# Module: /

## Skia Perf Technical Documentation

### 1. High-Level Overview

**Project Objectives:** Skia Perf is a performance monitoring system designed to
ingest, store, analyze, and visualize performance data for various projects,
with a primary focus on Skia and related systems (e.g., Flutter, Android,
Chrome). Its core objectives are:

1.  **Centralized Performance Data Storage:** Provide a robust and scalable
    repository for performance metrics collected from diverse benchmark runs.
2.  **Interactive Data Exploration:** Offer web-based dashboards that allow
    users to query, explore, and visualize performance trends over time, across
    different configurations and code revisions.
3.  **Automated Regression Detection:** Implement algorithms to automatically
    identify statistically significant performance regressions (and
    improvements) as new data is ingested.
4.  **Alerting and Notification:** Notify relevant stakeholders (developers,
    performance engineers) about detected regressions.
5.  **Triage and Investigation Support:** Provide tools to help users triage
    regressions, associate them with code changes, and track their resolution.
6.  **Integration with Developer Workflows:** Connect performance data with
    version control systems (Git) and issue trackers.

**Functionality:** Perf consists of several key components that work together:

- **Data Ingestion:** Processes performance data files (typically JSON)
  uploaded to Google Cloud Storage (GCS). These files are parsed, validated,
  and their metrics are stored against corresponding Git commit hashes.
- **Data Storage:** Uses SQL databases (primarily CockroachDB, with support
  for Spanner) to store metadata (commits, alert configurations, regression
  statuses) and specialized trace data.
- **Clustering & Regression Detection:** Employs k-means clustering and step
  detection algorithms on time-series trace data to identify regressions or
  significant performance shifts. This can be run continuously or triggered by
  new data events.
- **Frontend UI:** A web application (built with Go on the backend and
  TypeScript/Lit/Web Components on the frontend) that provides interactive
  dashboards for:
  - Plotting performance metrics over commit ranges.
  - Querying traces based on various parameters.
  - Configuring alerts.
  - Triaging detected regressions.
  - Viewing commit details and associated performance changes.
- **Alerting System:** Allows users to define alerts based on specific queries
  and thresholds. When regressions matching these alerts are found,
  notifications can be sent (e.g., email, issue tracker integration).
- **Command-Line Tools:** Provides `perfserver` (to run the different
  services) and `perf-tool` (for administrative tasks, data inspection, and
  database backups/restores).

### 2. Project-Specific Terminology

- **Trace:** A single time series of performance measurements for a specific
  test under a specific configuration (e.g., memory usage for `draw_a_circle`
  on `arch=x86,config=8888`). Trace IDs are structured key-value strings like
  `,arch=x86,config=8888,test=draw_a_circle,units=ms,`.
- **CommitNumber:** An internal, monotonically increasing integer assigned by
  Perf to each Git commit as it's processed. This provides a linear sequence
  for ordering data.
- **Tile:** A logical grouping of commits. Trace data is stored in relation to
  these tiles. The `tile_size` (number of commits per tile) is configurable
  and affects how data is sharded and queried.
- **ParamSet:** A collection of unique parameter key-value pairs observed in
  the data within a certain commit range (or tile range). Used to populate UI
  query builders.
- **DataFrame:** A tabular data structure, similar to R's dataframes or Pandas
  DataFrames, used on the backend and frontend. It holds trace values indexed
  by commit and trace ID, along with header information (commit details).
- **Cluster / Clustering:** The process of grouping similar traces together
  using k-means clustering. This is a core part of regression detection, as a
  significant change in the centroid of a cluster can indicate a regression.
- **Regression (Statistic):** A numerical value (StepSize / LeastSquaresError)
  calculated for a cluster's centroid after fitting a step function. It
  measures how much the centroid's behavior resembles a step change. High
  absolute values are "Interesting."
- **Alert (Configuration):** A user-defined configuration that specifies a
  query to select traces, a detection algorithm, grouping parameters, and
  notification settings for finding regressions.
- **Ingestion Format:** A specific JSON structure (documented in `FORMAT.md`)
  that Perf expects for input data files.
- **Shortcut:** A saved URL or configuration, often represented by a short,
  hashed ID, for quickly accessing a specific view or set of traces.
- **Triage:** The process of reviewing a detected regression, determining if
  it's a genuine issue or an expected change/noise, and marking it accordingly
  (e.g., "Bug," "Ignore").

### 3. Overall Architecture

Perf follows a services-oriented architecture, where the main `perfserver`
executable can run in different modes (frontend, ingest, cluster, maintenance).
Data flows from external benchmark systems into Perf, where it's processed,
stored, analyzed, and finally presented to users.

**Data Flow and Main Components:**

```
External Benchmark Systems
        |
        V
[Data Files (JSON) in Perf Ingestion Format]
        | (Uploaded to Google Cloud Storage - GCS)
        V
GCS Bucket (e.g., gs://skia-perf/nano-json-v1/)
        | (Pub/Sub event on new file arrival)
        V
Perf Ingest Service(s) (`perfserver ingest` mode)
        |   - Parses JSON files (see /go/ingest/parser)
        |   - Validates data (see /go/ingest/format)
        |   - Associates data with Git commits (see /go/git)
        |   - Writes trace data to TraceStore (SQL, tiled) (see /go/tracestore)
        |   - Updates ParamSets (for UI query builders)
        |   - (Optionally) Emits Pub/Sub events for "Event Driven Alerting"
        V
SQL Database (CockroachDB / Spanner)
    |   - Trace Data (values, parameters, indexed by commit/tile)
    |   - Commit Information (hashes, timestamps, messages)
    |   - Alert Configurations
    |   - Regression Records (details of detected regressions, triage status)
    |   - Shortcuts, User Favorites, etc.
    |
    +<--> Perf Cluster Service(s) (`perfserver cluster` or `perfserver frontend --do_clustering` mode)
    |       - Loads Alert configurations
    |       - Queries TraceStore for relevant data
    |       - Performs clustering (k-means) (see /go/clustering2, /go/ctrace2)
    |       - Fits step functions to cluster centroids (see /go/stepfit)
    |       - Calculates Regression statistic
    |       - Stores "Interesting" clusters/regressions in the database
    |       - Sends notifications (email, issue tracker) (see /go/notify)
    |
    +<--> Perf Frontend Service (`perfserver frontend` mode)
    |       - Serves HTML, CSS, JS (see /pages, /modules)
    |       - Handles API requests from the UI (see /go/frontend, /API.md)
    |       - Queries database for trace data, alert configs, regressions
    |       - Formats data for UI display (often as DataFrames)
    |       - Manages user authentication (via X-WEBAUTH-USER header)
    |
    +<--> Perf Maintenance Service (`perfserver maintenance` mode)
            - Git repository synchronization
            - Database schema migrations (see /migrations)
            - Old data cleanup
            - Cache refreshing (e.g., ParamSet cache)
```

**Rationale for Key Architectural Choices:**

- **Decoupled Ingestion via GCS and Pub/Sub:**
  - **Why:** This decouples data producers from Perf's internal processing.
    Producers only need to drop files in a GCS bucket. Pub/Sub provides a
    scalable and reliable way to notify ingesters about new files, allowing
    multiple ingester instances to pull work.
  - **How:** Ingesters subscribe to a Pub/Sub topic. GCS is configured to
    publish a message to this topic when a new file is finalized in the
    designated ingestion bucket/prefix.
- **SQL Database for Structured Data:**
  - **Why:** SQL databases like CockroachDB and Spanner provide
    transactional consistency, scalability, and powerful querying
    capabilities needed for metadata, alert configurations, and regression
    tracking. CockroachDB offers PostgreSQL compatibility, which is widely
    used. Spanner provides horizontal scalability for very large datasets.
  - **How:** Go's `database/sql` package is used, with schema defined and
    managed by `/go/sql` and migration scripts in `/migrations`.
- **Specialized TraceStore:**
  - **Why:** Performance trace data is time-series and can be voluminous. A
    generic relational model might not be optimal for the typical queries
    (fetching traces over commit ranges for specific parameter sets). The
    tiled approach with inverted indexes for parameters is designed for more
    efficient retrieval.
  - **How:** The `TraceStore` (/go/tracestore) implementation uses SQL
    tables but structures them to represent tiles of commits. `ParamSets`
    and `Postings` tables act as inverted indexes for fast lookup of traces
    matching specific key-value parameters.
- **Monolithic Executable (`perfserver`) with Modes:**
  - **Why:** Simplifies deployment and reduces the number of distinct
    binaries to manage. A single executable can be configured to run as a
    frontend, an ingester, a clusterer, or a maintenance task.
  - **How:** `perfserver` uses command-line flags and subcommands to
    determine its operational mode. Configuration files (`/configs/*.json`)
    further dictate behavior within each mode.
- **K-Means Clustering for Regression Detection:**
  - **Why:** K-means is a well-understood clustering algorithm suitable for
    grouping traces with similar performance characteristics. Changes in
    these groups over time can signal regressions. Traces are normalized
    before clustering to make them comparable despite different scales.
  - **How:** Implemented in `/go/clustering2` and `/go/kmeans`. `ctrace2`
    handles trace normalization.
- **Frontend/Backend Separation:**
  - **Why:** Standard practice for web applications. Allows independent
    development and scaling of the UI and the backend logic.
  - **How:** Backend (Go) serves JSON APIs. Frontend (TypeScript/Lit)
    consumes these APIs to render interactive views.
- **Event-Driven Alerting (Optional):**
  - **Why:** For very large and sparse datasets (like Android), continuous
    clustering over all alerts can be resource-intensive and slow.
    Event-driven alerting processes only the data relevant to recently
    updated traces, reducing latency and computational load.
  - **How:** Ingesters publish Pub/Sub events containing IDs of updated
    traces. Clusterers subscribe to these events and run relevant alert
    configurations only for the affected data.

### 4. Module Responsibilities and Key Components

This section focuses on significant modules beyond simple file/directory
descriptions.

- **`/go/config`**:

  - **Responsibility:** Defines and validates the structure for instance
    configuration files (`InstanceConfig`). This is the central place where
    all settings for a Perf deployment (database, ingestion sources, Git
    repo, UI features, notification settings) are specified.
  - **Why:** Configuration files allow Perf to be deployed for different
    projects with different data sources and requirements without code
    changes. A strongly-typed Go struct ensures that configurations are
    well-defined and can be validated.
  - **How:** `InstanceConfig` is a Go struct with fields for various aspects
    of the system. JSON files in `/configs` are unmarshaled into this
    struct. The module provides functions to load and validate these
    configurations.

- **`/go/ingest`**:

  - **Responsibility:** Orchestrates the entire data ingestion pipeline.
    This includes watching for new files, parsing them according to the
    `format.Format` specification, extracting performance metrics and
    metadata, associating them with Git commits, and writing the data to the
    `TraceStore`.
  - **Why:** This module is the entry point for all performance data into
    the Perf system. It needs to be robust, handle various data formats
    (though primarily the standard JSON format), and ensure data integrity.
  - **Key Sub-components:**
  - `ingest/format`: Defines the expected structure of input JSON files
    (`format.Format` Go struct) and provides validation. This ensures data
    consistency.
  - `ingest/parser`: Contains logic to parse the `format.Format` structure
    and extract individual trace measurements and their associated
    parameters.
  - `ingest/process`: Coordinates the steps: reading from a source (e.g.,
    GCS via `/go/file`), parsing, resolving commit information (via
    `/go/git`), and writing to the `TraceStore`.
  - **Workflow:**

  * A `Source` (e.g., `GCSSource` via PubSub) indicates a new file.
  * `process` reads the file.
  * `parser` and `format` validate and extract `Result`s.
  * For each `Result`, its `git_hash` is resolved to a `CommitNumber` using
    `/go/git`.
  * Traces are constructed and written to `/go/tracestore`.

- **`/go/tracestore`**:

  - **Responsibility:** Manages the storage and retrieval of performance
    trace data. This is a critical component for efficient querying.
  - **Why:** Trace data is time-series and multi-dimensional. The
    `TraceStore` is designed to efficiently retrieve trace values for
    specific parameter combinations over ranges of commits.
  - **How:** It uses a "tiled" storage approach. Commits are grouped into
    tiles.
  - `TraceValues` table: Stores the actual metric values, often sharded by
    tile.
  - `ParamSets` table: Stores unique key-value pairs found in trace
    identifiers within each tile.
  - `Postings` table: An inverted index mapping (tile, param_key,
    param_value) to a list of trace IDs that contain that key-value pair
    within that tile. This structure allows queries like "get all traces
    where `config=8888` and `arch=x86`" to be resolved efficiently by
    intersecting posting lists. `SQLTraceStore` is the primary
    implementation using the SQL database.

- **`/go/git`**:

  - **Responsibility:** Interacts with Git repositories to fetch commit
    information (hashes, authors, timestamps, messages). It also caches this
    information in the SQL database to avoid repeated Git operations.
  - **Why:** Perf needs to correlate performance data with specific code
    changes. This module provides the link between `git_hash` values in
    ingested data and Perf's internal `CommitNumber` sequence.
  - **How:** It can use either a local Git checkout (via `git` CLI) or a
    Gitiles service API. It maintains a `Commits` table in the SQL database,
    mapping commit hashes to `CommitNumber`s and storing other metadata. It
    periodically updates its local Git repository clone or queries Gitiles
    for new commits.

- **`/go/regression`**:

  - **Responsibility:** Handles the detection, storage, and management of
    performance regressions.
  - **Why:** This is a core function of Perf. It provides the logic to
    identify when performance has changed significantly and to track the
    status of these findings.
  - **How:**
  - It uses clustering results (from `/go/clustering2`) and step-fit
    analysis (from `/go/stepfit`) to identify "Interesting" clusters.
  - `Store` interface (implemented by `sqlregression2store`): Persists
    information about detected regressions, including the cluster summary,
    owning alert, commit hash, regression statistic, and triage status
    (`New`, `Ignore`, `Bug`).
  - The "Alerting" algorithm described in `DESIGN.md` (comparing new
    interesting clusters with existing ones based on trace fingerprints) is
    implemented here to manage the lifecycle of a regression.
  - **Key Workflow for Alerting/Regression Tracking:** `Run clustering
(e.g., hourly or event-driven) | V Identify "Interesting" new clusters
(high |Regression| score) | V For each new Interesting Cluster: Compare
fingerprint (top N traces) with existing relevant Clusters in DB | +--
No match? --> New Regression: Store in DB with status "New". | +-- Match
found? --> Update existing Regression if new one has better |Regression|
score. Keep triage status of existing.`

- **`/go/frontend`**:

  - **Responsibility:** Implements the backend for the Perf web user
    interface. It handles HTTP requests, interacts with data stores
    (TraceStore, AlertStore, RegressionStore, etc.), processes data, and
    serves JSON responses to the frontend.
  - **Why:** This module connects the user's browser interactions to Perf's
    data and analytical capabilities.
  - **How:** It uses Go's standard `net/http` package to define HTTP
    handlers for various API endpoints (e.g., fetching data for plots,
    listing alerts, updating triage statuses). It authenticates users based
    on the `X-WEBAUTH-USER` header. It often fetches data, converts it into
    `DataFrame` structures, and then serializes these to JSON for the
    frontend.

- **`/modules` (Frontend TypeScript)**:

  - **Responsibility:** Contains the TypeScript source code for all frontend
    custom elements (web components) and UI logic. These modules are
    compiled into JavaScript and CSS that run in the user's browser.
  - **Why:** This is where the user interface is built. Modularity (one
    component per file/directory) makes the frontend codebase manageable.
    Custom elements (often using Lit) provide encapsulation and reusability.
  - **How:** Each subdirectory typically defines one or more custom elements
    (e.g., `plot-simple-sk`, `alert-config-sk`, `query-sk`). These elements
    handle rendering, user interaction, and making API calls to the Go
    backend.
  - `perf-scaffold-sk`: Provides the main page layout (header, sidebar,
    content area).
  - `explore-simple-sk` / `explore-sk`: Core components for querying data
    and displaying plots.
  - `json/index.ts`: Contains TypeScript interfaces mirroring Go backend
    structs for type-safe API communication. This is crucial for ensuring
    frontend and backend data structures are compatible. It's often
    generated from Go source using `/go/ts/ts.go`.

- **`/pages`**:

  - **Responsibility:** Defines the top-level HTML structure for each
    distinct page of the Perf application (e.g., alerts page, exploration
    page).
  - **Why:** These files serve as the entry points for specific views. They
    are kept minimal, primarily including the `perf-scaffold-sk` and the
    main page-specific custom element.
  - **How:** Each HTML file (e.g., `alerts.html`) includes the
    `perf-scaffold-sk` and the relevant page element (e.g.,
    `<alerts-page-sk>`). An associated TypeScript file (e.g., `alerts.ts`)
    imports the necessary custom element definitions. Server-side Go
    templates inject initial context data (`window.perf = {%.context %};`)
    into the HTML.

- **`DESIGN.md`**:

  - **Significance:** This document is the primary source for understanding
    the high-level architecture, design rationale, and core algorithms of
    Perf, particularly for clustering and alerting.
  - **Key Concepts Explained:**
  - **Clustering:** Details the use of k-means clustering on normalized
    traces, the Euclidean distance metric, and the calculation of the
    "Regression" statistic (StepSize / LeastSquaresError) to identify
    "Interesting" clusters.
  - **Alerting Algorithm:** Explains how Perf identifies and tracks unique
    regressions over time by fingerprinting clusters and comparing new
    interesting clusters to existing ones. It outlines the schema for the
    `clusters` table (though the actual schema is in `/go/sql` and may have
    evolved to `Regressions` table).
  - **Event Driven Alerting:** Describes an alternative to continuous
    clustering, triggered by PubSub events when new data arrives. This is
    beneficial for large, sparse datasets.

- **`FORMAT.md`**:

  - **Significance:** Defines the precise JSON structure that Perf ingesters
    expect for input data files.
  - **Key Elements:** Specifies fields like `git_hash`, `key` (for global
    parameters), and `results` (an array of measurements). Each result can
    have its own `key` (for test-specific parameters like `test` name and
    `units`) and either a single `measurement` or a more complex
    `measurements` object for statistics (min, max, median). This document
    is crucial for data producers who need to integrate with Perf.

- **`BUILD.bazel` (Root)**:

  - **Significance:** Defines how the Perf application is built using Bazel.
    It specifies container images (`perfserver`, `backendserver`) that
    package the Go executables and necessary static resources (configs,
    frontend assets).
  - **How:** Uses `skia_app_container` rules to assemble Docker images. It
    copies the `perfserver` and `perf-tool` binaries, configuration files
    from `/configs`, and compiled frontend assets (HTML, JS, CSS from
    `/pages` built output) into the image. The `entrypoint` for the
    `perfserver` image is the `perfserver` executable itself.

### 5. Key Workflows Illustrated (Pseudographic Diagrams)

**A. New Alert Creation via UI and API:**

```
User (in Perf UI, e.g., on /alerts page)
  |
  | Fills out Alert configuration form (<alert-config-sk> element)
  | Clicks "Save"
  |
  V
Frontend JS (<alert-config-sk>)
  |
  | 1. If new alert, GET /_/alert/new
  |    (Server responds with a pre-populated Alert JSON with id: -1)
  |
  | 2. Modifies this Alert JSON based on form input
  |
  | 3. POST modified Alert JSON to /_/alert/update
  |    (Authorization: Bearer token if auth is enabled)
  |
  V
Perf Backend (`/go/frontend/service.go` - UpdateAlertHandler)
  |
  | Receives Alert JSON
  | If alert.ID == -1, it's a new alert.
  | Validates Alert configuration
  | Persists Alert to SQL Database (via `alerts.Store`)
  | Responds 200 OK
  |
  V
SQL Database (Alerts Table)
  |
  | New Alert record is created or existing one updated.
```

**Rationale:**

- The `GET /_/alert/new` step is a convenience. It provides the frontend with
  a valid `Alert` structure, including any instance-default values,
  simplifying new alert creation logic on the client.
- Using `id: -1` to signify a new alert during the `POST` to `/_/alert/update`
  is a common pattern to allow a single endpoint to handle both creation and
  updates. The backend inspects the ID to determine the correct action.
- The API interactions are documented in `API.md`.

**B. Data Ingestion and Event-Driven Regression Detection:**

```
Benchmark System
  |
  | Produces performance_data.json (Perf Ingestion Format)
  | Uploads to GCS: gs://[bucket]/[path]/YYYY/MM/DD/HH/performance_data.json
  |
  V
Google Cloud Storage
  |
  | File "OBJECT_FINALIZE" event
  | Publishes message to PubSub Topic (e.g., "perf-ingestion-topic")
  |
  V
Perf Ingest Service(s) (Subscribed to "perf-ingestion-topic")
  |
  | 1. Receives PubSub message (contains GCS file path)
  | 2. Downloads performance_data.json from GCS
  | 3. Parses JSON, validates data (see /go/ingest/format, /go/ingest/parser)
  | 4. Looks up git_hash in /go/git to get CommitNumber
  | 5. Writes trace data to TraceStore (SQL tables)
  | 6. If Event Driven Alerting enabled for this instance:
  |    Constructs a list of Trace IDs updated by this file
  |    Publishes message (containing gzipped Trace IDs) to another PubSub Topic (e.g., "trace-update-topic")
  |
  V
Perf Cluster Service(s) (Subscribed to "trace-update-topic")
  |
  | 1. Receives PubSub message (with updated Trace IDs)
  | 2. For each Alert Configuration (/go/alerts):
  |    If Alert's query matches any of the updated Trace IDs:
  |      Run clustering & regression detection for THIS Alert,
  |      focusing on the commit range and data relevant to the updated traces.
  |      (Reduces scope compared to full continuous clustering)
  | 3. If regressions found:
  |    Store in SQL Database (Regressions table)
  |    Send notifications (email, issue tracker)
```

**Rationale:**

- **GCS as Entry Point:** As described in `FORMAT.md` and `DESIGN.md`, GCS is
  the standard way data enters Perf. The YYYY/MM/DD/HH path structure is a
  convention.
- **Pub/Sub for Decoupling and Scalability:** Ingesters don't need to poll
  GCS. Pub/Sub handles event delivery, and multiple ingesters can process
  files in parallel.
- **Event-Driven Clustering Optimization:** `DESIGN.md` explicitly states this
  is for large/sparse datasets. Sending only updated Trace IDs significantly
  narrows the scope of clustering for each event, making it faster and less
  resource-intensive than re-clustering everything. PubSub's 10MB message
  limit is considered for gzipped trace ID lists.

This documentation provides a comprehensive starting point for a software
engineer to understand the Skia Perf project. It covers its purpose,
architecture, core concepts, and the rationale behind key design and
implementation choices, referencing existing documentation and source code
structure where appropriate.

# Module: /cockroachdb

The `/cockroachdb` module provides a set of shell scripts designed to facilitate
interaction with a CockroachDB instance, specifically one named
`perf-cockroachdb`, which is presumed to be running within a Kubernetes cluster.
These scripts abstract away some of the complexities of `kubectl` commands,
offering streamlined access for common database operations.

The primary motivation behind these scripts is to simplify development and
administrative workflows. Instead of requiring users to remember and type
lengthy `kubectl` commands with specific flags and resource names, these scripts
provide convenient, single-command access points.

**Key Components and Responsibilities:**

- **`admin.sh`**: This script focuses on providing access to the CockroachDB
  administrative web interface.

  - **Why**: The web UI is a crucial tool for monitoring database health,
    performance, and managing cluster settings. Direct access via `kubectl
port-forward` can be cumbersome to set up repeatedly.
  - **How**: It executes `kubectl port-forward` to map the local port `8080`
    to the port `8080` of the `perf-cockroachdb-0` pod. Crucially, it then
    immediately attempts to open this local address in Google Chrome,
    providing an instant user experience. This assumes Google Chrome is
    installed and available in the system's PATH. `User runs admin.sh | V
Script executes: kubectl port-forward perf-cockroachdb-0 8080 | V Local
port 8080 now forwards to CockroachDB pod's port 8080 | V Script
executes: google-chrome http://localhost:8080 | V CockroachDB Admin UI
opens in Chrome`

- **`connect.sh`**: This script is designed to provide a SQL shell connection
  to the CockroachDB instance.

  - **Why**: Developers and administrators frequently need to execute SQL
    queries directly against the database for debugging, data manipulation,
    or schema inspection. Setting up an interactive `kubectl run` command
    with the correct image and arguments can be error-prone.
  - **How**: It uses `kubectl run` to create a temporary, interactive pod
    named `androidx-cockroachdb`. This pod uses the
    `cockroachdb/cockroach:v19.2.5` Docker image. The `--rm` flag ensures
    the pod is deleted after the session ends, and `--restart=Never`
    prevents it from being restarted. The crucial part is the command passed
    to the pod: `sql --insecure --host=perf-cockroachdb-public`. This starts
    the CockroachDB SQL client, connecting insecurely to the database
    service exposed at `perf-cockroachdb-public`. `User runs connect.sh | V
Script executes: kubectl run androidx-cockroachdb -it --image=... --rm
--restart=Never -- sql --insecure --host=perf-cockroachdb-public | V
Temporary pod 'androidx-cockroachdb' is created | V CockroachDB SQL
client starts inside the pod, connecting to 'perf-cockroachdb-public' |
V User has an interactive SQL shell | V User exits shell -> Pod
'androidx-cockroachdb' is deleted`

- **`skia-infra-public-port-forward.sh`**: This script sets up a port forward
  for direct database connections, typically for use with a local CockroachDB
  SQL client or other database tools.

  - **Why**: While `connect.sh` provides an in-cluster SQL shell, sometimes
    a direct connection from the local machine is preferred, for instance,
    to use graphical SQL clients or specific client libraries that are not
    available within the temporary pod created by `connect.sh`. The
    `perf-cockroachdb` instance is likely within a private network in the
    Kubernetes cluster (namespace `perf`), and this script makes it
    accessible locally.
  - **How**: It leverages a helper script `../../kube/attach.sh
skia-infra-public` (the details of which are outside this module's scope
    but presumably handles Kubernetes context or authentication for the
    `skia-infra-public` cluster). This helper script is then used to execute
    `kubectl port-forward` specifically for the `perf-cockroachdb-0` pod
    within the `perf` namespace. It maps local port `25000` to the pod's
    CockroachDB port `26257`. The script also helpfully prints instructions
    for the user on how to connect using the `cockroach sql` command once
    the port forward is active. The `set -e` command ensures the script
    exits immediately if any command fails, and `set -x` enables command
    tracing for debugging. `User runs skia-infra-public-port-forward.sh | V
Script prints connection instructions | V Script executes:
../../kube/attach.sh skia-infra-public kubectl port-forward -n perf
perf-cockroachdb-0 25000:26257 | V Port forward is established:
local:25000 -> perf-cockroachdb-0:26257 (in 'perf' namespace) | V User
can now run 'cockroach sql --insecure --host=127.0.0.1:25000' in another
terminal`

These scripts collectively aim to make interacting with the `perf-cockroachdb`
instance as straightforward as possible by encapsulating the necessary `kubectl`
commands and providing context-specific instructions or actions. They rely on
the Kubernetes cluster being correctly configured and accessible, and on
`kubectl` and potentially `google-chrome` being available on the user's system.

# Module: /configs

The `/configs` directory houses JSON configuration files for various instances
of the Perf performance monitoring system. Each file defines the specific
behavior and data sources for a particular Perf deployment. These configurations
are crucial for tailoring Perf to different projects and environments, enabling
developers and performance engineers to monitor and analyze performance data
effectively.

The core idea is to provide a declarative way to set up a Perf instance. Instead
of hardcoding settings, these JSON files act as blueprints. Each file serializes
to and from a Go struct named `config.InstanceConfig`. This struct serves as the
canonical schema for all instance configurations, and its Go documentation
provides detailed explanations of each field. This approach ensures consistency
and makes it easier to manage and evolve the configuration options.

**Key Components and Responsibilities:**

The primary responsibility of this module is to define and store these instance
configurations. Each JSON file represents a distinct Perf instance, often
corresponding to a specific project or a particular version of a project (e.g.,
a public vs. internal build, or a stable vs. experimental branch).

- **Instance-Specific Configuration Files (e.g., `android2.json`,
  `chrome-public.json`):**

  - **Why:** Each project or system being monitored by Perf has unique
    requirements. These include where its performance data is stored (e.g.,
    GCS buckets), how it's ingested (e.g., Pub/Sub topics), which Git
    repository tracks its code changes, how users authenticate, and how
    notifications for regressions are handled.
  - **How:** These files use a JSON structure that maps directly to the
    `config.InstanceConfig` Go struct.
  - `URL`: The public-facing URL of the Perf instance.
  - `data_store_config`: Defines the backend database (e.g., CockroachDB,
    Spanner), connection strings, and parameters like `tile_size` which can
    impact query performance and data retrieval efficiency. The choice
    between CockroachDB and Spanner often depends on scalability needs and
    existing infrastructure.
  - `ingestion_config`: Specifies how performance data is brought into Perf.
    This includes the `source_type` (e.g., `gcs` for Google Cloud Storage,
    `dir` for local directories), the specific `sources` (e.g., GCS bucket
    paths or local file paths), and Pub/Sub topics for real-time ingestion.
    This section is vital for connecting Perf to the data producers.
  - `git_repo_config`: Links Perf to the source code repository. This allows
    Perf to correlate performance data with specific code changes (commits).
    It includes the repository `url`, the `provider` (e.g., `gitiles`,
    `git`), and sometimes a `commit_number_regex` to extract meaningful
    commit identifiers from commit messages.
  - `notify_config`: Configures how alerts and notifications are sent when
    regressions are detected. This can range from `none` to `html_email`,
    `markdown_issuetracker`, or `anomalygroup`. It often includes templates
    for notification subjects and bodies, leveraging placeholders like `{{
.Alert.DisplayName }}` to include dynamic information.
  - `auth_config`: Defines the authentication mechanism, commonly using a
    header like `X-WEBAUTH-USER` for integration with existing
    authentication systems.
  - `query_config`: Customizes how users can query and view data, including
    which parameters are available for filtering (`include_params`), default
    selections, and URL value defaults to tailor the user experience. It can
    also include caching configurations (e.g., using Redis) to improve query
    performance by specifying `cache_config` with `level1_cache_key` and
    `level2_cache_key`.
  - `anomaly_config`: Contains settings related to anomaly detection, such
    as `settling_time` which defines how long Perf waits before considering
    new data for anomaly detection, helping to avoid flagging transient
    issues.
  - Other fields like `contact`, `ga_measurement_id` (for Google Analytics),
    `feedback_url`, `trace_sample_proportion` (to control the volume of
    detailed trace data collected), and `favorites` (for pre-defined links
    on the Perf UI) further customize the instance.
  - **Example Workflow (Data Ingestion and Alerting for `android2.json`):**

  * **Data Production:** Android benchmarks generate performance data.
  * **Data Upload:** This data is uploaded to GCS buckets specified in
    `ingestion_config.source_config.sources` (e.g.,
    `gs://android-perf-2/android2`).
  * **Pub/Sub Notification:** A message is sent to the Pub/Sub topic
    `perf-ingestion-android2-production`.
  * **Perf Ingestion Service:** The Perf ingestion service, subscribed to
    this topic, reads the new data file from GCS.
  * **Data Processing & Storage:** Perf processes the data, associates it
    with the corresponding commit from the `git_repo_config` (e.g.,
    `https://android.googlesource.com/platform/superproject`), and stores it
    in the CockroachDB instance defined in `data_store_config`.
  * **Anomaly Detection:** Perf's anomaly detection algorithms analyze the
    new data points.
  * **Regression Found:** If a regression is detected based on the
    `anomaly_config`.
  * **Notification Sent:** A notification is generated according to
    `notify_config`. For `android2.json`, this means an issue is filed in an
    issue tracker (`"notifications": "markdown_issuetracker"`) with a
    subject and body formatted using the provided templates, including
    details like affected tests and devices.

- **`local.json`:**

  - **Why:** Provides a standardized configuration for local development and
    manual testing of Perf. It's designed to be self-contained and not rely
    on external production services.
  - **How:** It typically points the `ingestion_config` to a local directory
    (`integration/data`) that contains sample data. This data is often the
    same data used for unit tests, ensuring consistency between testing
    environments. The database connection will also point to a local
    instance.

- **`demo.json` and `demo_spanner.json`:**

  - **Why:** These configurations are likely used for demonstration purposes
    or for setting up small-scale, illustrative Perf instances. They
    showcase Perf's capabilities with sample data.
  - **How:** Similar to `local.json`, `demo.json` uses a local directory for
    data ingestion (`"./demo/data/"`) and a local CockroachDB instance.
    `demo_spanner.json` is analogous but configured to use Spanner as the
    backend, demonstrating flexibility in data store choices. They often
    include simpler `git_repo_config` pointing to public demo repositories
    (e.g., `https://github.com/skia-dev/perf-demo-repo.git`). The
    `favorites` section in `demo.json` shows how to add curated links to the
    Perf UI.

- **`/spanner` subdirectory:**

  - **Why:** This subdirectory groups configurations for Perf instances that
    specifically use Google Cloud Spanner as their backend data store.
    Spanner is chosen for its scalability, strong consistency, and global
    distribution capabilities, making it suitable for large-scale Perf
    deployments.
  - **How:** Files within this directory (e.g.,
    `spanner/chrome-public.json`, `spanner/skia-public.json`) will have
    their `data_store_config.datastore_type` set to `"spanner"`. They often
    include Spanner-specific settings or optimizations. For example,
    `enable_follower_reads` might be set to `true` in `data_store_config`
    for Spanner instances to distribute read load. Many of these
    configurations also define `redis_config` within their
    `query_config.cache_config` to further enhance query performance for
    frequently accessed data.
  - The `optimize_sqltracestore` flag, often set to `true` in Spanner
    configurations, indicates that specific optimizations for the SQL-based
    trace store are enabled, likely tailored to Spanner's characteristics.
  - Configurations like `chrome-internal.json` and `chrome-public.json`
    demonstrate sophisticated setups, including:
  - `commit_number_regex` in `git_repo_config` to extract structured commit
    positions.
  - `temporal_config` for integrating with Temporal workflows for tasks like
    regression grouping and bisection.
  - `enable_sheriff_config` to integrate with sheriffing systems for
    managing alerts.
  - `trace_format: "chrome"` indicates that the performance data adheres to
    the Chrome trace event format.

The choice of fields and their values within each JSON file reflects a series of
design decisions aimed at balancing flexibility, performance, and operational
manageability for each specific Perf instance. For instance, the `tile_size` in
`data_store_config` is adjusted based on expected data characteristics and query
patterns. Similarly, `trace_sample_proportion` is set to manage storage costs
and processing load while still capturing enough data for meaningful analysis.
The `notify_config` templates are crafted to provide actionable information to
developers when regressions occur.

# Module: /csv2days

## csv2days Module Documentation

### Overview

The `csv2days` module is a command-line utility designed to process CSV files
downloaded from the Perf performance monitoring system. Its primary purpose is
to simplify time-series data by consolidating multiple data points from the same
calendar day into a single representative value. This is particularly useful
when analyzing performance trends over longer periods, where daily granularity
is sufficient and finer-grained timestamps can introduce noise or unnecessary
complexity.

The core problem this module solves is the overabundance of data points when
Perf exports data at a high temporal resolution (e.g., multiple commits per
day). For certain types of analysis, this level of detail is not required and
can make it harder to discern broader trends. `csv2days` transforms such CSVs by
keeping only the first encountered data column for each unique day and
aggregating subsequent values from the same day into that single column using a
"max" aggregation strategy.

### Design and Implementation

The module operates as a streaming processor. It reads the input CSV file row by
row, processes the header to determine which columns to modify or drop, and then
transforms each subsequent data row accordingly before writing it to standard
output.

**Key Design Choices:**

1.  **Command-Line Interface:** The tool is designed as a simple command-line
    application for ease of integration into scripting workflows. It takes an
    input file path via the `--in` flag and outputs the transformed CSV to
    `stdout`. This follows common Unix philosophies for tool interoperability.
2.  **Streaming Processing:** Instead of loading the entire CSV into memory,
    which could be problematic for very large files, `csv2days` processes the
    file line by line. This makes the tool memory-efficient.
3.  **Date-Based Grouping:** The core logic revolves around identifying columns
    that represent timestamps. It uses a regular expression (`datetime`) to
    match RFC3339 formatted dates in the header row. The date part (YYYY-MM-DD)
    of these timestamps is used for grouping.
4.  **"First Seen" Column for a Day:** For each unique calendar day encountered
    in the header, only the first column corresponding to that day is retained
    in the output header. Subsequent columns from the same day are marked for
    removal.
5.  **"Max" Aggregation:** When multiple columns from the same day are
    encountered in a data row, the values from these columns are aggregated. The
    `csv2days` tool currently implements a "max" aggregation strategy: for the
    set of values corresponding to a single day, the maximum numerical value is
    chosen. If non-numerical values are encountered, the first value in the
    sequence is typically used.
6.  **Reverse Sorted Index Removal:** When removing columns, the indices of
    columns to be skipped (`skipCols`) are sorted in reverse order. This is
    crucial because removing an element from a slice shifts the indices of
    subsequent elements. Processing removals from right-to-left (largest index
    to smallest) ensures that the indices remain valid throughout the removal
    process.

**Workflow:**

The main workflow within `transformCSV` can be visualized as follows:

```
Read Input CSV File (--in flag)
        |
        v
Parse Header Row
        |
        +----------------------------------------------------------------------+
        | Identify Timestamp Columns (using RFC3339 regex)                     |
        | For each timestamp:                                                  |
        |   Extract Date (YYYY-MM-DD)                                          |
        |   If new date:                                                       |
        |     Add Date to Output Header                                        |
        |     Record current column as start of a new "run" for this day       |
        |   Else (same date as previous timestamp):                            |
        |     Mark current column for skipping (`skipCols`)                    |
        |     Increment length of current day's "run" (`runLengths`)           |
        |                                                                      |
        | Non-timestamp columns are added to Output Header as-is               |
        +----------------------------------------------------------------------+
        |
        v
Write Transformed Header to Output
        |
        v
Sort `skipCols` in Reverse Order
        |
        v
For each Data Row in Input CSV:
        |
        +----------------------------------------------------------------------+
        | Apply "Max" Aggregation:                                             |
        |   For each "run" of columns belonging to the same day (from header): |
        |     Find the maximum numerical value in the corresponding cells      |
        |     Replace the first cell of the run with this max value            |
        +----------------------------------------------------------------------+
        |
        v
Remove Skipped Columns (based on `skipCols` from header processing)
        |
        v
Write Transformed Data Row to Output
        |
        v
Flush Output Buffer
```

### Key Components/Files

- **`main.go`**: This is the heart of the module.
  - **`main()` function**: Handles command-line flag parsing (`--in` for the
    input CSV file). It orchestrates the reading of the input file and calls
    `transformCSV` to perform the core logic. Error handling and logging are
    also managed here.
  - **`transformCSV(input io.Reader, output io.Writer) error`**: This is the
    core function responsible for the CSV transformation.
  - It initializes `csv.Reader` for input and `csv.Writer` for output.
  - **Header Processing**: It reads the first line (header) of the CSV. It
    iterates through the header cells.
    - A regular expression (`datetime = regexp.MustCompile(...)`) is used
      to identify columns containing RFC3339 timestamps.
    - It maintains `lastDate` to detect when a new day starts in the
      header sequence.
    - `skipCols` (a slice of integers) stores the indices of columns that
      represent subsequent entries for an already seen day and should thus
      be removed from the data rows.
    - `runLengths` (a map of `int` to `int`) stores, for each column that
      starts a sequence of same-day entries, how many columns belong to
      that day. This is used later for aggregation. For example, if
      columns 5, 6, and 7 are all for "2023-01-15", `runLengths[5]` would
      be `3`.
    - The output header (`outHeader`) is constructed by keeping the date
      part (YYYY-MM-DD) for the first occurrence of each day and omitting
      subsequent columns for the same day. Non-date columns are passed
      through unchanged.
  - **Data Row Processing**: It then reads the rest of the CSV file row by
    row.
    - `applyMaxToRuns(s []string, runLengths map[int]int) []string`: For
      each "run" of columns identified in the header as belonging to the
      same day, this function takes the corresponding values from the
      current data row and replaces the value in the first column of that
      run with the maximum of those values. The `max(s []string) string`
      helper function is used here to find the maximum float value,
      falling back to the first string if parsing fails.
    - `removeAllIndexesFromSlices(s []string, skipCols []int) []string`:
      After aggregation, this function removes the data cells
      corresponding to the `skipCols` identified during header processing.
      It uses `removeValueFromSliceAtIndex` repeatedly. It's crucial that
      `skipCols` is sorted in reverse order for this to work correctly.
  - The transformed row is then written to the output CSV.
  - **Helper Functions**:
  - `removeValueFromSliceAtIndex(s []string, index int) []string`: A utility
    to remove an element at a specific index from a string slice.
  - `max(s []string) string`: Iterates through a slice of strings, attempts
    to parse them as floats, and returns the string representation of the
    maximum float found. If no floats are found or parsing errors occur, it
    defaults to returning the first string in the input slice. This function
    underpins the aggregation logic.
- **`main_test.go`**: Contains unit tests for the `transformCSV` function.
  - `TestTransformCSV_HappyPath`: Provides a simple input CSV string and the
    expected output string. It then calls `transformCSV` with these and
    asserts that the actual output matches the expected output. This serves
    as a concrete example of the module's behavior.
- **`BUILD.bazel`**: Defines how the `csv2days` Go binary and its associated
  library and tests are built using Bazel. It specifies source files,
  dependencies (like `skerr`, `sklog`, `util`), and visibility.

The design decision to use `strconv.ParseFloat` and handle potential errors by
continuing or defaulting implies that the tool is somewhat lenient with
non-numeric data in columns expected to be numeric. The "max" operation will
effectively ignore non-convertible strings unless all strings in a run are
non-convertible, in which case the first string is chosen.

# Module: /demo

The `demo` module provides the necessary data and tools to showcase the
capabilities of the Perf performance monitoring system. Its primary purpose is
to offer a tangible and reproducible example of how Perf ingests and processes
performance data. This allows users and developers to understand Perf's
functionality without needing to set up a complex real-world data pipeline.

The core of this module revolves around a set of pre-generated data files and a
Go program to create them.

**Key Components:**

- **`/demo/data/` (Directory):** This directory houses the actual demo data
  files in JSON format. Each file represents performance measurements
  associated with a specific commit hash.

  - **Why:** These static files serve as the input for a 'dir' type ingester
    in a demo Perf instance. They are structured according to the
    `format.Format` specification (defined in `perf/go/ingest/format`),
    which Perf understands. This allows for a simple and direct way to feed
    data into Perf for demonstration purposes.
  - **How:** Each JSON file (e.g., `demo_data_commit_1.json`) contains a
    `git_hash`, `key` (identifying the test environment like architecture
    and configuration), and `results`. The `results` section includes
    measurements for various tests (like "encode" and "decode") across
    different units (like "ms" and "kb"). Some files also include `links`
    which can point to external resources relevant to the data point or the
    overall commit. The data in these files is designed to show some
    variation over commits to demonstrate Perf's ability to track changes
    and detect regressions/improvements. For instance, the `decode`
    measurement and `encodeMemory` show a deliberate shift in values
    starting from `demo_data_commit_6.json`.

- **`generate_data.go`:** This Go program is responsible for creating the JSON
  data files located in the `/demo/data/` directory.

  - **Why:** While the static data files are sufficient for running the
    demo, this program provides the means to regenerate or modify the demo
    dataset. This is crucial if the demo requirements change, if new Perf
    features need to be showcased with different data patterns, or if the
    underlying `format.Format` evolves. It ensures the demo data remains
    relevant and can be adapted.
  - **How:**

  * It defines a list of Git commit hashes. These hashes are specifically
    chosen from the `skia-dev/perf-demo-repo` repository, establishing a
    direct link between the performance data and a version control history,
    a common scenario in real-world Perf usage.
  * It iterates through these hashes. For each hash: _ It programmatically
    generates performance values (e.g., `encode`, `decode`, `encodeMemory`).
    The generation includes some randomness (`rand.Float32()`) to make the
    data appear more realistic. _ A deliberate change in the data generation
    logic is introduced for commits at index 5 and onwards (e.g.,
    `multiplier = 1.2`), which leads to a noticeable shift in `decode` and
    `encodeMemory` values in the corresponding JSON files. This is done to
    demonstrate how Perf can track and visualize such changes. _ It
    populates a `format.Format` struct (from
    `go.skia.org/infra/perf/go/ingest/format`) with the generated data,
    including the Git hash, environment keys, and the measurement results. _
    The `format.Format` struct is then marshaled into JSON with indentation
    for readability. \* Finally, the JSON data is written to a file named
    according to the commit sequence (e.g., `demo_data_commit_1.json`)
    within the `data` subdirectory. The program uses the `runtime.Caller(0)`
    function to determine its own location, ensuring that the `data`
    directory is created relative to the Go file itself, making the script
    more portable.

**Workflow for Demo Data Usage:**

```
generate_data.go --(generates)--> /demo/data/*.json files
                                         |
                                         V
Perf Ingester (type 'dir', configured to read from /demo/data/)
                                         |
                                         V
Perf System (stores, analyzes, and visualizes the data)
```

The demo data is specifically designed to be used in conjunction with the
`perf/configs/demo.json` configuration file and the
`https://github.com/skia-dev/perf-demo-repo.git` repository. This linkage
provides a complete, albeit simplified, end-to-end scenario for demonstrating
Perf.

# Module: /go

## Module: /go

This main module, located at `/go`, serves as the root for all Go language
components of the Perf performance monitoring system. It encompasses a wide
array of functionalities, from data ingestion and storage to analysis, alerting,
and user interface backend logic. The design promotes modularity, with specific
responsibilities delegated to sub-modules.

The system is designed to handle large volumes of performance data, track it
against code revisions, detect regressions automatically, and provide tools for
developers and performance engineers to investigate and manage performance.

### Key Design Philosophies and Architectural Choices:

1.  **Modularity:** The system is broken down into numerous sub-modules (e.g.,
    `/go/alerts`, `/go/ingest`, `/go/regression`, `/go/frontend`), each with a
    well-defined responsibility. This promotes separation of concerns, making
    the system easier to develop, test, and maintain.
2.  **Interface-Based Design:** Many modules define interfaces for their core
    components (e.g., `tracestore.Store`, `alerts.Store`, `regression.Store`).
    This allows for different implementations to be swapped in (e.g., SQL-based
    stores vs. in-memory mocks for testing) and promotes loose coupling.
3.  **Configuration-Driven Behavior:** The `/go/config` module defines a
    comprehensive `InstanceConfig` structure, which is loaded from a JSON file.
    This configuration dictates many aspects of an instance's behavior,
    including database connections, data sources, alert settings, and UI
    features. This allows for flexible deployment and customization of Perf
    instances.
4.  **Asynchronous Processing and Workflows:** For long-running tasks like data
    ingestion, regression detection, and bisection, the system leverages
    asynchronous processing.
    - Go routines are widely used for concurrent operations.
    - The `/go/progress` module provides a mechanism for tracking and
      reporting the status of such tasks to the UI.
    - The `/go/workflows` module utilizes Temporal to orchestrate complex,
      multi-step processes like triggering bisections and processing their
      results. Temporal provides resilience and fault tolerance for these
      critical operations.
5.  **Data Storage and Retrieval:**
    - **SQL Database:** A relational database (primarily targeting
      CockroachDB, with Spanner compatibility) is the main persistence layer
      for most structured data, including alert configurations (`/go/alerts`),
      regression details (`/go/regression`), commit information (`/go/git`),
      user favorites (`/go/favorites`), subscriptions (`/go/subscription`),
      and more. The `/go/sql` module manages the database schema.
    - **Trace Data (`/go/tracestore`):** Performance trace data is stored in a
      tiled fashion, with inverted indexes to allow for efficient querying.
      This specialized storage approach is optimized for time-series
      performance metrics.
    - **File Storage (GCS):** Raw ingested data files and potentially other
      large artifacts are often stored in Google Cloud Storage. The `/go/file`
      and `/go/filestore` modules provide abstractions for interacting with
      these files.
6.  **Caching:** Various caching strategies are employed to improve performance:
    - In-memory LRU caches for frequently accessed data (e.g., in `/go/git`,
      `/go/progress`).
    - A dedicated `/go/tracecache` for trace IDs.
    - The `/go/psrefresh` module manages caching of `ParamSet`s (used for UI
      query builders), potentially using Redis (`/go/redis`).
    - `/go/graphsshortcut` offers an in-memory cache for graph shortcuts,
      especially for development.
7.  **External Service Integration:**
    - **Git:** The `/go/git` module interacts with Git repositories (via local
      CLI or Gitiles API) to fetch commit information.
    - **Issue Trackers:** Modules like `/go/issuetracker` and `/go/culprit`
      integrate with issue tracking systems (e.g., Buganizer) for automated
      bug filing.
    - **Chrome Perf:** The `/go/chromeperf` module allows communication with
      the Chrome Performance Dashboard for reporting regressions or fetching
      anomaly data.
    - **Pinpoint:** The `/go/pinpoint` module provides a client for the
      Pinpoint bisection service.
    - **LUCI Config:** The `/go/sheriffconfig` module integrates with LUCI
      Config for managing alert configurations.
8.  **Command-Line Tools:**
    - `/go/perfserver`: The main executable for running different Perf
      services (frontend, ingestion, clustering, maintenance).
    - `/go/perf-tool`: A CLI for various administrative and data inspection
      tasks.
    - `/go/initdemo`: A tool to initialize a database for demo or development.
    - `/go/ts`: A utility to generate TypeScript definitions from Go structs
      for frontend type safety.

### Core Workflows (Conceptual High-Level):

1.  **Data Ingestion:**

    ```
    External Data Source (e.g., GCS event)
        |
        V
    /go/file (Source Interface: DirSource, GCSSource) --> Raw File Data
        |
        V
    /go/ingest/process (Orchestrator)
        |
        +--> /go/ingest/parser (Parses file based on /go/ingest/format) --> Extracted Traces & Metadata
        |
        +--> /go/git (Resolves Git hash to CommitNumber)
        |
        V
    /go/tracestore (Writes traces, updates inverted index & ParamSets)
        |
        V
    /go/ingestevents (Publishes event: "File Ingested")
    ```

2.  **Regression Detection (Event-Driven Example):**

    ```
    /go/ingestevents (Receives "File Ingested" event)
        |
        V
    /go/regression/continuous (Controller)
        |
        +--> /go/alerts (Loads matching Alert configurations)
        |
        +--> /go/dfiter & /go/dataframe & /go/dfbuilder (Prepare DataFrames for analysis)
        |
        V
    /go/regression/detector (Core detection logic)
        |
        +--> /go/clustering2 (KMeans clustering)
        |
        +--> /go/stepfit (Individual trace step detection)
        |
        V
    Detected Regressions
        |
        +--> /go/regression (Store results using Store interface, e.g., sqlregression2store)
        |
        +--> /go/notify (Format & send notifications via Email, IssueTracker, Chromeperf)
        |
        +--> /go/workflows (MaybeTriggerBisectionWorkflow for potential bisection)
    ```

3.  **User Interaction (Frontend Request for Graph):**

    ```
    User in Browser (Requests graph)
        |
        V
    /go/frontend (HTTP Handlers, e.g., graphApi)
        |
        +--> /go/ui/frame (ProcessFrameRequest)
        |    |
        |    +--> /go/dataframe/dfbuilder (Builds DataFrame based on query)
        |    |      |
        |    |      +--> /go/tracestore (Fetch trace data)
        |    |      +--> /go/git (Fetch commit data)
        |    |
        |    +--> /go/calc (If formulas are used)
        |    |
        |    +--> /go/pivot (If pivot table requested)
        |    |
        |    +--> /go/anomalies (Fetch anomaly data to overlay)
        |
        V
    FrameResponse (JSON data for UI) --> User in Browser
    ```

4.  **Automated Bisection via Temporal Workflow:**
    `/go/workflows.MaybeTriggerBisectionWorkflow (Triggered by significant
regression) | +--> Waits for related anomalies to group | +-->
/go/anomalygroup (Loads anomaly group details) | +--> If GroupAction ==
BISECT: | | | +--> /go/gerrit (Activity: Get commit hashes from positions) |
| | +--> Executes Pinpoint.CulpritFinderWorkflow (Child Workflow) | |
(Pinpoint performs bisection) | V | Pinpoint calls back to
/go/workflows.ProcessCulpritWorkflow | | | +--> /go/culprit (Activity:
Persist culprit & Notify user) | +--> If GroupAction == REPORT: | +-->
/go/culprit (Activity: Notify user of anomaly group)`

### Sub-Module Summaries (Illustrative, not exhaustive):

- **/go/alertfilter**: Constants for alert filtering modes (e.g., `ALL`,
  `OWNER`). Ensures consistent filter definitions.
- **/go/alerts**: Manages `Alert` configurations, their storage
  (`sqlalertstore`), and efficient retrieval (`ConfigProvider` with caching).
  Defines how performance regressions are detected.
- **/go/anomalies**: Retrieves anomaly data, often by proxying to Chrome Perf,
  with a caching layer to improve performance.
- **/go/anomalygroup**: Groups related anomalies to consolidate actions like
  bug filing or bisection. Uses a gRPC service and SQL store.
- **/go/backend**: A gRPC backend service for internal, non-UI-facing APIs,
  promoting stable interfaces.
- **/go/builders**: Centralized factory for creating core components (data
  stores, Git client) based on instance configuration, preventing cyclic
  dependencies.
- **/go/bug**: Generates URLs for reporting bugs to issue trackers using
  configurable URI templates.
- **/go/calc**: (Not detailed in provided docs, but generally) Evaluates
  formulas on trace data.
- **/go/chromeperf**: Client for interacting with the Chrome Performance
  Dashboard API (reporting regressions, fetching anomalies).
- **/go/clustering2**: Implements k-means clustering for grouping similar
  performance traces.
- **/go/config**: Defines and validates the `InstanceConfig` structure (loaded
  from JSON) that governs a Perf instance.
- **/go/ctrace2**: Adapts trace data (normalization, handling missing points)
  for use with k-means clustering.
- **/go/culprit**: Manages identified culprits (commits causing regressions),
  their storage, and notification.
- **/go/dataframe**: Provides the `DataFrame` structure for handling
  performance data in a tabular, commit-centric way, inspired by R's
  dataframes.
- **/go/dfbuilder**: Constructs `DataFrame` objects from `TraceStore`,
  handling query logic and data aggregation.
- **/go/dfiter**: Iterates over `DataFrame`s, typically by slicing a larger
  fetched frame. Used in regression detection.
- **/go/dryrun**: Allows testing alert configurations without creating actual
  alerts, simulating regression detection.
- **/go/favorites**: Manages user-saved favorite configurations/views, stored
  in SQL.
- **/go/file**: Defines `File` and `Source` interfaces for abstracting file
  access from different origins (local, GCS via Pub/Sub).
- **/go/filestore**: Implements `fs.FS` for local and GCS file access,
  providing a unified way to read files.
- **/go/frontend**: Backend for the Perf web UI, handling HTTP requests,
  rendering templates, and interacting with data stores.
- **/go/git**: Abstraction for Git repository interaction, caching commit data
  in SQL, with providers for local CLI and Gitiles.
- **/go/graphsshortcut**: Manages shortcuts for collections of graph
  configurations, using hashed IDs for de-duplication.
- **/go/ingest**: Orchestrates the data ingestion pipeline: reading files,
  parsing formats, and writing to `TraceStore`.
- **/go/ingestevents**: Defines and handles serialization/deserialization of
  ingestion completion events for PubSub.
- **/go/initdemo**: CLI tool to initialize a database (CockroachDB or Spanner
  emulator) with the Perf schema.
- **/go/issuetracker**: Provides an interface and implementation for
  interacting with Google Issue Tracker (Buganizer).
- **/go/kmeans**: Generic k-means clustering algorithm implementation using
  interfaces for flexibility.
- **/go/maintenance**: Runs background tasks like Git repo sync, regression
  migration, query cache refresh, and old data deletion.
- **/go/notify**: Framework for formatting and sending notifications (email,
  issue tracker) about regressions.
- **/go/notifytypes**: Defines constants for different notification mechanisms
  and data providers.
- **/go/perf-tool**: CLI for Perf administration (config management, data
  inspection, database maintenance).
- **/go/perfclient**: Client for pushing performance data (typically from
  trybots) to Perf's GCS ingestion endpoint.
- **/go/perfresults**: Fetches, parses, and processes performance results from
  Telemetry benchmarks (Chromium).
- **/go/perfserver**: Main executable for Perf, consolidating frontend,
  ingestion, clustering, and maintenance services.
- **/go/pinpoint**: Client for the Pinpoint (Chromeperf) bisection service.
- **/go/pivot**: Aggregates and summarizes trace data within a `DataFrame`
  based on specified grouping criteria (like pivot tables).
- **/go/progress**: Tracks the progress of long-running backend tasks and
  exposes it to the UI via HTTP polling.
- **/go/psrefresh**: Manages and caches `paramtools.ParamSet` instances (used
  for UI query builders) to improve performance.
- **/go/redis**: Manages interaction with Redis for caching, primarily to
  support the query UI.
- **/go/regression**: Core module for detecting, storing, and managing
  performance regressions.
- **/go/samplestats**: Performs statistical analysis on sets of performance
  data to identify significant changes between "before" and "after" states.
- **/go/sheriffconfig**: Manages Sheriff Configurations (alerting rules
  defined in Protobuf, stored in LUCI Config), importing them into Perf.
- **/go/shortcut**: Manages shortcuts for lists of trace keys, using hashed
  IDs.
- **/go/sql**: Central module for SQL database schema management (definition,
  generation, validation, migration).
- **/go/stepfit**: Analyzes time-series data to detect significant changes
  ("steps") using various statistical algorithms.
- **/go/subscription**: Manages alerting subscriptions, defining how to react
  to anomalies (e.g., bug filing details).
- **/go/tracecache**: Caches trace identifiers for specific tiles and queries
  to improve performance.
- **/go/tracefilter**: Filters trace data based on hierarchical paths,
  identifying "leaf" traces.
- **/go/tracesetbuilder**: Efficiently constructs `TraceSet` and
  `ReadOnlyParamSet` objects from multiple, potentially disparate chunks of
  trace data using a worker pool.
- **/go/tracestore**: Defines interfaces and SQL implementations for storing
  and retrieving performance trace data, using a tiled storage approach.
- **/go/tracing**: Initializes and configures distributed tracing capabilities
  using OpenCensus.
- **/go/trybot**: Manages performance data from trybots (pre-submit tests),
  including ingestion, storage, and analysis.
- **/go/ts**: Utility to generate TypeScript definition files from Go structs
  for frontend type safety.
- **/go/types**: Defines core data types used throughout Perf (e.g.,
  `CommitNumber`, `TileNumber`, `Trace`).
- **/go/ui**: Handles frontend requests and prepares data for display,
  bridging UI interactions with backend data sources.
- **/go/urlprovider**: Generates URLs for various pages within the Perf
  application consistently.
- **/go/userissue**: Manages associations between specific data points (trace
  key + commit position) and Buganizer issues.
- **/go/workflows**: Defines and implements Temporal workflows for automating
  tasks like bisection triggering and culprit processing.

This comprehensive suite of modules works together to provide the Skia Perf
performance monitoring system.

# Module: /go/alertfilter

This module, `go/alertfilter`, provides constants that define different
filtering modes for alerts. These constants are used throughout the Perf
application to control which alerts are displayed or processed.

The primary motivation behind this module is to centralize the definition of
alert filtering options. By having these constants in a dedicated module, we
avoid scattering magic strings like "ALL" or "OWNER" throughout the codebase.
This improves maintainability, reduces the risk of typos, and makes it easier to
understand and modify the filtering logic. If new filtering modes are needed in
the future, they can be added here, providing a single source of truth.

**Key Components/Files:**

- **`alertfilter.go`**: This is the sole file in this module. It defines the
  string constants used for alert filtering.
  - **`ALL`**: This constant represents a filter that includes all alerts,
    irrespective of their owner or other properties. It is used when a user
    or a system process needs to view or operate on the entire set of active
    alerts.
  - **`OWNER`**: This constant represents a filter that includes only alerts
    assigned to a specific owner. This is crucial for user-specific views
    where individuals only want to see alerts relevant to their
    responsibilities.

**Workflow/Usage Example:**

Imagine a user interface for viewing alerts. The user might have a dropdown to
select how they want to filter the alerts.

```
User Interface:
  [Alert List]
  Filter: [Dropdown: "ALL", "OWNER"]

Backend Logic:
  func GetAlerts(filterMode string, userID string) []Alert {
    if filterMode == alertfilter.ALL {
      // Fetch all alerts from the database.
      return database.GetAllAlerts()
    } else if filterMode == alertfilter.OWNER {
      // Fetch alerts owned by the current user.
      return database.GetAlertsByOwner(userID)
    }
    // ... other filter modes or error handling
  }
```

In this scenario, the backend uses the constants from the `alertfilter` module
to determine the correct query to execute against the database. This ensures
consistency and clarity in how filtering is applied.

# Module: /go/alerts

The `/go/alerts` module is responsible for managing alert configurations within
the Perf application. These configurations define the conditions under which
users or systems should be notified about performance regressions. The module
handles the definition, storage, retrieval, and caching of these alert
configurations.

A core design principle is the separation of concerns between defining an
alert's structure (`config.go`), providing access to these configurations
(`configprovider.go`), and persisting them (`store.go` and its SQL
implementation in `sqlalertstore`). This modularity allows for flexibility in
how alerts are stored (e.g., potentially different database backends) and
accessed.

**Key Components and Responsibilities:**

- **`config.go`**: This file defines the `Alert` struct, which is the central
  data structure representing a single alert configuration.

  - **Why**: It encapsulates all the parameters necessary to define an
    alert, such as the query to select relevant performance traces, the
    notification destination (email or issue tracker), thresholds for
    triggering, clustering algorithms, and the desired action (e.g., report,
    bisect).
  - **How**: The `Alert` struct includes fields for:
  - `IDAsString`: A string representation of the alert's unique identifier.
    This is used for JSON serialization to avoid potential issues with large
    integer handling in JavaScript. The `BadAlertID` and
    `BadAlertIDAsAsString` constants represent an invalid/uninitialized ID.
  - `Query`: A URL-encoded string that defines the criteria for selecting
    traces from the performance data.
  - `GroupBy`: A comma-separated list of parameter keys. If specified, the
    `Query` is expanded into multiple sub-queries, one for each unique
    combination of values for the `GroupBy` keys found in the data. This
    allows for more granular alerting. The `GroupCombinations` and
    `QueriesFromParamset` methods handle this expansion.
  - `Alert`: The email address for notifications.
  - `IssueTrackerComponent`: The ID of the issue tracker component to file
    bugs against. A custom `SerializesToString` type is used for this field
    to handle JSON serialization of the int64 component ID as a string, with
    `0` serializing to `""`.
  - `DirectionAsString`: Specifies whether to alert on upward (`UP`),
    downward (`DOWN`), or both (`BOTH`) changes in performance. This
    replaces the deprecated `StepUpOnly` boolean.
  - `StateAsString`: Indicates if the alert is `ACTIVE` or `DELETED`. This
    is managed internally and affects whether an alert is processed.
  - `Action`: Defines what action to take when an anomaly is detected (e.g.,
    `types.AlertActionReport`, `types.AlertActionBisect`).
  - Other fields like `Interesting`, `Algo`, `Step`, `Radius`, `K`,
    `Sparse`, `MinimumNum`, `Category` control the specifics of regression
    detection and reporting.
  - The file also defines enums like `Direction` and `ConfigState` and
    helper functions for ID conversion and validation (`Validate`). The
    `Validate` function ensures consistency, for example, that `GroupBy`
    keys do not also appear in the main `Query`.

- **`store.go`**: This file defines the `Store` interface, which abstracts the
  persistence mechanism for `Alert` configurations.

  - **Why**: Decoupling the alert logic from the specific storage
    implementation (e.g., SQL, Datastore) makes the system more adaptable
    and testable.
  - **How**: The `Store` interface specifies methods for:
  - `Save`: Saving a new or updating an existing alert. It takes a
    `SaveRequest` which includes the `Alert` configuration and an optional
    `SubKey` (linking the alert to a subscription).
  - `ReplaceAll`: Atomically replacing all existing alerts with a new set.
    This is useful for bulk updates, often tied to configuration
    subscriptions. It requires a `pgx.Tx` to ensure transactional integrity.
  - `Delete`: Marking an alert as deleted.
  - `List`: Retrieving alerts, with an option to include deleted ones.
    Alerts are typically sorted by `DisplayName`.
  - `ListForSubscription`: Retrieving all active alerts associated with a
    specific subscription name.

- **`configprovider.go`**: This file implements a `ConfigProvider` that serves
  `Alert` configurations, incorporating a caching layer.

  - **Why**: To provide efficient and responsive access to alert
    configurations, especially in a high-traffic system. Repeatedly fetching
    from the underlying `Store` for every request would be inefficient.
  - **How**:
  - `configProviderImpl` implements the `ConfigProvider` interface.
  - It maintains two internal caches (`cache_active` for active alerts and
    `cache_all` for all alerts including deleted ones) using the
    `configCache` struct.
  - Upon initialization (`NewConfigProvider`), it performs an initial
    refresh and starts a background goroutine that periodically calls
    `Refresh` to update the caches from the `Store`.
  - `GetAllAlertConfigs` and `GetAlertConfig` serve data from these caches.
  - A `sync.RWMutex` is used to protect concurrent access to the caches.
  - The `Refresh` method explicitly fetches data from the `alertStore` and
    updates both caches.
  - The refresh interval is configurable.

- **Submodule `sqlalertstore`**: This submodule provides a SQL-based
  implementation of the `alerts.Store` interface.

  - **`sqlalertstore.go`**:
  - **Why**: To persist alert configurations in a relational database
    (specifically CockroachDB, with Spanner compatibility).
  - **How**: The `SQLAlertStore` struct holds a database connection pool
    (`pool.Pool`) and a map of SQL statements.
    - Alerts are stored as JSON strings in the `Alerts` table (schema
      defined in `sqlalertstore/schema/schema.go`). This simplifies schema
      evolution of the `Alert` struct itself, as changes to the struct
      don't always require immediate SQL schema migrations, though it
      makes querying based on specific alert fields harder directly in
      SQL.
    - `Save`: For new alerts (ID is `BadAlertIDAsAsString`), it performs
      an `INSERT` and retrieves the generated ID. For existing alerts, it
      performs an `UPSERT` (or an `INSERT ... ON CONFLICT DO UPDATE` for
      Spanner).
    - `Delete`: Marks an alert as deleted by setting its `config_state` to
      `1` (representing `alerts.DELETED`) and updates `last_modified`. It
      doesn't physically remove the row.
    - `ReplaceAll`: Within a transaction, it first marks all existing
      active alerts as deleted, then inserts the new set of alerts.
    - `List` and `ListForSubscription`: Query the `Alerts` table,
      deserialize the JSON `alert` column into `alerts.Alert` structs, and
      sort them by `DisplayName`.
  - **`spanner.go`**: Contains Spanner-specific SQL statements. This is
    necessary because CockroachDB and Spanner have slightly different SQL
    syntax for certain operations like UPSERTs and RETURNING clauses. The
    correct set of statements is chosen in `sqlalertstore.New` based on the
    `dbType`.
  - **`sqlalertstore/schema/schema.go`**: Defines the Go struct
    `AlertSchema` representing the `Alerts` table in the SQL database. Key
    fields include `id`, `alert` (TEXT, storing the JSON serialized
    `alerts.Alert`), `config_state` (INT), `last_modified` (INT, Unix
    timestamp), `sub_name`, and `sub_revision`.

**Key Workflows:**

1.  **Creating/Updating an Alert:**

    - User/System constructs an `alerts.Alert` struct.
    - `alerts.Store.Save()` is called.
    - If SQL-backed:
      - `sqlalertstore.Save()` serializes the `Alert` to JSON.
      - If `IDAsString` is `BadAlertIDAsAsString`, an `INSERT` statement is
        executed, and the new ID is populated back into the `Alert` struct.
      - Otherwise, an `UPSERT` or `INSERT ... ON CONFLICT DO UPDATE`
        statement is executed.
    - The `ConfigProvider`'s cache will eventually be updated during its next
      refresh cycle.

    ```
    [Client/Service] -- Alert Data --> [alerts.Store.Save()]
                                            |
                                            v
                              [sqlalertstore.Save()] -- Serializes Alert to JSON --> [Database]
                                            | (If new, DB returns ID)
                                            <---------------------------------------
                                            | (Updates Alert struct with ID)
                                            v
    [ConfigProvider.Refresh() periodically] --> [alerts.Store.List()]
                                                        |
                                                        v
                                          [sqlalertstore.List()] --> [Database]
                                                        | (Reads & deserializes)
                                                        v
                                              [ConfigProvider Cache Update]
    ```

2.  **Retrieving All Active Alerts:**

    - A service requests alert configurations via
      `alerts.ConfigProvider.GetAllAlertConfigs(ctx, false)`.
    - `configProviderImpl.GetAllAlertConfigs()` checks its `cache_active`.
    - If the cache is up-to-date (within refresh interval), it returns the
      cached `[]*Alert`.
    - If the cache needs refresh (or it's the first call), the background
      refresher (or an explicit `Refresh` call) would have populated it by:
      - Calling `alerts.Store.List(ctx, false)`.
      - Which in turn calls `sqlalertstore.List(ctx, false)`.
      - `sqlalertstore` queries the database for alerts where
        `config_state = 0` (ACTIVE), deserializes them, and returns the
        list.

    ```
    [Service] -- Request All Active Alerts --> [ConfigProvider.GetAllAlertConfigs(includeDeleted=false)]
                                                    | (Checks cache_active)
                                                    |
                                                    +-- [Cache Hit] ----> Returns cached []*Alert
                                                    |
                                                    +-- [Cache Miss/Stale (via periodic Refresh)]
                                                        |
                                                        v
                                         [alerts.Store.List(includeDeleted=false)]
                                                        |
                                                        v
                                     [sqlalertstore.List(includeDeleted=false)] -- SQL Query (WHERE config_state=0) --> [Database]
                                                        | (Reads & deserializes)
                                                        v
                                              [Updates & Returns from Cache]
    ```

3.  **Expanding `GroupBy` Queries:**

    - When an alert with a `GroupBy` clause is processed (e.g., by the
      regression detection system), `Alert.QueriesFromParamset(paramset)` is
      called.
    - `Alert.GroupCombinations(paramset)` is invoked to find all unique
      combinations of values for the keys specified in `GroupBy` from the
      provided `paramtools.ReadOnlyParamSet`.
    - For each combination, a new query string is generated by taking the
      original `Alert.Query` and appending the key-value pairs from the
      combination.
    - This results in a list of specific queries to be executed against the
      trace data.

    ```
    [Alert Processing System] -- Has Alert with GroupBy="config,arch", Query="metric=latency" & ParamSet --> [Alert.QueriesFromParamset()]
                                                                                                                |
                                                                                                                v
                                                                                                  [Alert.GroupCombinations()]
                                                                                                                | (e.g., finds {config:A, arch:X}, {config:B, arch:X})
                                                                                                                v
                                                                                                  [Generates specific queries:]
                                                                                                     - "metric=latency&config=A&arch=X"
                                                                                                     - "metric=latency&config=B&arch=X"
                                                                                                                |
                                                                                                                <-- Returns []string (list of queries)
    ```

The use of `SerializesToString` for `IssueTrackerComponent` highlights a common
challenge when interfacing Go backend systems with JavaScript frontends:
JavaScript's limitations with handling large integer IDs. Serializing them as
strings is a robust workaround.

The existence of a `mock` subdirectory with generated mocks for `Store` and
`ConfigProvider` (using `stretchr/testify/mock`) is standard Go practice,
facilitating unit testing of components that depend on these interfaces without
needing a real database or complex setup.

# Module: /go/anomalies

The `/go/anomalies` module is responsible for retrieving anomaly data. Anomalies
represent significant deviations in performance metrics. This module acts as an
intermediary between the application and the `chromeperf` service, which is the
source of truth for anomaly data. It provides an abstraction layer, potentially
including caching, to optimize anomaly retrieval.

### Key Components and Responsibilities

**1. `anomalies.go`:**

- **Purpose:** Defines the `Store` interface. This interface dictates the
  contract for any component that aims to provide anomaly data. It ensures
  that different implementations (e.g., a cached store or a direct passthrough
  store) can be used interchangeably.
- **Why:** Separating the interface from the implementation promotes loose
  coupling and testability. It allows for different strategies for fetching
  anomalies without changing the consuming code.
- **Key Methods:**
  - `GetAnomalies`: Retrieves anomalies for a list of trace names within a
    specific commit position range. This is useful for analyzing performance
    regressions or improvements tied to code changes.
  - `GetAnomaliesInTimeRange`: Fetches anomalies within a given time window.
    This is helpful for time-based analysis, independent of specific commit
    versions.
  - `GetAnomaliesAroundRevision`: Finds anomalies that occurred near a
    particular revision (commit). This helps pinpoint performance changes
    related to a specific code submission.

**2. `impl.go`:**

- **Purpose:** Provides a basic, non-caching implementation of the `Store`
  interface. It directly forwards requests to the
  `chromeperf.AnomalyApiClient`.
- **Why:** This serves as a foundational implementation. It's simple and
  directly reflects the capabilities of the underlying `chromeperf` service.
  It can be used when caching is not desired or not yet implemented.
- **How:** Each method in the `store` struct (the implementation of `Store`)
  makes a corresponding call to the `ChromePerf` client. For example,
  `GetAnomalies` calls `ChromePerf.GetAnomalies`. Error handling is included
  to log failures from the `chromeperf` service. Trace names are sorted before
  being passed to `chromeperf` which might be a requirement or an optimization
  for the `chromeperf` API.

**3. `/go/anomalies/cache/cache.go`:**

- **Purpose:** Implements a caching layer for the `Store` interface. This is
  designed to improve performance by reducing the number of direct calls to
  the `chromeperf` service, which can be network-intensive.
- **Why:** Repeatedly fetching the same anomaly data can be inefficient. A
  cache stores frequently accessed or recent anomalies locally, leading to
  faster response times and reduced load on the `chromeperf` service.
- **How:**
  - **LRU Cache:** Uses two Least Recently Used (LRU) caches: `testsCache`
    for anomalies queried by trace names and commit ranges, and
    `revisionCache` for anomalies queried around a specific revision. LRU
    ensures that the least accessed items are evicted when the cache reaches
    its `cacheSize` limit.
  - **Cache Invalidation:**
  - **TTL (Time-To-Live):** Cache entries have a `cacheItemTTL`. A periodic
    `cleanupCache` goroutine removes entries older than this TTL. This
    ensures that stale data doesn't persist indefinitely.
  - **`invalidationMap`:** This map tracks trace names for which anomalies
    have been modified (e.g., an alert was updated). If a trace name is in
    this map, any cached anomalies for that trace are considered invalid and
    will be re-fetched from `chromeperf`.
    - The `invalidationMap` itself is cleared periodically
      (`invalidationCleanupPeriod`) to prevent it from growing too large.
      This is a trade-off: it's simpler and has lower memory overhead but
      can lead to inaccuracies if a trace is invalidated and then the map
      is cleared before the next fetch for that trace.
  - **Metrics:** Tracks the `numEntriesInCache` to monitor cache
    utilization.
- **Key Methods (`store` struct in `cache.go`):**
  - `GetAnomalies`:
  * Attempts to retrieve anomalies from `testsCache`.
  * Checks the `invalidationMap`. If a trace is marked invalid, it's treated
    as a cache miss.
  * For any cache misses or invalidated traces, it fetches the data from
    `as.ChromePerf.GetAnomalies`.
  * Populates the `testsCache` with newly fetched data. `Client Request
(traceNames, startCommit, endCommit) | v [Cache Store] -- GetAnomalies()
| +---------------------------------+ | For each traceName: | | 1. Check
testsCache | ----> Cache Hit? -----> Add to Result | (Key:
trace:start:end) | | | 2. Check invalidationMap | No (Cache Miss or
Invalidated) +---------------------------------+ | |
(traceNamesMissingFromCache) | v | [ChromePerf Client] -- GetAnomalies()
-----------+ | v [Cache Store] -- Add new data to testsCache | v Return
Combined Result`
  - `GetAnomaliesInTimeRange`: This method currently bypasses the cache and
    directly calls `as.ChromePerf.GetAnomaliesTimeBased`. The decision to
    not cache time-based queries might be due to the potentially large and
    less frequently reused nature of such requests, or it might be a feature
    planned for later.
  - `GetAnomaliesAroundRevision`: Similar to `GetAnomalies`, it first checks
    `revisionCache`. If it's a miss, it fetches from
    `as.ChromePerf.GetAnomaliesAroundRevision` and updates the cache.
  - `InvalidateTestsCacheForTraceName`: Adds a `traceName` to the
    `invalidationMap`. This is likely called when an external event (e.g.,
    user updating an anomaly in Chrome Perf) indicates that the cached data
    for this trace is no longer accurate.

**4. `/go/anomalies/mock/Store.go`:**

- **Purpose:** Provides a mock implementation of the `Store` interface,
  generated using the `testify/mock` library.
- **Why:** Essential for unit testing. It allows other components that depend
  on the `anomalies.Store` to be tested in isolation, without needing a real
  `chromeperf` instance or a fully functional cache. Developers can define
  expected calls and return values for the mock store.
- **How:** It's an auto-generated file. The `mock.Mock` struct from
  `stretchr/testify` is embedded, providing methods like `On()`, `Return()`,
  and `AssertExpectations()` to control and verify the mock's behavior during
  tests.

### Design Decisions and Rationale

- **Interface-based Design (`anomalies.Store`):** This is a common and robust
  pattern in Go. It allows for flexibility in how anomalies are fetched and
  managed. For example, a new caching strategy or a different backend data
  source could be implemented without affecting code that consumes anomalies,
  as long as the new implementation adheres to the `Store` interface.
- **Caching Strategy (`cache.go`):**
  - **LRU:** A good general-purpose caching algorithm when memory is limited
    and recent/frequently accessed items are more likely to be requested
    again.
  - **TTL for Cache Items:** Prevents indefinitely storing stale data.
  - **`invalidationMap`:** A pragmatic approach to handling external data
    modifications. While not perfectly accurate (invalidates all anomalies
    for a trace even if only one changed, and susceptible to the
    `invalidationCleanupPeriod` timing), it's simpler and less
    memory-intensive than more granular invalidation schemes. This suggests
    a balance was struck between accuracy, complexity, and resource usage.
  - **Separate Caches (`testsCache`, `revisionCache`):** Likely done because
    the query patterns and cache keys for these two types of requests are
    different. `testsCache` uses a composite key
    (`traceName:startCommit:endCommit`), while `revisionCache` uses the
    `revision` number as the key.
- **Error Handling:** The implementations generally log errors from
  `chromeperf` but often return an empty `AnomalyMap` or `nil` slice to the
  caller in case of an error from the underlying service. This design choice
  means that callers might receive no data instead of an error, simplifying
  the caller's error handling logic but potentially obscuring issues if not
  monitored through logs.
- **Sorting Trace Names:** Before calling `chromeperf.GetAnomalies` or
  `chromeperf.GetAnomaliesTimeBased`, the list of `traceNames` is sorted. This
  could be a requirement of the `chromeperf` API for deterministic behavior,
  or an optimization to improve `chromeperf`'s internal processing or caching.
- **Tracing (`go.opencensus.io/trace`):** Spans are added to some methods
  (`GetAnomaliesInTimeRange`, `GetAnomaliesAroundRevision`). This is crucial
  for observability, allowing developers to track the performance and flow of
  requests through the system, especially in a distributed environment.

### Workflows

**Typical Anomaly Retrieval (with Cache):**

1.  A service needs anomalies (e.g., for displaying on a dashboard).
2.  It calls one of the `GetAnomalies*` methods on an `anomalies.Store` instance
    (which is likely the cached `store` from `cache.go`).
3.  **Cache Check:**
    - The cached `store` first checks its internal LRU cache(s) (`testsCache`
      or `revisionCache`) for the requested data.
    - For `GetAnomalies`, it also consults the `invalidationMap` to see if any
      relevant traces have been marked as stale.
4.  **Cache Hit:** If valid data is found in the cache, it's returned directly.
    `Caller -> anomalies.Store.GetAnomalies(traces, range) | v
Cache.GetAnomalies() | +--> Check testsCache (e.g., trace1:100:200) -> Found
& Valid | +--> Check testsCache (e.g., trace2:100:200) -> Not Found or
Invalid | Return cached data for trace1`
5.  **Cache Miss / Stale Data:** If data is not in the cache or is marked stale: - The cached `store` makes a network request to the
    `chromeperf.AnomalyApiClient`. - The response from `chromeperf` is received. - This new data is added to the LRU cache for future requests. - The data is returned to the caller. `Caller ->
anomalies.Store.GetAnomalies(traces, range) | v Cache.GetAnomalies() |
+--> Check testsCache (e.g., trace1:100:200) -> Found & Valid | +-->
Check testsCache (e.g., trace2:100:200) -> Not Found or Invalid | | |
(Data for trace1) v +---------------------------> [ ChromePerf API ] --
GetAnomalies(trace2, range) | v Cache.Add(trace2_data) | v Combine
trace1_data & trace2_data | v Return to Caller`

**Cache Invalidation Workflow:**

1.  An external event occurs (e.g., a user triages an anomaly in the Chrome Perf
    UI, which modifies its state).
2.  A mechanism (not detailed within this module, but implied) detects this
    change.
3.  This mechanism calls `cache.store.InvalidateTestsCacheForTraceName(ctx,
"affected_trace_name")`.
4.  The `affected_trace_name` is added to the `invalidationMap` in the
    `cache.store`.
5.  **Next `GetAnomalies` call for `affected_trace_name`:**
    - Even if `testsCache` contains an entry for this trace and range, the
      presence of `affected_trace_name` in `invalidationMap` will cause a
      cache miss.
    - Data will be re-fetched from `chromeperf`.
    - The `invalidationMap` entry for `affected_trace_name` typically remains
      until the `invalidationMap` is periodically cleared.

This module effectively decouples the rest of the Perf application from the
direct complexities of interacting with `chromeperf` for anomaly data, offering
performance benefits through caching and a consistent interface for data
retrieval.

# Module: /go/anomalygroup

The `anomalygroup` module is designed to group related anomalies (regressions in
performance metrics) together. This grouping allows for consolidated actions
like filing a single bug report for multiple related regressions or triggering a
single bisection job to find the common culprit for a set of anomalies. This
approach aims to reduce noise and improve the efficiency of triaging performance
regressions.

The core idea is to identify anomalies that share common characteristics, such
as the subscription (alert configuration), benchmark, and commit range. When a
new anomaly is detected, the system attempts to find an existing group that
matches these criteria. If a suitable group is found, the new anomaly is added
to it. Otherwise, a new group is created.

The module defines a gRPC service for managing anomaly groups, a storage
interface for persisting group data, and utilities for processing regressions
and interacting with the grouping logic.

### Key Components and Responsibilities

#### `store.go`: Anomaly Group Storage Interface

The `store.go` file defines the `Store` interface, which outlines the contract
for persisting and retrieving anomaly group data. This abstraction allows for
different storage backends (e.g., SQL databases) to be used.

**Key Responsibilities:**

- **Creating new anomaly groups:** When a new anomaly doesn't fit into an
  existing group, a new group record needs to be created. This involves
  storing metadata about the group, such as the subscription details,
  benchmark, initial commit range, and the action to be taken (e.g., REPORT,
  BISECT).
- **Loading anomaly groups:** Retrieving group information by its unique ID is
  essential for processing and taking actions on the group.
- **Finding existing groups:** This is a crucial part of the grouping logic.
  When a new anomaly is detected, the store is queried to find existing groups
  that match criteria like subscription, revision, domain (master), benchmark,
  commit range, and action type.
- **Updating anomaly groups:** Groups are dynamic. As new anomalies are added,
  or as actions are taken (e.g., bisection started, bug filed), the group
  record needs to be updated. This includes:
  - Adding new anomaly IDs to the group.
  - Adding culprit commit IDs once a bisection identifies them.
  - Storing the ID of a bisection job associated with the group.
  - Storing the ID of a reported issue (bug) associated with the group.

The `Store` interface ensures that the core logic for anomaly grouping is
decoupled from the specific implementation of data persistence.

#### `sqlanomalygroupstore/sqlanomalygroupstore.go`: SQL-backed Anomaly Group Store

This file provides a concrete implementation of the `Store` interface using a
SQL database (specifically designed with CockroachDB and Spanner in mind).

**Implementation Details:**

- **Schema:** The SQL schema for anomaly groups is defined in
  `sqlanomalygroupstore/schema/schema.go`. It includes fields for the group
  ID, creation time, list of anomaly IDs, metadata (stored as JSONB), common
  commit range, action type, and associated IDs for bisections, issues, and
  culprits.
- **Database Operations:**
  - `Create`: Inserts a new row into the `AnomalyGroups` table. It takes
    parameters like subscription details, benchmark, commit range, and
    action, and stores them. The group metadata (subscription name,
    revision, domain, benchmark) is marshaled into a JSON string before
    insertion.
  - `LoadById`: Selects an anomaly group from the database based on its ID.
    It retrieves core attributes of the group.
  - `UpdateBisectID`, `UpdateReportedIssueID`, `AddAnomalyID`,
    `AddCulpritIDs`: These methods execute SQL UPDATE statements to modify
    specific fields of an existing anomaly group record. They handle array
    appends for lists like `anomaly_ids` and `culprit_ids`, with specific
    syntax considerations for different SQL databases (e.g., Spanner's
    `COALESCE` for array concatenation).
  - `FindExistingGroup`: Constructs a SQL SELECT query with WHERE clauses to
    match the provided criteria (subscription, revision, domain, benchmark,
    commit range overlap, and action). This allows finding groups that a new
    anomaly might belong to.

**Design Choices:**

- **UUIDs for IDs:** Using UUIDs for group IDs, anomaly IDs, and culprit IDs
  ensures global uniqueness.
- **JSONB for Metadata:** Storing `group_meta_data` as JSONB provides
  flexibility in the metadata stored without requiring schema changes for
  minor additions.
- **Array Columns:** Storing `anomaly_ids` and `culprit_ids` as array types in
  the database is a natural way to represent lists of associated entities.
- **Database Type Abstraction:** While targeting SQL, there are minor
  conditional logic snippets (e.g., for array appending in Spanner vs.
  CockroachDB) to handle database-specific syntax, indicated by `dbType`
  checks.

#### `service/service.go`: gRPC Service Implementation

This file implements the `AnomalyGroupServiceServer` interface defined by the
protobuf definitions in `proto/v1/anomalygroup_service.proto`. It acts as the
entry point for external systems to interact with the anomaly grouping
functionality.

**Responsibilities:**

- **Exposing Store Operations via gRPC:** The service methods largely delegate
  to the corresponding methods of the `anomalygroup.Store` interface. For
  example, `CreateNewAnomalyGroup` calls `anomalygroupStore.Create`.
- **Handling gRPC Requests and Responses:** It translates incoming gRPC
  requests into calls to the store and formats the store's output into gRPC
  responses.
- **`FindTopAnomalies` Logic:** This method involves more than a simple store
  passthrough.
  1.  It loads the specified anomaly group.
  2.  It retrieves all regressions (anomalies) associated with that group
      using the `regression.Store`.
  3.  It sorts these regressions based on the percentage change in their
      median values (from `median_before` to `median_after`).
  4.  It formats the top N regressions (or all if N is not specified or is too
      large) into the `ag.Anomaly` protobuf message format, extracting
      relevant paramset values (bot, benchmark, story, measurement, stat).
- **`FindIssuesFromCulprits` Logic:**
  1.  Loads the specified anomaly group.
  2.  Retrieves the culprit IDs associated with the group.
  3.  Uses the `culprit.Store` to get the details of these culprits.
  4.  For each culprit, it checks its `GroupIssueMap` to find any issue IDs
      that are specifically associated with the given anomaly group ID. This
      allows correlation between a group (potentially containing multiple
      anomalies that led to a bisection) and the issues filed for the culprits
      found by that bisection.

**Design Choices:**

- **Dependency Injection:** The service takes instances of
  `anomalygroup.Store`, `culprit.Store`, and `regression.Store` as
  dependencies, promoting testability and decoupling.
- **Metric Collection:** It increments a counter (`newGroupCounter`) whenever
  a new group is created, allowing for monitoring of the system's behavior.

#### `proto/v1/anomalygroup_service.proto`: Protocol Buffer Definitions

This file defines the gRPC service `AnomalyGroupService` and the message types
used for requests and responses. This is the contract for how clients interact
with the anomaly grouping system.

**Key Messages:**

- `AnomalyGroup`: Represents a group of anomalies, including its ID, the
  action to take, lists of associated anomaly and culprit IDs, reported issue
  ID, and metadata like subscription and benchmark names.
- `Anomaly`: Represents a single regression, including its start and end
  commit positions, a `paramset` (key-value pairs describing the test),
  improvement direction, and median values before and after the regression.
- `GroupActionType`: An enum defining the possible actions for a group
  (NOACTION, REPORT, BISECT).
- Request/Response Messages: Specific messages for each RPC method (e.g.,
  `CreateNewAnomalyGroupRequest`, `FindExistingGroupsResponse`).

**Purpose:**

- Defines a clear, language-agnostic API for the service.
- Ensures type safety and structured data exchange.

#### `notifier/anomalygroupnotifier.go`: Anomaly Group Notifier

This component implements the `notify.Notifier` interface. It's invoked when a
new regression is detected by the alerting system. Its primary role is to
integrate the regression detection with the anomaly grouping logic.

**Workflow when `RegressionFound` is called:**

1.  Receive details of a newly detected regression (commit information, alert
    configuration, cluster summary, trace data, regression ID).
2.  Extract the `paramset` from the trace data.
3.  Validate the `paramset` to ensure it contains required keys (e.g., master,
    bot, benchmark, test, subtest_1). This is important because the grouping and
    subsequent actions (like bisection) rely on these parameters.
4.  Determine the `testPath` from the `paramset`. This path is used in finding
    or creating anomaly groups.
5.  Call `grouper.ProcessRegressionInGroup` (which eventually calls
    `utils.ProcessRegression`) to handle the grouping logic for this new
    regression.

**Design Choices:**

- **Interface Implementation:** Adheres to the `notify.Notifier` interface,
  allowing it to be plugged into the existing notification pipeline of the
  performance monitoring system.
- **Delegation to `AnomalyGrouper`:** It delegates the core grouping logic to
  an `AnomalyGrouper` instance (typically `utils.AnomalyGrouperImpl`). This
  keeps the notifier focused on the integration aspect.
- **Handling of Summary Traces:** It explicitly ignores regressions found on
  summary-level traces (traces representing an aggregation of multiple
  specific tests), as anomaly grouping is typically more meaningful for
  specific test cases.

#### `utils/anomalygrouputils.go`: Anomaly Grouping Utilities

This file contains the core logic for processing a new regression and
integrating it into an anomaly group.

**`ProcessRegression` Function - Key Steps:**

1.  **Synchronization:** Uses a `sync.Mutex` (`groupingMutex`). This is a
    critical point: it aims to prevent race conditions when multiple regressions
    are processed concurrently, especially around creating new groups. _However,
    the comment notes that with multiple containers, this mutex might not be
    sufficient and needs review._
2.  **Client Initialization:** Creates an `AnomalyGroupServiceClient` to
    communicate with the gRPC service.
3.  **Find Existing Group:** Calls the `FindExistingGroups` gRPC method to see
    if the new anomaly fits into any current groups based on subscription,
    revision, action type, commit range overlap, and test path.
4.  **Group Creation or Update:** - **If no existing group is found:** - Calls `CreateNewAnomalyGroup` to create a new group. - Calls `UpdateAnomalyGroup` to add the current `anomalyID` to this
    newly created group. - **Triggers a Temporal Workflow:** Initiates a
    `MaybeTriggerBisection` workflow. This workflow is responsible for
    deciding whether to start a bisection or file a bug based on the
    group's action type and other conditions. `Regression Detected -->
FindExistingGroups | +-- No Group Found --> CreateNewAnomalyGroup
--> UpdateAnomalyGroup (add anomaly) --> Start Temporal Workflow
(MaybeTriggerBisection)` - **If existing group(s) are found:** - For each matching group: - Calls `UpdateAnomalyGroup` to add the current `anomalyID` to that
    group. - Calls `FindIssuesToUpdate` to determine if any existing bug reports
    (either the group's own `ReportedIssueId` or issues linked via
    culprits) should be updated with information about this new anomaly. - If issues are found, it uses the `issuetracker` to add a comment to
    each relevant issue. `Regression Detected --> FindExistingGroups |
+-- Group(s) Found --> For each group: | +-- UpdateAnomalyGroup (add
anomaly) +-- FindIssuesToUpdate --> If issues exist --> Add Comment
to Issue(s)`
5.  **Return Group ID(s):** Returns a comma-separated string of group IDs the
    anomaly was associated with.

**`FindIssuesToUpdate` Function:**

This helper determines which existing issue tracker IDs should be updated with
information about a new anomaly being added to a group.

- If the `group_action` is `REPORT` and `reported_issue_id` is set on the
  group, that issue ID is returned.
- If the `group_action` is `BISECT`, it calls the `FindIssuesFromCulprits`
  gRPC method. This method looks up culprits associated with the group and
  then checks if those culprits have specific issues filed for them in the
  context of _this particular group_. This is important because a single
  culprit (commit) might be associated with multiple anomaly groups, and each
  might have its own context or bug report.

**Design Choices:**

- **Centralized Grouping Logic:** This package encapsulates the
  decision-making process of whether to create a new group or add to an
  existing one.
- **Temporal Workflow Integration:** Offloads the decision and execution of
  bisection or bug filing to a Temporal workflow. This makes the process
  asynchronous and more resilient.
- **Issue Tracker Interaction:** Directly interacts with the issue tracker to
  update existing bugs, keeping them relevant as new, related anomalies are
  found.

### Mocking Strategy

The module extensively uses mocks for testing:

- **`mocks/Store.go`:** A mock implementation of the `anomalygroup.Store`
  interface, generated by `testify/mock`. Used in `service/service_test.go`.
- **`proto/v1/mocks/AnomalyGroupServiceServer.go`:** A mock for the gRPC
  server interface `AnomalyGroupServiceServer`, generated by `testify/mock`
  (with manual adjustments noted in the file). Used by clients or other
  services that might call this gRPC service.
- **`utils/mocks/AnomalyGrouper.go`:** A mock for the `AnomalyGrouper`
  interface, used in `notifier/anomalygroupnotifier_test.go`.

This approach allows for unit testing components in isolation by providing
controlled behavior for their dependencies.

### Overall Workflow Example (Simplified)

1.  **Anomaly Detection:** Perf system detects a new regression (anomaly).
2.  **Notification:** `AnomalyGroupNotifier.RegressionFound` is called.
3.  **Preprocessing:** The notifier extracts `paramset`, validates it, and
    derives `testPath`.
4.  **Grouping Logic (`utils.ProcessRegression`):**
    - The system queries `AnomalyGroupService.FindExistingGroups` using the
      anomaly's properties (subscription, commit range, test path, action
      type).
    - **Scenario A: No existing group:**
      - `AnomalyGroupService.CreateNewAnomalyGroup` is called.
      - The new anomaly ID is added to this group via
        `AnomalyGroupService.UpdateAnomalyGroup`.
      - A Temporal workflow (`MaybeTriggerBisection`) is started for this
        new group.
    - **Scenario B: Existing group(s) found:**
      - The new anomaly ID is added to each matching group via
        `AnomalyGroupService.UpdateAnomalyGroup`.
      - `utils.FindIssuesToUpdate` is called for each group.
      - If the group's action is `REPORT` and it has a `ReportedIssueId`,
        that issue is updated.
      - If the group's action is `BISECT`,
        `AnomalyGroupService.FindIssuesFromCulprits` is called. If it
        returns issue IDs associated with this group's culprits, those
        issues are updated.
5.  **Temporal Workflow (`MaybeTriggerBisection` - not detailed here but
    implied):**
    - Based on the group's `GroupActionType`:
      - If `BISECT`: It might check conditions (e.g., number of anomalies in
        the group) and then trigger a bisection job (e.g., Pinpoint) using
        `AnomalyGroupService.FindTopAnomalies` to pick the most significant
        anomaly. The bisection ID is then saved to the group.
      - If `REPORT`: It might check conditions and then file a bug using
        `AnomalyGroupService.FindTopAnomalies` to gather details. The issue
        ID is saved to the group.

This system aims to automate and streamline the handling of performance
regressions by intelligently grouping them and initiating appropriate follow-up
actions.

# Module: /go/backend

The `/go/backend` module implements a gRPC-based backend service for Perf. This
service is designed to host API endpoints that are not directly user-facing,
promoting a separation of concerns and enabling better scalability and
maintainability.

**Core Purpose and Design Philosophy:**

The primary motivation for this backend service is to create a stable, internal
API layer. This decouples user-facing components (like the frontend) from the
direct implementation details of various backend tasks. For instance, if Perf
needs to trigger a Pinpoint job, the frontend doesn't interact with Pinpoint or
a workflow engine like Temporal directly. Instead, it makes a gRPC call to an
endpoint on this backend service. The backend service then handles the
interaction with the underlying system (e.g., Temporal).

This design offers several advantages:

- **Interface Stability:** If the underlying implementation for a task changes
  (e.g., replacing Temporal with another workflow orchestrator), the gRPC
  contract exposed by the backend service can remain the same. This minimizes
  changes required in calling services.
- **Load Offloading:** Computationally intensive operations that might
  otherwise burden the frontend can be delegated to this backend service.
  Examples include dry-running regression detection.
- **Centralized Internal Logic:** It provides a dedicated place for internal,
  non-UI-facing business logic.

**Key Components and Responsibilities:**

- **`backend.go`**: This is the heart of the backend service.

  - **`Backend` struct:** Encapsulates the state and configuration of the
    backend application, including gRPC server settings, ports, and loaded
    configuration.
  - **`BackendService` interface:** Defines a contract for any service that
    wishes to be hosted by this backend. Each such service must provide its
    gRPC service descriptor, registration logic, and an authorization
    policy. This interface-based approach allows for modular addition of new
    functionalities.
  - The `GetAuthorizationPolicy()` method returns a
    `shared.AuthorizationPolicy` which specifies whether unauthenticated
    access is allowed and which user roles are authorized to call the
    service or specific methods within it.
  - `RegisterGrpc()` is responsible for registering the specific gRPC
    service implementation with the main gRPC server.
  - `GetServiceDescriptor()` provides metadata about the gRPC service.
  - **`initialize()` function:** This is a crucial setup function. It:
  - Initializes common application components (like Prometheus metrics).
  - Loads and validates the application configuration (from a JSON file,
    e.g., `demo.json`).
  - Instantiates various data stores (for anomaly groups, culprits,
    subscriptions, regressions) by using builder functions that typically
    read connection details from the loaded configuration. This allows for
    flexibility in choosing data store implementations (e.g., Spanner,
    CockroachDB).
  - Sets up a culprit notifier, which is responsible for sending
    notifications about identified culprits.
  - Initializes a Temporal client if the `NotifyConfig.Notifications` is set
    to `AnomalyGrouper`, as this indicates that anomaly grouping workflows
    managed by Temporal are in use.
  - Dynamically configures and registers all `BackendService`
    implementations. This involves setting up authorization rules based on
    the policy defined by each service and then registering their gRPC
    handlers.
  - Starts listening for gRPC connections on the configured port.
  - **`configureServices()` and `registerServices()`:** These helper
    functions iterate over the list of `BackendService` implementations to
    set up authorization and register them with the main gRPC server.
  - **`configureAuthorizationForService()`:** This function applies the
    authorization policies defined by each individual service to the gRPC
    server's authorization policy. It uses `grpcsp.ServerPolicy` to define
    which roles can access the service or specific methods.
  - **`New()` constructor:** Creates and initializes a new `Backend`
    instance. It takes various store implementations and a notifier as
    arguments, allowing for dependency injection, particularly useful for
    testing. If these are `nil`, they are typically created within
    `initialize()` based on the configuration.
  - **`ServeGRPC()` and `Serve()`:** These methods start the gRPC server and
    block until it's shut down.
  - **`Cleanup()`:** Handles graceful shutdown of the gRPC server.

- **`pinpoint.go`**: This file defines a wrapper for the actual Pinpoint
  service implementation (which resides in `pinpoint/go/service`).

  - **`pinpointService` struct:** Implements the `BackendService` interface.
  - **`NewPinpointService()`:** Creates a new instance, taking a Temporal
    provider and a rate limiter as arguments. This indicates that Pinpoint
    operations might be rate-limited and potentially involve Temporal
    workflows.
  - It defines an authorization policy requiring users to have at least
    `roles.Editor` to access Pinpoint functionalities. This is a good
    example of how specific services define their own access control rules.

- **`shared/authorization.go`**:

  - **`AuthorizationPolicy` struct:** A simple struct used by
    `BackendService` implementations to declare their authorization
    requirements. This includes whether unauthenticated access is permitted,
    a list of roles authorized for the entire service, and a map for
    method-specific role authorizations. This promotes a consistent way for
    services to define their security posture.

- **`client/backendclientutil.go`**: This utility file provides helper
  functions for creating gRPC clients to connect to the backend service itself
  (or specific services hosted by it).

  - **`getGrpcConnection()`:** Abstracts the logic for establishing a gRPC
    connection. It handles both insecure (typically for local
    development/testing) and secure connections. For secure connections, it
    uses TLS (with `InsecureSkipVerify: true` as it's intended for internal
    GKE cluster communication) and OAuth2 for authentication, obtaining
    tokens for the service account running the client process.
  - **`NewPinpointClient()`, `NewAnomalyGroupServiceClient()`,
    `NewCulpritServiceClient()`:** These are factory functions that simplify
    the creation of typed gRPC clients for the specific services hosted on
    the backend. They first check if the backend service is
    configured/enabled before attempting to create a connection. This
    pattern makes it easy for other internal services to consume the APIs
    provided by this backend.

- **`backendserver/main.go`**: This is the entry point for the backend server
  executable.

  - It uses the `urfave/cli` library to define a command-line interface.
  - The `run` command initializes and starts the `Backend` service using the
    `backend.New()` constructor and then calls `b.Serve()`.
  - It primarily parses command-line flags (defined in
    `config.BackendFlags`) and passes them to the `backend` package. It
    doesn't instantiate stores or notifiers directly, relying on the
    `backend.New` (and subsequently `initialize`) to create them based on
    the loaded configuration if `nil` is passed.

**Workflow Example: Handling a gRPC Request**

1.  A client (e.g., the Perf frontend or another internal service) uses a
    generated gRPC client stub (potentially created with helpers from
    `client/backendclientutil.go`) to make a call to a specific method on a
    service hosted by the backend (e.g., `Pinpoint.ScheduleJob`).
2.  The gRPC request arrives at the `Backend` server's listener (`b.lisGRPC`).
3.  The `grpc.Server` routes the request to the appropriate service
    implementation (e.g., `pinpointService`).
4.  **Authentication/Authorization (via `grpcsp.ServerPolicy`):** Before the
    service method is executed, the `UnaryInterceptor` configured in
    `backend.go` (which uses `b.serverAuthPolicy`) intercepts the call.
    `Incoming gRPC Request --> UnaryInterceptor (grpcsp) | V Check Auth Policy
for Service/Method (defined by pinpointService.GetAuthorizationPolicy()) | V
Allow/Deny ----> Yes: Proceed to service method No: Return error`
5.  If authorized, the corresponding method on the `pinpointService` (which
    delegates to the actual `pinpoint_service.PinpointServer` implementation) is
    invoked.
6.  The service method performs its logic (e.g., interacting with Temporal to
    schedule a Pinpoint job, querying data stores).
7.  A response is sent back to the client.

**Configuration and Initialization:**

The system relies heavily on a configuration file (specified by
`flags.ConfigFilename`, often `demo.json` for local development as seen in
`backend_test.go` and `testdata/demo.json`). This file dictates:

- Data store connection strings and types (`data_store_config`).
- Notification settings (`notify_config`).
- The backend service's own host URL (`backend_host_url`), which it might use
  if it needs to call itself or if other components need to discover it.
- Temporal configuration (`temporal_config` - though not explicitly in
  `demo.json`, it's checked in `backend.go`).

The `initialize` function in `backend.go` is responsible for parsing this
configuration and setting up all necessary dependencies like database
connections, the Temporal client, and the culprit notifier. The use of builder
functions (e.g., `builders.NewAnomalyGroupStoreFromConfig`) allows the system to
be flexible with regard to the actual implementations of these components, as
long as they conform to the required interfaces.

This backend module serves as a crucial intermediary, enhancing the robustness
and maintainability of the Perf system by providing a well-defined internal API
layer.

# Module: /go/bug

The `go/bug` module is designed to facilitate the creation of URLs for reporting
bugs or regressions identified within the Skia performance monitoring system.
Its primary purpose is to dynamically generate these URLs based on a predefined
template and specific details about the identified issue. This approach allows
for flexible integration with various bug tracking systems, as the URL structure
can be configured externally.

**Core Functionality and Design:**

The module centers around the concept of URI templates. Instead of hardcoding
URL formats for specific bug trackers, it uses a template string that contains
placeholders for relevant information. This makes the system adaptable to
changes in bug tracker URL schemes or the adoption of new trackers without
requiring code modifications.

The key function, `Expand`, takes a URI template and populates it with details
about the regression. These details include:

1.  **`clusterLink`**: A URL pointing to the specific performance data cluster
    that exhibits the regression. This provides direct context for anyone
    investigating the bug.
2.  **`c provider.Commit`**: Information about the specific commit suspected of
    causing the regression. This includes the commit's URL, allowing for easy
    navigation to the code change. The use of the `provider.Commit` type from
    `perf/go/git/provider` indicates an integration with a system that can
    furnish commit details.
3.  **`message`**: A user-provided message describing the regression. This
    allows the reporter to add specific observations or context.

The `Expand` function utilizes the `gopkg.in/olivere/elastic.v5/uritemplates`
library to perform the actual substitution of placeholders in the template
string with the provided values. This library handles URL encoding of the
substituted values, ensuring the generated URL is valid.

**Key Components/Files:**

- **`bug.go`**: This file contains the core logic for expanding URI templates.

  - `Expand(uriTemplate string, clusterLink string, c provider.Commit,
message string) string`: This is the primary function responsible for
    generating the bug reporting URL. It takes the template and the
    contextual information as input and returns the fully formed URL. If the
    template expansion fails (e.g., due to a malformed template), it logs an
    error using `go.skia.org/infra/go/sklog` and returns an empty string or
    a partially formed URL depending on the nature of the error.
  - `ExampleExpand(uriTemplate string) string`: This function serves as a
    utility or example for demonstrating how to use the `Expand` function.
    It calls `Expand` with pre-defined example data for the cluster link,
    commit, and message. This can be useful for testing the template
    expansion logic or for providing a quick way to see how a given template
    would be populated.

- **`bug_test.go`**: This file contains unit tests for the functionality in
  `bug.go`.

  - `TestExpand(t *testing.T)`: This test function verifies that the
    `Expand` function correctly substitutes the provided values into the URI
    template and produces the expected URL. It uses the
    `github.com/stretchr/testify/assert` library for assertions, ensuring
    that the generated URL matches the anticipated output, including proper
    URL encoding.

**Workflow:**

A typical workflow involving this module would be:

1.  **Configuration**: An external system (e.g., the Perf frontend) is
    configured with a URI template for the desired bug tracking system. This
    template will contain placeholders like `{cluster_url}`, `{commit_url}`, and
    `{message}`. Example Template:
    `https://bugtracker.example.com/new?summary=Regression%20Found&description=Regression%20details:%0ACluster:%20{cluster_url}%0ACommit:%20{commit_url}%0AMessage:%20{message}`

2.  **Regression Identification**: A user or an automated system identifies a
    performance regression.

3.  **Information Gathering**: The system gathers the necessary information:

    - The URL to the Perf cluster graph showing the regression.
    - Details of the commit suspected to have introduced the regression.
    - An optional message from the user.

4.  **URL Generation**: The `Expand` function in `go/bug` is called with the
    configured URI template and the gathered information.

    ```
    template := "https://bugtracker.example.com/new?summary=Regression%20Found&description=Cluster:%20{cluster_url}%0ACommit:%20{commit_url}%0AMessage:%20{message}"
    clusterURL := "https://perf.skia.org/t/?some_params"
    commitData := provider.Commit{URL: "https://skia.googlesource.com/skia/+show/abcdef123"}
    userMessage := "Significant drop in frame rate on TestXYZ."

    bugReportURL := bug.Expand(template, clusterURL, commitData, userMessage)
    ```

5.  **Redirection/Display**: The generated `bugReportURL` is then presented to
    the user, who can click it to navigate to the bug tracker with the
    pre-filled information.

This design decouples the bug reporting logic from the specifics of any single
bug tracking system, promoting flexibility and maintainability. The use of a
standard URI template expansion library ensures robustness in URL generation.

# Module: /go/builders

The `builders` module is responsible for constructing various core components of
the Perf system based on instance configuration. This centralized approach to
object creation prevents cyclical dependencies that could arise if configuration
objects were directly responsible for building the components they configure.
The module acts as a factory, taking an `InstanceConfig` and returning fully
initialized and operational objects like data stores, file sources, and caches.

The primary design goal is to decouple the configuration of Perf components from
their instantiation. This allows for cleaner dependencies and makes it easier to
manage the lifecycle of different parts of the system. For example, a
`TraceStore` needs a database connection, but the `InstanceConfig` that defines
the database connection string shouldn't also be responsible for creating the
`TraceStore` itself. The `builders` module bridges this gap.

Key components and their instantiation logic:

- **`builders.go`**: This is the central file containing all the builder
  functions.
  - **Database Pool (`NewDBPoolFromConfig`)**: This function is crucial as
    many other components rely on a database connection. It establishes a
    connection pool to the configured database (e.g., CockroachDB, Spanner).
  - **Why**: A connection pool is used to manage database connections
    efficiently, reusing existing connections to reduce the overhead of
    establishing new ones for each request.
  - **How**: It parses the connection string from the `InstanceConfig`,
    configures pool parameters like maximum and minimum connections, and
    sets up a logging adapter (`pgxLogAdaptor`) to integrate database logs
    with the application's logging system.
  - **Singleton**: A key design choice here is the `singletonPool`. This
    ensures that only one database connection pool is created per
    application instance, preventing resource exhaustion and ensuring
    consistent database interaction. A mutex (`singletonPoolMutex`) protects
    the creation of this singleton.
  - **Schema Check**: Optionally, it can verify that the connected database
    schema matches the expected schema defined for the application. This is
    important for ensuring data integrity and compatibility.
  - **Timeout Wrapper**: The raw database pool is wrapped with a
    `timeout.New` wrapper. This enforces that all database operations are
    performed within a context that has a timeout, preventing indefinite
    blocking. `InstanceConfig --> NewDBPoolFromConfig -->
pgxpool.ParseConfig | +-> pgxpool.ConnectConfig --> rawPool | +->
timeout.New(rawPool) --> singletonPool (if schema check passes)`
  - **PerfGit (`NewPerfGitFromConfig`)**: Constructs a `perfgit.Git` object,
    which provides an interface to Git repository data.
  - **Why**: Perf needs to associate performance data with specific code
    revisions.
  - **How**: It first obtains a database pool using `getDBPool` (which in
    turn uses `NewDBPoolFromConfig`) and then instantiates `perfgit.New`
    with this pool and the instance configuration.
  - **TraceStore (`NewTraceStoreFromConfig`)**: Creates a
    `tracestore.TraceStore` for managing performance trace data.
  - **Why**: This is the core component for storing and retrieving
    time-series performance metrics.
  - **How**: It gets a database pool and a `TraceParamStore` (for managing
    trace parameter sets) and then instantiates the appropriate
    `sqltracestore`.
  - **MetadataStore (`NewMetadataStoreFromConfig`)**: Creates a
    `tracestore.MetadataStore` for managing metadata associated with traces.
  - **How**: Similar to `TraceStore`, it obtains a database pool and then
    creates an `sqltracestore.NewSQLMetadataStore`.
  - **AlertStore, RegressionStore, ShortcutStore, GraphsShortcutStore,
    AnomalyGroupStore, CulpritStore, SubscriptionStore, FavoriteStore,
    UserIssueStore**: These functions follow a similar pattern: they obtain
    a database pool via `getDBPool` and then instantiate their respective
    SQL-backed store implementations (e.g., `sqlalertstore`,
    `sqlregression2store`).
  - **Why**: These stores manage various aspects of Perf's functionality,
    such as alerting configurations, regression tracking, saved shortcuts,
    etc. Centralizing their creation based on the common database
    configuration simplifies the system.
  - **RegressionStore Variation**: `NewRegressionStoreFromConfig` has a
    conditional logic based on `instanceConfig.UseRegression2` to
    instantiate either `sqlregression2store` or `sqlregressionstore`. This
    allows for migrating to a new regression store implementation controlled
    by configuration.
  - **GraphsShortcutStore Caching**: `NewGraphsShortcutStoreFromConfig` can
    return a cached version
    (`graphsshortcutstore.NewCacheGraphsShortcutStore`) if `localToProd` is
    true, indicating a local development or testing environment where a
    simpler in-memory cache might be preferred over a database-backed store.
  - **Source (`NewSourceFromConfig`)**: Creates a `file.Source` which
    defines where Perf ingests data from (e.g., Google Cloud Storage, local
    directories).
  - **Why**: Perf needs to be flexible in terms of where it reads input data
    files.
  - **How**: It uses a `switch` statement based on
    `instanceConfig.IngestionConfig.SourceConfig.SourceType` to instantiate
    either a `gcssource` or a `dirsource`.
  - **IngestedFS (`NewIngestedFSFromConfig`)**: Creates a `fs.FS` (file
    system interface) that provides access to already ingested files.
  - **Why**: To provide a consistent way to access files regardless of their
    underlying storage (GCS or local).
  - **How**: Similar to `NewSourceFromConfig`, it switches on the source
    type to return a GCS or local file system implementation.
  - **Cache (`GetCacheFromConfig`)**: Returns a `cache.Cache` instance
    (either Redis-backed or local in-memory).
  - **Why**: Caching is used to improve the performance of frequently
    accessed data or computationally intensive queries.
  - **How**: It checks `instanceConfig.QueryConfig.CacheConfig.Type` to
    determine whether to create a `redisCache` (connecting to a Google Cloud
    Redis instance) or a `localCache`.

The `getDBPool` helper function is used internally by many builder functions. It
acts as a dispatcher based on `instanceConfig.DataStoreConfig.DataStoreType`,
calling `NewDBPoolFromConfig` with appropriate schema checking flags. This
abstracts the direct call to `NewDBPoolFromConfig` and centralizes the logic for
selecting the database type.

The test file (`builders_test.go`) ensures that these builder functions
correctly instantiate objects and handle different configurations, including
invalid ones. A notable aspect of the tests is the management of the
`singletonPool`. Since `NewDBPoolFromConfig` creates a singleton, tests that
require fresh database instances must explicitly clear this singleton
(`singletonPool = nil`) before calling the builder to avoid reusing a connection
from a previous test. This is handled in `newDBConfigForTest`.

# Module: /go/chromeperf

The `chromeperf` module facilitates interaction with the Chrome Perf backend,
which is the system of record for performance data for Chromium. This module
allows Perf to send and receive data from Chrome Perf.

## Key Responsibilities

The primary responsibility of this module is to abstract the communication
details with the Chrome Perf API. It provides a typed Go interface to various
Chrome Perf endpoints, handling request formatting, authentication, and response
parsing.

This interaction is crucial for:

- **Reporting Regressions:** When Perf detects a performance regression, it
  needs to inform Chrome Perf to create an alert and potentially file a bug.
- **Fetching Anomaly Data:** Perf needs to retrieve information about existing
  anomalies and alert groups from Chrome Perf to display them in its UI or use
  them in its analysis. This includes details about the commit range, affected
  tests, and associated bug IDs.
- **Maintaining Test Path Consistency:** Chrome Perf and Perf may have
  slightly different representations of test paths (e.g., due to character
  restrictions). This module, in conjunction with the `sqlreversekeymapstore`
  submodule, helps manage these differences.

## Key Components

### `chromeperfClient.go`

This file defines the generic `ChromePerfClient` interface and its
implementation, `chromePerfClientImpl`. This is the core component responsible
for making HTTP GET and POST requests to the Chrome Perf API.

**Why:** Abstracting the HTTP client allows for easier testing (by mocking the
client) and centralizes the logic for handling authentication (using OAuth2
Google default token source) and constructing target URLs.

**How:**

- It uses `google.DefaultTokenSource` for authentication.
- `generateTargetUrl` constructs the correct API endpoint URL, differentiating
  between the Skia-Bridge proxy
  (`https://skia-bridge-dot-chromeperf.appspot.com`) and direct calls to the
  legacy Chrome Perf endpoint (`https://chromeperf.appspot.com`). The
  Skia-Bridge is generally preferred.
- `SendGetRequest` and `SendPostRequest` handle the actual HTTP communication,
  JSON marshalling/unmarshalling, and basic error handling, including checking
  for accepted HTTP status codes.

Example workflow for a POST request:

```
Caller -> chromePerfClient.SendPostRequest(ctx, "anomalies", "add", requestBody, &responseObj, []int{200})
  |
  |  (Serializes requestBody to JSON)
  v
  |--------------------------------------------------------------------------------------------------------|
  | generateTargetUrl("https://skia-bridge-dot-chromeperf.appspot.com/anomalies/add")                      |
  |--------------------------------------------------------------------------------------------------------|
  |
  v
httpClient.Post(targetUrl, "application/json", jsonBody)
  |
  v
  (HTTP Request to Chrome Perf API)
  |
  v
  (Receives HTTP Response)
  |
  v
  (Checks if response status code is in acceptedStatusCodes)
  |
  v
  (Deserializes response body into responseObj)
  |
  v
Caller (receives populated responseObj or error)
```

### `anomalyApi.go`

This file builds upon `chromeperfClient.go` to provide a specialized client for
interacting with the `/anomalies` endpoint in Chrome Perf. It defines the
`AnomalyApiClient` interface and its implementation `anomalyApiClientImpl`.

**Why:** This client encapsulates the logic specific to anomaly-related
operations, such as formatting requests for reporting regressions or fetching
anomaly details, and parsing the specific JSON structures returned by these
endpoints. It also handles the translation between Perf's trace identifiers and
Chrome Perf's `test_path` format.

**How:**

- **`ReportRegression`**: Constructs a `ReportRegressionRequest` and sends it
  to the `anomalies/add` endpoint. This is how Perf informs Chrome Perf about
  a new regression.
- **`GetAnomalyFromUrlSafeKey`**: Fetches details for a specific anomaly using
  its key from the `anomalies/get` endpoint.
- **`GetAnomalies`**: Retrieves anomalies for a list of tests within a
  specific commit range (`min_revision`, `max_revision`) by calling the
  `anomalies/find` endpoint.
  - It performs a crucial translation step: `traceNameToTestPath` converts
    Perf's comma-separated key-value trace names (e.g.,
    `,benchmark=Blazor,bot=MacM1,...`) into Chrome Perf's slash-separated
    `test_path` (e.g., `ChromiumPerf/MacM1/Blazor/...`).
  - It also handles potential discrepancies in commit numbers if Chrome Perf
    returns commit hashes. It uses `perfGit.CommitNumberFromGitHash` to
    resolve these.
- **`GetAnomaliesTimeBased`**: Similar to `GetAnomalies`, but fetches
  anomalies based on a time range (`start_time`, `end_time`) by calling the
  `anomalies/find_time` endpoint.
- **`GetAnomaliesAroundRevision`**: Fetches anomalies that occurred around a
  specific revision number.
- **`traceNameToTestPath`**: This function is key for interoperability. It
  parses a Perf trace name (which is a string of key-value pairs) and
  constructs the corresponding `test_path` string that Chrome Perf expects. It
  also handles an experimental feature (`EnableSkiaBridgeAggregation`) which
  can modify how test paths are generated, particularly for aggregated
  statistics (e.g., ensuring `testName_avg` is used if the `stat` is `value`).
  - The logic for `statToSuffixMap` and `hasSuffixInTestValue` addresses
    historical inconsistencies where test names in Perf might or might not
    include statistical suffixes (like `_avg`, `_max`). The goal is to
    derive the correct Chrome Perf `test_path`.

Workflow for fetching anomalies:

```
Perf UI/Backend -> anomalyApiClient.GetAnomalies(ctx, ["trace_A,key=val", "trace_B,key=val"], 100, 200)
  |
  v
  (For each traceName)
  traceNameToTestPath("trace_A,key=val") -> "chromeperf/test/path/A"
  |
  v
  chromeperfClient.SendPostRequest(ctx, "anomalies", "find", {Tests: ["path/A", "path/B"], MinRevision: "100", MaxRevision: "200"}, &anomaliesResponse, ...)
  |
  v
  (Parses anomaliesResponse, potentially resolving commit hashes to commit numbers)
  |
  v
Perf UI/Backend (receives AnomalyMap)
```

### `alertGroupApi.go`

This file provides a client for interacting with Chrome Perf's `/alert_group`
API, specifically to get details about alert groups. An alert group in Chrome
Perf typically corresponds to a set of related anomalies (regressions).

**Why:** When Perf displays information about an alert (which might have
originated from Chrome Perf), it needs to fetch details about the associated
alert group, such as the specific anomalies included, the commit range, and
other metadata.

**How:**

- **`GetAlertGroupDetails`**: Takes an alert group key and calls the
  `alert_group/details` endpoint on Chrome Perf.
- The `AlertGroupDetails` struct holds the response, including a map of
  `Anomalies` (where the value is the Chrome Perf `test_path`) and start/end
  commit numbers/hashes.
- **`GetQueryParams` and `GetQueryParamsPerTrace`**: These methods are
  utilities to transform the `AlertGroupDetails` into query parameters that
  can be used to construct URLs for Perf's own explorer page. This allows
  users to easily navigate from a Chrome Perf alert to viewing the
  corresponding data in Perf.
  - `GetQueryParams` aggregates all test path components (masters, bots,
    benchmarks, etc.) from all anomalies in the group into a single set of
    parameters.
  - `GetQueryParamsPerTrace` generates a separate set of query parameters
    for _each_ individual anomaly in the alert group.
  - They parse the slash-separated `test_path` from Chrome Perf back into
    individual components.

Workflow for getting alert group details:

```
Perf Backend (e.g., when processing an incoming alert from Chrome Perf)
  |
  v
  alertGroupApiClient.GetAlertGroupDetails(ctx, "chrome_perf_group_key")
  |
  v
  chromeperfClient.SendGetRequest(ctx, "alert_group", "details", {key: "chrome_perf_group_key"}, &alertGroupResponse)
  |
  v
  (alertGroupResponse is populated)
  |
  v
  alertGroupResponse.GetQueryParams(ctx) -> Perf Explorer URL query params
```

### `store.go` and the `sqlreversekeymapstore` submodule

`store.go` defines the `ReverseKeyMapStore` interface. The
`sqlreversekeymapstore` directory and its `schema` subdirectory provide an
SQL-based implementation of this interface.

**Why:** Test paths in Chrome Perf can contain characters that are considered
"invalid" or are handled differently by Perf's parameter parsing (e.g., Perf's
trace keys are comma-separated key-value pairs, and the values themselves should
ideally not interfere with this). When data is ingested into Perf from Chrome
Perf, or when Perf constructs test paths to query Chrome Perf, these "invalid"
characters in Chrome Perf test path components (like subtest names) might be
replaced (e.g., with underscores).

This creates a problem: if Perf has `test/foo_bar` and Chrome Perf has
`test/foo?bar`, Perf needs a way to know that `foo_bar` corresponds to `foo?bar`
when querying Chrome Perf. The `ReverseKeyMapStore` is designed to store these
mappings.

**How:**

- `sqlreversekeymapstore/schema/schema.go` defines the SQL table schema
  `ReverseKeyMapSchema` with columns:
  - `ModifiedValue`: The value as it appears in Perf (e.g., `foo_bar`).
  - `ParamKey`: The parameter key this value belongs to (e.g., `subtest_1`).
  - `OriginalValue`: The original value as it was in Chrome Perf (e.g.,
    `foo?bar`).
  - The primary key is a combination of `ModifiedValue` and `ParamKey`.
- `sqlreversekeymapstore/sqlreversekeymapstore.go` implements the
  `ReverseKeyMapStore` interface using a SQL database (configurable for
  CockroachDB or Spanner via different SQL statements).
  - `Create`: Inserts a new mapping. If a mapping for the `ModifiedValue`
    and `ParamKey` already exists (conflict), it does nothing. This is
    important because the mapping should be stable.
  - `Get`: Retrieves the `OriginalValue` given a `ModifiedValue` and
    `ParamKey`.

This store is likely used during the process of converting between Perf trace
parameters and Chrome Perf test paths, especially when generating requests _to_
Chrome Perf. If a parameter value in Perf might have been modified from its
Chrome Perf original, this store can be queried to get the original value needed
for the Chrome Perf API call. The exact point of integration for creating these
mappings (i.e., when are `Create` calls made) is not explicitly detailed within
this module but would typically happen when Perf first encounters/ingests a test
path from Chrome Perf that requires modification.

For example, if `anomalyApi.go` needs to construct a `test_path` to query Chrome
Perf based on parameters from Perf:

1.  Perf has params: `test=my_test, subtest_1=value_with_question_mark`
2.  When constructing the `test_path` segment for `subtest_1`: - Call `reverseKeyMapStore.Get(ctx, "value_with_question_mark",
"subtest_1")`. - If it returns an original value like `"value?with?question?mark"`, use
    that for the Chrome Perf API call. - Otherwise, use `"value_with_question_mark"`.

The `store.go` file simply defines the interface, allowing for different backend
implementations of this mapping store if needed, though `sqlreversekeymapstore`
is the provided concrete implementation.

# Module: /go/clustering2

## Overview

The `clustering2` module is responsible for grouping similar performance traces
together using k-means clustering. This helps in identifying patterns and
regressions in performance data by analyzing the collective behavior of traces
rather than individual ones. The core idea is to represent each trace as a point
in a multi-dimensional space and then find `k` clusters of these points.

## Design and Implementation

### Why K-Means?

K-means is a well-understood and relatively efficient clustering algorithm
suitable for the scale of performance data encountered. It partitions data into
`k` distinct, non-overlapping clusters. Each data point belongs to the cluster
with the nearest mean (cluster centroid). This approach allows for the
summarization of large numbers of traces into a smaller set of representative
"shapes" or behaviors.

### Key Components and Files

#### `clustering.go`

This file contains the primary logic for performing k-means clustering on
performance traces.

- **`ClusterSummary`**: This struct represents a single cluster found by the
  k-means algorithm.

  - `Centroid`: The average shape of all traces in this cluster. This is the
    core representation of the cluster's behavior.
  - `Keys`: A list of identifiers for the traces belonging to this cluster.
    These are sorted by their distance to the `Centroid`, allowing users to
    quickly see the most representative traces. This is not serialized to
    JSON to keep the payload manageable, as it can be very large.
  - `Shortcut`: An identifier for a pre-computed set of `Keys`, used for
    efficient retrieval and display in UIs.
  - `ParamSummaries`: A breakdown of the parameter key-value pairs present
    in the cluster and their prevalence (see `valuepercent.go`). This helps
    in understanding what distinguishes this cluster (e.g., "all traces in
    this cluster are for `arch=x86`").
  - `StepFit`: Contains information about how well the `Centroid` fits a
    step function. This is crucial for identifying regressions or
    improvements that manifest as sudden shifts in performance.
  - `StepPoint`: The specific data point (commit/timestamp) where the step
    (if any) in the `Centroid` is detected.
  - `Num`: The total number of traces in this cluster.
  - `Timestamp`: Records when the cluster analysis was performed.
  - `NotificationID`: Stores the ID of any alert or notification sent
    regarding a significant step change detected in this cluster.

- **`ClusterSummaries`**: A container for all the `ClusterSummary` objects
  produced by a single clustering run, along with metadata like the `K` value
  used and the `StdDevThreshold`.

- **`CalculateClusterSummaries` function**: This is the main entry point for
  the clustering process.

  - **Trace Conversion**: It takes a `dataframe.DataFrame` (which holds
    traces and their metadata) and converts each trace into a
    `kmeans.Clusterable` object. The `ctrace2.NewFullTrace` function is used
    here, which likely involves some form of normalization or feature
    extraction to make traces comparable. The `stddevThreshold` parameter is
    used during this conversion, potentially to filter out noisy or flat
    traces.
  - **Initial Centroid Selection (`chooseK`)**: K-means requires an initial
    set of `k` centroids. This function randomly selects `k` traces from the
    input data to serve as the initial centroids. Random selection is a
    common and simple initialization strategy.
  - **K-Means Iteration**:
  - The `kmeans.Do` function performs one iteration of the k-means
    algorithm:
    1.  Assign each observation (trace) to the nearest centroid.
    2.  Recalculate the centroids based on the mean of the observations
        assigned to them. The `ctrace2.CalculateCentroid` function is likely
        responsible for computing the mean of a set of traces.
  - This process is repeated for a maximum of `MAX_KMEANS_ITERATIONS` or
    until the change in `totalError` (sum of squared distances from each
    point to its centroid) between iterations falls below `KMEAN_EPSILON`.
    This convergence criterion prevents unnecessary computations once the
    clusters stabilize.
  - A `Progress` callback can be provided to monitor the clustering process,
    reporting the `totalError` at each iteration.
  - **Summary Generation (`getClusterSummaries`)**: After the k-means
    algorithm converges, this function takes the final centroids and the
    original observations to generate `ClusterSummary` objects for each
    cluster.
  - For each cluster, it identifies the member traces.
  - It calculates `ParamSummaries` (see `valuepercent.go`) to describe the
    common characteristics of traces in that cluster.
  - It performs step detection (`stepfit.GetStepFitAtMid`) on the cluster's
    centroid to identify significant performance shifts. The `interesting`
    parameter likely defines a threshold for what constitutes a noteworthy
    step change, and `stepDetection` specifies the algorithm or method used
    for step detection.
  - It sorts the traces within each cluster by their distance to the
    centroid, ensuring `ClusterSummary.Keys` lists the most representative
    traces first. A limited number of sample keys
    (`config.MaxSampleTracesPerCluster`) are stored.
  - Finally, the resulting `ClusterSummary` objects are sorted, likely by
    the magnitude or significance of the detected step
    (`StepFit.Regression`), to highlight the most impactful changes first.

- **Constants**:

  - `K`: The default number of clusters to find. 50 is chosen as a balance
    between granularity and computational cost.
  - `MAX_KMEANS_ITERATIONS`: A safeguard against non-converging k-means
    runs.
  - `KMEAN_EPSILON`: A threshold to determine convergence, balancing
    precision with computation time.

#### `valuepercent.go`

This file defines how to summarize and present the parameter distributions
within a cluster.

- **`ValuePercent` struct**: Represents a specific parameter key-value pair
  (e.g., "config=8888") and the percentage of traces in a cluster that have
  this pair. This provides a quantitative measure of how characteristic a
  parameter is for a given cluster.

- **`SortValuePercentSlice` function**: This is crucial for making the
  `ParamSummaries` in `ClusterSummary` human-readable and informative. The
  goal is to:

  1.  Group parameter values by their key (e.g., all "config=..." values
      together).
  2.  Within each key group, sort by the percentage (highest first).
  3.  Sort the key groups themselves by the highest percentage of their top
      value. If percentages are equal, an alphabetical sort of the value is
      used as a tie-breaker.

  This complex sorting logic ensures that the most dominant and distinguishing
  parameters for a cluster are presented prominently. For example:

  ```
  config=8888 90%
  config=565  10%
  arch=x86    80%
  arch=arm    20%
  ```

  Here, "config" is listed before "arch" because its top value ("config=8888")
  has a higher percentage (90%) than the top value for "arch" ("arch=x86" at
  80%).

### Workflow: Calculating Cluster Summaries

```
Input: DataFrame (traces, headers), K, StdDevThreshold, ProgressCallback, InterestingThreshold, StepDetectionMethod

1.  [clustering.go: CalculateClusterSummaries]
    a.  Initialize empty list of observations.
    b.  For each trace in DataFrame.TraceSet:
        i.  Create ClusterableTrace (ctrace2.NewFullTrace) using trace data and StdDevThreshold.
        ii. Add to observations list.
    c.  If no observations, return error.
    d.  [clustering.go: chooseK]
        i.  Randomly select K observations to be initial centroids.
    e.  Initialize lastTotalError = 0.0
    f.  Loop MAX_KMEANS_ITERATIONS times OR until convergence:
        i.   [kmeans.Do] -> new_centroids
             1.  Assign each observation to its closest centroid (from previous iteration or initial).
             2.  Recalculate centroids (ctrace2.CalculateCentroid) based on assigned observations.
        ii.  [kmeans.TotalError] -> currentTotalError
        iii. If ProgressCallback provided, call it with currentTotalError.
        iv.  If |currentTotalError - lastTotalError| < KMEAN_EPSILON, break loop.
        v.   lastTotalError = currentTotalError
    g.  [clustering.go: getClusterSummaries] -> clusterSummaries
        i.   [kmeans.GetClusters] -> allClusters (list of observations per centroid)
        ii.  For each cluster in allClusters and its corresponding centroid:
             1.  Create new ClusterSummary.
             2.  [clustering.go: getParamSummaries] (using cluster members) -> ParamSummaries
                 a.  [clustering.go: GetParamSummariesForKeys]
                     i.   Count occurrences of each param=value in cluster keys.
                     ii.  Convert counts to ValuePercent structs.
                     iii. [valuepercent.go: SortValuePercentSlice] -> sorted ParamSummaries.
             3.  [stepfit.GetStepFitAtMid] (on centroid values, StdDevThreshold, InterestingThreshold, StepDetectionMethod) -> StepFit, StepPoint.
             4.  Set ClusterSummary.Num = number of members in cluster.
             5.  Sort cluster members by distance to centroid.
             6.  Populate ClusterSummary.Keys with top N sorted member keys.
             7.  Populate ClusterSummary.Centroid with centroid values.
        iii. Sort all ClusterSummary objects (e.g., by StepFit.Regression).
    h.  Populate ClusterSummaries struct with results, K, and StdDevThreshold.
    i.  Return ClusterSummaries.

Output: ClusterSummaries object or error.
```

This process effectively transforms raw trace data into a structured summary
that highlights significant patterns and changes, facilitating performance
analysis and regression detection.

# Module: /go/config

The `/go/config` module defines the configuration structure for Perf instances
and provides utilities for loading, validating, and managing these
configurations. It plays a crucial role in customizing the behavior of a Perf
instance, from data ingestion and storage to alert notifications and UI
presentation.

**Core Responsibilities and Design:**

The primary responsibility of this module is to define and manage the
`InstanceConfig` struct. This struct is a comprehensive container for all
settings that govern a Perf instance. The design emphasizes:

1.  **Centralized Configuration:** By consolidating all instance-specific
    settings into a single `InstanceConfig` struct (`config.go`), the module
    provides a single source of truth. This simplifies understanding the state
    of an instance and reduces the chances of configuration drift.
2.  **Typed Configuration:** Using Go structs with explicit types ensures that
    configuration values are of the expected format, catching many potential
    errors at compile-time or during validation. This is preferable to using
    untyped maps or generic configuration formats.
3.  **JSON Serialization/Deserialization:** Configuration files are expected to
    be in JSON format. The module uses standard Go `encoding/json` for this,
    making it easy to create, read, and modify configurations.
4.  **Schema Validation:** To ensure the integrity and correctness of
    configuration files, the module employs JSON Schema validation
    (`/go/config/validate/validate.go`,
    `/go/config/validate/instanceConfigSchema.json`).
    - A JSON schema (`instanceConfigSchema.json`) formally defines the
      structure and types of the `InstanceConfig`. This schema is
      automatically generated from the Go struct definition using the
      `/go/config/generate/main.go` program, ensuring the schema stays in sync
      with the code.
    - The `validate.InstanceConfigFromFile` function uses this schema to
      validate a configuration file before attempting to deserialize it. This
      allows for early detection of malformed or incomplete configurations.
5.  **Command-Line Flag Integration:** The module defines structs like
    `BackendFlags`, `FrontendFlags`, `IngestFlags`, and `MaintenanceFlags`
    (`config.go`). These structs group related command-line flags and provide
    methods (`AsCliFlags`) to convert them into `cli.Flag` slices, compatible
    with the `github.com/urfave/cli/v2` library. This design keeps flag
    definitions organized and associated with the components they configure.
6.  **Extensibility:** The `InstanceConfig` is designed to be extensible. New
    configuration options can be added as new fields to the relevant
    sub-structs. The JSON schema generation and validation mechanisms will
    automatically adapt to these changes.

**Key Components and Files:**

- **`config.go`:** This is the heart of the module.
  - It defines the main `InstanceConfig` struct, which aggregates various
    sub-configuration structs like `AuthConfig`, `DataStoreConfig`,
    `IngestionConfig`, `GitRepoConfig`, `NotifyConfig`,
    `IssueTrackerConfig`, `AnomalyConfig`, `QueryConfig`, `TemporalConfig`,
    and `DataPointConfig`. Each of these sub-structs groups settings related
    to a specific aspect of the Perf system (e.g., authentication, data
    storage, data ingestion).
  - It defines various enumerated types (e.g., `DataStoreType`,
    `SourceType`, `GitAuthType`, `GitProvider`, `TraceFormat`) to provide
    clear and constrained options for certain configuration values.
  - It includes `DurationAsString`, a custom type for handling
    `time.Duration` serialization and deserialization as strings in JSON,
    which is more human-readable than nanosecond integers. It also provides
    a custom JSON schema for this type.
  - It defines structs for command-line flags used by different Perf
    services (backend, frontend, ingest, maintenance). This helps in
    organizing and parsing command-line arguments.
  - Global constants like `MaxSampleTracesPerCluster`, `MinStdDev`,
    `GotoRange`, and `QueryMaxRunTime` are defined here, providing default
    values or limits used across the application.
- **`/go/config/validate/validate.go`:**
  - This file contains the logic for validating an `InstanceConfig` beyond
    what the JSON schema can enforce. This includes semantic checks, such as
    ensuring that required fields are present based on the values of other
    fields (e.g., API keys for issue tracker notifications).
  - The `InstanceConfigFromFile` function is the primary entry point for
    loading and validating a configuration file. It first performs schema
    validation and then calls the `Validate` function for further business
    logic checks.
  - It also validates the Go text templates used in `NotifyConfig` by
    attempting to format them with sample data. This helps catch template
    syntax errors early.
- **`/go/config/validate/instanceConfigSchema.json`:**
  - This is an automatically generated JSON Schema file that defines the
    expected structure and data types for `InstanceConfig` JSON files. It is
    used by `validate.go` to perform initial validation of configuration
    files.
- **`/go/config/generate/main.go`:**
  - This is a small utility program that generates the
    `instanceConfigSchema.json` file based on the `InstanceConfig` struct
    definition in `config.go`. This ensures that the schema is always
    up-to-date with the Go code. The `//go:generate` directive at the top of
    the file allows for easy regeneration of the schema.
- **`config_test.go` and `/go/config/validate/validate_test.go`:**
  - These files contain unit tests for the configuration loading,
    serialization/deserialization (especially for custom types like
    `DurationAsString`), and validation logic. The tests for `validate.go`
    include checks against actual configuration files used in production
    (`//perf:configs`), ensuring that the validation logic is robust and
    correctly handles real-world scenarios.

**Workflows:**

**1. Loading and Validating a Configuration File:**

```
User provides config file path (e.g., "configs/nano.json")
  |
  V
Application calls validate.InstanceConfigFromFile("configs/nano.json")
  |
  V
validate.go: Reads the JSON file content.
  |
  V
validate.go: Validates content against instanceConfigSchema.json (using jsonschema.Validate).
  |                                 \
  | (If schema violation)            \
  V                                   V
Error returned with schema violations.  Deserializes JSON into config.InstanceConfig struct.
                                      |
                                      V
                                    validate.go: Calls Validate(instanceConfig) for further business logic checks.
                                      |           (e.g., API key presence, template validity)
                                      |
                                      | (If validation error)
                                      V
                                    Error returned.
                                      |
                                      V (If all valid)
                                    Returns the populated config.InstanceConfig struct.
                                      |
                                      V
                                    Application sets config.Config = returnedInstanceConfig
                                      |
                                      V
                                    Perf instance uses config.Config for its operations.
```

**2. Generating the JSON Schema:**

This is typically done during development when the `InstanceConfig` struct
changes.

```
Developer modifies config.InstanceConfig struct in config.go
  |
  V
Developer runs `go generate` in the /go/config/generate directory (or via bazel)
  |
  V
/go/config/generate/main.go: Calls jsonschema.GenerateSchema("../validate/instanceConfigSchema.json", &config.InstanceConfig{})
  |
  V
jsonschema library: Introspects the config.InstanceConfig struct and its fields.
  |
  V
jsonschema library: Generates a JSON Schema definition.
  |
  V
/go/config/generate/main.go: Writes the generated schema to /go/config/validate/instanceConfigSchema.json.
```

The design prioritizes robustness through schema and semantic validation,
maintainability through structured Go types and centralized configuration, and
ease of use through standard JSON format and command-line flag integration. The
separation of schema generation (`generate` subdirectory) and validation
(`validate` subdirectory) keeps concerns distinct.

# Module: /go/ctrace2

## ctrace2 Module Documentation

### Overview

The `ctrace2` module provides the functionality to adapt trace data (represented
as a series of floating-point values) for use with k-means clustering
algorithms. The primary goal is to transform raw trace data into a format that
is suitable for distance calculations and centroid computations, which are
fundamental operations in k-means. This involves normalization and handling of
missing data points.

### Why and How

In performance analysis, traces often represent measurements over time or across
different configurations. Clustering these traces helps identify groups of
similar performance characteristics. However, raw trace data might have issues
that hinder effective clustering:

1.  **Varying Scales:** Different traces might have values in vastly different
    ranges, leading to biased distance calculations where traces with larger
    absolute values dominate.
2.  **Missing Data:** Traces can have missing data points, which need to be
    handled appropriately during normalization and distance computation.
3.  **Zero Standard Deviation:** Traces with constant values (zero standard
    deviation) can cause division by zero errors during normalization.

The `ctrace2` module addresses these by:

- **Normalization:** Each trace is normalized to have a standard deviation of
  1.0. This ensures that the scale of the values does not disproportionately
  influence the clustering. The `vec32.Norm` function from the `go/vec32`
  module is leveraged for this. Before normalization, any missing data points
  (`vec32.MissingDataSentinel`) are filled in using `vec32.Fill`, which likely
  interpolates or uses a similar strategy to replace them.
- **Minimum Standard Deviation:** To prevent division by zero or issues with
  extremely small standard deviations, a `minStdDev` parameter is used during
  normalization. If the calculated standard deviation of a trace is below this
  minimum, the `minStdDev` value is used instead. This is a practical approach
  to handle traces with very little variation without excluding them from
  clustering.
- **`ClusterableTrace` Structure:** This structure wraps the trace data (`Key`
  and `Values`) and implements the `kmeans.Clusterable` and `kmeans.Centroid`
  interfaces from the `perf/go/kmeans` module. This makes `ClusterableTrace`
  instances directly usable by the k-means algorithm.

### Responsibilities and Key Components

- **`ctrace.go`:** This is the core file of the module.
  - **`ClusterableTrace` struct:**
  - **Purpose:** Represents a single trace that is ready for clustering. It
    holds a `Key` (a string identifier for the trace) and `Values` (a slice
    of `float32` representing the normalized data points).
  - **Why:** This struct is designed to be directly consumable by the
    k-means clustering algorithm by implementing necessary interfaces.
  - **`Distance(c kmeans.Clusterable) float64` method:** Calculates the
    Euclidean distance between the current `ClusterableTrace` and another
    `ClusterableTrace`. This is crucial for the k-means algorithm to
    determine how similar two traces are. The calculation assumes that both
    traces have the same number of data points (a guarantee maintained by
    `NewFullTrace`). `For each point i in trace1 and trace2: diff_i =
trace1.Values[i] - trace2.Values[i] squared_diff_i = diff_i * diff_i Sum
all squared_diff_i Distance = Sqrt(Sum)`
  - **`AsClusterable() kmeans.Clusterable` method:** Returns the
    `ClusterableTrace` itself, satisfying the `kmeans.Centroid` interface
    requirement.
  - **`Dup(newKey string) *ClusterableTrace` method:** Creates a deep copy
    of the `ClusterableTrace` with a new key. This is useful when you need
    to manipulate a trace without affecting the original.
  - **`NewFullTrace(key string, values []float32, minStdDev float32)
*ClusterableTrace` function:**
  - **Purpose:** The primary factory function for creating
    `ClusterableTrace` instances from raw trace data.
  - **How:** 1. It takes a `key` (string identifier), raw `values` (`[]float32`),
    and a `minStdDev`. 2. Creates a copy of the input `values` to avoid modifying the original
    slice. 3. Calls `vec32.Fill()` on the copied values. This step handles missing
    data points by filling them, likely through interpolation or a
    similar imputation technique provided by the `go/vec32` module. 4. Calls `vec32.Norm()` on the filled values, using `minStdDev`. This
    normalizes the trace data so that its standard deviation is
    effectively 1.0 (or adjusted if the original standard deviation was
    below `minStdDev`). 5. Returns a new `ClusterableTrace` with the provided `key` and the
    processed (filled and normalized) `values`. `Input: key, raw_values,
minStdDev ------------------------------------ copied_values =
copy(raw_values) filled_values = vec32.Fill(copied_values)
normalized_values = vec32.Norm(filled_values, minStdDev) Output:
ClusterableTrace{Key: key, Values: normalized_values}`
  - **`CalculateCentroid(members []kmeans.Clusterable) kmeans.Centroid`
    function:**
  - **Purpose:** Implements the `kmeans.CalculateCentroid` function type.
    Given a slice of `ClusterableTrace` instances (which are members of a
    cluster), it computes their centroid.
  - **How:** 1. It initializes a new slice of `float32` (`mean`) with the same
    length as the `Values` of the first member trace. 2. It iterates through each member trace in the `members` slice. 3. For each member, it iterates through its `Values` and adds each
    value to the corresponding element in the `mean` slice. 4. After summing up all values component-wise, it divides each element
    in the `mean` slice by the total number of `members` to get the
    average value for each dimension. 5. It returns a new `ClusterableTrace` representing the centroid. The
    key for this centroid trace is set to `CENTROID_KEY`
    ("special_centroid"). `Input: members (list of
ClusterableTraces) ------------------------------------------
Initialize mean_values = [0.0, 0.0, ..., 0.0] (same length as
members[0].Values) For each member_trace in members: For each i from
0 to len(member_trace.Values) - 1: mean_values[i] = mean_values[i] +
member_trace.Values[i] For each i from 0 to len(mean_values) - 1:
mean_values[i] = mean_values[i] / len(members) Output:
ClusterableTrace{Key: CENTROID_KEY, Values: mean_values}`
  - **`CENTROID_KEY` constant:**
  - **Purpose:** Defines a standard key ("special_centroid") to be used for
    traces that represent the centroid of a cluster.
  - **Why:** This provides a consistent way to identify centroid traces if
    they are, for example, added back into a collection of traces (e.g., in
    a DataFrame).

The interaction with the `go/vec32` module is crucial for data preprocessing
(filling missing values and normalization), while the `perf/go/kmeans` module
provides the interfaces that `ctrace2` implements to be compatible with k-means
clustering algorithms.

# Module: /go/culprit

The `culprit` module is responsible for identifying, storing, and notifying
about commits that are likely causes of performance regressions. It integrates
with anomaly detection and subscription systems to automate the process of
pinpointing culprits and alerting relevant parties.

## Key Responsibilities

- **Culprit Identification:** While the actual bisection logic might reside
  elsewhere, this module is responsible for receiving information about
  potential culprit commits.
- **Culprit Persistence:** Storing identified culprits in a database, linking
  them to the anomaly groups they are associated with.
- **Notification:** Generating and sending notifications (e.g., creating
  issues in an issue tracker) when new culprits are found or when new anomaly
  groups are reported.
- **Data Formatting:** Formatting notification messages (subjects and bodies)
  based on configurable templates.

## Key Components and Files

### `store.go` & `sqlculpritstore/sqlculpritstore.go`

- **Purpose:** These files define the interface and implementation for storing
  and retrieving culprit data. The primary goal is to persist information
  about commits identified as culprits, associating them with specific anomaly
  groups and any filed issues.
- **How it Works:**
  - `store.go` defines the `Store` interface, which outlines the contract
    for culprit data operations like `Get`, `Upsert`, and `AddIssueId`.
  - `sqlculpritstore/sqlculpritstore.go` provides a SQL-based implementation
    of this interface. It uses a SQL database (configured via `pool.Pool`)
    to store culprit information.
  - The `Upsert` method is crucial. It either inserts a new culprit record
    or updates an existing one if a commit has already been identified as a
    culprit for a different anomaly group. This prevents duplicate culprit
    entries for the same commit. It also links the culprit to the
    `anomaly_group_id`.
  - The `AddIssueId` method updates a culprit record to include the ID of an
    issue (e.g., a bug tracker ticket) that was created for it, and also
    maintains a map between the anomaly group and the issue ID. This is
    important for tracking and referencing.
  - The database schema (defined in `sqlculpritstore/schema/schema.go`)
    includes fields for commit details (host, project, ref, revision),
    associated anomaly group IDs, and associated issue IDs. An index on
    `(revision, host, project, ref)` helps in efficiently querying for
    existing culprits.
- **Design Choices:**
  - Using an interface (`Store`) decouples the rest of the module from the
    specific database implementation, allowing for easier testing and
    potential future changes in the storage backend.
  - The `Upsert` logic is designed to handle cases where the same commit
    might be identified as a culprit for multiple regressions (different
    anomaly groups). Instead of creating duplicate entries, it appends the
    new `anomaly_group_id` to the existing record.
  - Storing `group_issue_map` as JSONB allows flexible storage of the
    mapping between anomaly groups and the specific issue filed for that
    group in the context of this culprit.

### `formatter/formatter.go`

- **Purpose:** This component is responsible for constructing the content
  (subject and body) of notifications. It allows for customizable message
  formats.
- **How it Works:**
  - Defines the `Formatter` interface with methods
    `GetCulpritSubjectAndBody` (for new culprit notifications) and
    `GetReportSubjectAndBody` (for new anomaly group reports).
  - `MarkdownFormatter` is the concrete implementation. It uses Go's
    `text/template` package to render notification messages.
  - Templates for subjects and bodies can be provided via `InstanceConfig`.
    If not provided, default templates are used.
  - `TemplateContext` and `ReportTemplateContext` provide the data that can
    be used within the templates (e.g., commit details, subscription
    information, anomaly group details).
  - Helper functions like `buildCommitURL`, `buildAnomalyGroupUrl`, and
    `buildAnomalyDetails` are available within the templates to construct
    URLs and format anomaly details.
- **Design Choices:**
  - The use of interfaces and templates promotes flexibility. Users can
    define their own notification formats without modifying the core
    notification logic.
  - Default templates ensure that the system can function even without
    explicit template configuration.
  - Separating formatting from the transport mechanism (how notifications
    are sent) adheres to the single responsibility principle.
- **`formatter/noop.go`**: Provides a `NoopFormatter` that generates empty
  subjects and bodies, useful for disabling notifications or for testing
  scenarios where actual formatting is not needed.

### `transport/transport.go`

- **Purpose:** This component handles the actual sending of notifications to
  external systems, primarily issue trackers.
- **How it Works:**
  - Defines the `Transport` interface with the `SendNewNotification` method.
  - `IssueTrackerTransport` is the concrete implementation for interacting
    with an issue tracker (e.g., Google Issue Tracker/Buganizer).
  - It uses the `go.skia.org/infra/go/issuetracker/v1` client library.
  - Authentication is handled using an API key retrieved via the `secret`
    package.
  - When `SendNewNotification` is called, it constructs an
    `issuetracker.Issue` object based on the provided subject, body, and
    subscription details (like component ID, priority, CCs, hotlists).
  - It then calls the issue tracker API to create a new issue.
  - Metrics (`SendNewNotificationSuccess`, `SendNewNotificationFail`) are
    recorded to monitor the success rate of sending notifications.
- **Design Choices:**
  - The `Transport` interface allows for different notification mechanisms
    to be plugged in (e.g., email, Slack) in the future.
  - Configuration for the issue tracker (API key, secret project/name) is
    externalized, promoting better security and manageability.
  - Error handling and metrics provide visibility into the notification
    delivery process.
- **`transport/noop.go`**: Provides a `NoopTransport` that doesn't actually
  send any notifications, useful for disabling notifications or for testing.

### `notify/notify.go`

- **Purpose:** This component orchestrates the notification process by
  combining a `Formatter` and a `Transport`.
- **How it Works:**
  - Defines the `CulpritNotifier` interface with methods
    `NotifyCulpritFound` and `NotifyAnomaliesFound`.
  - `DefaultCulpritNotifier` implements this interface. It takes a
    `formatter.Formatter` and a `transport.Transport` as dependencies.
  - The `GetDefaultNotifier` factory function determines which `Formatter`
    and `Transport` to use based on the
    `InstanceConfig.IssueTrackerConfig.NotificationType`. If `NoneNotify`,
    it uses `NoopFormatter` and `NoopTransport`. If `IssueNotify`, it sets
    up `MarkdownFormatter` and `IssueTrackerTransport`.
  - `NotifyCulpritFound`:
  * Calls the formatter's `GetCulpritSubjectAndBody` to get the message
    content.
  * Calls the transport's `SendNewNotification` to send the message.
  * Returns the ID of the created issue (or an empty string if no
    notification was sent).
  - `NotifyAnomaliesFound`:
  * Calls the formatter's `GetReportSubjectAndBody`.
  * Calls the transport's `SendNewNotification`.
  * Returns the ID of the created issue.
- **Design Choices:**
  - Decouples the high-level notification logic from the specifics of
    message formatting and sending.
  - Configuration-driven selection of formatter and transport makes the
    notification behavior adaptable.

### `service/service.go`

- **Purpose:** Implements the gRPC service defined in `culprit.proto`. This is
  the main entry point for external systems (like a bisection service or an
  anomaly detection pipeline) to interact with the culprit module.
- **How it Works:**
  - Implements the `pb.CulpritServiceServer` interface.
  - It depends on `anomalygroup.Store`, `culprit.Store`,
    `subscription.Store`, and `notify.CulpritNotifier`.
  - **`PersistCulprit` RPC:**
  * Calls `culpritStore.Upsert` to save the identified culprit commits and
    associate them with the `anomaly_group_id`.
  * Calls `anomalygroupStore.AddCulpritIDs` to link the newly
    created/updated culprit IDs back to the anomaly group. ``[Client (e.g.,
Bisection Service)] | v [PersistCulpritRequest {Commits,
AnomalyGroupID}] | v [culpritService.PersistCulprit] | \ | `->
[culpritStore.Upsert(AnomalyGroupID, Commits)] -> Returns CulpritIDs | |
| v `<-----------------------
[anomalygroupStore.AddCulpritIDs(AnomalyGroupID, CulpritIDs)] | v
[PersistCulpritResponse {CulpritIDs}] | v [Client]``
  - **`GetCulprit` RPC:**
  * Calls `culpritStore.Get` to retrieve culprit details by their IDs.
  - **`NotifyUserOfCulprit` RPC:**
  * Retrieves culprit details using `culpritStore.Get`.
  * Loads the corresponding `AnomalyGroup` using
    `anomalygroupStore.LoadById`.
  * Loads the `Subscription` associated with the anomaly group using
    `subscriptionStore.GetSubscription`.
  * Calls `notifier.NotifyCulpritFound` for each culprit to send a
    notification (e.g., file a bug).
  * Calls `culpritStore.AddIssueId` to store the generated issue ID with the
    culprit and the specific anomaly group. ``[Client (e.g., Bisection
Service after PersistCulprit)] | v [NotifyUserOfCulpritRequest
{CulpritIDs, AnomalyGroupID}] | v [culpritService.NotifyUserOfCulprit]
|-> [culpritStore.Get(CulpritIDs)] -> Culprits |->
[anomalygroupStore.LoadById(AnomalyGroupID)] -> AnomalyGroup | | ->
[subscriptionStore.GetSubscription(AnomalyGroup.SubName,
AnomalyGroup.SubRev)] -> Subscription | (For each Culprit in Culprits) |
| | `-> [notifier.NotifyCulpritFound(Culprit, Subscription)] -> Returns
IssueID | | | v | `-> [culpritStore.AddIssueId(Culprit.ID, IssueID,
AnomalyGroupID)] | v [NotifyUserOfCulpritResponse {IssueIDs}] | v
[Client]``
  - **`NotifyUserOfAnomaly` RPC:**
  * Loads the `AnomalyGroup` and its associated `Subscription`.
  * Calls `notifier.NotifyAnomaliesFound` to send a notification about the
    group of anomalies (e.g., file a summary bug). ``[Client (e.g., Anomaly
Detection Service)] | v [NotifyUserOfAnomalyRequest {AnomalyGroupID,
Anomalies[]}] | v [culpritService.NotifyUserOfAnomaly] |->
[anomalygroupStore.LoadById(AnomalyGroupID)] -> AnomalyGroup | | ->
[subscriptionStore.GetSubscription(AnomalyGroup.SubName,
AnomalyGroup.SubRev)] -> Subscription | `->
[notifier.NotifyAnomaliesFound(AnomalyGroup, Subscription, Anomalies[])]
-> Returns IssueID | v [NotifyUserOfAnomalyResponse {IssueID}] | v
[Client]``
  - `PrepareSubscription` is a helper function used to potentially override
    or mock subscription details for testing or during transitional phases
    before full sheriff configuration is active. This is a temporary
    measure.
- **Design Choices:**
  - Clear separation of concerns: the service layer orchestrates actions by
    calling appropriate stores and notifiers.
  - gRPC provides a well-defined, language-agnostic interface for the
    service.
  - The authorization policy (`GetAuthorizationPolicy`) is currently set to
    allow unauthenticated access, which might need to be revisited for
    production environments.

### `proto/v1/culprit_service.proto`

- **Purpose:** Defines the gRPC service contract for culprit-related
  operations.
- **Key Messages and RPCs:**
  - `Commit`: Represents a source code commit.
  - `Culprit`: Represents an identified culprit commit, including its ID,
    the commit details, associated anomaly group IDs, and issue IDs. It also
    includes `group_issue_map` to track which issue was filed for which
    anomaly group in the context of this culprit.
  - `Anomaly`: Represents a detected performance anomaly (duplicated from
    anomalygroup service for potential independent evolution).
  - `PersistCulpritRequest`/`Response`: For storing new culprits.
  - `GetCulpritRequest`/`Response`: For retrieving existing culprits.
  - `NotifyUserOfAnomalyRequest`/`Response`: For triggering notifications
    about a new set of anomalies (anomaly group).
  - `NotifyUserOfCulpritRequest`/`Response`: For triggering notifications
    about newly identified culprits.
- **Design Choices:**
  - Proto definitions provide a clear, typed contract for communication.
  - The `Anomaly` message is duplicated from the `anomalygroup` service.
    This choice was made to allow the `culprit` service and `anomalygroup`
    service to evolve their respective `Anomaly` definitions independently
    if needed in the future, avoiding tight coupling.
  - The `group_issue_map` in the `Culprit` message is important for
    scenarios where a single culprit might be associated with multiple
    anomaly groups, and each of those (culprit, group) pairs might result in
    a distinct bug being filed.

### Mocks (`mocks/` subdirectories)

- These directories contain generated mock implementations for the interfaces
  defined within the `culprit` module (e.g., `Store`, `Formatter`,
  `Transport`, `CulpritNotifier`, `CulpritServiceServer`).
- **Purpose:** Facilitate unit testing by allowing dependencies to be easily
  mocked. This is standard practice for writing testable Go code. They are
  generated using tools like `mockery`.

## Overall Workflow Example: Finding and Notifying a Culprit

1.  **Anomaly Detection:** An external system detects a performance regression
    and groups related anomalies into an `AnomalyGroup`.
2.  **Bisection (External):** A bisection process is triggered (potentially
    externally) to find the commit(s) responsible for the anomalies in the
    `AnomalyGroup`.
3.  **Persist Culprit:** The bisection service calls the
    `CulpritService.PersistCulprit` RPC with the identified `Commit`(s) and the
    `AnomalyGroupID`.
    - `culpritService` uses `culpritStore.Upsert` to save these commits as
      `Culprit` records, linking them to the `AnomalyGroupID`.
    - It then calls `anomalygroupStore.AddCulpritIDs` to update the
      `AnomalyGroup` record with the IDs of these new culprits.
4.  **Notify User of Culprit:** The bisection service (or another orchestrator)
    then calls `CulpritService.NotifyUserOfCulprit` RPC with the `CulpritID`(s)
    and the `AnomalyGroupID`.
    - `culpritService` retrieves the full `Culprit` details and the associated
      `Subscription`.
    - `DefaultCulpritNotifier` is invoked:
      - `MarkdownFormatter` generates the subject and body for the
        notification.
      - `IssueTrackerTransport` sends this formatted message to the issue
        tracker, creating a new bug.
    - The ID of the created bug is returned.
    - `culpritService` calls `culpritStore.AddIssueId` to associate this bug
      ID with the specific `Culprit` and `AnomalyGroupID`.

This flow ensures that culprits are stored, linked to their regressions, and
users are notified through the configured channels. The modular design allows
for flexibility in how each step (storage, formatting, transport) is
implemented.

# Module: /go/dataframe

## Module: /go/dataframe

The `dataframe` module provides the `DataFrame` data structure and related
functionality for handling and manipulating performance trace data. It is a core
component for querying, analyzing, and visualizing performance metrics within
the Skia Perf system.

**Key Design Principles:**

- **Tabular Data Representation:** Inspired by R's DataFrame, this module
  represents performance data as a table. Rows correspond to individual traces
  (identified by a structured key), and columns represent distinct commit
  points or patch levels. This structure facilitates efficient querying and
  analysis of time-series data across different configurations.
- **TraceSet and ParamSet:** A `DataFrame` encapsulates a `types.TraceSet`,
  which is a map of trace keys to their corresponding performance values. It
  also maintains a `paramtools.ReadOnlyParamSet`, which describes the unique
  parameter key-value pairs present in the `TraceSet`. This allows for
  efficient filtering and aggregation based on trace characteristics.
- **Commit-Centric Columns:** The columns of a `DataFrame` are defined by
  `ColumnHeader` structs, each containing a commit offset and a timestamp.
  This ties the performance data directly to specific points in the codebase's
  history.
- **Data Retrieval Abstraction:** The `DataFrameBuilder` interface decouples
  the `DataFrame` creation logic from the underlying data source. This allows
  for different implementations to fetch data (e.g., from a database) while
  providing a consistent API for consumers.
- **Efficiency for Common Operations:** The module provides functions for
  common data manipulation tasks like merging DataFrames (`Join`), filtering
  traces (`FilterOut`), slicing data (`Slice`), and compressing data by
  removing empty columns (`Compress`). These operations are designed with
  performance considerations in mind.

**Key Components and Files:**

- **`dataframe.go`**: This is the central file defining the `DataFrame` struct
  and its associated methods.

  - **`DataFrame` struct**:
  - `TraceSet`: Stores the actual performance data, mapping trace keys
    (strings representing parameter combinations like
    ",arch=x86,config=8888,") to `types.Trace` (slices of float32 values).
  - `Header`: A slice of `*ColumnHeader` pointers, defining the columns of
    the DataFrame. Each `ColumnHeader` links a column to a specific commit
    (`Offset`) and its `Timestamp`.
  - `ParamSet`: A `paramtools.ReadOnlyParamSet` that contains all unique
    key-value pairs from the keys in `TraceSet`. This is crucial for
    understanding the dimensions of the data and for building UI controls
    for filtering. It's rebuilt by `BuildParamSet()`.
  - `Skip`: An integer indicating if any commits were skipped during data
    retrieval to keep the DataFrame size manageable (related to
    `MAX_SAMPLE_SIZE`).
  - **`DataFrameBuilder` interface**: Defines the contract for objects that
    can construct `DataFrame` instances. This allows for different data
    sources or retrieval strategies. Key methods include:
  - `NewFromQueryAndRange`: Creates a DataFrame based on a query and a time
    range.
  - `NewFromKeysAndRange`: Creates a DataFrame for specific trace keys over
    a time range.
  - `NewNFromQuery` / `NewNFromKeys`: Creates a DataFrame with the N most
    recent data points for matching traces or specified keys.
  - `NumMatches` / `PreflightQuery`: Used to estimate the size of the data
    that a query will return, often for UI feedback or to refine queries.
  - **`ColumnHeader` struct**: Represents a single column in the DataFrame,
    typically corresponding to a commit. It contains:
  - `Offset`: A `types.CommitNumber` identifying the commit.
  - `Timestamp`: The timestamp of the commit in seconds since the Unix
    epoch.
  - **Key Functions**:
  - `NewEmpty()`: Creates an empty DataFrame.
  - `NewHeaderOnly()`: Creates a DataFrame with populated headers (commits
    within a time range) but no trace data. This can be useful for setting
    up the structure before fetching actual data.
  - `FromTimeRange()`: Retrieves commit information (headers and commit
    numbers) for a given time range from a `perfgit.Git` instance. This is a
    foundational step in populating the `Header` of a DataFrame.
  - `MergeColumnHeaders()`: A utility function that takes two slices of
    `ColumnHeader` and merges them into a single sorted slice, returning
    mapping indices to reconstruct traces. This is essential for the `Join`
    operation.
  - `Join()`: Combines two DataFrames into a new DataFrame. It merges their
    headers and trace data. If traces exist in one DataFrame but not the
    other for a given key, missing data points (`vec32.MissingDataSentinel`)
    are inserted. The `ParamSet` of the resulting DataFrame is the union of
    the input ParamSets. `DataFrame A (Header: [C1, C3], TraceX: [v1, v3]) |
V DataFrame B (Header: [C2, C3], TraceX: [v2', v3']) | V Joined
DataFrame (Header: [C1, C2, C3], TraceX: [v1, v2', v3/v3']) (TraceY from
A or B padded with missing data)`
  - `BuildParamSet()`: Recalculates the `ParamSet` for a DataFrame based on
    the current keys in its `TraceSet`. This is called after operations like
    `FilterOut` that might change the set of traces.
  - `FilterOut()`: Removes traces from the `TraceSet` based on a provided
    `TraceFilter` function. It then calls `BuildParamSet()` to update the
    `ParamSet`.
  - `Slice()`: Returns a new DataFrame that is a view into a sub-section of
    the original DataFrame's columns. The underlying trace data is sliced,
    not copied, for efficiency.
  - `Compress()`: Creates a new DataFrame by removing any columns (and
    corresponding data points in traces) that contain only missing data
    sentinels across all traces. This helps in reducing data size and
    focusing on relevant data points.

- **`dataframe_test.go`**: Contains unit tests for the functionality in
  `dataframe.go`. These tests cover various scenarios, including empty
  DataFrames, different merging and joining cases, filtering, slicing, and
  compression. The tests often use `gittest` for creating mock Git
  repositories to test time range queries.

- **`/go/dataframe/mocks/DataFrameBuilder.go`**: This file contains a mock
  implementation of the `DataFrameBuilder` interface, generated using the
  `testify/mock` library. This mock is used in tests of other packages that
  depend on `DataFrameBuilder`, allowing them to simulate DataFrame creation
  without needing a real data source or Git repository.

**Workflows:**

1.  **Fetching Data for Display/Analysis:**

    - A client (e.g., a web UI) specifies a query and a time range.
    - An implementation of `DataFrameBuilder` (e.g., one that queries a
      CockroachDB instance) uses `NewFromQueryAndRange`.
    - Internally, this likely involves:
      1.  Resolving the time range to a list of commits using `FromTimeRange`
          (which calls `perfgit.Git.CommitSliceFromTimeRange`). This populates
          the `Header`.
      2.  Querying the data source for traces matching the query and falling
          within the identified commit range.
      3.  Populating the `TraceSet`.
      4.  Building the `ParamSet` using `BuildParamSet()`.
    - The resulting `DataFrame` is returned.

    ```
    Client Request (Query, TimeRange)
        |
        V
    DataFrameBuilder.NewFromQueryAndRange(ctx, begin, end, query, ...)
        |
        +-> FromTimeRange(ctx, git, begin, end, ...)  // Get commit headers
        |       |
        |       V
        |   perfgit.Git.CommitSliceFromTimeRange()
        |       |
        |       V
        |   [ColumnHeader{Offset, Timestamp}, ...]
        |
        +-> DataSource.QueryTraces(query, commit_numbers) // Fetch trace data
        |       |
        |       V
        |   types.TraceSet
        |
        +-> DataFrame.BuildParamSet() // Populate ParamSet
        |
        V
    DataFrame{Header, TraceSet, ParamSet}
    ```

2.  **Joining DataFrames (e.g., from different sources or queries):**

    - Two `DataFrame` instances, `dfA` and `dfB`, are available.
    - `Join(dfA, dfB)` is called.
    - `MergeColumnHeaders(dfA.Header, dfB.Header)` creates a unified header
      and maps to align traces.
    - A new `TraceSet` is built. For each key:
      - If a key is in `dfA` but not `dfB`, its trace is copied, padded with
        missing values for columns unique to `dfB`.
      - If a key is in `dfB` but not `dfA`, its trace is copied, padded with
        missing values for columns unique to `dfA`.
      - If a key is in both, values are merged based on the unified header.
    - The `ParamSet`s of `dfA` and `dfB` are combined.
    - A new, joined `DataFrame` is returned.

3.  **Filtering Data:**

    - A `DataFrame` `df` exists.
    - A `TraceFilter` function `myFilter` is defined (e.g., to remove traces
      with all zero values).
    - `df.FilterOut(myFilter)` is called.
    - The method iterates through `df.TraceSet`. If `myFilter` returns `true`
      for a trace, that trace is deleted from the `TraceSet`.
    - `df.BuildParamSet()` is called to reflect the potentially reduced set of
      parameters.

**Constants:**

- `DEFAULT_NUM_COMMITS`: Default number of commits to fetch when using methods
  like `NewNFromQuery`. Set to 50.
- `MAX_SAMPLE_SIZE`: A limit on the number of commits (columns) a DataFrame
  might contain, especially when downsampling. Set to 5000. (Note: The
  `downsample` parameter in `FromTimeRange` is currently ignored, meaning this
  might not be strictly enforced by that specific function directly but could
  be a target for other parts of the system or future enhancements.)

# Module: /go/dfbuilder

The `dfbuilder` module is responsible for constructing `DataFrame` objects.
`DataFrames` are fundamental data structures in Perf, representing a collection
of performance traces (time series data) along with their associated parameters
and commit information. This module acts as an intermediary between the raw
trace data stored in a `TraceStore` and the higher-level analysis and
visualization components that consume `DataFrames`.

The core design revolves around efficiently fetching and organizing trace data
based on various querying criteria. This involves interacting with a
`perfgit.Git` instance to resolve commit ranges and timestamps, and a
`tracestore.TraceStore` to retrieve the actual trace data.

**Key Responsibilities and Components:**

- **`dfbuilder.go`**: This is the central file implementing the
  `DataFrameBuilder` interface.
  - **`builder` struct**: This struct holds the necessary dependencies like
    `perfgit.Git`, `tracestore.TraceStore`, `tracecache.TraceCache`, and
    configuration parameters (e.g., `tileSize`, `numPreflightTiles`,
    `QueryCommitChunkSize`). It also maintains metrics for various DataFrame
    construction operations.
  - **Construction (`NewDataFrameBuilderFromTraceStore`)**: Initializes a
    `builder` instance. An important configuration here is
    `filterParentTraces`. If enabled, the builder will attempt to remove
    redundant parent traces when child traces (more specific traces) exist.
    For example, if traces for `test=foo,subtest=bar` and `test=foo` both
    exist, the latter might be filtered out if `filterParentTraces` is true.
  - **Fetching by Time Range and Query (`NewFromQueryAndRange`)**:
  - **Why**: This is a common use case where users want to see traces
    matching a specific query (e.g., `config=8888`) within a given time
    period.
  - **How**: 1. It first uses `dataframe.FromTimeRange` (which internally queries
    `perfgit.Git`) to get a list of `ColumnHeader` (commit information)
    and `CommitNumber`s within the specified time range. It also handles
    downsampling if requested. 2. It then determines the relevant tiles to query from the `TraceStore`
    based on the commit numbers (`sliceOfTileNumbersFromCommits`). 3. The core data fetching happens in the `new` method. This method
    queries the `TraceStore` for matching traces _per tile_ concurrently
    using `errgroup.Group` for parallelism. This is a key optimization
    to speed up data retrieval, especially over large time ranges
    spanning multiple tiles. 4. A `tracesetbuilder.TraceSetBuilder` is used to efficiently aggregate
    the traces fetched from different tiles into a single
    `types.TraceSet` and `paramtools.ParamSet`. 5. Finally, it constructs and returns a compressed `DataFrame`.
    `NewFromQueryAndRange | -> dataframe.FromTimeRange (get commits in
time range from Git) | -> sliceOfTileNumbersFromCommits (determine
tiles to query) | -> new (concurrently query TraceStore for each
tile) | -> TraceStore.QueryTraces (for each tile) | ->
tracesetbuilder.Add (aggregate results) | -> tracesetbuilder.Build |
-> DataFrame.Compress`
  - **Fetching by Keys and Time Range (`NewFromKeysAndRange`)**:
  - **Why**: Used when the specific trace keys are already known, and data
    for these keys is needed within a time range.
  - **How**: Similar to `NewFromQueryAndRange` in terms of getting commit
    information for the time range. However, instead of querying by a
    `query.Query` object, it directly calls `TraceStore.ReadTraces` for each
    relevant tile, providing the list of trace keys. Results are then
    aggregated. This is generally faster if the exact trace keys are known
    as it avoids the overhead of query parsing and matching within the
    `TraceStore`.
  - **Fetching N Most Recent Data Points (`NewNFromQuery`,
    `NewNFromKeys`)**:
  - **Why**: Often, users are interested in the N most recent data points
    for a query or a set of keys, typically for displaying recent trends or
    for alert evaluation.
  - **How**: These methods work by iterating backward in time, tile by tile
    (or by `QueryCommitChunkSize` if configured), until `N` data points are
    collected for the matching traces. 1. It starts from a given `end` time (or the latest commit if `end` is
    zero). 2. It determines an initial `beginIndex` and `endIndex` for commit
    numbers. The `QueryCommitChunkSize` can influence this `beginIndex`
    to fetch a larger chunk of commits at once, potentially improving
    parallelism in the `new` method. 3. In a loop: - It fetches commit headers and indices for the current
    `beginIndex`-`endIndex` range. - It calls the `new` method (for `NewNFromQuery`) or a similar
    tile-based fetching logic (for `NewNFromKeys`) to get a
    `DataFrame` for this smaller range. - It counts non-missing data points in the fetched `DataFrame`. If
    no data is found for `maxEmptyTiles` consecutive attempts, it
    stops to prevent searching indefinitely through sparse data. - It appends the data from the fetched `DataFrame` to the result
    `DataFrame`, working backward from the `N`th slot. - It then adjusts `beginIndex` and `endIndex` to move to the
    previous chunk of commits/tiles. 4. If `filterParentTraces` is enabled, it calls `filterParentTraces` to
    remove redundant parent traces from the final `TraceSet`. 5. The resulting `DataFrame` might have traces of length less than `N`
    if not enough data points were found. It trims the traces if
    necessary. `NewNFromQuery (or NewNFromKeys) | -> findIndexForTime
(get commit number for 'end' time) | -> Loop (until N points are
found or maxEmptyTiles reached): | -> fromIndexRange (get commits
for current chunk) | -> new (or similar logic for keys) (fetch data
for this chunk) | -> Aggregate data into result DataFrame | ->
Update beginIndex/endIndex to previous chunk | -> [Optional]
filterParentTraces | -> Trim traces if fewer than N points found`
  - **Preflighting Queries (`PreflightQuery`)**:
  - **Why**: Before executing a potentially expensive query to fetch a full
    `DataFrame`, it's useful to get an estimate of how many traces will
    match and what the resulting `ParamSet` will look like. This allows UIs
    to present filter options dynamically.
  - **How**: 1. It fetches the latest tile number from the `TraceStore`. 2. It queries the `numPreflightTiles` most recent tiles (concurrently)
    for trace IDs matching the query `q`. This uses `getTraceIds`, which
    first attempts to fetch from `tracecache` and falls back to
    `TraceStore.QueryTracesIDOnly`. 3. The trace IDs (which are `paramtools.Params`) found are used to
    build up a `ParamSet`. 4. The count of matching traces from the tile with the most matches is
    taken as the estimated count. 5. Crucially, for parameter keys _present in the input query `q`_, it
    replaces the values in the computed `ParamSet` with _all_ values for
    those keys from the `referenceParamSet`. This ensures that the UI
    can still offer all possible filter options for parameters the user
    has already started filtering on. 6. The resulting `ParamSet` is normalized. `PreflightQuery | ->
TraceStore.GetLatestTile | -> Loop (for numPreflightTiles,
concurrently): | -> getTraceIds (TileN, query) // Checks tracecache
first, then TraceStore.QueryTracesIDOnly | -> [If cache miss]
TraceStore.QueryTracesIDOnly | -> [If cache miss & tracecache
enabled] tracecache.CacheTraceIds | -> Aggregate Params into a new
ParamSet -> Update max count | -> Update ParamSet with values from
referenceParamSet for keys in the original query | -> Normalize
ParamSet`
  - **Counting Matches (`NumMatches`)**:
  - **Why**: A simpler version of `PreflightQuery` that only returns the
    estimated number of matching traces.
  - **How**: It queries the two most recent tiles using
    `TraceStore.QueryTracesIDOnly` and returns the higher of the two counts.
  - **Parent Trace Filtering (`filterParentTraces` function)**:
  - **Why**: To reduce data redundancy and present a cleaner set of traces
    to the user, especially in UIs where deeply nested subtests can create
    many similar-looking parent traces.
  - **How**: It uses `tracefilter.NewTraceFilter()`. For each trace key in
    the input `TraceSet`:
    1.  The key is parsed into `paramtools.Params`.
    2.  A "path" is constructed from the parameter values based on a
        predefined order of keys (e.g., "master", "bot", "benchmark",
        "test", "subtest_1", ...).
    3.  This path and the original trace key are added to the `traceFilter`.
    4.  After processing all keys, `traceFilter.GetLeafNodeTraceKeys()`
        returns only the keys corresponding to the most specific (leaf)
        traces in the hierarchical structure implied by the paths.
    5.  A new `TraceSet` is built containing only these leaf node traces.
  - **Caching (`getTraceIds`, `cacheTraceIdsIfNeeded`)**:
  - **Why**: `QueryTracesIDOnly` can still be somewhat expensive if
    performed frequently on the same tiles and queries (e.g., during
    `PreflightQuery`). Caching the results (the list of matching trace
    IDs/params) can significantly speed this up.
  - **How**: The `getTraceIds` function first attempts to retrieve trace IDs
    from the `tracecache.TraceCache`. If there's a cache miss or the cache
    is not configured, it queries the `TraceStore`. If a database query was
    performed and the cache is configured, `cacheTraceIdsIfNeeded` is called
    to store the results in the cache for future requests. The cache key is
    typically a combination of the tile number and the query string.

**Design Choices and Trade-offs:**

- **Tile-Based Processing**: The `TraceStore` organizes data into tiles. Most
  `dfbuilder` operations that involve fetching data across a range of commits
  are designed to process these tiles concurrently. This improves performance
  by parallelizing I/O and computation.
- **`tracesetbuilder`**: This utility is used to efficiently merge trace data
  coming from different tiles (which might have different sets of commits)
  into a coherent `TraceSet` and `ParamSet`.
- **`QueryCommitChunkSize`**: This parameter in `NewNFromQuery` allows
  fetching data in larger chunks than a single tile. This can increase
  parallelism in the underlying `new` method call, but fetching too large a
  chunk might lead to excessive memory usage or longer latency for the first
  chunk.
- **`maxEmptyTiles` / `newNMaxSearch`**: When searching backward for N data
  points, these constants prevent indefinite searching if the data is very
  sparse or the query matches very few traces.
- **`singleTileQueryTimeout`**: This guards against queries on individual
  tiles taking too long, which could happen with "bad" tiles containing
  excessive data or due to backend issues. This is particularly important for
  operations like `NewNFromQuery` or `PreflightQuery` which might issue many
  such single-tile queries.
- **Caching for `PreflightQuery`**: `PreflightQuery` is often called by UIs to
  populate filter options. Caching the results of `QueryTracesIDOnly` (which
  provides the raw data for `ParamSet` construction in preflight) via
  `tracecache` helps make these UI interactions faster.
- **Parent Trace Filtering**: This is an opinionated feature that aims to
  improve usability by default. The specific heuristic for identifying
  "parent" vs. "child" traces is based on a predefined order of parameter
  keys.

The `dfbuilder_test.go` file provides comprehensive unit tests for these
functionalities, covering various scenarios including empty queries, queries
matching data in single or multiple tiles, N-point queries, and preflight
operations with and without caching. It uses `gittest` for creating a mock Git
history and `sqltest` (for Spanner) or mock implementations for the `TraceStore`
and `TraceCache`.

# Module: /go/dfiter

## `dfiter` Module Documentation

### Overview

The `dfiter` module is responsible for efficiently creating and providing
`dataframe.DataFrame` objects, which are fundamental data structures used in
regression detection within the Perf application. It acts as an iterator,
allowing consuming code to process DataFrames one by one. This is particularly
useful for performance reasons, as constructing and holding all possible
DataFrames in memory simultaneously could be resource-intensive.

The core purpose of `dfiter` is to abstract away the complexities of fetching
and structuring data from the underlying trace store and Git history. It ensures
that DataFrames are generated with the correct dimensions and data points based
on user-defined queries, commit ranges, and alert configurations.

### Design and Implementation Choices

The `dfiter` module employs a "slicing" strategy for generating DataFrames. This
means it typically fetches a larger, encompassing DataFrame from the
`dataframe.DataFrameBuilder` and then yields smaller, overlapping
sub-DataFrames.

**Why this approach?**

- **Efficiency:** Fetching a larger chunk of data once from the database (via
  `DataFrameBuilder`) is often more efficient than making numerous small
  queries. The slicing operation itself is a relatively cheap in-memory
  operation.
- **Context for Regression Detection:** Regression detection algorithms often
  need to look at data points before and after a specific commit (the "radius"
  of an alert). The slicing approach naturally provides this sliding window of
  context.

**Key Components and Responsibilities:**

- **`DataFrameIterator` Interface:**
  - **Why:** Defines a standard contract for iterating over DataFrames. This
    promotes loose coupling, allowing different implementations of DataFrame
    generation if needed in the future, and simplifies how other parts of
    the system consume DataFrames.
  - **How:** It provides two methods:
  - `Next() bool`: Advances the iterator to the next DataFrame. Returns
    `true` if a next DataFrame is available, `false` otherwise.
  - `Value(ctx context.Context) (*dataframe.DataFrame, error)`: Returns the
    current DataFrame.
- **`dataframeSlicer` struct:**
  - **Why:** This is the concrete implementation of `DataFrameIterator`. It
    embodies the slicing strategy described above.
  - **How:** It holds a reference to a larger, source `dataframe.DataFrame`
    (`df`), the desired `size` of the sliced DataFrames (determined by
    `alert.Radius`), and the current `offset` for slicing. The `Next()`
    method checks if another slice of the specified `size` can be made, and
    `Value()` performs the actual slicing using `df.Slice()`.
- **`NewDataFrameIterator` Function:**
  - **Why:** This is the factory function for creating `DataFrameIterator`
    instances. It encapsulates the logic for determining how the initial,
    larger DataFrame should be fetched based on the input parameters.
  - **How:**
  * **Parameter Parsing:** Parses the input `queryAsString` into a
    `query.Query` object.
  * **Mode Determination (Implicit):** The function behaves differently
    based on `domain.Offset`: - **`domain.Offset == 0` (Continuous/Sliding Window Mode):** - This mode is typically used for ongoing regression detection
    across a range of recent commits. - It fetches a DataFrame of `domain.N` commits ending at
    `domain.End`. - **Settling Time:** If `anomalyConfig.SettlingTime` is
    configured, it adjusts `domain.End` to exclude very recent data
    points that might not have "settled" (e.g., due to data
    ingestion delays or pending backfills). This prevents alerts on
    potentially incomplete or volatile fresh data. - The `dataframeSlicer` will then produce overlapping DataFrames
    of size `2*alert.Radius + 1`. - **`domain.Offset != 0` (Specific Commit/Exact DataFrame Mode):** - This mode is used when analyzing a specific commit or a small,
    fixed window around it (e.g., when a user clicks on a specific
    point in a chart to see its details or re-runs detection for a
    particular regression). - It aims to return a _single_ DataFrame. - The size of this DataFrame is `2*alert.Radius + 1`. - To determine the `End` time for fetching data, it calculates the
    commit `alert.Radius` positions _after_ the `domain.Offset`.
    This ensures the commit at `domain.Offset` is centered within
    the radius. For example, if `domain.Offset` is commit 21 and
    `alert.Radius` is 3, it will fetch data up to commit 24 (`21 +
3`). The resulting DataFrame will then contain commits `[18, 19,
20, 21, 22, 23, 24]`. This is a specific requirement to ensure
    consistency with how different step detection algorithms expect
    their input DataFrames.
  * **Data Fetching:** Uses the injected `dataframe.DataFrameBuilder`
    (`dfBuilder`) to construct the initial DataFrame
    (`dfBuilder.NewNFromQuery`). This involves querying the trace store and
    potentially Git history.
  * **Data Sufficiency Check:** Verifies if the fetched DataFrame contains
    enough data points (at least `2*alert.Radius + 1` commits). If not, it
    returns `ErrInsufficientData`. This is crucial because regression
    detection algorithms require a minimum amount of data to operate
    correctly.
  * **Metrics:** Records the number of floating-point values queried from
    the database using
    `metrics2.GetCounter("perf_regression_detection_floats")`. This helps in
    monitoring the data processing load.
  * **Iterator Instantiation:** Creates and returns a `dataframeSlicer`
    instance initialized with the fetched DataFrame and the calculated slice
    size.
- **`ErrInsufficientData`:**
  - **Why:** A specific error type to indicate that while the queries were
    successful, the available data didn't meet the minimum requirements
    (e.g., not enough commits within the requested range or matching the
    query). This allows calling code to handle this scenario gracefully,
    perhaps by informing the user or adjusting parameters.

### Key Workflows

**1. Continuous Regression Detection (Sliding Window):**

This typically happens when `domain.Offset` is 0.

```
[Caller]                                     [NewDataFrameIterator]                                      [DataFrameBuilder]
   | -- Request with query, domain (N, End), alert (Radius) --> |                                                            |
   |                                                            | -- Parse query                                             |
   |                                                            | -- (If anomalyConfig.SettlingTime > 0) Adjust domain.End --> |
   |                                                            | -- dfBuilder.NewNFromQuery(ctx, domain.End, q, domain.N) --> |
   |                                                            |                                                            | -- Query TraceStore
   |                                                            |                                                            | -- Build large DataFrame
   |                                                            |                                                            | <----- DataFrame (df)
   |                                                            | -- Check if len(df.Header) >= 2*Radius+1                   |
   |                                                            | -- (If insufficient) Return ErrInsufficientData ----------- |
   |                                                            | -- Create dataframeSlicer(df, size=2*Radius+1, offset=0)   |
   | <----------------- DataFrameIterator (slicer) ------------- |                                                            |

[Caller]                                     [dataframeSlicer]
   |                                                            |
   | -- it.Next() ---------------------------------------------> |
   |                                                            | -- return offset+size <= len(df.Header)
   | <------------------------------ true ---------------------- |
   | -- it.Value() --------------------------------------------> |
   |                                                            | -- subDf = df.Slice(offset, size)
   |                                                            | -- offset++
   | <-------------------------- subDf, nil -------------------- |
   | -- (Process subDf)                                         |
   | ... (loop Next()/Value() until Next() returns false) ...   |
```

**2. Specific Commit Analysis (Exact DataFrame):**

This typically happens when `domain.Offset` is non-zero.

```
[Caller]                                           [NewDataFrameIterator]                                    [Git]      [DataFrameBuilder]
   | -- Request with query, domain (Offset), alert (Radius) --> |                                                          |                   |
   |                                                          | -- Parse query                                           |                   |
   |                                                          | -- targetCommitNum = domain.Offset + alert.Radius        |                   |
   |                                                          | -- perfGit.CommitFromCommitNumber(targetCommitNum) ------> |                   |
   |                                                          |                                                          | -- Lookup commit  |
   |                                                          | <----------------------------- commitDetails, nil --------- |                   |
   |                                                          | -- dfBuilder.NewNFromQuery(ctx, commitDetails.Timestamp, |                   |
   |                                                          |                            q, n=2*Radius+1) ------------> |                   |
   |                                                          |                                                          |                   | -- Query TraceStore
   |                                                          |                                                          |                   | -- Build DataFrame (size 2*R+1)
   |                                                          | <-------------------------------------------------------- DataFrame (df) ----- |
   |                                                          | -- Check if len(df.Header) >= 2*Radius+1                 |                   |
   |                                                          | -- (If insufficient) Return ErrInsufficientData --------- |                   |
   |                                                          | -- Create dataframeSlicer(df, size=2*Radius+1, offset=0) |                   |
   | <----------------------- DataFrameIterator (slicer) ------ |                                                          |                   |

[Caller]                                     [dataframeSlicer]
   |                                                            |
   | -- it.Next() ---------------------------------------------> |
   |                                                            | -- return offset+size <= len(df.Header) (true for the first call)
   | <------------------------------ true ---------------------- |
   | -- it.Value() --------------------------------------------> |
   |                                                            | -- subDf = df.Slice(offset, size) (returns the whole df)
   |                                                            | -- offset++
   | <-------------------------- subDf, nil -------------------- |
   | -- (Process subDf)                                         |
   | -- it.Next() ---------------------------------------------> |
   |                                                            | -- return offset+size <= len(df.Header) (false for subsequent calls)
   | <------------------------------ false --------------------- |
```

This design allows for flexible and efficient generation of DataFrames tailored
to the specific needs of regression detection, whether it's scanning a wide
range of recent commits or focusing on a particular point in time. The use of an
iterator pattern also helps manage memory consumption by processing DataFrames
sequentially.

# Module: /go/dryrun

## Dryrun Module Documentation

### Overview

The `dryrun` module provides the capability to test an alert configuration and
preview the regressions it would identify without actually creating an alert or
sending notifications. This is a crucial tool for developers and performance
engineers to fine-tune alert parameters and ensure they accurately capture
relevant performance changes.

The core idea is to simulate the regression detection process for a given alert
configuration over a historical range of data. This allows users to iterate on
alert definitions, observe the potential impact of those definitions, and avoid
alert fatigue caused by poorly configured alerts.

### Responsibilities and Key Components

The primary responsibility of the `dryrun` module is to handle HTTP requests for
initiating and reporting the progress of these alert simulations.

#### Key Files and Components:

- **`dryrun.go`**: This is the heart of the `dryrun` module. It defines the
  `Requests` struct, which manages the state and dependencies required for
  processing dry run requests. It also contains the HTTP handler
  (`StartHandler`) that orchestrates the dry run process.

  - **`Requests` struct**:

  - **Why**: Encapsulates all necessary dependencies (like `perfgit.Git` for
    Git interactions, `shortcut.Store` for shortcut lookups,
    `dataframe.DataFrameBuilder` for data retrieval, `progress.Tracker` for
    reporting progress, and `regression.ParamsetProvider` for accessing
    parameter sets) into a single unit. This promotes modularity and makes
    it easier to manage and test the dry run functionality.

  - **How**: It is instantiated via the `New` function, which takes these
    dependencies as arguments. This allows for dependency injection, making
    the component more testable and flexible.

  - **`StartHandler` function**:

  - **Why**: This is the entry point for initiating a dry run. It handles
    the incoming HTTP request, validates the alert configuration, and kicks
    off the asynchronous regression detection process.

  - **How**:

    1.  It decodes the alert configuration from the HTTP request body.
    2.  It performs initial validation on the alert query and other
        parameters. If validation fails, an error is immediately reported to
        the client.
    3.  It uses a `progress.Tracker` to allow clients to monitor the status
        of the long-running dry run operation.
    4.  Crucially, it launches the actual regression detection in a separate
        goroutine. This is essential because regression detection can be a
        time-consuming process, and blocking the HTTP handler would lead to
        timeouts and poor user experience.
    5.  It defines a `detectorResponseProcessor` callback function. This
        function is invoked by the underlying
        `regression.ProcessRegressions` function whenever potential
        regressions are found.
        - **Why (callback)**: This design decouples the core regression
          detection logic from the specifics of how dry run results are
          formatted and reported. It allows the `regression` module to
          focus on detection, while the `dryrun` module handles the
          presentation and progress updates for the dry run scenario.
        - **How (callback)**: The callback processes the raw
          `ClusterResponse` objects from the regression detection,
          converts them into user-friendly `RegressionAtCommit` structures
          (which include commit details and the detected regression), and
          updates the `Progress` object with these results. This enables
          real-time feedback to the user as regressions are identified.
    6.  The `regression.ProcessRegressions` function is then called in the
        goroutine, passing the alert request, the callback, and other
        necessary dependencies. This function iterates through the relevant
        data, applies the alert's clustering and detection logic, and
        invokes the callback for each identified cluster.
    7.  The handler immediately responds to the client with the initial
        `Progress` object, allowing the client to start polling for updates.

  - **`RegressionAtCommit` struct**:

  - **Why**: Provides a structured way to represent a regression found at a
    specific commit. This includes both the commit information (`CID`) and
    the details of the regression itself (`Regression`).

  - **How**: It's a simple struct used for marshalling the results into JSON
    for the client.

### Workflows

#### Dry Run Initiation and Processing:

```
Client (UI/API) --HTTP POST /dryrun/start with AlertConfig--> Requests.StartHandler
                                                                    |
                                                                    V
                                                        [Validate AlertConfig]
                                                                    |
                                 +----------------------------------+----------------------------------+
                                 | (Validation Fails)                                                | (Validation Succeeds)
                                 V                                                                   V
                        [Update Progress with Error]                                     [Add to Progress Tracker]
                                 |                                                                   |
                                 V                                                                   V
                  Respond to Client with Error Progress             Launch Goroutine: regression.ProcessRegressions(...)
                                                                    |
                                                                    V
                                                  [Iterate through data, detect regressions]
                                                                    |
                                                                    V
                                             For each potential regression cluster:
                                             Invoke `detectorResponseProcessor` callback
                                                                    |
                                                                    V
                                              Callback: [Convert ClusterResponse to RegressionAtCommit]
                                                        [Update Progress with new RegressionAtCommit]
                                                                    |
                                                                    V
                                                      (Client polls for Progress updates)
                                                                    |
                                                                    V
                                                   When ProcessRegressions completes:
                                                   [Update Progress: Finished or Error]

```

The `StartHandler` effectively acts as a controller that receives the request,
performs initial setup and validation, and then delegates the heavy lifting of
regression detection to the `regression.ProcessRegressions` function, ensuring
the HTTP request can return quickly while the background processing continues.
The callback mechanism allows the `dryrun` module to react to findings from the
`regression` module in a way that's specific to the dry run use case (i.e.,
accumulating and formatting results for client display).

# Module: /go/favorites

## Favorites Module

The `favorites` module provides functionality for users to save and manage
"favorite" configurations or views within the Perf application. This allows
users to quickly return to specific data explorations or commonly used settings.

The core design philosophy is to provide a persistent storage mechanism for
user-specific preferences related to application state (represented as URLs).
This is achieved through a `Store` interface, which abstracts the underlying
data storage, and a concrete SQL-based implementation.

### Key Components and Responsibilities

- **`store.go`**: This file defines the central `Store` interface.

  - **Why**: The interface decouples the business logic of managing
    favorites from the specific database implementation. This promotes
    testability (using mocks) and allows for potential future changes to the
    storage backend without impacting the core application logic.
  - **How**: It specifies the fundamental CRUD (Create, Read, Update,
    Delete) operations for favorites, along with a `List` operation to
    retrieve all favorites for a specific user and a `Liveness` check.
  - `Favorite`: This struct represents a single favorite item, containing
    fields like `ID`, `UserId`, `Name`, `Url`, `Description`, and
    `LastModified`. The `Url` is a key piece of data, as it allows the
    application to reconstruct the state the user wants to save.
  - `SaveRequest`: This struct is used for creating and updating favorites,
    encapsulating the data needed for these operations, notably excluding
    the `ID` (which is generated or already known) and `LastModified` (which
    is handled by the store).
  - **Liveness**: The `Liveness` method is a bit of an outlier. It's used to
    check the health of the database connection. It was placed in this store
    "arbitrarily because of its lack of essential function" in other more
    critical stores, making it a relatively safe place to perform this check
    without impacting core performance data operations.

- **`sqlfavoritestore/sqlfavoritestore.go`**: This file provides the SQL
  implementation of the `Store` interface.

  - **Why**: CockroachDB (or a similar SQL database) is used as the
    persistent storage for favorites. This choice provides a robust,
    scalable, and transactional way to manage this data.
  - **How**:
  - It defines SQL statements for each operation in the `Store` interface.
    These statements interact with a `Favorites` table.
  - The `FavoriteStore` struct holds a database connection pool
    (`pool.Pool`).
  - Methods like `Get`, `Create`, `Update`, `Delete`, and `List` execute
    their corresponding SQL statements against the database.
  - Timestamps (`LastModified`) are handled automatically during create and
    update operations to track when a favorite was last changed.
  - Error handling is done using `skerr.Wrapf` to provide context to any
    database errors.

- **`sqlfavoritestore/schema/schema.go`**: This file defines the SQL schema
  for the `Favorites` table.

  - **Why**: It provides a structured, Go-based representation of the
    database table. This can be useful for schema management, migrations,
    and ORM-like interactions (though a full ORM isn't used here).
  - **How**: The `FavoriteSchema` struct uses struct tags (`sql:"..."`) to
    define column names, types, constraints (like `PRIMARY KEY`, `NOT
NULL`), and indexes. The `byUserIdIndex` is crucial for efficiently
    listing favorites for a specific user.

- **`mocks/Store.go`**: This file contains a generated mock implementation of
  the `Store` interface.

  - **Why**: Mocks are essential for unit testing components that depend on
    the `Store` interface. They allow tests to simulate different store
    behaviors (e.g., successful operations, errors) without requiring an
    actual database connection.
  - **How**: This file is auto-generated by the `mockery` tool. It provides
    a `Store` struct that embeds `mock.Mock` from the `testify` library.
    Each method of the interface has a corresponding mock function that can
    be configured to return specific values or errors.

### Key Workflows

**1. Creating a New Favorite:**

```
User Action (e.g., clicks "Save as Favorite" in UI)
    |
    V
Application Handler
    |
    V
[favorites.Store.Create] is called with user ID, name, URL, description
    |
    V
[sqlfavoritestore.FavoriteStore.Create]
    |
    V
Generates current timestamp for LastModified
    |
    V
Executes INSERT SQL statement:
  INSERT INTO Favorites (user_id, name, url, description, last_modified) VALUES (...)
    |
    V
Database stores the new favorite record
    |
    V
Returns success/error to Application Handler
```

**2. Listing User's Favorites:**

```
User navigates to "My Favorites" page
    |
    V
Application Handler
    |
    V
[favorites.Store.List] is called with the current user's ID
    |
    V
[sqlfavoritestore.FavoriteStore.List]
    |
    V
Executes SELECT SQL statement:
  SELECT id, name, url, description FROM Favorites WHERE user_id=$1
    |
    V
Database returns rows matching the user ID
    |
    V
[sqlfavoritestore.FavoriteStore.List] scans rows into []*favorites.Favorite
    |
    V
Returns list of favorites to Application Handler
    |
    V
UI displays the list
```

**3. Retrieving a Specific Favorite (e.g., when a user clicks on a favorite to
load it):**

```
User clicks on a specific favorite in their list
    |
    V
Application Handler (obtains favorite ID)
    |
    V
[favorites.Store.Get] is called with the favorite ID
    |
    V
[sqlfavoritestore.FavoriteStore.Get]
    |
    V
Executes SELECT SQL statement:
  SELECT id, user_id, name, url, description, last_modified FROM Favorites WHERE id=$1
    |
    V
Database returns the single matching favorite row
    |
    V
[sqlfavoritestore.FavoriteStore.Get] scans row into a *favorites.Favorite struct
    |
    V
Returns the favorite object to Application Handler
    |
    V
Application uses the `Url` from the favorite object to restore the application state
```

# Module: /go/file

The `file` module and its submodules are responsible for providing a unified
interface for accessing files from different sources, such as local directories
or Google Cloud Storage (GCS). This abstraction allows the Perf ingestion system
to treat files consistently regardless of their origin.

## Core Concepts

The central idea is to define a `file.Source` interface that abstracts the
origin of files. Implementations of this interface are then responsible for
monitoring their respective sources (e.g., a GCS bucket via Pub/Sub
notifications, or a local directory) and emitting `file.File` objects through a
channel when new files become available.

The `file.File` struct encapsulates the essential information about a file: its
name, an `io.ReadCloser` for its contents, its creation timestamp, and
optionally, the associated `pubsub.Message` if the file originated from a GCS
Pub/Sub notification. This optional field is crucial for acknowledging the
message after successful processing, or nack'ing it if an error occurs, ensuring
reliable message handling in a distributed system.

### `file.go`

This file defines the core `File` struct and the `Source` interface.

- **`File` struct:** Represents a single file.

  - `Name`: The identifier for the file (e.g., `gs://bucket/object` or a
    local path).
  - `Contents`: An `io.ReadCloser` to read the file's content. This design
    allows for streaming file data, which is memory-efficient, especially
    for large files. The consumer is responsible for closing this reader.
  - `Created`: The timestamp when the file was created or last modified
    (depending on the source).
  - `PubSubMsg`: A pointer to a `pubsub.Message`. This is populated if the
    file notification came from a Pub/Sub message (e.g., GCS object change
    notifications). It's used to `Ack` or `Nack` the message, indicating
    successful processing or a desire to retry/dead-letter.

- **`Source` interface:** Defines the contract for file sources.

  - `Start(ctx context.Context) (<-chan File, error)`: This method initiates
    the process of watching for new files. It returns a read-only channel
    (`<-chan File`) through which `File` objects are sent as they are
    discovered. The method is designed to be called only once per `Source`
    instance. This design ensures that the resource setup and monitoring
    logic (like starting a Pub/Sub subscription listener or initiating a
    directory walk) is done once.

## Implementations of `file.Source`

### `dirsource`

The `dirsource` submodule provides an implementation of `file.Source` that reads
files from a local filesystem directory.

- **Purpose:** Primarily intended for testing and demonstration purposes. It
  allows developers to simulate file ingestion locally without needing to set
  up GCS or Pub/Sub.
- **Mechanism:**
  - `New(dir string)`: Constructs a `DirSource` for a given directory path.
    It resolves the path to an absolute path.
  - `Start(_ context.Context)`: When called, it initiates a `filepath.Walk`
    over the specified directory.
  - For each regular file encountered, it opens the file and creates a
    `file.File` object.
  - The `ModTime` of the file is used as the `Created` timestamp, which is a
    known simplification for its intended use cases.
  - The `file.File` objects are sent to an unbuffered channel.
  - The channel is closed after the directory walk is complete.
- **Limitations:**
  - It performs a one-time walk of the directory. It does not watch for new
    files or changes to existing files after the initial walk.
  - It uses the file's modification time as the creation time.
- **Workflow:** `New(directory) -> DirSource instance | V DirSource.Start()
--> Goroutine starts | V filepath.Walk(directory) | +----------------------+
| | V V For each file: For each directory: os.Open(path) (skip) Create
file.File{Name, Contents, ModTime} Send file.File to channel | V Caller
receives file.File from channel`

### `gcssource`

The `gcssource` submodule implements `file.Source` for files stored in Google
Cloud Storage, using Pub/Sub notifications for new file events.

- **Purpose:** This is the production-grade implementation for ingesting files
  from GCS. It's designed to be robust and scalable.
- **Mechanism:**
  - `New(ctx context.Context, instanceConfig *config.InstanceConfig)`:
  - Initializes GCS and Pub/Sub clients using default application
    credentials.
  - Constructs a Pub/Sub subscription name. It can either use a
    pre-configured `Subscription` name from `instanceConfig` or generate one
    based on the `Topic` (often adding a suffix like `-prod` or using a
    round-robin scheme for load distribution if multiple ingester instances
    are running).
  - Creates a `sub.Subscription` object to manage receiving messages from
    the configured Pub/Sub topic/subscription. A key configuration here is
    `ReceiveSettings.MaxExtension = -1`. This disables automatic ack
    deadline extension by the Pub/Sub client library. The rationale is that
    the `gcssource` itself will explicitly `Ack` or `Nack` messages. If
    automatic extension were enabled and the processing of a file took
    longer than the extension period, the message might be redelivered while
    still being processed, leading to duplicate processing or other issues.
    By disabling it, the ingester has full control over the message
    lifecycle.
  - Initializes a `filter.Filter` based on `AcceptIfNameMatches` and
    `RejectIfNameMatches` regular expressions provided in the
    `instanceConfig`. This allows for fine-grained control over which files
    are processed based on their GCS object names.
  - Determines if dead-lettering is enabled based on the instance
    configuration.
  - `Start(ctx context.Context)`:
  - Creates the output channel for `file.File` objects.
  - Launches a goroutine that continuously calls
    `s.subscription.Receive(ctx, s.receiveSingleEventWrapper)`.
    - The `Receive` method blocks until a message is available or the
      context is cancelled.
    - `receiveSingleEventWrapper` is called for each Pub/Sub message.
- **File Event Processing (`receiveSingleEventWrapper` and
  `receiveSingleEvent`):**
  1.  **Deserialize Event:** The Pub/Sub message `Data` is expected to be a
      JSON payload describing a GCS object event (specifically, `{"bucket":
"...", "name": "..."}`).
  2.  **Filename Construction:** A `gs://` URI is constructed from the bucket
      and name.
  3.  **Filename Filtering:** The `filter.Filter` (configured with regexes) is
      applied. If the filename is rejected, the message is acked (as there's
      no point retrying), and processing stops for this event.
  4.  **Source Prefix Check:** The filename is checked against the `Sources`
      list in `instanceConfig.IngestionConfig.SourceConfig.Sources`. These are
      typically `gs://` prefixes. If the filename doesn't match any of these
      prefixes, it's considered an unexpected file, the message is acked, and
      processing stops. This ensures that the ingester only processes files
      from explicitly configured GCS locations.
  5.  **Fetch GCS Object Attributes:** `obj.Attrs(ctx)` is called to get
      metadata like the creation time. If this fails (e.g., object deleted
      between notification and processing, or transient network error), the
      message is nacked (if dead-lettering is not enabled) or handled by the
      dead-letter policy, as retrying might succeed.
  6.  **Stream GCS Object Contents:** `obj.NewReader(ctx)` is called to get an
      `io.ReadCloser` for the file's content. If this fails, the message is
      nacked (or dead-lettered).
  7.  **Send `file.File`:** A `file.File` struct is created with the GCS path,
      the reader, the `attrs.Created` time, and the original `pubsub.Message`.
      This `file.File` is sent to the `fileChannel`.
  8.  **Message Acknowledgement:**
      - The `receiveSingleEvent` function returns `true` if the initial
        stages of processing (up to sending to the channel) were successful
        and the message should be acked from Pub/Sub's perspective (meaning
        it was valid, filtered appropriately, and the object was
        accessible). It returns `false` for transient errors where a retry
        might help (e.g., failing to get object attributes or reader).
      - The `receiveSingleEventWrapper` then uses this boolean:
      - If dead-lettering is enabled (`s.deadLetterEnabled`):
        - If `receiveSingleEvent` returned `false` (transient error or
          should retry), the message is `Nack()`-ed. This typically sends
          it to a dead-letter topic if configured, or allows Pub/Sub to
          redeliver it after a backoff.
        - If `receiveSingleEvent` returned `true`, the message is _not_
          explicitly `Ack()`-ed here. The acknowledgement is deferred to
          the consumer of the `file.File` (i.e., the ingester). This is a
          critical design choice: the message is only truly "done" when
          the file content has been fully processed by the downstream
          system.
      - If dead-lettering is _not_ enabled:
        - If `receiveSingleEvent` returned `true`, the message is
          `Ack()`-ed.
        - If `receiveSingleEvent` returned `false`, the message is
          `Nack()`-ed.
- **Key Design Choices:**
  - **Decoupling from Pub/Sub Ack/Nack:** The `gcssource` itself doesn't
    always immediately `Ack` messages upon successful GCS interaction.
    Instead, it passes the `*pubsub.Message` along in the `file.File`
    struct. This allows the ultimate consumer of the file's content (e.g.,
    the Perf ingestion pipeline) to `Ack` the message only after it has
    successfully processed and stored the data. This provides end-to-end
    processing guarantees. If processing fails downstream, the message can
    be `Nack`-ed, leading to a retry or dead-lettering.
  - **Filtering:** Multiple layers of filtering (regex-based `filter.Filter`
    and prefix-based `SourceConfig.Sources`) ensure that only desired files
    are processed.
  - **Error Handling:** Distinguishes between errors that warrant an `Ack`
    (e.g., file explicitly filtered out) and those that warrant a `Nack`
    (e.g., transient GCS errors), especially when dead-letter queues are in
    use.
  - **Scalability:** Uses a configurable number of parallel receivers
    (`maxParallelReceives`) for Pub/Sub messages, although currently set
    to 1. This can be tuned for performance.
- **Workflow (Simplified):** `New(config) -> GCSSource instance (GCS/PubSub
clients, filter initialized) | V GCSSource.Start() --> Goroutine starts
PubSub subscription.Receive loop | V PubSub message arrives | V
receiveSingleEventWrapper(msg) | V receiveSingleEvent(msg) | +-> Deserialize
msg data (JSON: bucket, name) -> Error? Ack, return. | +-> Filter filename
(regex) -> Rejected? Ack, return. | +-> Check if filename matches
config.Sources prefixes -> No match? Ack, return. | +-> GCS:
storageClient.Object(bucket, name).Attrs() -> Error? Nack (retryable),
return. | +-> GCS: object.NewReader() -> Error? Nack (retryable), return. |
V Create file.File{Name, Contents, Created, PubSubMsg: msg} Send file.File
to fileChannel | V Caller receives file.File from channel (Caller later
Acks/Nacks msg via file.File.PubSubMsg)`

This modular approach to file sourcing makes the Perf ingestion system flexible
and easier to test and maintain. New file sources can be added by simply
implementing the `file.Source` interface.

# Module: /go/filestore

The `filestore` module provides an abstraction layer for interacting with
different file storage systems. It defines a common interface, leveraging Go's
`io/fs.FS`, allowing the application to read files regardless of whether they
are stored locally or in a cloud storage service like Google Cloud Storage
(GCS). This design promotes flexibility and testability by decoupling file
access logic from the specific storage implementation.

The primary goal is to enable Perf, the performance monitoring system, to
seamlessly access data files from various sources. Perf often deals with large
datasets and trace files, which might be stored in GCS for scalability and
durability or locally during development and testing. By using this module, Perf
components can be written to consume data using the standard `fs.FS` interface
without needing to know the underlying storage details.

Key components:

- **`local`**: This submodule provides an implementation of `fs.FS` for the
  local file system.

  - **Why**: It's essential for local development, testing, and scenarios
    where data is directly available on the machine running Perf.
  - **How**: The `local.New(rootDir string)` function initializes a
    `filesystem` struct. This struct stores the absolute path to a `rootDir`
    and uses `os.DirFS(rootPath)` to create an `fs.FS` instance scoped to
    that directory. When `Open(name string)` is called, it calculates the
    path relative to `rootDir` and then uses the underlying `os.DirFS` to
    open the file. This ensures that file access is contained within the
    specified root directory.
  - The `local.go` file contains the `filesystem` struct and its methods.
    The core logic resides in the `New` function for initialization and the
    `Open` method for file access. `filepath.Abs` and `filepath.Rel` are
    used to correctly handle and relativize paths.

- **`gcs`**: This submodule implements `fs.FS` for Google Cloud Storage.

  - **Why**: GCS is a common choice for storing large amounts of data in a
    scalable and accessible manner. Perf relies on GCS for storing trace
    files and other performance artifacts.
  - **How**: The `gcs.New(ctx context.Context)` function initializes a
    `filesystem` struct. It authenticates with GCS using
    `google.DefaultTokenSource` to obtain an OAuth2 token source and then
    creates a `*storage.Client`. The `Open(name string)` method expects a
    GCS URI (e.g., `gs://bucket-name/path/to/file`). It parses this URI into
    a bucket name and object path using `parseNameIntoBucketAndPath`. Then,
    it uses the `storage.Client` to get a `*storage.Reader` for the
    specified object. This reader is wrapped in a custom `file` struct which
    implements `fs.File`.
  - The `gcs.go` file defines the `filesystem` struct, which holds the
    `*storage.Client`, and the `file` struct, which wraps `*storage.Reader`.
    The `New` function handles GCS client initialization and authentication.
    The `Open` method is responsible for parsing GCS URIs and obtaining a
    reader for the object. Notably, the `Stat()` method for `gcs.file` is
    intentionally not implemented (returns `ErrNotImplemented`) because
    Perf's current usage patterns do not require it, simplifying the
    implementation. The `parseNameIntoBucketAndPath` helper function is
    crucial for translating the GCS URI format into the bucket and object
    path components required by the GCS client library.

**Workflow: Opening a File (Conceptual)**

The client code (e.g., a component within Perf) would typically decide which
filestore implementation to use based on configuration or the nature of the file
path.

1.  **Initialization**:

    - For local files: `fsImpl, err := local.New("/path/to/data/root")`
    - For GCS files: `fsImpl, err := gcs.New(context.Background())`

2.  **File Access**:

        - The client calls `file, err :=

    fsImpl.Open("relative/path/to/file.json")`(for local) or`file, err :=
    fsImpl.Open("gs://my-bucket/data/some_trace.json")` (for GCS).

3.  **Behind the Scenes**:

        - **Local**: `local.Open("relative/path/to/file.json") | V Calculates

    absolute path based on rootDir | V Calls
    os.DirFS(rootDir).Open("relative/path/to/file.json") | V Returns fs.File
    (os.File)`    - **GCS**:`gcs.Open("gs://my-bucket/data/some_trace.json") | V
    parseNameIntoBucketAndPath("gs://my-bucket/data/some_trace.json") -->
    "my-bucket", "data/some_trace.json" | V
    gcsClient.Bucket("my-bucket").Object("data/some_trace.json").NewReader()
    | V Wraps storage.Reader in gcs.file | V Returns fs.File (gcs.file)`

4.  **Reading Data**:

    - The client can then use the returned `fs.File` (e.g.,
      `file.Read(buffer)`) in a standard way, irrespective of whether it's an
      `os.File` or a `gcs.file` wrapping a `storage.Reader`.

This abstraction allows Perf to be agnostic to the underlying storage mechanism
when reading files, simplifying its data processing pipelines.

# Module: /go/frontend

The `frontend` module serves as the backbone for the Perf web UI. It's
responsible for handling HTTP requests, rendering HTML templates, and
interacting with various backend services and data stores to provide a
comprehensive performance analysis platform.

The design philosophy emphasizes a separation of concerns. The core
`frontend.go` file initializes and wires together various components, while the
`api` subdirectory houses specific handlers for different categories of user
interactions (e.g., alerts, graphs, regressions). This modular approach
simplifies development, testing, and maintenance.

**Key Components and Responsibilities:**

- **`frontend.go`**:

  - **Initialization (`New`, `initialize`)**: This is the entry point. It
    sets up logging, metrics, reads configuration (`config.Config`),
    initializes database connections (TraceStore, AlertStore,
    RegressionStore, etc.), and establishes connections to external services
    like Git and potentially Chrome Perf.
  - **Template Handling (`loadTemplates`, `templateHandler`)**: It loads
    HTML templates from the `dist` directory (produced by the build system).
    These templates are Go templates, allowing for dynamic data injection.
    Snippets for Google Analytics (`googleanalytics.html`) and cookie
    consent (`cookieconsent.html`) are embedded and can be included in the
    rendered pages.
  - **Page Context (`getPageContext`)**: This crucial function generates a
    JavaScript object (`window.perf`) that is embedded in every HTML page.
    This object contains configuration values and settings that the
    client-side JavaScript needs to function correctly, such as API URLs,
    display preferences, and feature flags. This avoids hardcoding such
    values in the JavaScript and allows for easier configuration.
  - **Routing (`GetHandler`, `getFrontendApis`)**: It defines the HTTP
    routes and associates them with their respective handler functions. This
    is where `chi` router is configured. It also instantiates and registers
    all the API handlers from the `api` sub-module.
  - **Authentication and Authorization (`loginProvider`,
    `RoleEnforcedHandler`)**: It integrates with an authentication system
    (e.g., `proxylogin`) to determine user identity and roles.
    `RoleEnforcedHandler` is a middleware to protect certain endpoints based
    on user roles.
  - **Long-Running Task Management (`progressTracker`)**: For operations
    that might take a significant amount of time (e.g., generating complex
    data frames for graphs, running regression detection), it uses a
    `progress.Tracker`. This allows the frontend to initiate a task, return
    an ID to the client, and let the client poll for status and results,
    preventing HTTP timeouts for long operations.
  - _Workflow Example (Frame Request):_
    1.  Client POSTs to `/_/frame/start` with query details.
    2.  `frameStartHandler` creates a `progress` object, adds it to
        `progressTracker`.
    3.  A goroutine is launched to process the frame request using
        `frame.ProcessFrameRequest`.
    4.  `frameStartHandler` immediately returns the `progress` object's ID.
    5.  Client polls `/_/status/{id}`.
    6.  Client fetches results from `/_/frame/results/{id}` (managed by
        `progressTracker`) once finished.
  - **Redirections (`gotoHandler`, old URL handlers)**: Handles redirects
    for old URLs to new ones and provides a `/g/` endpoint to navigate to
    specific views based on a Git hash.
  - **Liveness Probe (`liveness`)**: Provides a `/liveness` endpoint that
    checks the health of critical dependencies (like the database
    connection) for Kubernetes.

- **`api` (subdirectory)**: This directory contains the specific HTTP handlers
  for various features of Perf. Each API is typically encapsulated in its own
  file (e.g., `alertsApi.go`, `graphApi.go`) and implements the `FrontendApi`
  interface, primarily its `RegisterHandlers` method. This design promotes
  modularity.

  - **`alertsApi.go`**: Manages CRUD operations for alert configurations
    (`alerts.Alert`). It interacts with `alerts.ConfigProvider` (for
    fetching configurations, potentially cached) and `alerts.Store` (for
    persistence). It also handles trying out bug filing and notification
    sending for alerts. Includes endpoints to list subscriptions and manage
    dry-run requests for alert configurations.
  - **`anomaliesApi.go`**: Provides endpoints for fetching anomaly data. It
    has two modes of operation:
  - **Legacy (Chromeperf-backed)**: Proxies requests to an external
    Chromeperf instance for sheriff lists, anomaly lists, and group reports.
    This was likely an initial integration or for instances that rely on
    Chromeperf's anomaly detection. The test name cleaning logic
    (`cleanTestName`) addresses potential incompatibilities in test naming
    conventions or characters between systems.
  - **Skia-internal**: Fetches sheriff (subscription) lists and associated
    alerts directly from the instance's own database (`subscription.Store`,
    `alerts.Store`). This allows Perf instances to manage their own anomaly
    data.
  - **`favoritesApi.go`**: Manages user-specific and instance-wide favorite
    links. User favorites are stored in `favorites.Store`, while
    instance-wide favorites can be defined in the main configuration file
    (`config.Config.Favorites`). It provides endpoints to list, create,
    delete, and update favorites.
  - **`graphApi.go`**: Handles requests related to plotting graphs.
  - **Frame Requests (`frameStartHandler`)**: As described above, this
    initiates the potentially long process of fetching trace data and
    constructing a `dataframe.DataFrame`. It uses
    `dfbuilder.DataFrameBuilder` for this.
  - **Commit Information (`cidHandler`, `cidRangeHandler`,
    `shiftHandler`)**: Provides details about specific commits or ranges of
    commits by interacting with `perfgit.Git`.
  - **Trace Details (`detailsHandler`, `linksHandler`)**: Fetches raw data
    or metadata for a specific trace point at a particular commit. This
    involves reading from `tracestore.TraceStore` and potentially the
    `ingestedFS` (filesystem where raw ingested data is stored) to get
    information like associated benchmark links from the original JSON
    files.
  - **`pinpointApi.go`**: Facilitates interaction with the Pinpoint
    bisection service. It allows users to create bisection jobs (to identify
    the commit that caused a performance regression) or try jobs (to test a
    patch). It can proxy requests to a legacy Pinpoint service or a newer
    backend service.
  - **`queryApi.go`**: Supports the query construction UI.
  - **Parameter Set (`initpageHandler`, `getParamSet`)**: Provides the
    initial set of queryable parameters (keys and their possible values) to
    populate the UI. This uses `psrefresh.ParamSetRefresher` which
    periodically updates this canonical paramset based on recent data,
    ensuring the UI reflects available data.
  - **Query Preflighting/Counting (`countHandler`,
    `nextParamListHandler`)**: As the user builds a query in the UI, these
    handlers can estimate the number of matching traces or provide the next
    relevant parameter values based on the current partial query. This gives
    users immediate feedback. The `nextParamListHandler` is tailored for UIs
    where parameter selection is ordered (e.g., Chromeperf's UI).
  - **`regressionsApi.go`**: Deals with detected regressions.
  - **Listing/Counting Regressions (`regressionRangeHandler`,
    `regressionCountHandler`, `alertsHandler`, `regressionsHandler`)**:
    Fetches regression data from `regression.Store` based on time ranges,
    alert configurations, or subscriptions. It can filter by user ownership
    or category.
  - **Triage (`triageHandler`)**: Allows users (editors) to mark regressions
    as triaged (e.g., "positive", "negative", "ignored") and associate them
    with bug reports. If a regression is marked as negative, it can generate
    a bug report URL using a configurable template.
  - **Manual Clustering (`clusterStartHandler`)**: Allows users to initiate
    the regression detection process for a specific query or set of
    parameters. This is also a long-running operation managed by
    `progressTracker`.
  - **Anomaly/Group Redirection (`anomalyHandler`,
    `alertGroupQueryHandler`)**: Provides redirect URLs to the appropriate
    graph view for a given anomaly ID or alert group ID from Chromeperf.
    This involves generating graph shortcuts.
  - **`sheriffConfigApi.go`**: Handles interactions related to LUCI Config
    for sheriff configurations.
  - **Metadata (`getMetadataHandler`)**: Provides metadata to LUCI Config,
    indicating which configuration files (e.g., `skia-sheriff-configs.cfg`)
    Perf owns and the URL for validating changes to these files. This is
    part of an automated config management system.
  - **Validation (`validateConfigHandler`)**: Receives configuration content
    from LUCI Config and validates it (e.g., using
    `sheriffconfig.ValidateContent`). Returns success or a structured error
    message.
  - **`shortcutsApi.go`**: Manages the creation and retrieval of shortcuts.
  - **Key Shortcuts (`keysHandler`)**: Allows storing a set of trace keys
    (queries) and getting a short ID for them. This is used, for example, by
    the "Share" button on the explore page.
  - **Graph Shortcuts (`getGraphsShortcutHandler`,
    `createGraphsShortcutHandler`)**: Manages shortcuts for more complex
    graph configurations, which can include multiple queries and formulas.
    These are used for sharing multi-graph views.
  - **`triageApi.go`**: Provides endpoints for triaging anomalies,
    specifically those originating from or managed by Chromeperf. This
    includes filing new bugs, associating anomalies with existing bugs, and
    performing actions like ignoring or nudging anomalies. It interacts with
    `chromeperf.ChromePerfClient` and potentially an
    `issuetracker.IssueTracker` implementation.
  - **`userIssueApi.go`**: Manages user-reported issues (Buganizer
    annotations) associated with specific data points (a trace at a commit).
    This allows users to link external bug reports directly to performance
    data points in the UI. It uses `userissue.Store` for persistence.

The overall goal of the `frontend` module is to provide a responsive and
informative user interface by efficiently querying and presenting performance
data, while also enabling users to configure alerts, triage regressions, and
collaborate on performance analysis. The interaction with various stores and
services is abstracted to keep the request handling logic focused.

# Module: /go/git

The `go/git` module provides an abstraction layer for interacting with Git
repositories. It is designed to efficiently retrieve and cache commit
information, which is essential for performance analysis in Skia Perf. The
primary goal is to offer a consistent interface for accessing commit data,
regardless of whether the underlying data source is a local Git checkout or a
remote Gitiles API.

**Design Decisions and Implementation Choices:**

- **Database Caching:** To avoid repeated and potentially slow Git operations,
  commit information is cached in an SQL database. This allows for quick
  lookups of commit details, commit numbers, and commit ranges. The schema for
  this database is defined in `/go/git/schema/schema.go`.
- **Provider Abstraction:** The module utilizes a `provider.Provider`
  interface (defined in `/go/git/provider/provider.go`). This allows for
  different implementations of how Git data is fetched. Currently, two
  providers are implemented:
  - `git_checkout`: Interacts with a local Git repository by shelling out to
    `git` commands. This is suitable for environments where a local checkout
    is available and preferred.
  - `gitiles`: Uses the Gitiles API to fetch commit data. This is useful
    when direct repository access is not feasible or when leveraging
    Google's infrastructure for Git operations. The choice of provider is
    determined by the instance configuration, as seen in
    `/go/git/providers/builder.go`.
- **Commit Numbering:**
  - **Sequential:** By default, the system assigns sequential integer
    `CommitNumber`s to commits as they are ingested. This provides a simple,
    ordered way to refer to commits.
  - **Repo-Supplied:** The system can also be configured to extract commit
    numbers directly from commit messages using a regular expression
    (specified in `instanceConfig.GitRepoConfig.CommitNumberRegex`). This is
    useful for repositories like Chromium that embed a commit position in
    their messages. The `repoSuppliedCommitNumber` flag in `impl.go`
    controls this behavior.
- **LRU Cache:** In addition to the database cache, an in-memory LRU (Least
  Recently Used) cache (`cache` in `impl.go`) is used for frequently accessed
  commit details (`CommitFromCommitNumber`). This further speeds up lookups
  for commonly requested commits. The size of this cache is defined by
  `commitCacheSize`.
- **Background Polling:** The `StartBackgroundPolling` method in `impl.go`
  initiates a goroutine that periodically calls the `Update` method. This
  ensures that the local database cache stays synchronized with the remote
  repository.
- **SQL Statements:** All SQL queries are predefined as constants in
  `impl.go`. This helps in organizing and managing the queries. Separate
  statements are defined for different SQL dialects if needed (e.g., `insert`
  vs `insertSpanner`).
- **Error Handling:** The `BadCommit` constant provides a sentinel value for
  functions returning `provider.Commit` to indicate an error or an invalid
  commit.

**Key Responsibilities and Components:**

- **`interface.go` (Git Interface):**
  - Defines the `Git` interface, which is the public contract for this
    module. It specifies all the operations that can be performed to
    retrieve commit information.
  - This interface decouples the consumers of Git data from the specific
    implementation details (e.g., whether data comes from a local repo or
    Gitiles).
- **`impl.go` (Git Implementation):**
  - Contains the `Impl` struct, which is the primary implementation of the
    `Git` interface.
  - **Data Synchronization (`Update` method):** This is a crucial method
    responsible for fetching new commits from the configured
    `provider.Provider` and storing them in the SQL database. It determines
    the last known commit and fetches all subsequent commits.
  - If `repoSuppliedCommitNumber` is true, it parses the commit number from
    the commit body using `commitNumberRegex`.
  - It handles potential race conditions where multiple services might try
    to update simultaneously by checking if a commit already exists before
    insertion.
  - **Commit Retrieval Methods:** Implements various methods for fetching
    commit data, such as:
  - `CommitNumberFromGitHash`: Retrieves the sequential `CommitNumber` for a
    given Git hash.
  - `CommitFromCommitNumber`: Retrieves the full `provider.Commit` details
    for a given `CommitNumber`. Uses the LRU cache.
  - `CommitNumberFromTime`: Finds the `CommitNumber` closest to (but not
    after) a given timestamp.
  - `CommitSliceFromTimeRange`, `CommitSliceFromCommitNumberRange`: Fetches
    slices of commits based on time or commit number ranges.
  - `GitHashFromCommitNumber`: Retrieves the Git hash for a given
    `CommitNumber`.
  - `PreviousGitHashFromCommitNumber`,
    `PreviousCommitNumberFromCommitNumber`: Finds the Git hash or commit
    number of the commit immediately preceding a given commit number.
  - `CommitNumbersWhenFileChangesInCommitNumberRange`: Identifies commit
    numbers within a range where a specific file was modified. This involves
    converting commit numbers to hashes and then querying the
    `provider.Provider`.
  - **URL Generation (`urlFromParts`):** Constructs a URL to view a specific
    commit, respecting configurations like `DebouceCommitURL` or custom
    `CommitURL` formats.
  - **Metrics:** Collects various metrics (e.g., `updateCalled`,
    `commitNumberFromGitHashCalled`) to monitor the usage and performance of
    different operations.
- **`provider/provider.go` (Provider Interface and Commit Struct):**
  - Defines the `provider.Provider` interface, which abstracts the source of
    Git commit data. Implementations of this interface (like `git_checkout`
    and `gitiles`) handle the actual fetching of data.
  - Defines the `provider.Commit` struct, which is the standard
    representation of a commit used throughout the `go/git` module and its
    providers. It includes fields like `GitHash`, `Timestamp`, `Author`,
    `Subject`, and `Body`. The `Body` is particularly important when
    `repoSuppliedCommitNumber` is true, as it's parsed to extract the commit
    number.
- **`providers/builder.go` (Provider Factory):**
  - Contains the `New` function, which acts as a factory for creating
    `provider.Provider` instances based on the
    `instanceConfig.GitRepoConfig.Provider` setting. This allows the system
    to dynamically choose between `git_checkout` or `gitiles` (or
    potentially other future providers).
- **`providers/git_checkout/git_checkout.go` (CLI Git Provider):**
  - Implements `provider.Provider` by executing `git` command-line
    operations.
  - Handles cloning the repository if it doesn't exist.
  - Manages Git authentication (e.g., via Gerrit) if configured.
  - `CommitsFromMostRecentGitHashToHead`: Uses `git rev-list` to get commit
    information.
  - `GitHashesInRangeForFile`: Uses `git log` to find changes to a specific
    file.
  - `parseGitRevLogStream`: A helper function to parse the output of `git
rev-list --pretty`.
- **`providers/gitiles/gitiles.go` (Gitiles Provider):**
  - Implements `provider.Provider` by interacting with a Gitiles API
    endpoint.
  - `CommitsFromMostRecentGitHashToHead`: Uses `gr.LogFnBatch` to fetch
    commits in batches. It handles logic for main branches versus other
    branches and respects the `startCommit`.
  - `GitHashesInRangeForFile`: Uses `gr.Log` with appropriate path
    filtering.
  - `Update` is a no-op for Gitiles as the API always provides the latest
    data.
- **`schema/schema.go` (Database Schema):**
  - Defines the `Commit` struct with SQL annotations, representing the
    structure of the `Commits` table in the database. This table stores the
    cached commit information.
- **`gittest/gittest.go` (Test Utilities):**
  - Provides helper functions (`NewForTest`) for setting up test
    environments. This includes creating a temporary Git repository,
    populating it with commits, and initializing a test database. This is
    crucial for writing reliable unit and integration tests for the `go/git`
    module and its components.
- **`mocks/Git.go` (Mock Implementation):**
  - Provides a mock implementation of the `Git` interface, generated by
    `mockery`. This is used in tests of other modules that depend on
    `go/git`, allowing them to isolate their tests from actual Git
    operations or database interactions.

**Key Workflows:**

1.  **Initial Population / Update:**

    ```
    Application -> Impl.Update()
        |
        '-> Provider.Update()  (e.g., git pull for git_checkout)
        |
        '-> Impl.getMostRecentCommit() (from local DB)
        |
        '-> Provider.CommitsFromMostRecentGitHashToHead(mostRecentDBHash, ...)
            |
            '-> (For each new commit from Provider)
                |
                '-> [If repoSuppliedCommitNumber] Impl.getCommitNumberFromCommit(commit.Body)
                |
                '-> Impl.CommitNumberFromGitHash(commit.GitHash) (Check if already exists)
                |
                '-> DB.Exec(INSERT INTO Commits ...)
    ```

2.  **Fetching Commit Details by CommitNumber:**

    ```
    Application -> Impl.CommitFromCommitNumber(commitNum)
        |
        '-> Check LRU Cache (cache.Get(commitNum))
        |   |
        |   '-> [If found] Return cached provider.Commit
        |
        '-> [If not in LRU] DB.QueryRow(SELECT ... FROM Commits WHERE commit_number=$1)
            |
            '-> Construct provider.Commit
            |
            '-> Add to LRU Cache (cache.Add(commitNum, commit))
            |
            '-> Return provider.Commit
    ```

3.  **Finding Commits Where a File Changed:** `Application ->
Impl.CommitNumbersWhenFileChangesInCommitNumberRange(beginNum, endNum, file)
| '-> Impl.PreviousGitHashFromCommitNumber(beginNum) -> beginHash (or
Impl.GitHashFromCommitNumber if beginNum is 0 and start commit is used) |
'-> Impl.GitHashFromCommitNumber(endNum) -> endHash | '->
Provider.GitHashesInRangeForFile(beginHash, endHash, file) ->
changedGitHashes[] | '-> (For each changedGitHash) | '->
Impl.CommitNumberFromGitHash(changedGitHash) -> commitNum | '-> Add
commitNum to result list | '-> Return result list`

This structure allows Perf to efficiently query and manage Git commit
information, supporting its core functionality of tracking performance data
across different versions of the codebase.

# Module: /go/graphsshortcut

The `graphsshortcut` module provides a mechanism for storing and retrieving
shortcuts for graph configurations in Perf. Users often define complex sets of
graphs for analysis. Instead of redefining these configurations each time or
relying on cumbersome URL sharing, this module allows users to save a collection
of graph configurations and access them via a unique, shorter identifier. This
significantly improves usability and sharing of common graph views.

The core idea is to represent a set of graphs, each with its own configuration
(queries, formulas, keys), as a `GraphsShortcut` object. This object can then be
persisted and retrieved using a `Store` interface. A key design decision is the
generation of a unique ID for each `GraphsShortcut`. This ID is a hash (MD5) of
the content of the shortcut, ensuring that identical graph configurations will
always have the same ID. This also provides a form of de-duplication. To ensure
consistent ID generation, the queries and formulas within each graph
configuration are sorted alphabetically before hashing. However, the order of
the `GraphConfig` objects within a `GraphsShortcut` _does_ affect the generated
ID.

```
User defines graph configurations --> [GraphsShortcut object] -- InsertShortcut --> [Store] --> Generates ID (MD5 hash) --> Persists (ID, Shortcut)
                                                                     ^
                                                                     |
User provides ID -------------------> [Store] -- GetShortcut --------+------> [GraphsShortcut object] --> Display Graphs
```

### Key Components:

- **`graphsshortcut.go`**: This file defines the central data structures and
  the `Store` interface.

  - `GraphConfig`: Represents the configuration for a single graph. It
    contains:
  - `Queries`: A slice of strings, where each string represents a query used
    to fetch data for the graph.
  - `Formulas`: A slice of strings, representing any formulas applied to the
    data.
  - `Keys`: A string, likely representing a pre-selected set of traces or
    keys to focus on.
  - `GraphsShortcut`: This is the primary object that is stored and
    retrieved. It's essentially a list of `GraphConfig` objects.
  - `GetID()`: A method on `GraphsShortcut` that calculates a unique MD5
    hash based on its content. This method is crucial for identifying and
    de-duplicating shortcuts. It sorts queries and formulas within each
    `GraphConfig` before hashing to ensure that the order of these internal
    elements doesn't change the ID.
  - `Store`: An interface defining the contract for persisting and
    retrieving `GraphsShortcut` objects. It has two methods:
  - `InsertShortcut`: Takes a `GraphsShortcut` and stores it, returning its
    generated ID.
  - `GetShortcut`: Takes an ID and returns the corresponding
    `GraphsShortcut`.

- **`graphsshortcutstore/`**: This subdirectory contains implementations of
  the `graphsshortcut.Store` interface.

  - **`graphsshortcutstore.go` (`GraphsShortcutStore`)**: This provides an
    SQL-backed implementation of the `Store`.
  - **Why SQL?**: SQL databases offer robust, persistent storage suitable
    for production environments where data integrity and concurrent access
    are important.
  - **How it works**:
    - It uses a connection pool (`sql.Pool`) to manage database
      connections.
    - `InsertShortcut`: Marshals the `GraphsShortcut` object into JSON and
      stores it as a string in the `GraphsShortcuts` table along with its
      pre-computed ID. It uses `ON CONFLICT (id) DO NOTHING` to avoid
      errors if the same shortcut (and thus same ID) is inserted multiple
      times.
    - `GetShortcut`: Retrieves the JSON string from the database based on
      the ID and unmarshals it back into a `GraphsShortcut` object.
  - **`cachegraphsshortcutstore.go` (`cacheGraphsShortcutStore`)**: This
    provides an in-memory cache-backed implementation of the `Store`.
  - **Why a cache implementation?**: This is primarily useful for local
    development or testing scenarios, especially when connecting to a
    production database. It allows developers to use features that rely on
    graph shortcuts (like multigraph) without needing write access (or
    breakglass permissions) to the production database. The shortcuts are
    stored locally and ephemerally.
  - **How it works**:
    - It utilizes a generic `cache.Cache` client.
    - `InsertShortcut`: Marshals the `GraphsShortcut` to JSON and stores
      it in the cache using the shortcut's ID as the cache key.
    - `GetShortcut`: Retrieves the JSON string from the cache by ID and
      unmarshals it.
  - **`schema/schema.go`**: Defines the SQL table schema for
    `GraphsShortcuts`. The table primarily stores the `id` (TEXT, PRIMARY
    KEY) and the `graphs` (TEXT, storing the JSON representation of the
    `GraphsShortcut`).

- **`graphsshortcuttest/graphsshortcuttest.go`**: This file provides a suite
  of common tests that can be run against any implementation of the
  `graphsshortcut.Store` interface.

  - **Why shared tests?**: This promotes consistency and ensures that all
    store implementations adhere to the same contract. It makes it easier to
    add new store implementations and verify their correctness.
  - **Key Tests**:
  - `InsertGet`: Verifies that a shortcut can be inserted and then
    retrieved, and that the retrieved shortcut is identical to the original
    (accounting for sorted queries/formulas).
  - `GetNonExistent`: Ensures that attempting to retrieve a shortcut with an
    unknown ID results in an error.

- **`mocks/Store.go`**: This file contains a mock implementation of the
  `graphsshortcut.Store` interface, generated by the `testify/mock` library.

  - **Why mocks?**: Mocks are essential for unit testing components that
    depend on the `Store` interface without needing a real database or
    cache. They allow for controlled testing of different scenarios, such as
    simulating errors from the store.

In summary, the `graphsshortcut` module provides a flexible way to save and
share complex graph views by defining a clear data structure (`GraphsShortcut`),
a standardized way to identify them (`GetID`), and an interface (`Store`) for
various persistence mechanisms, with current implementations for SQL databases
and in-memory caches.

# Module: /go/ingest

The `/go/ingest` module is responsible for the entire process of taking
performance data files, parsing them, and storing the data into a trace store.
This involves identifying the format of the input file, extracting relevant
measurements and metadata, associating them with specific commits, and then
writing this information to the configured data storage backend.

A key design principle is to support multiple ingestion file formats and to be
resilient to errors in individual files. The system attempts to parse files in a
specific order, falling back to legacy formats if the primary parsing fails.
This allows for graceful upgrades of the ingestion format over time without
breaking existing data producers.

The ingestion process also handles trybot data, extracting issue and patchset
information, which is crucial for pre-submit performance analysis.

## Key Components and Files

### `/go/ingest/filter/filter.go`

This component provides a mechanism to selectively process or ignore input files
based on their names using regular expressions.

**Why:** In many scenarios, not all files in a data source are relevant for
performance analysis. For example, temporary files, logs, or files matching
specific patterns might need to be excluded. This filter allows for fine-grained
control over which files are ingested.

**How:**

- It uses two regular expressions: `accept` and `reject`.
- An `accept` regex, if provided, means only filenames matching this regex
  will be considered for processing. If empty, all files are initially
  accepted.
- A `reject` regex, if provided, means any filename matching this regex will
  be ignored, even if it matched the `accept` regex. If empty, no files are
  rejected based on this rule.
- The `Reject(name string) bool` method implements this logic: a file is
  rejected if it _doesn't_ match the `accept` regex (if one is provided) OR if
  it _does_ match the `reject` regex (if one is provided).

**Workflow:**

```
File Name -> Filter.Reject()
              |
              +-- accept_regex_exists? -- Yes -> name_matches_accept? -- No -> REJECT
              |                             |
              |                             +-------------------------- Yes --+
              +----------------------------- No -----------------------------+
                                                                             |
                                                                             V
                                                               reject_regex_exists? -- Yes -> name_matches_reject? -- Yes -> REJECT
                                                                             |                             |
                                                                             |                             +-- No --+
                                                                             +----------------------------- No -----+
                                                                                                                    |
                                                                                                                    V
                                                                                                                  ACCEPT
```

### `/go/ingest/format/format.go` and `/go/ingest/format/legacyformat.go`

These files define the structure of the data files that the ingestion system can
understand. `format.go` defines the current standard format (Version 1), while
`legacyformat.go` defines an older format primarily used by nanobench.

**Why:** A well-defined input format is essential for reliable data ingestion.
Versioning allows the format to evolve while maintaining backward compatibility
or clear error handling for older, unsupported versions. The current format
(`Format` struct) is designed to be flexible, allowing for common metadata (like
git hash, issue/patchset), global key-value pairs applicable to all results, and
a list of individual results. Each result can have its own set of keys and
either a single measurement or a map of "sub-measurements" (e.g., min, max,
median for a single test). This structure allows for rich and varied performance
data to be represented. The legacy format (`BenchData`) exists to support older
systems that still produce data in that schema.

**How:**

- **`format.go` (Version 1):**
  - `Format` struct: The top-level structure. Contains `Version`, `GitHash`,
    optional trybot info (`Issue`, `Patchset`), a global `Key` map, a slice
    of `Result` structs, and global `Links`.
  - `Result` struct: Represents one or more measurements. It has its own
    `Key` map (which gets merged with the global `Key`), and critically,
    either a single `Measurement` (float32) or a `Measurements` map.
  - `SingleMeasurement` struct: Used within `Measurements` map. It allows
    associating a `value` (e.g., "min", "median") with a `Measurement`
    (float32) and optional `Links`. This is how multiple metrics for a
    single conceptual test run are represented.
  - `Parse(r io.Reader)`: Decodes JSON data from a reader into a `Format`
    struct. It specifically checks `fileFormat.Version ==
FileFormatVersion`.
  - `Validate(r io.Reader)`: Uses a JSON schema (`formatSchema.json`) to
    validate the structure of the input data. This ensures that incoming
    files adhere to the expected contract, preventing malformed data from
    causing issues downstream.
  - `GetLinksForMeasurement(traceID string)`: Retrieves links associated
    with a specific measurement, combining global links with
    measurement-specific ones.
- **`legacyformat.go`:**
  - `BenchData` struct: Defines the older nanobench format. It has fields
    like `Hash`, `Issue`, `PatchSet`, `Key`, `Options`, and `Results`. The
    `Results` are nested maps leading to `BenchResult`.
  - `BenchResult`: A map representing individual test results, typically
    `map[string]interface{}` where values are float64s, except for an
    "options" key.
  - `ParseLegacyFormat(r io.Reader)`: Decodes JSON data into a `BenchData`
    struct.

The system will first attempt to parse an input file using `format.Parse`. If
that fails (e.g., due to a version mismatch or JSON parsing error), it may then
attempt to parse it using `format.ParseLegacyFormat` as a fallback.

### `/go/ingest/format/formatSchema.json`

This file contains the JSON schema definition for the `Format` struct defined in
`format.go`.

**Why:** A JSON schema provides a formal, machine-readable definition of the
expected data structure. This is used for validation, ensuring that ingested
files conform to the specified format. This helps catch errors early and
provides clear feedback on what is wrong with a non-conforming file.

**How:** It's a standard JSON Schema file. The `format.Validate` function uses
this schema to check the structure and types of the fields in an incoming JSON
file. The schema is embedded into the Go binary.

### `/go/ingest/format/generate/main.go`

This is a utility program used to automatically generate `formatSchema.json`
from the Go `Format` struct definition.

**Why:** Manually keeping a JSON schema synchronized with Go struct definitions
is error-prone. This generator ensures that the schema always accurately
reflects the Go types.

**How:** It uses the `go.skia.org/infra/go/jsonschema` library, which can
reflect on Go structs and produce a corresponding JSON schema. The
`//go:generate` directive in the file allows this program to be run easily
(e.g., via `go generate`).

### `/go/ingest/parser/parser.go`

This is the core component responsible for taking an input file (as
`file.File`), attempting to parse it using the defined formats, and extracting
the performance data into a standardized intermediate representation.

**Why:** This component decouples the specifics of file formats from the process
of writing data to the trace store. It handles the logic of trying different
parsers, extracting common information like Git hashes and trybot details, and
transforming the data into lists of parameter maps (`paramtools.Params`) and
corresponding measurement values (`float32`). It also enforces rules like branch
name filtering and parameter key/value validation.

**How:**

- **`New(...)`**: Initializes a `Parser` with instance-specific
  configurations, such as recognized branch names and a regex for invalid
  characters in parameter keys/values.
- **`Parse(ctx context.Context, file file.File)`**: This is the main entry
  point for processing a regular data file.
  1.  It first attempts to parse the file using `extractFromVersion1File`
      (which uses `format.Parse`).
  2.  If that fails, it falls back to `extractFromLegacyFile` (which uses
      `format.ParseLegacyFormat`).
  3.  It checks if the branch name (if present in the file's common keys) is
      in the allowed list. If not, it returns `ErrFileShouldBeSkipped`.
  4.  It ensures that the extracted parameter keys and values are valid,
      potentially modifying them using `query.ForceValidWithRegex` based on
      the `invalidParamCharRegex` from the instance configuration. This is
      crucial because trace IDs (which are derived from these parameters)
      often have restrictions on allowed characters.
  5.  Returns `params` (a slice of `paramtools.Params`), `values` (a slice of
      `float32`), the `gitHash`, any global `links` from the file, and an
      error.
- **`ParseTryBot(file file.File)`**: A specialized function to extract only
  the `Issue` and `Patchset` information from a file, trying both V1 and
  legacy formats. This is likely used for systems that only need to identify
  the tryjob associated with a file without processing all the measurement
  data.
- **`ParseCommitNumberFromGitHash(gitHash string)`**: Extracts an integer
  commit number from a specially formatted git hash string (e.g., "CP:12345"
  -> 12345). This supports systems that use such commit identifiers.
- Helper functions like `getParamsAndValuesFromLegacyFormat` and
  `getParamsAndValuesFromVersion1Format` do the actual work of traversing the
  parsed file structures (`BenchData` or `Format`) and flattening them into
  the `params` and `values` slices.
  - For the V1 format, it iterates through `f.Results`. If a `Result` has a
    single `Measurement`, it combines `f.Key` and `result.Key` to form the
    `paramtools.Params`.
  - If a `Result` has `Measurements` (a map of `string` to
    `[]SingleMeasurement`), it iterates through this map. For each entry, it
    takes the map's key and the `Value` from `SingleMeasurement` to add more
    key-value pairs to the `paramtools.Params`.
- **`GetSamplesFromLegacyFormat(b *format.BenchData)`**: Extracts raw sample
  data (if present) from the legacy format. This seems to be for specific use
  cases where individual sample values, rather than just aggregated metrics,
  are needed.

**Key Workflow (Simplified `Parse`):**

```
Input: file.File
Output: ([]paramtools.Params, []float32, gitHash, links, error)

1. Read file contents.
2. Attempt Parse as Version 1 Format:
   `f, err := format.Parse(contents)`
   If success:
     `params, values := getParamsAndValuesFromVersion1Format(f, p.invalidParamCharRegex)`
     `gitHash = f.GitHash`
     `links = f.Links`
     `commonKeys = f.Key`
   Else (error):
     Reset reader.
     Attempt Parse as Legacy Format:
       `benchData, err := format.ParseLegacyFormat(contents)`
       If success:
         `params, values := getParamsAndValuesFromLegacyFormat(benchData)`
         `gitHash = benchData.Hash`
         `links = nil` (legacy format doesn't have global links in the same way)
         `commonKeys = benchData.Key`
       Else (error):
         Return error.

3. `branch, ok := p.checkBranchName(commonKeys)`
   If !ok:
     Return `ErrFileShouldBeSkipped`.

4. If len(params) == 0:
   Return `ErrFileShouldBeSkipped`.

5. Return `params, values, gitHash, links, nil`.
```

### `/go/ingest/process/process.go`

This component orchestrates the entire ingestion pipeline. It takes files from a
source (e.g., a directory, GCS bucket), uses the `parser` to extract data,
interacts with `git` to resolve commit information, and then writes the
processed data to a `tracestore.TraceStore` and `tracestore.MetadataStore`. It
also handles sending Pub/Sub events for ingested files.

**Why:** This provides the high-level control flow for ingestion. It manages
concurrency (multiple worker goroutines), error handling at a macro level
(retries for writing to the store), and integration with external systems like
Git and Pub/Sub.

**How:**

- **`Start(...)`**:
  1.  Initializes tracing, Pub/Sub client (if a topic is configured), the
      `file.Source` (to get files), the `tracestore.TraceStore` and
      `tracestore.MetadataStore` (to write data), and `perfgit.Git` (to map
      git hashes to commit numbers).
  2.  Starts a number of `worker` goroutines specified by
      `numParallelIngesters`.
  3.  Each `worker` listens on a channel provided by the `file.Source`.
- **`worker(...)`**:
  1.  Creates a `parser.Parser` instance.
  2.  Enters a loop, receiving `file.File` objects from the channel.
  3.  For each file, it calls `workerInfo.processSingleFile`.
- **`workerInfo.processSingleFile(f file.File)`**: This is the heart of the
  per-file processing.
  1.  Increments metrics for files received.
  2.  Calls `p.Parse(ctx, f)` to get `params`, `values`, `gitHash`, and
      `fileLinks`.
  3.  Handles errors from `Parse`:
      - If `parser.ErrFileShouldBeSkipped`, acks the Pub/Sub message (if
        any) and skips.
      - For other parsing errors, increments metrics and nacks the Pub/Sub
        message (if dead-lettering is enabled, allowing for retries or
        manual inspection).
  4.  If `gitHash` is empty, logs an error and nacks.
  5.  If the Git repo supplies commit numbers directly (e.g. "CP:12345"), it
      calls `p.ParseCommitNumberFromGitHash`.
  6.  Calls `g.GetCommitNumber(ctx, gitHash, commitNumberFromFile)` to resolve
      the `gitHash` (or verify the supplied commit number) against the Git
      repository. It includes logic to update the local Git repository clone
      if the hash isn't initially found. If the commit cannot be resolved, it
      logs an error, acks the Pub/Sub message (as retrying won't help for an
      unknown commit), and skips.
  7.  Builds a `paramtools.ParamSet` from all the extracted `params`.
  8.  Writes the data to the `tracestore.TraceStore` using `store.WriteTraces`
      or `store.WriteTraces2` (depending on
      `instanceConfig.IngestionConfig.TraceValuesTableInlineParams`). This
      involves retries in case of transient store errors.
      - `WriteTraces2` suggests an optimized path where some parameter data
        might be stored directly with trace values, potentially for
        performance reasons.
  9.  If writing fails after retries, increments metrics and nacks.
  10. If writing succeeds, acks the Pub/Sub message and increments success
      metrics.
  11. Calls `sendPubSubEvent` to publish information about the ingested file
      (trace IDs, paramset, filename) to a configured Pub/Sub topic. This
      allows other services to react to new data ingestion.
  12. If `fileLinks` were present in the input, it calls
      `metadataStore.InsertMetadata` to store these links.
- **`sendPubSubEvent(...)`**: If a `FileIngestionTopicName` is configured,
  this function constructs an `ingestevents.IngestEvent` containing the trace
  IDs, the overall `ParamSet` for the file, and the filename. It then
  publishes this event to the specified Pub/Sub topic.

**Overall Ingestion Workflow:**

```
File Source (e.g., GCS bucket watcher)
     |
     v
[ file.File channel ] -> Worker Goroutine(s)
                             |
                             v
                       processSingleFile(file)
                             |
  +--------------------------+--------------------------+
  |                          |                          |
  v                          v                          v
Parser.Parse(file) --> Git.GetCommitNumber(hash) --> TraceStore.WriteTraces(...)
  |      ^                   |                          |   ^
  |      | (if parsing fails)|                          |   | (retries)
  |      +-------------------| (update repo if needed)  |   |
  |                          |                          |   |
  +-----> ParamSet Creation  +--------------------------+   |
  |                                                        |
  v                                                        |
sendPubSubEvent (if success) ------------------------------+
  |
  v
MetadataStore.InsertMetadata (if links exist)
```

This architecture allows for robust and scalable ingestion of performance data
from various sources and formats, with clear separation of concerns between
parsing, data transformation, Git interaction, and storage. The use of Pub/Sub
facilitates downstream processing and real-time reactions to newly ingested
data.

# Module: /go/ingestevents

The `ingestevents` module is designed to facilitate the communication of
ingestion completion events via PubSub. This is a critical part of the
event-driven alerting system within Perf, where the completion of data ingestion
for a file triggers subsequent processes like regression detection in a
clusterer.

The core of this module revolves around the `IngestEvent` struct. This struct
encapsulates the necessary information to be transmitted when a file has been
successfully ingested. It includes:

- `TraceIDs`: A slice of strings representing all the unencoded trace
  identifiers found within the ingested file. These IDs are fundamental for
  identifying the specific data points that have been processed.
- `ParamSet`: An unencoded, read-only representation of the
  `paramtools.ParamSet` that summarizes the `TraceIDs`. This provides a
  consolidated view of the parameters associated with the ingested traces.
- `Filename`: The name of the file that was ingested. This helps in tracking
  the source of the ingested data.

To handle the transmission of `IngestEvent` data over PubSub, the module
provides two key functions:

- `CreatePubSubBody`: This function takes an `IngestEvent` struct as input and
  prepares it for PubSub transmission. The "how" here involves a two-step
  process:

  1.  The `IngestEvent` is first encoded into a JSON format. This provides a
      structured and widely compatible representation of the data.
  2.  The resulting JSON data is then compressed using gzip. The "why" for
      this step is to ensure that the message size stays within the PubSub
      message size limits (currently 10MB). This is particularly important
      when dealing with files that contain a large number of traces, as the
      raw JSON representation could exceed the limit. The function returns the
      gzipped JSON data as a byte slice.

  ```
  IngestEvent (struct) ---> JSON Encoding ---> Gzip Compression ---> []byte (for PubSub)
  ```

- `DecodePubSubBody`: This function performs the reverse operation. It takes a
  byte slice (presumably received from a PubSub message) and decodes it back
  into an `IngestEvent` struct. The process is:

  1.  The input byte slice is first decompressed using gzip.
  2.  The decompressed data, which is expected to be in JSON format, is then
      decoded into an `IngestEvent` struct. Error handling is incorporated at
      each step to manage potential issues during decompression or JSON
      decoding.

  ```
  []byte (from PubSub) ---> Gzip Decompression ---> JSON Decoding ---> IngestEvent (struct)
  ```

The primary responsibility of this module is therefore to provide a standardized
and efficient way to serialize and deserialize ingestion event information for
PubSub communication. The design choice of using JSON for structure and gzip for
compression balances readability, interoperability, and an efficient use of
PubSub resources.

The file `ingestevents.go` contains the definition of the `IngestEvent` struct
and the implementation of the `CreatePubSubBody` and `DecodePubSubBody`
functions. The corresponding test file, `ingestevents_test.go`, ensures that the
encoding and decoding processes work correctly, verifying that an `IngestEvent`
can be successfully round-tripped through the serialization and deserialization
process.

# Module: /go/initdemo

The `initdemo` module provides a command-line application designed to initialize
a database instance, specifically targeting CockroachDB or a Spanner emulator,
for demonstration or development purposes.

Its primary purpose is to automate the creation of a named database and the
application of the latest database schema. This ensures a consistent and
ready-to-use database environment, removing the manual steps often required for
setting up a database for applications like Skia Perf.

The core functionality revolves around connecting to a specified database URL,
attempting to create the database (gracefully handling cases where it already
exists), and then executing the appropriate schema definition. The choice of
schema (standard SQL or Spanner-specific) is determined by a command-line flag.

**Key Components and Responsibilities:**

- **`main.go`**: This is the entry point and sole Go source file for the
  application.
  - **Flag Parsing**: It defines and parses command-line flags to configure
    the database connection and behavior.
  - `--databasename`: Specifies the name of the database to be created
    (defaults to "demo"). This allows users to customize the database name
    for different environments or purposes.
  - `--database_url`: Provides the connection string for the CockroachDB
    instance (defaults to a local instance
    `postgresql://root@127.0.0.1:26257/?sslmode=disable`). This allows
    connection to different database servers or configurations.
  - `--spanner`: A boolean flag that, when set, instructs the application to
    use the Spanner-specific schema. This is crucial for ensuring
    compatibility when targeting a Spanner emulator, which may have
    different SQL syntax or feature support compared to CockroachDB.
  - **Database Connection**: It establishes a connection to the database
    using the `pgxpool` library, which is a PostgreSQL driver and connection
    pool for Go. This library was chosen for its robustness and performance
    in handling PostgreSQL-compatible databases like CockroachDB.
  - **Database Creation**: It attempts to execute a `CREATE DATABASE` SQL
    statement. The implementation includes error handling to log an
    informational message if the database already exists, rather than
    failing, making the script idempotent in terms of database creation.
  - **Database Selection (CockroachDB specific)**: If not targeting Spanner,
    it executes `SET DATABASE` to switch the current session's context to
    the newly created (or existing) database. This is a CockroachDB-specific
    command.
  - **Schema Selection**: Based on the `--spanner` flag, it selects the
    appropriate schema definition.
  - If `--spanner` is false, it uses `sql.Schema` from the `//perf/go/sql`
    module, which contains the standard SQL schema for Perf.
  - If `--spanner` is true, it uses `spanner.Schema` from the
    `//perf/go/sql/spanner` module, which contains the schema adapted for
    Spanner. This separation allows maintaining distinct schema versions
    tailored to the nuances of each database system.
  - **Schema Application**: It executes the selected schema DDL statements
    against the connected database. This step creates all the necessary
    tables, indexes, and other database objects required by the Perf
    application.
  - **Connection Closure**: Finally, it closes the database connection pool
    to release resources.

**Workflow:**

The typical workflow of the `initdemo` application can be visualized as:

1.  **Parse Flags**: `Application Start` -> `Read --databasename,
--database_url, --spanner`

2.  **Connect to Database**: `Use --database_url` -> `pgxpool.Connect()` ->
    `Connection Pool (conn)`

3.  **Create Database**: `conn` + `Use --databasename` -> `Execute "CREATE
DATABASE <name>"` `|` `+-- Success` `|` `+-- Error (e.g., already exists)`
    -> `Log Info "Database <name> already exists."`

4.  **Set Active Database (if not Spanner)**: `Is --spanner false?` `|` `+--
Yes` -> `conn` + `Use --databasename` -> `Execute "SET DATABASE <name>"` `|
|` `| +-- Error` -> `sklog.Fatal()` `|` `+-- No (Spanner enabled)` -> `Skip
this step`

5.  **Select Schema**: `Is --spanner true?` `|` `+-- Yes` -> `dbSchema =
spanner.Schema` `|` `+-- No` -> `dbSchema = sql.Schema`

6.  **Apply Schema**: `conn` + `dbSchema` -> `Execute schema DDL` `|` `+--
Error` -> `sklog.Fatal()`

7.  **Close Connection**: `conn.Close()` -> `Application End`

This process ensures that a target database is either created or confirmed to
exist, and then the correct schema is applied, making it ready for use. The
choice of using `pgxpool` for database interaction and providing separate schema
definitions for standard SQL and Spanner demonstrates a design focused on
supporting multiple database backends for the Perf system. The error handling,
particularly for the database creation step, aims for robust and user-friendly
operation.

# Module: /go/issuetracker

## Perf Issue Tracker Module

This module provides an interface and implementation for interacting with the
Google Issue Tracker API, specifically tailored for Perf's needs. The primary
goal is to abstract the complexities of the Issue Tracker API and provide a
simpler, more focused way to retrieve issue details and add comments to existing
issues. This allows other parts of the Perf system to integrate with issue
tracking without needing to directly handle API authentication, request
formatting, or response parsing.

### Core Functionality and Design

The module is designed around the `IssueTracker` interface, which defines the
core operations:

1.  **Listing Issues (`ListIssues`)**: This function allows retrieving details
    for a set of specified issue IDs.

        - **Why**: Perf often needs to fetch information about bugs that have been
          filed (e.g., to display their status or link to them from alerts).
          Providing a bulk retrieval mechanism based on IDs is efficient.
        - **How**: The implementation takes a `ListIssuesRequest` containing a
          slice of integer issue IDs. It constructs a query string by joining
          these IDs with " | " (OR operator in Issue Tracker query language) and
          prepending "id:()". This formatted query is then sent to the Issue
          Tracker API.
        - **Example Workflow**: `Perf System --- ListIssuesRequest (IDs: [123,

    456]) ---> issuetracker Module | v Construct Query: "id:(123 | 456)" | v
    Issue Tracker API <--- GET Request --- issueTrackerImpl | v Perf System
    <--- []\*issuetracker.Issue --- Response Parsing <--- API Response`

2.  **Creating Comments (`CreateComment`)**: This function allows adding a new
    comment to an existing issue.

        - **Why**: Perf might need to automatically update bugs with new
          information, such as when a regression is fixed or when more data about
          an alert becomes available.
        - **How**: It takes a `CreateCommentRequest` containing the `IssueId` and
          the `Comment` string. The implementation constructs an
          `issuetracker.IssueComment` object and uses the Issue Tracker client
          library to post this comment to the specified issue.
        - **Example Workflow**: `Perf System --- CreateCommentRequest (ID: 789,

    Comment: "...") ---> issuetracker Module | v Issue Tracker API <--- POST
    Request --- issueTrackerImpl | v Perf System <--- CreateCommentResponse
    <--- Response Parsing <--- API Response`

### Key Components

- **`issuetracker.go`**:

  - **`IssueTracker` interface**: Defines the contract for interacting with
    the issue tracker. This allows for decoupling the client code from the
    specific implementation and facilitates testing using mocks.
  - **`issueTrackerImpl` struct**: The concrete implementation of the
    `IssueTracker` interface. It holds an instance of the
    `issuetracker.Service` client, which is the generated Go client for the
    Google Issue Tracker API.
  - **`NewIssueTracker` function**: This is the factory function for
    creating an `issueTrackerImpl` instance.
  - **Authentication**: It handles the authentication by fetching an API key
    from Google Secret Manager. The secret project and name are configurable
    via `config.IssueTrackerConfig`. It then uses `google.DefaultClient`
    with the "https://www.googleapis.com/auth/buganizer" scope to obtain an
    authenticated HTTP client. This client and the API key are then used to
    initialize the `issuetracker.Service`.
  - **Configuration**: The `BasePath` of the `issuetracker.Service` is
    explicitly set to "https://issuetracker.googleapis.com" to ensure it
    points to the correct API endpoint.
  - **Request/Response Structs (`ListIssuesRequest`, `CreateCommentRequest`,
    `CreateCommentResponse`)**: These simple structs define the data
    structures for requests and responses, making the interface clear and
    easy to use. They are designed to be minimal and specific to the needs
    of the Perf system.

- **`mocks/IssueTracker.go`**:

  - This file contains a mock implementation of the `IssueTracker`
    interface, generated using the `testify/mock` library.
  - **Why**: Mocks are crucial for unit testing components that depend on
    the `issuetracker` module. They allow tests to simulate various
    responses (success, failure, specific data) from the issue tracker
    without making actual API calls. This makes tests faster, more reliable,
    and independent of external services.
  - **How**: The `IssueTracker` mock struct embeds `mock.Mock` and provides
    mock implementations for `ListIssues` and `CreateComment`. The
    `NewIssueTracker` function in this file is a constructor for the mock,
    which also sets up test cleanup to assert that all expected mock calls
    were made.

### Design Decisions and Trade-offs

- **Interface-based design**: Using an interface (`IssueTracker`) promotes
  loose coupling and testability. Consumers depend on the abstraction rather
  than the concrete implementation.
- **Simplified API**: The module exposes only the functionality currently
  needed by Perf (listing issues by ID and creating comments). It doesn't
  attempt to be a full-fledged Issue Tracker client, which simplifies its own
  implementation and usage. If more advanced features are needed in the
  future, the interface can be extended.
- **Secret Management for API Key**: Storing the API key in Google Secret
  Manager is a security best practice, preventing it from being hardcoded or
  checked into version control.
- **Error Handling**: The module uses `skerr.Wrapf` to wrap errors, providing
  context and making debugging easier. It also includes input validation for
  `CreateCommentRequest` to prevent invalid API calls.
- **Logging**: Debug logs (`sklog.Debugf`) are included to trace requests and
  responses, which can be helpful during development and troubleshooting.

The module relies on the external `go.skia.org/infra/go/issuetracker/v1`
library, which is the auto-generated client for the Google Issue Tracker API.
This design choice leverages existing, well-tested client libraries instead of
reimplementing API interaction from scratch.

# Module: /go/kmeans

## K-Means Clustering Module

This module provides a generic implementation of the k-means clustering
algorithm. The primary goal is to offer a flexible way to group a set of data
points (observations) into a predefined number of clusters (k) based on their
similarity. The "similarity" is determined by a distance metric, and the
"center" of each cluster is represented by a centroid.

### Design and Implementation Choices

The module is designed with **generality** in mind. Instead of being tied to a
specific data type or distance metric, it uses interfaces (`Clusterable`,
`Centroid`) and a function type (`CalculateCentroid`). This approach allows
users to define their own data structures and distance calculations, making the
k-means algorithm applicable to a wide variety of problems.

**Interfaces for Flexibility:**

- **`Clusterable`**: This is a marker interface. Any data type that needs to
  be clustered must satisfy this interface. In practice, this means you can
  use `interface{}` and then perform type assertions within your custom
  distance and centroid calculation functions. This design choice prioritizes
  ease of use for simple cases, where the same type might represent both an
  observation and a centroid.
- **`Centroid`**: This interface defines the contract for centroids.
  - `AsClusterable() Clusterable`: This method is crucial for situations
    where a centroid itself can be treated as a data point (e.g., when
    calculating distances or when a centroid is part of the initial
    observation set). It allows the algorithm to seamlessly integrate
    centroids into lists of clusterable items. If a centroid cannot be
    meaningfully converted to a `Clusterable`, it returns `nil`.
  - `Distance(c Clusterable) float64`: This method is the core of the
    similarity measure. It calculates the distance between the centroid and
    a given `Clusterable` data point. The user provides the specific
    implementation for this, enabling the use of various distance metrics
    (Euclidean, Manhattan, etc.).
- **`CalculateCentroid func([]Clusterable) Centroid`**: This function type
  defines how a new centroid is computed from a set of `Clusterable` items
  belonging to a cluster. This allows users to implement different strategies
  for centroid calculation, such as taking the mean, median, or other
  representative points.

**Lloyd's Algorithm Implementation:**

The core clustering logic is implemented in the `Do` function, which performs a
single iteration of Lloyd's algorithm. This is a common and relatively
straightforward iterative approach to k-means.

The `KMeans` function orchestrates multiple iterations of `Do`. A key design
consideration here is the **convergence criteria**. Currently, it runs for a
fixed number of iterations (`iters`). A more sophisticated approach, would be to iterate until the total error (or the change in
centroid positions) falls below a certain threshold, indicating that the
clusters have stabilized. This was likely deferred for simplicity in the initial
implementation, but it's an important aspect for practical applications to avoid
unnecessary computations or premature termination.

**Why modify centroids in-place in `Do`?**

The `Do` function modifies the `centroids` slice passed to it. The documentation
explicitly advises calling it as `centroids = Do(observations, centroids, f)`.
This design choice might have been made for efficiency, avoiding the allocation
of a new centroids slice in every iteration if the number of centroids remains
the same. However, it also means the caller needs to be aware of this side
effect. The function does return the potentially new slice of centroids, which
is important because centroids can be "lost" if a cluster becomes empty.

### Key Responsibilities and Components

- **`kmeans.go`**: This is the sole source file and contains all the logic for
  the k-means algorithm.

  - **`Clusterable` (interface)**: Defines the contract for data points that
    can be clustered. Its main purpose is to allow generic collections of
    items.
  - **`Centroid` (interface)**: Defines the contract for cluster centers,
    including how to calculate their distance to data points and how to
    treat them as data points themselves.
  - **`CalculateCentroid` (function type)**: A user-provided function that
    defines the logic for computing a new centroid from a group of data
    points. This separation of concerns is key to the module's flexibility.
  - **`closestCentroid(observation Clusterable, centroids []Centroid) (int,
float64)`**: A helper function that finds the index of the centroid
    closest to a given observation and the distance to it. This is a
    fundamental step in assigning observations to clusters.
  - **`Do(observations []Clusterable, centroids []Centroid, f
CalculateCentroid) []Centroid`**:
  - **Responsibility**: Performs a single iteration of the k-means algorithm
    (Lloyd's algorithm).
  - **How it works**: 1. Assigns each observation to its nearest centroid, forming temporary
    clusters. `Observations --> [Find Closest Centroid for each] -->
Temporary Cluster Assignments` 2. For each temporary cluster, it recalculates a new centroid using the
    user-provided `f` function. `Temporary Cluster Assignments -->
[Group by Cluster] --> Sets of Clusterable items | V [Apply 'f'] -->
New Centroids` 3. If a cluster becomes empty (no observations are closest to its
    centroid), that centroid is effectively removed in this iteration,
    as `f` will not be called for an empty set of `Clusterable` items,
    and `newCentroids` will not include it.
  - **Design Rationale**: Encapsulates one core step of the iterative
    process, making the overall `KMeans` function clearer. The in-place
    modification (and return value) addresses the potential for the number
    of centroids to change.
  - **`GetClusters(observations []Clusterable, centroids []Centroid)
([][]Clusterable, float64)`**:
  - **Responsibility**: Organizes the observations into their final clusters
    based on the provided (presumably converged) centroids and calculates
    the total sum of squared errors (or whatever distance metric is used).
  - **How it works**:
    1.  Initializes a list of clusters, where each cluster initially
        contains only its centroid (if `AsClusterable()` is not nil).
    2.  Iterates through all observations, assigning each to its closest
        centroid and adding it to the corresponding cluster list.
    3.  Accumulates the distance from each observation to its assigned
        centroid to compute the `totalError`.
  - **Design Rationale**: Provides a way to retrieve the actual cluster
    memberships after the algorithm has run. The inclusion of the centroid
    as the first element in each returned cluster is a convention for easy
    identification.
  - **`KMeans(observations []Clusterable, centroids []Centroid, k, iters
int, f CalculateCentroid) ([]Centroid, [][]Clusterable)`**:
  - **Responsibility**: The main entry point for running the k-means
    algorithm for a specified number of iterations.
  - **How it works**: `Initial Centroids --(iter 1)--> Do() --(updates)-->
Centroids' | --(iter 2)--> Do() --(updates)--> Centroids'' ... --(iter
'iters')--> Do() --(updates)--> Final Centroids | V GetClusters() -->
Final Clusters`
  - **Design Rationale**: Provides a simple interface to run the entire
    process. The fixed number of iterations (`iters`) is a straightforward
    stopping condition, though, as mentioned, convergence-based stopping
    would be more robust. The `k` parameter seems redundant given that the
    initial number of centroids is determined by `len(centroids)`. If `k`
    was intended to specify the _desired_ number of clusters and the initial
    `centroids` were just starting points, the implementation would need to
    handle cases where `len(centroids)` != `k`. However, the current `Do`
    function naturally adjusts the number of centroids if some clusters
    become empty.
  - **`TotalError(observations []Clusterable, centroids []Centroid)
float64`**:
  - **Responsibility**: Calculates the sum of distances from each
    observation to its closest centroid. This is often used as a measure of
    the "goodness" of the clustering.
  - **How it works**: It simply calls `GetClusters` and returns the
    `totalError` computed by it.
  - **Design Rationale**: Provides a convenient way to evaluate the
    clustering quality without needing to manually iterate and sum
    distances.

### Key Workflows

**1. Single K-Means Iteration (`Do` function):**

```
Input: Observations (O), Current Centroids (C_curr), CalculateCentroid function (f)

1. For each Observation o in O:
   Find c_closest in C_curr such that Distance(o, c_closest) is minimized.
   Assign o to the cluster associated with c_closest.
   ---> Result: A mapping of each Observation to a Centroid index.

2. Initialize NewCentroids (C_new) as an empty list.

3. For each unique Centroid index j (from 0 to k-1):
   a. Collect all Observations (O_j) assigned to cluster j.
   b. If O_j is not empty:
      Calculate new_centroid_j = f(O_j).
      Add new_centroid_j to C_new.
   ---> Potentially, some original centroids might not have any observations assigned,
        so C_new might have fewer centroids than C_curr.

Output: New Centroids (C_new)
```

**2. Full K-Means Clustering (`KMeans` function):**

```
Input: Observations (O), Initial Centroids (C_init), Number of Iterations (iters), CalculateCentroid function (f)

1. Set CurrentCentroids = C_init.

2. Loop 'iters' times:
   CurrentCentroids = Do(O, CurrentCentroids, f)  // Perform one iteration
   ---> CurrentCentroids are updated.

3. FinalCentroids = CurrentCentroids.

4. Clusters, TotalError = GetClusters(O, FinalCentroids)
   ---> Assigns each observation to its final cluster based on FinalCentroids.
        The first element of each sub-array in Clusters is the centroid itself.

Output: FinalCentroids, Clusters
```

The unit tests in `kmeans_test.go` provide excellent examples of how to
implement the `Clusterable`, `Centroid`, and `CalculateCentroid` requirements
for a simple 2D point scenario. They demonstrate the expected behavior of the
`Do` and `KMeans` functions, including edge cases like empty inputs or losing
centroids when clusters become empty.

# Module: /go/maintenance

## Maintenance Module Documentation

### High-Level Overview

The `maintenance` module in Perf is responsible for executing a set of
long-running background processes that are essential for the health and
operational integrity of a Perf instance. These tasks ensure that data is kept
up-to-date, system configurations are current, and storage is managed
efficiently. The module is designed to be started once and run continuously,
performing its duties at predefined intervals.

### Design Rationale and Implementation Choices

The core design principle behind the `maintenance` module is to centralize
various periodic tasks that would otherwise be scattered or require manual
intervention. By consolidating these operations, the system becomes more robust
and easier to manage.

Key design choices include:

- **Asynchronous Operations:** Most maintenance tasks are designed to run in
  separate goroutines, triggered by timers. This allows the main application
  thread (if any) to remain responsive and prevents one maintenance task from
  blocking others.
- **Configurability via Flags and Instance Configuration:** The behavior of
  the maintenance tasks (e.g., whether to perform regression migration,
  refresh query cache, or delete old data) is controlled by command-line flags
  (`config.MaintenanceFlags`) and the instance-specific configuration
  (`config.InstanceConfig`). This provides flexibility for different Perf
  deployments and operational needs.
- **Dependency Injection:** Components like database connections
  (`builders.NewDBPoolFromConfig`), Git interfaces
  (`builders.NewPerfGitFromConfig`), and caching mechanisms
  (`builders.GetCacheFromConfig`) are created and passed into the respective
  maintenance tasks. This promotes modularity and testability.
- **Error Handling and Logging:** Each maintenance task incorporates error
  handling and logging (`sklog`) to provide visibility into its operations and
  to aid in diagnosing issues. While errors in one task might be logged, the
  overall `Start` function aims to keep other independent tasks running.
- **Idempotency (Implicit):** While not explicitly stated for all tasks, many
  maintenance operations are inherently idempotent or designed to be safe to
  run repeatedly (e.g., schema migration, data deletion based on age).
- **Phased Introduction of Features:** Features like regression migration or
  Sheriff config integration are gated by flags (`flags.MigrateRegressions`,
  `instanceConfig.EnableSheriffConfig`). This allows for gradual rollouts and
  testing in production environments.

### Responsibilities and Key Components

The `maintenance` module orchestrates several distinct background processes.

**1. Core Initialization and Schema Management (`maintenance.go`)**

- **Why:** Before any maintenance tasks can run, essential services like
  tracing need to be initialized. Crucially, the database schema must be
  validated and migrated to the expected version. This ensures that all
  subsequent database operations are performed against a compatible and
  up-to-date schema.
- **How:**
  - `tracing.Init`: Sets up the distributed tracing system.
  - `builders.NewDBPoolFromConfig`: Establishes a connection pool to the
    database.
  - `expectedschema.ValidateAndMigrateNewSchema`: Checks the current
    database schema version against the expected version defined in the
    codebase. If they don't match, it applies the necessary migrations to
    bring the schema up to date. This is a critical step to prevent data
    corruption or application errors due to schema mismatches.

**2. Git Repository Synchronization (`maintenance.go`)**

- **Why:** Perf relies on an up-to-date view of the monitored Git repository
  to associate performance data with specific commits. This process ensures
  that new commits are continuously ingested into the Perf system.
- **How:**
  - `builders.NewPerfGitFromConfig`: Creates an instance of `perfgit.Git`,
    which provides an interface to the Git repository.
  - `g.StartBackgroundPolling(ctx, gitRepoUpdatePeriod)`: This method
    launches a goroutine within the `perfgit` component. This goroutine
    periodically fetches the latest changes from the remote Git repository
    (origin) and updates the local representation, typically also updating a
    `Commits` table in the database with new commit information. The
    `gitRepoUpdatePeriod` constant (e.g., 1 minute) defines how frequently
    this update occurs.

**3. Regression Schema Migration (`maintenance.go`)**

- **Why:** Over time, the way regression data is stored might need to be
  changed for performance, new features, or data integrity reasons. This
  component handles the migration of existing regression data from an older
  schema/table to a newer one. This is often a long-running process for
  instances with a large history of regressions.
- **How:**
  - Controlled by the `flags.MigrateRegressions` flag.
  - `migration.New`: Creates a `Migrator` instance, likely configured with
    database connections for both the old and new regression storage
    mechanisms.
  - `migrator.RunPeriodicMigration(regressionMigratePeriod,
regressionMigrationBatchSize)`: Starts a goroutine that, at intervals
    defined by `regressionMigratePeriod`, processes a
    `regressionMigrationBatchSize` number of regressions, moving them from
    the old storage to the new. This batching approach prevents overwhelming
    the database and allows the migration to proceed incrementally.

**4. Sheriff Configuration Import (`maintenance.go`)**

- **Why:** Perf allows defining alert configurations (Sheriff configs) that
  specify how and when alerts should be triggered for performance regressions.
  These configurations can be managed externally (e.g., via LUCI Config). This
  component ensures that Perf stays synchronized with the latest
  configurations.
- **How:**
  - Conditional on `instanceConfig.EnableSheriffConfig` and a non-empty
    `instanceConfig.InstanceName`.
  - It initializes `AlertStore` and `SubscriptionStore` for managing alert
    and subscription data within Perf.
  - `luciconfig.NewApiClient`: Creates a client to communicate with the LUCI
    Config service.
  - `sheriffconfig.New`: Initializes the `SheriffConfig` service, which
    encapsulates the logic for fetching, parsing, and applying Sheriff
    configurations.
  - `sheriffConfig.StartImportRoutine(configImportPeriod)`: Launches a
    goroutine that periodically (every `configImportPeriod`) polls the LUCI
    Config service for the specified instance. If new or updated
    configurations are found, they are processed and stored/updated in
    Perf's database (e.g., in the `Alerts` and `Subscriptions` tables).

**5. Query Cache Refresh (`maintenance.go`)**

- **Why:** To speed up common queries (e.g., retrieving the set of available
  trace parameters, known as ParamSets), Perf can cache this information. This
  component is responsible for periodically rebuilding and refreshing these
  caches.
- **How:**

  - Controlled by the `flags.RefreshQueryCache` flag.
  - `builders.NewTraceStoreFromConfig`: Gets an interface to the trace data.
  - `dfbuilder.NewDataFrameBuilderFromTraceStore`: Creates a utility for
    building data frames from traces, which is likely used to derive the
    ParamSet.
  - `psrefresh.NewDefaultParamSetRefresher`: Initializes a component
    specifically designed to refresh ParamSets. It uses the
    `DataFrameBuilder` to scan trace data and determine the current set of
    unique parameter key-value pairs.
  - `psRefresher.Start(time.Hour)`: Starts a goroutine to refresh the
    primary ParamSet (perhaps stored directly in the database or an
    in-memory representation) hourly.
  - `builders.GetCacheFromConfig`: If a distributed cache like Redis is
    configured, this obtains a client for it.
  - `psrefresh.NewCachedParamSetRefresher`: Wraps the primary `psRefresher`
    with a caching layer.
  - `cacheParamSetRefresher.StartRefreshRoutine(redisCacheRefreshPeriod)`:
    Starts another goroutine that takes the ParamSet generated by
    `psRefresher` and populates the external cache (e.g., Redis) at
    `redisCacheRefreshPeriod` intervals (e.g., every 4 hours). This provides
    a faster lookup path for frequently accessed ParamSet data.

  Workflow:

  ```
  Trace Data --> DataFrameBuilder --> ParamSetRefresher (generates primary ParamSet)
                                        |
                                        v
                                   CachedParamSetRefresher --> External Cache (e.g., Redis)
  ```

**6. Old Data Deletion (`deletion/deleter.go`, `maintenance.go`)**

- **Why:** Over time, Perf accumulates a large amount of data, including
  regression information and associated shortcuts (which are often links or
  identifiers for specific data views). To manage storage costs and maintain
  system performance, very old data that is unlikely to be accessed needs to
  be periodically deleted.
- **How:**

  - Controlled by the `flags.DeleteShortcutsAndRegressions` flag.
  - **`deletion.New(db, ...)`:** Initializes a `Deleter` object. This object
    encapsulates the logic for identifying and removing outdated regressions
    and shortcuts. It takes a database connection pool (`db`) and the
    datastore type. Internally, it creates instances of `sqlregressionstore`
    and `sqlshortcutstore` to interact with the respective database tables.
  - **`deleter.RunPeriodicDeletion(deletionPeriod, deletionBatchSize)`:**
    This method in `maintenance.go` calls the `RunPeriodicDeletion` method
    on the `Deleter` instance.
  - Inside `deleter.go`, `RunPeriodicDeletion` starts a goroutine.
  - This goroutine ticks at intervals specified by `deletionPeriod` (e.g.,
    every 15 minutes).
  - On each tick, it calls `d.DeleteOneBatch(deletionBatchSize)`.
  - **`Deleter.DeleteOneBatch(shortcutBatchSize)`:**

  * Calls `d.getBatch(ctx, shortcutBatchSize)` to identify a batch of
    regressions and shortcuts eligible for deletion.
    - **`Deleter.getBatch(...)`:**
      - Finds the oldest commit number present in the `Regressions`
        table.
      - Iteratively queries the `Regressions` table for ranges of
        commits, starting from the oldest.
      - For each regression found, it checks the timestamp of its `Low`
        and `High` `StepPoint`s.
      - If a `StepPoint`'s timestamp is older than the defined `ttl`
        (Time-To-Live, currently -18 months), the associated shortcut
        and the commit number of the regression are marked for deletion.
      - It continues collecting these until the number of shortcuts to
        be deleted reaches approximately `shortcutBatchSize`.
      - Returns the list of commit numbers whose regressions will be
        deleted and the list of shortcut IDs to be deleted.
  * Calls `d.deleteBatch(ctx, commitNumbers, shortcuts)` to perform the
    actual deletion.
    - **`Deleter.deleteBatch(...)`:**
      - Starts a database transaction.
      - Iterates through the `commitNumbers` and calls
        `d.regressionStore.DeleteByCommit()` for each, removing the
        regression data associated with that commit.
      - Iterates through the `shortcuts` and calls
        `d.shortcutStore.DeleteShortcut()` for each, removing the
        shortcut entry.
      - If all deletions are successful, it commits the transaction. If
        any error occurs, it rolls back the transaction to ensure data
        consistency.

  Deletion Workflow:

  ```
  Timer (every deletionPeriod) --> DeleteOneBatch
                                     |
                                     v
                                  getBatch (identifies old data based on TTL)
                                     |
                                     | Returns (commit_numbers_to_delete, shortcut_ids_to_delete)
                                     v
                                  deleteBatch (deletes in a transaction)
                                     |
                                     +--> RegressionStore.DeleteByCommit
                                     +--> ShortcutStore.DeleteShortcut
  ```

  The `ttl` variable in `deleter.go` is set to -18 months, meaning regressions
  and their associated shortcuts older than 1.5 years are targeted for
  deletion. This value was determined based on stakeholder requirements for
  data retention.

The `select {}` at the end of the `Start` function in `maintenance.go` is a
common Go idiom to make the main goroutine (the one that called `Start`) block
indefinitely. Since all the actual work is done in background goroutines
launched by `Start`, this prevents the `Start` function from returning and thus
keeps the maintenance processes alive.

# Module: /go/notify

The `notify` module in Perf is responsible for handling notifications related to
performance regressions. It provides a flexible framework for formatting and
sending notifications through various channels like email, issue trackers, or
custom endpoints like Chromeperf.

**Core Concepts and Design:**

The notification system is built around a few key abstractions:

1.  **`Notifier` Interface (`notify.go`):** This is the central interface for
    sending notifications. It defines methods for:

    - `RegressionFound`: Called when a new regression is detected.
    - `RegressionMissing`: Called when a previously detected regression is no
      longer found (e.g., due to new data or fixes).
    - `ExampleSend`: Used for sending test/dummy notifications to verify
      configuration.
    - `UpdateNotification`: For updating an existing notification (e.g.,
      adding a comment to an issue).

2.  **`Formatter` Interface (`notify.go`):** This interface is responsible for
    constructing the content (body and subject) of a notification.
    Implementations exist for:

    - `HTMLFormatter` (`html.go`): Generates HTML-formatted notifications,
      suitable for email.
    - `MarkdownFormatter` (`markdown.go`): Generates Markdown-formatted
      notifications, suitable for issue trackers or other systems that support
      Markdown. The formatters use Go's `text/template` package, allowing for
      customizable notification messages. Templates can access a
      `TemplateContext` (or `AndroidBugTemplateContext` for Android-specific
      notifications) which provides data about the regression, commit, alert,
      etc.

3.  **`Transport` Interface (`notify.go`):** This interface defines how a
    formatted notification is actually sent. Implementations include:

    - `EmailTransport` (`email.go`): Sends notifications via email using the
      `emailclient` module.
    - `IssueTrackerTransport` (`issuetracker.go`): Interacts with an issue
      tracking system (configured for Google's Issue Tracker/Buganizer) to
      create or update issues. It uses the `go/issuetracker/v1` client and
      requires an API key for authentication.
    - `NoopTransport` (`noop.go`): A "do nothing" implementation, useful for
      disabling notifications or for testing.

4.  **`NotificationDataProvider` Interface (`notification_provider.go`):** This
    interface is responsible for gathering the necessary data to populate the
    notification templates.

    - The `defaultNotificationDataProvider` uses a `Formatter` to generate the
      notification body and subject based on `RegressionMetadata`.
    - `androidNotificationProvider` (`android_notification_provider.go`) is a
      specialized provider for Android-specific bug reporting. It uses its own
      `AndroidBugTemplateContext` which includes Android-specific details like
      Build ID diff URLs. It leverages the `MarkdownFormatter` for content
      generation but with Android-specific templates.

**Workflow for Sending a Notification (Simplified):**

1.  A regression is detected (e.g., by the `alerter` module).
2.  The `Notifier`'s `RegressionFound` method is called with details about the
    regression (commit, alert configuration, cluster summary, etc.).
3.  The `Notifier` (typically `defaultNotifier`) uses its
    `NotificationDataProvider` to get the raw notification data (body and
    subject).
    - The `NotificationDataProvider` populates a context object (e.g.,
      `TemplateContext` or `AndroidBugTemplateContext`).
    - It then uses a `Formatter` (e.g., `MarkdownFormatter`) to execute the
      appropriate template with this context, producing the final body and
      subject.
4.  The `Notifier` then calls its `Transport`'s `SendNewRegression` method,
    passing the formatted body and subject.
5.  The `Transport` implementation handles the actual sending (e.g., makes an
    API call to the issue tracker or sends an email).

```
Regression Detected --> Notifier.RegressionFound(...)
                           |
                           v
                  NotificationDataProvider.GetNotificationDataRegressionFound(...)
                           |
                           | (Populates Context, e.g., TemplateContext)
                           v
                  Formatter.FormatNewRegressionWithContext(...)
                           |  (Uses Go templates)
                           v
                  Formatted Body & Subject
                           |
                           v
                  Transport.SendNewRegression(body, subject)
                           |
                           +------------------> EmailTransport --> Email Server
                           |
                           +------------------> IssueTrackerTransport --> Issue Tracker API
                           |
                           +------------------> NoopTransport --> (Does nothing)
```

**Key Files and Responsibilities:**

- **`notify.go`**:

  - Defines the core interfaces: `Notifier`, `Formatter`, `Transport`.
  - Provides the `defaultNotifier` implementation, which orchestrates the
    notification process by composing a `NotificationDataProvider`,
    `Formatter`, and `Transport`.
  - Contains the `New()` factory function that constructs the appropriate
    `Notifier` based on the `NotifyConfig`. This is the main entry point for
    creating a notifier.
  - Defines `TemplateContext` used by generic formatters.
  - Includes logic in `getRegressionMetadata` to fetch additional
    information like source file links from `TraceStore` if the alert is for
    an individual trace.

- **`notification_provider.go`**:

  - Defines the `NotificationDataProvider` interface.
  - Provides `defaultNotificationDataProvider` which uses a generic
    `Formatter`.
  - The purpose is to abstract the data gathering logic for notifications,
    allowing for different data providers (like the Android-specific one)
    without changing the core `Notifier` or `Transport` mechanisms.

- **`android_notification_provider.go`**:

  - Implements `NotificationDataProvider` specifically for Android bug
    creation.
  - Uses `AndroidBugTemplateContext` to provide Android-specific data to
    templates, such as `GetBuildIdUrlDiff` for generating links to compare
    Android build CLs.
  - Relies on `MarkdownFormatter` but configures it with Android-specific
    notification templates defined in the `NotifyConfig`. This allows
    Android teams to customize their bug reports.

- **`markdown.go` & `html.go`**:

  - Implement the `Formatter` interface for Markdown and HTML respectively.
  - Define default templates for new regressions and when regressions go
    missing.
  - `MarkdownFormatter` can be configured with custom templates via
    `NotifyConfig`. It also provides a `buildIDFromSubject` template
    function, specifically designed for Android's commit message format, to
    extract build IDs.
  - `viewOnDashboard` is a utility function to construct a URL to the Perf
    explore page for the given regression.

- **`email.go` & `issuetracker.go` & `noop.go`**:

  - Implement the `Transport` interface.
  - `email.go`: Uses `emailclient` to send emails. Splits
    comma/space-separated recipient lists.
  - `issuetracker.go`: Interacts with the Google Issue Tracker API. It
    requires API key secrets (configured via `NotifyConfig`) and uses OAuth2
    for authentication. It can create new issues and update existing ones
    (e.g., to mark them obsolete).
  - `noop.go`: A null implementation for disabling notifications.

- **`chromeperfnotifier.go`**:

  - Implements the `Notifier` interface directly, without using the
    `Formatter` or `Transport` abstractions in the same way as
    `defaultNotifier`. This is because it communicates directly with the
    Chrome Performance Dashboard's Anomaly API.
  - It translates Perf's regression data into the format expected by the
    Chromeperf API (`ReportRegression`).
  - Includes logic (`isParamSetValid`, `getTestPath`) to ensure the data
    conforms to Chromeperf's requirements (e.g., specific param keys like
    `master`, `bot`, `benchmark`, `test`).
  - Determines if a regression is an improvement based on the
    `improvement_direction` parameter and the step direction.

- **`commitrange.go`**:

  - Provides `URLFromCommitRange`, a utility function to generate a URL for
    a commit or a range of commits. If a `commitRangeURLTemplate` is
    provided (e.g., via configuration), it will be used to create a URL
    showing the diff between two commits. Otherwise, it defaults to the
    individual commit's URL. This is used by formatters to create links in
    notifications.

- **`common/notificationData.go`**:

  - Defines `NotificationData` (simple struct for body and subject) and
    `RegressionMetadata` (a comprehensive struct holding all relevant
    information about a regression needed for notification generation). This
    promotes a common data structure for passing regression details.

**Configuration and Customization (`NotifyConfig`):**

The behavior of the `notify` module is heavily influenced by
`config.NotifyConfig`. This configuration allows users to:

- Choose the notification type (`Notifications` field): `None`, `HTMLEmail`,
  `MarkdownIssueTracker`, `ChromeperfAlerting`, `AnomalyGrouper`.
- Specify the `NotificationDataProvider`: `DefaultNotificationProvider` or
  `AndroidNotificationProvider`.
- Customize the subject and body of notifications using Go templates
  (`Subject`, `Body`, `MissingSubject`, `MissingBody`). This is particularly
  relevant for `MarkdownFormatter` and `androidNotificationProvider`.
- Provide settings for `IssueTrackerTransport` (API key secret locations).

This design allows for flexibility in how notifications are generated and
delivered, catering to different needs and integrations. For instance, the
Android team can have highly customized bug reports, while other users might
prefer standard email notifications. The `ChromeperfNotifier` demonstrates a
direct integration with another system, bypassing some of the general-purpose
formatting/transport layers when a specific API is targeted.

# Module: /go/notifytypes

## Perf Notifytypes Module

### Overview

The `notifytypes` module in Perf defines the various types of notification
mechanisms that can be triggered in response to performance regressions or other
significant events. It also defines types for data providers that supply the
necessary information for these notifications. This module serves as a central
point for enumerating and categorizing notification strategies, enabling
flexible and extensible notification handling within the Perf system.

### Why: Design Decisions

The primary goal of this module is to provide a structured and type-safe way to
manage notification types.

- **Extensibility:** By defining notification types as constants of a custom
  `Type` string, new notification methods can be easily added in the future
  without requiring significant code changes in consuming modules. This
  promotes loose coupling and allows the notification system to evolve
  independently.
- **Clarity and Readability:** Using named constants (e.g., `HTMLEmail`,
  `MarkdownIssueTracker`) instead of raw strings makes the code more
  self-documenting and reduces the likelihood of errors due to typos.
- **Centralized Definition:** Having all notification types defined in one
  place simplifies maintenance and provides a clear overview of the available
  notification options.
- **Separation of Concerns:** The `NotificationDataProviderType` allows for
  different sources or formats of data to be used for generating
  notifications, separating the concern of _what_ data is needed from _how_
  the notification is delivered. This is crucial, for example, when different
  platforms (like Android) might require specific data formatting or
  additional information.

### How: Implementation Choices

- **`Type` (string alias):** The `Type` is defined as an alias for `string`.
  This allows for string-based storage and transmission of notification types
  (e.g., in configuration files or database entries) while still providing a
  degree of type safety within Go code.
- **Constants for Notification Types:** Specific notification mechanisms are
  defined as constants of type `Type`. This ensures that only valid,
  predefined notification types can be used.
  - `HTMLEmail`: Indicates notifications sent as HTML-formatted emails. This
    is suitable for rich content and direct user communication.
  - `MarkdownIssueTracker`: Represents notifications formatted in Markdown,
    intended for integration with issue tracking systems. This facilitates
    automated ticket creation or updates.
  - `ChromeperfAlerting`: Specifies that regression data should be sent to
    the Chromeperf alerting system. This allows for integration with a
    specialized alerting infrastructure.
  - `AnomalyGrouper`: Designates that regressions should be processed by an
    anomaly grouping logic, which then determines the appropriate action.
    This enables more sophisticated handling of multiple related anomalies.
  - `None`: A special type indicating that no notification should be sent.
    This is useful for disabling notifications in certain contexts or for
    configurations where alerting is not desired.
- **`AllNotifierTypes` Slice:** This public variable provides a convenient way
  for other parts of the system to iterate over or validate against all known
  notification types.
- **`NotificationDataProviderType` (string alias):** Similar to `Type`, this
  defines the kind of data provider to use for notifications.
  - `DefaultNotificationProvider`: Represents the standard or default data
    provider.
  - `AndroidNotificationProvider`: Indicates a specialized data provider
    tailored for Android-specific notification requirements. This might
    involve fetching different metrics, formatting data in a particular way,
    or including Android-specific metadata.

### Responsibilities and Key Components

- **`notifytypes.go`:** This is the sole file in the module and contains all
  the definitions.
  - **Defines Notification Types:** Its primary responsibility is to
    enumerate the supported notification mechanisms (`HTMLEmail`,
    `MarkdownIssueTracker`, `ChromeperfAlerting`, `AnomalyGrouper`, `None`).
    This acts as a contract for other modules that implement or consume
    notification functionalities.
  - **Defines Data Provider Types:** It also defines the types of data
    providers (`DefaultNotificationProvider`, `AndroidNotificationProvider`)
    that can be used to source information for notifications. This allows
    the notification system to adapt to different data sources or formats.
  - **Provides an Exhaustive List:** The `AllNotifierTypes` variable makes
    it easy for other components to get a list of all valid notification
    types, for example, for display in a UI or for validation purposes.

### Key Workflows/Processes

While this module itself doesn't implement workflows, it underpins them. A
typical conceptual workflow where these types would be used is:

1.  **Regression Detected:** The Perf system identifies a performance
    regression. `Regression Event -->`
2.  **Configuration Checked:** The system checks the configuration associated
    with the metric/test that regressed. This configuration would specify a
    `notifytypes.Type`. `Configuration Lookup (specifies notifytypes.Type, e.g.,
HTMLEmail) -->`
3.  **Notifier Selected:** Based on the `notifytypes.Type` from the
    configuration, the appropriate notifier implementation is selected.
    `Notification System -->`
4.  **Data Provider Selected (if applicable):** If the configuration also
    specifies a `notifytypes.NotificationDataProviderType`, the corresponding
    data provider is chosen. `Data Provider (e.g., AndroidNotificationProvider)
-->`
5.  **Notification Sent:** The selected notifier uses the data (potentially from
    the selected data provider) to construct and send the notification.
    `Notification Delivered (e.g., Email Sent)`

For example, if a regression is detected for an Android benchmark and the
configuration specifies `HTMLEmail` as the `Type` and
`AndroidNotificationProvider` as the `NotificationDataProviderType`:

`Regression Event` -> `Config: {Type: HTMLEmail, DataProvider:
AndroidNotificationProvider}` -> `Select EmailNotifier` -> `Select
AndroidDataProvider` -> `AndroidDataProvider fetches data` -> `EmailNotifier
formats and sends HTML email`

# Module: /go/perf-tool

The `perf-tool` module provides a command-line interface (CLI) for interacting
with various aspects of the Perf performance monitoring system. It allows
developers and administrators to manage configurations, inspect data, perform
database maintenance tasks, and validate ingestion files.

The primary motivation behind `perf-tool` is to offer a centralized and
scriptable way to perform common Perf operations that would otherwise require
manual intervention or direct database interaction. This simplifies workflows
and enables automation of routine tasks.

The core functionality is organized into subcommands, each addressing a specific
area of Perf:

- **`config`**: Manages Perf instance configurations.
  - `create-pubsub-topics-and-subscriptions`: Sets up the necessary Google
    Cloud Pub/Sub topics and subscriptions required for data ingestion. This
    is crucial for ensuring that Perf instances can receive and process
    performance data.
  - `validate`: Checks the syntax and validity of a Perf instance
    configuration file. This helps prevent deployment of misconfigured
    instances.
- **`tiles`**: Interacts with the tiled data storage used by Perf's
  `tracestore`. Tiles are segments of time-series data.
  - `last`: Displays the index of the most recent tile, providing insight
    into the current state of data ingestion.
  - `list`: Shows a list of recent tiles and the number of traces they
    contain, useful for understanding data volume and distribution.
- **`traces`**: Allows querying and exporting trace data.
  - `list`: Retrieves and displays the IDs of traces that match a given
    query within a specific tile. This is useful for ad-hoc data
    exploration.
  - `export`: Exports trace data matching a query and commit range to a JSON
    file. This enables external analysis or data migration.
- **`ingest`**: Manages the data ingestion process.
  - `force-reingest`: Triggers the re-ingestion of data files from Google
    Cloud Storage (GCS) for a specified time range. This is useful for
    reprocessing data after configuration changes or to fix ingestion
    errors. The workflow is:
  * Parse start and stop time parameters.
  * Iterate through configured GCS source prefixes.
  * For each prefix, determine hourly GCS directories within the time range.
  * List files in each directory.
  * For each file, create a Pub/Sub message with the GCS object attributes
    (bucket and name).
  * Publish these messages to the configured ingestion topic. This simulates
    the GCS notification events that trigger ingestion.
  - `validate`: Validates the format and content of an ingestion file
    against the expected schema and parsing rules. This helps ensure data
    quality before ingestion.
- **`database`**: Provides tools for backing up and restoring Perf database
  components. This is critical for disaster recovery and data migration.
  - `backup`:
  - `alerts`: Backs up alert configurations to a zip file.
  - `shortcuts`: Backs up saved shortcut configurations to a zip file.
  - `regressions`: Backs up regression data (detected performance changes)
    and associated shortcuts to a zip file. It backs up data up to a
    specified date (defaulting to four weeks ago). The process involves
    iterating backward through commits in batches, fetching regressions for
    each commit range, and storing them along with any shortcuts referenced
    in those regressions.
  - `restore`:
  - `alerts`: Restores alert configurations from a backup file.
  - `shortcuts`: Restores shortcut configurations from a backup file.
  - `regressions`: Restores regression data and their associated shortcuts
    from a backup file. It's important to note that restoring regressions
    also attempts to re-create the associated shortcuts.
- **`trybot`**: Contains experimental functionality related to trybot
  (pre-submit testing) data.
  - `reference`: Generates a synthetic nanobench reference file. This file
    is constructed by loading a specified trybot results file, identifying
    all trace IDs within it, and then fetching historical sample data for
    these traces from the main Perf instance (specifically, from the last N
    ingested files). The aggregated historical samples are then formatted
    into a new nanobench JSON file. This allows for comparing trybot results
    against a baseline derived from recent production data using tools like
    `nanostat`.
- **`markdown`**: Generates Markdown documentation for the `perf-tool` CLI
  itself.

The `main.go` file sets up the CLI application using the `urfave/cli` library.
It defines flags, commands, and subcommands, and maps them to corresponding
functions in the `application` package. It handles flag parsing, configuration
loading (from a file, with optional connection string overrides), and
initialization of logging.

The `application/application.go` file defines the `Application` interface and
its concrete implementation `app`. This interface abstracts the core logic for
each command, promoting testability and separation of concerns. The `app` struct
implements methods that interact with various Perf components like `tracestore`,
`alertStore`, `shortcutStore`, `regressionStore`, and GCS.

Key design choices include:

- **Interface-based application logic (`Application` interface):** This allows
  for mocking the application logic during testing (as seen in `main_test.go`
  and `application/mocks/Application.go`), ensuring that the CLI command
  parsing and flag handling can be tested independently of the actual backend
  operations.
- **Configuration-driven:** Most operations require an instance configuration
  file (`--config_filename`), which defines data store connections, GCS
  sources, etc. This makes the tool adaptable to different Perf deployments.
- **Use of helper builders:** Functions from `perf/go/builders` are used to
  instantiate components like `TraceStore`, `AlertStore`, etc., based on the
  provided instance configuration. This centralizes component creation logic.
- **Zip format for backups:** Database backups for alerts, shortcuts, and
  regressions are stored in zip files. Inside these zip files, data is
  typically serialized using `encoding/gob`. This provides a simple and
  portable backup solution.
- **Batching for large operations:** When backing up regressions, data is
  fetched in batches of commits (`regressionBatchSize`) to manage memory and
  avoid overwhelming the database.
- **Pub/Sub for re-ingestion:** The `ingest force-reingest` command leverages
  Pub/Sub by publishing messages that mimic GCS notifications, effectively
  triggering the standard ingestion pipeline.

The `application/mocks/Application.go` file contains a mock implementation of
the `Application` interface, generated by the `mockery` tool. This is used in
`main_test.go` to test the command-line argument parsing and dispatch logic
without actually performing the underlying operations.

# Module: /go/perfclient

The `perfclient` module provides an interface for sending performance data to
Skia Perf's ingestion system. The primary goal of this module is to abstract the
complexities of interacting with Google Cloud Storage (GCS), which is the
underlying mechanism Perf uses for data ingestion. By providing a dedicated
client, it simplifies the process for other applications and services that need
to report performance metrics.

The core design centers around a `ClientInterface` and its concrete
implementation, `Client`. This approach allows for easy mocking and testing,
promoting loose coupling between the `perfclient` and its consumers.

**Key Components and Responsibilities:**

- **`perf_client.go`**:

  - **`ClientInterface`**: This interface defines the contract for pushing
    performance data. The key method is `PushToPerf`. The decision to use an
    interface here is crucial for testability and dependency injection. It
    allows consumers to use a real GCS-backed client in production and a
    mock client in tests.
  - **`Client`**: This struct is the concrete implementation of
    `ClientInterface`. It holds a `gcs.GCSClient` instance, which is
    responsible for the actual communication with Google Cloud Storage, and
    a `basePath` string that specifies the root directory within the GCS
    bucket where performance data will be stored. The constructor `New`
    takes these as arguments, allowing users to configure the GCS bucket and
    the top-level folder for their data.
  - **`PushToPerf` method**: This is the workhorse of the module.

  * It takes a `time.Time` object (`now`), a `folderName`, a `filePrefix`,
    and a `format.BenchData` struct (which represents the performance
    metrics).
  * The `format.BenchData` is first marshaled into a JSON string. This is
    the standard format Perf expects for ingestion.
  * The JSON data is then compressed using `gzip`. This is a performance
    optimization, as GCS can automatically decompress gzipped files with the
    correct `ContentEncoding` header, reducing storage costs and transfer
    times.
  * A deterministic GCS object path is constructed using the `objectPath`
    helper function. This path incorporates the `basePath`, the current
    timestamp (formatted as `YYYY/MM/DD/HH/`), the `folderName`, and a
    filename composed of the `filePrefix`, an MD5 hash of the JSON data, and
    a millisecond-precision timestamp. The inclusion of the MD5 hash helps
    in avoiding duplicate uploads of identical data and can be useful for
    debugging or data verification. The timestamp in the path and filename
    ensures that data from different runs or times are stored separately and
    can be easily queried.
  * Finally, the compressed data is uploaded to GCS using the
    `storageClient.SetFileContents` method. Crucially, it sets
    `ContentEncoding: "gzip"` and `ContentType: "application/json"` in the
    `gcs.FileWriteOptions`. This metadata informs GCS about the compression
    and data type, enabling features like automatic decompression.

  - **`objectPath` function**: This helper function is responsible for
    constructing the unique GCS path for each performance data file. The
    rationale for this specific path structure
    (`basePath/YYYY/MM/DD/HH/folderName/filePrefix_hash_timestamp.json`) is
    to organize data chronologically and by task, making it easier to
    browse, query, and manage within GCS. The hash ensures uniqueness and
    integrity.

- **`mock_perf_client.go`**:

  - **`MockPerfClient`**: This provides a mock implementation of
    `ClientInterface` using the `testify/mock` library. This is essential
    for unit testing components that depend on `perfclient` without
    requiring actual GCS interaction. It allows developers to define
    expected calls to `PushToPerf` and verify that their code interacts with
    the client correctly. The `NewMockPerfClient` constructor returns a
    pointer to ensure that the methods provided by `mock.Mock` (like `On`
    and `AssertExpectations`) are accessible.

**Workflow: Pushing Performance Data**

The primary workflow involves a client application using `perfclient` to send
performance data:

```
Client App                   perfclient.Client                  gcs.GCSClient
     |                            |                                 |
     | -- Call PushToPerf(now,    |                                 |
     |    folder, prefix, data) ->|                                 |
     |                            | -- Marshal data to JSON         |
     |                            | -- Compress JSON (gzip)         |
     |                            | -- Construct GCS objectPath     |
     |                            |    (includes time, folder,     |
     |                            |     prefix, data hash)          |
     |                            |                                 |
     |                            | -- Call SetFileContents(path,   |
     |                            |    options, compressed_data) -> |
     |                            |                                 | -- Upload to GCS
     |                            |                                 |    with gzip encoding
     |                            |                                 |    and JSON content type
     |                            | <-------------------------------| -- Return success/error
     | <--------------------------|                                 |
     | -- Receive success/error   |                                 |
```

The design emphasizes creating a clear separation of concerns: the `perfclient`
handles the formatting, compression, and path generation logic specific to
Perf's ingestion requirements, while the underlying `gcs.GCSClient` handles the
raw GCS communication. This makes the `perfclient` a focused and reusable
component for any system needing to integrate with Skia Perf.

# Module: /go/perfresults

## Module Overview

The `perfresults` module is responsible for fetching, parsing, and processing
performance results data generated by Telemetry-based benchmarks in the Chromium
project. This data typically resides in `perf_results.json` files. The module
provides functionalities to:

1.  **Load Performance Data**: Retrieve performance results from various
    sources, primarily Buildbucket builds. This involves interacting with
    Buildbucket to get build information, Swarming to identify relevant tasks
    and their outputs, and RBE-CAS (Content Addressable Storage) to download the
    actual `perf_results.json` files.
2.  **Parse Performance Data**: Interpret the structure of `perf_results.json`
    files. These files contain sets of histograms, where each histogram
    represents a specific benchmark measurement. The parser extracts these
    histograms and associated metadata.
3.  **Process and Transform Data**: Convert the parsed performance data into a
    format suitable for ingestion by other systems, such as the Perf ingestion
    pipeline. This includes aggregating histogram samples (e.g., calculating
    mean, max, min) and structuring the data according to a defined schema.

The primary goal is to provide a reliable and efficient way to access and
utilize Chromium's performance data for analysis and monitoring.

## Design Decisions and Implementation Choices

### Data Loading Workflow

The process of loading performance results from a Buildbucket build involves
several steps:

`Buildbucket ID -> BuildInfo -> Swarming Task ID -> Child Swarming Task IDs ->
CAS Outputs -> PerfResults`

1.  **Buildbucket Interaction (`buildbucket.go`)**:

    - **Why**: Buildbucket is the entry point for CI/CQ builds. It contains
      information about the build, including the associated Swarming task and
      crucial metadata like git revision and commit position.
    - **How**: The `bbClient` interacts with the Buildbucket PRPC API to fetch
      build details using a given `buildID`. It specifically requests fields
      like `builder`, `status`, `infra.backend.task.id` (for the Swarming task
      ID), `output.properties` (for git revision information), and
      `input.properties` (for `perf_dashboard_machine_group`).
    - The `BuildInfo` struct is populated with this information, providing a
      consolidated view of the build's context. The `GetPosition()` method on
      `BuildInfo` is crucial as it determines the commit identifier (either
      commit position or git hash) used for associating the performance data
      with a specific point in the codebase.

2.  **Swarming Interaction (`swarming.go`)**:

    - **Why**: The main Buildbucket task often spawns multiple child Swarming
      tasks, each running a subset of benchmarks. We need to identify all
      these child tasks to gather all performance results.
    - **How**: The `swarmingClient` uses the Swarming PRPC API.
      - `findChildTaskIds`: Given a parent Swarming task ID (obtained from
        `BuildInfo`), this function lists all child tasks by querying for
        tasks with a matching `parent_task_id` tag. The query is scoped by
        the parent task's creation and completion timestamps to narrow down
        the search.
      - `findTaskCASOutputs`: For each child task ID, this function
        retrieves the task result, specifically looking for the
        `CasOutputRoot`. This reference points to the RBE-CAS location where
        the task's output files (including `perf_results.json`) are stored.

3.  **RBE-CAS Interaction (`rbecas.go`)**:

    - **Why**: `perf_results.json` files are stored in RBE-CAS. RBE-CAS
      provides efficient and reliable storage for large build artifacts.
    - **How**: The `RBEPerfLoader` uses the RBE SDK to interact with CAS.
      - `fetchPerfDigests`: Given a CAS reference (pointing to the root
        directory of a task's output), this function:
      * Reads the root `Directory` proto.
      * Retrieves the entire directory tree using `GetDirectoryTree`.
      * Flattens the tree to get a map of file paths to their digests.
      * Filters for files named `perf_results.json`. The path structure is
        expected to be `benchmark_name/perf_results.json`, allowing
        association of results with a specific benchmark.
      - `loadPerfResult`: Given a digest for a `perf_results.json` file,
        this reads the blob from CAS and parses it using `NewResults`.
      - `LoadPerfResults`: This orchestrates the loading for multiple CAS
        references (from multiple child Swarming tasks). It iterates through
        each CAS reference, fetches the digests of `perf_results.json`
        files, loads each file, and then merges results from the same
        benchmark. Merging is important because a single benchmark might
        have its results split across multiple files or tasks.

4.  **Orchestration (`perf_loader.go`)**:

    - **Why**: A central loader is needed to tie together the interactions
      with Buildbucket, Swarming, and RBE-CAS.
    - **How**: The `loader.LoadPerfResults` method coordinates the entire
      workflow:
      1.  Initializes `bbClient` to get `BuildInfo`.
      2.  Initializes `swarmingClient` to find child task IDs and then their
          CAS outputs.
      3.  It performs a sanity check (`checkCasInstances`) to ensure all CAS
          outputs come from the same RBE instance, simplifying client
          initialization.
      4.  Initializes `RBEPerfLoader` (via `rbeProvider` for testability) for
          the determined CAS instance.
      5.  Calls `RBEPerfLoader.LoadPerfResults` with the list of CAS
          references to fetch and parse all `perf_results.json` files.
    - The use of `rbeProvider` is a good example of dependency injection,
      allowing tests to mock the RBE-CAS interaction.

### Performance Data Parsing (`perf_results_parser.go`)

- **Why**: `perf_results.json` files have a specific, somewhat complex
  structure. A dedicated parser is needed to extract meaningful data
  (histograms and their metadata).
- **How**:
  - The `PerfResults` struct is the main container, holding a map of
    `TraceKey` to `Histogram`.
  - `TraceKey` uniquely identifies a trace, composed of `ChartName` (metric
    name), `Unit`, `Story` (user journey/test case), `Architecture`, and
    `OSName`. These fields are extracted from the histogram's own properties
    and its associated "diagnostics" which are references to other metadata
    objects within the JSON file.
  - `Histogram` stores the `SampleValues` (the actual measurements).
  - **Streaming JSON Decoding**: `NewResults` uses `json.NewDecoder` to
    process the input `io.Reader` in a streaming fashion.
  - **Why Streaming?**: `perf_results.json` files can be very large (10MB+).
    Reading the entire file into memory before parsing would be inefficient
    and could lead to high memory usage. Streaming allows processing the
    JSON array element by element.
  - **Implementation**:
    1.  It first expects and consumes the opening `[` of the JSON array.
    2.  It then iterates while `decoder.More()` is true, decoding each
        element into a `singleEntry` struct.
    3.  `singleEntry` is a union-like struct that can hold different types
        of objects found in the JSON (histograms, generic sets, date ranges,
        related name maps). This is determined by checking fields like
        `Name` (present for histograms) or `Type`.
    4.  If an entry is a histogram (`entry.Name != ""`), it's converted to
        `TraceKey` and `Histogram` via
        `histogramRaw.asTraceKeyAndHistogram`. This conversion involves
        looking up GUIDs from the histogram's `Diagnostics` map in a locally
        maintained `metadata` map (`md`).
    5.  Other entry types (`GenericSet`, `DateRange`, `RelatedNameMap`) are
        stored in the `md` map, keyed by their `GUID`, so they can be
        referenced by histograms later in the stream.
    6.  Parsed histograms are merged into `pr.Histograms`. If a `TraceKey`
        already exists, sample values are appended.
    7.  Finally, it consumes the closing `]` of the JSON array.
  - **Aggregation**: The `Histogram` type provides methods for common
    aggregations (Min, Max, Mean, Stddev, Sum, Count). `AggregationMapping`
    provides a convenient way to access these aggregation functions by
    string keys, which is used by downstream consumers like the ingestion
    module.
  - **Legacy `UnmarshalJSON`**: An `UnmarshalJSON` method exists, which
    reads the entire byte slice into memory. This is less efficient and
    marked for deprecation in favor of `NewResults`.

### Data Ingestion Preparation (`ingest/`)

This submodule focuses on transforming the parsed `PerfResults` into the
`format.Format` structure required by the Perf ingestion system.

- **`json.go` (`ConvertPerfResultsFormat`)**:

  - **Why**: The raw `PerfResults` structure is not directly ingestible. It
    needs to be reshaped.
  - **How**:

  * Iterates through each `(TraceKey, Histogram)` pair in the input
    `PerfResults`.
  * For each pair, it creates a `format.Result`. The `Key` map within
    `format.Result` is populated from `TraceKey` fields (chart, unit, story,
    arch, os).
  * The `Measurements` map within `format.Result` is populated by calling
    `toMeasurement` on the `Histogram`.
  * `toMeasurement` iterates through `perfresults.AggregationMapping`,
    applying each aggregation function to the histogram's samples. Each
    resulting aggregation (e.g., "max", "mean") becomes a
    `format.SingleMeasurement` with the aggregation type as its `Value` and
    the computed metric as its `Measurement`.
  * The final `format.Format` object includes the version, commit hash
    (`GitHash`), and any provided headers and links.

- **`gcs.go`**:

  - **Why**: Provides utilities for determining the correct Google Cloud
    Storage (GCS) path where the transformed JSON files should be stored.
    This is based on conventions used by the Perf ingestion system.
  - **How**:
  - `convertPath`: Constructs a GCS path like
    `gs://<bucket>/ingest/<time_path>/<build_info_path>/<benchmark>`.
  - `convertTime`: Formats a `time.Time` into `YYYY/MM/DD/HH` (UTC).
  - `convertBuildInfo`: Formats `BuildInfo` into
    `<MachineGroup>/<BuilderName>`. It defaults `MachineGroup` to
    "ChromiumPerf" and `BuilderName` to "BuilderNone" if they are empty.
  - `isInternal`: Determines if the results are internal or public based on
    the `BuilderName`. It checks against a list of known external bot
    configurations (`pinpoint/go/bot_configs`). If not found, it defaults to
    internal. This determines whether `PublicBucket` (`chrome-perf-public`)
    or `InternalBucket` (`chrome-perf-non-public`) is used.

## Key Components and Files

- **`perf_loader.go`**: Orchestrates the loading of performance results from
  Buildbucket. `NewLoader().LoadPerfResults()` is the main entry point.
- **`buildbucket.go`**: Handles interaction with the Buildbucket API to fetch
  build metadata. Defines `BuildInfo`.
- **`swarming.go`**: Handles interaction with the Swarming API to find child
  tasks and their CAS outputs.
- **`rbecas.go`**: Handles interaction with RBE-CAS to download and parse
  `perf_results.json` files. Defines `RBEPerfLoader`.
- **`perf_results_parser.go`**: Parses the content of `perf_results.json`
  files. Defines `PerfResults`, `TraceKey`, `Histogram`, and the streaming
  `NewResults` parser.
- **`ingest/json.go`**: Transforms parsed `PerfResults` into the
  `format.Format` structure for ingestion.
- **`ingest/gcs.go`**: Provides utilities to determine GCS paths for storing
  transformed results.
- **`cli/main.go`**: A command-line interface utility that uses the
  `perfresults` library to fetch results for a given Buildbucket ID and
  outputs them as JSON files in the ingestion format. This serves as a
  practical example and a tool for ad-hoc data retrieval.
- **`testdata/`**: Contains JSON files used for replaying HTTP and gRPC
  interactions during tests (`*.json`, `*.rpc`), and sample
  `perf_results.json` files for parser testing. `replay_test.go` sets up the
  replay mechanism.

## Workflows

### Primary Workflow: Loading Perf Results from Buildbucket

```
User/System --Buildbucket ID--> perf_loader.LoadPerfResults()
  |
  +--> buildbucket.findBuildInfo() --PRPC call--> Buildbucket API
  |      (Returns BuildInfo: Swarming Task ID, Git Revision, Machine Group, etc.)
  |
  +--> swarming.findChildTaskIds() --PRPC call--> Swarming API (using Parent Task ID)
  |      (Returns list of Child Swarming Task IDs)
  |
  +--> swarming.findTaskCASOutputs() --PRPC calls--> Swarming API (for each Child Task ID)
  |      (Returns list of CASReference objects)
  |
  (Error if CAS instances differ for CASReferences)
  |
  +--> rbecas.RBEPerfLoader.LoadPerfResults() (with list of CASReferences)
       |
       +--> For each CASReference:
       |    |
       |    +--> rbecas.fetchPerfDigests() --RBE SDK calls--> RBE-CAS
       |    |      (Returns map of benchmark_name to digest of perf_results.json)
       |    |
       |    +--> For each (benchmark_name, digest):
       |         |
       |         +--> rbecas.loadPerfResult() --RBE SDK call (ReadBlob)--> RBE-CAS
       |         |      |
       |         |      +--> perf_results_parser.NewResults() (Parses JSON stream)
       |         |             (Returns PerfResults object for this file)
       |         |
       |         +--> (Merge with existing PerfResults for the same benchmark_name)
       |
       (Returns map[benchmark_name]*PerfResults and BuildInfo)
```

### CLI Workflow: Fetching and Converting Perf Results

```
CLI User --Build ID, Output Dir--> cli/main.main()
  |
  +--> perfresults.NewLoader().LoadPerfResults(Build ID)
  |      (Executes the Primary Workflow described above)
  |      (Returns BuildInfo, map[benchmark]*PerfResults)
  |
  +--> For each (benchmark, perfResult) in results:
       |
       +--> ingest.ConvertPerfResultsFormat(perfResult, buildInfo.GetPosition(), headers, links)
       |      (Transforms PerfResults to ingest.Format)
       |
       +--> Marshal ingest.Format to JSON
       |
       +--> Write JSON to output file: <outputDir>/<benchmark>_<BuildID>.json
       |
       +--> Print output filename to stdout
```

### Temporal Worker (Placeholder)

The `workflows/worker/main.go` file sets up a Temporal worker. Currently, it's a
basic skeleton that initializes a worker and connects to a Temporal server. It
doesn't register any specific activities or workflows from the `perfresults`
module itself. Its presence suggests an intention to integrate `perfresults`
functionalities into Temporal workflows in the future, possibly for automated
ingestion or processing tasks. The worker itself is a generic Temporal worker
setup.

## Testing Strategy

The module employs a robust testing strategy:

- **Unit Tests**: Each Go file generally has a corresponding `_test.go` file
  with unit tests for its specific logic. For example,
  `perf_results_parser_test.go` tests the JSON parsing, and
  `buildbucket_test.go` tests `BuildInfo` logic.
- **Replay Testing (`replay_test.go`, `testdata/`)**:
  - **Why**: Directly calling external services (Buildbucket, Swarming,
    RBE-CAS) in tests makes them slow, flaky, and dependent on external
    state. Replay testing records actual interactions once and then
    "replays" them during subsequent test runs.
  - **How**:
  - HTTP interactions (with Buildbucket and Swarming PRPC servers) are
    replayed using `cloud.google.com/go/httpreplay`. Recorded interactions
    are stored as `.json` files in `testdata/`.
  - gRPC interactions (with RBE-CAS) are replayed using
    `cloud.google.com/go/rpcreplay`. Recorded interactions are stored as
    gzipped `.rpc` files in `testdata/`.
  - A command-line flag (`-record_path`) controls whether tests run in
    replay mode (reading from `testdata/`) or record mode (writing new
    replay files to the specified path). This allows updating replay files
    when external APIs change or new test cases are needed.
  - `setupReplay()` and `newRBEReplay()` in `replay_test.go` are helper
    functions that configure the HTTP client and RBE client for either
    recording or replaying.
- **Test Data (`testdata/perftest/`)**: Contains various `perf_results.json`
  files (e.g., `full.json`, `empty.json`, `merged.json`) to test different
  scenarios for the `perf_results_parser.go`. This ensures the parser
  correctly handles different valid and edge-case inputs.
- **Example Usage as Test (`cli/main.go`)**: The CLI itself serves as an
  integration test for the core loading and conversion logic. Its tests
  (`perf_loader_test.go` for example) often use the replay mechanism to test
  the end-to-end flow from Build ID to parsed `PerfResults`.

This combination ensures both isolated unit correctness and reliable integration
testing without external dependencies during typical test runs.

# Module: /go/perfserver

The `perfserver` module serves as the central executable for the Perf
performance monitoring system. It consolidates various essential components into
a single command-line tool, simplifying deployment and management. The primary
goal is to provide a unified entry point for running the web UI, data ingestion
processes, regression detection, and maintenance tasks. This approach avoids the
complexity of managing multiple separate services and their configurations.

The module leverages the `urfave/cli` library to define and manage sub-commands,
each corresponding to a distinct functional area of Perf. This design allows for
clear separation of concerns while maintaining a single binary. Configuration
for each sub-command is handled through flags, with the `config` package
providing structured types for these flags.

Key components and their responsibilities:

- **`main.go`**: This is the entry point of the `perfserver` executable.

  - **Why**: It orchestrates the initialization and execution of the
    different Perf sub-systems.
  - **How**: It defines a `cli.App` with several sub-commands:

  - **`frontend`**: This sub-command launches the main web user interface
    for Perf.

    - **Why**: To provide users with a visual way to explore performance
      data, configure alerts, and view regressions.
    - **How**: It initializes and runs the `frontend` component (from
      `//perf/go/frontend`). Configuration is passed via
      `config.FrontendFlags`. The `frontend` component itself handles
      serving HTTP requests and rendering the UI.

  - **`maintenance`**: This sub-command starts background maintenance tasks.

    - **Why**: Certain operations, like data cleanup, schema migrations,
      or periodic recalculations, are necessary for the long-term health
      and efficiency of the Perf system. These tasks often need to be run
      as singletons to avoid conflicts.
    - **How**: It initializes and runs the `maintenance` component (from
      `//perf/go/maintenance`). It first validates the instance
      configuration (using `//perf/go/config/validate`) and then starts
      the maintenance routines. Prometheus metrics are exposed for
      monitoring.

  - **`ingest`**: This sub-command runs the data ingestion process.

        - **Why**: To continuously import performance data from various
          sources (e.g., build artifacts, test results) and populate the
          central data store (TraceStore).
        - **How**: It initializes and runs the ingestion process logic (from
          `//perf/go/ingest/process`). Similar to `maintenance`, it validates
          the instance configuration. It supports parallel ingestion for
          improved throughput. Prometheus metrics are also exposed.
        - Data Ingestion Workflow: `Configured Sources --> [Ingest Process]

    --Parses/Validates--> [TraceStore] | Handles incoming files
    Populates data`

  - **`cluster`**: This sub-command runs the regression detection process.

        - **Why**: To automatically analyze incoming performance data against
          configured alerts and identify significant performance regressions.
        - **How**: Interestingly, this sub-command also utilizes the
          `frontend.New` and `f.Serve()` mechanism, similar to the `frontend`
          sub-command. This suggests that the regression detection logic might
          be tightly coupled with or exposed through the same underlying
          service framework as the main UI, potentially for sharing
          configuration or common infrastructure. It uses
          `config.FrontendFlags` but specifically for clustering-related
          settings (indicated by `AsCliFlags(true)`).
        - Regression Detection Workflow: `[TraceStore] --New Data--> [Cluster

    Process] --Applies Alert Rules--> [Alerts/Notifications] ^ | |
    Identifies Regressions +-------------------------+`

  - **`markdown`**: A utility sub-command to generate Markdown documentation
    for `perfserver` itself.

    - **Why**: To provide up-to-date command-line help in a portable
      format.
    - **How**: It uses the `ToMarkdown()` method provided by the
      `urfave/cli` library.

  - **Logging**: The `Before` hook in the `cli.App` configures `sklog` to
    output logs to standard output, ensuring that operational messages from
    any sub-command are visible.

  - **Configuration Loading**: For sub-commands like `ingest` and
    `maintenance`, instance configuration is loaded from a specified file
    (`ConfigFilename` flag) and validated using `//perf/go/config/validate`.
    The database connection string can be overridden via a command-line
    flag.

  - **Metrics**: The `ingest` and `maintenance` sub-commands initialize
    Prometheus metrics, allowing for monitoring of their operational health
    and performance.

The design emphasizes modularity by delegating the core logic of each function
(UI, ingestion, clustering, maintenance) to dedicated packages
(`//perf/go/frontend`, `//perf/go/ingest/process`, `//perf/go/maintenance`).
`perfserver` acts as the conductor, parsing command-line arguments, loading
appropriate configurations, and invoking the correct sub-system. This structure
makes the overall Perf system more maintainable and easier to understand, as
each component has a well-defined responsibility.

# Module: /go/pinpoint

The `/go/pinpoint` module provides a Go client for interacting with the Pinpoint
service, which is part of Chromeperf. Pinpoint is a performance testing and
analysis tool used to identify performance regressions and improvements. This
client enables other Go applications within the Skia infrastructure to
programmatically trigger Pinpoint jobs.

**Core Functionality:**

The primary purpose of this module is to abstract the complexities of making
HTTP requests to the Pinpoint API. It handles authentication, request
formatting, and response parsing. This allows other services to easily initiate
two main types of Pinpoint jobs:

1.  **Bisect Jobs:** These jobs are used to identify the specific commit that
    caused a performance regression or improvement between two given git
    revisions. The client constructs the appropriate URL and parameters for the
    `pinpointURL` endpoint.
2.  **Try Jobs (A/B Testing):** These jobs compare the performance of a base
    commit (or patch) against an experimental commit (or patch). This is
    particularly useful for evaluating the performance impact of a pending code
    change. The client uses the `pinpointLegacyURL` for these types of jobs.

**Design Decisions and Implementation Choices:**

- **Separate Endpoints for Bisect and Try Jobs:** The Pinpoint service has
  distinct API endpoints for creating bisect jobs (`pinpointURL`) and legacy
  try jobs (`pinpointLegacyURL`). The client reflects this by having separate
  methods (`CreateBisect` and `CreateTryJob`) and corresponding request URL
  builder functions (`buildBisectRequestURL` and `buildTryJobRequestURL`).
  This design choice directly maps to the underlying Pinpoint API structure,
  making it clear which type of job is being created.
- **URL Parameter Encoding:** Both types of Pinpoint jobs are initiated via
  HTTP GET requests where all parameters are encoded in the URL query string.
  The `buildBisectRequestURL` and `buildTryJobRequestURL` functions are
  responsible for constructing these URLs by populating `url.Values` and then
  encoding them. This is a direct consequence of how the Pinpoint API is
  designed.
- **Authentication:** The client utilizes Google's default token source for
  authentication (`google.DefaultTokenSource`) with the
  `auth.ScopeUserinfoEmail` scope. This is a standard approach for
  service-to-service authentication within the Google Cloud ecosystem,
  ensuring secure communication with the Pinpoint API.
- **Metrics Collection:** The client integrates with `go/metrics2` to track
  the number of times bisect and try jobs are called and the number of times
  these calls fail. This is crucial for monitoring the reliability and usage
  of the Pinpoint integration.
- **Error Handling:** The module uses `go/skerr` for wrapping errors. This
  provides more context to errors, making debugging easier. For example, if a
  Pinpoint request fails, the HTTP status code and response body are included
  in the error message.
- **Dependency on `pinpoint/go/bot_configs`:** For try jobs, the `target`
  parameter is required by the Pinpoint API. This `target` is derived from the
  `Configuration` (bot) and `Benchmark` using the
  `bot_configs.GetIsolateTarget` function. This indicates a specific
  configuration setup for running the performance tests.
- **`test_path` Parameter for Bisect Jobs:** The Pinpoint API requires a
  `test_path` parameter for bisect jobs. This parameter is constructed by
  joining several components like "ChromiumPerf", configuration, benchmark,
  chart, and story. This specific formatting is a legacy requirement of the
  Chromeperf API.
- **Mandatory `bug_id` for Bisect Jobs:** The Pinpoint API mandates the
  `bug_id` parameter for bisect jobs. If not provided by the caller, the
  client defaults it to `"null"`. This reflects a specific constraint of the
  upstream service.
- **`tags` Parameter:** Both job types include a `tags` parameter set to
  `{"origin":"skia_perf"}`. This helps in tracking and filtering jobs
  originating from the Skia infrastructure within the Pinpoint system.

**Key Components/Files:**

- **`pinpoint.go`:** This is the sole Go file in the module and contains all
  the logic.
  - **`Client` struct:** Represents the Pinpoint client. It holds the
    authenticated `http.Client` and counters for metrics.
  - **`New()` function:** The constructor for the `Client`. It initializes
    the HTTP client with appropriate authentication.
  - **`CreateLegacyTryRequest` and `CreateBisectRequest` structs:** Define
    the structure of the data required to create try jobs and bisect jobs,
    respectively. These fields directly map to the parameters expected by
    the Pinpoint API.
  - **`CreatePinpointResponse` struct:** Defines the structure of the JSON
    response from Pinpoint, which includes the `JobID` and `JobURL`.
  - **`CreateTryJob()` method:**
  - Takes a `CreateLegacyTryRequest` and a `context.Context`.
  - Calls `buildTryJobRequestURL` to construct the request URL.
  - Makes an HTTP POST request (though parameters are in URL, Pinpoint
    endpoint expects POST for creation) to the `pinpointLegacyURL`.
  - Parses the JSON response into a `CreatePinpointResponse`.
  - Handles errors and increments metrics.
  - **`CreateBisect()` method:**
  - Similar to `CreateTryJob()`, but takes a `CreateBisectRequest`.
  - Calls `buildBisectRequestURL`.
  - Makes an HTTP POST request to the `pinpointURL`.
  - Parses the response and handles errors/metrics.
  - **`buildTryJobRequestURL()` function:**
  - Takes a `CreateLegacyTryRequest`.
  - Validates required fields like `Benchmark` and `Configuration`.
  - Retrieves the `target` using `bot_configs.GetIsolateTarget`.
  - Populates `url.Values` with all relevant parameters from the request,
    including hardcoded values like `comparison_mode` and `tags`.
  - Returns the fully formed URL string.
  - **`buildBisectRequestURL()` function:**
  - Takes a `CreateBisectRequest`.
  - Populates `url.Values` with parameters from the request.
  - Sets a default value for `bug_id` if not provided.
  - Constructs the `test_path` parameter based on available request fields.
  - Includes the `tags` parameter.
  - Returns the fully formed URL string.

**Key Workflows:**

1.  **Creating a Bisect Job:**

    ```
    Application Code                   go/pinpoint.Client                    Pinpoint API
    ----------------                   ------------------                    ------------
    1. CreateBisectRequest data ---->
    2. Calls client.CreateBisect() -->
                                       3. buildBisectRequestURL()
                                          (constructs URL with params)
                                       4. HTTP POST to pinpointURL -------->
                                                                             5. Processes request
                                                                             6. Returns JSON response
                                       <----------------------------------- 7. Receives HTTP response
                                       8. Parses JSON into
                                          CreatePinpointResponse
    <--------------------------------- 9. Returns CreatePinpointResponse
    ```

2.  **Creating a Try Job (A/B Test):**

    ```
    Application Code                   go/pinpoint.Client                         Pinpoint API (Legacy)
    ----------------                   ------------------                         ---------------------
    1. CreateLegacyTryRequest data ->
    2. Calls client.CreateTryJob() -->
                                       3. buildTryJobRequestURL()
                                          (gets 'target' from bot_configs,
                                           constructs URL with params)
                                       4. HTTP POST to pinpointLegacyURL ----->
                                                                                  5. Processes request
                                                                                  6. Returns JSON response
                                       <---------------------------------------- 7. Receives HTTP response
                                       8. Parses JSON into
                                          CreatePinpointResponse
    <--------------------------------- 9. Returns CreatePinpointResponse
    ```

# Module: /go/pivot

## Pivot Module Documentation

### High-Level Overview

The `pivot` module provides functionality analogous to pivot tables in
spreadsheets or `GROUP BY` operations in SQL. Its primary purpose is to
aggregate and summarize trace data within a `DataFrame` based on specified
grouping criteria and operations. This allows users to transform raw trace data
into more insightful, summarized views, facilitating comparisons and analysis
across different dimensions of the data. For example, one might want to compare
the performance of 'arm' architecture machines against 'intel' architecture
machines by summing or averaging their respective performance metrics.

### Design and Implementation

The core of the `pivot` module revolves around the `Request` struct and the
`Pivot` function.

**`Request` Struct:**

The `Request` struct encapsulates the parameters for a pivot operation. It
defines:

- **`GroupBy`**: A slice of strings representing the parameter keys to group
  the traces by. This is the fundamental dimension along which the data will
  be aggregated. For instance, if `GroupBy` is `["arch"]`, all traces with the
  same 'arch' value will be grouped together.
- **`Operation`**: An `Operation` type (e.g., `Sum`, `Avg`, `Geo`) that
  specifies how the values within each group of traces should be combined.
  This operation is applied to each point in the traces within a group,
  resulting in a new, summarized trace for that group.
- **`Summary`**: An optional slice of `Operation` types. If provided, these
  operations are applied to the _resulting_ traces from the `GroupBy` step.
  Each `Summary` operation generates a single value (a column in the final
  output if viewed as a table) for each grouped trace. If `Summary` is empty,
  the output is a `DataFrame` where each row is a summarized trace (suitable
  for plotting).

**`Pivot` Function Workflow:**

The `Pivot` function executes the aggregation and summarization process. Here's
a breakdown of its key steps and the reasoning behind them:

1.  **Input Validation (`req.Valid()`):**

    - **Why:** To ensure the request is well-formed before proceeding with
      potentially expensive computations. This prevents errors due to missing
      `GroupBy` keys or invalid `Operation` or `Summary` values.
    - **How:** It checks if `GroupBy` is non-empty and if the specified
      `Operation` and `Summary` operations are among the predefined valid
      operations (`AllOperations`).

2.  **Initialization and Grouping Structure (`groupedTraceSets`):**

    - **Why:** To efficiently organize traces into their respective groups. A
      map is used where keys are the group identifiers (e.g., ",arch=arm,")
      and values are `types.TraceSet` containing traces belonging to that
      group.
    - **How:**
      - It pre-populates `groupedTraceSets` by determining all possible
        unique combinations of values for the `GroupBy` keys present in the
        input `DataFrame`'s `ParamSet`. This is done using
        `df.ParamSet.CartesianProduct(req.GroupBy)`. This pre-population
        ensures that even groups with no matching traces are considered,
        although they will be filtered out later if they remain empty.
      - It then iterates through each trace in the input `DataFrame`
        (`df.TraceSet`).
      - For each trace, it extracts the relevant parameter values specified
        in `req.GroupBy` to form a `groupKey` using `groupKeyFromTraceKey`.
        This function ensures that only traces containing _all_ the
        `GroupBy` keys contribute to a group. If a trace is missing a
        `GroupBy` key, it's ignored.
      - The trace is then added to the `types.TraceSet` associated with its
        `groupKey` in `groupedTraceSets`.

    ```
    Input DataFrame (df.TraceSet)
            |
            v
    For each traceID, trace in df.TraceSet:
      Parse traceID into params
      groupKey = groupKeyFromTraceKey(params, req.GroupBy)
      If groupKey is valid:
        Add trace to groupedTraceSets[groupKey]
            |
            v
    Grouped Traces (groupedTraceSets)
    ```

3.  **Applying the GroupBy Operation:**

    - **Why:** To perform the primary aggregation based on the
      `req.Operation`.
    - **How:**
      - It iterates through the `groupedTraceSets`.
      - For each non-empty group, it applies the `groupByOperation` function
        corresponding to `req.Operation` (obtained from `opMap`) to the
        `types.TraceSet` of that group. The `opMap` is a crucial design
        choice, mapping `Operation` constants to their respective
        implementation functions (one for grouping traces, another for
        summarizing single traces). This provides a clean and extensible way
        to manage different aggregation functions.
      - The result of this operation is a single summarized trace for that
        group, which is stored in the `ret.TraceSet` of the new `DataFrame`.
      - Context cancellation (`ctx.Err()`) is checked periodically to allow
        for early termination if the operation is cancelled.

    ```
    Grouped Traces (groupedTraceSets)
            |
            v
    For each groupID, traces in groupedTraceSets:
      If len(traces) > 0:
        summarizedTrace = opMap[req.Operation].groupByOperation(traces)
        ret.TraceSet[groupID] = summarizedTrace
            |
            v
    DataFrame with GroupBy Applied (ret)
    ```

4.  **Building ParamSet for the Result:**

    - **Why:** The resulting `DataFrame` needs its own `ParamSet` reflecting
      the new structure where trace keys only contain the `GroupBy`
      parameters.
    - **How:** `ret.BuildParamSet()` is called.

5.  **Applying Summary Operations (Optional):**

    - **Why:** To further reduce the data into single summary values per group
      if `req.Summary` is specified. This is useful for generating tabular
      summaries rather than plots.
    - **How:**
      - If `req.Summary` is empty, the original `DataFrame`'s `Header` is
        used for the new `DataFrame`, and the function returns. The result
        is a `DataFrame` of summarized traces.
      - If `req.Summary` is not empty:
      - It iterates through each summarized trace in `ret.TraceSet`.
      - For each trace, it creates a new `types.Trace` (called
        `summaryValues`) whose length is equal to the number of `Summary`
        operations.
      - For each `Operation` in `req.Summary`, it applies the corresponding
        `summaryOperation` function (from `opMap`) to the current grouped
        trace. The result is stored in `summaryValues`.
      - The original summarized trace in `ret.TraceSet[groupKey]` is
        replaced with `summaryValues`.
      - The `Header` of the `ret` `DataFrame` is rebuilt. Each column in the
        header now corresponds to one of the `Summary` operations, with
        offsets from 0 to `len(req.Summary) - 1`.

    ```
    DataFrame with GroupBy Applied (ret)
            |
            v
    If len(req.Summary) > 0:
      For each groupKey, trace in ret.TraceSet:
        summaryValues = new Trace of length len(req.Summary)
        For i, op in enumerate(req.Summary):
          summaryValues[i] = opMap[op].summaryOperation(trace)
        ret.TraceSet[groupKey] = summaryValues
      Adjust ret.Header to match Summary operations
            |
            v
    Final Pivoted DataFrame (ret)
    ```

**Operations (`Operation` type and `opMap`):**

The module defines a set of standard operations like `Sum`, `Avg`, `Geo`, `Std`,
`Count`, `Min`, `Max`.

- **Why:** To provide common aggregation methods.
- **How:**
  - Each `Operation` is a string constant.
  - The `opMap` is a map where each `Operation` key maps to an
    `operationFunctions` struct. This struct holds two function pointers:
  - `groupByOperation`: Takes a `types.TraceSet` (a group of traces) and
    returns a single aggregated `types.Trace`. These functions are typically
    sourced from the `go/calc` module.
  - `summaryOperation`: Takes a single `[]float32` (a trace) and returns a
    single `float32` summary value. These functions are typically sourced
    from `go/vec32` or defined locally (like `stdDev`).
  - This design makes it easy to add new operations by defining the constant
    and adding corresponding entries to `opMap` with the appropriate
    implementation functions.

**Error Handling:**

- **Why:** To provide clear feedback on invalid inputs or issues during
  processing.
- **How:** The `Pivot` function returns an error if `req.Valid()` fails or if
  an error occurs during grouping (e.g., a `GroupBy` key is not found in the
  `ParamSet` of the input DataFrame). Context cancellation is also handled,
  allowing long-running pivot operations to be interrupted. Errors are wrapped
  using `skerr.Wrap` to provide context.

### Key Components and Files

- **`pivot.go`**: This is the main file containing all the logic for the pivot
  functionality.

  - **`Request` struct**: Defines the parameters for a pivot operation. Its
    design allows for flexible grouping and summarization.
  - **`Operation` type and constants**: Define the set of available
    aggregation operations.
  - **`opMap` variable**: A critical data structure mapping `Operation`
    types to their respective implementation functions for both grouping and
    summarizing. This is the heart of how different operations are
    dispatched.
  - **`Pivot` function**: The primary public function that performs the
    pivot operation. Its step-by-step process of grouping, applying the main
    operation, and then optionally applying summary operations is central to
    its functionality.
  - **`groupKeyFromTraceKey` function**: A helper function responsible for
    constructing the group identifier for each trace based on the `GroupBy`
    keys. It handles cases where a trace might not have all the required
    keys.
  - **`Valid()` method on `Request`**: Ensures that the pivot request is
    well-formed before processing begins.

- **`pivot_test.go`**: Contains unit tests for the `pivot` module.

  - **Why:** To ensure the correctness and robustness of the pivot logic
    under various scenarios, including valid inputs, invalid inputs, edge
    cases (like empty groups or traces not matching any group), and context
    cancellation.
  - **How:** It uses the `testify` assertion library and defines test cases
    that cover different aspects of the `Request` validation,
    `groupKeyFromTraceKey` logic, and the `Pivot` function itself with
    various combinations of `Operation` and `Summary` settings. The
    `dataframeForTesting()` helper function provides a consistent dataset
    for testing.

This module is designed to be a general-purpose tool for transforming and
understanding large datasets of traces by allowing users to aggregate data along
arbitrary dimensions and apply various statistical operations.

# Module: /go/progress

## Module: /go/progress

The `/go/progress` module provides a mechanism for tracking the progress of
long-running tasks on the backend and exposing this information to the UI. This
is crucial for user experience in applications where operations like data
queries or complex computations can take a significant amount of time. Without
progress tracking, users might perceive the application as unresponsive or
encounter timeouts.

### Why: The Need for Asynchronous Task Monitoring

Many backend operations, such as those initiated by API endpoints like
`/frame/start` or `/dryrun/start`, are asynchronous. The initial HTTP request
might return quickly, but the actual work continues in the background. This
module addresses the need to:

1.  **Provide Feedback:** Inform the user that a task is ongoing and how it's
    progressing.
2.  **Avoid Timeouts:** Prevent HTTP requests from timing out while waiting for
    a long task to complete. The UI can poll for updates instead of holding a
    connection open.
3.  **Communicate Complex State:** Allow tasks to report detailed, multi-stage
    progress information, not just a simple percentage. For example, a "dry run"
    might involve several distinct steps, each with its own status and relevant
    data.

### How: Design and Implementation

The core idea is to represent the state of a long-running task as a `Progress`
object. This object can be updated by the task as it executes. A `Tracker` then
manages multiple `Progress` objects, making them accessible via HTTP polling.

**Key Components:**

- **`progress.go`**: Defines the `Progress` interface and its concrete
  implementation `progress`.

  - **`Progress` Interface**: This is the central abstraction for a single
    long-running task.
  - **Why an interface?** It allows for potential future extensions or
    alternative implementations (e.g., different storage mechanisms for
    progress data if needed, though the current implementation is
    in-memory).
  - **Key Methods:**
    - `Message(key, value string)`: Allows the task to report arbitrary
      key-value string pairs. This is flexible enough to accommodate
      diverse progress information (e.g., current step, commit being
      processed, number of items filtered). If a key already exists, its
      value is updated.
    - `Results(interface{})`: Stores intermediate or final results of the
      task. The `interface{}` type allows any JSON-serializable data to be
      stored. This is useful for showing partial results or accumulating
      data incrementally.
    - `Error(string)`: Marks the task as failed and stores an error
      message.
    - `Finished()`: Marks the task as successfully completed.
    - `FinishedWithResults(interface{})`: Atomically sets the results and
      marks the task as finished. This is preferred over separate
      `Results()` and `Finished()` calls to avoid race conditions where
      the UI might poll between the two calls.
    - `Status() Status`: Returns the current status (`Running`,
      `Finished`, `Error`).
    - `URL(string)`: Sets the URL that the client should poll for further
      updates. This is typically set by the `Tracker`.
    - `JSON(w io.Writer) error`: Serializes the current progress state
      (status, messages, results, next URL) into JSON and writes it to the
      provided writer.
  - **`progress` struct (concrete implementation)**:
  - Uses a `sync.Mutex` to ensure thread-safe updates to its internal
    `SerializedProgress` state. This is critical because long-running tasks
    often execute in separate goroutines, and the `Progress` object might be
    accessed concurrently by the task updating its state and by the
    `Tracker` serving HTTP requests.
  - Maintains its state in a `SerializedProgress` struct, which is designed
    for easy JSON serialization.
  - **State Transitions:** A `Progress` object starts in the `Running`
    state. Once it transitions to `Finished` or `Error`, it becomes
    immutable. Any attempt to modify it (e.g., calling `Message()` or
    `Results()` again) will result in a panic. This design simplifies
    reasoning about the lifecycle of a task's progress.
  - **`SerializedProgress` struct**: Defines the JSON structure sent to the
    client. It includes the `Status`, an array of `Message` (key-value
    pairs), the `Results` (if any), and the `URL` for the next poll.
  - **`Status` enum**: `Running`, `Finished`, `Error`.

- **`tracker.go`**: Defines the `Tracker` interface and its concrete
  implementation `tracker`.

  - **`Tracker` Interface**: Manages a collection of `Progress` objects.
  - `Add(prog Progress)`: Registers a new `Progress` object with the
    tracker. The tracker assigns a unique ID to this progress and sets its
    polling URL.
  - `Handler(w http.ResponseWriter, r *http.Request)`: An HTTP handler
    function that clients use to poll for progress updates. It extracts the
    progress ID from the request URL, retrieves the corresponding `Progress`
    object, and sends its JSON representation.
  - `Start(ctx context.Context)`: Starts a background goroutine for periodic
    cleanup of completed tasks from the cache.
  - **`tracker` struct (concrete implementation)**:
  - **`lru.Cache`**: Uses a Least Recently Used (LRU) cache
    (`github.com/hashicorp/golang-lru`) to store `cacheEntry` objects.
    - **Why LRU?** To prevent unbounded memory growth if many tasks are
      tracked. Older, completed tasks are eventually evicted.
  - **`basePath`**: A string prefix for the polling URLs (e.g.,
    `/_/status/`). Each progress object gets a unique ID appended to this
    base path to form its polling URL.
  - **`cacheEntry` struct**: Wraps a `Progress` object and a `Finished`
    timestamp. The timestamp is used by the cleanup routine to determine
    when a completed task can be removed from the cache.
  - **Cleanup Mechanism**:
    - The `Start` method launches a goroutine that periodically calls
      `singleStep`.
    - `singleStep` iterates through the cache:
    - It updates the `Finished` timestamp in a `cacheEntry` when the
      corresponding `Progress` object transitions out of the `Running`
      state.
    - It removes entries from the cache if they have been in a `Finished`
      or `Error` state for longer than `cacheDuration` (currently 5
      minutes). This prevents the cache from holding onto completed tasks
      indefinitely.
    - This ensures that resources are eventually freed up while still
      allowing clients a reasonable window to fetch the final results of a
      completed task.
  - **UUIDs for IDs**: Uses `github.com/google/uuid` to generate unique IDs
    for each tracked `Progress`. This makes the polling URLs distinct and
    hard to guess.

### Key Workflows

1.  **Starting and Tracking a Long-Running Task:**

    ```
    Backend HTTP Handler (e.g., /api/start_long_task)
        |
        | 1. Create a new Progress object:
        |    prog := progress.New()
        |
        | 2. Add it to the global Tracker instance:
        |    trackerInstance.Add(prog)  // Tracker sets prog.URL() internally
        |
        | 3. Respond to the initial HTTP request with the Progress JSON.
        |    // The client now has prog.URL() to poll.
        |    prog.JSON(w)
        |
        V
    Goroutine (executing the long-running task)
        |
        | 1. Periodically update progress:
        |    prog.Message("Step", "Processing item X")
        |    prog.Message("PercentComplete", "30%")
        |    prog.Results(partialData) // Optional: intermediate results
        |
        | 2. When finished:
        |    If error:
        |        prog.Error("Something went wrong")
        |    Else:
        |        prog.FinishedWithResults(finalData)
    ```

2.  **Client Polling for Updates:**

    ```
    Client (e.g., browser UI)
        |
        | 1. Receives initial response with prog.URL (e.g., /_/status/some-uuid)
        |
        | 2. Makes a GET request to prog.URL
        V
    Backend Tracker.Handler
        |
        | 1. Extracts "some-uuid" from the request path.
        |
        | 2. Looks up the Progress object in its cache using "some-uuid".
        |    If not found --> HTTP 404 Not Found
        |
        | 3. Calls prog.JSON(w) to send the current state.
        V
    Client
        |
        | 1. Receives JSON with current status, messages, results.
        |
        | 2. If Status is "Running", schedules another poll to prog.URL.
        |
        | 3. If Status is "Finished" or "Error", displays final results/error and stops polling.
    ```

3.  **Tracker Cache Management (Background Process):**

    ```
    Tracker.Start()
        |
        V
    Goroutine (periodic execution, e.g., every minute)
        |
        | Calls tracker.singleStep()
        |   |
        |   V
        |   Iterate through cache entries:
        |     - If Progress.Status() is not "Running" AND cacheEntry.Finished is zero:
        |         Set cacheEntry.Finished = now()
        |     - If cacheEntry.Finished is not zero AND now() > cacheEntry.Finished + cacheDuration:
        |         Remove entry from cache
        |     - Update metrics (numEntriesInCache)
        |
        V
    (Loop back to periodic execution)
    ```

This system provides a robust and flexible way to communicate the progress of
backend tasks to the user interface, improving the overall user experience for
operations that might otherwise seem opaque or unresponsive. The use of JSON for
data interchange makes it easy for web frontends to consume the progress
information.

# Module: /go/psrefresh

The `psrefresh` module is designed to manage and provide access to
`paramtools.ParamSet` instances, which are collections of key-value pairs
representing the parameters of traces in a performance monitoring system. The
primary goal is to efficiently retrieve and cache these parameter sets,
especially for frequently accessed queries, to reduce database load and improve
response times.

The module addresses the need for up-to-date parameter sets by periodically
fetching data from a trace store (represented by the `OPSProvider` interface).
It combines parameter sets from recent time intervals (tiles) to provide a
comprehensive view of available parameters.

A key challenge is handling potentially large and complex parameter sets. To
mitigate this, the module offers a caching layer (`CachedParamSetRefresher`).
This caching mechanism is configurable and can pre-populate caches (e.g., local
in-memory or Redis) with filtered parameter sets based on predefined query
levels. This pre-population significantly speeds up queries that match these
common filter patterns.

**Key Components and Responsibilities:**

- **`psrefresh.go`**:

  - Defines the core interfaces `OPSProvider` and `ParamSetRefresher`.
  - `OPSProvider`: Abstractly represents a source of ordered parameter sets
    (e.g., a trace data store). It provides methods to get the latest "tile"
    (a time-based segment of data) and the parameter set for a specific
    tile. This abstraction allows `psrefresh` to be independent of the
    underlying data storage implementation.
  - `ParamSetRefresher`: Defines the contract for components that can
    provide the full parameter set and parameter sets filtered by a query.
    It also includes a `Start` method to initiate the refresh process.
  - Implements `defaultParamSetRefresher`, which is the standard
    implementation of `ParamSetRefresher`.
  - **Why**: This struct is responsible for the fundamental logic of
    periodically fetching parameter sets from the `OPSProvider`. It merges
    parameter sets from a configurable number of recent tiles to create a
    comprehensive view.
  - **How**: It uses a background goroutine (`refresh`) that periodically
    calls `oneStep`. The `oneStep` method fetches the latest tile, then
    iterates backward through the configured number of previous tiles,
    retrieving and merging their parameter sets using
    `paramtools.ParamSet.AddParamSet`. The resulting merged set is then
    normalized and stored.
  - A `sync.Mutex` is used to protect concurrent access to the `ps`
    (paramtools.ReadOnlyParamSet) field, ensuring thread safety when
    `GetAll` is called.
  - `GetParamSetForQuery` delegates the actual filtering and counting of
    traces to a `dataframe.DataFrameBuilder`, demonstrating a separation of
    concerns.
  - `UpdateQueryValueWithDefaults` is a helper to automatically add default
    parameter selections to queries if configured, simplifying common query
    patterns.

- **`cachedpsrefresh.go`**:

  - Implements `CachedParamSetRefresher`, which wraps a
    `defaultParamSetRefresher` and adds a caching layer.
  - **Why**: To improve performance for common queries by avoiding repeated
    database lookups or expensive filtering operations. For frequently
    accessed subsets of data (e.g., specific benchmarks or configurations),
    retrieving pre-computed parameter sets from a cache is much faster.
  - **How**: It takes a `cache.Cache` instance (which could be local, Redis,
    etc.) and a `defaultParamSetRefresher`.
  - `PopulateCache`: This is a crucial method that proactively fills the
    cache. It uses the `QueryCacheConfig` (part of `config.QueryConfig`) to
    determine which levels of parameter sets to cache.
    - It starts by getting the full parameter set from the underlying
      `psRefresher`.
    - It then iterates through configured "Level 1" parameter keys and
      their specified values. For each combination, it performs a
      `PreflightQuery` (via the `dfBuilder`) to get the filtered parameter
      set and the count of matching traces.
    - Both the filtered parameter set (as a string) and the count are
      stored in the cache using distinct keys.
    - If "Level 2" keys and values are configured, it recursively calls
      `populateChildLevel` to cache parameter sets for combinations of
      Level 1 and Level 2 parameters.
    - The cache keys are generated by `paramSetKey` and `countKey`,
      ensuring a consistent naming scheme.
  - `GetParamSetForQuery`: When a query is made,
    `getParamSetForQueryInternal` first tries to retrieve the result from
    the cache.
    - It determines the appropriate cache key based on the query
      parameters and the configured cache levels (`getParamSetKey`). It
      only attempts to serve from the cache if the query matches the
      configured cache levels (1 or 2 parameters, potentially adjusted for
      default parameters).
    - If a cache hit occurs, it reconstructs the `paramtools.ParamSet`
      from the cached string and retrieves the count.
    - If there's a cache miss or an error, it falls back to the underlying
      `psRefresher.GetParamSetForQuery`.
  - `StartRefreshRoutine`: This method starts a goroutine that periodically
    calls `PopulateCache` to keep the cached data fresh.

**Key Workflows:**

1.  **Initialization and Periodic Refresh (Default Refresher):**

    ```
    NewDefaultParamSetRefresher(opsProvider, ...) -> pf
    pf.Start(refreshPeriod)
      -> pf.oneStep() // Initial fetch
           -> opsProvider.GetLatestTile() -> latestTile
           -> LOOP (numParamSets times):
                -> opsProvider.GetParamSet(tile) -> individualPS
                -> mergedPS.AddParamSet(individualPS)
                -> tile = tile.Prev()
           -> mergedPS.Normalize()
           -> pf.ps = mergedPS.Freeze()
      -> GO pf.refresh()
           -> LOOP (every refreshPeriod):
                -> pf.oneStep() // Subsequent fetches
    ```

2.  **Cache Population (Cached Refresher):**

    ```
    NewCachedParamSetRefresher(defaultRefresher, cacheImpl) -> cr
    cr.StartRefreshRoutine(cacheRefreshPeriod)
      -> cr.PopulateCache() // Initial population
           -> defaultRefresher.GetAll() -> fullPS
           -> // For each configured Level 1 key/value:
                -> qValues = {level1Key: [level1Value]}
                -> defaultRefresher.UpdateQueryValueWithDefaults(qValues) // If applicable
                -> query.New(qValues) -> lv1Query
                -> defaultRefresher.dfBuilder.PreflightQuery(ctx, lv1Query, fullPS) -> count, filteredPS
                -> psCacheKey = paramSetKey(qValues, [level1Key])
                -> cr.addToCache(ctx, psCacheKey, filteredPS.ToString(), count)
                -> // If Level 2 is configured:
                     -> cr.populateChildLevel(ctx, level1Key, level1Value, filteredPS, level2Key, level2Values)
                          -> // For each configured Level 2 value:
                               -> qValues = {level1Key: [level1Value], level2Key: [level2Value]}
                               -> ... (similar PreflightQuery and addToCache)
      -> GO LOOP (every cacheRefreshPeriod):
           -> cr.PopulateCache() // Subsequent cache refreshes
    ```

3.  **Querying with Cache:** `cr.GetParamSetForQuery(ctx, queryObj, queryValues)
-> cr.getParamSetForQueryInternal(ctx, queryObj, queryValues) ->
cr.getParamSetKey(queryValues) -> cacheKey, err -> IF cacheKey is valid AND
exists: -> cache.GetValue(ctx, cacheKey) -> cachedParamSetString ->
cache.GetValue(ctx, countKey(cacheKey)) -> cachedCountString ->
paramtools.FromString(cachedParamSetString) -> paramSet ->
strconv.ParseInt(cachedCountString) -> count -> RETURN count, paramSet, nil
-> ELSE (cache miss or invalid key for caching): ->
defaultRefresher.GetParamSetForQuery(ctx, queryObj, queryValues) -> count,
paramSet, err -> RETURN count, paramSet, err`

The use of `config.QueryConfig` and `config.Experiments` allows for
instance-specific tuning of caching behavior (which keys/values to pre-populate)
and handling of default parameters. The separation between
`defaultParamSetRefresher` and `CachedParamSetRefresher` promotes modularity,
allowing the caching layer to be optional or replaced with different caching
strategies if needed.

# Module: /go/redis

The `redis` module in Skia Perf is designed to manage interactions with Redis
instances, primarily to support and optimize the query UI. It leverages Redis
for caching frequently accessed data, thereby improving the responsiveness and
performance of the Perf frontend.

The core idea is to periodically fetch information about available Redis
instances within a Google Cloud Project and then interact with a specific,
configured Redis instance to store or retrieve cached data. This cached data
typically represents results of expensive computations or frequently requested
data points, like recent trace data for specific queries.

**Key Responsibilities and Components:**

- **`redis.go`**: This is the central file of the module.

  - **`RedisWrapper` interface**: Defines the contract for Redis-related
    operations. This abstraction allows for easier testing and potential
    future replacements of the underlying Redis client implementation. The
    key methods are:
  - `StartRefreshRoutine`: Initiates a background process (goroutine) that
    periodically discovers and interacts with the configured Redis instance.
  - `ListRedisInstances`: Retrieves a list of all Redis instances available
    within a specified GCP project and location.
  - **`RedisClient` struct**: This is the concrete implementation of the
    `RedisWrapper` interface.
  - It holds a `gcp_redis.CloudRedisClient` for interacting with the Google
    Cloud Redis API (e.g., listing instances).
  - It also has a reference to `tracestore.TraceStore`, which is likely used
    to fetch the data that needs to be cached in Redis.
  - The `tilesToCache` field suggests that the caching strategy might
    involve pre-calculating and storing "tiles" of data, which is a common
    pattern in Perf systems for displaying graphs over time.
  - **`NewRedisClient`**: The constructor for `RedisClient`.
  - **`StartRefreshRoutine`**: - **Why**: To ensure that Perf is always aware of the correct Redis
    instance to use and to periodically update the cache. Network
    configurations or instance details might change, and this routine
    helps adapt to such changes. - **How**: It takes a `refreshPeriod` and a `config.InstanceConfig`
    (which is actually `redis_client.RedisConfig` in the current
    implementation, indicating the target project, zone, and instance
    name). It then starts a goroutine that, at regular intervals defined
    by `refreshPeriod`:
    _ Calls `ListRedisInstances` to get all Redis instances in the
    configured project/zone.
    _ Iterates through the instances to find the one matching the
    `config.Instance` name. \* If the target instance is found, it calls `RefreshCachedQueries`.
    `[StartRefreshRoutine] | V (Goroutine - Ticks every 'refreshPeriod')
| V [ListRedisInstances] -> (GCP API Call) -> [List of Redis
Instances] | V (Find Target Instance by Name) | V (If Target Found)
[RefreshCachedQueries]`
  - **`ListRedisInstances`**:
    - **Why**: To discover available Redis instances within the specified
      GCP project and location. This is the first step before Perf can
      connect to and use a specific Redis instance.
    - **How**: It uses the `gcpClient` (an instance of
      `cloud.google.com/go/redis/apiv1.CloudRedisClient`) to make an API
      call to GCP to list instances under the given `parent` (e.g.,
      "projects/my-project/locations/us-central1"). It iterates through
      the results and returns a slice of `redispb.Instance` objects.
  - **`RefreshCachedQueries`**:
    - **Why**: This is the heart of the caching mechanism. Its purpose is
      to update the data stored in the target Redis instance. The specific
      data to be cached would depend on the needs of the Perf query UI.
    - **How**:
    * It establishes a connection to the specified Redis instance
      (`instance.Host` and `instance.Port`) using
      `github.com/redis/go-redis/v9`.
    * It acquires a mutex (`r.mutex.Lock()`) to prevent concurrent
      modifications to the cache or shared resources, though the current
      implementation only has placeholder logic.
    * The current implementation contains placeholder logic:
      - It attempts to `GET` a key named "FullPS".
      - It then `SET`s the key "FullPS" to the current time, with an
        expiration of 30 seconds.
    * **Future Work (as hinted by `TODO(wenbinzhang)` and
      `tilesToCache`)**: This method is expected to be expanded to:
      - Identify which queries or data segments are good candidates for
        caching.
      - Fetch the necessary data, potentially using the `traceStore`.
      - Store this data in Redis, likely with appropriate keys and
        expiration times. The `tilesToCache` parameter suggests it might
        pre-cache a certain number of recent "tiles" of trace data.

- **`mocks/RedisWrapper.go`**: This file contains a mock implementation of the
  `RedisWrapper` interface, generated by the `mockery` tool.

  - **Why**: To facilitate unit testing of components that depend on
    `RedisWrapper`. By using a mock, tests can simulate various Redis
    behaviors (e.g., successful connection, instance not found, errors)
    without needing an actual Redis instance or GCP connectivity.
  - **How**: It provides a `RedisWrapper` struct that embeds `mock.Mock`
    from the `testify` library. For each method in the `RedisWrapper`
    interface, there's a corresponding method in the mock that records calls
    and can be configured to return specific values or errors, allowing test
    authors to define expected interactions.

**Design Decisions and Rationale:**

- **Interface-based Design (`RedisWrapper`)**: Using an interface decouples
  the rest of the Perf system from the concrete Redis client implementation.
  This is good for:
  - **Testability**: As seen with the `mocks` package.
  - **Flexibility**: If Skia decides to switch to a different Redis client
    library or even a different caching technology in the future, the
    changes would be localized to the implementation of `RedisWrapper`
    without affecting its consumers.
- **Periodic Refresh Routine**: Instead of connecting to Redis on-demand for
  every operation or assuming a static configuration, the
  `StartRefreshRoutine` provides a more robust approach.
  - It handles potential changes in the Redis instance's availability or
    address.
  - It centralizes the logic for keeping the cache up-to-date.
- **Separation of Concerns**:
  - The module clearly separates GCP Redis instance management (listing
    instances via GCP API) from data interaction with a specific Redis
    instance (using a Redis client library like `go-redis`).
- **Use of Standard Libraries**:
  - `cloud.google.com/go/redis/apiv1` for GCP infrastructure management.
  - `github.com/redis/go-redis/v9` for standard Redis data operations. This
    ensures reliance on well-maintained and feature-rich libraries.

**Workflow: Cache Refresh Process**

The primary workflow driven by this module is the periodic refresh of cached
data:

```
System Starts
    |
    V
Initialize RedisClient (NewRedisClient)
    |
    V
Call StartRefreshRoutine
    |
    V
[Background Goroutine - Loop every 'refreshPeriod']
    |
    |--> 1. List GCP Redis Instances (ListRedisInstances)
    |       - Input: GCP project, location
    |       - Output: List of *redispb.Instance
    |
    |--> 2. Identify Target Redis Instance
    |       - Based on configuration (e.g., instance name)
    |
    |--> 3. If Target Instance Found: Refresh Cache (RefreshCachedQueries)
            |
            |--> a. Connect to Target Redis (using go-redis)
            |       - Host, Port from *redispb.Instance
            |
            |--> b. Determine data to cache (e.g., recent trace data for popular queries)
            |       - Likely involves `traceStore`
            |
            |--> c. Write data to Redis (SET commands)
            |       - Use appropriate keys and expiration times
            |
            |--> (Current placeholder: SET "FullPS" = current_time with 30s TTL)
```

This module provides the foundational components for integrating Redis as a
caching layer in Skia Perf, aiming to improve UI performance by serving
frequently requested data quickly from an in-memory store. The current
implementation focuses on instance discovery and has placeholder logic for the
actual caching, which is expected to be expanded based on Perf's specific
caching needs.

# Module: /go/regression

The `/go/regression` module is responsible for detecting, storing, and managing
performance regressions in Skia. It analyzes performance data over time,
identifies significant changes (regressions or improvements), and provides
mechanisms for triaging and tracking these changes.

**Core Functionality & Design:**

The primary goal is to automatically flag performance changes that might
indicate a problem or an unexpected improvement. This involves:

1.  **Data Analysis:** Analyzing time-series performance data (traces) across
    different commits.
2.  **Clustering:** Grouping similar traces together to identify patterns and
    changes affecting multiple tests or configurations.
3.  **Step Detection:** Identifying abrupt changes (steps) in performance
    metrics that signify a potential regression or improvement.
4.  **Alerting & Notification:** Informing relevant parties when a potential
    regression is detected.
5.  **Persistence:** Storing detected regressions and their triage status.

**Key Components & Files:**

- **`detector.go`**: This file contains the core logic for processing
  regression detection requests.

  - **Why:** It orchestrates the process of fetching data, applying
    clustering algorithms, and identifying regressions. It's designed to
    handle potentially large datasets and long-running analyses.
  - **How:** `ProcessRegressions` is the main entry point. It takes a
    `RegressionDetectionRequest` (which specifies the alert configuration
    and the time domain to analyze) and a `DetectorResponseProcessor`
    callback.
  - It can expand a single alert configuration with `GroupBy` parameters
    into multiple, more specific requests using
    `allRequestsFromBaseRequest`. This allows for targeted analysis of
    specific trace groups.
  - It iterates over data using `dfiter.DataFrameIterator`, which provides
    dataframes for analysis.
  - For each dataframe, it filters out traces with too much missing data
    (`tooMuchMissingData`) to ensure the reliability of the detection.
  - It applies a clustering algorithm (either K-Means via
    `clustering2.CalculateClusterSummaries` or individual StepFit via
    `StepFit`) to identify clusters of traces exhibiting similar behavior.
    The choice of K (number of clusters for K-Means) can be automatic or
    user-specified.
  - It generates `RegressionDetectionResponse` objects containing the
    cluster summaries and the relevant data frame. These responses are
    passed to the `DetectorResponseProcessor`.
  - Shortcuts for identified clusters are created using `shortcutFromKeys`
    for easier referencing.
  - **Workflow:** `RegressionDetectionRequest -> Expand (if GroupBy) ->
Multiple Requests | V For each Request: DataFrameIterator -> DataFrame
-> Filter Traces -> Apply Clustering (KMeans or StepFit) | | V V
Shortcut Creation <- ClusterSummaries -> DetectorResponseProcessor`

- **`regression.go`**: Defines the primary data structures for representing
  regressions and their triage status.

  - **Why:** Provides a standardized way to model regressions, including
    details about the low (performance degradation) and high (performance
    improvement) steps, the associated data frame, and triage information.
  - **How:**
  - `Regression`: The central struct holding `Low` and `High`
    `ClusterSummary` objects (from `clustering2`), the `FrameResponse` (data
    context), and `TriageStatus` for both low and high. It also includes
    fields for the newer `regression2` schema (like `Id`, `CommitNumber`,
    `AlertId`, `MedianBefore`, `MedianAfter`).
  - `TriageStatus`: Represents whether a regression is `Untriaged`,
    `Positive` (expected/acceptable), or `Negative` (a bug).
  - `AllRegressionsForCommit`: A container for all regressions found for a
    specific commit, keyed by the alert ID.
  - `Merge`: A method to combine information from two `Regression` objects,
    typically used when new data provides a more significant regression for
    an existing alert.

- **`types.go`**: Defines the `Store` interface, which abstracts the
  persistence layer for regressions.

  - **Why:** Decouples the regression detection logic from the specific
    database implementation, allowing for different storage backends.
  - **How:** The `Store` interface specifies methods for:
  - `Range`: Retrieving regressions within a commit range.
  - `SetHigh`/`SetLow`: Storing newly detected high/low regressions.
  - `TriageHigh`/`TriageLow`: Updating the triage status of regressions.
  - `Write`: Bulk writing of regressions.
  - `GetRegressionsBySubName`, `GetByIDs`: Retrieving regressions based on
    subscription names or specific IDs (primarily for the `regression2`
    schema).
  - `GetOldestCommit`, `GetRegression`: Utility methods for fetching
    specific data.
  - `DeleteByCommit`: Removing regressions associated with a commit.

- **`stepfit.go`**: Implements an alternative regression detection strategy
  that analyzes each trace individually using step fitting.

  - **Why:** Useful when `GroupBy` is used in an alert, or when K-Means
    clustering is not the desired approach. It focuses on finding
    significant steps in individual time series.
  - **How:** The `StepFit` function iterates through each trace in the input
    `DataFrame`.
  - For each trace, it calls `stepfit.GetStepFitAtMid` to determine if
    there's a significant step (low or high) around the midpoint of the
    trace.
  - If an "interesting" step is found (based on `stddevThreshold` and
    `interesting` parameters), the trace is added to either the `low` or
    `high` `ClusterSummary`.
  - The `low` and `high` summaries collect all traces that show a downward
    or upward step, respectively.
  - Parametric summaries (`ParamSummaries`) are generated for the keys
    within these clusters.

- **`fromsummary.go`**: Provides a utility function to convert a
  `RegressionDetectionResponse` into a `Regression` object.

  - **Why:** Bridges the output of the detection process with the structured
    `Regression` type used for storage and display.
  - **How:** `RegressionFromClusterResponse` takes a
    `RegressionDetectionResponse`, an `alerts.Alert` configuration, and a
    `perfgit.Git` instance.
  - It identifies the commit at the midpoint of the response's data frame.
  - It iterates through the `ClusterSummary` objects in the response.
  - If a cluster's step point matches the midpoint commit and meets the
    alert's criteria (minimum number of traces, direction), it populates the
    `Low` or `High` fields of the `Regression` object. It prioritizes the
    regression with the largest absolute magnitude if multiple are found.

**Submodules:**

- **`continuous/` (`continuous.go`)**: Manages the continuous, background
  detection of regressions.

  - **Why:** Ensures that new performance data is promptly analyzed for
    regressions as it arrives, without requiring manual intervention.
  - **How:**
  - `Continuous` struct: Holds dependencies like `perfgit.Git`,
    `regression.Store`, `alerts.ConfigProvider`, `notify.Notifier`, etc.
  - `Run()`: The main entry point, which starts either event-driven or
    polling-based regression detection.
  - **Event-Driven (`RunEventDrivenClustering`)**:
    - Listens to Pub/Sub messages from `FileIngestionTopicName` indicating
      new data ingestion (`ingestevents.IngestEvent`).
    - For each event, it identifies matching alert configurations using
      `getTraceIdConfigsForIngestEvent` (which calls
      `matchingConfigsFromTraceIDs`).
    - `matchingConfigsFromTraceIDs` refines alert queries if `GroupBy` is
      present to be more specific to the incoming trace.
    - It then calls `ProcessAlertConfig` (or `ProcessAlertConfigForTraces`
      if `StepFitGrouping` is used) for each matching config and the
      specific traces.
  - **Polling (`RunContinuousClustering`)**:
    - Periodically (defined by `pollingDelay`), fetches all alert
      configurations using `buildConfigAndParamsetChannel`.
    - Shuffles the configs to distribute the load if multiple detectors
      are running.
    - Calls `ProcessAlertConfig` for each configuration.
  - `ProcessAlertConfig()`:
    - Sets the current config being processed.
    - Optionally performs a "smoketest" for `GroupBy` alerts to ensure the
      query is valid and returns data.
    - Calls `regression.ProcessRegressions` to perform the actual
      detection.
    - The `clusterResponseProcessor` (which is `reportRegressions`) is
      called with the detection results.
  - `reportRegressions()`:
    - For each detected regression (`RegressionDetectionResponse`), it
      determines the commit and previous commit details.
    - It checks if the regression meets the alert criteria (direction,
      minimum number).
    - It calls `updateStoreAndNotification` to persist the regression and
      send notifications.
  - `updateStoreAndNotification()`:
    - Checks if the regression already exists in the `regression.Store`.
    - If new, it stores the regression (using `store.SetLow` or
      `store.SetHigh`) and sends a notification via
      `notifier.RegressionFound`. The notification ID is stored with the
      regression.
    - If existing, but the direction of the regression has changed (e.g.,
      was low, now also high), it updates the store and the notification
      using `notifier.UpdateNotification`.
    - If existing and the direction is the same, it only updates the
      store.
  - **Key Decision:** The system supports both event-driven (preferred for
    responsiveness) and polling-based detection (as a fallback or for
    periodic full checks). The choice is controlled by the
    `EventDrivenRegressionDetection` flag.
  - **Workflow (Event-Driven):** `Pub/Sub Message (New Data) -> Decode
IngestEvent -> Get Matching Alert Configs | V For each (Config, Matched
Traces): ProcessAlertConfig -> regression.ProcessRegressions | V
reportRegressions -> updateStoreAndNotification | | V V Store Notifier`

- **`migration/` (`migrator.go`)**: Handles the data migration from an older
  `regressions` table schema to the newer `regressions2` schema.

  - **Why:** Facilitates schema evolution without data loss. The newer
    schema (`Regression2Schema`) aims to store regression data more
    granularly, typically one row per detected step (high or low), rather
    than combining high and low for the same commit/alert into a single JSON
    blob.
  - **How:**
  - `RegressionMigrator`: Contains instances of the legacy
    `sqlregressionstore.SQLRegressionStore` and the new
    `sqlregression2store.SQLRegression2Store`.
  - `RunPeriodicMigration`: Sets up a ticker to periodically run
    `RunOneMigration`.
  - `RunOneMigration` / `migrateRegressions`:
    - Fetches a batch of unmigrated regressions from the legacy store
      (`legacyStore.GetRegressionsToMigrate`).
    - For each legacy `Regression` object:
    - It begins a database transaction.
    - It populates fields specific to the `regression2` schema (e.g.,
      `Id`, `PrevCommitNumber`, `MedianBefore`, `MedianAfter`,
      `IsImprovement`, `ClusterType`) if they are not already present from
      the legacy data. This is crucial as the
      `sqlregression2store.WriteRegression` expects these.
    - The `sqlregression2store.WriteRegression` function might split a
      single legacy `Regression` object (if it has both `High` and `Low`
      components) into two separate entries in the `Regressions2` table,
      one for `HighClusterType` and one for `LowClusterType`.
    - It then marks the corresponding row in the legacy `Regressions`
      table as migrated using `legacyStore.MarkMigrated`, storing the new
      regression ID.
    - Commits the transaction. If any step fails, the transaction is
      rolled back.
  - **Key Decision:** Migration is performed in batches and within
    transactions to ensure atomicity and prevent data duplication or loss
    during the migration process.

- **`sqlregressionstore/`**: Implements the `regression.Store` interface using
  a generic SQL database. This is the _older_ SQL storage mechanism.

  - **Why:** Provides a persistent storage solution for regressions
    identified by the detection system.
  - **How:**
  - `SQLRegressionStore`: The main struct, holding a database connection
    pool (`pool.Pool`) and prepared SQL statements. It supports different
    SQL dialects (e.g., CockroachDB via `statements`, Spanner via
    `spannerStatements`).
  - The schema (`sqlregressionstore/schema/RegressionSchema.go`) typically
    stores one row per `(commit_number, alert_id)` pair. The actual
    `regression.Regression` object (which might contain both high and low
    details, along with the frame) is serialized into a JSON string and
    stored in a `regression TEXT` column.
  - `readModifyWrite`: A core helper function that encapsulates the common
    pattern of reading a `Regression` from the DB, allowing a callback to
    modify it, and then writing it back. This is done within a transaction
    to prevent lost updates. If `mustExist` is true, it errors if the
    regression isn't found; otherwise, it creates a new one.
  - `SetHigh`/`SetLow`: Use `readModifyWrite` to update the `High` or `Low`
    part of the JSON-serialized `Regression` object. They also update the
    triage status to `Untriaged` if it was previously `None`.
  - `TriageHigh`/`TriageLow`: Use `readModifyWrite` to update the
    `HighStatus` or `LowStatus` within the JSON-serialized `Regression`.
  - `GetRegressionsToMigrate`: Fetches regressions that haven't been
    migrated to the `regression2` schema.
  - `MarkMigrated`: Updates a row to indicate it has been migrated, storing
    the new `regression_id` from the `regression2` table.
  - **Limitation:** Storing the entire `Regression` object as JSON can make
    querying for specific aspects of the regression (e.g., only high
    regressions, or regressions with a specific triage status) less
    efficient and more complex. This is one of the motivations for the
    `sqlregression2store`.

- **`sqlregression2store/`**: Implements the `regression.Store` interface
  using a newer SQL schema (`Regressions2`).

  - **Why:** Addresses limitations of the older `sqlregressionstore` by
    storing regression data in a more normalized and queryable way.
  - **How:**
  - `SQLRegression2Store`: The main struct.
  - Schema (`sqlregression2store/schema/Regression2Schema.go`): Designed to
    store each regression step (high or low) as a separate row. Key columns
    include `id` (UUID, primary key), `commit_number`, `prev_commit_number`,
    `alert_id`, `creation_time`, `median_before`, `median_after`,
    `is_improvement`, `cluster_type` (e.g., "high", "low"),
    `cluster_summary` (JSONB), `frame` (JSONB), `triage_status`, and
    `triage_message`.
  - `writeSingleRegression`: The core writing function. It takes a
    `regression.Regression` object and writes its relevant parts (either
    high or low, but not both in the same DB row) to the `Regressions2`
    table.
  - `convertRowToRegression`: Converts a database row from `Regressions2`
    back into a `regression.Regression` object. Depending on the
    `cluster_type` in the row, it populates either the `High` or `Low` part
    of the `Regression` object.
  - `SetHigh`/`SetLow`:
    - These methods now interact with `updateBasedOnAlertAlgo`.
    - `updateBasedOnAlertAlgo`: This function is crucial. It considers the
      `Algo` type of the alert (`KMeansGrouping` vs. `StepFitGrouping`).
    - For `KMeansGrouping`, it expects to potentially update an existing
      regression for the same `(commit_number, alert_id)` as new data
      might refine the cluster. It uses `readModifyWriteCompat` to achieve
      this.
    - For `StepFitGrouping` (individual trace analysis), it generally
      expects to create a _new_ regression entry if one doesn't exist for
      the exact frame, avoiding updates to pre-existing ones unless it's
      truly a new detection.
    - The `updateFunc` passed to `updateBasedOnAlertAlgo` populates the
      necessary fields in the `regression.Regression` object (e.g.,
      setting `r.High` or `r.Low`, and calling
      `populateRegression2Fields`).
  - `populateRegression2Fields`: This helper populates the fields specific
    to the `Regressions2` schema (like `PrevCommitNumber`, `MedianBefore`,
    `MedianAfter`, `IsImprovement`) from the `ClusterSummary` and
    `FrameResponse` within the `Regression` object.
  - `WriteRegression` (used by migrator): If a legacy `Regression` object
    has both `High` and `Low` components, this function splits it and calls
    `writeSingleRegression` twice, creating two rows in `Regressions2`.
  - `Range`: When retrieving regressions, if multiple rows from
    `Regressions2` correspond to the same `(commit_number, alert_id)` (e.g.,
    one for high, one for low), it merges them back into a single
    `regression.Regression` object for compatibility with how the rest of
    the system might expect the data.
  - **Key Improvement:** Storing regression components (high/low) as
    separate rows with dedicated columns for medians, triage status, etc.,
    allows for much more efficient and direct SQL querying compared to
    parsing JSON in the older store.

**Overall Workflow Example (Simplified):**

1.  **Continuous Detection (`continuous.go`):**
    - New data arrives (e.g., via Pub/Sub).
    - `Continuous` identifies relevant `alerts.Alert` configurations.
    - `ProcessAlertConfig` is called.
2.  **Regression Processing (`detector.go`):**
    - `ProcessRegressions` fetches data, builds `DataFrame`s.
    - Clustering (KMeans or `stepfit.go`) is applied.
    - `RegressionDetectionResponse`s are generated.
3.  **Reporting & Storing (`continuous.go` calls back into `regression`
    store):**
    - `reportRegressions` processes these responses.
    - `updateStoreAndNotification` interacts with a `regression.Store`
      implementation (e.g., `sqlregression2store.go`):
      - Checks if the regression is new or an update.
      - Calls `SetLow` or `SetHigh` on the store.
      - The store (`sqlregression2store`) writes the data to the
        `Regressions2` table, potentially creating a new row or updating an
        existing one based on the alert's algorithm type.
      - A notification might be sent.

The system is designed to be modular, with interfaces like `regression.Store`
and `alerts.ConfigProvider` allowing for flexibility in implementation details.
The migration path from `sqlregressionstore` to `sqlregression2store` highlights
the evolution towards a more structured and queryable data model for
regressions.

# Module: /go/samplestats

The `samplestats` module is designed to perform statistical analysis on sets of
performance data, specifically to identify significant changes between two
sample sets, often referred to as "before" and "after" states. This is crucial
for detecting regressions or improvements in performance metrics over time or
across different code versions.

The core functionality revolves around comparing these two sets of samples for
each trace (a unique combination of parameters identifying a specific test or
metric). It calculates various statistical metrics for each set and then employs
statistical tests to determine if the observed differences are statistically
significant.

**Key Design Choices and Implementation Details:**

- **Statistical Significance:** The module uses p-values to determine
  significance. A user-configurable alpha level (defaulting to 0.05) acts as
  the threshold. If the calculated p-value for a trace is below this alpha,
  the change is considered significant.
- **Choice of Statistical Tests:** The module offers two common statistical
  tests:
  - **Mann-Whitney U Test (default):** This is a non-parametric test used to
    compare two independent samples. It's often preferred when the data
    doesn't necessarily follow a normal distribution.
  - **Two Sample Welch's t-test:** This parametric test is used to compare
    the means of two independent samples, particularly when their variances
    might be unequal. The choice allows users to select the most appropriate
    test based on the characteristics of their data.
- **Outlier Removal:** An optional Interquartile Range Rule (IQRR) can be
  applied to remove outliers from the sample data before calculating
  statistics. This helps in reducing the influence of extreme values that
  might skew the results. The decision to make this optional acknowledges that
  outlier removal isn't always desired or appropriate.
- **Delta Calculation:** For changes deemed significant, the module calculates
  the percentage change in the mean between the "before" and "after" samples.
  If a change isn't significant, the delta is reported as NaN (Not a Number),
  clearly distinguishing it from actual zero-percentage changes.
- **Configurability:** The `Config` struct provides a centralized way to
  control the analysis process. This includes setting the alpha level,
  choosing the statistical test, enabling outlier removal, and deciding
  whether to include all traces in the output or only those with significant
  changes. This configurability makes the module adaptable to various analysis
  needs.
- **Result Structure:** The `Result` struct encapsulates the outcome of the
  analysis, including a list of `Row` structs (one per trace) and a count of
  skipped traces. Each `Row` contains the trace identifier, its parameters,
  the calculated metrics for both "before" and "after" samples, the percentage
  delta, the p-value, and any informational notes (e.g., errors during
  statistical test calculation). This structured output facilitates further
  processing or display of the results.
- **Sorting:** The results can be sorted based on different criteria, with the
  default being by the calculated `Delta`. This allows users to quickly
  identify the most impactful changes. The `Order` type and functions like
  `ByName`, `ByDelta`, and `Reverse` provide a flexible sorting mechanism.

**Responsibilities and Key Components:**

- **`analyze.go`**: This is the heart of the module.

  - **`Analyze` function**: This is the primary entry point. It takes the
    `Config` and two maps of samples (`before` and `after`, where keys are
    trace IDs and values are `parser.Samples`).
  - It iterates through all unique trace IDs present in either the "before"
    or "after" sets.
  - For each trace, it retrieves the corresponding samples, skipping the
    trace if data isn't present in both sets.
  - It calls `calculateMetrics` (from `metrics.go`) for both "before" and
    "after" samples.
  - Based on the `Config.Test` setting, it performs either the Mann-Whitney
    U test or the Two Sample Welch's t-test using functions from the
    `github.com/aclements/go-moremath/stats` library.
  - It compares the resulting p-value with the configured alpha level. If `p
< alpha`, it calculates the percentage `Delta` between the means.
    Otherwise, `Delta` is `NaN`.
  - It constructs a `Row` struct with all the calculated information.
  - It optionally filters out rows where no significant change was detected
    if `Config.All` is false.
  - Finally, it sorts the resulting `Row`s based on `Config.Order` (or by
    `Delta` if no order is specified) using the `Sort` function from
    `sort.go`.
  - It returns a `Result` struct containing the list of `Row`s and the count
    of skipped traces.
  - **`Config` struct**: Defines the parameters that control the analysis,
    such as `Alpha` for p-value cutoff, `Order` for sorting, `IQRR` for
    outlier removal, `All` for including all results, and `Test` for
    selecting the statistical test.
  - **`Result` struct**: Encapsulates the output of the `Analyze` function,
    holding the `Rows` of analysis data and the `Skipped` count.
  - **`Row` struct**: Represents the analysis results for a single trace,
    including its name, parameters, "before" and "after" `Metrics`, the
    percentage `Delta`, the `P` value, and any `Note`.

- **`metrics.go`**: This file is responsible for calculating basic statistical
  metrics from a given set of sample values.

  - **`calculateMetrics` function**: Takes a `Config` (primarily to check
    `IQRR`) and `parser.Samples`.
  - If `Config.IQRR` is true, it applies the Interquartile Range Rule to
    filter out outliers from `samples.Values`. The values within 1.5 \* IQR
    from the first and third quartiles are retained.
  - It then calculates the `Mean`, `StdDev` (standard deviation), and
    `Percent` (coefficient of variation: `StdDev / Mean * 100`) of the
    (potentially filtered) values.
  - It returns these calculated statistics in a `Metrics` struct, along with
    the (potentially filtered) `Values`.
  - **`Metrics` struct**: Holds the calculated `Mean`, `StdDev`, raw
    `Values` (after potential outlier removal), and `Percent` (coefficient
    of variation).

- **`sort.go`**: This file provides utilities for sorting the results (`Row`
  slices).

  - **`Order` type**: A function type `func(rows []Row, i, j int) bool`
    defining a less-than comparison for sorting `Row`s.
  - **`ByName` function**: An `Order` implementation that sorts rows
    alphabetically by `Row.Name`.
  - **`ByDelta` function**: An `Order` implementation that sorts rows by
    `Row.Delta`. It specifically places `NaN` delta values (insignificant
    changes) at the beginning.
  - **`Reverse` function**: A higher-order function that takes an `Order`
    and returns a new `Order` that represents the reverse of the input
    order.
  - **`Sort` function**: A convenience function that sorts a slice of `Row`s
    in place using `sort.SliceStable` and a given `Order`.

**Illustrative Workflow (Simplified `Analyze` Process):**

```
Input: before_samples, after_samples, config

For each trace_id in (before_samples keys + after_samples keys):
  If trace_id not in before_samples OR trace_id not in after_samples:
    Increment skipped_count
    Continue

  before_metrics = calculateMetrics(config, before_samples[trace_id])
  after_metrics  = calculateMetrics(config, after_samples[trace_id])

  If config.Test == UTest:
    p_value = MannWhitneyUTest(before_metrics.Values, after_metrics.Values)
  Else (config.Test == TTest):
    p_value = TwoSampleWelchTTest(before_metrics.Values, after_metrics.Values)

  alpha = config.Alpha (or defaultAlpha if config.Alpha is 0)

  If p_value < alpha:
    delta = ((after_metrics.Mean / before_metrics.Mean) - 1) * 100
  Else:
    delta = NaN
    If NOT config.All:
      Continue // Skip if not showing all results and change is not significant

  Add new Row{Name: trace_id, Delta: delta, P: p_value, ...} to results_list

Sort results_list using config.Order (or ByDelta by default)

Return Result{Rows: results_list, Skipped: skipped_count}
```

# Module: /go/sheriffconfig

The `sheriffconfig` module is responsible for managing configurations for Skia
Perf's anomaly detection and alerting system. These configurations, known as
"Sheriff Configs," are defined in Protocol Buffer format and are typically
stored in LUCI Config. This module handles fetching these configurations,
validating them, and transforming them into a format suitable for storage and
use by other Perf components, specifically the `alerts` and `subscription`
modules.

The core idea is to allow users to define rules for which performance metrics
they care about and how anomalies in those metrics should be detected and
handled. This provides a flexible and centralized way to manage alerting for a
large number of performance tests.

**Key Responsibilities and Components:**

- **Protocol Buffer Definitions (`/proto/v1`):**

  - This directory defines the structure of Sheriff Configurations using
    Protocol Buffers. This is the "source of truth" for what constitutes a
    valid configuration.
  - `sheriff_config.proto`: Defines the main messages like `SheriffConfig`,
    `Subscription`, `AnomalyConfig`, and `Rules`.
  - `SheriffConfig`: The top-level message, containing a list of
    `Subscription`s. This represents the entire set of alerting
    configurations for a Perf instance.
  - `Subscription`: Represents a user's or team's interest in a specific set
    of metrics. It includes details for creating bug reports (e.g., contact
    email, bug component, labels, priority, severity) and a list of
    `AnomalyConfig`s that define how to detect anomalies for the metrics
    covered by this subscription.
  - `AnomalyConfig`: Specifies the parameters for anomaly detection for a
    particular subset of metrics. This includes:
    - `Rules`: Define which metrics this `AnomalyConfig` applies to, using
      `match` and `exclude` patterns. These patterns are query strings
      (e.g., "master=ChromiumPerf&benchmark=Speedometer2").
    - Detection parameters: `step` (algorithm for step detection),
      `radius` (commits to consider), `threshold` (sensitivity),
      `minimum_num` (number of interesting traces to trigger an alert),
      `sparse` (handling of missing data), `k` (for K-Means clustering),
      `group_by` (for breaking down clustering), `direction` (up, down, or
      both), `action` (no action, triage, or bisect), and `algo`
      (clustering algorithm like StepFit or KMeans).
  - `Rules`: Contains lists of `match` and `exclude` strings. Match strings
    define positive criteria for selecting metrics, while exclude strings
    define negative criteria. The combination allows for precise targeting
    of metrics.
  - `sheriff_config.pb.go`: The Go code generated from
    `sheriff_config.proto`. This provides the Go structs and methods to work
    with these configurations programmatically.
  - `generate.go`: Contains `go:generate` directives used to regenerate
    `sheriff_config.pb.go` whenever `sheriff_config.proto` changes. This
    ensures the Go code stays in sync with the proto definition.

- **Validation (`/validate`):**

  - `validate.go`: This is crucial for ensuring the integrity and
    correctness of Sheriff Configurations before they are processed or
    stored. It performs a series of checks:
  - **Pattern Validation:** Ensures that `match` and `exclude` strings in
    `Rules` are well-formed query strings. It checks for valid regex if a
    value starts with `~`. It also enforces that exclude patterns only
    target a single key-value pair.
  - **AnomalyConfig Validation:** Ensures that each `AnomalyConfig` has at
    least one `match` pattern.
  - **Subscription Validation:** Verifies that essential fields like `name`,
    `contact_email`, `bug_component`, and `instance` are present. It also
    checks that each subscription has at least one `AnomalyConfig`.
  - **SheriffConfig Validation:** Ensures there's at least one
    `Subscription` and that all subscription names within a config are
    unique.
  - `DeserializeProto`: A helper function to convert a base64 encoded string
    (as typically retrieved from LUCI Config) into a `SheriffConfig`
    protobuf message.

- **Service (`/service`):**

  - `service.go`: This component orchestrates the process of fetching
    Sheriff Configurations from LUCI Config, processing them, and storing
    them in the database.
  - **`New` function:** Initializes the `sheriffconfigService`, taking
    dependencies like a database connection pool (`sql.Pool`),
    `subscription.Store`, `alerts.Store`, and a `luciconfig.ApiClient`. If
    no `luciconfig.ApiClient` is provided, it creates one.
  - **`ImportSheriffConfig` method:** This is the main entry point for
    importing configurations.

  * It uses the `luciconfig.ApiClient` to fetch configurations from a
    specified LUCI Config path (e.g., "skia-sheriff-configs.cfg").
  * For each fetched configuration file content:
    - It calls `processConfig`.
  * It then inserts all derived `subscription_pb.Subscription` objects into
    the `subscriptionStore` and all `alerts.SaveRequest` objects into the
    `alertStore` within a single database transaction. This ensures
    atomicity – either all changes are saved, or none are.

  - **`processConfig` method:**

  * Deserializes the raw configuration content (string) into a
    `pb.SheriffConfig` protobuf message using `prototext.Unmarshal`.
  * Validates the deserialized `pb.SheriffConfig` using
    `validate.ValidateConfig`.
  * Iterates through each `pb.Subscription` in the config:
    - It filters subscriptions based on the `instance` field, only
      processing those matching the service's configured instance (e.g.,
      "chrome-internal"). This allows multiple Perf instances to share a
      config file but only import relevant subscriptions.
    - It calls `makeSubscriptionEntity` to convert the `pb.Subscription`
      into a `subscription_pb.Subscription` (the format used by the
      `subscription` module).
    - **Revision Check:** Crucially, it checks if a subscription with the
      same name and revision already exists in the `subscriptionStore`. If
      it does, it means this specific version of the subscription has
      already been imported, so it's skipped. This prevents redundant
      database writes and processing if the LUCI config file hasn't
      actually changed for that subscription.
    - If the subscription is new or has a new revision, it calls
      `makeSaveRequests` to generate `alerts.SaveRequest` objects for each
      alert defined within that subscription.

  - **`makeSubscriptionEntity` function:** Transforms a `pb.Subscription`
    (from Sheriff Config proto) into a `subscription_pb.Subscription` (for
    the `subscription` datastore), mapping fields and applying default
    priorities/severities if not specified.
  - **`makeSaveRequests` function:**

  * Iterates through each `pb.AnomalyConfig` within a `pb.Subscription`.
  * For each `match` rule within the `pb.AnomalyConfig.Rules`:
    - Calls `buildQueryFromRules` to construct the actual query string
      that will be used to select metrics for this alert.
    - Calls `createAlert` to create an `alerts.Alert` object, populating
      it with parameters from the `pb.AnomalyConfig` and the parent
      `pb.Subscription`.
    - Wraps the `alerts.Alert` in an `alerts.SaveRequest` along with the
      subscription name and revision.

  - **`createAlert` function:** Populates an `alerts.Alert` struct. This
    involves:
  - Mapping enum values from the Sheriff Config proto (e.g.,
    `AnomalyConfig_Step`, `AnomalyConfig_Direction`, `AnomalyConfig_Action`,
    `AnomalyConfig_Algo`) to their corresponding internal types used by the
    `alerts` module (e.g., `alerts.Direction`,
    `types.RegressionDetectionGrouping`, `types.StepDetection`,
    `types.AlertAction`). This is done using maps like `directionMap`,
    `clusterAlgoMap`, etc.
  - Applying default values for parameters like `radius`, `minimum_num`,
    `sparse`, `k`, `group_by` if they are not explicitly set in the
    `AnomalyConfig`.
  - **`buildQueryFromRules` function:** Constructs a canonical query string
    from a `match` string and a list of `exclude` strings. It parses them as
    URL query parameters, combines them (with `!` for excludes), sorts the
    parts alphabetically, and joins them with `&`. This ensures that
    equivalent rules always produce the same query string.
  - **`getPriorityFromProto` and `getSeverityFromProto` functions:** Convert
    the enum values for priority and severity from the proto definition to
    the integer values expected by the `subscription` module, applying
    defaults if the proto value is "unspecified."
  - **`StartImportRoutine` and `ImportSheriffConfigOnce`:** Provide
    functionality to periodically fetch and import configurations, making
    the system self-updating when LUCI configs change.

**Workflow: Importing a Sheriff Configuration**

```
LUCI Config Change (e.g., new revision of skia-sheriff-configs.cfg)
      |
      v
Sheriffconfig Service (triggered by timer or manual call)
      |
      |--- 1. luciconfigApiClient.GetProjectConfigs("skia-sheriff-configs.cfg") --> Fetches raw config content + revision
      |
      v
For each config file content:
      |
      |--- 2. processConfig(configContent, revision)
      |      |
      |      |--- 2a. prototext.Unmarshal(configContent) --> pb.SheriffConfig
      |      |
      |      |--- 2b. validate.ValidateConfig(pb.SheriffConfig) --> Error or OK
      |      |
      |      v
      |      For each pb.Subscription in pb.SheriffConfig:
      |            |
      |            |--- 2c. If subscription.Instance != service.Instance --> Skip
      |            |
      |            |--- 2d. subscriptionStore.GetSubscription(name, revision) --> ExistingSubscription?
      |            |
      |            |--- 2e. If ExistingSubscription == nil (new or updated):
      |            |      |
      |            |      |--- makeSubscriptionEntity(pb.Subscription, revision) --> subscription_pb.Subscription
      |            |      |
      |            |      |--- makeSaveRequests(pb.Subscription, revision)
      |            |      |     |
      |            |      |     v
      |            |      |     For each pb.AnomalyConfig in pb.Subscription:
      |            |      |           |
      |            |      |           v
      |            |      |           For each matchRule in pb.AnomalyConfig.Rules:
      |            |      |                 |
      |            |      |                 |--- buildQueryFromRules(matchRule, excludeRules) --> queryString
      |            |      |                 |
      |            |      |                 |--- createAlert(queryString, pb.AnomalyConfig, pb.Subscription, revision) --> alerts.Alert
      |            |      |                 |
      |            |      |                 ---> Collect alerts.SaveRequest
      |            |      |
      |            |      ---> Collect subscription_pb.Subscription
      |
      v
Database Transaction (BEGIN)
      |
      |--- 3. subscriptionStore.InsertSubscriptions(collected_subscriptions)
      |
      |--- 4. alertStore.ReplaceAll(collected_save_requests)
      |
Database Transaction (COMMIT or ROLLBACK)
```

This module acts as a critical bridge, translating human-readable (and
machine-parsable via proto) alerting definitions into the concrete data
structures used by Perf's backend alerting and subscription systems. The
validation step is key to preventing malformed configurations from breaking the
alerting pipeline. The revision checking mechanism ensures efficiency by only
processing changes.

# Module: /go/shortcut

The `shortcut` module provides functionality for creating, storing, and
retrieving "shortcuts". A shortcut is essentially a named list of trace keys.
These trace keys typically represent specific performance metrics or
configurations. The primary purpose of shortcuts is to provide a convenient way
to refer to a collection of traces with a short, memorable identifier, rather
than having to repeatedly specify the full list of keys. This is particularly
useful for sharing links to specific views in the Perf UI or for programmatic
access to predefined sets of performance data.

The core component is the `Store` interface, defined in `shortcut.go`. This
interface abstracts the underlying storage mechanism, allowing different
implementations to be used (e.g., in-memory for testing, SQL database for
production). The key operations defined by the `Store` interface are:

- `Insert`: Adds a new shortcut to the store. It takes an `io.Reader`
  containing the shortcut data (typically JSON) and returns a unique ID for
  the shortcut.
- `InsertShortcut`: Similar to `Insert`, but takes a `Shortcut` struct
  directly.
- `Get`: Retrieves a shortcut given its ID.
- `GetAll`: Returns a channel that streams all stored shortcuts. This is
  useful for tasks like data migration.
- `DeleteShortcut`: Removes a shortcut from the store.

A `Shortcut` itself is a simple struct containing a slice of strings, where each
string is a trace key.

The generation of shortcut IDs is handled by the `IDFromKeys` function. This
function takes a `Shortcut` struct, sorts its keys alphabetically (to ensure
that the order of keys doesn't affect the ID), and then computes an MD5 hash of
the concatenated keys. A prefix "X" is added to this hash for historical
reasons, maintaining compatibility with older systems. This deterministic ID
generation ensures that the same set of keys will always produce the same
shortcut ID.

Workflow for creating and retrieving a shortcut:

1.  **Creation**: `Client Code` ---(JSON data or Shortcut struct)--->
    `Store.Insert` or `Store.InsertShortcut` `Store` ---(Generates ID using
    IDFromKeys, marshals to JSON if needed)---> `Underlying Storage (e.g., SQL
DB)` `Underlying Storage` ---> `Store` ---(Returns Shortcut ID)---> `Client
Code`

2.  **Retrieval**: `Client Code` ---(Shortcut ID)---> `Store.Get` `Store`
    ---(Queries by ID)---> `Underlying Storage (e.g., SQL DB)` `Underlying
Storage` ---(Returns stored JSON or data)---> `Store` `Store` ---(Unmarshals
    to Shortcut struct, sorts keys)---> `Client Code` (receives Shortcut struct)

The `sqlshortcutstore` subdirectory provides a concrete implementation of the
`Store` interface using an SQL database (specifically designed for CockroachDB,
as indicated by test setup and migration references). The `sqlshortcutstore.go`
file contains the logic for interacting with the database, including SQL
statements for inserting, retrieving, and deleting shortcuts. Shortcut data is
stored as JSON strings in the database. The schema for the `Shortcuts` table is
implicitly defined by the SQL statements and further clarified in
`sqlshortcutstore/schema/schema.go`, which defines a `ShortcutSchema` struct
mirroring the table structure (though this struct is primarily for documentation
or ORM-like purposes and not directly used in the raw SQL interaction in
`sqlshortcutstore.go`).

Testing is a significant aspect of this module:

- `shortcut_test.go` contains unit tests for the `IDFromKeys` function,
  ensuring its correctness and deterministic behavior.
- `shortcuttest` provides a suite of common tests (`InsertGet`,
  `GetNonExistent`, `GetAll`, `DeleteShortcut`) that can be run against any
  implementation of the `shortcut.Store` interface. This promotes consistency
  and ensures that different store implementations behave as expected. The
  `InsertGet` test, for example, verifies that a stored shortcut can be
  retrieved and that the keys are sorted upon retrieval, even if they were not
  sorted initially.
- `sqlshortcutstore_test.go` utilizes the tests from `shortcuttest` to
  validate the `SQLShortcutStore` implementation against a test database.
- `mocks/Store.go` provides a mock implementation of the `Store` interface,
  generated by the `mockery` tool. This is useful for testing components that
  depend on `shortcut.Store` without needing a real storage backend.

# Module: /go/sql

The `go/sql` module serves as the central hub for managing the SQL database
schema used by the Perf application. It defines the structure of the database
tables and provides utilities for schema generation, validation, and migration.
This module ensures that the application's database schema is consistent,
well-defined, and can evolve smoothly over time.

**Key Responsibilities and Components:**

- **Schema Definition (`schema.go`, `spanner/schema_spanner.go`):**

  - **Why:** These files contain the SQL `CREATE TABLE` statements that
    define the structure of all tables used by Perf. Having the schema
    defined in code (generated from Go structs) provides a single source of
    truth and allows for easier version control and programmatic
    manipulation.
  - **How:**
  - `schema.go`: Defines the schema for CockroachDB.
  - `spanner/schema_spanner.go`: Defines the schema for Spanner. Spanner has
    slightly different SQL syntax and features (e.g., `TTL INTERVAL`),
    necessitating a separate schema definition.
  - The schema is not written manually but is _generated_ by the `tosql`
    utility (see below). This ensures that the SQL schema accurately
    reflects the Go struct definitions in other modules (e.g.,
    `perf/go/alerts/sqlalertstore/schema`).
  - Along with the `CREATE TABLE` statements, these files also export slices
    of strings representing the column names for each table. This can be
    useful for constructing SQL queries programmatically.

- **Table Struct Definition (`tables.go`):**

  - **Why:** This file defines a Go struct `Tables` which aggregates all the
    individual table schema structs from various Perf sub-modules (like
    `alerts`, `anomalygroup`, `git`, etc.).
  - **How:** The `Tables` struct serves as the input to the `tosql` schema
    generator. By referencing schema structs from other modules, it ensures
    that the generated SQL schema is consistent with how data is represented
    and manipulated throughout the application. The `//go:generate`
    directives at the top of this file trigger the `tosql` utility to
    regenerate the schema files when necessary.

- **Schema Generation Utility (`tosql/main.go`):**

  - **Why:** Manually writing and maintaining complex SQL schemas is
    error-prone. This utility automates the generation of the SQL schema
    files (`schema.go` and `spanner/schema_spanner.go`) from the Go struct
    definitions.
  - **How:** It takes the `sql.Tables` struct (defined in `tables.go`) as
    input and uses the `go/sql/exporter` module to translate the Go struct
    tags and field types into corresponding SQL `CREATE TABLE` statements.
    It supports different SQL dialects (CockroachDB and Spanner) and can
    handle specific features like Spanner's TTL (Time To Live) for tables.
    The `schemaTarget` flag controls which database dialect is generated.

- **Expected Schema and Migration (`expectedschema/`):**

  - **Why:** As the application evolves, the database schema needs to
    change. This submodule manages schema migrations, ensuring that the live
    database can be updated to new versions without downtime or data loss.
    It also validates that the current database schema matches an expected
    version.
  - **How:**

  - **`embed.go`:** This file uses `go:embed` to embed JSON representations
    of the _current_ (`schema.json`, `schema_spanner.json`) and _previous_
    (`schema_prev.json`, `schema_prev_spanner.json`) expected database
    schemas. These JSON files are generated by the `exportschema` utility.
    `Load()` and `LoadPrev()` functions provide access to these deserialized
    schema descriptions.

  - **`migrate.go`:** This is the core of the schema migration logic.

    - It defines SQL statements (`FromLiveToNext`, `FromNextToLive`, and
      their Spanner equivalents) that describe how to upgrade the database
      from the "previous" schema version to the "next" (current) schema
      version, and how to roll back that change. **Crucially, schema
      changes must be backward and forward compatible** because during a
      deployment, old and new versions of the application might run
      concurrently.
    - `ValidateAndMigrateNewSchema` is the key function. It:

    * Loads the "next" (target) and "previous" expected schemas from the
      embedded JSON files.
    * Gets the _actual_ schema description from the live database.
    * Compares the actual schema with the previous and next expected
      schemas.
      - If `actual == next`, no migration is needed.
      - If `actual == prev` and `actual != next`, it executes the
        `FromLiveToNext` SQL statements to upgrade the database schema.
      - If `actual` matches neither `prev` nor `next`, it indicates an
        unexpected schema state and returns an error, preventing
        application startup. This is a critical safety check.

    - The migration process is designed to be run by a maintenance task
      during deployment. Old instances (frontend, ingesters) ->
      Maintenance task (runs `ValidateAndMigrateNewSchema`) -> New
      instances (frontend, ingesters)

    ```
    Deployment Starts
         |
         V
    Maintenance Task Runs
         |
         +------------------------------------+
         | Calls ValidateAndMigrateNewSchema  |
         +------------------------------------+
              |
              V
    Is schema == previous_expected_schema? --Yes--> Apply `FromLiveToNext` SQL
              | No                                     |
              V                                      V
    Is schema == current_expected_schema? ---Yes---> Migration Successful / No Action
              | No
              V
    Error: Schema mismatch! Halt.
              |
              V
    New Application Instances Start (if migration was successful)
    ```

  - **Test files (`migrate_test.go`, `migrate_spanner_test.go`):** These
    files contain unit tests to verify the schema migration logic for both
    CockroachDB and Spanner. They test scenarios where no migration is
    needed, migration is required, and the schema is in an invalid state.

- **Schema Export Utility (`exportschema/main.go`):**

  - **Why:** The `expectedschema` submodule needs JSON representations of
    the "current" and "previous" database schemas to perform validation and
    migration. This utility generates these JSON files.
  - **How:** It takes the `sql.Tables` struct (for CockroachDB) or
    `spanner.Schema` (for Spanner) and uses the `go/sql/schema/exportschema`
    module to serialize the schema description into a JSON format. The
    output of this utility is typically checked into version control as
    `schema.json`, `schema_prev.json`, etc., within the `expectedschema`
    directory. The typical workflow for a schema change involves:

  * Make schema changes in relevant Go structs (e.g., add a new field to
    `alerts.AlertSchema`).
  * Run `go generate ./...` within `perf/go/sql/` to regenerate `schema.go`
    and `spanner/schema_spanner.go`.
  * Copy the _old_ `expectedschema/schema.json` to
    `expectedschema/schema_prev.json` (and similarly for Spanner).
  * Run the `exportschema` binary (e.g., `bazel run
//perf/go/sql/exportschema -- --out
perf/go/sql/expectedschema/schema.json`) to generate the new
    `expectedschema/schema.json`.
  * Update the `FromLiveToNext` and `FromNextToLive` SQL statements in
    `expectedschema/migrate.go`.
  * Update test constants in `sql_test.go` (`LiveSchema`, `DropTables`) if
    necessary.

- **Testing Utilities (`sqltest/sqltest.go`):**

  - **Why:** Provides standardized ways to set up temporary CockroachDB or
    Spanner emulator instances for testing components that interact with the
    database.
  - **How:**
  - `NewCockroachDBForTests`: Sets up a connection to a local CockroachDB
    instance (managed by `cockroachdb_instance.Require`), creates a new
    temporary database for the test, applies the current `sql.Schema`, and
    registers a cleanup function to drop the database after the test.
  - `NewSpannerDBForTests`: Similarly, sets up a connection to a local
    Spanner emulator (via PGAdapter, required by `pgadapter.Require`),
    applies the current `spanner.Schema`, and prepares it for tests.
  - These functions abstract away the complexities of emulator management
    and initial schema setup, making tests cleaner and more reliable.

- **Schema Tests (`sql_test.go`):**

  - **Why:** Verifies that the schema migration scripts correctly transform
    a database from a "live-like" previous state to the current expected
    state.
  - **How:**
  - Defines constants like `DropTables` (to clean up) and `LiveSchema` /
    `LiveSchemaSpanner`. `LiveSchema` represents the schema _before_ the
    latest change defined in `expectedschema/migrate.go`'s `FromLiveToNext`.
  - The tests typically:
    1.  Create a test database.
    2.  Apply `DropTables` to ensure a clean slate.
    3.  Apply `LiveSchema` to simulate the state of the database _before_
        the pending migration.
    4.  Execute `expectedschema.FromLiveToNext` (or its Spanner equivalent).
    5.  Fetch the schema description from the migrated database.
    6.  Compare this migrated schema with the schema obtained by applying
        `sql.Schema` (or `spanner.Schema`) directly to a fresh database
        (which represents the target state). They should be identical.

This comprehensive approach to schema management ensures that Perf's database
can be reliably deployed, maintained, and evolved. The separation of concerns
(schema definition, generation, validation, migration, and testing) makes the
system robust and easier to understand.

# Module: /go/stepfit

The `stepfit` module is designed to analyze time-series data, specifically
performance traces, to detect significant changes or "steps." It employs various
statistical algorithms to determine if a step up (performance improvement), a
step down (performance regression), or no significant change has occurred in the
data. This module is crucial for automated performance monitoring, allowing for
the identification of impactful changes in system behavior.

The core idea is to fit a step function to the input trace data. A step function
is a simple function that is constant except for a single jump (the "step") at a
particular point (the "turning point"). The module calculates the best fit for
such a function and then evaluates the characteristics of this fit to determine
the nature and significance of the step.

**Key Components and Logic:**

The primary entity in this module is the `StepFit` struct. It encapsulates the
results of the step detection analysis:

- `LeastSquares`: This field stores the Least Squares Error (LSE) of the
  fitted step function. A lower LSE generally indicates a better fit of the
  step function to the data. It's important to note that not all step
  detection algorithms calculate or use LSE; in such cases, this field is set
  to `InvalidLeastSquaresError`.
- `TurningPoint`: This integer indicates the index in the input trace where
  the step function changes its value. It essentially marks the location of
  the detected step.
- `StepSize`: This float represents the magnitude of the change in the step
  function. A negative `StepSize` implies a step _up_ in the trace values
  (conventionally a performance regression, e.g., increased latency).
  Conversely, a positive `StepSize` indicates a step _down_ (conventionally a
  performance improvement, e.g., decreased latency).
- `Regression`: This value is a metric used to quantify the significance or
  "interestingness" of the detected step. Its calculation varies depending on
  the chosen `stepDetection` algorithm.
  - For the `OriginalStep` algorithm, it's calculated as `StepSize / LSE`
    (or `StepSize / stddevThreshold` if LSE is too small). A larger absolute
    value of `Regression` implies a more significant step.
  - For other algorithms like `AbsoluteStep`, `PercentStep`, and
    `CohenStep`, `Regression` is directly related to the `StepSize` (or a
    normalized version of it).
  - For `MannWhitneyU`, `Regression` represents the p-value of the test.
- `Status`: This is an enumerated type (`StepFitStatus`) indicating the
  overall assessment of the step:
  - `LOW`: A step down was detected, often interpreted as a performance
    improvement.
  - `HIGH`: A step up was detected, often interpreted as a performance
    regression.
  - `UNINTERESTING`: No significant step was found.

The main function responsible for performing the analysis is `GetStepFitAtMid`.
It takes the following inputs:

- `trace`: A slice of `float32` representing the time-series data to be
  analyzed.
- `stddevThreshold`: A threshold for standard deviation. This is used in the
  `OriginalStep` algorithm for normalizing the trace and as a floor for
  standard deviation in other algorithms like `CohenStep` to prevent division
  by zero or near-zero values.
- `interesting`: A threshold value used to determine if a calculated
  `Regression` value is significant enough to be classified as `HIGH` or
  `LOW`. The exact interpretation of this threshold depends on the
  `stepDetection` algorithm.
- `stepDetection`: An enumerated type (`types.StepDetection`) specifying which
  algorithm to use for step detection.

**Workflow of `GetStepFitAtMid`:**

1.  **Initialization and Preprocessing:**

    - A new `StepFit` struct is initialized with `Status` set to
      `UNINTERESTING`.
    - If the trace length is less than `minTraceSize` (currently 3), the
      function returns the initialized `StepFit` as there isn't enough data to
      analyze.
    - **Trace Normalization/Adjustment:**
      - If `stepDetection` is `types.OriginalStep`, the input `trace` is
        duplicated and normalized (mean centered and scaled by its standard
        deviation, unless the standard deviation is below
        `stddevThreshold`).
      - For all other `stepDetection` types, if the trace has an odd length,
        the last element is dropped to make the trace length even. This is
        because these algorithms typically compare the first half of the
        trace with the second half.

2.  **Step Detection Algorithm Execution:** The function then proceeds based on
    the selected `stepDetection` algorithm. The core logic involves splitting
    the (potentially modified) trace roughly in half at the `TurningPoint`
    (which is `len(trace) / 2`) and comparing statistics of the two halves.

        - **`types.OriginalStep`:**

          - Calculates the mean of the first half (`y0`) and the second half
            (`y1`) of the (normalized) trace.
          - Computes the Sum of Squared Errors (SSE) for fitting `y0` to the
            first half and `y1` to the second half. The `LeastSquares` error
            (`lse`) is derived from this SSE.
          - `StepSize` is `y0 - y1`.
          - `Regression` is calculated as `StepSize / lse` (or `StepSize /

    stddevThreshold`if`lse` is too small). _Note: The original
    implementation has a slight deviation from the standard definition
    of standard error in this calculation._

        - **`types.AbsoluteStep`:**

          - `StepSize` is `y0 - y1`.
          - `Regression` is simply the `StepSize`.
          - The step is considered interesting if the absolute value of
            `StepSize` meets the `interesting` threshold.

        - **`types.Const`:**

          - This algorithm behaves differently. It focuses on the absolute value
            of the trace point at the `TurningPoint` (`trace[i]`).
          - `StepSize` is `abs(trace[i]) - interesting`.
          - `Regression` is `-1 * abs(trace[i])`. This is done so that larger
            deviations (regressions) result in more negative `Regression`
            values, which are then flagged as `HIGH`.

        - **`types.PercentStep`:**

          - `StepSize` is `(y0 - y1) / y0`, representing the percentage change
            relative to the mean of the first half.
          - Handles potential `Inf` or `NaN` results from the division (e.g., if
            `y0` is zero).
          - `Regression` is the `StepSize`.

        - **`types.CohenStep`:**

          - Calculates Cohen's d, a measure of effect size.
          - `StepSize` is `(y0 - y1) / s_pooled`, where `s_pooled` is the pooled
            standard deviation of the two halves (or `stddevThreshold` if
            `s_pooled` is too small or NaN).
          - `Regression` is the `StepSize`.

        - **`types.MannWhitneyU`:**

          - Performs a Mann-Whitney U test (a non-parametric test) to determine
            if the two halves of the trace come from different distributions.
          - `StepSize` is `y0 - y1`.
          - `Regression` is the p-value of the test.
          - `LeastSquares` is set to the U-statistic from the test.

3.  **Status Determination:**

    - For `types.MannWhitneyU`:
      - If `Regression` (p-value) is less than or equal to the `interesting`
        threshold (e.g., 0.05), a significant difference is detected.
      - The `Status` (`HIGH` or `LOW`) is then determined by the sign of
        `StepSize`. If `StepSize` is negative (step up), `Status` is `HIGH`.
        Otherwise, it's `LOW`.
      - The `Regression` value is then negated if the status is `HIGH` to
        align with the convention that more negative values are "worse."
    - For all other algorithms:
      - If `Regression` is greater than or equal to `interesting`, `Status`
        is `LOW`.
      - If `Regression` is less than or equal to `-interesting`, `Status` is
        `HIGH`.
      - Otherwise, `Status` remains `UNINTERESTING`.

4.  **Return Result:** The populated `StepFit` struct, containing
    `LeastSquares`, `TurningPoint`, `StepSize`, `Regression`, and `Status`, is
    returned.

**Design Rationale:**

- **Multiple Algorithms:** The inclusion of various step detection algorithms
  (`OriginalStep`, `AbsoluteStep`, `PercentStep`, `CohenStep`, `MannWhitneyU`)
  provides flexibility. Different datasets and performance characteristics may
  be better suited to different statistical approaches. For instance,
  `MannWhitneyU` is non-parametric and makes fewer assumptions about the data
  distribution, which can be beneficial for noisy or non-Gaussian data.
  `AbsoluteStep` and `PercentStep` offer simpler, more direct ways to define a
  regression based on absolute or relative changes.
- **Centralized Logic:** The `GetStepFitAtMid` function consolidates the logic
  for all supported algorithms, making it easier to manage and extend.
- **Clear `StepFit` Structure:** The `StepFit` struct provides a well-defined
  way to communicate the results of the analysis, separating the raw metrics
  (like `StepSize`, `LeastSquares`) from the final interpretation (`Status`).
- **`interesting` Threshold:** The `interesting` parameter allows users to
  customize the sensitivity of the step detection. This is crucial because
  what constitutes a "significant" change can vary greatly depending on the
  context of the performance metric being monitored.
- **`stddevThreshold`:** This parameter helps in handling cases with very low
  variance, preventing numerical instability (like division by zero) and
  ensuring that normalization in `OriginalStep` behaves reasonably.
- **Focus on the Middle:** The `GetStepFitAtMid` name implies that the step
  detection is focused around the middle of the trace. This is a common
  approach for detecting a single, prominent step. More complex scenarios with
  multiple steps would require different techniques.

**Why specific implementation choices?**

- **Normalization in `OriginalStep`:** Normalizing the trace in the
  `OriginalStep` algorithm (as described in the linked blog post) aims to make
  the detection less sensitive to the absolute scale of the data and more
  focused on the relative change.
- **Symmetric Traces for Non-`OriginalStep`:** For algorithms other than
  `OriginalStep`, ensuring an even trace length by potentially dropping the
  last point simplifies the division of the trace into two equal halves for
  comparison.
- **Handling of `Inf` and `NaN` in `PercentStep`:** Explicitly checking for
  and handling `Inf` and `NaN` values that can arise from division by zero
  (when `y0` is zero) makes the `PercentStep` calculation more robust.
- **`Regression` as p-value for `MannWhitneyU`:** Using the p-value as the
  `Regression` metric for `MannWhitneyU` directly reflects the statistical
  significance of the observed difference between the two halves of the trace.
  The `interesting` threshold then acts as the significance level (alpha).
- **`InvalidLeastSquaresError`:** This constant provides a clear way to
  indicate when LSE is not applicable or not calculated by a particular
  algorithm, avoiding confusion with a calculated LSE of 0 or a negative
  value.

In essence, the `stepfit` module provides a toolkit for identifying abrupt
changes in performance data, offering different lenses (algorithms) through
which to view and quantify these changes. The design prioritizes flexibility in
algorithm choice and user-configurable sensitivity to cater to diverse
performance analysis needs.

# Module: /go/subscription

The `subscription` module manages alerting configurations, known as
subscriptions, for anomalies detected in performance data. It provides the means
to define, store, and retrieve these configurations.

The core concept is that a "subscription" dictates how the system should react
when an anomaly is found. This includes details like which bug tracker component
to file an issue under, what labels to apply, who to CC on the bug, and the
priority/severity of the issue. This allows for automated and consistent
handling of performance regressions.

Subscriptions are versioned using an `infra_internal` Git hash (revision). This
allows for tracking changes to subscription configurations over time and ensures
that the correct configuration is used based on the state of the infrastructure
code.

**Key Components and Files:**

- **`store.go`**: Defines the `Store` interface. This interface is the central
  abstraction for interacting with subscription data. It dictates the
  operations that any concrete subscription storage implementation must
  provide. This design choice allows for flexibility in the underlying storage
  mechanism (e.g., SQL database, in-memory store for testing).

  - **Why an interface?** Decouples the business logic from the specific
    storage implementation. This promotes testability (using mocks) and
    allows for easier migration to different database technologies in the
    future if needed.
  - **Key methods:**
  - `GetSubscription`: Retrieves a specific version of a subscription.
  - `GetActiveSubscription`: Retrieves the currently active version of a
    subscription by its name. This is likely the most common retrieval
    method for active alerting.
  - `InsertSubscriptions`: Allows for batch insertion of new subscriptions.
    This is typically done within a database transaction to ensure atomicity
    – either all subscriptions are inserted, or none are. This is crucial
    when updating configurations, as it prevents a partially updated state.
    The implementation in `sqlsubscriptionstore` deactivates all existing
    subscriptions before inserting the new ones as active, effectively
    replacing the entire active set.
  - `GetAllSubscriptions`: Retrieves all historical versions of all
    subscriptions.
  - `GetAllActiveSubscriptions`: Retrieves all currently active
    subscriptions. This is useful for systems that need to know all current
    alerting rules.

- **`proto/v1/subscription.proto`**: Defines the structure of a `Subscription`
  using Protocol Buffers. This is the canonical data model for subscriptions.

  - **Why Protocol Buffers?** Provides a language-neutral, platform-neutral,
    extensible mechanism for serializing structured data. This is beneficial
    for potential interoperability with other services or for persisting
    data in a well-defined format. It also generates efficient serialization
    and deserialization code.
  - **Key fields:** `name`, `revision`, `bug_labels`, `hotlists`,
    `bug_component`, `bug_priority`, `bug_severity`, `bug_cc_emails`,
    `contact_email`. Each field directly maps to a configuration aspect for
    bug filing and contact information.

- **`sqlsubscriptionstore/sqlsubscriptionstore.go`**: Provides a concrete
  implementation of the `Store` interface using an SQL database (specifically
  designed for CockroachDB, as indicated by the use of `pgx`).

  - **Why SQL?** Relational databases offer robust data integrity,
    transaction support (ACID properties), and powerful querying
    capabilities, which are well-suited for managing structured
    configuration data like subscriptions.
  - **How it works:** It defines SQL statements for each operation in the
    `Store` interface. When inserting subscriptions, it first deactivates
    all existing subscriptions and then inserts the new ones as active. This
    ensures that only the latest set of configurations is considered active.
  - The `is_active` boolean column in the database schema
    (`sqlsubscriptionstore/schema/schema.go`) is key to this "active
    version" concept.

- **`sqlsubscriptionstore/schema/schema.go`**: Defines the SQL table schema
  for storing subscriptions.

  - **Key design choice:** The primary key is a composite of `name` and
    `revision`. This allows multiple versions of the same named subscription
    to exist, identified by their revision. The `is_active` field
    differentiates the current version from historical ones.

- **`mocks/Store.go`**: Contains a mock implementation of the `Store`
  interface, generated by the `mockery` tool.

  - **Why mocks?** Essential for unit testing components that depend on the
    `Store` interface without requiring an actual database connection. This
    makes tests faster, more reliable, and isolates the unit under test.

**Key Workflows:**

1.  **Updating Subscriptions:** This typically happens when configurations in
    `infra_internal` are changed.

    ```
    External Process (e.g., config syncer)
        |
        v
    Reads new subscription definitions (likely from files)
        |
        v
    Parses definitions into []*pb.Subscription
        |
        v
    Calls store.InsertSubscriptions(ctx, newSubscriptions, tx)
        |
        |--> [SQL Transaction Start]
        |       |
        |       v
        |    sqlsubscriptionstore: Deactivate all existing subscriptions (UPDATE Subscriptions SET is_active=false WHERE is_active=true)
        |       |
        |       v
        |    sqlsubscriptionstore: Insert each new subscription with is_active=true (INSERT INTO Subscriptions ...)
        |       |
        |       v
        |--> [SQL Transaction Commit/Rollback]
    ```

    This ensures that the update is atomic. If any part fails, the transaction
    is rolled back, leaving the previous set of active subscriptions intact.

2.  **Anomaly Detection Triggering Alerting:** `Anomaly Detector | v Identifies
an anomaly and the relevant subscription name (e.g., based on metric
patterns) | v Calls store.GetActiveSubscription(ctx, subscriptionName) | v
sqlsubscriptionstore: Retrieves the active subscription (SELECT ... FROM
Subscriptions WHERE name=$1 AND is_active=true) | v Anomaly Detector uses
the pb.Subscription details (bug component, labels, etc.) to file a bug.`

This module provides a robust and versioned way to manage alerting rules,
ensuring that performance regressions are handled consistently and routed
appropriately. The separation of interface and implementation, along with the
use of Protocol Buffers, contributes to a maintainable and extensible system.

# Module: /go/tracecache

## TraceCache Module Documentation

The `tracecache` module provides a mechanism for caching trace identifiers
(trace IDs) associated with specific tiles and queries. This caching layer
significantly improves performance by reducing the need to repeatedly compute or
fetch trace IDs, which can be a computationally expensive operation.

**Core Functionality & Design Rationale:**

The primary purpose of `tracecache` is to store and retrieve lists of trace IDs.
Trace IDs are represented as `paramtools.Params`, which are essentially
key-value pairs that uniquely identify a specific trace within the performance
monitoring system.

The caching strategy is built around the concept of a "tile" and a "query."

- **Tile:** In the context of Skia Perf, a tile represents a chunk of commit
  history. Caching trace IDs per tile allows for efficient retrieval of
  relevant traces when analyzing a specific range of commits.
- **Query:** A query, represented by `query.Query`, defines the specific
  parameters used to filter traces. Different queries will yield different
  sets of trace IDs.

By combining the tile number and a string representation of the query, a unique
cache key is generated. This ensures that cached data is specific to the exact
combination of commit range and filter criteria.

The module relies on an external caching implementation provided via the
`go/cache.Cache` interface. This design choice promotes flexibility, allowing
different caching backends (e.g., in-memory, Redis, Memcached) to be used
without modifying the `tracecache` logic itself. This separation of concerns is
crucial for adapting to various deployment environments and performance
requirements.

**Key Components:**

- **`traceCache.go`**: This is the sole file in the module and contains the
  implementation of the `TraceCache` struct and its associated methods.
  - **`TraceCache` struct**:
  - Holds an instance of `cache.Cache`. This is the underlying cache client
    used for storing and retrieving data.
  - **`New(cache cache.Cache) *TraceCache`**:
  - The constructor for `TraceCache`. It takes a `cache.Cache` instance as
    an argument, which will be used for all caching operations. This
    dependency injection allows the caller to provide any cache
    implementation that conforms to the `cache.Cache` interface.
  - **`CacheTraceIds(ctx context.Context, tileNumber types.TileNumber, q
*query.Query, traceIds []paramtools.Params) error`**:
  - This method is responsible for storing a list of trace IDs into the
    cache.
  - It first generates a unique `cacheKey` using the `tileNumber` and the
    `query.Query`.
  - The `traceIds` (a slice of `paramtools.Params`) are then serialized into
    a JSON string using the `toJSON` helper function. This serialization is
    necessary because most cache backends store data as strings or byte
    arrays. JSON is chosen for its human-readability and widespread support.
  - Finally, it uses the `cacheClient.SetValue` method to store the JSON
    string under the generated `cacheKey`.
  - **`GetTraceIds(ctx context.Context, tileNumber types.TileNumber, q
*query.Query) ([]paramtools.Params, error)`**:
  - This method retrieves a list of trace IDs from the cache.
  - It generates the `cacheKey` in the same way as `CacheTraceIds`.
  - It then attempts to fetch the value associated with this key using
    `cacheClient.GetValue`.
  - If the value is not found in the cache (i.e., `cacheJson` is empty), it
    returns `nil` for both the trace IDs and the error, indicating a cache
    miss.
  - If a value is found, it deserializes the JSON string back into a slice
    of `paramtools.Params` using `json.Unmarshal`.
  - **`traceIdCacheKey(tileNumber types.TileNumber, q query.Query)
string`**:
  - A private helper function that constructs the cache key. It combines the
    `tileNumber` (an integer) and a string representation of the
    `query.Query` (obtained via `q.KeyValueString()`) separated by an
    underscore. This format ensures uniqueness and provides some
    human-readable context within the cache keys.
  - **`toJSON(obj interface{}) (string, error)`**:
  - A private generic helper function to marshal any given object into a
    JSON string. This is used specifically for serializing the
    `[]paramtools.Params` before caching.

**Workflow for Caching Trace IDs:**

1.  **Input:** `tileNumber`, `query.Query`, `[]paramtools.Params` (trace IDs to
    cache)
2.  `CacheTraceIds` is called.
3.  `traceIdCacheKey(tileNumber, query)` generates a unique key. `tileNumber +
"_" + query.KeyValueString() ---> cacheKey`
4.  `toJSON(traceIds)` serializes the list of trace IDs into a JSON string.
    `[]paramtools.Params --json.Marshal--> jsonString`
5.  `t.cacheClient.SetValue(ctx, cacheKey, jsonString)` stores the JSON string
    in the underlying cache.

**Workflow for Retrieving Trace IDs:**

1.  **Input:** `tileNumber`, `query.Query`
2.  `GetTraceIds` is called.
3.  `traceIdCacheKey(tileNumber, query)` generates the cache key (same logic as
    above). `tileNumber + "_" + query.KeyValueString() ---> cacheKey`
4.  `t.cacheClient.GetValue(ctx, cacheKey)` attempts to retrieve the value from
    the cache. `cacheClient --GetValue(cacheKey)--> jsonString (or empty if not
found)`
5.  **If `jsonString` is empty (cache miss):** Return `nil`, `nil`.
6.  **If `jsonString` is not empty (cache hit):**
    `json.Unmarshal([]byte(jsonString), &traceIds)` deserializes the JSON string
    back into `[]paramtools.Params`. `jsonString --json.Unmarshal-->
[]paramtools.Params`
7.  Return the deserialized `[]paramtools.Params` and `nil` error.

# Module: /go/tracefilter

## Tracefilter Module Documentation

The `tracefilter` module provides a mechanism to organize and filter trace data
based on their hierarchical paths. The core idea is to represent traces within a
tree structure, where each node in the tree corresponds to a segment of the
trace's path. This allows for efficient filtering of traces, specifically to
identify "leaf" traces – those that do not have any further sub-paths.

This approach is particularly useful in scenarios where traces have a
parent-child relationship implied by their path structure. For instance, in
performance analysis, a trace like `/root/p1/p2/p3/t1` might represent a
specific test (`t1`) under a series of nested configurations (`p1`, `p2`, `p3`).
If there's another trace `/root/p1/p2`, it could be considered a "parent" or an
aggregate trace. The `tracefilter` helps in identifying only the most specific,
or "leaf," traces, effectively filtering out these higher-level parent traces.

### Key Components and Responsibilities

The primary component is the `TraceFilter` struct.

**`TraceFilter` struct:**

- **Purpose:** Represents a node within the trace path tree.
- **Fields:**
  - `traceKey`: A string identifier associated with the trace path ending at
    this node. For the root of the tree, this is initialized to "HEAD".
  - `value`: The string value of the current path segment this node
    represents.
  - `children`: A map where keys are the next path segments and values are
    pointers to child `TraceFilter` nodes. This map forms the branches of
    the tree.
- **Why this structure?**
  - A tree is a natural way to represent hierarchical path data.
  - Using a map for `children` allows for efficient lookup and addition of
    child nodes based on the next path segment.
  - Storing the `traceKey` at each node allows associating an identifier
    with a complete path as it's being built.

**`NewTraceFilter()` function:**

- **Purpose:** Acts as the constructor for the `TraceFilter` tree.
- **How it works:** It initializes a root `TraceFilter` node. The `traceKey`
  is set to "HEAD" as a sentinel value for the root, and its `children` map is
  initialized as empty, ready to have paths added to it.
- **Why this design?** Provides a clear and simple entry point for creating a
  new filter structure.

**`AddPath(path []string, traceKey string)` method:**

- **Purpose:** Adds a new trace, defined by its `path` (a slice of strings
  representing path segments) and its unique `traceKey`, to the filter tree.
- **How it works:**
  1.  It traverses the tree, creating new nodes as needed for each segment in
      the input `path`.
  2.  If a segment in the `path` already exists as a child of the current
      node, it moves to that existing child.
  3.  If a segment does not exist, a new `TraceFilter` node is created for
      that segment, its `value` is set to the segment string, its `traceKey`
      is set to the input `traceKey`, and it's added to the `children` map of
      the current node.
  4.  This process repeats recursively for the remaining segments in the
      `path`.
- **Why this design?**

  - This incremental build process efficiently constructs the tree by
    reusing existing nodes for common path prefixes.
  - The recursive nature elegantly handles paths of arbitrary length.
  - Associating the `traceKey` with each newly created node ensures that
    even intermediate nodes (which might later become leaves if no further
    sub-paths are added) have an associated key.

  ```
  Example: Adding path ["root", "p1", "p2"] with key "keyA"

  Initial Tree:
  (HEAD)

  After AddPath(["root", "p1", "p2"], "keyA"):

  (HEAD)
    |
    +-- ("root", key="keyA")
         |
         +-- ("p1", key="keyA")
              |
              +-- ("p2", key="keyA")  <- Leaf node initially
  ```

  If we then add `["root", "p1", "p2", "t1"]` with key `"keyB"`:

  ```
  (HEAD)
    |
    +-- ("root", key="keyB")  // traceKey updated if path is prefix
         |
         +-- ("p1", key="keyB")
              |
              +-- ("p2", key="keyB")
                   |
                   +-- ("t1", key="keyB") <- New leaf node
  ```

  _Note: The `traceKey` of an existing node is updated by `AddPath` if the new
  path being added shares that node as a prefix. This ensures that the
  `traceKey` stored at a node corresponds to the longest path ending at that
  node if it's also a prefix of other paths._ However, the primary use of
  `GetLeafNodeTraceKeys` relies on the `traceKey` of nodes that _become_
  leaves.

**`GetLeafNodeTraceKeys()` method:**

- **Purpose:** Retrieves the `traceKey`s of all traces that are considered
  "leaf" nodes in the tree. A leaf node is a node that has no children.
- **How it works:**
  1.  It performs a depth-first traversal of the tree.
  2.  If the current node has no children (i.e., `len(tf.children) == 0`), its
      `traceKey` is considered a leaf key and is added to the result list.
  3.  If the current node has children, the method recursively calls itself on
      each child node and aggregates the results.
- **Why this design?**

  - This is the core filtering logic. By only returning keys from nodes
    without children, it effectively filters out traces that serve as
    prefixes (parents) to other, more specific traces.
  - Recursion is a natural fit for traversing tree structures.

  ```
  Workflow for GetLeafNodeTraceKeys:

  Start at (CurrentNode)
      |
      V
  Is CurrentNode a leaf (no children)?
      |
      +-- YES --> Add CurrentNode.traceKey to results
      |
      +-- NO  --> For each ChildNode in CurrentNode.children:
                      |
                      V
                  Recursively call GetLeafNodeTraceKeys on ChildNode
                      |
                      V
                  Append results from ChildNode to overall results
      |
      V
  Return aggregated results
  ```

### Example Scenario and How it Works

Consider the following traces and their paths:

1.  `traceA`: path `["config", "test_group", "test1"]`, key `"keyA"`
2.  `traceB`: path `["config", "test_group"]`, key `"keyB"`
3.  `traceC`: path `["config", "test_group", "test2"]`, key `"keyC"`
4.  `traceD`: path `["config", "other_group", "test3"]`, key `"keyD"`

5.  **Tree Construction (`AddPath` calls):**

    - `tf.AddPath(["config", "test_group", "test1"], "keyA")`
    - `tf.AddPath(["config", "test_group"], "keyB")`
      - When this is added, the node for `"test_group"` initially created by
        `keyA` will have its `traceKey` updated to `"keyB"`.
    - `tf.AddPath(["config", "test_group", "test2"], "keyC")`
    - `tf.AddPath(["config", "other_group", "test3"], "keyD")`

    The tree would look something like this (simplified, showing relevant
    traceKeys for leaf potential):

    ```
    (HEAD)
      |
      +-- ("config")
           |
           +-- ("test_group", traceKey likely updated by "keyB" during AddPath)
           |    |
           |    +-- ("test1", traceKey="keyA")  <-- Leaf
           |    |
           |    +-- ("test2", traceKey="keyC")  <-- Leaf
           |
           +-- ("other_group")
                |
                +-- ("test3", traceKey="keyD")  <-- Leaf
    ```

6.  **Filtering (`GetLeafNodeTraceKeys()` call):**

    - When `GetLeafNodeTraceKeys()` is called on the root:
      - It traverses to `"config"`.
      - It traverses to `"test_group"`. This node has children (`"test1"`
        and `"test2"`), so its key (`"keyB"`) is _not_ added.
      - It traverses to `"test1"`. This is a leaf. `"keyA"` is added.
      - It traverses to `"test2"`. This is a leaf. `"keyC"` is added.
      - It traverses to `"other_group"`.
      - It traverses to `"test3"`. This is a leaf. `"keyD"` is added.

    The result would be `["keyA", "keyC", "keyD"]`. Notice that `"keyB"` is
    excluded because the path `["config", "test_group"]` has sub-paths
    (`.../test1` and `.../test2`), making it a non-leaf node in the context of
    trace specificity.

This module provides a clean and efficient way to identify the most granular
traces in a dataset where hierarchy is defined by path structure.

# Module: /go/tracesetbuilder

The `tracesetbuilder` module is designed to efficiently construct a
`types.TraceSet` and its corresponding `paramtools.ReadOnlyParamSet` from
multiple, potentially disparate, sets of trace data. This is particularly useful
when dealing with performance data that might arrive in chunks (e.g., from
different "Tiles" of data) and needs to be aggregated into a coherent view
across a series of commits.

The core challenge this module addresses is the concurrent and distributed
nature of processing trace data. If multiple traces with the same identifier
(key) were processed by different workers simultaneously without coordination,
it could lead to race conditions and incorrect data. Similarly, simply locking
the entire `TraceSet` for each update would create a bottleneck.

The `tracesetbuilder` solves this by employing a worker pool (`mergeWorkers`).
The key design decision here is to distribute the work based on the trace key.
Each trace key is hashed (using `crc32.ChecksumIEEE`), and this hash determines
which `mergeWorker` is responsible for that specific trace. This ensures that
all data points for a single trace are always processed by the same worker,
thereby avoiding the need for explicit locking at the individual trace level
within the worker. Each `mergeWorker` maintains its own `types.TraceSet` and
`paramtools.ParamSet`.

**Key Components and Workflow:**

1.  **`TraceSetBuilder`:**

        - **Responsibilities:**
          - Manages a pool of `mergeWorker` instances.
          - Provides the `Add` method to ingest new trace data.
          - Provides the `Build` method to consolidate results from all workers
            and return the final `TraceSet` and `ReadOnlyParamSet`.
          - Provides the `Close` method to shut down the worker pool.
        - **`New(size int)`:** Initializes the `TraceSetBuilder`. The `size`
          parameter is crucial as it defines the expected length of each trace in
          the final, consolidated `TraceSet`. This allows the builder to
          pre-allocate trace slices of the correct length, filling in missing data
          points as necessary. It creates `numWorkers` instances of `mergeWorker`.
        - **`Add(commitNumberToOutputIndex map[types.CommitNumber]int32, commits

    []provider.Commit, traces types.TraceSet)`:\*\* This is the entry point
    for feeding data into the builder.

    - `traces`: A `types.TraceSet`representing a chunk of data (e.g.,
      from a single tile). -`commits`: A slice of `provider.Commit`objects corresponding to the
      data points in the`traces`.
    - `commitNumberToOutputIndex`: A map that dictates where each data
      point from the input `traces`(identified by its
      `types.CommitNumber`) should be placed in the _final_ output trace.
      This mapping is essential for correctly aligning data points that
      might come from different sources or represent different commit
      ranges.
    - For each trace in the input `traces`:
    - It parses the trace key into `paramtools.Params`.
    - It creates a `request`struct containing the key, params, the trace
      data itself, the`commitNumberToOutputIndex`map, and the`commits` slice.
    - It calculates an index based on the CRC32 hash of the trace key
      modulo`numWorkers`.
    - It sends this `request`to the`ch`channel of the selected
      `mergeWorker`.
    - A `sync.WaitGroup`is incremented for each trace added, ensuring
      `Build` waits for all processing to complete.
    - **`Build(ctx context.Context)`:**
      - Waits for all `Add`operations to be processed by the workers (using
        `t.wg.Wait()`).
      - Iterates through all `mergeWorkers`.
      - Merges the `traceSet`and`paramSet`from each`mergeWorker`into a
        single, final`types.TraceSet`and`paramtools.ParamSet`.
      - Normalizes and freezes the final `paramSet`to create a
        `paramtools.ReadOnlyParamSet`.
      - Returns the consolidated `TraceSet`and`ReadOnlyParamSet`.
    - **`Close()`:** Iterates through the `mergeWorkers` and closes their
      respective input channels (`ch`). This signals the worker goroutines to
      terminate once they have processed all pending requests.

2.  **`mergeWorker`:**

    - **Responsibilities:**
      - Processes `request` objects sent to its channel.
      - Maintains its own local `types.TraceSet` and `paramtools.ParamSet`.
      - Updates its local `TraceSet` with new data points, placing them
        correctly according to `request.commitNumberToOutputIndex`.
      - Adds the parameters from each processed trace to its local
        `ParamSet`.
    - **`newMergeWorker(wg *sync.WaitGroup, size int)`:** Creates a
      `mergeWorker` and starts its goroutine.
      - It initializes an empty `types.TraceSet` and `paramtools.ParamSet`.
      - The goroutine continuously reads `request` objects from its `ch`
        channel.
      - For each `request`:
      - It retrieves or creates a trace in its `m.traceSet` for the given
        `req.key`. If creating, it uses `types.NewTrace(size)` to ensure the
        trace has the correct final length.
      - It iterates through the `req.commits` and uses
        `req.commitNumberToOutputIndex` to determine the correct destination
        index in its local trace for each data point in `req.trace`.
      - It updates the trace value at that destination index.
      - It adds `req.params` to its `m.paramSet`.
      - It decrements the shared `sync.WaitGroup` (`m.wg.Done()`) to signal
        completion of this piece of work.
    - **`Process(req *request)`:** Sends a request to the worker's channel.
    - **`Close()`:** Closes the worker's input channel.

3.  **`request` struct:**

    - A simple data structure used to pass all necessary information for
      processing a single trace segment through the pipeline to a
      `mergeWorker`. It encapsulates the trace key, its parsed parameters, the
      actual trace data segment, the mapping of commit numbers to output
      indices, and the corresponding commit metadata.

**Workflow Diagram:**

```
                       TraceSetBuilder.New(outputTraceLength)
                                   |
                                   V
  +-----------------------------------------------------------------------+
  | TraceSetBuilder (manages WaitGroup and pool of mergeWorkers)          |
  +-----------------------------------------------------------------------+
      |                                          ^
      | Add(commitMap1, commits1, traces1)       | Build() waits for WaitGroup
      | Add(commitMap2, commits2, traces2)       |
      V                                          |
  +-----------------------------------------------------------------------+
  | For each trace in input:                                              |
  |  1. Parse key -> params                                               |
  |  2. Create 'request' struct                                           |
  |  3. Hash key -> workerIndex                                           |
  |  4. Send 'request' to mergeWorkers[workerIndex].ch                    |
  |  5. Increment WaitGroup                                               |
  +-----------------------------------------------------------------------+
      |         |         | ... (numWorkers times)
      V         V         V
  +--------+ +--------+ +--------+
  | mergeW_0 | | mergeW_1 | | mergeW_N |  (Each runs in its own goroutine)
  | .ch    | | .ch    | | .ch    |
  | .traceSet| | .traceSet| | .traceSet|
  | .paramSet| | .paramSet| | .paramSet|
  +--------+ +--------+ +--------+
      ^         ^         ^
      | Process request:  |
      |  - Get/Create local trace for req.key (length: outputTraceLength) |
      |  - For each point in req.trace:                                   |
      |    - Use req.commitNumberToOutputIndex[commitNum] to find dstIdx  |
      |    - localTrace[dstIdx] = req.trace[srcIdx]                       |
      |  - Add req.params to local paramSet                               |
      |  - Decrement WaitGroup                                            |
      |         |         |
      --------------------- (When TraceSetBuilder.Build() is called)
              |
              V
  +-----------------------------------------------------------------------+
  | TraceSetBuilder.Build():                                              |
  |  1. Wait for all 'Add' operations (WaitGroup.Wait())                  |
  |  2. Create finalTraceSet, finalParamSet                               |
  |  3. For each mergeWorker:                                             |
  |     - Merge worker.traceSet into finalTraceSet                        |
  |     - Merge worker.paramSet into finalParamSet                        |
  |  4. Normalize and Freeze finalParamSet                                |
  |  5. Return finalTraceSet, finalParamSet (ReadOnly)                    |
  +-----------------------------------------------------------------------+
      |
      V
  +-----------------------------------------------------------------------+
  | TraceSetBuilder.Close():                                              |
  |  - Close channels of all mergeWorkers (signals them to terminate)     |
  +-----------------------------------------------------------------------+
```

The use of `numWorkers` and `channelBufferSize` are constants that can be tuned
for performance based on the expected workload and system resources. The CRC32
hash provides a reasonably good distribution of keys across workers, minimizing
the chance of one worker becoming a bottleneck. The `sync.WaitGroup` is
essential for ensuring that the `Build` method doesn't prematurely try to
aggregate results before all input data has been processed by the workers.

The design allows for efficient, concurrent processing of large volumes of trace
data by partitioning the work based on trace identity and then merging the
results, making it suitable for building comprehensive views of performance
metrics over time.

# Module: /go/tracestore

The `tracestore` module defines interfaces and implementations for storing and
retrieving performance trace data. It's a core component of the Perf system,
enabling the analysis of performance metrics over time and across different
configurations.

## Design Philosophy

The primary goal of `tracestore` is to provide an efficient and scalable way to
manage large volumes of trace data. This involves:

- **Tiled Storage:** Data is organized into "tiles," which are fixed-size
  blocks of commits. This approach simplifies data management and allows for
  efficient querying of data within specific time ranges. Each tile has its
  own inverted index and ParamSet, making searches within a tile fast.
- **Inverted Indexing:** To quickly find traces matching specific criteria
  (e.g., "arch=x86" and "config=8888"), `tracestore` uses an inverted index.
  This index maps key-value pairs to the trace IDs that contain them within
  each tile.
- **Caching:** Various caching mechanisms are employed to improve performance,
  including:
  - In-memory LRU caches for frequently accessed data like ParamSets and
    recently written Postings/ParamSet entries.
  - An optional external cache (like Memcached via `go/cache/memcached`) for
    broader caching strategies.
  - A `tracecache` for caching the results of `QueryTracesIDOnly` to speed
    up repeated queries.
- **Interface-Based Design:** The module defines interfaces (`TraceStore`,
  `MetadataStore`, `TraceParamStore`) to allow for different backend
  implementations. This promotes flexibility and testability. The primary
  implementation provided is `sqltracestore`, which uses an SQL database.
- **Concurrency:** Operations like writing traces and querying are designed to
  be concurrent, leveraging Go routines and parallel processing to handle
  large datasets efficiently. For instance, writing large batches of traces or
  postings is often chunked and processed in parallel.
- **Separation of Concerns:**
  - `TraceStore` handles the core logic of reading and writing trace values
    and their associated parameters.
  - `MetadataStore` manages metadata associated with source files (e.g.,
    links to dashboards or logs).
  - `TraceParamStore` specifically handles the mapping between trace IDs
    (MD5 hashes of trace names) and their full parameter sets. This
    separation helps in optimizing storage and retrieval for these distinct
    types of data.

## Key Components and Responsibilities

The `tracestore` module is primarily defined by a set of interfaces and their
SQL-based implementations.

### `tracestore.go`

This file defines the main `TraceStore` interface. It outlines the contract for
any system that wants to store and retrieve performance traces. Key
responsibilities include:

- **Writing Traces (`WriteTraces`, `WriteTraces2`):** Ingesting new
  performance data points. Each data point is associated with a specific
  commit, a set of parameters (defining the trace, e.g.,
  `config=8888,arch=x86`), a value, the source file it came from, and a
  timestamp.
  - The `WriteTraces` method is designed to handle potentially large batches
    of data efficiently. Implementations often involve chunking data and
    performing parallel writes to the underlying storage.
  - `WriteTraces2` is a newer variant, potentially for different storage
    schemas or optimizations (e.g., denormalizing common params directly
    into the trace values table as seen in `TraceValues2Schema`).
- **Reading Traces (`ReadTraces`, `ReadTracesForCommitRange`):** Retrieving
  trace data for specific keys (trace names) within a given tile or commit
  range.
- **Querying Traces (`QueryTraces`, `QueryTracesIDOnly`):**
  - `QueryTraces` allows searching for traces based on a `query.Query`
    object (which specifies parameter key-value pairs). It returns the
    actual trace values and associated commit information.
  - `QueryTracesIDOnly` is an optimization that returns only the
    `paramtools.Params` (effectively the identifying parameters) of traces
    matching a query. This is useful when only the list of matching traces
    is needed, not their values.
- **Tile Management (`GetLatestTile`, `TileNumber`, `TileSize`,
  `CommitNumberOfTileStart`):** Provides methods for interacting with the
  tiled storage system.
- **ParamSet Management (`GetParamSet`):** Retrieving the
  `paramtools.ReadOnlyParamSet` for a specific tile. A ParamSet represents all
  unique key-value pairs present in the traces within that tile, which is
  crucial for UI elements like query builders.
- **Source Information (`GetSource`, `GetLastNSources`,
  `GetTraceIDsBySource`):** Retrieving information about the origin of trace
  data, such as the ingested file name.

### `metadatastore.go`

This file defines the `MetadataStore` interface. Its responsibility is to manage
metadata associated with source files.

- **`InsertMetadata`:** Stores links or other metadata for a given source file
  name.
- **`GetMetadata`:** Retrieves the stored metadata for a source file. This can
  be used, for example, to link from a data point back to the original log
  file or a specific dashboard view related to the data ingestion.

### `traceparamstore.go`

This file defines the `TraceParamStore` interface. This store is dedicated to
managing the relationship between a trace's unique identifier (typically an MD5
hash of its full parameter string) and the actual `paramtools.Params` object.

- **`WriteTraceParams`:** Stores the mapping from trace IDs to their parameter
  sets. This is done to avoid repeatedly parsing or storing the full parameter
  string for every data point of a trace.
- **`ReadParams`:** Retrieves the `paramtools.Params` for a given set of trace
  IDs.

### Submodule: `sqltracestore`

This submodule provides the SQL-based implementation of the `TraceStore`,
`MetadataStore`, and `TraceParamStore` interfaces.

- **`sqltracestore.go`:** Implements the `TraceStore` interface.

  - **Schema:** It relies on a specific SQL schema (defined conceptually in
    the package documentation and concretely in
    `sqltracestore/schema/schema.go`) involving tables like `TraceValues`
    (for actual metric values), `Postings` (the inverted index), `ParamSets`
    (per-tile parameter information), and `SourceFiles`.
  - **Writing Data:** When `WriteTraces` is called, it performs several
    actions:

  * Updates the `SourceFiles` table with the new source filename if it's not
    already present.
  * Updates the `ParamSets` table for the current tile with any new
    key-value pairs from the incoming traces. This uses a cache to avoid
    redundant writes.
  * For each incoming trace: _ Calculates its MD5 hash (trace ID). _ Inserts
    the value into the `TraceValues` table (or `TraceValues2` for
    `WriteTraces2`). _ If the trace ID and its key-value pairs are not
    already in the `Postings` table for the current tile (checked via
    cache), it inserts them. _ Stores the mapping of the trace ID to its
    `paramtools.Params` in the `TraceParams` table via the
    `TraceParamStore`. All these writes are typically batched and
    parallelized for efficiency.

  - **Querying Data (`QueryTracesIDOnly`):**

  * Retrieves the `ParamSet` for the target tile.
  * Generates a query plan based on the input `query.Query` and the tile's
    `ParamSet`.
  * **Optimization (`restrictByCounting`):** It attempts to optimize the
    query by first running `COUNT(*)` queries for each part of the query
    plan. The part of the plan that matches the fewest traces (below a
    threshold) is then used to fetch its corresponding trace IDs. These IDs
    are then used to construct a `restrictClause` (e.g., `AND trace_id IN
(...)`) that is appended to the queries for the other parts of the plan.
    This significantly speeds up queries where one filter is much more
    selective than others.
  * For each part of the query plan (each key and its OR'd values), it
    executes an SQL query against the `Postings` table (using the
    `restrictClause` if applicable) to get a stream of matching
    `traceIDForSQL`.
  * The streams of `traceIDForSQL` from each part of the plan are then
    intersected (using `newIntersect`) to find the trace IDs that satisfy
    all AND conditions of the query.
  * These resulting trace IDs are then passed to the `TraceParamStore` to
    fetch their full `paramtools.Params`.

  - **Reading Data (`QueryTraces`, `ReadTraces`):** Once the trace IDs (and
    thus their full names) are known (either from `QueryTracesIDOnly` or
    directly provided), it queries the `TraceValues` table to fetch the
    actual floating-point values for those traces within the specified
    commit range or tile. It also fetches commit information from the
    `Commits` table.
  - **Follower Reads:** Supports `enableFollowerReads` configuration, which
    adds `AS OF SYSTEM TIME '-5s'` to certain read queries, allowing them to
    potentially hit read replicas and reduce load on the primary, at the
    cost of slightly stale data.
  - **Dialect Specificity:** It has distinct SQL templates and statement
    strings for CockroachDB (default) and Spanner (`spanner.go`) to account
    for syntax differences or performance characteristics (e.g., `UPSERT`
    vs. `ON CONFLICT`).

- **`sqlmetadatastore.go`:** Implements the `MetadataStore` interface. It uses
  an `Metadata` SQL table that links a `source_file_id` (from `SourceFiles`)
  to a JSONB column storing the metadata map.

- **`sqltraceparamstore.go`:** Implements the `TraceParamStore` interface. It
  uses a `TraceParams` SQL table that stores `trace_id` (bytes) and their
  corresponding `params` (JSONB). Writes are chunked and can be parallelized.

- **`intersect.go`:** Provides helper functions (`newIntersect`,
  `newIntersect2`) to compute the intersection of multiple sorted channels of
  `traceIDForSQL`. This is crucial for implementing the AND logic in
  `QueryTracesIDOnly`. It builds a binary tree of `newIntersect2` operations
  for efficiency, avoiding slower reflection-based approaches.

- **`schema/schema.go`:** Defines Go structs that mirror the SQL table
  schemas. This is used for documentation and potentially could be used with
  ORM-like tools if needed, though the current implementation uses direct SQL
  templating.

  - `TraceValuesSchema`: Stores individual data points (value, commit,
    source file) keyed by trace ID.
  - `TraceValues2Schema`: An alternative/extended schema for trace values,
    potentially denormalizing common parameters like `benchmark`, `bot`,
    `test`, etc., for direct querying.
  - `SourceFilesSchema`: Maps source file names to integer IDs.
  - `ParamSetsSchema`: Stores the unique key-value pairs present in each
    tile.
  - `PostingsSchema`: The inverted index, mapping (tile, key-value) to trace
    IDs.
  - `MetadataSchema`: Stores JSON metadata for source files.
  - `TraceParamsSchema`: Maps trace IDs (MD5 hashes) to their full
    `paramtools.Params` (stored as JSON).

- **`spanner.go`:** Contains SQL templates and specific configurations (like
  parallel pool sizes for writes) tailored for Google Cloud Spanner.

### Submodule: `mocks`

- **`TraceStore.go`:** Provides a mock implementation of the `TraceStore`
  interface, generated by the `mockery` tool. This is essential for unit
  testing components that depend on `TraceStore` without needing a full
  database setup.

## Key Workflows

### Writing Traces

```
Caller (e.g., ingester) -> TraceStore.WriteTraces(ctx, commitNumber, params[], values[], paramset, sourceFile, timestamp)
  |
  `-> SQLTraceStore.WriteTraces
      |
      | 1. Tile Calculation: tileNumber = TileNumber(commitNumber)
      |
      | 2. Source File ID:
      |    `-> updateSourceFile(ctx, sourceFile) -> sourceFileID
      |        (Queries SourceFiles table, inserts if not exists)
      |
      | 3. ParamSet Update (for the tile):
      |    For each key, value in paramset:
      |      If not in cache(tileNumber, key, value):
      |        Add to batch for ParamSets table insertion
      |    Execute batch insert into ParamSets, update cache
      |
      | 4. For each trace (params[i], values[i]):
      |    | a. Trace ID Calculation: traceID_md5_hex = md5(query.MakeKey(params[i]))
      |    |
      |    | b. Store Trace Params:
      |    |    `-> TraceParamStore.WriteTraceParams(ctx, {traceID_md5_hex: params[i]})
      |    |        (Inserts into TraceParams table if not exists)
      |    |
      |    | c. Add to TraceValues Batch: (traceID_md5_hex, commitNumber, values[i], sourceFileID)
      |    |
      |    | d. Postings Update (for the tile):
      |    |    If not in cache(tileNumber, traceID_md5_hex): // Marks this whole trace as processed for postings
      |    |      For each key, value in params[i]:
      |    |        Add to batch for Postings table: (tileNumber, "key=value", traceID_md5_hex)
      |
      | 5. Execute batch insert into TraceValues (or TraceValues2)
      |
      | 6. Execute batch insert into Postings, update postings cache
```

### Querying for Trace IDs (`QueryTracesIDOnly`)

```
Caller -> TraceStore.QueryTracesIDOnly(ctx, tileNumber, query)
  |
  `-> SQLTraceStore.QueryTracesIDOnly
      |
      | 1. Get ParamSet for tile:
      |    `-> GetParamSet(ctx, tileNumber) -> tileParamSet
      |        (Checks OPS cache, falls back to querying ParamSets table)
      |
      | 2. Generate Query Plan: plan = query.QueryPlan(tileParamSet)
      |    (If plan is empty or invalid for tile, return empty channel)
      |
      | 3. Optimization (restrictByCounting):
      |    | For each part of 'plan' (key, or_values[]):
      |    |   `-> DB: COUNT(*) FROM Postings WHERE tile_number=... AND key_value IN (...) LIMIT threshold
      |    | Find the plan part (minKey, minValues) with the smallest count (if count < threshold).
      |    | If any count is 0, plan is skippable.
      |    | If minKey found:
      |    |   `-> DB: SELECT trace_id FROM Postings WHERE tile_number=... AND key_value IN (minValues)
      |    |   `-> restrictClause = "AND trace_id IN (result_ids...)"
      |
      | 4. Execute Query for each plan part (concurrently):
      |    For each key, values[] in 'plan' (excluding minKey if restrictClause is used):
      |      `-> DB: SELECT trace_id FROM Postings
      |               WHERE tile_number=tileNumber AND key_value IN ("key=value1", "key=value2"...)
      |               [restrictClause]
      |               ORDER BY trace_id
      |         -> channel_for_key_N (stream of traceIDForSQL)
      |
      | 5. Intersect Results:
      |    `-> newIntersect(ctx, [channel_for_key_1, channel_for_key_2,...]) -> finalTraceIDsChannel (stream of unique traceIDForSQL)
      |
      | 6. Fetch Full Params (concurrently, in chunks):
      |    For each batch of unique traceIDForSQL from finalTraceIDsChannel:
      |      `-> TraceParamStore.ReadParams(ctx, batch_of_ids) -> map[traceID]Params
      |      For each Params in map:
      |        Send Params to output channel
      |
      `-> Returns output channel of paramtools.Params
```

This structured approach, combining interfaces with a robust SQL implementation,
allows `tracestore` to serve as a reliable and performant foundation for Perf's
data storage needs.

# Module: /go/tracing

## Tracing Module Documentation

**High-Level Overview**

The `/go/tracing` module is responsible for initializing and configuring tracing
capabilities within the Perf application. It leverages the OpenCensus library to
provide distributed tracing, allowing developers to understand the flow of
requests across different services and components. This is crucial for debugging
performance issues, identifying bottlenecks, and gaining insights into the
application's behavior in a distributed environment.

**Design Decisions and Implementation Choices**

The core design principle behind this module is to centralize tracing
initialization. This ensures consistency in how tracing is set up across
different parts of the application.

- **Conditional Initialization:** The `Init` function provides different
  initialization paths based on whether the application is running in a
  `local` development environment or a deployed environment.

  - **Local Environment:** In a local setup, `loggingtracer.Initialize()` is
    called. This likely configures a simpler, console-based tracer. The
    rationale is that in local development, detailed, distributed tracing
    might be overkill, and logging traces to the console is often sufficient
    for debugging.
  - **Deployed Environment:** For deployed instances, the
    `tracing.Initialize` function from the shared
    `go.skia.org/infra/go/tracing` library is used. This enables more
    sophisticated tracing, likely integrating with a backend tracing system
    like Jaeger or Stackdriver Trace.

- **Configuration-Driven Sampling:** The `cfg.TraceSampleProportion` (of type
  `config.InstanceConfig`) determines the sampling rate for traces. This
  allows administrators to control the volume of trace data generated,
  balancing the need for detailed information with the cost and overhead of
  storing and processing traces. A value of `0.0` would likely disable
  tracing, while `1.0` would trace every request.

- **Automatic Project ID Detection:** The `autoDetectProjectID` constant being
  an empty string suggests that the underlying `tracing.Initialize` function
  is capable of automatically determining the Google Cloud Project ID when
  running in a GCP environment. This simplifies configuration as the project
  ID doesn't need to be explicitly passed.

- **Metadata Enrichment:** The `map[string]interface{}` passed to
  `tracing.Initialize` includes:

  - `podName`: This value is retrieved from the `MY_POD_NAME` environment
    variable. This is a common practice in Kubernetes environments to
    identify the specific pod generating the trace, which is invaluable for
    pinpointing issues.
  - `instance`: This is derived from `cfg.InstanceName`. This helps
    differentiate traces originating from different Perf instances (e.g.,
    "perf-prod", "perf-staging").

**Responsibilities and Key Components/Files**

- **`tracing.go`:** This is the sole file in this module and contains the
  `Init` function.

  - **`Init(local bool, cfg *config.InstanceConfig) error` function:**
  - **Responsibility:** To initialize the tracing system for the
    application. It acts as the single entry point for tracing setup.
  - **How it works:** 1. It takes a `local` boolean flag and an `InstanceConfig` pointer as
    input. 2. If `local` is `true`, it calls `loggingtracer.Initialize()`. This
    indicates a preference for a simpler, possibly console-based,
    tracing mechanism for local development. `local=true ---->
loggingtracer.Initialize()` 3. If `local` is `false`, it proceeds to initialize tracing for a
    deployed environment. - It retrieves the `TraceSampleProportion` from the `cfg`. - It retrieves the `InstanceName` from `cfg` to be used as an
    attribute. - It calls `tracing.Initialize` from the shared
    `go.skia.org/infra/go/tracing` library. - It passes the sampling proportion, `autoDetectProjectID` (an
    empty string, relying on automatic detection), and a map of
    attributes (`podName` from the environment and `instance` from
    the config). `local=false | V Read cfg.TraceSampleProportion
Read cfg.InstanceName Read os.Getenv("MY_POD_NAME") | V
tracing.Initialize(sample_proportion, "", {podName, instance})`
  - **Why this approach:**
    - Centralizes tracing setup, making it easier to manage and modify.
    - Provides a clear distinction between local and deployed tracing
      configurations, catering to different needs.
    - Leverages shared tracing libraries (`go.skia.org/infra/go/tracing`)
      for common functionality, promoting code reuse.

- **Dependencies:**

  - `//go/tracing` (likely `go.skia.org/infra/go/tracing`): This is the core
    shared tracing library providing the `Initialize` function for robust,
    distributed tracing. It handles the actual setup of exporters (e.g., to
    Stackdriver, Jaeger) and samplers.
  - `//go/tracing/loggingtracer`: This dependency provides a simpler tracer
    implementation, probably for logging traces to standard output, suitable
    for local development environments where a full-fledged tracing backend
    might not be available or necessary.
  - `//perf/go/config`: This module provides the `InstanceConfig` struct,
    which contains application-specific configuration, including the
    `TraceSampleProportion` and `InstanceName` used by the tracing
    initialization. This decouples tracing configuration from the tracing
    logic itself.

**Key Workflows/Processes**

**Tracing Initialization Workflow:**

```
Application Startup
       |
       V
Call perf/go/tracing.Init(isLocal, instanceConfig)
       |
       +---- isLocal is true? ----> Call loggingtracer.Initialize() --> Tracing active (console/simple)
       |                                                                    |
       |                                                                    V
       |                                                               Application proceeds
       |
       +---- isLocal is false? ---> Read TraceSampleProportion from instanceConfig
                                    Read InstanceName from instanceConfig
                                    Read MY_POD_NAME environment variable
                                           |
                                           V
                                    Call shared go.skia.org/infra/go/tracing.Initialize(...)
                                    with sampling rate and attributes (podName, instance)
                                           |
                                           V
                                    Tracing active (distributed, e.g., Stackdriver)
                                           |
                                           V
                                    Application proceeds
```

This workflow illustrates how the `Init` function adapts the tracing setup based
on the execution context (local vs. deployed) and external configuration. The
goal is to provide appropriate tracing capabilities with minimal boilerplate in
the rest of the application.

# Module: /go/trybot

The `/go/trybot` module is responsible for managing performance data generated
by trybots. Trybots are automated systems that run tests on code changes
(patches or changelists) before they are merged into the main codebase. This
module handles the ingestion, storage, and retrieval of these trybot results,
allowing developers and performance engineers to analyze the performance impact
of proposed code changes.

The core idea is to provide a way to compare the performance characteristics of
a pending change against the baseline performance of the current codebase. This
helps in identifying potential performance regressions or improvements early in
the development cycle.

## Key Components and Responsibilities

### `/go/trybot/trybot.go`

This file defines the central data structure `TryFile`.

- **`TryFile`**: This struct represents a single file containing trybot
  results.
  - `CL`: The identifier of the changelist (e.g., a Gerrit change ID). This
    is crucial for associating results with a specific code change.
  - `PatchNumber`: The specific patchset within the changelist. Code review
    systems often allow multiple iterations (patchsets) for a single
    changelist.
  - `Filename`: The name of the file where the trybot results are stored,
    often including a scheme like `gs://` indicating its location (e.g., in
    Google Cloud Storage).
  - `Timestamp`: When the result file was created. This is important for
    tracking and ordering results.

### `/go/trybot/ingester`

This submodule is responsible for taking raw result files and transforming them
into the `TryFile` format that the rest of the system understands.

- **`/go/trybot/ingester/ingester.go`**: Defines the `Ingester` interface.

  - **`Ingester` interface**: Specifies a contract for components that can
    process incoming files (represented by `file.File`) and produce a stream
    of `trybot.TryFile` objects. The `Start` method initiates this
    processing, typically in a background goroutine. This design allows for
    different sources or formats of trybot results to be plugged into the
    system.

- **`/go/trybot/ingester/gerrit/gerrit.go`**: Provides a concrete
  implementation of the `Ingester` interface, specifically for handling trybot
  results originating from Gerrit code reviews.

  - **`Gerrit` struct**: Implements `ingester.Ingester`. It uses a
    `parser.Parser` (from `/perf/go/ingest/parser`) to understand the
    content of the result files.
  - **`New` function**: Constructor for the `Gerrit` ingester.
  - **`Start` method**:
  - It receives a channel of `file.File` objects.
  - For each file, it attempts to parse it using `parser.ParseTryBot`. This
    method extracts the changelist ID (`issue`) and patchset number.
  - If parsing is successful, it converts the patchset string to an integer.
  - A `trybot.TryFile` is created with the extracted CL, patch number,
    filename, and creation timestamp.
  - This `TryFile` is then sent to an output channel.
  - It includes metrics (`parseCounter`, `parseFailCounter`) to track the
    success and failure rates of parsing.
  - The use of channels for input (`files`) and output (`ret`) facilitates
    asynchronous processing, meaning the ingester can process files as they
    become available without blocking other operations.

### `/go/trybot/store`

This submodule is responsible for persisting and retrieving `TryFile`
information and the associated performance measurements.

- **`/go/trybot/store/store.go`**: Defines the `TryBotStore` interface.
  - **`TryBotStore` interface**: This interface outlines the contract for
    storing and retrieving trybot data. This abstraction allows different
    database backends (e.g., CockroachDB, in-memory stores for testing) to
    be used.
  - `Write(ctx context.Context, tryFile trybot.TryFile) error`: Persists a
    `TryFile` and its associated data.
  - `List(ctx context.Context, since time.Time) ([]ListResult, error)`:
    Retrieves a list of unique changelist/patchset combinations that have
    been processed since a given time. `ListResult` contains the `CL` (as a
    string) and `Patch` number.
  - `Get(ctx context.Context, cl types.CL, patch int) ([]GetResult, error)`:
    Fetches all performance results for a specific changelist and patch
    number. `GetResult` contains the `TraceName` (a unique identifier for a
    specific metric and parameter combination) and its measured `Value`.
- **`/go/trybot/store/mocks/TryBotStore.go`**: Provides a mock implementation
  of `TryBotStore`, generated by the `mockery` tool. This is essential for
  unit testing components that depend on `TryBotStore` without needing a real
  database.

### `/go/trybot/results`

This submodule focuses on loading and preparing trybot results for analysis and
presentation, often by comparing them to baseline data.

- **`/go/trybot/results/results.go`**: Defines the structures for requesting
  and representing analyzed trybot results.

  - **`Kind` type (`TryBot`, `Commit`)**: Distinguishes whether the analysis
    request is for trybot data (pre-submit) or for data from an already
    landed commit (post-submit). This allows the system to handle both
    scenarios.
  - **`TryBotRequest` struct**: Represents a request from a client (e.g., a
    UI) to get analyzed performance data. It includes the `Kind`, `CL` and
    `PatchNumber` (for `TryBot` kind), `CommitNumber` and `Query` (for
    `Commit` kind). The `Query` is used to filter the traces to be analyzed
    when looking at landed commits.
  - **`TryBotResult` struct**: Contains the analysis results for a single
    trace.
  - `Params`: The key-value parameters that uniquely identify the trace.
  - `Median`, `Lower`, `Upper`, `StdDevRatio`: Statistical measures derived
    from the trace data. `StdDevRatio` is a key metric indicating how much a
    new value deviates from the historical distribution, helping to flag
    regressions or improvements.
  - `Values`: A slice of recent historical values for the trace, with the
    last value being either the trybot result or the value at the specified
    commit.
  - **`TryBotResponse` struct**: The overall response to a `TryBotRequest`.
  - `Header`: Column headers for the data, typically representing commit
    information.
  - `Results`: A slice of `TryBotResult` for each analyzed trace.
  - `ParamSet`: A collection of all unique parameter key-value pairs present
    in the results, useful for filtering in a UI.
  - **`Loader` interface**: Defines a contract for components that can take
    a `TryBotRequest` and produce a `TryBotResponse`. This involves fetching
    relevant data, performing statistical analysis, and formatting it.

- **`/go/trybot/results/dfloader/dfloader.go`**: Implements the
  `results.Loader` interface using a `dataframe.DataFrameBuilder`. DataFrames
  are a common way to represent tabular data for analysis.

  - **`Loader` struct**: Holds references to a `dataframe.DataFrameBuilder`
    (for constructing DataFrames from trace data), a `store.TryBotStore`
    (for fetching trybot-specific measurements), and `perfgit.Git` (for
    resolving commit information).
  - **`TraceHistorySize` constant**: Defines how many historical data points
    to load for each trace for comparison.
  - **`New` function**: Constructor for the `Loader`.
  - **`Load` method**: This is the core logic for generating the
    `TryBotResponse`.
  - **Workflow**:
    1.  `Determine Timestamp`: If the request is for a `Commit`, it fetches
        the commit details (including its timestamp) using `perfgit.Git`.
        Otherwise, it uses the current time.
    2.  `Parse Query`: If the request kind is `Commit`, the provided `Query`
        string is parsed. An empty query for a `Commit` request is an error.
    3.  `Fetch Baseline Data (DataFrame)`:
        - If `Kind` is `Commit`: It uses `dfb.NewNFromQuery` to load a
          DataFrame containing the last `TraceHistorySize+1` data points
          for traces matching the query, up to the commit's timestamp. The
          "+1" is to hold the value at the commit itself or to be a
          placeholder.
        - If `Kind` is `TryBot`: a. It first calls `store.Get` to retrieve
          the specific trybot measurements for the given `CL` and
          `PatchNumber`. b. It then extracts the trace names from these
          trybot results. c. It calls `dfb.NewNFromKeys` to load a
          DataFrame with `TraceHistorySize+1` historical data points for
          these specific trace names. d. Crucially, it then _replaces_ the
          last value in each trace within the DataFrame with the
          corresponding value obtained from the `store.Get` call. This
          effectively injects the trybot's measurement into the historical
          context for comparison. e. If a trybot result exists for a trace
          that has no historical data in the DataFrame, that trace is
          removed from the analysis, and `rebuildParamSet` is flagged.
    4.  `Prepare Response Header`: The DataFrame's header (commit
        information) is used for the response. If it's a `TryBot` request,
        the last header entry (representing the trybot data point) has its
        `Offset` set to `types.BadCommitNumber` to indicate it's not a
        landed commit.
    5.  `Calculate Statistics`: For each trace in the DataFrame:
        - The trace name (key) is parsed into `paramtools.Params`.
        - `vec32.StdDevRatio` is called with the trace values (which now
          includes the trybot value at the end if applicable). This
          function calculates the median, lower/upper bounds, and the
          standard deviation ratio.
        - A `results.TryBotResult` is created.
        - If `StdDevRatio` calculation fails (e.g., insufficient data),
          the trace is skipped, and `rebuildParamSet` is flagged.
    6.  `Sort Results`: The `TryBotResult` slice is sorted by `StdDevRatio`
        in descending order. This prioritizes potential regressions (high
        positive ratio) and significant improvements (high negative ratio).
    7.  `Normalize ParamSet`: If `rebuildParamSet` is true (due to missing
        traces or parsing errors), the `ParamSet` for the response is
        regenerated from the final set of `TryBotResult`s.
    8.  The `results.TryBotResponse` is assembled and returned.
  - This process allows a direct comparison of a tryjob's performance
    numbers against the recent history of the same metrics on the main
    branch.

### `/go/trybot/samplesloader`

This submodule deals with loading raw sample data from trybot result files.
Sometimes, instead of just a single aggregated value, trybots might output
multiple raw measurements (samples) for a metric.

- **`/go/trybot/samplesloader/samplesloader.go`**: Defines the `SamplesLoader`
  interface.

  - **`SamplesLoader` interface**: Specifies a method `Load(ctx
context.Context, filename string) (parser.SamplesSet, error)` that takes
    a filename (URL to the result file) and returns a `parser.SamplesSet`. A
    `SamplesSet` is a map where keys are trace identifiers and values are
    `parser.Samples` (which include parameters and a slice of raw float64
    sample values).

- **`/go/trybot/samplesloader/gcssamplesloader/gcssamplesloader.go`**:
  Implements `SamplesLoader` for files stored in Google Cloud Storage (GCS).

  - **`loader` struct**: Holds a `gcs.GCSClient` for interacting with GCS
    and a `parser.Parser`.
  - **`New` function**: Constructor for the GCS samples loader.
  - **`Load` method**:
  - Parses the input `filename` (which is a GCS URL like
    `gs://bucket/path/file.json`) to extract the bucket and path.
  - Uses the `storageClient` to read the content of the file from GCS.
  - Parses the file content using `format.ParseLegacyFormat` (assuming a
    specific JSON structure for these sample files).
  - Converts the parsed data into a `parser.SamplesSet` using
    `parser.GetSamplesFromLegacyFormat`.
  - This component is essential when detailed analysis of raw samples is
    needed, rather than just aggregated metrics.

## Overall Workflow (Ingestion and Analysis)

A simplified workflow could look like this:

1.  **File Arrival**: A new trybot result file appears (e.g., uploaded to GCS).

    ```
    New File (e.g., in GCS)
    ```

2.  **Ingestion**: An `ingester.Ingester` (like `ingester.gerrit.Gerrit`)
    detects and processes this file.

    ```
    File --> [Gerrit Ingester] --parses--> trybot.TryFile{CL, PatchNum, Filename, Timestamp}
    ```

3.  **Storage**: The `TryFile` metadata and potentially the parsed values are
    written to the `store.TryBotStore`.

    ```
    trybot.TryFile --> [TryBotStore.Write] --> Database
    ```

    (The actual performance values might be stored alongside the `TryFile`
    metadata or linked via the `Filename` if they are in a separate detailed
    file).

4.  **Analysis Request**: A user or an automated system requests analysis for a
    particular CL/Patch via a UI or API, sending a `results.TryBotRequest`.

    ```
    UI/API --sends--> results.TryBotRequest{Kind=TryBot, CL="123", PatchNumber=1}
    ```

5.  **Data Loading and Comparison**: The `results.dfloader.Loader` handles this
    request. `results.TryBotRequest | v [dfloader.Loader.Load] | +--(A)-->
[TryBotStore.Get(CL, PatchNum)] --> Trybot specific values (Value_T) for
traces T1, T2... | +--(B)-->
[DataFrameBuilder.NewNFromKeys(traceNames=[T1,T2...])] --> Historical data
for T1, T2... | (e.g., [V1_hist1, V1_hist2, ..., V1_histN, _placeholder_]) |
+--(C)--> Combine: Replace _placeholder_ with Value_T | (e.g., for T1:
[V1_hist1, V1_hist2, ..., V1_histN, V1_T]) | +--(D)--> Calculate
StdDevRatio, Median, etc. for each trace | +--(E)--> Sort results | v
results.TryBotResponse (sent back to UI/API)`

This module is crucial for proactive performance monitoring, enabling teams to
catch performance regressions before they land in the main codebase, by
systematically ingesting, storing, and analyzing the performance data generated
during the pre-submit testing phase. The use of interfaces for storage
(`TryBotStore`), ingestion (`Ingester`), and results loading (`results.Loader`)
makes the system flexible and extensible.

# Module: /go/ts

The `go/ts` module serves as a utility to generate TypeScript definition files
from Go structs. This is crucial for maintaining type safety and consistency
between the Go backend and the TypeScript frontend, particularly when dealing
with JSON data structures that are exchanged between them. The core problem this
module solves is bridging the gap between Go's static typing and TypeScript's
type system for data interchange, ensuring that changes in Go struct definitions
are automatically reflected in the frontend's TypeScript types.

The primary component is the `main.go` file. Its responsibility is to:

1.  **Parse command-line arguments**: It accepts an output path (`-o`) where the
    generated TypeScript file will be written.
2.  **Instantiate a `go2ts.Generator`**: This is the core engine from the
    `go/go2ts` library responsible for the Go-to-TypeScript conversion.
3.  **Configure the generator**:
    - `GenerateNominalTypes = true`: This setting likely ensures that the
      generated TypeScript types are nominal (i.e., types are distinct based
      on their name, not just their structure), which can provide stronger
      type checking.
    - `AddIgnoreNil`: This is used for specific Go types like
      `paramtools.Params`, `paramtools.ParamSet`,
      `paramtools.ReadOnlyParamSet`, and `types.TraceSet`. This suggests that
      `nil` values for these types in Go should likely be treated as optional
      or nullable fields in TypeScript, or perhaps excluded from the generated
      types if they are always expected to be non-nil when serialized.
4.  **Register Go structs and unions for conversion**:
    - The code extensively uses `generator.AddMultiple` to register a wide
      array of Go structs from various `perf` submodules (e.g., `alerts`,
      `chromeperf`, `clustering2`, `frontend/api`, `regression`). These are
      the structs that are serialized to JSON and consumed by the frontend. By
      registering them, the generator knows which Go types to convert into
      corresponding TypeScript interfaces or types.
    - The `addMultipleUnions` helper function and
      `generator.AddUnionToNamespace` are used to register Go union types
      (often represented as a collection of constants or an interface
      implemented by several types). This ensures that TypeScript enums or
      union types are generated, reflecting the possible values or types a Go
      field can hold. The `typeName` argument in `unionAndName` and the
      namespace argument in `AddUnionToNamespace` control how these unions are
      named and organized in the generated TypeScript.
    - `generator.AddToNamespace` is used to group related types under a
      specific namespace in the generated TypeScript, improving organization
      (e.g., `pivot.Request{}` is added to the `pivot` namespace).
5.  **Render the TypeScript output**: Finally, `generator.Render(w)` writes the
    generated TypeScript definitions to the specified output file.

The design decision to use a dedicated program for this generation task, rather
than manual synchronization or other methods, highlights the importance of
automation and reducing the likelihood of human error in keeping backend and
frontend types aligned. The reliance on the `go/go2ts` library centralizes the
core conversion logic, making this module a consumer and orchestrator of that
library for the specific needs of the Skia Perf application.

A key workflow is triggered by the `//go:generate` directive at the top of
`main.go`: `//go:generate bazelisk run --config=mayberemote //:go -- run . -o
../../modules/json/index.ts`

This command, when `go generate` is run (typically as part of a build process),
executes the compiled `go/ts` program.

Workflow:

1.  Developer modifies a Go struct in a `perf` submodule that is serialized to
    JSON for the UI.
2.  Developer (or an automated build step) runs `go generate` within the `go/ts`
    module's directory (or a higher-level directory that includes it).
3.  The `go:generate` directive executes the `main` function in `go/ts/main.go`.
4.  `main.go` -> Uses `go2ts.Generator` -> Registers relevant Go structs and
    unions.
5.  `go2ts.Generator` -> Analyzes registered Go types -> Generates corresponding
    TypeScript definitions.
6.  `main.go` -> Writes the TypeScript definitions to
    `../../modules/json/index.ts`.
7.  The frontend can now import and use these up-to-date TypeScript types,
    ensuring type safety when interacting with JSON data from the backend.

The choice of specific structs and unions registered in `main.go` reflects the
data contracts between the Perf backend and its frontend UI. Any Go struct that
is part of an API response or request payload handled by the frontend needs to
be included here.

# Module: /go/types

## Go Types Module

This module defines core data types used throughout the Perf application. These
types provide a standardized way to represent fundamental concepts related to
commits, performance data (traces), and alert configurations. The design
prioritizes clarity, type safety, and consistency across different parts of the
system.

### Key Concepts and Components:

#### Commit and Tile Numbering:

- **`CommitNumber` (`types.go`)**: Represents a unique, sequential identifier
  for a commit within a repository.

  - **Why**: To provide a simple, linear way to reference commits. It
    assumes a straightforward, non-branching history for easier indexing and
    retrieval of performance data associated with specific code changes. The
    first commit in a repository is assigned `CommitNumber(0)`.
  - **How**: Implemented as an `int32`. It includes an `Add` method for safe
    offsetting and a `BadCommitNumber` constant (`-1`) to represent invalid
    or non-existent commit numbers.
  - **`CommitNumberSlice` (`types.go`)**: A utility type to enable sorting
    of `CommitNumber` slices, which is useful for various data processing
    and display tasks.

- **`TileNumber` (`types.go`)**: Represents an index for a "tile" in the
  `TraceStore`. Performance data (traces) are often stored in chunks or tiles
  for efficient storage and retrieval.

  - **Why**: Tiling allows for optimized access to performance data,
    especially for large datasets. Instead of loading entire traces, only
    relevant tiles need to be accessed.
  - **How**: Implemented as an `int32`. Functions like
    `TileNumberFromCommitNumber` and `TileCommitRangeForTileNumber` manage
    the mapping between commit numbers and tile numbers based on a
    configurable `tileSize`. The `Prev()` method allows navigation to the
    preceding tile, and `BadTileNumber` (`-1`) indicates an invalid tile.

  **Workflow: Commit to Tile Mapping**

  ```
  CommitNumber ----(tileSize)----> TileNumberFromCommitNumber() ----> TileNumber
                                                                    |
                                                                    V
                                        TileCommitRangeForTileNumber() ----> (StartCommit, EndCommit)
  ```

#### Performance Data Representation:

- **`Trace` (`types.go`)**: Represents a sequence of performance measurements,
  typically corresponding to a specific metric over a series of commits.

  - **Why**: To provide a simple and efficient way to store and manipulate
    time-series performance data.
  - **How**: Implemented as a `[]float32`. The `NewTrace` function
    initializes a trace of a given length with a special
    `vec32.MISSING_DATA_SENTINEL` value, which is crucial for distinguishing
    between actual zero values and missing data points. This leverages the
    `go.skia.org/infra/go/vec32` package for optimized float32 vector
    operations.

- **`TraceSet` (`types.go`)**: A collection of `Trace`s, keyed by a string
  identifier (trace ID).

  - **Why**: To group related traces, often corresponding to different
    metrics measured for the same test or configuration.
  - **How**: Implemented as a `map[string]Trace`.

#### Regression Detection and Alerting:

- **`RegressionDetectionGrouping` (`types.go`)**: An enumeration defining how
  traces are grouped for regression detection.

  - **Why**: Different grouping strategies can be more effective for
    different types of performance data. This allows flexibility in the
    regression detection process.
  - **How**: Defined as a string type with constants like `KMeansGrouping`
    (cluster traces by shape) and `StepFitGrouping` (analyze each trace
    individually for steps). `ToClusterAlgo` provides a safe way to convert
    strings to this type.

- **`StepDetection` (`types.go`)**: An enumeration defining the algorithms
  used to detect significant steps (changes) in individual traces or cluster
  centroids.

  - **Why**: Various statistical methods can be employed to identify
    meaningful performance regressions or improvements. This allows
    selection of the most appropriate method for the data characteristics.
  - **How**: Defined as a string type with constants representing different
    detection methods, such as `OriginalStep`, `AbsoluteStep`,
    `PercentStep`, `CohenStep`, and `MannWhitneyU`. `ToStepDetection`
    ensures type-safe conversion from strings.

- **`AlertAction` (`types.go`)**: An enumeration defining the actions to be
  taken when an anomaly (potential regression) is detected by an alert
  configuration.

  - **Why**: To allow configurable responses to detected anomalies, ranging
    from no action to filing issues or triggering bisection jobs.
  - **How**: Defined as a string type with constants like `NoAction`,
    `FileIssue`, and `Bisection`.

- **`Domain` (`types.go`)**: Specifies the range of commits over which an
  operation (like regression detection) should be performed.

  - **Why**: To precisely define the scope of analysis.
  - **How**: A struct containing either `N` (number of commits) and `End`
    (timestamp for the end of the range) or an `Offset` (a specific commit
    number).

- **`ProgressCallback` (`types.go`)**: A function type used to provide
  feedback on the progress of long-running operations.

  - **Why**: To enable user interfaces or logging systems to display the
    status of tasks like regression detection.
  - **How**: Defined as `func(message string)`.

- **`CL` (`types.go`)**: Represents a Change List identifier (e.g., a GitHub
  Pull Request number).

  - **Why**: To associate performance data or alerts with specific code
    changes under review.
  - **How**: Defined as a `string`.

- **`AnomalyDetectionNotifyType` (`types.go`)**: Defines the notification
  mechanism for anomalies.

  - **Why**: Allows flexibility in how users are informed about detected
    performance issues.
  - **How**: String type with constants `IssueNotify` (send to issue
    tracker) and `NoneNotify` (no notification).

#### Miscellaneous:

- **`ProjectId` (`types.go`)**: Represents a project identifier.

  - **Why**: Useful in multi-project environments to scope data or
    configurations.
  - **How**: Defined as a `string` with a predefined list `AllProjectIds`.

- **`AllMeasurementStats` (`types.go`)**: A list of valid statistical suffixes
  that can be part of performance measurement keys (e.g., "avg", "max").

  - **Why**: To ensure consistency and provide a reference for valid stat
    types when parsing or generating metric keys.
  - **How**: A `[]string` slice.

The unit tests in `types_test.go` focus on validating the logic of
`CommitNumber` arithmetic and the mapping between `CommitNumber` and
`TileNumber`, ensuring the core indexing mechanisms are correct.

# Module: /go/ui

The `/go/ui` module is responsible for handling frontend requests and preparing
data for display in the Perf UI. Its primary purpose is to bridge the gap
between user interactions on the frontend (e.g., selecting time ranges, defining
queries, or applying formulas) and the backend data sources and processing
logic.

This module is designed to be the central point for fetching and transforming
performance data into a format that can be readily consumed by the UI. It
orchestrates interactions with various other modules, such as those responsible
for accessing Git history (`/go/git`), building dataframes (`/go/dataframe`),
handling data shortcuts (`/go/shortcut`), and calculating derived metrics
(`/go/calc`).

The key rationale behind this module's existence is to encapsulate the
complexity of data retrieval and preparation, providing a clean and consistent
API for the frontend. This separation of concerns allows the frontend to focus
on presentation and user interaction, while the backend handles the intricacies
of data access and manipulation.

The main workflow involves receiving a `FrameRequest` from the frontend,
processing it to fetch and transform data, and then returning a `FrameResponse`
containing the prepared data and display instructions.

### Key Components and Files:

- **`/go/ui/frame/frame.go`**: This is the core file of the module.
  - **Responsibilities**:
  - Defines the structure of frontend requests (`FrameRequest`) and backend
    responses (`FrameResponse`). `FrameRequest` captures user inputs like
    time ranges, queries, formulas, and pivot table configurations.
    `FrameResponse` packages the resulting data, along with display hints
    and any relevant messages.
  - Manages the processing of `FrameRequest` objects. This involves
    dispatching tasks to other modules based on the request parameters. For
    example, it uses the `dataframe.DataFrameBuilder` to fetch data based on
    queries or trace keys, the `calc` module to evaluate formulas, and the
    `pivot` module to restructure data for pivot tables.
  - Handles different types of requests, such as those based on a specific
    time range (`REQUEST_TIME_RANGE`) or a fixed number of recent commits
    (`REQUEST_COMPACT`).
  - Orchestrates the retrieval of anomalies from an `anomalies.Store` and
    associates them with the relevant traces in the response. This can be
    done based on time ranges or commit revision numbers.
  - Includes logic to determine the appropriate display mode for the
    frontend (e.g., plot, pivot table, or just a query input).
  - Implements safeguards like truncating the number of traces in the
    response if it exceeds a predefined limit, to prevent overwhelming the
    frontend or the network.
  - Provides functionality to identify "SKP changes" (significant file
    changes in the Git repository, historically related to Skia Picture
    files) within the requested commit range, which can be highlighted in
    the UI.
  - **Design Choices & Implementation Details**:
  - The `ProcessFrameRequest` function is the main entry point for handling
    a request. It creates a `frameRequestProcess` struct to manage the state
    of the request processing.
  - The processing is broken down into distinct steps: handling queries,
    formulas, and keys (shortcuts). Each step typically involves fetching
    data and then joining it into a single `DataFrame`.
  - Error handling is centralized in `reportError` to ensure consistent
    logging and error propagation.
  - Progress tracking is integrated via the `progress.Progress` interface,
    allowing the frontend to display updates during long-running requests.
  - The decision to support both `REQUEST_TIME_RANGE` and `REQUEST_COMPACT`
    request types caters to different user needs: exploring specific
    historical periods versus viewing the latest trends.
  - The inclusion of anomaly data directly in the `FrameResponse` aims to
    provide users with immediate context about significant performance
    changes alongside the raw data. The system supports fetching anomalies
    based on either time or revision ranges, offering flexibility depending
    on how anomalies are tracked and stored.
  - The `ResponseFromDataFrame` function acts as a final assembly step,
    taking a processed `DataFrame` and enriching it with SKP change
    information, display mode, and handling potential truncation.

A typical request processing flow might look like this:

```
Frontend Request (FrameRequest)
        |
        V
ProcessFrameRequest() in frame.go
        |
        +------------------------------+-----------------------------+--------------------------+
        |                              |                             |                          |
        V                              V                             V                          V
  (If Queries exist)             (If Formulas exist)         (If Keys exist)           (If Pivot requested)
  doSearch()                     doCalc()                    doKeys()                  pivot.Pivot()
    |                              |                             |                          |
    V                              V                             V                          V
  dfBuilder.NewFromQuery...()    calc.Eval() with               dfBuilder.NewFromKeys...()   Restructure DataFrame
                                 rowsFromQuery/Shortcut()
        |                              |                             |                          |
        +------------------------------+-----------------------------+--------------------------+
        |
        V
  DataFrame construction and merging
        |
        V
  (If anomaly search enabled)
  addTimeBasedAnomaliesToResponse() OR addRevisionBasedAnomaliesToResponse()
    |
    V
  anomalyStore.GetAnomalies...()
        |
        V
ResponseFromDataFrame()
        |
        V
  getSkps() (Find significant file changes)
        |
        V
  Truncate response if too large
        |
        V
  Set DisplayMode
        |
        V
Backend Response (FrameResponse)
        |
        V
Frontend UI
```

# Module: /go/urlprovider

## URL Provider Module

The `urlprovider` module is designed to generate URLs for various pages within
the Perf application. This centralized approach ensures consistency in URL
generation across different parts of the application and simplifies the process
of linking to specific views with pre-filled parameters. The key motivation is
to abstract away the complexities of URL query parameter construction and to
provide a simple interface for generating links to common Perf views like
"Explore", "MultiGraph", and "GroupReport".

The core component of this module is the `URLProvider` struct. An instance of
`URLProvider` is initialized with a `perfgit.Git` object. This dependency is
crucial because some URL generation, particularly for time-range-based views,
requires fetching commit information (specifically timestamps) from the Git
repository to define the "begin" and "end" parameters of the URL.

### Key Responsibilities and Components:

- **`urlprovider.go`**: This file contains the primary logic for the URL
  provider.
  - **`URLProvider` struct**: Holds a reference to a `perfgit.Git` instance.
    This allows it to interact with the Git repository to fetch commit
    details needed for constructing time-based query parameters.
  - **`New(perfgit perfgit.Git) *URLProvider`**: This constructor function
    creates and returns a new instance of `URLProvider`. It takes a
    `perfgit.Git` object as an argument, which is stored within the struct.
    This design choice makes the `URLProvider` stateful with respect to its
    Git interaction capabilities.
  - **`Explore(...) string`**: This method generates a URL for the "Explore"
    page (`/e/`).
  - **Why**: The "Explore" page is used for in-depth analysis of performance
    data based on various parameters and a specific commit range.
  - **How**:
    1.  It calls `getQueryParams` to construct the common query parameters
        like `begin`, `end`, and `disable_filter_parent_traces`. The `begin`
        and `end` timestamps are derived from the provided
        `startCommitNumber` and `endCommitNumber` by querying the `perfGit`
        instance. The `end` timestamp is intentionally shifted forward by
        one day to ensure that anomalies at the very end of the selected
        range are visible on the graph.
    2.  It then serializes the `parameters` map (which contains key-value
        pairs for filtering traces) into a URL-encoded query string using
        `GetQueryStringFromParameters`. This encoded string is assigned to
        the `queries` parameter of the final URL.
    3.  Additional `queryParams` (passed as `url.Values`) can be merged into
        the URL.
    4.  The final URL is constructed by appending the encoded query
        parameters to the base path `/e/?`.
  - **`MultiGraph(...) string`**: This method generates a URL for the
    "MultiGraph" page (`/m/`).
  - **Why**: The "MultiGraph" page allows users to view multiple graphs
    simultaneously, often identified by a shortcut ID.
  - **How**:
    1.  Similar to `Explore`, it uses `getQueryParams` to build the common
        time-range and filtering parameters.
    2.  It specifically adds the `shortcut` parameter with the provided
        `shortcutId`.
    3.  Additional `queryParams` can also be merged.
    4.  The final URL is constructed by appending the encoded query
        parameters to the base path `/m/?`.
  - **`GroupReport(param string, value string) string`**: This _static_
    function generates a URL for the "Group Report" page (`/u/`).
  - **Why**: The "Group Report" page displays information related to groups
    of anomalies, specific anomalies, bugs, or revisions. Unlike `Explore`
    and `MultiGraph`, it does not inherently depend on a time range derived
    from commits, nor does it require complex parameter encoding.
  - **How**:
    1.  It validates the input `param` against a predefined list of allowed
        parameters (`anomalyGroupID`, `anomalyIDs`, `bugID`, `rev`, `sid`).
        This is a security and correctness measure to prevent arbitrary
        parameters from being injected.
    2.  If the `param` is valid, it constructs a simple URL with the
        provided `param` and `value`.
    3.  It returns an empty string if the `param` is invalid.
    4.  This function is static (not a method on `URLProvider`) because it
        doesn't need access to the `perfGit` instance or any other state
        within `URLProvider`. This simplifies its usage for cases where only
        a group report URL is needed without initializing a full
        `URLProvider`.
  - **`getQueryParams(...) url.Values`**: This private helper method is
    responsible for creating the base set of query parameters common to
    `Explore` and `MultiGraph`.
  - **How**:
    1.  It calls `fillCommonParams` to set the `begin` and `end` parameters
        based on commit numbers.
    2.  It conditionally adds `disable_filter_parent_traces=true` if
        requested.
    3.  It merges any additional `queryParams` provided by the caller.
  - **`fillCommonParams(...)`**: This private helper populates the `begin`
    and `end` timestamp parameters in the provided `url.Values`.
  - **How**: It uses the `perfGit` instance to look up the `Commit` objects
    corresponding to the `startCommitNumber` and `endCommitNumber`. The
    timestamps from these commits are then used. As mentioned earlier, the
    `end` timestamp is adjusted by adding one day. This separation of
    concerns keeps the main `Explore` and `MultiGraph` methods cleaner.
  - **`GetQueryStringFromParameters(parameters map[string][]string)
string`**: This helper method converts a map of string slices
    (representing query parameters where a single key can have multiple
    values) into a URL-encoded query string.

### Key Workflows:

1.  **Generating an "Explore" Page URL:**

    ```
    Caller provides: context, startCommitNum, endCommitNum, filterParams, disableFilterParent, otherQueryParams
      |
      v
    URLProvider.Explore()
      |
      +-------------------------------------+
      |                                     |
      v                                     v
    getQueryParams()                     GetQueryStringFromParameters(filterParams)
      |                                     |
      +--> fillCommonParams()               +--> Encode filterParams
      |      |                                        |
      |      +--> perfGit.CommitFromCommitNumber() -> Get start timestamp
      |      |                                        |
      |      +--> perfGit.CommitFromCommitNumber() -> Get end timestamp, add 1 day
      |      |                                        |
      |      +----------------------------------------+
      |      |
      |      v
      |    Combine begin, end, disableFilterParent, otherQueryParams into url.Values
      |                                     |
      +-------------------------------------+
      |
      v
    Combine base URL ("/e/?"), common query params, and encoded filterParams string
      |
      v
    Return final URL string
    ```

2.  **Generating a "MultiGraph" Page URL:**

    ```
    Caller provides: context, startCommitNum, endCommitNum, shortcutId, disableFilterParent, otherQueryParams
      |
      v
    URLProvider.MultiGraph()
      |
      v
    getQueryParams()
      |
      +--> fillCommonParams()
      |      |
      |      +--> perfGit.CommitFromCommitNumber() -> Get start timestamp
      |      |
      |      +--> perfGit.CommitFromCommitNumber() -> Get end timestamp, add 1 day
      |      |
      |      +----------------------------------------+
      |      |
      |      v
      |    Combine begin, end, disableFilterParent, otherQueryParams into url.Values
      |
      v
    Add "shortcut=shortcutId" to url.Values
      |
      v
    Combine base URL ("/m/?") and all query params
      |
      v
    Return final URL string
    ```

3.  **Generating a "Group Report" Page URL:**

    ```
    Caller provides: paramName, paramValue
      |
      v
    urlprovider.GroupReport()
      |
      v
    Validate paramName against allowed list
      |
      +-- (Valid) --> Construct URL: "/u/?" + paramName + "=" + paramValue
      |                 |
      |                 v
      |               Return URL string
      |
      +-- (Invalid) --> Return "" (empty string)
    ```

The design emphasizes reusability of common parameter generation logic
(`getQueryParams`, `fillCommonParams`) and clear separation of concerns for
generating URLs for different Perf pages. The dependency on `perfgit.Git` is
explicitly managed through the `URLProvider` struct, making it clear when Git
interaction is necessary.

# Module: /go/userissue

The `userissue` module is responsible for managing the association between
specific data points in Perf (identified by a trace key and a commit position)
and Buganizer issues. This allows users to flag specific performance regressions
or anomalies and link them directly to a tracking issue.

The core of this module is the `Store` interface, which defines the contract for
persisting and retrieving these user-issue associations. The primary
implementation of this interface is `sqluserissuestore`, which leverages a SQL
database (specifically CockroachDB in this context) to store the data.

**Key Responsibilities and Components:**

- **`store.go`**: This file defines the central `UserIssue` struct and the
  `Store` interface.

  - **`UserIssue` struct**: Represents a single association. It contains:
  - `UserId`: The email of the user who made the association.
  - `TraceKey`: A string uniquely identifying a performance metric's trace
    (e.g., ",arch=x86,config=Release,test=MyTest,").
  - `CommitPosition`: An integer representing a specific point in the commit
    history where the data point exists.
  - `IssueId`: The numerical ID of the Buganizer issue.
  - **`Store` interface**: This interface dictates the operations that any
    backing store for user issues must support:
  - `Save(ctx context.Context, req *UserIssue) error`: Persists a new
    `UserIssue` association. The implementation must handle potential
    conflicts, such as trying to save a duplicate entry (same trace key and
    commit position).
  - `Delete(ctx context.Context, traceKey string, commitPosition int64)
error`: Removes an existing user-issue association based on its unique
    trace key and commit position. It should handle cases where the
    specified association doesn't exist.
  - `GetUserIssuesForTraceKeys(ctx context.Context, traceKeys []string,
startCommitPosition int64, endCommitPosition int64) ([]UserIssue,
error)`: Retrieves all `UserIssue` associations for a given set of trace
    keys within a specified range of commit positions. This is crucial for
    displaying these associations on performance graphs or reports.

- **`sqluserissuestore/sqluserissuestore.go`**: This is the SQL-backed
  implementation of the `Store` interface.

  - **Design Rationale**: Using a SQL database provides robust data
    integrity, transactional guarantees, and the ability to perform complex
    queries if needed in the future. CockroachDB is chosen for its
    scalability and compatibility with PostgreSQL syntax.
  - **Implementation Details**:
  - It uses a `go.skia.org/infra/go/sql/pool` for managing database
    connections.
  - SQL statements are defined as constants and, in the case of
    `listUserIssues`, use Go's `text/template` package to dynamically
    construct the `IN` clause for multiple `traceKeys`. This is a common
    pattern to avoid SQL injection vulnerabilities and handle variadic
    inputs efficiently.
  - `Save`: Inserts a new row into the `UserIssues` table. It includes a
    `last_modified` timestamp.
  - `Delete`: First, it attempts to retrieve the issue to ensure it exists
    before attempting deletion. This provides a more informative error
    message if the record is not found.
  - `GetUserIssuesForTraceKeys`: Constructs a SQL query using a template to
    select issues matching the provided trace keys and commit position
    range. It then iterates over the query results and populates a slice of
    `UserIssue` structs.

- **`sqluserissuestore/schema/schema.go`**: This file defines the Go struct
  `UserIssueSchema` which directly maps to the SQL table schema for
  `UserIssues`.

  - **Purpose**: This provides a typed representation of the database table,
    making it easier to reason about the data structure and to potentially
    use with ORM-like tools or schema migration utilities.
  - **Key Fields**:
  - `user_id TEXT NOT NULL`
  - `trace_key TEXT NOT NULL`
  - `commit_position INT NOT NULL`
  - `issue_id INT NOT NULL`
  - `last_modified TIMESTAMPTZ DEFAULT now()`
  - `PRIMARY KEY(trace_key, commit_position)`: The combination of
    `trace_key` and `commit_position` uniquely identifies a user issue,
    preventing multiple issues from being associated with the exact same
    data point.

- **`mocks/Store.go`**: This contains a mock implementation of the `Store`
  interface, generated using the `testify/mock` library.

  - **Purpose**: This is essential for unit testing components that depend
    on the `userissue.Store` without requiring a live database connection.
    It allows developers to define expected calls and return values for the
    store's methods.

**Workflow Example: Saving a User Issue**

1.  **User Action**: A user on the Perf frontend identifies a data point (e.g.,
    on a graph) and associates it with a Buganizer issue ID.
2.  **API Request**: The frontend sends a request to a backend API endpoint.
3.  **Backend Handler**: The API handler receives the request, which includes
    the user's ID, the trace key, the commit position, and the issue ID.
4.  **Store Interaction**: The handler creates a `userissue.UserIssue` struct
    and calls the `Save` method on an instance of `userissue.Store` (likely
    `sqluserissuestore.UserIssueStore`). `User Request (UI) | v API Endpoint | v
Backend Handler | | Creates userissue.UserIssue{UserId:"...",
TraceKey:"...", CommitPosition:123, IssueId:45678} v
userissue.Store.Save(ctx, &issue) | v
sqluserissuestore.UserIssueStore.Save() | | Constructs SQL: INSERT INTO
UserIssues (...) VALUES ($1, $2, $3, $4, $5) v SQL Database (UserIssues
Table) <-- Row inserted`

**Workflow Example: Retrieving User Issues for a Chart**

1.  **User Action**: A user views a performance chart displaying multiple traces
    over a range of commits.
2.  **Frontend Request**: The frontend needs to know if any data points on the
    visible traces and commit range have associated issues. It requests this
    information from a backend API.
3.  **Backend Handler**: The API handler receives the list of trace keys visible
    on the chart and the start/end commit positions.
4.  **Store Interaction**: The handler calls `GetUserIssuesForTraceKeys` on the
    `userissue.Store`. `Chart Display Request (UI) | | Provides:
traceKeys=["trace1", "trace2"], startCommit=100, endCommit=200 v API
Endpoint | v Backend Handler | v
userissue.Store.GetUserIssuesForTraceKeys(ctx, traceKeys, startCommit,
endCommit) | v sqluserissuestore.UserIssueStore.GetUserIssuesForTraceKeys()
| | Constructs SQL: SELECT ... FROM UserIssues WHERE trace_key IN ('trace1',
'trace2') AND commit_position>=100 AND commit_position<=200 v SQL Database
(UserIssues Table) | | Returns rows matching the query v Backend Handler | |
Formats response v API Endpoint | v UI (displays issue markers on chart)`

The design emphasizes a clear separation of concerns with the `Store` interface,
allowing for different storage backends if necessary (though SQL is the current
and likely long-term choice). The SQL implementation is straightforward, using
parameterized queries for security and templates for dynamic query construction
where appropriate.

# Module: /go/workflows

## Module: /go/workflows

### Overview

This module defines and implements Temporal workflows for automating tasks
related to performance anomaly detection and analysis in Skia Perf. It
orchestrates interactions between various services like the AnomalyGroup
service, Culprit service, and Gerrit service to achieve end-to-end automation.
The primary goal is to streamline the process of identifying performance
regressions, finding their root causes (culprits), and notifying relevant
parties.

The workflows are designed to be resilient and fault-tolerant, leveraging
Temporal's capabilities for retries and state management. This ensures that even
if individual steps or external services encounter transient issues, the overall
process can continue and eventually complete.

### Responsibilities and Key Components

The module is structured into a public API (`workflows.go`) and an internal
implementation package (`internal/`).

**`workflows.go`**:

- **Purpose**: Defines the public interface for the workflows, including their
  names and the data structures for their parameters and results.
- **Why**: This separation allows other modules (clients) to trigger these
  workflows without needing to know the internal implementation details or
  depend on the specific libraries used within the workflows. It acts as a
  contract.
- **Key Contents**:
  - **Workflow Name Constants (`ProcessCulprit`, `MaybeTriggerBisection`)**:
    These string constants are the canonical names used to invoke the
    respective workflows via the Temporal client. Using constants helps
    avoid typos and ensures consistency.
  - **Parameter and Result Structs (`ProcessCulpritParam`,
    `ProcessCulpritResult`, `MaybeTriggerBisectionParam`,
    `MaybeTriggerBisectionResult`)**: These structs define the data that
    needs to be passed into a workflow and the data that a workflow is
    expected to return upon completion. They ensure type safety and clarity
    in communication.

**`internal/` package**: This package contains the actual implementation of the
workflows and their associated activities. Activities are the building blocks of
Temporal workflows, representing individual units of work that can be executed,
retried, and timed out independently.

- **`options.go`**:

  - **Purpose**: Centralizes the configuration for Temporal activities and
    child workflows.
  - **Why**: Provides a consistent way to define timeouts and retry
    policies. This makes it easier to manage and adjust these settings
    globally or for specific categories of operations. For example,
    short-lived activities interacting with external services have different
    reliability characteristics than long-running child workflows.
  - **Key Components**:
  - `regularActivityOptions`: Defines default options (e.g., 1-minute
    timeout, 10 retry attempts) for standard activities that are expected to
    complete quickly, like API calls to other services.
  - `childWorkflowOptions`: Defines options for child workflows (e.g.,
    12-hour execution timeout, 4 retry attempts). This longer timeout
    accommodates potentially resource-intensive tasks like bisections which
    involve compilation and testing.

- **`maybe_trigger_bisection.go`**:

  - **Purpose**: Implements the `MaybeTriggerBisectionWorkflow`, which is
    the core logic for deciding whether to automatically find the cause of a
    performance regression (bisection) or to simply report the anomaly.
  - **Why**: This workflow automates a critical decision point in the
    performance analysis pipeline. It aims to reduce manual intervention by
    automatically initiating bisections for significant regressions while
    still allowing for manual reporting of less critical issues.
  - **Key Workflow Steps**:

  * **Wait**: Pauses for a defined duration (`_WAIT_TIME_FOR_ANOMALIES`,
    e.g., 30 minutes). This allows time for related anomalies to be detected
    and grouped together, potentially providing a more comprehensive picture
    before taking action. `Wait for more anomalies ->`
  * **Load Anomaly Group**: Retrieves details of the specific anomaly group
    using an activity that calls the AnomalyGroup service. `Load Anomaly
Group (Activity) AnomalyGroup Service <---> Workflow`
  * **Decision (Bisect or Report)**: Based on the `GroupAction` field of the
    anomaly group: - **If `BISECT`**: a. **Load Top Anomaly**: Fetches the most
    significant anomaly within the group. b. **Resolve Commit Hashes**:
    Converts the start and end commit positions of the anomaly into Git
    commit hashes using an activity that interacts with a Gerrit/Crrev
    service. `Get Commit Hashes (Activity) Gerrit/Crrev Service <--->
Workflow` c. **Launch Bisection (Child Workflow)**: Triggers a
    separate `CulpritFinderWorkflow` (defined in the
    `pinpoint/go/workflows` module) as a child workflow. This child
    workflow is responsible for performing the actual bisection. - A
    unique ID is generated for the Pinpoint job. - The child workflow is
    configured with `ParentClosePolicy: ABANDON`, meaning it will
    continue running even if this parent workflow terminates. This is
    crucial because bisections can be long-running. - Callback
    parameters are passed to the child workflow so it knows how to
    report its findings back (e.g., which Anomaly Group ID it's
    associated with, which Culprit service to use). `Launch Pinpoint
Bisection Workflow -----------------> Pinpoint.CulpritFinderWorkflow
(Child)` d. **Update Anomaly Group**: Records the ID of the launched
    bisection job back into the AnomalyGroup. `Update Anomaly Group with
Bisection ID (Activity) AnomalyGroup Service <---> Workflow` - **If `REPORT`**: a. **Load Top Anomalies**: Fetches a list of the
    top N anomalies in the group. b. **Notify User**: Calls an activity
    that uses the Culprit service to file a bug or send a notification
    about these anomalies. `Notify User of Anomalies (Activity) Culprit
Service <--------> Workflow`

  - **Helper Functions**:
  - `parseStatisticNameFromChart`, `benchmarkStoriesNeedUpdate`,
    `updateStoryDescriptorName`: These functions handle specific data
    transformations needed to correctly format parameters for the Pinpoint
    bisection request, often due to legacy conventions or differences in how
    metrics are named.

- **`process_culprit.go`**:

  - **Purpose**: Implements the `ProcessCulpritWorkflow`, which handles the
    results of a completed bisection (i.e., when one or more culprits are
    identified).
  - **Why**: This workflow bridges the gap between a successful bisection
    and making that information actionable. It ensures that found culprits
    are stored and that users are notified appropriately.
  - **Key Workflow Steps**:

  * **Convert Commits**: Transforms the commit data from the Pinpoint format
    to the format expected by the Culprit service. This involves parsing
    repository URLs.
  * **Persist Culprit**: Calls an activity to store the identified
    culprit(s) in a persistent datastore via the Culprit service. `Persist
Culprit (Activity) Culprit Service <--------> Workflow`
  * **Notify User of Culprit**: Calls an activity to notify users (e.g., by
    filing or updating a bug) about the identified culprit(s) via the
    Culprit service. `Notify User of Culprit (Activity) Culprit Service
<--------> Workflow`

  - **Helper Function**:
  - `ParsePinpointCommit`: Handles the parsing of repository URLs from the
    Pinpoint commit format (e.g., `https://{host}/{project}.git`) into
    separate host and project components required by the Culprit service.

- **`anomalygroup_service_activity.go`**:

  - **Purpose**: Defines activities that interact with the AnomalyGroup gRPC
    service.
  - **Why**: Encapsulates the client-side logic for communicating with the
    AnomalyGroup service. This makes the workflows themselves cleaner and
    focuses them on orchestration rather than low-level RPC details.
  - **Key Activities**:
  - `LoadAnomalyGroupByID`: Fetches an anomaly group by its ID.
  - `FindTopAnomalies`: Retrieves the most significant anomalies within a
    group.
  - `UpdateAnomalyGroup`: Updates an existing anomaly group (e.g., to add a
    bisection ID).

- **`culprit_service_activity.go`**:

  - **Purpose**: Defines activities that interact with the Culprit gRPC
    service.
  - **Why**: Similar to `anomalygroup_service_activity.go`, this
    encapsulates communication with the Culprit service.
  - **Key Activities**:
  - `PeristCulprit`: Stores culprit information.
  - `NotifyUserOfCulprit`: Notifies users about a found culprit (e.g., by
    creating a bug).
  - `NotifyUserOfAnomaly`: Notifies users about a set of anomalies (used
    when the group action is `REPORT`).

- **`gerrit_service_activity.go`**:

  - **Purpose**: Defines activities for interacting with Gerrit or a
    Gerrit-like service (specifically Crrev in this case) to resolve commit
    positions to commit hashes.
  - **Why**: Bisection workflows often start with commit positions (which
    are easier for humans or detection systems to reason about initially)
    but need actual Git hashes to perform the bisection. This activity
    provides that translation.
  - **Key Activity**:
  - `GetCommitRevision`: Takes a commit position (as an integer) and returns
    its corresponding Git hash.

**`worker/main.go`**:

- **Purpose**: This is the entry point for the Temporal worker process that
  hosts and executes the workflows and activities defined in this module.
- **Why**: Temporal workers are the processes that actually run the workflow
  and activity code. This `main` function sets up the worker, connects it to
  the Temporal server, and registers the workflows and activities it's capable
  of handling.
- **Key Operations**:
  1.  **Initialization**: Sets up logging and Prometheus metrics.
  2.  **Temporal Client Creation**: Establishes a connection to the Temporal
      frontend service.
  3.  **Worker Creation**: Creates a new Temporal worker associated with a
      specific task queue (e.g., `localhost.dev` or a production queue name).
      Workflows and activities are dispatched to workers listening on the
      correct task queue.
  4.  **Workflow Registration**: Registers `ProcessCulpritWorkflow` and
      `MaybeTriggerBisectionWorkflow` with the worker, associating them with
      their public names (e.g., `workflows.ProcessCulprit`).
  5.  **Activity Registration**: Registers instances of the activity structs
      (e.g., `CulpritServiceActivity`, `AnomalyGroupServiceActivity`,
      `GerritServiceActivity`) with the worker.
  6.  **Worker Start**: Starts the worker, which begins polling the specified
      task queue for tasks to execute.

### Key Workflows/Processes

**1. Anomaly Group Processing and Potential Bisection
(`MaybeTriggerBisectionWorkflow`)**

```
External Trigger (e.g., new AnomalyGroup created)
       |
       v
Start MaybeTriggerBisectionWorkflow(AG_ID)
       |
       +----------------------------------+
       | Wait (e.g., 30 mins)             |
       +----------------------------------+
       |
       v
LoadAnomalyGroupByID(AG_ID) ----> AnomalyGroup Service
       |
       +-----------+
       | GroupAction?|
       +-----------+
          /       \
         /         \
    BISECT         REPORT
      |              |
      v              v
FindTopAnomalies(AG_ID, Limit=1)  FindTopAnomalies(AG_ID, Limit=10)
      |              |
      v              v
GetCommitRevision(StartCommit) --> Gerrit   Anomalies --> Convert to CulpritService format
      |              |
      v              v
GetCommitRevision(EndCommit)   --> Gerrit   NotifyUserOfAnomaly(AG_ID, Anomalies) --> Culprit Service
      |
      v
Execute Pinpoint.CulpritFinderWorkflow (Child)
      |   (Async, ParentClosePolicy=ABANDON)
      |   Params: {StartHash, EndHash, Config, Benchmark, Story, ...
      |            CallbackParams: {AG_ID, CulpritServiceURL, GroupingTaskQueue}}
      |
      v
UpdateAnomalyGroup(AG_ID, BisectionID) --> AnomalyGroup Service
      |
      v
End Workflow
```

**2. Processing Bisection Results (`ProcessCulpritWorkflow`)**

This workflow is typically triggered as a callback by the Pinpoint
`CulpritFinderWorkflow` when it successfully identifies a culprit.

```
Pinpoint.CulpritFinderWorkflow completes
       | (Calls back to Temporal, invoking ProcessCulpritWorkflow)
       v
Start ProcessCulpritWorkflow(Commits, AG_ID, CulpritServiceURL)
       |
       +----------------------------------+
       | Convert Pinpoint Commits to       |
       | Culprit Service Format           |
       | (Parse Repository URLs)          |
       +----------------------------------+
       |
       v
PersistCulprit(Commits, AG_ID) --------> Culprit Service
       | (Returns CulpritIDs)
       v
NotifyUserOfCulprit(CulpritIDs, AG_ID) -> Culprit Service
       | (Returns IssueIDs, e.g., bug numbers)
       v
End Workflow
```

# Module: /integration

The `/integration` module provides a dataset and tools for conducting
integration tests on the Perf performance monitoring system. Its primary purpose
is to offer a controlled and reproducible environment for verifying the
ingestion and processing capabilities of Perf.

The core of this module is the `data` subdirectory. This directory houses a
collection of JSON files, each representing performance data associated with
specific commits from the `perf-demo-repo`
(https://github.com/skia-dev/perf-demo-repo.git). These files are structured
according to the `format.Format` schema defined in
`go.skia.org/infra/perf/go/ingest/format`. This standardized format is crucial
as it allows Perf's 'dir' type ingester to directly consume these files. The
dataset is intentionally designed to include a mix of valid data points and
specific error conditions:

- **Nine "good" files:** These represent typical, valid performance data that
  Perf should successfully ingest and process. Each file corresponds to a
  known commit in the `perf-demo-repo`.
- **One file with a "bad" commit:** This file (`demo_data_commit_10.json`)
  contains a `git_hash` that does not correspond to an actual commit in the
  `perf-demo-repo`. This allows testing how Perf handles data associated with
  unknown or invalid commit identifiers.
- **One malformed JSON file:** `malformed.json` is intentionally not a valid
  JSON file. This is used to test Perf's error handling capabilities when
  encountering incorrectly formatted input data.

The generation of these data files is handled by `generate_data.go`. This Go
program is responsible for creating the JSON files in the `data` directory. It
uses a predefined list of commit hashes from the `perf-demo-repo` and generates
random but plausible performance metrics for each. The inclusion of this
generator script is important because it allows developers to easily modify,
expand, or regenerate the test dataset if the testing requirements change or if
new scenarios need to be covered. The script uses `math/rand` for generating
some variability in the measurement values, ensuring the data isn't entirely
static while still being predictable.

The key workflow for utilizing this module in an integration test scenario would
look something like this:

1.  **Setup Perf:** Configure a local instance of Perf.
2.  **Configure Ingester:** Point Perf's 'dir' type ingester to the
    `/integration/data` directory. `Perf Instance --> Ingester (type: 'dir') -->
/integration/data/*.json`
3.  **Run Ingestion:** Trigger the ingestion process in Perf.
4.  **Verify:**
    - Confirm that the data from the nine "good" files is correctly ingested
      and displayed in Perf.
    - Check that Perf appropriately handles the file with the "bad" commit
      (e.g., logs an error, flags the data).
    - Verify that Perf correctly identifies and reports the error with the
      `malformed.json` file.

The `BUILD.bazel` file defines how the components of this module are built.

- The `data` `filegroup` makes the JSON test files available to other parts of
  the system, specifically for use in performance testing
  (`//perf:__subpackages__`).
- The `integration_lib` `go_library` encapsulates the logic from
  `generate_data.go`.
- The `integration` `go_binary` provides an executable to run
  `generate_data.go`, allowing for easy regeneration of the test data.

In essence, the `/integration` module provides a self-contained,
version-controlled set of test data and a mechanism to regenerate it. This is
crucial for ensuring the stability and correctness of Perf's data ingestion
pipeline by providing a consistent baseline for integration testing. The choice
to include both valid and intentionally erroneous data points allows for
comprehensive testing of Perf's data handling capabilities, including its
robustness in the face of invalid input.

# Module: /jupyter

The `/jupyter` module provides tools and examples for interacting with Skia's
performance data, specifically data from `perf.skia.org`. The primary goal is to
enable users to programmatically query, analyze, and visualize performance
metrics using the power of Python libraries like Pandas, NumPy, and Matplotlib
within a Jupyter Notebook environment.

The core functionality revolves around fetching and processing performance data.
This is achieved by providing Python functions that abstract the complexities of
interacting with the `perf.skia.org` API. This allows users to focus on the data
analysis itself rather than the underlying data retrieval mechanisms.

**Key Components/Files:**

- **`/jupyter/Perf+Query.ipynb`**: This is a Jupyter Notebook that serves as
  both an example and a utility library.

  - **Why**: It demonstrates how to use the provided Python functions to
    query performance data. It also contains the definitions of these key
    functions, making it a self-contained environment for performance
    analysis. The notebook format is chosen for its interactive nature,
    allowing users to execute code snippets, see results immediately, and
    experiment with different queries and visualizations.
  - **How**:

  - **`perf_calc(formula)`**: This function is designed to evaluate a
    specific formula against the performance data. It takes a string
    `formula` (e.g., `'count(filter(\"\"))'`) as input. The formula is sent
    to the `perf.skia.org` backend for processing. This function is useful
    when you need to perform calculations or aggregations on the data
    directly on the server side before retrieving it.

  - **`perf_query(query)`**: This function allows for more direct querying
    of performance data based on key-value pairs. It takes a query string
    (e.g., `'source_type=skp&sub_result=min_ms'`) that specifies the
    parameters for data retrieval. This is suitable when you want to fetch
    raw or filtered trace data.

  - **`perf_impl(body)`**: This is an internal helper function used by both
    `perf_calc` and `perf_query`. It handles the actual HTTP communication
    with `perf.skia.org`. It first determines the time range for the query
    (typically the last 50 commits by default) by fetching initial page
    data. Then, it sends the query or formula to the `/_/frame/start`
    endpoint, polls the `/_/frame/status` endpoint until the request is
    successful, and finally retrieves the results from `/_/frame/results`.
    The results are then processed into a Pandas DataFrame, which is a
    powerful data structure for analysis in Python. A special value `1e32`
    from the backend (often representing missing or invalid data) is
    converted to `np.nan` (Not a Number) for better handling in Pandas.

  - **`paramset()`**: This utility function fetches the available parameter
    set from `perf.skia.org`. This is useful for discovering the possible
    values for different dimensions like 'model', 'test', 'cpu_or_gpu',
    etc., which can then be used to construct more targeted queries.

  - **Examples**: The notebook is rich with examples showcasing how to use
    `perf_calc` and `perf_query`, plot the resulting DataFrames using
    Pandas' built-in plotting capabilities or Matplotlib directly, normalize
    data, calculate means, and perform more complex analyses like finding
    the noisiest hardware models or comparing CPU vs. GPU performance for
    specific tests. These examples serve as practical starting points for
    users.

  - **Workflow (Simplified `perf_impl`):**

  * `Client (Jupyter Notebook)` -- `GET /_/initpage/` --> `perf.skia.org`
    (Get time bounds)

  * `perf.skia.org` -- `Initial Data (JSON)` --> `Client`

  * `Client` -- `POST /_/frame/start (with query/formula & time bounds)` -->
    `perf.skia.org`

  * `perf.skia.org` -- `Request ID (JSON)` --> `Client`

  * `Client` -- `GET /_/frame/status/{ID}` --> `perf.skia.org` (Loop until
    'Success')

  * `perf.skia.org` -- `Status (JSON)` --> `Client`

  * `Client` -- `GET /_/frame/results/{ID}` --> `perf.skia.org`

  * `perf.skia.org` -- `Performance Data (JSON)` --> `Client`

  * `Client (Python)`: Parse JSON -> Create Pandas DataFrame -> Return
    DataFrame to user.

- **`/jupyter/README.md`**: This file provides instructions on setting up the
  necessary Python environment to run Jupyter Notebooks and the required
  libraries (Pandas, SciPy, Matplotlib).

  - **Why**: Python environment management can be tricky, especially with
    system-wide installations. Using a virtual environment (`virtualenv`) is
    recommended to isolate project dependencies and avoid conflicts.
  - **How**: It guides the user through installing `pip`, `python-dev`, and
    `python-virtualenv` using `apt-get` (assuming a Debian-based Linux
    system). It then shows how to create a virtual environment, activate it,
    upgrade `pip`, and install `jupyter`, `notebook`, `scipy`, `pandas`, and
    `matplotlib` within that isolated environment. Finally, it explains how
    to run the Jupyter Notebook server and deactivate the environment when
    done. This ensures a reproducible and clean setup for users wanting to
    utilize the `Perf+Query.ipynb` notebook.

The design emphasizes ease of use for data analysts and developers who need to
interact with Skia's performance data. By leveraging Jupyter Notebooks, it
provides an interactive and visual way to explore performance trends and issues.
The abstraction of API calls into simple Python functions (`perf_calc`,
`perf_query`) significantly lowers the barrier to entry for accessing this rich
dataset.

# Module: /lint

The `/lint` module is responsible for ensuring code quality and consistency
within the project by integrating and configuring JSHint, a popular JavaScript
linting tool.

The primary goal of this module is to provide a standardized way to identify and
report potential errors, stylistic issues, and anti-patterns in the JavaScript
codebase. This helps maintain code readability, reduces the likelihood of bugs,
and promotes adherence to established coding conventions.

The core component of this module is the `reporter.js` file. This file defines a
custom reporter function that JSHint will use to format and output the linting
results.

The decision to implement a custom reporter stems from the need to present
linting errors in a clear, concise, and actionable format. Instead of relying on
JSHint's default output, which might be too verbose or not ideally suited for
the project's workflow, `reporter.js` provides a tailored presentation.

The `reporter` function within `reporter.js` takes an array of error objects
(`res`) as input, where each object represents a single linting issue found by
JSHint. It then iterates through these error objects and constructs a formatted
string for each error. The format chosen is `filename:line:character message`,
which directly points developers to the exact location of the issue in the
source code.

For example: `src/myFile.js:10:5 Missing semicolon`

This specific format is chosen for its commonality in development tools and its
ease of integration with various editors and IDEs, allowing developers to
quickly navigate to the reported errors.

After processing all errors, if any were found, the `reporter` function
aggregates the formatted error strings and prints them to the standard output
(`process.stdout.write`). Additionally, it appends a summary line indicating the
total number of errors found, ensuring that developers have a quick overview of
the linting status. The pluralization of "error" vs. "errors" is also handled
for grammatical correctness.

The workflow can be visualized as:

```
JSHint analysis --[error objects]--> reporter.js --[formatted errors & summary]--> stdout
```

By controlling the output format, this module ensures that linting feedback is
consistently presented and easily digestible, contributing to a more efficient
development process. The design prioritizes providing actionable information to
developers, enabling them to address code quality issues promptly.

# Module: /migrations

This module is responsible for managing SQL database schema migrations for Perf.
Perf utilizes SQL backends to store various data, including trace data,
shortcuts, and alerts. As the application evolves, the database schema may need
to change. This module provides the mechanism to apply these changes and to
upgrade existing databases to the schema expected by the current Perf version.

The core of this system relies on the `github.com/golang-migrate/migrate/v4`
library. This library provides a robust framework for versioning database
schemas and applying migrations in a controlled manner.

The key design principle is to have a versioned set of SQL scripts for each
supported SQL dialect. This allows Perf to:

1.  **Initialize a new database** with the correct schema.
2.  **Upgrade an existing database** from an older schema version to the current
    one.
3.  **Rollback schema changes** if necessary, by providing "down" migrations.

Each SQL dialect (e.g., CockroachDB) has its own subdirectory within the
`/migrations` module. The naming convention for these directories is critical:
they must match the values defined in `sql.Dialect`.

Inside each dialect-specific directory, migration files are organized by
version.

- File names are prefixed with a 0-padded version number (e.g., `0001_`,
  `0002_`).
- For each version, there are two files:
  - An `.up.` file (e.g., `0001_create_initial_tables.up.sql`): Contains SQL
    statements to apply the schema changes for that version.
  - A `.down.` file (e.g., `0001_create_initial_tables.down.sql`): Contains
    SQL statements to revert the schema changes introduced by the
    corresponding `.up.` file.

This paired approach ensures that migrations can be applied and rolled back
smoothly.

**Key Files and Responsibilities:**

- `README.md`: Provides a high-level overview of the migration system,
  explaining its purpose and the use of the `golang-migrate/migrate` library.
  It also details the directory structure and file naming conventions for
  migration scripts.
- `cockroachdb/`: This directory contains the migration scripts specifically
  for the CockroachDB dialect.
  - `cockroachdb/0001_create_initial_tables.up.sql`: This is the first
    migration script for CockroachDB. It defines the initial schema for
    Perf, creating tables such as `TraceValues`, `SourceFiles`, `ParamSets`,
    `Postings`, `Shortcuts`, `Alerts`, `Regressions`, and `Commits`. The
    table definitions include primary keys, indexes, and column types
    tailored for efficient data storage and retrieval specific to Perf's
    needs (e.g., storing trace data, associating traces with source files,
    managing alert configurations, and tracking commit history). The schema
    is designed to support the various functionalities of Perf, such as
    querying traces by parameters, retrieving trace values over commit
    ranges, and linking regressions to specific alerts and commits.
  - `cockroachdb/0001_create_initial_tables.down.sql`: This file is intended
    to contain SQL statements to drop the tables created by its
    corresponding `.up.` script. However, as a safety precaution against
    accidental data loss, it is currently empty. The design acknowledges the
    potential danger of automated table drops in a production environment.
- `cdb.sql`: This is a utility SQL script designed for developers to interact
  with and test queries against a CockroachDB instance populated with Perf
  data. It includes sample `INSERT` statements to populate tables with test
  data and various `SELECT` queries demonstrating common data retrieval
  patterns used by Perf. This file is not part of the automated migration
  process but serves as a helpful tool for development and debugging. It
  showcases how to query for traces based on parameters, retrieve trace
  values, find the most recent tile, and get source file information. It also
  includes examples of more complex queries involving `INTERSECT` and `JOIN`
  operations, reflecting the kinds of queries Perf might execute.
- `test.sql`: Similar to `cdb.sql`, this script is for testing and
  experimentation, but it's tailored for a SQLite database. It creates a
  schema similar to the CockroachDB one (though potentially simplified or with
  slight variations due to dialect differences) and populates it with test
  data. It contains a series of `CREATE TABLE`, `INSERT`, and `SELECT`
  statements that developers can use to quickly set up a local test
  environment and verify SQL logic.
- `batch-delete.sh` and `batch-delete.sql`: These files provide a mechanism
  for performing batch deletions of specific parameter data from the
  `ParamSets` table in a CockroachDB instance.
  - `batch-delete.sql`: Contains the `DELETE` SQL statement. It is designed
    to be edited directly to specify the deletion criteria (e.g.,
    `tile_number`, `param_key`, `param_value` ranges) and the `LIMIT` for
    the number of rows deleted in each batch. This batching approach is
    crucial for deleting large amounts of data without overwhelming the
    database or causing long-running transactions.
  - `batch-delete.sh`: A shell script that repeatedly executes
    `batch-delete.sql` using the `cockroach sql` command-line tool. It runs
    in a loop with a short sleep interval, allowing for controlled,
    iterative deletion. This script assumes that a port-forward to the
    CockroachDB instance is already established. This utility is likely used
    for data cleanup or maintenance tasks that require removing specific,
    potentially large, datasets.

**Migration Workflow (Conceptual):**

When Perf starts or when a migration command is explicitly run:

1.  **Determine Current Schema Version:** The `golang-migrate/migrate` library
    connects to the database and checks the current schema version (often stored
    in a dedicated migrations table managed by the library itself).
2.  **Identify Target Schema Version:** This is typically the highest version
    number found among the migration files for the configured SQL dialect.
3.  **Apply Pending Migrations:**

        - If the current schema version is lower than the target version, the
          library iteratively executes the `.up.sql` files in ascending order of
          their version numbers, starting from the version immediately following
          the current one, up to the target version.
        - Each successful `.up.` migration updates the schema version in the
          database.

        Example: Current Version = 0, Target Version = 2 `DB State (v0) --> Run

    0001*\*.up.sql --> DB State (v1) --> Run 0002*\*.up.sql --> DB State (v2)`

4.  **Rollback Migrations (if needed):**

        - If a user needs to revert to an older schema version, the library can
          execute the `.down.sql` files in descending order.

        Example: Current Version = 2, Target Rollback Version = 0 `DB State (v2) -->

    Run 0002*\*.down.sql --> DB State (v1) --> Run 0001*\*.down.sql --> DB State
    (v0)`

The `BUILD.bazel` file defines a `filegroup` named `cockroachdb` which bundles
all files under the `cockroachdb/` subdirectory. This is likely used by other
parts of the Perf build system, perhaps to package these migration scripts or
make them accessible to the Perf application when it needs to perform
migrations.

# Module: /modules

## Modules Documentation

### Overview

The `modules` directory contains a collection of frontend TypeScript modules
that constitute the building blocks of the Perf web application's user
interface. These modules primarily define custom HTML elements (web components)
and utility functions for various UI functionalities, data processing, and
interaction with backend services. The architecture emphasizes modularity,
reusability, and a component-based approach, largely leveraging the Lit library
for creating custom elements and `elements-sk` for common UI widgets.

The design philosophy encourages separation of concerns:

- **UI Components:** Dedicated custom elements encapsulate specific UI
  features like plotting, alert configuration, data tables, dialogs, and input
  controls.
- **Data Handling:** Modules like `dataframe` and `progress` manage data
  fetching, processing, and state.
- **Utilities:** Modules like `paramtools`, `pivotutil`, `cid`, and `trybot`
  provide common functionalities for data manipulation, key parsing, and
  specific calculations.
- **Styling and Theming:** A centralized `themes` module ensures a consistent
  visual appearance, building upon `infra-sk`'s theming capabilities.
- **JSON Contracts:** The `json` module defines TypeScript interfaces that
  mirror backend Go structures, ensuring type safety in client-server
  communication.

This modular structure aims to create a maintainable and scalable frontend
codebase. Each module typically includes its core logic, associated styles, demo
pages for isolated development and testing, and unit/integration tests.

### Key Responsibilities and Components

A significant portion of the modules is dedicated to creating custom HTML
elements that serve as interactive UI components. These elements often
encapsulate complex behavior and interactions, simplifying their use in
higher-level page components.

**Data Visualization and Interaction:**

- `plot-simple-sk`: A custom-built canvas-based plotting element for rendering
  interactive line graphs, optimized for performance with features like dual
  canvases, Path2D objects, and k-d trees for point proximity.
- `plot-google-chart-sk`: An alternative plotting element that wraps the
  Google Charts library, offering a rich set of features and interactivity
  like panning, zooming, and trace visibility toggling.
- `plot-summary-sk`: Displays a summary plot (often using Google Charts) and
  allows users to select a range, which is useful for overview and drill-down
  scenarios.
- `chart-tooltip-sk`: Provides a detailed, interactive tooltip for data points
  on charts, showing commit information, anomaly details, and actions like
  bisection or requesting traces.
- `graph-title-sk`: Displays a structured title for graphs, showing key-value
  parameter pairs associated with the plotted data.
- `word-cloud-sk`: Visualizes key-value pairs and their frequencies as a
  textual list with proportional bars.

**Alert and Regression Management:**

- `alert-config-sk`: A UI for creating and editing alert configurations,
  including query definition, detection algorithms, and notification settings.
- `alerts-page-sk`: A page for viewing, creating, and managing all alert
  configurations.
- `cluster-summary2-sk`: Displays a detailed summary of a performance cluster,
  including a plot, statistics, and triage controls.
- `anomalies-table-sk`: Renders a sortable and interactive table of detected
  performance anomalies, allowing for grouping and bulk actions like triage
  and graphing.
- `anomaly-sk`: Displays detailed information about a single performance
  anomaly.
- `triage-status-sk`: A simple button-like element indicating the current
  triage status of a cluster and allowing users to initiate the triage
  process.
- `triage-menu-sk`: Provides a menu for bulk triage actions on selected
  anomalies, including assigning bugs or marking them as ignored.
- `new-bug-dialog-sk`: A dialog for filing new bugs related to anomalies,
  pre-filling details.
- `existing-bug-dialog-sk`: A dialog for associating anomalies with existing
  bug reports.
- `user-issue-sk`: Manages the association of user-reported Buganizer issues
  with specific data points.
- `bisect-dialog-sk`: A dialog for initiating a Pinpoint bisection process to
  find the commit causing a regression.
- `pinpoint-try-job-dialog-sk`: A (legacy) dialog for initiating Pinpoint A/B
  try jobs to request additional traces.
- `triage-page-sk`: A page dedicated to viewing and triaging regressions based
  on time range and filters.
- `regressions-page-sk`: A page for viewing regressions associated with
  specific "subscriptions" (e.g., sheriff configs).
- `subscription-table-sk`: Displays details of a subscription and its
  associated alerts.
- `revision-info-sk`: Displays information about anomalies detected around a
  specific revision.

**Data Input and Selection:**

- `query-sk`: A comprehensive UI for constructing complex queries by selecting
  parameters and their values.
- `paramset-sk`: Displays a set of parameters and their values, often used to
  summarize a query or data selection.
- `query-chooser-sk`: Combines `paramset-sk` (for summary) and `query-sk` (in
  a dialog) for a compact query selection experience.
- `query-count-sk`: Shows the number of items matching a given query, fetching
  this count from a backend endpoint.
- `commit-detail-picker-sk`: Allows users to select a specific commit from a
  range, typically presented in a dialog with date range filtering.
- `commit-detail-panel-sk`: Displays a list of commit details, making them
  selectable.
- `commit-detail-sk`: Displays information about a single commit with action
  buttons.
- `calendar-input-sk`: A date input field combined with a calendar picker
  dialog.
- `calendar-sk`: A standalone interactive calendar widget.
- `day-range-sk`: Allows selection of a "begin" and "end" date.
- `domain-picker-sk`: Allows selection of a data domain either by date range
  or by a number of recent commits.
- `test-picker-sk`: A guided, multi-step picker for selecting tests or traces
  by sequentially choosing parameter values.
- `picker-field-sk`: A text input field with a filterable dropdown menu of
  predefined options, built using Vaadin ComboBox.
- `algo-select-sk`: A dropdown for selecting a clustering algorithm.
- `split-chart-menu-sk`: A menu for selecting an attribute by which to split a
  chart.
- `pivot-query-sk`: A UI for configuring pivot table requests (group by,
  operations, summaries).
- `triage2-sk`: A set of three buttons for selecting a triage status
  (positive, negative, untriaged).
- `tricon2-sk`: An icon that visually represents one of the three triage
  states.

**Data Display and Structure:**

- `pivot-table-sk`: Displays pivoted DataFrame data in a sortable table.
- `json-source-sk`: A dialog for viewing the raw JSON source data for a
  specific trace point.
- `ingest-file-links-sk`: Displays relevant links (e.g., to Swarming,
  Perfetto) associated with an ingested data point.
- `point-links-sk`: Displays links from ingestion files and generates commit
  range links between data points.
- `commit-range-sk`: Dynamically generates a URL to a commit range viewer
  based on begin and end commits.

**Scaffolding and Application Structure:**

- `perf-scaffold-sk`: Provides the consistent layout, header, and navigation
  sidebar for all Perf application pages.
- `explore-simple-sk`: The core element for exploring and visualizing
  performance data, including querying, plotting, and anomaly interaction.
- `explore-sk`: Wraps `explore-simple-sk`, adding features like user
  authentication, default configurations, and optional integration with
  `test-picker-sk`.
- `explore-multi-sk`: Allows displaying and managing multiple
  `explore-simple-sk` graphs simultaneously, with shared controls and shortcut
  management.
- `favorites-dialog-sk`: A dialog for adding or editing bookmarked "favorites"
  (named URLs).
- `favorites-sk`: Displays and manages a user's list of favorites.

**Backend Interaction and Data Processing Utilities:**

- `cid/cid.ts`: Provides `lookupCids` to fetch detailed commit information
  based on commit numbers.
- `common/plot-builder.ts` & `common/plot-util.ts`: Utilities for transforming
  `DataFrame` and `TraceSet` data into formats suitable for plotting libraries
  (especially Google Charts) and for creating consistent chart options.
- `common/test-util.ts`: Sets up mocked API responses (`fetch-mock`) for
  various backend endpoints, facilitating isolated testing and demo page
  development.
- `const/const.ts`: Defines shared constants, notably `MISSING_DATA_SENTINEL`
  for representing missing data points, ensuring consistency with the backend.
- `csv/index.ts`: Converts `DataFrame` objects into CSV format for data
  export.
- `dataframe/index.ts` & `dataframe/dataframe_context.ts`: Core logic for
  managing and manipulating `DataFrame` objects. `DataFrameRepository` (a
  LitElement context provider) handles fetching, caching, merging, and
  providing `DataFrame` and `DataTable` objects to consuming components.
- `dataframe/traceset.ts`: Utilities for extracting and formatting information
  from trace keys within DataFrames/DataTables, such as generating chart
  titles and legends.
- `errorMessage/index.ts`: A wrapper around `elements-sk`'s `errorMessage` to
  display persistent error messages by default.
- `json/index.ts`: Contains TypeScript interfaces and types that define the
  structure of JSON data exchanged with the backend, crucial for type safety
  and often auto-generated from Go structs.
- `paramtools/index.ts`: Client-side utilities for creating, parsing, and
  manipulating `ParamSet` objects and structured trace keys (e.g., `makeKey`,
  `fromKey`, `queryFromKey`).
- `pivotutil/index.ts`: Utilities for validating pivot table requests
  (`pivot.Request`) and providing descriptions for pivot operations.
- `progress/progress.ts`: Implements `startRequest` for initiating and polling
  the status of long-running server-side tasks, providing progress updates to
  the UI.
- `trace-details-formatter/traceformatter.ts`: Provides `TraceFormatter`
  implementations (default and Chrome-specific) for converting trace parameter
  sets to display strings and vice-versa for querying.
- `trybot/calcs.ts`: Calculates and aggregates `stddevRatio` values from Perf
  trybot results, grouping them by parameter to identify performance impacts.
- `trybot-page-sk`: A page for analyzing performance regressions based on
  commit or trybot run, using `trybot/calcs` for analysis.
- `window/index.ts`: Utilities related to the browser `window` object,
  including parsing build tag information from `window.perf.image_tag`.

**Core Architectural Patterns:**

- **Custom Elements (Web Components):** The UI is primarily built using custom
  elements, promoting encapsulation, reusability, and interoperability. Most
  elements extend `ElementSk` from `infra-sk`.
- **Lit Library:** Widely used for defining custom elements, providing
  efficient templating (`lit-html`) and reactive updates.
- **State Management:**
  - Local component state is managed within the elements themselves.
  - `stateReflector` (from `infra-sk`) is frequently used to synchronize
    component state with URL query parameters, enabling bookmarking and
    shareable views (e.g., `alerts-page-sk`, `explore-simple-sk`,
    `triage-page-sk`).
  - Lit contexts (`@lit/context`) are used for providing shared data down
    the component tree without prop drilling, notably in
    `dataframe/dataframe_context.ts` for `DataFrame` objects.
- **Event-Driven Communication:** Components often communicate using custom
  DOM events. Child components emit events, and parent components listen and
  react to them (e.g., `query-sk` emits `query-change`, `triage-status-sk`
  emits `start-triage`).
- **Asynchronous Operations:** `fetch` API is used for backend communication.
  Promises and `async/await` are standard for handling these asynchronous
  operations. Spinners (`spinner-sk`) provide user feedback during loading.
- **Modularity and Dependencies:** Modules are designed to be relatively
  self-contained, with clear dependencies declared in `BUILD.bazel` files.
  This allows for better organization and easier maintenance.
- **Testing:** Each module typically has associated demo pages (`*-demo.html`,
  `*-demo.ts`) for isolated development and visual testing, Karma unit tests
  (`*_test.ts`), and Puppeteer end-to-end/screenshot tests
  (`*_puppeteer_test.ts`). `fetch-mock` is extensively used in demos and tests
  to simulate backend responses.

This comprehensive set of modules forms a rich ecosystem for building and
maintaining the Perf application's frontend, with a strong emphasis on modern
web development practices and reusability.

# Module: /modules/alert

## Alert Module Documentation

### Overview

The `alert` module is responsible for validating the configuration of alerts
within the Perf system. Its primary function is to ensure that alert definitions
adhere to a set of predefined rules, guaranteeing their proper functioning and
preventing errors. This module plays a crucial role in maintaining the
reliability of the alerting system by catching invalid configurations before
they are deployed.

### Design Decisions and Implementation Choices

The core design principle behind this module is simplicity and focused
responsibility. Instead of incorporating complex validation logic directly into
other parts of the system (like the UI or backend services that handle alert
creation/modification), this module provides a dedicated, reusable validation
function. This promotes modularity and makes the validation logic easier to
maintain and update.

The choice of using a simple function (`validate`) that returns a string (empty
for valid, error message for invalid) is intentional. This approach is
straightforward to understand and integrate into various parts of the
application. It avoids throwing exceptions for validation failures, which can
sometimes complicate control flow, and instead provides clear, human-readable
feedback.

The current validation is intentionally minimal, focusing on the essential
requirement of a non-empty query. This is a pragmatic approach, starting with
the most critical validation and allowing for the addition of more complex rules
as the system evolves. The dependency on `//perf/modules/json:index_ts_lib`
indicates that the structure of an `Alert` is defined externally, and this
module consumes that definition.

### Key Components and Responsibilities

- **`index.ts`**: This is the central file of the module.
  - **Responsibility**: It houses the primary validation logic for `Alert`
    configurations.
  - **`validate(alert: Alert): string` function**:
  - **Purpose**: This function is the public API of the module. It takes an
    `Alert` object (as defined in the `../json` module) as input.
  - **How it works**: It performs a series of checks on the properties of
    the `alert` object. Currently, it verifies that the `query` property of
    the `Alert` is present and not an empty string.
  - **Output**: If all checks pass, it returns an empty string, signifying
    that the `Alert` configuration is valid. If any check fails, it returns
    a string containing a descriptive error message indicating why the
    `Alert` is considered invalid. This message is intended to be
    user-friendly and help in correcting the configuration.

### Key Workflows

**Alert Validation Workflow:**

```
External System (e.g., UI, API)  -- Passes Alert object --> [alert/index.ts: validate()]
                                                                    |
                                                                    V
                                                       [ Is alert.query non-empty? ]
                                                                    |
                                         +--------------------------+--------------------------+
                                         | (Yes)                                            | (No)
                                         V                                                  V
                                [ Returns "" (empty string) ]           [ Returns "An alert must have a non-empty query." ]
                                         |                                                  |
                                         V                                                  V
External System <-- Receives validation result -- [ Interprets result (valid/invalid) ]
```

This workflow illustrates how an external system would interact with the
`validate` function. The external system provides an `Alert` object, and the
`validate` function returns a string. The external system then uses this string
to determine if the alert configuration is valid and can proceed accordingly
(e.g., save the alert, display an error to the user).

# Module: /modules/alert-config-sk

The `alert-config-sk` module provides a custom HTML element,
`<alert-config-sk>`, designed for creating and editing alert configurations
within the Perf application. This element serves as a user interface for
defining the conditions under which an alert should be triggered, how
regressions are detected, and where notifications should be sent.

**Core Functionality and Design:**

The primary goal of `alert-config-sk` is to offer a comprehensive yet
user-friendly way to manage alert settings. It encapsulates all the necessary
input fields and logic for defining an `Alert` object, which is a central data
structure in Perf for representing alert configurations.

Key design considerations include:

- **Modularity and Reusability:** By packaging the alert configuration UI as a
  custom element, it can be easily integrated into various parts of the Perf
  application where alert management is needed.
- **Dynamic UI based on Context:** The UI adapts based on global settings
  (e.g., `window.perf.notifications`, `window.perf.display_group_by`,
  `window.perf.need_alert_action`). This allows the same component to present
  different options depending on the specific Perf instance's configuration or
  the user's context. For example, the notification options (email vs. issue
  tracker) and the visibility of "Group By" settings can change.
- **Data Binding and Reactivity:** The element uses Lit library for templating
  and reactivity. Changes in the input fields directly update the internal
  `_config` object, and changes to the element's properties (like `config`,
  `paramset`) trigger re-renders.
- **Integration with other Perf modules:** It leverages other custom elements
  like `query-chooser-sk` for selecting traces, `algo-select-sk` for choosing
  clustering algorithms, and various `elements-sk` components (e.g.,
  `select-sk`, `multi-select-sk`, `checkbox-sk`) for standard UI inputs. This
  promotes consistency and reduces redundant code.
- **User Feedback and Validation:** The component provides immediate feedback,
  such as displaying different threshold units based on the selected step
  detection algorithm and validating input for fields like the Issue Tracker
  Component ID. It also includes "Test" buttons to verify alert notification
  and bug template configurations.

**Key Components and Files:**

- **`alert-config-sk.ts`:** This is the heart of the module, defining the
  `AlertConfigSk` class which extends `ElementSk`.
  - **Properties:**
  - `config`: An `Alert` object representing the current alert configuration
    being edited. This is the primary data model for the component.
  - `paramset`: A `ParamSet` object providing the available parameters and
    their values for constructing queries (used by `query-chooser-sk`).
  - `key_order`: An array of strings dictating the preferred order of keys
    in the `query-chooser-sk`.
  - **Templating (`template` static method):** Uses `lit-html` to define the
    structure and content of the element. It dynamically renders sections
    based on the current configuration and global settings (e.g.,
    `window.perf.notifications`).
  - **Event Handling:** Listens to events from child components (e.g.,
    `query-change` from `query-chooser-sk`, `selection-changed` from
    `select-sk`) to update the `_config` object.
  - **Logic for Dynamic UI:**
  - The `thresholdDescriptors` object maps step detection algorithms to
    their corresponding units and descriptive labels, ensuring the
    "Threshold" input field is always relevant.
  - Conditional rendering (e.g., using `?` operator in lit-html or `if`
    statements in helper functions like `_groupBy`) is used to show/hide UI
    elements based on `window.perf` flags.
  - **API Interaction:**
  - `testBugTemplate()`: Sends a `POST` request to `/_/alert/bug/try` to
    test the configured bug URI template.
  - `testAlert()`: Sends a `POST` request to `/_/alert/notify/try` to test
    the alert notification setup.
  - **Helper Functions:**
  - `toDirection()`, `toConfigState()`: Convert string values from UI
    selections to the appropriate enum types for the `Alert` object.
  - `indexFromStep()`: Determines the correct selection index for the "Step
    Detection" dropdown based on the current `_config.step` value.
- **`alert-config-sk.scss`:** Contains the SASS styles for the element,
  ensuring a consistent look and feel within the Perf application. It imports
  styles from `themes_sass_lib` and `buttons_sass_lib` for theming and button
  styling.
- **`alert-config-sk-demo.html` and `alert-config-sk-demo.ts`:** Provide a
  demonstration page for the `alert-config-sk` element.
  - The HTML sets up a basic page structure with an instance of
    `alert-config-sk` and buttons to manipulate global `window.perf`
    settings, allowing developers to test different UI states of the
    component.
  - The TypeScript file initializes the demo, sets up mock `paramset` and
    `config` data, and provides event listeners for the control buttons to
    refresh the `alert-config-sk` component and display its current state.
    This is crucial for development and testing.
- **`alert-config-sk_puppeteer_test.ts`:** Contains Puppeteer tests for the
  component. These tests verify that the component renders correctly in
  different states (e.g., with/without group_by, different notification
  options) by interacting with the demo page and taking screenshots.
- **`index.ts`:** A simple entry point that imports and thereby registers the
  `alert-config-sk` custom element, making it available for use in HTML.

**Workflow Example: Editing an Alert**

1.  **Initialization:**

    - An instance of `alert-config-sk` is added to the DOM.
    - The `paramset` property is set, providing the available trace
      parameters.
    - The `config` property is set with the `Alert` object to be edited (or a
      default new configuration).
    - Global `window.perf` settings influence which UI sections are initially
      visible.

2.  **User Interaction:**

    - The user modifies various fields: Display Name, Category, Query (via
      `query-chooser-sk`), Grouping (via `algo-select-sk`), Step Detection,
      Threshold, etc.
    - As the user changes a field (e.g., selects a new "Step Detection"
      algorithm from the `select-sk`):
      - An event is dispatched by the child component (e.g.,
        `selection-changed`).
      - `alert-config-sk` listens for this event.
      - The event handler in `alert-config-sk.ts` updates the corresponding
        property in its internal `_config` object (e.g.,
        `this._config.step = newStepValue`).
      - The component re-renders (managed by Lit) to reflect the change. For
        instance, if the "Step Detection" changes, the "Threshold" label and
        units dynamically update.

    ```
    User interacts with <select-sk id="step">
        |
        V
    <select-sk> emits 'selection-changed' event
        |
        V
    AlertConfigSk.stepSelectionChanged(event) is called
        |
        V
    this._config.step is updated
        |
        V
    this._render() is (indirectly) called by Lit
        |
        V
    UI updates, e.g., label for "Threshold" input changes
    ```

3.  **Testing Configuration (Optional):**

    - User clicks "Test" for bug template:
      - `AlertConfigSk.testBugTemplate()` is called.
      - A POST request is made to `/_/alert/bug/try`.
      - The response (a URL to the bug) is opened in a new tab, or an error
        is shown.
    - User clicks "Test" for alert notification:
      - `AlertConfigSk.testAlert()` is called.
      - A POST request is made to `/_/alert/notify/try`.
      - A success/error message is displayed.

4.  **Saving Changes:**

    - The parent component or application logic that hosts `alert-config-sk`
      is responsible for retrieving the updated `config` object from the
      `alert-config-sk` element (e.g., `element.config`) and persisting it
      (e.g., by sending it to a backend API). `alert-config-sk` itself does
      not handle the saving of the configuration to a persistent store.

This element aims to simplify the complex task of configuring alerts by
providing a structured and reactive interface, abstracting away the direct
manipulation of the underlying `Alert` JSON object for the end-user.

# Module: /modules/alerts-page-sk

## alerts-page-sk Module Documentation

### High-Level Overview

The `alerts-page-sk` module provides a user interface for managing and
configuring alerts within the Perf application. Users can view, create, edit,
and delete alert configurations. The page displays existing alerts in a table
and provides a dialog for detailed configuration of individual alerts. It
interacts with a backend API to fetch and persist alert data.

### Design Decisions and Implementation Choices

**Why a dedicated page for alerts?** Centralizing alert management provides a
clear and focused interface for users responsible for monitoring performance
metrics. This separation of concerns simplifies the overall application
structure and user experience.

**How are alerts displayed and managed?** Alerts are displayed in a tabular
format, offering a quick overview of key information like name, query, owner,
and status. Icons are used for common actions like editing and deleting,
enhancing usability. A modal dialog, utilizing the `<dialog>` HTML element and
the `alert-config-sk` component, is employed for focused editing of individual
alert configurations. This approach avoids cluttering the main page and provides
a dedicated space for detailed settings.

**Why use Lit for templating?** Lit is used for its efficient rendering and
component-based architecture. This allows for a declarative way to define the UI
and manage its state, making the code more maintainable and easier to
understand. The use of `html` tagged template literals provides a clean and
JavaScript-native way to write templates.

**How is user authorization handled?** The page checks if the logged-in user has
an 'editor' role. This is determined by fetching the user's status from
`/_/login/status`. Editing and creation functionalities are disabled if the user
lacks the necessary permissions, preventing unauthorized modifications. The
logged-in user's email is also pre-filled as the owner for new alerts.

**Why is `fetch-mock` used in the demo?** `fetch-mock` is utilized in the demo
(`alerts-page-sk-demo.ts`) to simulate backend API responses. This allows for
isolated testing and development of the frontend component without requiring a
running backend. It enables developers to define expected responses for various
API endpoints, facilitating a predictable environment for UI development and
testing.

**How are API interactions handled?** The component uses the `fetch` API to
communicate with the backend. Helper functions like `jsonOrThrow` and
`okOrThrow` are used to simplify response handling and error management.
Specific endpoints are used for listing (`/_/alert/list/...`), creating
(`/_/alert/new`), updating (`/_/alert/update`), and deleting
(`/_/alert/delete/...`) alerts.

**Why distinguish between "Alert" and "Component" in the UI?** The UI adapts to
display either an "Alert" field or an "Issue Tracker Component" field based on
the `window.perf.notifications` global setting. This allows the application to
integrate with different notification systems. If `markdown_issuetracker` is
configured, it links directly to the relevant issue tracker component.

### Responsibilities and Key Components/Files

- **`alerts-page-sk.ts`**: This is the core TypeScript file defining the
  `AlertsPageSk` custom element.

  - **Responsibilities**:
  - Fetching and displaying a list of alert configurations.
  - Providing functionality to create new alerts.
  - Enabling editing of existing alerts through a modal dialog.
  - Allowing deletion of alerts.
  - Handling user authorization for edit/create operations.
  - Managing the state of the "show deleted alerts" checkbox.
  - Interacting with the backend API for all alert-related operations.
  - Rendering the UI using Lit templates.
  - **Key Methods**:
  - `connectedCallback()`: Initializes the component by fetching initial
    data (paramset and alert list).
  - `list()`: Fetches and re-renders the list of alerts.
  - `add()`: Initiates the creation of a new alert by fetching a default
    configuration from the server and opening the edit dialog.
  - `edit()`: Opens the edit dialog for an existing alert.
  - `accept()`: Handles the submission of changes from the edit dialog,
    sending an update request to the server.
  - `delete()`: Sends a request to the server to delete an alert.
  - `openOnLoad()`: Checks the URL for an alert ID on page load and, if
    present, opens the edit dialog for that specific alert. This allows for
    direct linking to an alert's configuration.
  - **Key Properties**:
  - `alerts`: An array holding the currently displayed alert configurations.
  - `_cfg`: The `Alert` object currently being edited in the dialog.
  - `isEditor`: A boolean indicating if the current user has editing
    privileges.
  - `dialog`: A reference to the HTML `<dialog>` element used for editing.
  - `alertconfig`: A reference to the `alert-config-sk` element within the
    dialog.

- **`alerts-page-sk.scss`**: Contains the SASS/CSS styles for the
  `alerts-page-sk` element.

  - **Responsibilities**: Defines the visual appearance of the alerts table,
    buttons, dialog, and other UI elements within the page. It ensures a
    consistent look and feel, including theming (dark mode).

- **`alerts-page-sk-demo.ts`**: Provides a demonstration and development
  environment for the `alerts-page-sk` component.

  - **Responsibilities**:
  - Sets up `fetch-mock` to simulate backend API responses for
    `/login/status`, `/_/count/`, `/_/alert/update`, `/_/alert/list/...`,
    `/_/initpage/`, and `/_/alert/new`. This allows the component to be
    developed and tested in isolation.
  - Initializes global `window.perf` properties that might affect the
    component's behavior (e.g., `key_order`, `display_group_by`,
    `notifications`).
  - Dynamically inserts `alerts-page-sk` elements into the demo HTML page.

- **`alerts-page-sk-demo.html`**: The HTML structure for the demo page.

  - **Responsibilities**: Provides the basic HTML layout where the
    `alerts-page-sk` component is rendered for demonstration purposes.
    Includes an `<error-toast-sk>` for displaying error messages.

- **`alerts-page-sk_puppeteer_test.ts`**: Contains Puppeteer tests for the
  `alerts-page-sk` component.

  - **Responsibilities**: Performs automated UI testing, ensuring the
    component renders correctly and basic interactions function as expected.
    It takes screenshots for visual regression testing.

- **`index.ts`**: A simple entry point that imports and thereby registers the
  `alerts-page-sk` custom element.

### Key Workflows

**1. Viewing Alerts:**

```
User navigates to the alerts page
  |
  V
alerts-page-sk.connectedCallback()
  |
  +----------------------+
  |                      |
  V                      V
fetch('/_/initpage/')  fetch('/_/alert/list/false')  // Fetch paramset and initial alert list
  |                      |
  V                      V
Update `paramset`      Update `alerts` array
  |                      |
  +----------------------+
  |
  V
_render() // Lit renders the table with alerts
```

**2. Creating a New Alert:**

```
User clicks "New" button (if isEditor === true)
  |
  V
alerts-page-sk.add()
  |
  V
fetch('/_/alert/new') // Get a template for a new alert
  |
  V
Update `cfg` with the new alert template (owner set to current user)
  |
  V
dialog.showModal() // Show the alert-config-sk dialog
  |
  V
User fills in alert details in alert-config-sk
  |
  V
User clicks "Accept"
  |
  V
alerts-page-sk.accept()
  |
  V
cfg = alertconfig.config // Get updated config from alert-config-sk
  |
  V
fetch('/_/alert/update', { method: 'POST', body: JSON.stringify(cfg) }) // Send new alert to backend
  |
  V
alerts-page-sk.list() // Refresh the alert list
```

**3. Editing an Existing Alert:**

```
User clicks "Edit" icon next to an alert (if isEditor === true)
  |
  V
alerts-page-sk.edit() with the selected alert's data
  |
  V
Set `origCfg` (deep copy of current `cfg`)
Set `cfg` to the selected alert's data
  |
  V
dialog.showModal() // Show the alert-config-sk dialog pre-filled with alert data
  |
  V
User modifies alert details in alert-config-sk
  |
  V
User clicks "Accept"
  |
  V
alerts-page-sk.accept()
  |
  V
cfg = alertconfig.config // Get updated config
  |
  V
IF JSON.stringify(cfg) !== JSON.stringify(origCfg) THEN
  fetch('/_/alert/update', { method: 'POST', body: JSON.stringify(cfg) }) // Send updated alert
  |
  V
  alerts-page-sk.list() // Refresh list
ENDIF
```

**4. Deleting an Alert:**

```
User clicks "Delete" icon next to an alert (if isEditor === true)
  |
  V
alerts-page-sk.delete() with the selected alert's ID
  |
  V
fetch('/_/alert/delete/{alert_id}', { method: 'POST' }) // Send delete request
  |
  V
alerts-page-sk.list() // Refresh the alert list
```

**5. Toggling "Show Deleted Configs":**

```
User clicks "Show deleted configs" checkbox
  |
  V
alerts-page-sk.showChanged()
  |
  V
Update `showDeleted` property based on checkbox state
  |
  V
alerts-page-sk.list() // Fetches alerts based on the new `showDeleted` state
```

# Module: /modules/algo-select-sk

## Algo Select SK Module

The `algo-select-sk` module provides a custom HTML element that allows users to
select a clustering algorithm. This component is crucial for applications where
different clustering approaches might yield better results depending on the data
or the analytical goal.

### High-Level Overview

The core purpose of this module is to present a user-friendly way to switch
between available clustering algorithms, specifically "k-means" and "stepfit".
It encapsulates the selection logic and emits an event when the chosen algorithm
changes, allowing other parts of the application to react accordingly.

### Design and Implementation

The "why" behind this module is the need for a standardized and reusable UI
component for algorithm selection. Instead of each part of an application
implementing its own dropdown or radio buttons for algorithm choice,
`algo-select-sk` provides a consistent look and feel.

The "how" involves leveraging the `select-sk` custom element from the
`elements-sk` library to provide the actual dropdown functionality.
`algo-select-sk` builds upon this by:

1.  **Defining specific algorithm options:** It hardcodes "k-means" and
    "stepfit" as the available choices, along with descriptive tooltips.
2.  **Managing state:** It uses an `algo` attribute (and corresponding property)
    to store and reflect the currently selected algorithm.
3.  **Emitting a custom event:** When the selection changes, it dispatches an
    `algo-change` event with the new algorithm in the `detail` object. This
    decoupling allows other components to listen for changes without direct
    dependencies on `algo-select-sk`.

The choice to use `select-sk` as a base provides a consistent styling and
behavior aligned with other elements in the Skia infrastructure.

### Responsibilities and Key Components

- **`algo-select-sk.ts`**: This is the heart of the module.
  - **`AlgoSelectSk` class**: This `ElementSk` subclass defines the custom
    element's behavior.
  - **`template`**: Uses `lit-html` to render the underlying `select-sk`
    element with predefined `div` elements representing the algorithm
    options ("K-Means" and "Individual" which maps to "stepfit"). The
    `selected` attribute on these divs is dynamically updated based on the
    current `algo` property.
  - **`connectedCallback` and `attributeChangedCallback`**: Ensure the
    element renders correctly when added to the DOM or when its `algo`
    attribute is changed programmatically.
  - **`_selectionChanged` method**: This is the event handler for the
    `selection-changed` event from the inner `select-sk` element. When
    triggered, it updates the `algo` property of `algo-select-sk` and then
    dispatches the `algo-change` custom event. This is the primary mechanism
    for communicating the selected algorithm to the outside world. `User
interacts with <select-sk> | V <select-sk> emits 'selection-changed'
event | V AlgoSelectSk._selectionChanged() is called | V Updates
internal 'algo' property | V Dispatches 'algo-change' event with { algo:
"new_value" }`
  - **`algo` getter/setter**: Provides a programmatic way to get and set the
    selected algorithm. The setter ensures that only valid algorithm values
    ('kmeans' or 'stepfit') are set, defaulting to 'kmeans' for invalid
    inputs. This adds a layer of robustness.
  - **`toClusterAlgo` function**: A utility function to validate and
    normalize the input string to one of the allowed `ClusterAlgo` types.
    This prevents invalid algorithm names from being propagated.
  - **`AlgoSelectAlgoChangeEventDetail` interface**: Defines the structure
    of the `detail` object for the `algo-change` event, ensuring type safety
    for event consumers.
- **`algo-select-sk.scss`**: Provides minimal styling, primarily ensuring that
  the cursor is a pointer when hovering over the element, indicating
  interactivity. It imports shared color and theme styles.
- **`index.ts`**: A simple entry point that imports `algo-select-sk.ts`,
  ensuring the custom element is defined and available for use when the module
  is imported.
- **`algo-select-sk-demo.html` and `algo-select-sk-demo.ts`**: These files
  provide a demonstration page for the `algo-select-sk` element.
  - The HTML sets up a few instances of `algo-select-sk`, including one with
    a pre-selected algorithm and one in dark mode, to showcase its
    appearance.
  - The TypeScript for the demo listens to the `algo-change` event from one
    of the instances and displays the event detail in a `<pre>` tag. This
    serves as a live example of how to consume the event.
- **`algo-select-sk_puppeteer_test.ts`**: Contains Puppeteer tests to verify
  the component renders correctly and basic functionality. It checks for the
  presence of the elements on the demo page and takes a screenshot for visual
  regression testing.

The component is designed to be self-contained and easy to integrate. By simply
including the element in HTML and listening for the `algo-change` event,
developers can incorporate algorithm selection functionality into their
applications.

# Module: /modules/anomalies-table-sk

## Anomalies Table (`anomalies-table-sk`)

The `anomalies-table-sk` module provides a custom HTML element for displaying a
sortable and interactive table of performance anomalies. Its primary purpose is
to present anomaly data in a clear, actionable format, allowing users to quickly
identify, group, triage, and investigate performance regressions or
improvements.

### Key Responsibilities:

- **Displaying Anomalies:** Renders a list of `Anomaly` objects in a tabular
  format. Each row represents an anomaly and displays key information such as
  bug ID, revision range, test path, and metrics like delta percentage and
  absolute delta.
- **Grouping Anomalies:** Automatically groups anomalies that share
  overlapping revision ranges. This helps users identify related issues or
  multiple manifestations of the same underlying problem. Groups can be
  expanded or collapsed for better readability.
- **User Interaction:**
  - **Sorting:** Allows users to sort the table by various columns (e.g.,
    Bug ID, Revisions, Test, Delta %).
  - **Selection:** Users can select individual anomalies or entire groups of
    anomalies using checkboxes.
  - **Bulk Actions:** Provides "Triage" and "Graph" buttons that operate on
    the currently selected anomalies.
- **Triage Integration:** Integrates with `triage-menu-sk` to allow users to
  assign bug IDs, mark anomalies as invalid or ignored, or reset their triage
  state.
- **Navigation and Investigation:**
  - Provides links to individual anomaly reports (e.g.,
    `/u/?anomalyIDs=...`).
  - Generates links to view graphs of selected anomalies in the multi-graph
    explorer (`/m/...`).
  - Links bug IDs to the configured bug tracking system (e.g.,
    `/u/?bugID=...`).
  - Allows unassociating a bug ID from an anomaly.

### Design and Implementation Choices:

- **LitElement for Web Component:** The component is built using LitElement, a
  lightweight library for creating Web Components. This promotes
  encapsulation, reusability, and interoperability with other web
  technologies.
- **Client-Side Grouping:** Anomaly grouping based on revision range
  intersection is performed client-side. This simplifies the backend and
  provides immediate feedback to the user as they interact with the table. The
  `groupAnomalies` method iterates through the anomaly list, merging anomalies
  into existing groups if their revision ranges intersect, or creating new
  groups otherwise.
- **Client-Side Sorting:** Sorting is handled by the `sort-sk` element, which
  observes changes to data attributes on the table rows. This avoids server
  roundtrips for simple sorting operations.
- **Selective Rendering:** The table is re-rendered (using `this._render()`)
  only when necessary, such as when data changes, groups are
  expanded/collapsed, or selections are updated. This improves performance.
- **`AnomalyGroup` Class:** A simple `AnomalyGroup` class is used to manage
  collections of related anomalies and their expanded state. This provides a
  clear structure for handling grouped data.
- **Popup for Triage:** The triage menu is presented in a popup to save screen
  real estate and provide a focused interface for triage actions. The popup's
  visibility is controlled by the `showPopup` boolean property.
- **Event-Driven Communication:** The component emits a custom event
  `anomalies_checked` when the selection state of an anomaly changes. This
  allows parent components or other parts of the application to react to user
  selections.
- **API Integration for Graphing and Reporting:**
  - When graphing multiple anomalies, it first calls the
    `/_anomalies/group_report` backend API. This API is designed to provide
    a consolidated view or a shared identifier (`sid`) for a group of
    anomalies, which is then used to construct the graph URL. This is
    preferred over constructing potentially very long URLs with many
    individual anomaly IDs.
  - For single anomaly graphing, it fetches additional time range
    information via the same `group_report` API to provide context (one week
    before and after the anomaly) in the graph.
- **Trace Formatting:** Uses `ChromeTraceFormatter` to correctly format trace
  queries for linking to the graph explorer.
- **Styling:** SCSS is used for styling, importing shared styles from
  `themes_sass_lib`, `buttons_sass_lib`, and `select_sass_lib` for a
  consistent look and feel. Specific styles handle the appearance of
  regression vs. improvement, expanded rows, and the triage popup.

### Key Files:

- **`anomalies-table-sk.ts`:** This is the core file containing the LitElement
  class definition for `AnomaliesTableSk`. It implements all the logic for
  rendering the table, handling user interactions, grouping anomalies, and
  interacting with backend services for triage and graphing.
  - `populateTable(anomalyList: Anomaly[])`: The primary method to load data
    into the table. It triggers grouping and rendering.
  - `generateTable()`, `generateGroups()`, `generateRows()`: Template
    methods responsible for constructing the HTML structure of the table
    using `lit-html`.
  - `groupAnomalies()`: Implements the logic for grouping anomalies based on
    overlapping revision ranges.
  - `openReport()`: Handles the logic for generating a URL to graph the
    selected anomalies, potentially calling the `/_anomalies/group_report`
    API.
  - `togglePopup()`: Manages the visibility of the triage menu popup.
  - `anomalyChecked()`: Handles checkbox state changes and updates the
    `checkedAnomaliesSet`.
  - `openMultiGraphUrl()`: Constructs the URL for viewing an anomaly's trend
    in the multi-graph explorer, fetching time range context via an API
    call.
- **`anomalies-table-sk.scss`:** Contains the SCSS styles specific to the
  anomalies table, defining its layout, appearance, and the styling for
  different states (e.g., improvement, regression, expanded rows).
- **`index.ts`:** A simple entry point that imports and registers the
  `anomalies-table-sk` custom element.
- **`anomalies-table-sk-demo.ts` and `anomalies-table-sk-demo.html`:** Provide
  a demonstration page for the component, showcasing its usage with sample
  data and interactive buttons to populate the table and retrieve checked
  anomalies. The demo also sets up a global `window.perf` object with
  configuration typically provided by the Perf application environment.

### Workflows:

**1. Displaying and Grouping Anomalies:**

```
[User Action: Page Load with Anomaly Data]
  |
  v
AnomaliesTableSk.populateTable(anomalyList)
  |
  v
AnomaliesTableSk.groupAnomalies()
  |-> For each Anomaly in anomalyList:
  |     |-> Try to merge with existing AnomalyGroup (if revision ranges intersect)
  |     |-> Else, create new AnomalyGroup
  |
  v
AnomaliesTableSk._render()
  |
  v
[DOM Update: Table is rendered with grouped anomalies, groups initially collapsed]
```

**2. Selecting and Triaging Anomalies:**

```
[User Action: Clicks checkbox for an anomaly or group]
  |
  v
AnomaliesTableSk.anomalyChecked() or AnomalySk.toggleChildrenCheckboxes()
  |-> Updates `checkedAnomaliesSet`
  |-> Updates header checkbox state if needed
  |-> Emits 'anomalies_checked' event
  |-> Enables/Disables "Triage" and "Graph" buttons based on selection
  |
  v
[User Action: Clicks "Triage" button (if enabled)]
  |
  v
AnomaliesTableSk.togglePopup()
  |-> Shows TriageMenuSk popup
  |-> TriageMenuSk.setAnomalies(checkedAnomalies)
  |
  v
[User interacts with TriageMenuSk (e.g., assigns bug, marks invalid)]
  |
  v
TriageMenuSk makes API request (e.g., to /_/triage)
  |
  v
[Application reloads data or updates table based on triage result]
```

**3. Graphing Selected Anomalies:**

```
[User Action: Selects one or more anomalies]
  |
  v
[User Action: Clicks "Graph" button (if enabled)]
  |
  v
AnomaliesTableSk.openReport()
  |
  |--> If single anomaly selected:
  |     |-> window.open(`/u/?anomalyIDs={id}`, '_blank')
  |
  |--> If multiple anomalies selected:
        |-> Call fetchGroupReportApi(idString)
        |    |-> POST to /_/anomalies/group_report with anomaly IDs
        |    |-> Receives response with `sid` (shared ID)
        |
        |-> window.open(`/u/?sid={sid}`, '_blank')
```

**4. Expanding/Collapsing an Anomaly Group:**

```
[User Action: Clicks expand/collapse button on a group row]
  |
  v
AnomaliesTableSk.expandGroup(anomalyGroup)
  |-> Toggles `anomalyGroup.expanded` boolean
  |
  v
AnomaliesTableSk._render()
  |
  v
[DOM Update: Rows within the group are shown or hidden]
```

# Module: /modules/anomaly-sk

The `anomaly-sk` module provides a custom HTML element `<anomaly-sk>` and
related functionalities for displaying details about performance anomalies. It's
designed to present information about a specific anomaly, including its
severity, the affected revision range, and a link to the associated bug report.
A key utility function, `getAnomalyDataMap`, is also provided to process raw
anomaly data into a format suitable for plotting.

**Key Responsibilities and Components:**

- **`anomaly-sk.ts`**: This is the core file defining the `<anomaly-sk>`
  custom element.

  - **Why**: To encapsulate the logic and presentation of individual anomaly
    details in a reusable web component. This promotes modularity and makes
    it easy to integrate anomaly information into various parts of the Perf
    application.
  - **How**: It extends `ElementSk` and uses the `lit-html` library for
    templating. It accepts an `Anomaly` object as a property and dynamically
    renders a table displaying information like the score before and after
    the anomaly, percentage change, revision range, improvement status, and
    bug ID.
  - It fetches commit details (hashes) using the `lookupCids` function from
    the `cid` module to construct a clickable link to the commit range.
  - It formats numbers and percentages for better readability.
  - It handles different bug ID states (e.g., 0 for no bug, -1 for invalid
    alert, -2 for ignored alert) by displaying appropriate text or a link to
    the bug tracking system. The `bug_host_url` property allows
    customization of the bug tracker URL.
  - The `formatRevisionRange` method asynchronously fetches commit hashes
    for the start and end revisions of the anomaly to create a link to the
    commit range view. If `window.perf.commit_range_url` is not defined, it
    simply displays the revision numbers.

- **`getAnomalyDataMap` (function in `anomaly-sk.ts`)**:

  - **Why**: To transform raw trace data and anomaly information into a
    structured format that can be easily consumed by plotting components
    like `plot-simple-sk`. This function bridges the gap between the raw
    data representation and the visual representation of anomalies on a
    graph.
  - **How**: It takes a `TraceSet` (a collection of traces),
    `ColumnHeader[]` (representing commit points on the x-axis), an
    `AnomalyMap` (mapping trace IDs and commit IDs to `Anomaly` objects),
    and a list of `highlight_anomalies` IDs.
  - It iterates through each trace in the `TraceSet`. If a trace has
    anomalies listed in the `AnomalyMap`, it then iterates through those
    anomalies.
  - For each anomaly, it finds the corresponding x-coordinate by matching
    the anomaly's commit ID (`cid`) with the `offset` in the `ColumnHeader`.
    A crucial detail is that if an exact commit ID match isn't found in the
    header (e.g., due to a data upload failure for that specific commit), it
    will associate the anomaly with the _next available_ commit point. This
    ensures that anomalies are still visualized even if their precise commit
    data point is missing, rather than being omitted entirely.
  - The y-coordinate is taken directly from the trace data at that
    x-coordinate.
  - It determines if an anomaly should be highlighted based on the
    `highlight_anomalies` input.
  - The output is an object where keys are trace IDs and values are arrays
    of `AnomalyData` objects, each containing the `x`, `y` coordinates, the
    `Anomaly` object itself, and a `highlight` flag.

  ```
  Input:
    TraceSet: { "traceA": [10, 12, 15*], ... } (*value at commit 101)
    Header: [ {offset: 99}, {offset: 100}, {offset: 101} ]
    AnomalyMap: { "traceA": { "101": AnomalyObjectA } }
    HighlightList: []

  getAnomalyDataMap
        |
        V
  Output:
    {
      "traceA": [
        { x: 2, y: 15, anomaly: AnomalyObjectA, highlight: false }
      ],
      ...
    }
  ```

- **`anomaly-sk.scss`**: This file contains the SCSS styles for the
  `<anomaly-sk>` element.

  - **Why**: To provide a consistent visual appearance for the anomaly
    details table, aligning with the overall theme of the application
    (`themes_sass_lib`).
  - **How**: It defines basic table styling, such as text alignment and
    padding for `th` and `td` elements within the `anomaly-sk` component.

- **`anomaly-sk-demo.html` and `anomaly-sk-demo.ts`**: These files set up a
  demonstration page for the `<anomaly-sk>` element.

  - **Why**: To provide a sandbox environment for developers to see the
    component in action with various inputs and to facilitate isolated
    testing and development.
  - **How**: `anomaly-sk-demo.html` includes instances of `<anomaly-sk>`
    with different IDs. `anomaly-sk-demo.ts` initializes these components
    with sample `Anomaly` data. It also mocks the `/_/cid/` API endpoint
    using `fetch-mock` to simulate responses for commit detail lookups,
    which is crucial for the `formatRevisionRange` functionality to work in
    the demo. Global `window.perf` configurations are also set up, as the
    component relies on them (e.g., `commit_range_url`).

- **Test Files (`anomaly-sk_test.ts`, `anomaly-sk_puppeteer_test.ts`)**:

  - **Why**: To ensure the correctness and reliability of the component's
    logic and rendering.
  - **`anomaly-sk_test.ts`**: Contains unit tests for the
    `getAnomalyDataMap` function (verifying its mapping logic, especially
    the handling of missing commit points) and for static utility methods
    within `AnomalySk` like `formatPercentage` and the asynchronous
    `formatRevisionRange`. It uses `fetch-mock` to control API responses for
    CID lookups.
  - **`anomaly-sk_puppeteer_test.ts`**: Contains browser-based integration
    tests using Puppeteer. It verifies that the demo page renders correctly
    and takes screenshots for visual regression testing.

**Workflow for Displaying an Anomaly:**

1.  An `Anomaly` object is passed to the `anomaly` property of the
    `<anomaly-sk>` element. `<anomaly-sk
.anomaly=${someAnomalyObject}></anomaly-sk>`
2.  The `set anomaly()` setter in `AnomalySk` is triggered.
3.  It calls `this.formatRevisionRange()` to asynchronously prepare the revision
    range display.
    - `formatRevisionRange` extracts `start_revision` and `end_revision`.
    - It calls `lookupCids([start_rev_num, end_rev_num])` which makes a POST
      request to `/_/cid/`.
    - The response provides commit hashes.
    - If `window.perf.commit_range_url` is set, it constructs an `<a>` tag
      with the URL populated with the fetched hashes. Otherwise, it just
      formats the revision numbers as text.
    - The resulting `TemplateResult` is stored in `this._revision`.
4.  `this._render()` is called, which re-renders the component's template.
5.  The template (`AnomalySk.template`) displays the table:
    - Score, Prior Score, Percent Change (calculated using
      `getPercentChange`).
    - Revision Range (using the `this.revision` template generated in step 3).
    - Improvement status.
    - Bug ID (formatted using `AnomalySk.formatBug`, potentially linking to
      `this.bugHostUrl`).

This module effectively isolates the presentation and data transformation logic
related to individual anomalies, making it a maintainable and reusable piece of
the Perf frontend. The handling of potentially missing data points in
`getAnomalyDataMap` shows a robust design choice for dealing with real-world
data imperfections.

# Module: /modules/bisect-dialog-sk

## Bisect Dialog (`bisect-dialog-sk`)

The `bisect-dialog-sk` module provides a user interface element for initiating a
bisection process within the Perf application. This is specifically designed to
help pinpoint the commit that introduced a performance regression or
improvement, primarily for Chrome.

### Core Responsibility

The primary responsibility of this module is to present a dialog to the user,
pre-filled with relevant information extracted from a chart tooltip (e.g., when
a user identifies an anomaly in a performance graph). It allows the user to
confirm or modify these parameters and then submit a request to the backend to
start a bisection task.

### Why a Dedicated Dialog?

Performance analysis often involves identifying the exact change that caused a
shift in metrics. A manual bisection process can be tedious and error-prone.
This dialog streamlines this by:

1.  **Pre-filling Data:** It leverages context from the chart (like the test
    path and commit range) to pre-populate the necessary fields, reducing manual
    data entry and potential mistakes.
2.  **Structured Input:** It provides a clear form for all required parameters
    for a bisection request, ensuring that the backend receives all necessary
    information.
3.  **User Authentication Awareness:** It integrates with the `alogin-sk` module
    to fetch the logged-in user's email, which is a required parameter for the
    bisect request.
4.  **Feedback Mechanism:** It provides visual feedback to the user during the
    submission process (e.g., a spinner) and communicates success or failure via
    toast messages.

### How it Works

1.  **Initialization and Pre-filling:**

    - The dialog is typically instantiated and hidden until needed.
    - When a user triggers a bisection (e.g., from a chart tooltip), the
      `setBisectInputParams` method is called with details like the
      `testPath`, `startCommit`, `endCommit`, `bugId`, `story`, and
      `anomalyId`.
    - These parameters are used to populate the input fields within the
      dialog's form.

2.  **User Interaction and Submission:**

    - The `open()` method displays the modal dialog.
    - The user can review and, if necessary, modify the pre-filled values
      (e.g., adjust the commit range or add a bug ID). They can also provide
      an optional patch to be applied during the bisection.
    - Upon clicking the "Bisect" button, the `postBisect` method is invoked.

3.  **Request Construction and API Call:**

    - `postBisect` gathers the current values from the form fields.
    - It parses the `testPath` to extract components like the `benchmark`,
      `chart`, and `statistic`. The logic for deriving `chart` and `statistic`
      involves checking the last part of the test name against a predefined
      list of `STATISTIC_VALUES` (e.g., "avg", "count").
    - A `CreateBisectRequest` object is constructed with all the necessary
      parameters.
    - A `fetch` call is made to the `/_/bisect/create` endpoint with the JSON
      payload.

4.  **Response Handling:**

    - If the request is successful, a success message is typically displayed
      (often as a toast by the calling context, as this dialog focuses on the
      submission itself), and the dialog closes.
    - If the request fails, an error message is displayed using
      `errorMessage`, and the dialog remains open, allowing the user to
      correct any issues or retry.

**Simplified Bisect Request Workflow:**

```
User Clicks Bisect Trigger (e.g., on chart)
      |
      V
Calling Code prepares `BisectPreloadParams`
      |
      V
`bisect-dialog-sk.setBisectInputParams(params)`
      |
      V
`bisect-dialog-sk.open()`
      |
      V
Dialog is Displayed (pre-filled)
      |
      V
User reviews/modifies data & Clicks "Bisect"
      |
      V
`bisect-dialog-sk.postBisect()`
      |
      V
`testPath` is parsed (extract benchmark, chart, statistic)
      |
      V
`CreateBisectRequest` object is built
      |
      V
`fetch POST /_/bisect/create` with request data
      |
      V
Handle API Response:
  - Success -> Close dialog, Show success notification (external)
  - Error   -> Show error message, Keep dialog open
```

### Key Components and Files

- **`bisect-dialog-sk.ts`**: This is the core TypeScript file defining the
  `BisectDialogSk` custom element.

  - **`BisectDialogSk` class**: Extends `ElementSk` and manages the dialog's
    state, rendering, and interaction logic.
  - `BisectPreloadParams` interface: Defines the structure of the initial
    data passed to the dialog.
  - `template`: A lit-html template defining the dialog's HTML structure,
    including input fields for test path, bug ID, start/end commits, story,
    and an optional patch. It also includes a close icon, a spinner for
    loading states, and submit/close buttons.
  - `connectedCallback()`: Initializes the element, sets up property
    upgrades, queries for DOM elements (dialog, form, spinner, button), and
    attaches an event listener to the form's submit event. It also fetches
    the logged-in user's status.
  - `setBisectInputParams()`: Populates the internal state and input fields
    with data provided externally.
  - `open()`: Shows the modal dialog and ensures the submit button is
    enabled.
  - `closeBisectDialog()`: Closes the dialog.
  - `postBisect()`: This is the heart of the submission logic. It:
    - Activates the spinner and disables the submit button.
    - Parses the `testPath` to extract various components required for the
      bisect request (like `benchmark`, `chart`, `story`, `statistic`).
      The logic for `chart` and `statistic` derivation is particularly
      important here.
    - Constructs the `CreateBisectRequest` payload.
    - Makes a `POST` request to the `/_/bisect/create` endpoint.
    - Handles the response, either closing the dialog on success or
      displaying an error message on failure.
  - `STATISTIC_VALUES`: A constant array used to determine if the last part
    of a test name is a statistic (e.g., `avg`, `min`, `max`).

- **`bisect-dialog-sk.scss`**: Contains the SASS styles for the dialog,
  ensuring it aligns with the application's theme. It styles the dialog
  itself, input fields, and the footer elements.

- **`index.ts`**: A simple entry point that imports and thus registers the
  `bisect-dialog-sk` custom element.

- **`BUILD.bazel`**: Defines the build rules for this module, specifying its
  dependencies (SASS, TypeScript, other SK elements like `alogin-sk`,
  `select-sk`, `spinner-sk`, `close-icon-sk`) and sources. The dependencies
  highlight its reliance on common UI components and infrastructure modules
  for features like login status and error messaging.

### Design Choices

- **Custom Element (`ElementSk`)**: Encapsulating the dialog as a custom
  element promotes reusability and modularity. It can be easily integrated
  into different parts of the Perf application where bisection capabilities
  are needed.
- **`lit-html` for Templating**: Provides an efficient and declarative way to
  define the dialog's HTML structure and update it based on its state.
- **Pre-computation of Request Parameters**: The dialog takes a "test path"
  and derives several other parameters (benchmark, chart, statistic) from it.
  This simplifies the input required from the user or the calling component,
  as they only need to provide the full test identifier.
- **Specific to Chrome**: The comment "The bisect logic is only specific to
  Chrome" indicates that the backend service this dialog interacts with
  (`/_/bisect/create`) is tailored for Chrome's bisection infrastructure. The
  `project: 'chromium'` in the request payload confirms this.
- **Error Handling**: The use of `jsonOrThrow` and `errorMessage` provides a
  standard way to handle API errors and inform the user.
- **Spinner for Feedback**: The `spinner-sk` element gives visual feedback
  during the asynchronous `fetch` operation, improving user experience.

# Module: /modules/calendar-input-sk

## Calendar Input Element (`calendar-input-sk`)

The `calendar-input-sk` module provides a user-friendly way to select dates. It
combines a standard text input field for manual date entry with a button that
reveals a `calendar-sk` element within a dialog for visual date picking. This
approach offers flexibility for users who prefer typing dates directly and those
who prefer a visual calendar interface.

### Responsibilities and Key Components

- **`calendar-input-sk.ts`**: This is the core file defining the
  `CalendarInputSk` custom element.

  - **Why**: It orchestrates the interaction between the text input, the
    calendar button, and the pop-up calendar dialog. The goal is to provide
    a seamless date selection experience.
  - **How**:
  - It uses a standard HTML `<input type="text">` element for direct date
    input. A `pattern` attribute (`[0-9]{4}-[0-9]{1,2}-[0-9]{1,2}`) and a
    `title` are used to guide the user on the expected `YYYY-MM-DD` format.
    An error indicator (`&cross;`) is shown if the input doesn't match the
    pattern.
  - A `<button>` element, styled with a `date-range-icon-sk`, triggers the
    display of the calendar.
  - A standard HTML `<dialog>` element is used to present the `calendar-sk`
    element. This choice simplifies the implementation of modal behavior.
  - The `openHandler` method is responsible for showing the dialog. It uses
    a `Promise` to manage the asynchronous nature of user interaction with
    the dialog (either selecting a date or canceling). This makes the event
    handling logic cleaner and easier to follow.
  - The `inputChangeHandler` is triggered when the user types into the text
    field. It validates the input against the defined pattern. If valid, it
    parses the date string and updates the `displayDate` property.
  - The `calendarChangeHandler` is invoked when a date is selected from the
    `calendar-sk` component within the dialog. It resolves the
    aforementioned `Promise` with the selected date.
  - The `dialogCancelHandler` is called when the dialog is closed without a
    date selection (e.g., by pressing the "Cancel" button or the Escape
    key). It rejects the `Promise`.
  - An `input` custom event (of type `CustomEvent<Date>`) is dispatched
    whenever the selected date changes, whether through the text input or
    the calendar dialog. This allows parent components to react to date
    selections.
  - The `displayDate` property acts as the single source of truth for the
    currently selected date. Setting this property will update both the text
    input and the date displayed in the `calendar-sk` when it's opened.
  - It leverages the `lit-html` library for templating, providing a
    declarative way to define the element's structure and efficiently update
    the DOM.
  - The element extends `ElementSk`, inheriting common functionalities for
    Skia custom elements.

- **`calendar-input-sk.scss`**: This file contains the styling for the
  `calendar-input-sk` element.

  - **Why**: To provide a consistent visual appearance that integrates well
    with the Skia design system (themes).
  - **How**: It uses SASS to define styles for the input field, the calendar
    button, the error indicator, and the dialog. It leverages CSS variables
    (e.g., `--error`, `--on-surface`, `--surface-1dp`) for theming, allowing
    the component's appearance to adapt to different contexts (like dark
    mode). The `.invalid` class is conditionally displayed based on the
    input field's validity state using the `:invalid` pseudo-class.

- **`index.ts`**: This file simply imports and thereby registers the
  `calendar-input-sk` custom element.

  - **Why**: This is a common pattern for making custom elements available
    for use in an application. It acts as the entry point for the component.

- **`calendar-input-sk-demo.html` / `calendar-input-sk-demo.ts`**: These files
  constitute a demonstration page for the `calendar-input-sk` element.

  - **Why**: To showcase the element's functionality, different states
    (including invalid input and dark mode), and provide a simple way for
    developers to interact with and understand the component. It also serves
    as a testbed during development.
  - **How**: The HTML file includes multiple instances of
    `<calendar-input-sk>` in various configurations. The TypeScript file
    initializes these instances, sets initial `displayDate` values, and
    demonstrates how to listen for the `input` event. It also shows an
    example of programmatically setting an invalid value in one of the input
    fields.

### Key Workflows

**1. Selecting a Date via Text Input:**

```
User types "2023-10-26" into text input
    |
    V
inputChangeHandler in calendar-input-sk.ts
    |
    +-- (Input is valid: matches pattern "YYYY-MM-DD") --> Parse "2023-10-26" into a Date object
    |                                                       |
    |                                                       V
    |                                                   Update _displayDate property
    |                                                       |
    |                                                       V
    |                                                   Render component (updates input field's .value)
    |                                                       |
    |                                                       V
    |                                                   Dispatch "input" CustomEvent<Date>
    |
    +-- (Input is invalid: e.g., "2023-") --> Do nothing (CSS shows error indicator)
```

**2. Selecting a Date via Calendar Dialog:**

```
User clicks calendar button
    |
    V
openHandler in calendar-input-sk.ts
    |
    V
dialog.showModal() is called
    |
    V
<dialog> with <calendar-sk> is displayed
    |
    +-- User selects a date in <calendar-sk> --> <calendar-sk> dispatches "change" event
    |                                                |
    |                                                V
    |                                           calendarChangeHandler in calendar-input-sk.ts
    |                                                |
    |                                                V
    |                                           dialog.close()
    |                                                |
    |                                                V
    |                                           Promise resolves with the selected Date
    |
    +-- User clicks "Cancel" button or presses Esc --> dialog dispatches "cancel" event
                                                         |
                                                         V
                                                    dialogCancelHandler in calendar-input-sk.ts
                                                         |
                                                         V
                                                    dialog.close()
                                                         |
                                                         V
                                                    Promise rejects
```

**If Promise resolves (date selected):**

```
openHandler continues after await
    |
    V
Update _displayDate property with the resolved Date
    |
    V
Render component (updates input field's .value)
    |
    V
Dispatch "input" CustomEvent<Date>
    |
    V
Focus on the text input field
```

The design emphasizes a clear separation of concerns: the `calendar-sk` handles
the visual calendar logic, while `calendar-input-sk` manages the integration of
text input and the dialog presentation. The use of a `Promise` in `openHandler`
simplifies the handling of the asynchronous dialog interaction, leading to more
readable and maintainable code.

# Module: /modules/calendar-sk

The `calendar-sk` module provides a custom HTML element `<calendar-sk>` that
displays an interactive monthly calendar. This element was created to address
limitations with the native HTML `<input type="date">` element, specifically its
lack of Safari support and the inability to style the pop-up calendar.
Furthermore, it aims to be more themeable and accessible than other existing web
component solutions like Elix.

The core philosophy behind `calendar-sk` is to provide a user-friendly,
accessible, and customizable date selection experience. Accessibility is a key
consideration, with design choices informed by WAI-ARIA practices for date
pickers. This includes keyboard navigation and appropriate ARIA attributes.

**Key Responsibilities and Components:**

- **`calendar-sk.ts`**: This is the heart of the module, defining the
  `CalendarSk` custom element which extends `ElementSk`.
  - **Rendering the Calendar:** It uses the `lit-html` library for
    templating, dynamically generating the HTML for the calendar grid. The
    calendar displays one month at a time.
  - The main template (`CalendarSk.template`) constructs the overall table
    structure, including navigation buttons for changing the year and month,
    and headers for the year and month.
  - `CalendarSk.rowTemplate` is responsible for rendering each week (row) of
    the calendar.
  - `CalendarSk.buttonForDateTemplate` creates the individual day buttons.
    It handles logic for disabling buttons for dates outside the current
    month and highlighting the selected date and today's date.
  - **Date Management:**
  - It internally manages a `_displayDate` (a JavaScript `Date` object)
    which represents the currently selected or focused date.
  - The `CalendarDate` class is a helper to simplify comparisons of year,
    month, and date, as JavaScript `Date` objects can be tricky with
    timezones and direct comparisons.
  - Helper functions like `getNumberOfDaysInMonth` and
    `firstDayIndexOfMonth` are used to correctly layout the days within the
    grid.
  - **Navigation:**
  - Provides UI buttons (using `navigate-before-icon-sk` and
    `navigate-next-icon-sk`) for incrementing/decrementing the month and
    year. Methods like `incYear`, `decYear`, `incMonth`, and `decMonth`
    handle the logic for updating `_displayDate` and re-rendering. A crucial
    detail in month/year navigation is handling cases where the current day
    (e.g., 31st) doesn't exist in the target month (e.g., February). In such
    scenarios, the date is adjusted to the last valid day of the target
    month.
  - **Keyboard Navigation:**
  - The `keyboardHandler` method implements navigation using arrow keys
    (day/week changes) and PageUp/PageDown keys (month changes). This
    handler is designed to be attached to a parent element (like a dialog or
    the document) to allow for controlled event handling, especially when
    multiple keyboard-interactive elements are on a page. When a key is
    handled, it prevents further event propagation and focuses the newly
    selected date button.
  - **Internationalization (i18n):**
  - Leverages `Intl.DateTimeFormat` to display month names and weekday
    headers according to the specified `locale` property or the browser's
    default locale. The `buildWeekDayHeader` method dynamically generates
    these headers.
  - **Events:**
  - Dispatches a `change` custom event ( `CustomEvent<Date>`) whenever a new
    date is selected by clicking on a day. The event detail contains the
    selected `Date` object.
  - **Theming:**
  - The component is themeable through CSS custom properties, as defined in
    `calendar-sk.scss`. It imports styles from
    `//perf/modules/themes:themes_sass_lib` and
    `//elements-sk/modules/styles:buttons_sass_lib`.
- **`calendar-sk.scss`**: This file contains the SASS/CSS styles for the
  `<calendar-sk>` element. It defines the visual appearance of the calendar
  grid, buttons, headers, and how selected or "today" dates are highlighted.
  It relies on CSS variables (e.g., `--background`, `--secondary`,
  `--surface-1dp`) for theming, allowing the look and feel to be customized by
  the consuming application.
- **`calendar-sk-demo.html` and `calendar-sk-demo.ts`**: These files set up a
  demonstration page for the `calendar-sk` element.
  - `calendar-sk-demo.html` includes instances of the calendar, some in dark
    mode and one configured for a different locale (`zh-Hans-CN`), to
    showcase its versatility.
  - `calendar-sk-demo.ts` initializes these calendar instances, sets their
    initial `displayDate` and `locale`, and attaches event listeners to log
    the `change` event. It also demonstrates how to hook up the
    `keyboardHandler`.
- **`index.ts`**: A simple entry point that imports and thus registers the
  `calendar-sk` custom element, making it available for use in HTML.

**Key Workflows:**

1.  **Initialization and Rendering:** `ElementSk constructor` ->
    `connectedCallback` -> `buildWeekDayHeader` -> `_render` (calls
    `CalendarSk.template`)

    - When the `<calendar-sk>` element is added to the DOM, its
      `connectedCallback` is invoked.
    - This triggers the initial rendering process, including building the
      weekday headers based on the current locale.
    - The main template then renders the calendar grid for the month of the
      initial `_displayDate`.

2.  **Date Selection (Click):** User clicks on a date button -> `dateClick`
    method -> Updates `_displayDate` -> Dispatches `change` event with the new
    `Date` -> `_render` (to update UI, e.g., highlight new selection)

    User clicks a date button. `[date button]` --click--> `dateClick(event)` |
    +--> `new Date(this._displayDate)` (create copy) | +-->
    `d.setDate(event.target.dataset.date)` (update day) | +-->
    `dispatchEvent(new CustomEvent<Date>('change', { detail: d }))` | +-->
    `this._displayDate = d` | +--> `this._render()`

3.  **Month/Year Navigation (Click):** User clicks "Previous Month" button ->
    `decMonth` method -> Calculates new year, monthIndex, and date (adjusting
    for days in month) -> Updates `_displayDate` with the new `Date` ->
    `_render` (to display the new month/year)

        User clicks "Previous Month" button. `[Previous Month button]` --click-->
        `decMonth()` | +--> Calculate new year, month, date (adjusting for month
        boundaries and days in month) | +--> `this._displayDate = new Date(newYear,

    newMonthIndex, newDate)`| +-->`this.\_render()`

4.  **Keyboard Navigation:** User presses "ArrowRight" while calendar (or its
    container) has focus -> `keyboardHandler(event)` -> `case 'ArrowRight':
this.incDay();` -> `incDay` method updates `_displayDate` (e.g., from May 21
    to May 22) -> `this._render()` -> `e.stopPropagation(); e.preventDefault();`
    ->
    `this.querySelector<HTMLButtonElement>('button[aria-selected="true"]')!.focus();`

User presses ArrowRight key. `keydown event (ArrowRight)` --->
`keyboardHandler(event)` | + (matches `case 'ArrowRight'`) | +-->
`this.incDay()` | | | +--> `this.\_displayDate = new Date(year, monthIndex,

date + 1)`| | | +-->`this.\_render()`| +-->`event.stopPropagation()`|
+-->`event.preventDefault()` | +--> Focus the newly selected day button.

The use of zero-indexed months (`monthIndex`) internally, as is common with the
JavaScript `Date` object, is a deliberate choice for consistency with the
underlying API, though it requires careful handling to avoid off-by-one errors,
especially when calculating things like the number of days in a month.

# Module: /modules/chart-tooltip-sk

## chart-tooltip-sk Module Documentation

### Overview

The `chart-tooltip-sk` module provides a custom HTML element,
`<chart-tooltip-sk>`, designed to display detailed information about a specific
data point on a chart. This tooltip is intended to be interactive, offering
context-sensitive actions and information relevant to performance monitoring and
analysis. It can be triggered by hovering over or clicking on a chart point.

The design philosophy behind this module is to centralize the presentation of
complex data point information and related actions. Instead of scattering this
logic across various chart implementations, `chart-tooltip-sk` encapsulates it,
promoting reusability and maintainability. It aims to provide a rich user
experience by surfacing relevant details like commit information, anomaly
status, bug tracking, and actions like bisection or requesting further traces.

### Key Responsibilities and Components

The primary responsibility of `chart-tooltip-sk` is to render a tooltip with
relevant information and interactive elements based on the data point it's
associated with.

**Core Functionality & Design Choices:**

- **Data Loading and Display:**
  - The `load()` method is the main entry point for populating the tooltip
    with data. It accepts various parameters like the trace index, test
    name, y-value, date, commit position, anomaly details, and bug
    information. This comprehensive loading mechanism allows the parent
    charting component (e.g., `explore-simple-sk`) to provide all necessary
    context.
  - It displays fundamental information such as the test name, data value,
    units, and the date of the data point.
  - **Why:** Consolidating data loading into a single method simplifies the
    interface for parent components.
- **Commit Information:**
  - The tooltip can display details about the commit associated with the
    data point, including the author, message, and a link to the commit in
    the version control system.
  - The `fetch_details()` method is responsible for asynchronously
    retrieving commit details using the `/_/cid/` endpoint. This is done to
    avoid loading all commit details upfront for every point on a chart,
    which could be performance-intensive.
  - The `_always_show_commit_info` and `_skip_commit_detail_display` flags
    (sourced from `window.perf`) allow for configurable display of commit
    details, catering to different instance needs.
  - **Why:** On-demand fetching of commit details optimizes initial load
    times. Configuration flags provide flexibility for different deployment
    scenarios.
- **Anomaly Detection and Triage:**
  - If a data point is identified as an anomaly, the tooltip will highlight
    this and display relevant anomaly metrics (e.g., median before/after,
    percentage change).
  - It integrates with `anomaly-sk` for consistent formatting of anomaly
    data.
  - It incorporates `triage-menu-sk` to allow users to triage new anomalies
    (e.g., create bugs, mark as not a bug).
  - If a bug is already associated with an anomaly, it displays the bug ID
    and provides an option to unassociate it.
  - **Why:** Centralizing anomaly display and triage actions within the
    tooltip provides a focused user workflow.
- **Bug Association:**
  - Integrates with `user-issue-sk` to display and manage Buganizer issues
    linked to a data point (even if it's not a formal anomaly). Users can
    associate existing bugs or create new ones.
  - The `bug_host_url` (from `window.perf`) is used to construct links to
    the bug tracking system.
  - **Why:** Direct integration with bug tracking streamlines the process of
    linking performance data to actionable issues.
- **Interactive Actions:**
  - **Bisect:** Provides a "Bisect" button (if `_show_pinpoint_buttons` is
    true, typically for Chromium instances) that opens `bisect-dialog-sk`.
    This allows users to initiate a bisection to find the exact commit that
    caused a regression.
  - **Request Trace:** Offers a "Request Trace" button (also gated by
    `_show_pinpoint_buttons`) that opens `pinpoint-try-job-dialog-sk`. This
    is used to request more detailed trace data for a specific commit.
  - **Point Links:** Integrates `point-links-sk` to show relevant links for
    a data point based on instance configuration (e.g., links to V8 or
    WebRTC specific commit ranges). This is configured via
    `keys_for_commit_range` and `keys_for_useful_links` in `window.perf`.
  - **JSON Source:** If enabled (`show_json_file_display` in `window.perf`),
    it provides a way to view the raw JSON data for the point via
    `json-source-sk`.
  - **Why:** Placing these actions directly in the tooltip makes them easily
    discoverable and accessible in the context of the selected data point.
- **Positioning and Visibility:**
  - The `moveTo()` method handles the dynamic positioning of the tooltip
    relative to the mouse cursor or the selected chart point. It
    intelligently adjusts its position to stay within the viewport and avoid
    overlapping critical chart elements.
  - The tooltip can be "fixed" (typically on click) or transient (on hover).
    A fixed tooltip remains visible and offers more interactive elements.
  - **Why:** Smart positioning ensures the tooltip is always usable and
    doesn't obstruct the underlying chart. The fixed/transient behavior
    balances information density with unobtrusiveness.
- **Styling:**
  - Uses SCSS for styling (`chart-tooltip-sk.scss`), including themes
    imported from `//perf/modules/themes:themes_sass_lib`.
  - Employs `md-elevation` for a Material Design-inspired shadow effect.
  - **Why:** SCSS allows for organized and maintainable styles. Material
    Design elements provide a consistent look and feel.

**Key Files:**

- **`chart-tooltip-sk.ts`:** The core TypeScript file defining the
  `ChartTooltipSk` class, its properties, methods, and HTML template (using
  `lit-html`). This is where the primary logic for data display, interaction
  handling, and integration with sub-components resides.
- **`chart-tooltip-sk.scss`:** The SASS file containing the styles for the
  tooltip element.
- **`index.ts`:** A simple entry point that imports and registers the
  `chart-tooltip-sk` custom element.
- **`chart-tooltip-sk-demo.html` & `chart-tooltip-sk-demo.ts`:** Files for
  demonstrating the tooltip's functionality. The demo sets up mock data and
  `fetchMock` to simulate API responses, allowing isolated testing and
  visualization of the component.
- **`BUILD.bazel`:** Defines how the element and its demo page are built,
  including dependencies on other Skia Elements and Perf modules like
  `anomaly-sk`, `commit-range-sk`, `triage-menu-sk`, etc.

**Workflow Example: Displaying Tooltip on Chart Point Click (Fixed Tooltip)**

```
User clicks a point on a chart
        |
        V
Parent Chart Component (e.g., explore-simple-sk)
    1. Determines data for the clicked point (coordinates, commit, trace info).
    2. Optionally fetches commit details if not already available.
    3. Optionally checks its anomaly map for anomaly data.
    4. Calls `chartTooltipSk.load(...)` with all relevant data,
       setting `tooltipFixed = true` and providing a close button action.
    5. Calls `chartTooltipSk.moveTo({x, y})` to position the tooltip.
        |
        V
chart-tooltip-sk
    1. `load()` method populates internal properties (_test_name, _y_value, _commit_info, _anomaly, etc.).
    2. `_render()` is triggered (implicitly or explicitly).
    3. The lit-html template in `static template` is evaluated:
        - Basic info (test name, value, date) is displayed.
        - If `commit_info` is present, commit details (author, message, hash) are shown.
        - If `_anomaly` is present:
            - Anomaly metrics are displayed.
            - If `anomaly.bug_id === 0`, `triage-menu-sk` is shown.
            - If `anomaly.bug_id > 0`, bug ID is shown with an unassociate button.
            - Pinpoint job links are shown if available.
        - If `tooltip_fixed` is true:
            - "Bisect" and "Request Trace" buttons are shown (if configured).
            - `user-issue-sk` is shown (if not an anomaly).
            - `json-source-sk` button/link is shown (if configured).
            - The close icon is visible.
    4. Child components like `commit-range-sk`, `point-links-sk`, `user-issue-sk`, `triage-menu-sk`
       are updated with their respective data.
    5. `moveTo()` positions the rendered `div.container` on the screen.
        |
        V
User interacts with buttons (e.g., "Bisect", "Triage", "Close")
        |
        V
chart-tooltip-sk or its child components handle the interaction
    - e.g., clicking "Bisect" calls `openBisectDialog()`, which shows `bisect-dialog-sk`.
    - e.g., clicking "Close" executes the `_close_button_action` passed during `load()`.
```

This modular approach ensures that `chart-tooltip-sk` is a self-contained,
feature-rich component for displaying detailed contextual information and
actions related to data points in performance charts.

# Module: /modules/cid

## CID Module Documentation

This module, `/modules/cid`, provides functionality for interacting with Commit
IDs (CIDs), which are also referred to as CommitNumbers. The primary purpose of
this module is to facilitate the retrieval of detailed commit information based
on a set of commit numbers and their corresponding sources.

### Design and Implementation

The core functionality revolves around the `lookupCids` function. This function
is designed to be a simple and efficient way to fetch commit details from a
backend endpoint.

**Why Asynchronous Operations?**

The lookup of commit information involves a network request to a backend service
(`/_/cid/`). Network requests are inherently asynchronous. Therefore,
`lookupCids` returns a `Promise`. This allows the calling code to continue
execution while the commit information is being fetched and to handle the
response (or any potential errors) when it becomes available. This non-blocking
approach is crucial for maintaining a responsive user interface or efficient
server-side processing.

**Why JSON for Data Exchange?**

JSON (JavaScript Object Notation) is used as the data format for both the
request and the response.

- **Request:** The input `cids` (an array of `CommitNumber` objects) is
  serialized into a JSON string and sent in the body of the HTTP POST request.
  JSON is a lightweight and widely supported format, making it ideal for
  client-server communication.
- **Response:** The backend endpoint is expected to return a JSON response
  conforming to the `CIDHandlerResponse` type. The `jsonOrThrow` utility
  (imported from `../../../infra-sk/modules/jsonOrThrow`) is used to parse
  this JSON response. This utility simplifies error handling by automatically
  throwing an error if the response is not valid JSON or if the HTTP request
  itself fails.

**Why POST Request?**

A POST request is used instead of a GET request for sending the `cids`. While
GET requests are often used for retrieving data, they are typically limited in
the amount of data that can be sent in the URL (e.g., through query parameters).
Since the number of `cids` to look up could be large, sending them in the
request body via a POST request is a more robust and scalable approach. The
`Content-Type: application/json` header informs the server that the request body
contains JSON data.

### Key Components and Files

- **`cid.ts`**: This is the sole TypeScript file in the module and contains
  the implementation of the `lookupCids` function.
  - **`lookupCids(cids: CommitNumber[]): Promise<CIDHandlerResponse>`**:
  - **Responsibility**: Takes an array of `CommitNumber` objects and
    asynchronously fetches detailed commit information for each from the
    `/_/cid/` backend endpoint.
  - **How it works**:
    1.  It constructs an HTTP POST request to the `/_/cid/` endpoint.
    2.  The `cids` array is converted into a JSON string and included as the
        request body.
    3.  Appropriate headers (`Content-Type: application/json`) are set.
    4.  The `fetch` API is used to make the network request.
    5.  The response from the server is then processed by `jsonOrThrow`. If
        the request is successful and the response is valid JSON, it
        resolves the promise with the parsed `CIDHandlerResponse`.
        Otherwise, it rejects the promise with an error.
  - **Dependencies**:
    - `jsonOrThrow` (from `../../../infra-sk/modules/jsonOrThrow`): For
      robust JSON parsing and error handling.
    - `CommitNumber`, `CIDHandlerResponse` (from `../json`): These are
      type definitions that define the structure of the input commit
      identifiers and the expected response from the backend.

### Workflow: Looking up Commit IDs

The typical workflow for using this module is as follows:

```
Caller                     | /modules/cid/cid.ts (lookupCids) | Backend Server (/_/cid/)
---------------------------|----------------------------------|-------------------------
1. Has array of CommitNumber objects.
                           |                                  |
2. Calls `lookupCids(cids)`|                                  |
   `---------------------->`|                                  |
                           | 3. Serializes `cids` to JSON.    |
                           | 4. Creates POST request with JSON body.
                           |    `--------------------------->`| 5. Receives POST request.
                           |                                  | 6. Processes `cids`.
                           |                                  | 7. Generates `CIDHandlerResponse`.
                           |    `<---------------------------`| 8. Sends JSON response.
                           | 9. Receives response.            |
                           | 10. `jsonOrThrow` parses response.|
                           |    (Throws error on failure)     |
                           |                                  |
11. Receives Promise that  |                                  |
    resolves with          |                                  |
    `CIDHandlerResponse`   |                                  |
    (or rejects with error).
   `<----------------------`|                                  |
```

# Module: /modules/cluster-lastn-page-sk

The `cluster-lastn-page-sk` module provides a user interface for testing and
configuring alert configurations by running them against a recent range of
commits. This allows users to "dry run" an alert to see what regressions it
would detect before saving it to run periodically.

**Core Functionality:**

The primary purpose of this module is to facilitate the iterative process of
defining effective alert configurations. Instead of deploying an alert and
waiting for it to trigger (potentially with undesirable results), users can
simulate its behavior on historical data. This helps in fine-tuning parameters
like the detection algorithm, radius, sparsity, and interestingness threshold.

**Key Components and Files:**

- **`cluster-lastn-page-sk.ts`**: This is the heart of the module, defining
  the `ClusterLastNPageSk` custom element.

  - **State Management**: It manages the current alert configuration
    (`this.state`), the commit range (`this.domain`), and the results of the
    dry run (`this.regressions`). It utilizes `stateReflector` to
    potentially persist and restore parts of this state in the URL, allowing
    users to share specific configurations or test setups.
  - **User Interaction**: It handles user actions such as:
  - Editing the alert configuration via a dialog (`alert-config-dialog`
    which hosts an `alert-config-sk` element).
  - Modifying the commit range using a `domain-picker-sk` element.
  - Initiating the dry run (`run()` method).
  - Saving the configured alert (`writeAlert()` method).
  - Viewing details of detected regressions in a dialog
    (`triage-cluster-dialog` which hosts a `cluster-summary2-sk` element).
  - **API Communication**:
  - Fetches initial data (paramset for alert configuration, default new
    alert template) from `/_/initpage/` and `/_/alert/new` respectively.
  - Sends the alert configuration and commit range to the `/_/dryrun/start`
    endpoint to initiate the clustering and regression detection process. It
    uses the `startRequest` utility from `../progress/progress` to handle
    the asynchronous request and display progress.
  - Sends the finalized alert configuration to `/_/alert/update` to save or
    update it in the backend.
  - **Rendering**: It uses `lit-html` for templating and dynamically renders
    the UI based on the current state, including the controls, the progress
    of a running dry run, and a table of detected regressions. The table
    displays commit details (`commit-detail-sk`) and triage status
    (`triage-status-sk`) for each detected regression.
  - **Error Handling**: It displays error messages if the dry run or alert
    saving fails.

- **`cluster-lastn-page-sk.html` (Demo Page)**: A simple HTML file that
  includes the `cluster-lastn-page-sk` element and an `error-toast-sk` for
  displaying global error messages. This is primarily used for demonstration
  and testing purposes.

- **`cluster-lastn-page-sk-demo.ts`**: Sets up mock HTTP responses using
  `fetch-mock` for the demo page. This allows the `cluster-lastn-page-sk`
  element to function in isolation without needing a live backend. It mocks
  endpoints like `/_/initpage/`, `/_/alert/new`, `/_/count/`, and
  `/_/loginstatus/`.

- **`cluster-lastn-page-sk.scss`**: Provides the styling for the
  `cluster-lastn-page-sk` element and its dialogs, ensuring a consistent look
  and feel with the rest of the Perf application. It uses shared SASS
  libraries for buttons and themes.

**Workflow for Testing an Alert Configuration:**

1.  **Load Page**: User navigates to the page.

    - `cluster-lastn-page-sk` fetches initial paramset and a default new alert
      configuration.

    ```
    User -> cluster-lastn-page-sk
           cluster-lastn-page-sk -> GET /_/initpage/ (fetches paramset)
           cluster-lastn-page-sk -> GET /_/alert/new (fetches default alert)
    ```

2.  **Configure Alert**: User clicks the "Configure Alert" button.

    - A dialog (`alert-config-dialog`) opens, showing `alert-config-sk`.
    - User modifies alert parameters (algorithm, radius, query, etc.).
    - User clicks "Accept".
    - The `state` in `cluster-lastn-page-sk` is updated with the new
      configuration.

    ```
    User --clicks--> "Configure Alert" button
           cluster-lastn-page-sk --shows--> alert-config-dialog
           User --interacts with--> alert-config-sk
           User --clicks--> "Accept"
           alert-config-sk --updates--> cluster-lastn-page-sk.state
    ```

3.  **(Optional) Adjust Commit Range**: User interacts with `domain-picker-sk`
    to define the number of recent commits or a specific date range for the dry
    run.

    - `cluster-lastn-page-sk.domain` is updated.

4.  **Run Dry Run**: User clicks the "Run" button.

    - `cluster-lastn-page-sk` constructs a `RegressionDetectionRequest` using
      the current alert `state` and `domain`.
    - It sends this request to `/_/dryrun/start`.
    - The UI shows a spinner and progress messages.
    - As results (regressions) become available, they are displayed in a
      table.

    ```
    User --clicks--> "Run" button
           cluster-lastn-page-sk --creates--> RegressionDetectionRequest
           cluster-lastn-page-sk --POSTs to--> /_/dryrun/start (with request body)
                                       (progress updates via startRequest callback)
           Backend --processes & clusters-->
           Backend --sends progress/results--> cluster-lastn-page-sk
           cluster-lastn-page-sk --updates--> UI (regressions table, status messages)
    ```

5.  **Review Results**: User examines the table of regressions.

    - Each row shows a commit and the regressions (low/high) found at that
      commit.
    - User can click on a regression to open a `triage-cluster-dialog`
      (showing `cluster-summary2-sk`) for more details.
    - From the summary dialog, user can open related traces in the explorer
      view.

6.  **Iterate or Save**:

        - If results are not satisfactory, user goes back to step 2 to adjust the
          alert configuration and re-runs.
        - If results are satisfactory, user clicks "Create Alert" (or "Update
          Alert" if modifying an existing one).
        - `cluster-lastn-page-sk` sends the current alert `state` to
          `/_/alert/update`. `User --clicks--> "Create Alert" / "Update Alert"

    button cluster-lastn-page-sk --POSTs to--> /\_/alert/update (with alert
    config) Backend --saves/updates alert--> Backend --responds with ID-->
    cluster-lastn-page-sk cluster-lastn-page-sk --updates--> UI (button text
    might change to "Update Alert")`

**Design Decisions:**

- **Client-Side Dry Run Initiation**: The "dry run" is initiated from the
  client, sending the full alert configuration. This allows immediate feedback
  and iteration without needing to first save an incomplete or experimental
  alert to the backend.
- **Component-Based UI**: The UI is built using custom elements (e.g.,
  `alert-config-sk`, `domain-picker-sk`, `cluster-summary2-sk`). This promotes
  modularity, reusability, and separation of concerns.
- **Asynchronous Operations with Progress**: Long-running operations like the
  dry run are handled asynchronously with visual feedback (spinners, status
  messages) provided by the `../progress/progress` utility, enhancing user
  experience.
- **State Reflection**: Using `stateReflector` allows parts of the page's
  state (like the alert configuration) to be encoded in the URL. This is
  useful for sharing specific test scenarios or bookmarking them.
- **Dialogs for Focused Interaction**: Modal dialogs are used for alert
  configuration and viewing regression summaries, preventing users from
  interacting with the main page content while these tasks are in progress,
  thus guiding their focus.
- **Mocking for Demo/Testing**: The demo page
  (`cluster-lastn-page-sk-demo.ts`) heavily relies on `fetch-mock`. This
  enables isolated development and testing of the UI component without a
  backend dependency, which is crucial for frontend unit/integration tests and
  local development.

# Module: /modules/cluster-page-sk

The `cluster-page-sk` module provides the user interface for Perf's trace
clustering functionality. This allows users to identify groups of traces that
exhibit similar behavior, which is crucial for understanding performance
regressions or improvements across different configurations and tests.

**Core Functionality and Design:**

The primary goal of this page is to allow users to define a set of traces and
then apply a clustering algorithm to them. The "why" behind this is to simplify
the analysis of large datasets by grouping related performance changes. Instead
of manually inspecting hundreds or thousands of individual traces, users can
focus on a smaller number of clusters, each representing a distinct performance
pattern.

The "how" involves several key components:

1.  **Defining the Scope of Analysis:**

    - **Commit Selection:** Users first select a central commit around which
      the analysis will be performed. This is handled by
      `commit-detail-picker-sk`. The clustering will typically look at commits
      before and after this selected point. The `state.offset` property stores
      the selected commit's offset.
    - **Query:** Users define the set of traces to consider using a query
      string. This is managed by `query-sk` and `paramset-sk`. The
      `state.query` holds this query. The `query-count-sk` element provides
      feedback on how many traces match the current query.
    - **Time Range/Commit Radius:** Users can specify a "radius" (in terms of
      number of commits) around the selected commit to include in the
      analysis. This is stored in `state.radius`.

2.  **Clustering Algorithm and Parameters:**

    - **Algorithm Selection:** Users can choose the clustering algorithm
      (e.g., k-means). This is facilitated by `algo-select-sk` and stored in
      `state.algo`. The choice of algorithm impacts how clusters are formed
      and what "similarity" means.
    - **Number of Clusters (K):** For algorithms like k-means, the user can
      suggest the number of clusters to find. A value of 0 typically means the
      server will try to determine an optimal K. This is stored in `state.k`.
    - **Interestingness Threshold:** Users can define a threshold for what
      constitutes an "interesting" cluster, often based on the magnitude of
      regression or step size. This is `state.interesting`.
    - **Sparse Data Handling:** An option (`state.sparse`) allows users to
      indicate if the data is sparse, meaning not all traces have data points
      for all commits. This affects how the clustering algorithm processes
      missing data.

3.  **Executing the Clustering and Displaying Results:**

    - **Initiating the Request:** The "Run" button triggers the clustering
      process. The `start()` method constructs a `RegressionDetectionRequest`
      object containing all the user-defined parameters. This request is sent
      to the `/_/cluster/start` endpoint.
    - **Background Processing and Progress:** Clustering can be a long-running
      operation. The module uses the `progress` utility to manage the
      asynchronous request. It displays a spinner (`spinner-sk`) and status
      messages (`ele.status`, `ele.runningStatus`) to keep the user informed.
      The `requestId` property tracks the active request.
    - **Displaying Clusters:** Once the server responds, the
      `RegressionDetectionResponse` contains a list of `FullSummary` objects.
      Each `FullSummary` represents a discovered cluster. These are rendered
      using multiple `cluster-summary2-sk` elements. This component is
      responsible for visualizing the details of each cluster, including its
      member traces and regression information.
    - **Sorting Results:** Users can sort the resulting clusters by various
      metrics (size, regression score, etc.) using `sort-sk`.

**State Management:**

The `cluster-page-sk` component maintains its internal state in a `State`
object. This includes user selections like the query, commit offset, algorithm,
and various parameters. Crucially, this state is reflected in the URL using the
`stateReflector` utility. This design decision ensures that:

- The page is bookmarkable: Users can save and share URLs that directly lead
  to a specific clustering configuration and its results.
- Browser history (back/forward buttons) works as expected.
- The application state is serializable and easily reproducible.

The `stateHasChanged()` method is called whenever a piece of the state is
modified, triggering the `stateReflector` to update the URL and potentially
re-render the component.

**Key Files and Their Roles:**

- **`cluster-page-sk.ts`:** This is the main TypeScript file defining the
  `ClusterPageSk` custom element. It orchestrates all the sub-components,
  manages the application state, handles user interactions (e.g., button
  clicks, input changes), makes API calls for clustering, and renders the
  results. It defines the overall layout and logic of the clustering page.
- **`cluster-page-sk.html` (inferred, as it's a LitElement):** The HTML
  template is defined within `cluster-page-sk.ts` using `lit-html`. This
  template structures the page, embedding various custom elements for commit
  selection, query building, algorithm choice, and result display.
- **`cluster-page-sk.scss`:** Provides the specific styling for the
  `cluster-page-sk` element and its layout, ensuring a consistent look and
  feel.
- **`index.ts`:** A simple entry point that imports and registers the
  `cluster-page-sk` custom element, making it available for use in HTML.
- **`cluster-page-sk-demo.ts` & `cluster-page-sk-demo.html`:** These files set
  up a demonstration page for the `cluster-page-sk` element.
  `cluster-page-sk-demo.ts` uses `fetch-mock` to simulate API responses,
  allowing the component to be developed and tested in isolation without
  needing a live backend. This is crucial for rapid development and ensuring
  the UI behaves correctly under various backend scenarios.
- **`State` class (within `cluster-page-sk.ts`):** Defines the structure of
  the data that is persisted in the URL and drives the component's behavior.
  It encapsulates all user-configurable options for the clustering process.

**Workflow Example: Performing a Cluster Analysis**

```
User Interaction                         | Component/State Change        | Backend Interaction
-----------------------------------------|-------------------------------|---------------------
1. User navigates to the cluster page.   | `ClusterPageSk` initializes.  | Fetches initial paramset (`/_/initpage/`)
                                         | `stateReflector` initializes  |
                                         | from URL or defaults.         |
                                         |                               |
2. User selects a commit.                | `commit-detail-picker-sk`     | (Potentially fetches commit details if not cached)
                                         | emits `commit-selected`.      |
                                         | `state.offset` updates.       |
                                         | `stateHasChanged()` called.   |
                                         |                               |
3. User types a query (e.g., "config=gpu").| `query-sk` emits            | (Potentially `/`_`/count/` to update trace count)
                                         | `query-change`.               |
                                         | `state.query` updates.        |
                                         | `stateHasChanged()` called.   |
                                         |                               |
4. User selects an algorithm (e.g., kmeans).| `algo-select-sk` emits      |
                                         | `algo-change`.                |
                                         | `state.algo` updates.         |
                                         | `stateHasChanged()` called.   |
                                         |                               |
5. User adjusts advanced parameters      | Input elements update         |
   (K, radius, interestingness).         | corresponding `state` props.  |
                                         | `stateHasChanged()` called.   |
                                         |                               |
6. User clicks "Run".                    | `start()` method is called.   | POST to `/_/cluster/start` with `RegressionDetectionRequest`
                                         | `requestId` is set.           | (This is a long-running request)
                                         | Spinner becomes active.       |
                                         |                               |
7. Page periodically updates status.     | `progress` utility polls for  | GET requests to check progress.
                                         | updates.                      |
                                         | `ele.runningStatus` updates.  |
                                         |                               |
8. Clustering completes.                 | `progress` utility resolves.  | Final response from `/_/cluster/start` (or progress endpoint)
                                         | `summaries` array is populated| containing `RegressionDetectionResponse`.
                                         | with cluster data.            |
                                         | `requestId` is cleared.       |
                                         | Spinner stops.                |
                                         |                               |
9. Results are displayed.                | `ClusterPageSk` re-renders,   |
                                         | showing `cluster-summary2-sk` |
                                         | elements for each cluster.    |
```

This workflow highlights how user inputs are translated into state changes,
which then drive API requests and ultimately update the UI to present the
clustering results. The separation of concerns among various sub-components (for
query, commit selection, etc.) makes the main `cluster-page-sk` element more
manageable.

# Module: /modules/cluster-summary2-sk

The `cluster-summary2-sk` module provides a custom HTML element for displaying
detailed information about a cluster of performance test results. This includes
visualizing the trace data, showing regression statistics, and allowing users to
triage the cluster.

**Core Functionality and Design:**

The primary purpose of this element is to present a comprehensive summary of a
performance cluster. It aims to provide all necessary information for a user to
understand the nature of a performance change (regression or improvement) and
take appropriate action (e.g., filing a bug, marking it as expected).

Key design considerations include:

- **Data Visualization:** A `plot-simple-sk` element is used to display the
  centroid trace of the cluster over time. This visual representation helps
  users quickly grasp the trend and identify the point of change. An "x-bar"
  can be displayed on the plot to highlight the specific commit where a step
  change is detected.
- **Statistical Summary:** The element displays key statistics about the
  cluster, such as its size, the regression factor, step size, and least
  squares error. The labels and formatting of these statistics dynamically
  adapt based on the `StepDetection` algorithm used (e.g., 'absolute',
  'percent', 'mannwhitneyu'). This ensures that the presented information is
  relevant and interpretable for the specific detection method.
- **Commit Details:** Integration with `commit-detail-panel-sk` allows users
  to view details of the commit associated with the detected step point or any
  selected point on the trace plot. This is crucial for correlating
  performance changes with specific code modifications.
- **Triaging:** If not disabled via the `notriage` attribute, the element
  includes a `triage2-sk` component. This allows authenticated users with
  "editor" privileges to set the triage status (e.g., "positive", "negative",
  "untriaged") and add a message. This functionality is essential for tracking
  the investigation and resolution of performance issues.
- **Contextual Actions:** Buttons are provided to:
  - "View on dashboard": Opens the current cluster view in a broader
    explorer context, pre-filling relevant parameters like shortcut ID and
    time range.
  - "Word Cloud": Toggles the visibility of a `word-cloud-sk` element, which
    displays a summary of the parameters that make up the traces in the
    cluster. This helps in understanding the common characteristics of the
    affected tests.
  - A permalink is generated to directly link to the triage page for the
    specific step point.
- **Interactive Exploration:** The `commit-range-sk` component allows users to
  define a range around the detected step or a selected commit, facilitating
  further investigation within the Perf application.

**Key Components and Their Roles:**

- **`cluster-summary2-sk.ts`**: This is the main TypeScript file defining the
  `ClusterSummary2Sk` custom element.
  - **`ClusterSummary2Sk` class:** Extends `ElementSk` and manages the
    element's state, rendering, and event handling.
  - **Data Properties (`full_summary`, `triage`, `alert`):** These
    properties receive the core data for the cluster. When `full_summary` is
    set, it triggers the rendering of the plot, statistics, and commit
    details. The `alert` property determines the labels and formatting for
    regression statistics. The `triage` property reflects the current triage
    state.
  - **Template (`template` static method):** Uses `lit-html` to define the
    element's structure, binding data to various sub-components and display
    areas.
  - **Event Handling:**
    - `open-keys`: Fired when the "View on dashboard" button is clicked,
      providing details for opening the explorer.
    - `triaged`: Fired when the triage status is updated, containing the
      new status and the relevant commit information.
    - `trace_selected`: Handles events from `plot-simple-sk` when a point
      on the graph is clicked, triggering a lookup for the corresponding
      commit details.
  - **Helper Methods:**
    - `statusClass()`: Determines the CSS class for the regression display
      based on the severity (e.g., "high", "low").
    - `permaLink()`: Generates a URL to the triage page focused on the
      step point.
    - `lookupCids()` (static): A static method (delegating to
      `../cid/cid.ts`) used to fetch commit details based on commit
      numbers.
  - **`labelsForStepDetection`:** A crucial constant object that maps
    different `StepDetection` algorithm names (e.g., 'percent',
    'mannwhitneyu', 'absolute') to specific labels and number formatting
    functions for the regression statistics. This ensures that the displayed
    information is meaningful and correctly interpreted for the algorithm
    used to detect the cluster.
- **`cluster-summary2-sk.html` (template, rendered by
  `cluster-summary2-sk.ts`):** Defines the visual layout using HTML and
  embedded custom elements. It uses a CSS grid for positioning the main
  sections: regression summary, statistics, plot, triage status, commit
  details, actions, and word cloud.
- **`cluster-summary2-sk.scss`**: Provides the styling for the element. It
  defines how different sections are displayed, including styles for
  regression severity (e.g., red for "high" regressions, green for "low"),
  button appearances, and responsive behavior (hiding the plot on smaller
  screens).
- **`cluster-summary2-sk-demo.html` and `cluster-summary2-sk-demo.ts`**: These
  files set up a demonstration page for the `cluster-summary2-sk` element. The
  `.ts` file provides mock data for `FullSummary`, `Alert`, and `TriageStatus`
  to populate the demo instances of the element. It also demonstrates how to
  listen for the `triaged` and `open-keys` custom events.

**Workflows:**

1.  **Initialization and Data Display:**

    - The host application provides `full_summary` (containing cluster data
      and trace frame), `alert` (details of the alert that triggered this
      cluster), and optionally `triage` (current triage status) properties to
      the `cluster-summary2-sk` element.
    - `set full_summary()`:
      - Updates internal `summary` and `frame` data.
      - Populates dataset attributes for sorting (e.g., `data-clustersize`).
      - Clears and redraws the `plot-simple-sk` with the centroid trace from
        `summary.centroid` and time labels from `frame.dataframe.header`.
      - If a step point is identified and the status is not "Uninteresting",
        an x-bar is placed on the plot at the corresponding commit.
      - `lookupCids` is called to fetch and display details for the commit
        at the step point in `commit-detail-panel-sk`.
    - `set alert()`:
      - Updates the `labels` used for displaying regression statistics based
        on `alert.step` and `labelsForStepDetection`.
    - `set triage()`:
      - Updates the `triageStatus` and re-renders the triage controls.
    - The element renders based on the provided data, displaying statistics,
      plot, commit details, and triage controls.

    ```
    Host Application           cluster-summary2-sk
    ----------------           -------------------
    [Set full_summary data] --> Process data
                                 |
                                 +-> plot-simple-sk (Draws trace)
                                 |
                                 +-> commit-detail-panel-sk (Shows step commit)
                                 |
                                 +-> Display stats (regression, size, etc.)

    [Set alert data] ---------> Update regression labels/formatters

    [Set triage data] --------> Update triage2-sk state
    ```

2.  **User Triage:**

    - User interacts with `triage2-sk` (selects status) and the message input
      field.
    - User clicks the "Update" button.
    - `update()` method is called:
      - An `ClusterSummary2SkTriagedEventDetail` object is created
        containing the `step_point` (as `columnHeader`) and the current
        `triageStatus`.
      - A `triaged` custom event is dispatched with this detail.
    - The host application listens for the `triaged` event to persist the
      triage status.

    ```
    User                  cluster-summary2-sk             Host Application
    ----                  -------------------             ----------------
    Selects status ---->  [triage2-sk updates value]
    Types message  ---->  [Input updates value]
    Clicks "Update" --->  update()
                           |
                           +-> Creates TriagedEventDetail
                           |
                           +-> Dispatches "triaged" event --> Listens and handles event
                                                               (e.g., saves to backend)
    ```

3.  **Viewing on Dashboard:**

    - User clicks the "View on dashboard" button.
    - `openShortcut()` method is called:
      - An `ClusterSummary2SkOpenKeysEventDetail` object is created with the
        `shortcut` ID, `begin` and `end` timestamps from the frame, and the
        `step_point` as `xbar`.
      - An `open-keys` custom event is dispatched.
    - The host application listens for `open-keys` and navigates the user to
      the explorer view with the provided parameters.

    ```
    User                      cluster-summary2-sk             Host Application
    ----                      -------------------             ----------------
    Clicks "View on dash" --> openShortcut()
                               |
                               +-> Creates OpenKeysEventDetail
                               |
                               +-> Dispatches "open-keys" event --> Listens and handles event
                                                                    (e.g., navigates to explorer)
    ```

The `cluster-summary2-sk` element plays a vital role in the Perf frontend by
providing a focused and interactive view for analyzing individual performance
regressions or improvements identified through clustering. Its integration with
plotting, commit details, and triaging makes it a key tool for performance
analysis workflows.

# Module: /modules/commit-detail-panel-sk

## Commit Detail Panel SK

**High-level Overview:**

The `commit-detail-panel-sk` module provides a custom HTML element
`<commit-detail-panel-sk>` designed to display a list of commit details. It
offers functionality to make these commit entries selectable and emits an event
when a commit is selected. This component is primarily used in user interfaces
where users need to browse and interact with a sequence of commits.

**Why and How:**

The core purpose of this module is to present commit information in a structured
and interactive way. Instead of simply displaying raw commit data, it leverages
the `commit-detail-sk` element (an external dependency) to render each commit
with relevant information like author, message, and a link to the commit.

The design decision to make commits selectable (via the `selectable` attribute)
enhances user interaction. When a commit is clicked in "selectable" mode, it
triggers a `commit-selected` custom event. This event carries detailed
information about the selected commit, including its index in the list, a
concise description, and the full commit object. This allows parent components
or applications to react to user selections and perform actions based on the
chosen commit (e.g., loading further details, navigating to a specific state).

The implementation uses Lit library for templating and rendering. The commit
data is provided via the `details` property, which expects an array of `Commit`
objects (defined in `perf/modules/json`). The component dynamically generates
table rows for each commit.

The visual appearance is controlled by `commit-detail-panel-sk.scss`. It defines
styles for the panel, including highlighting the selected row and adjusting
opacity based on the `selectable` state. The styling aims for a clean and
readable presentation of commit information.

A `hide` property is also available to conditionally show or hide the entire
commit list. This is useful for scenarios where the panel's visibility needs to
be controlled dynamically by the parent application.

**Key Components/Files:**

- **`commit-detail-panel-sk.ts`**: This is the heart of the module. It defines
  the `CommitDetailPanelSk` class, which extends `ElementSk`.
  - **Responsibilities**:
  - Manages the list of `Commit` objects (`_details` property).
  - Renders the list of commits as an HTML table using Lit templates
    (`template` and `rows` static methods).
  - Handles user clicks on table rows (`_click` method).
  - When a commit is selected (and the `selectable` attribute is present),
    it dispatches the `commit-selected` custom event with relevant commit
    data.
  - Manages the `selectable`, `selected`, and `hide` attributes and their
    corresponding properties, re-rendering the component when these change.
  - Integrates the `commit-detail-sk` element to display individual commit
    details within each row.
- **`commit-detail-panel-sk.scss`**: This file contains the SASS styles for
  the component.
  - **Responsibilities**:
  - Defines the visual appearance of the commit panel, including link
    colors, table cell padding, and selected row highlighting.
  - Adjusts the opacity and cursor style based on whether the panel is
    `selectable`.
  - Leverages theme variables (e.g., `--primary`, `--surface-1dp`) from
    `//perf/modules/themes:themes_sass_lib` for consistent theming.
- **`commit-detail-panel-sk-demo.ts` and `commit-detail-panel-sk-demo.html`**:
  These files provide a demonstration page for the component.
  - **Responsibilities**:
  - Illustrate how to use the `<commit-detail-panel-sk>` element in an HTML
    page.
  - Show examples of the component in both selectable and non-selectable
    states, and in light/dark themes.
  - Demonstrate how to provide commit data to the `details` property and how
    to listen for the `commit-selected` event.
- **`index.ts`**: A simple entry point that imports and registers the
  `commit-detail-panel-sk` custom element, making it available for use.
- **`BUILD.bazel`**: Defines how the module is built and its dependencies. For
  instance, it declares `commit-detail-sk` as a runtime dependency and Lit as
  a TypeScript dependency.
- **`commit-detail-panel-sk_puppeteer_test.ts`**: Contains Puppeteer tests to
  verify the component's rendering and basic functionality.

**Key Workflows:**

1.  **Initialization and Rendering:**

    ```
    Parent Application --> Sets 'details' property of <commit-detail-panel-sk> with Commit[]
                                      |
                                      V
    commit-detail-panel-sk.ts --> _render() is called
                                      |
                                      V
                          Lit template generates <table>
                                      |
                                      V
                          For each Commit in 'details':
                               Generates <tr> containing <commit-detail-sk .cid=Commit>
    ```

2.  **Commit Selection (when `selectable` is true):** `User --> Clicks on a <tr>
in the <commit-detail-panel-sk> | V commit-detail-panel-sk.ts -->
_click(event) handler is invoked | V Determines the clicked commit's index
and data | V Sets 'selected' attribute/property to the index of the clicked
commit | V Dispatches 'commit-selected' CustomEvent with { selected: index,
description: string, commit: Commit } | V Parent Application --> Listens for
'commit-selected' event and processes the event.detail`

The design favors declarative attribute-based configuration (e.g., `selectable`,
`selected`) and event-driven communication for user interactions, which are
common patterns in web component development.

# Module: /modules/commit-detail-picker-sk

The `commit-detail-picker-sk` module provides a user interface element for
selecting a specific commit from a range of commits. It's designed to be a
reusable component that simplifies the process of commit selection within
applications that need to interact with commit histories.

**Core Functionality and Design:**

The primary purpose of `commit-detail-picker-sk` is to allow users to browse and
select a commit. This is achieved by presenting a button that, when clicked,
opens a dialog.

- **Button as Entry Point:** The button displays a summary of the currently
  selected commit (author and message) or a default message like "Choose a
  commit." This provides immediate context to the user. Clicking this button
  triggers the opening of the selection dialog. `[Button: "Author - Commit
Message"] --- (click) ---> [Dialog Opens]`
- **Dialog for Selection:** The dialog is the main interaction point for
  choosing a commit. It contains:
  - `commit-detail-panel-sk`: This submodule is responsible for displaying
    the list of commits fetched from the backend. Users can click on a
    commit in this panel to select it.
  - **Date Range Selection:** A `day-range-sk` component allows users to
    specify a time window for fetching commits. This is crucial for
    performance and usability, as it prevents loading an overwhelming number
    of commits at once. When the date range changes, the component
    automatically fetches the relevant commits. `[day-range-sk] -- (date
range change) --> [Fetch Commits for New Range] | V
[commit-detail-panel-sk updates]`
  - **Spinner:** A `spinner-sk` element provides visual feedback to the user
    while commits are being fetched, indicating that an operation is in
    progress.
  - **Close Button:** Allows the user to dismiss the dialog without making a
    selection or after a selection is made.

**Data Flow and State Management:**

1.  **Initialization:** When the component is first loaded, it initializes with
    a default date range (typically the last 24 hours). It then fetches the
    commits within this initial range.
2.  **Fetching Commits:** The component makes a POST request to the
    `/_/cidRange/` endpoint. The request body includes the `begin` and `end`
    timestamps of the desired range and optionally the `offset` of a currently
    selected commit (to ensure it's included in the results if it falls outside
    the new range). `User Action (e.g., change date range) | V
[commit-detail-picker-sk] | V (Constructs RangeRequest: {begin, end,
offset}) POST /_/cidRange/ | V (Receives Commit[] array)
[commit-detail-picker-sk] | V (Updates internal 'details' array)
[commit-detail-panel-sk] (Re-renders with new commit list)`
3.  **Commit Selection:** - When a user selects a commit in the `commit-detail-panel-sk`, the panel
    emits a `commit-selected` event. - `commit-detail-picker-sk` listens for this event and updates its
    internal `selected` index. - The dialog is then closed, and the main button's text updates to reflect
    the new selection. - Crucially, `commit-detail-picker-sk` itself emits a `commit-selected`
    event. This allows parent components to react to the user's choice. The
    detail of this event is of type
    `CommitDetailPanelSkCommitSelectedDetails`, containing information about
    the selected commit. `[commit-detail-panel-sk] -- (internal click on a
commit) --> Emits 'commit-selected' (internal) | V
[commit-detail-picker-sk] -- (handles internal event) --> Updates
'selected' index Updates button text Closes dialog Emits
'commit-selected' (external)`
4.  **External Selection (`selection` property):** The component exposes a
    `selection` property (of type `CommitNumber`). If this property is set
    externally, the component will attempt to fetch commits around that
    `CommitNumber` and pre-select it in the panel.

**Key Files and Responsibilities:**

- **`commit-detail-picker-sk.ts`:** This is the core TypeScript file defining
  the `CommitDetailPickerSk` custom element.
  - **Why:** It orchestrates the interactions between the button, dialog,
    `commit-detail-panel-sk`, and `day-range-sk`. It handles fetching commit
    data, managing the selection state, and emitting the final
    `commit-selected` event.
  - **How:** It uses the Lit library for templating and rendering. It
    defines methods for opening/closing the dialog (`open()`, `close()`),
    handling range changes (`rangeChange()`), updating the commit list
    (`updateCommitSelections()`), and processing selections from the panel
    (`panelSelect()`). The `selection` getter/setter allows for programmatic
    control of the selected commit.
- **`commit-detail-picker-sk.scss`:** Contains the SASS/CSS styles for the
  component.
  - **Why:** To provide a consistent visual appearance and layout for the
    button and the dialog, ensuring it integrates well with the overall
    application theme (e.g., light and dark modes via CSS variables like
    `--on-background`, `--background`).
  - **How:** It styles the `dialog` element, the buttons within it, and
    ensures proper display and spacing of child components like
    `day-range-sk`.
- **`commit-detail-picker-sk-demo.html` & `commit-detail-picker-sk-demo.ts`:**
  These files provide a demonstration page for the component.
  - **Why:** To showcase the component's functionality in isolation, making
    it easier to test and understand its usage. The demo also includes
    examples for light and dark themes.
  - **How:** The HTML sets up basic page structure and placeholders for the
    component. The TypeScript file initializes instances of
    `commit-detail-picker-sk`, mocks the backend API call (`/_/cidRange/`)
    using `fetch-mock` to provide sample commit data, and sets up an event
    listener to display the `commit-selected` event details.
- **Dependencies:**
  - `commit-detail-panel-sk`: Used within the dialog to list and allow
    selection of individual commits. `commit-detail-picker-sk` passes the
    fetched `details` (array of `Commit` objects) to this panel.
  - `day-range-sk`: Used to allow the user to define the time window for
    which commits should be fetched. Its `day-range-change` event triggers a
    refetch in the picker.
  - `spinner-sk`: Provides visual feedback during data loading.
  - `ElementSk`: Base class from `infra-sk` providing common custom element
    functionality.
  - `jsonOrThrow`: Utility for parsing JSON responses and throwing an error
    if parsing fails or the response is not OK.
  - `errorMessage`: Utility for displaying error messages to the user.

The design focuses on encapsulation: the `commit-detail-picker-sk` component
manages its internal state (current range, fetched commits, selected index) and
exposes a clear interface for interaction (a button to open, a `selection`
property, and a `commit-selected` event). This makes it easy to integrate into
larger applications that require users to pick a commit from a potentially large
history.

# Module: /modules/commit-detail-sk

## commit-detail-sk

The `commit-detail-sk` module provides a custom HTML element
`<commit-detail-sk>` designed to display concise information about a single
commit. This element is crucial for user interfaces where presenting commit
details in a structured and interactive manner is necessary.

### Why

In applications dealing with version control systems, there's often a need to
display details of individual commits. This could be for reviewing changes,
navigating commit history, or linking to related actions like exploring code
changes, viewing clustered data, or triaging issues associated with a commit.
The `commit-detail-sk` element encapsulates this functionality, offering a
reusable and consistent way to present commit information.

### How

The core of the module is the `CommitDetailSk` class, which extends `ElementSk`.
This class defines the structure and behavior of the `<commit-detail-sk>`
element.

**Key Responsibilities and Components:**

- **`commit-detail-sk.ts`**: This is the heart of the module.

  - It defines the `CommitDetailSk` custom element.
  - The element takes a `Commit` object (defined in `perf/modules/json`) as
    input via the `cid` property. This object contains details like the
    commit hash, author, message, timestamp, and URL.
  - The `template` function, using `lit-html`, defines the HTML structure of
    the element. It displays:
  - A truncated commit hash.
  - The commit author.
  - The time elapsed since the commit (human-readable, via `diffDate`).
  - The commit message.
  - It also renders a set of Material Design outlined buttons: "Explore",
    "Cluster", "Triage", and "Commit". These buttons are intended to
    navigate the user to different views or actions related to the specific
    commit. The links for these buttons are dynamically generated based on
    the commit hash and the `cid.url`.
  - The `openLink` method handles the click events on these buttons, opening
    the respective links in a new browser window/tab.
  - `upgradeProperty` is used to ensure that the `cid` property is correctly
    initialized if it's set before the element is fully connected to the
    DOM.

- **`commit-detail-sk.scss`**: This file contains the styling for the
  `<commit-detail-sk>` element.

  - It defines styles for the layout, typography, and appearance of the
    commit information and the action buttons.
  - It utilizes CSS variables for theming (e.g., `--blue`, `--primary`),
    allowing the component to adapt to different visual themes (light and
    dark mode, as demonstrated in the demo).
  - It includes styles from `//perf/modules/themes:themes_sass_lib` and
    `//elements-sk/modules:colors_sass_lib` to ensure consistency with the
    broader application's design system.

- **`commit-detail-sk-demo.html` and `commit-detail-sk-demo.ts`**: These files
  provide a demonstration page for the `<commit-detail-sk>` element.

  - The HTML sets up basic page structure and includes instances of
    `<commit-detail-sk>` in both light and dark mode contexts.
  - The TypeScript file initializes these demo elements with sample `Commit`
    data. It also simulates a click on the element to potentially reveal
    more details or actions if such functionality were implemented (though
    in the current version, the "tip" div with buttons is always visible).
    The `Date.now` function is mocked to ensure consistent output for the
    `diffDate` calculation in the demo and tests.

**Workflow Example: Displaying Commit Information and Actions**

```
1. Application provides a `Commit` object.
   e.g., { hash: "abc123...", author: "user@example.com", ... }

2. The `Commit` object is assigned to the `cid` property of a `<commit-detail-sk>` element.
   <commit-detail-sk .cid=${commitData}></commit-detail-sk>

3. `CommitDetailSk` element renders:
   [abc123...] - [user@example.com] - [2 days ago] - [Commit message]
   +----------------------------------------------------------------+
   | [Explore] [Cluster] [Triage] [Commit (link to commit source)]  |  <- Action buttons
   +----------------------------------------------------------------+

4. User clicks an action button (e.g., "Explore").

5. `openLink` method is called with a generated URL (e.g., "/g/e/abc123...").

6. A new browser tab opens to the specified URL.
```

This design promotes reusability and separation of concerns. The element focuses
solely on presenting commit information and providing relevant action links,
making it easy to integrate into various parts of an application that need to
display commit details. The use of `lit-html` for templating allows for
efficient rendering and updates.

# Module: /modules/commit-range-sk

The `commit-range-sk` module provides a custom HTML element,
`<commit-range-sk>`, designed to display a link representing a range of commits
within a Git repository. This functionality is particularly useful in
performance analysis tools where identifying the specific commits that
introduced a performance regression or improvement is crucial.

**Core Functionality and Design:**

The primary purpose of `commit-range-sk` is to dynamically generate a URL that
points to a commit range viewer (e.g., a Git web interface like Gerrit or
GitHub). This URL is constructed based on a "begin" and an "end" commit.

- **Identifying the Commit Range:**

  - The element takes a `trace` (an array of numerical data points, where
    each point corresponds to a commit), a `commitIndex` (the index within
    the `trace` array that represents the "end" commit of interest), and
    `header` information (which maps trace indices to commit metadata like
    `offset` or commit number).
  - The "end" commit is directly determined by the `commitIndex` and the
    `header`.
  - The "begin" commit is found by iterating backward from the
    `commitIndex - 1` in the `trace`. It skips over any entries marked with
    `MISSING_DATA_SENTINEL` (indicating commits for which there's no data
    point) until it finds a valid previous commit.
  - This logic ensures that the range always spans from a commit with actual
    data to the target commit, even if there are intermediate commits with
    missing data.

- **Converting Commit Numbers to Hashes:**

  - The commit range URL template, configured globally via
    `window.perf.commit_range_url`, typically requires Git commit hashes
    (SHAs) rather than internal commit numbers or offsets.
  - The `commit-range-sk` element uses a `commitNumberToHashes` function to
    perform this conversion.
  - The default implementation, `defaultcommitNumberToHashes`, makes an
    asynchronous call to a backend service (likely `/`/cid/``) by
invoking`lookupCids`from the`//perf/modules/cid:cid_ts_lib` module. This
    service is expected to return the commit hashes corresponding to the
    provided commit numbers.
  - This design allows for testability by enabling the replacement of
    `commitNumberToHashes` with a mock function during testing (as seen in
    `commit-range-sk_test.ts`).

- **URL Construction and Display:**

  - Once the "begin" and "end" commit numbers are identified and their
    corresponding hashes are retrieved, the element populates the
    `window.perf.commit_range_url` template. This template usually contains
    placeholders like `{begin}` and `{end}` which are replaced with the
    actual commit hashes.
  - The displayed text for the link is also dynamically generated. If the
    "begin" and "end" commits are not consecutive (i.e., there's at least
    one commit between them, or the "begin" commit had to skip missing data
    points), the text will show a range like "`<begin_offset + 1> -
<end_offset>`". Otherwise, it will just show the "`<end_offset>`". The
    `+1` for the begin offset in a range is to ensure the displayed range
    starts _after_ the last known good commit.
  - The element supports two display modes controlled by the `showLinks`
    property:
  - If `showLinks` is `false` (default, or when the element is merely
    hovered over in some UIs), only the text representing the commit(s) is
    displayed.
  - If `showLinks` is `true`, a fully formed hyperlink (`<a>` tag) is
    rendered.

**Key Components/Files:**

- **`commit-range-sk.ts`**: This is the core file defining the `CommitRangeSk`
  custom element.

  - It extends `ElementSk`, a base class for custom elements in the Skia
    infrastructure.
  - It manages the state of the component through properties like `_trace`,
    `_commitIndex`, `_header`, `_url`, `_text`, and `_commitIds`.
  - The `recalcLink()` method is central to its operation. It's triggered
    whenever relevant input properties (`trace`, `commitIndex`, `header`)
    change. This method orchestrates the process of finding commit IDs,
    converting them to hashes, and generating the URL and display text.
  - `setCommitIds()` implements the logic for determining the start and end
    commit numbers based on the input trace and header, handling missing
    data points.
  - It uses the `lit/html` library for templating, allowing for efficient
    rendering and updates to the DOM.

- **`commit-range-sk-demo.ts` and `commit-range-sk-demo.html`**: These files
  provide a demonstration page for the `commit-range-sk` element.

  - `commit-range-sk-demo.ts` sets up a mock environment, including mocking
    the `fetch` call to `/`/cid/``using`fetch-mock`. This is crucial for
    demonstrating the element's behavior without needing a live backend.
  - It also initializes the global `window.perf` object with necessary
    configuration, such as the `commit_range_url` template.
  - It then instantiates the `<commit-range-sk>` element and populates its
    properties to showcase its functionality.

- **`commit-range-sk_test.ts`**: This file contains unit tests for the
  `CommitRangeSk` element.

  - It utilizes `chai` for assertions and `setUpElementUnderTest` for easy
    instantiation of the element in a test environment.
  - A key testing strategy involves overriding the `commitNumberToHashes`
    method on the element instance to provide controlled hash values and
    assert the correctness of the generated URL and text, especially in
    scenarios involving `MISSING_DATA_SENTINEL`.

- **`BUILD.bazel`**: Defines how the module is built, its dependencies (e.g.,
  `//infra-sk/modules/ElementSk`, `//perf/modules/json`, `lit`), and how the
  demo page and tests are structured.

**Workflow Example: Generating a Commit Range Link**

1.  **Initialization:**

    - The application using `<commit-range-sk>` sets the global
      `window.perf.commit_range_url` (e.g.,
      `"http://example.com/range/{begin}/{end}"`).
    - The `<commit-range-sk>` element is added to the DOM.

2.  **Property Setting:**

              - The application provides data to the element:
                - `element.trace = [10, MISSING_DATA_SENTINEL, 12, 15];`
                - `element.header = [{offset: C1}, {offset: C2}, {offset: C3},

        {offset: C4}];`(where C1-C4 are commit numbers)

    -`element.commitIndex = 3;`(points to the data`15`and commit`C4`)

        - `element.showLinks = true;`

3.  **`recalcLink()` Triggered:**

    - Changing any of the above properties automatically calls `recalcLink()`.

4.  **Determine Commit IDs (`setCommitIds()`):**

    - End commit: `header[commitIndex].offset` => `C4`.
    - Previous commit search:
      - Start at `commitIndex - 1 = 2`. `trace[2]` is `12` (not missing).
        So, `header[2].offset` => `C3`.
    - `_commitIds` becomes `[C3, C4]`.

5.  **Check if Range (`isRange()`):**

    - Is `C3 + 1 === C4`? Let's assume `C3` and `C4` are not consecutive
      (e.g., `C3=100`, `C4=102`). `isRange()` returns `true`.
    - Text becomes: `"${C3 + 1} - ${C4}"` (e.g., `"101 - 102"`).

6.  **Convert Commit IDs to Hashes (`commitNumberToHashes`):**

        - `commitNumberToHashes([C3, C4])` is called.
        - Internally, this likely makes a POST request to `/`/cid/``with`[C3,
          C4]`.
        - Backend returns: `{ commitSlice: [{hash: "hash_for_C3"}, {hash:

    "hash_for_C4"}] }`.

    - The function resolves with `["hash_for_C3", "hash_for_C4"]`.

7.  **Construct URL:**

    - `url = window.perf.commit_range_url` (e.g.,
      `"http://example.com/range/{begin}/{end}"`)
    - `url = url.replace('{begin}', "hash_for_C3")`
    - `url = url.replace('{end}', "hash_for_C4")`
    - `_url` becomes `"http://example.com/range/hash_for_C3/hash_for_C4"`.

8.  **Render:**

        - Since `showLinks` is true, the template becomes: `<a

    href="http://example.com/range/hash_for_C3/hash_for_C4"
    target="\_blank">101 - 102</a>` - The element updates its content with this HTML.

This workflow demonstrates how `commit-range-sk` encapsulates the logic for
finding relevant commits, converting their identifiers, and presenting a
user-friendly link to explore changes between them, abstracting away the
complexities of interacting with commit data and URL templates.

# Module: /modules/common

## Common Module

The `common` module houses utility functions and data structures that are shared
across various parts of the Perf application, particularly those related to data
visualization and testing. Its primary purpose is to promote code reuse and
maintain consistency in how data is processed and displayed.

### Responsibilities and Key Components

The module's responsibilities can be broken down into the following areas:

1.  **Plot Data Construction and Formatting**:

    - **Why**: Visualizing performance data often involves transforming raw
      data into formats suitable for charting libraries (like Google Charts).
      This process needs to be standardized to ensure plots are consistent and
      correctly represent the underlying information.
    - **How**:

      - `plot-builder.ts`: This file is central to preparing data for
        plotting.

      - `convertFromDataframe`: This function is crucial for adapting data
        organized in a `DataFrame` structure (where traces are rows) into a
        format suitable for Google Charts, which typically expects data in
        columns. It essentially transposes the `TraceSet`. The `domain`
        parameter allows specifying whether the x-axis should represent
        commit positions, dates, or both, providing flexibility in how
        time-series data is visualized.

        ```
        Input DataFrame (TraceSet):
        TraceA: [val1, val2, val3]
        TraceB: [valA, valB, valC]
        Header: [commit1, commit2, commit3]

        convertFromDataframe (domain='commit') ->

        Output for Google Chart:
        ["Commit Position", "TraceA", "TraceB"]
        [commit1_offset,  val1,     valA    ]
        [commit2_offset,  val2,     valB    ]
        [commit3_offset,  val3,     valC    ]
        ```

      - `ConvertData`: This function takes a `ChartData` object, which is a
        more abstract representation of plot data (lines with x, y
        coordinates and labels), and transforms it into the specific
        array-of-arrays format required by Google Charts. This abstraction
        allows other parts of the application to work with `ChartData`
        without needing to know the exact details of the charting library's
        input format.

        ```
        Input ChartData:
        xLabel: "Time"
        lines: {
          "Line1": [{x: t1, y: v1}, {x: t2, y: v2}],
          "Line2": [{x: t1, y: vA}, {x: t2, y: vB}]
        }

        ConvertData ->

        Output for Google Chart:
        ["Time", "Line1", "Line2"]
        [t1,     v1,      vA     ]
        [t2,     v2,      vB     ]
        ```

      - `mainChartOptions` and `SummaryChartOptions`: These functions
        provide pre-configured option objects for Google Line Charts. They
        encapsulate common styling and behavior (like colors, axis
        formatting, tooltip behavior, and null interpolation) to ensure a
        consistent look and feel for different types of charts (main detail
        charts vs. summary overview charts). This avoids repetitive
        configuration and makes it easier to maintain visual consistency.
        The options are also designed to adapt to the current theme
        (light/dark mode) by using CSS custom properties.

      - `defaultColors`: A predefined array of colors used for chart series,
        ensuring a consistent and visually distinct palette.

2.  **Plotting Utilities**:

    - **Why**: Beyond basic data transformation, there are common tasks
      related to preparing data specifically for plotting, such as associating
      anomalies with data points and handling missing values.
    - **How**:

      - `plot-util.ts`: This file contains helper functions that build upon
        `plot-builder.ts`.

      - `CreateChartDataFromTraceSet`: This function serves as a
        higher-level constructor for `ChartData`. It takes a raw `TraceSet`
        (a dictionary where keys are trace identifiers and values are arrays
        of numbers), corresponding x-axis labels (commit numbers or dates),
        the desired x-axis format, and anomaly information. It then iterates
        through the traces, constructs `DataPoint` objects (which include x,
        y, and any associated anomaly), and organizes them into the
        `ChartData` structure. A key aspect is its handling of
        `MISSING_DATA_SENTINEL` to exclude missing points from the chart
        data, relying on the charting library's interpolation. It also uses
        `findMatchingAnomaly` to link anomalies to their respective data
        points.

        ```
        Input TraceSet:
        "trace_foo": [10, 12, MISSING_DATA_SENTINEL, 15]
        xLabels: [c1, c2, c3, c4]
        Anomalies: { "trace_foo": [{x: c2, y: 12, anomaly: {...}}] }

        CreateChartDataFromTraceSet ->

        Output ChartData:
        lines: {
          "trace_foo": [
            {x: c1, y: 10, anomaly: null},
            {x: c2, y: 12, anomaly: {...}},
            // Point for c3 is skipped due to MISSING_DATA_SENTINEL
            {x: c4, y: 15, anomaly: null}
          ]
        }
        ...
        ```

      - `findMatchingAnomaly`: A utility to efficiently check if a given
        data point (identified by its trace key, x-coordinate, and
        y-coordinate) corresponds to a known anomaly. This is used by
        `CreateChartDataFromTraceSet` to enrich data points with anomaly
        details.

3.  **Test Utilities**:

    - **Why**: Writing effective unit and integration tests, as well as
      creating demo pages, often requires mock data and simulated API
      responses. Centralizing these test utilities avoids duplication and
      makes tests easier to write and maintain.
    - **How**:
      - `test-util.ts`: This file provides functions to set up a common
        testing and demo environment.
      - `setUpExploreDemoEnv`: This is a comprehensive function that uses
        `fetch-mock` to intercept various API calls that are typically made
        by Perf frontend components (e.g., explore page, alert details). It
        returns predefined, static responses for endpoints like
        `/_/login/status`, `/_/initpage/...`, `/_/count/`, `/_/frame/start`,
        `/_/defaults/`, `/_/status/...`, `/_/cid/`, `/_/details/`,
        `/_/shortcut/get`, `/_/nextParamList/`, and `/_/shortcut/update`.
      - The purpose of mocking these endpoints is to allow frontend
        components to be tested or demonstrated in isolation, without
        requiring a live backend. The mocked data is designed to be
        representative of real API responses, enabling realistic testing
        scenarios. For example, it provides sample `paramSet` data,
        `DataFrame` structures, commit information, and default
        configurations. This ensures that components relying on these API
        calls behave predictably in a test or demo environment. The function
        also checks for a `proxy_endpoint` cookie to avoid mocking if a real
        backend is being proxied for development or demo purposes.

# Module: /modules/const

The `/modules/const` module serves as a centralized repository for constants
utilized throughout the Perf UI. Its primary purpose is to ensure consistency
and maintainability by providing a single source of truth for values that are
shared across different parts of the user interface.

A key design decision behind this module is to manage values that might also be
defined in the backend. This avoids potential discrepancies and ensures that
frontend and backend systems operate with the same understanding of specific
sentinel values or configurations.

The core responsibility of this module is to define and export these shared
constants.

One of the key components is the `const.ts` file. This file contains the actual
definitions of the constants. A notable constant defined here is
`MISSING_DATA_SENTINEL`.

The `MISSING_DATA_SENTINEL` constant (value: `1e32`) is critical for
representing missing data points within traces. The backend uses this specific
floating-point value to indicate that a sample is absent. The choice of `1e32`
is deliberate. JSON, the data interchange format used, does not natively support
`NaN` (Not a Number) or infinity values (`+/- Inf`). Therefore, a valid
`float32` that has a compact JSON representation and is unlikely to clash with
actual data values was chosen. It is imperative that this frontend constant
remains synchronized with the `MissingDataSentinel` constant defined in the
backend Go package `//go/vec32/vec`. This synchronization ensures that both the
UI and the backend correctly interpret missing data.

Any part of the Perf UI that needs to interpret or display trace data,
especially when dealing with potentially incomplete datasets, will rely on this
`MISSING_DATA_SENTINEL`. For instance, charting libraries or data table
components might use this constant to visually differentiate missing points or
to exclude them from calculations.

Workflow involving `MISSING_DATA_SENTINEL`:

Backend Data Generation --> Data contains `MissingDataSentinel` from
`//go/vec32/vec` | V Data Serialization (JSON) --> `1e32` is used for missing
data | V Frontend Data Fetching | V Frontend UI Component (e.g., a chart) | V UI
uses `MISSING_DATA_SENTINEL` from `/modules/const/const.ts` to identify missing
points | V Appropriate rendering (e.g., gap in a line chart, specific
placeholder in a table)

# Module: /modules/csv

The `/modules/csv` module provides functionality to convert `DataFrame` objects,
a core data structure representing performance or experimental data, into the
Comma Separated Values (CSV) format. This conversion is essential for users who
wish to export data for analysis in external tools, spreadsheets, or for
archival purposes.

The primary challenge in converting a `DataFrame` to CSV lies in representing
the potentially sparse and varied parameter sets associated with each trace
(data series) in a flat, tabular format. The `DataFrame` stores traces indexed
by a "trace ID," which is a string encoding of key-value pairs representing the
parameters that uniquely identify that trace.

The conversion process addresses this challenge through a multi-step approach:

1.  **Parameter Key Consolidation**:

    - The `parseIdsIntoParams` function takes an array of trace IDs and
      transforms each ID string back into its constituent key-value parameter
      pairs. This is achieved by leveraging the `fromKey` function from the
      `//perf/modules/paramtools` module.
    - The `allParamKeysSorted` function then iterates through all these parsed
      parameter sets to identify the complete, unique set of all parameter
      keys present across all traces. These keys are then sorted
      alphabetically. This sorted list of unique parameter keys will form the
      initial set of columns in the CSV, ensuring a consistent order and
      comprehensive representation of all parameters.

    _Pseudocode for parameter key consolidation:_

    ```
    traceIDs = ["key1=valueA,key2=valueB", "key1=valueC,key3=valueD"]
    parsedParams = {}
    for each id in traceIDs:
      parsedParams[id] = fromKey(id) // e.g., {"key1=valueA,key2=valueB": {key1:"valueA", key2:"valueB"}}

    allKeys = new Set()
    for each params in parsedParams.values():
      for each key in params.keys():
        allKeys.add(key)

    sortedColumnNames = sorted(Array.from(allKeys)) // e.g., ["key1", "key2", "key3"]
    ```

2.  **Header Row Generation**:

    - The `dataFrameToCSV` function begins by constructing the header row of
      the CSV.
    - This row starts with the `sortedColumnNames` derived in the previous
      step.
    - It then appends column headers derived from the `DataFrame`'s `header`
      property. Each element in `df.header` typically represents a point in
      time (or a commit, build, etc.), and its `timestamp` field is converted
      into an ISO 8601 formatted date string.

    _Pseudocode for header row generation:_

    ```
    csvHeader = sortedColumnNames
    for each columnHeader in df.header:
      csvHeader.push(new Date(columnHeader.timestamp * 1000).toISOString())
    csvLines.push(csvHeader.join(','))
    ```

3.  **Data Row Generation**:

    - For each trace in the `df.traceset` (excluding "special\_" traces, which
      are likely internal or metadata traces not intended for direct CSV
      export):
      - The corresponding parameter values for the `sortedColumnNames` are
        retrieved. If a trace does not have a value for a particular
        parameter key, an empty string is used, ensuring that each row has
        the same number of columns corresponding to the parameter keys.
      - The actual data points for the trace are then appended. The
        `MISSING_DATA_SENTINEL` (defined in `//perf/modules/const`) is a
        special value indicating missing data; this is converted to an empty
        string in the CSV to represent a null or missing value. Other
        numerical values are appended directly.
      - Each fully constructed row is then joined by commas.

    _Pseudocode for data row generation:_

    ```
    for each traceId, traceData in df.traceset:
      if traceId starts with "special_":
        continue

      traceParams = parsedParams[traceId]
      rowData = []
      for each columnName in sortedColumnNames:
        rowData.push(traceParams[columnName] or "") // Add parameter value or empty string

      for each value in traceData:
        if value is MISSING_DATA_SENTINEL:
          rowData.push("")
        else:
          rowData.push(value)
      csvLines.push(rowData.join(','))
    ```

4.  **Final CSV String Assembly**:

    - Finally, all the generated lines (header and data rows) are joined
      together with newline characters (`\n`) to produce the complete CSV
      string.

The design prioritizes creating a CSV that is both human-readable and easily
parsable by other tools. By dynamically determining the parameter columns based
on the input `DataFrame` and sorting them, it ensures that all relevant trace
metadata is included in a consistent manner. The explicit handling of
`MISSING_DATA_SENTINEL` ensures that missing data is represented clearly as
empty fields.

The key files in this module are:

- `index.ts`: This file contains the core logic for the CSV conversion. It
  houses the `parseIdsIntoParams`, `allParamKeysSorted`, and the main
  `dataFrameToCSV` functions. It leverages helper functions from
  `//perf/modules/paramtools` for parsing trace ID strings and relies on
  constants from `//perf/modules/const` for identifying missing data.
- `index_test.ts`: This file provides unit tests for the `dataFrameToCSV`
  function. It defines a sample `DataFrame` with various scenarios, including
  different parameter sets per trace and missing data points, and asserts that
  the generated CSV matches the expected output. This is crucial for ensuring
  the correctness and robustness of the CSV generation logic.

The dependencies on `//perf/modules/const` (for `MISSING_DATA_SENTINEL`) and
`//perf/modules/json` (for `DataFrame`, `ColumnHeader`, `Params` types) indicate
that this module is tightly integrated with the broader data representation and
handling mechanisms of the Perf system. The dependency on
`//perf/modules/paramtools` (for `fromKey`) highlights its role in interpreting
the structured information encoded within trace IDs.

# Module: /modules/dataframe

The `dataframe` module is designed to manage and manipulate time-series data,
specifically performance testing traces, within the Perf application. It
provides a centralized way to fetch, store, and process trace data, enabling
functionalities like visualizing performance trends, identifying anomalies, and
managing user-reported issues.

The core idea is to have a reactive data repository that components can consume.
This allows for efficient data loading and updates, especially when dealing with
large datasets and dynamic time ranges. Instead of each component fetching and
managing its own data, they can rely on a shared `DataFrameRepository` to handle
these tasks. This promotes consistency and reduces redundant data fetching.

## Key Components and Responsibilities

### `dataframe_context.ts`

This file defines the `DataFrameRepository` class, which acts as the central
data store and manager. It's implemented as a LitElement
(`<dataframe-repository-sk>`) that doesn't render any UI itself but provides
data and loading states through Lit contexts.

**Why a LitElement with Contexts?** Using a LitElement allows easy integration
into the existing component-based architecture. Lit contexts (`@lit/context`)
provide a clean and reactive way for child components to consume the `DataFrame`
and related information without prop drilling or complex event bus
implementations.

**Core Functionalities:**

- **Data Fetching:**

  - `resetTraces(range, paramset)`: Fetches an initial set of traces based
    on a time range and a `ParamSet` (a set of key-value pairs defining the
    traces to query). This is typically called when the user defines a new
    query. `User defines query -> explore-simple-sk calls resetTraces() | V
DataFrameRepository -> Fetches data from /_/frame/start | V Updates
internal _header, _traceset, anomaly, userIssues | V Provides DataFrame,
DataTable, AnomalyMap, UserIssueMap via context`
  - `extendRange(offsetInSeconds)`: Fetches additional data to extend the
    current time range, either forwards or backwards. This is used for
    infinite scrolling or when the user wants to see more data. To improve
    performance for large range extensions, it slices the requested range
    into smaller chunks (`chunkSize`) and fetches them concurrently. `User
scrolls/requests more data -> UI calls extendRange() | V
DataFrameRepository -> Slices range into chunks if needed | V Fetches
data for each chunk from /_/frame/start concurrently | V Merges new data
with existing _header, _traceset, anomaly | V Provides updated
DataFrame, DataTable, AnomalyMap via context`
  - The fetching mechanism uses the `/_/frame/start` endpoint, sending a
    `FrameRequest` which includes the time range, query (derived from
    `ParamSet`), and timezone.
  - It handles responses, including potential errors or "Finished" status
    with no data (e.g., no commits in the requested range).

- **Data Caching and Merging:**

  - Maintains an internal representation of the data: `_header` (array of
    `ColumnHeader` objects, representing commit points/timestamps) and
    `_traceset` (a `TraceSet` object mapping trace keys to their data
    arrays).
  - When new data is fetched (either initial load or extension), it's merged
    with the existing cached data. The merging logic ensures that headers
    are correctly ordered and trace data is appropriately prepended or
    appended. If a trace being extended isn't present in a new data chunk,
    it's padded with `MISSING_DATA_SENTINEL` to maintain alignment with the
    header.

- **Anomaly Management:**

  - Fetches anomaly data (`AnomalyMap`) along with the trace data.
  - `updateAnomalies(anomalies, id)`: Allows merging new anomalies and
    removing specific anomalies (e.g., when an anomaly is nudged or
    re-triaged). This uses `mergeAnomaly` and `removeAnomaly` from
    `index.ts`.

- **User-Reported Issue Management:**

  - `getUserIssues(traceKeys, begin, end)`: Fetches user-reported issues
    (e.g., Buganizer bugs linked to specific data points) from the
    `/_/user_issues/` endpoint for a given set of traces and commit range.
  - `updateUserIssue(traceKey, commitPosition, bugId)`: Updates the local
    cache of user issues, typically after a new issue is filed or an
    existing one is modified.
  - Trace keys are normalized by removing special functions (e.g., `norm()`)
    before querying for user issues to ensure issues are found even if the
    displayed trace is a transformed version of the original.

- **Google DataTable Conversion:**

  - Converts the internal `DataFrame` into a
    `google.visualization.DataTable` format using `convertFromDataframe`
    (from `perf/modules/common:plot-builder_ts_lib`). This `DataTable` is
    then provided via `dataTableContext` and is typically consumed by
    charting components like `<plot-google-chart-sk>`.
  - The Google Chart library is loaded asynchronously
    (`DataFrameRepository.loadPromise`).

- **State Management:**

  - `loading`: A boolean provided via `dataframeLoadingContext` to indicate
    if a data request is in flight.
  - `_requestComplete`: A Promise that resolves when the current data
    fetching operation completes. This can be used to coordinate actions
    that depend on data being available.

**Contexts Provided:**

- `dataframeContext`: Provides the current `DataFrame` object.
- `dataTableContext`: Provides the `google.visualization.DataTable` derived
  from the `DataFrame`.
- `dataframeAnomalyContext`: Provides the `AnomalyMap` for the current data.
- `dataframeUserIssueContext`: Provides the `UserIssueMap` for the current
  data.
- `dataframeLoadingContext`: Provides a boolean indicating if data is
  currently being loaded.
- `dataframeRepoContext`: Provides the `DataFrameRepository` instance itself,
  allowing consumers to call its methods (e.g., `extendRange`).

### `index.ts`

This file contains utility functions for manipulating `DataFrame` structures,
similar to its Go counterpart (`//perf/go/dataframe/dataframe.go`). These
functions are crucial for merging, slicing, and analyzing the data.

**Key Functions:**

- `findSubDataframe(header, range, domain)`: Given a `DataFrame` header and a
  time/offset range, this function finds the start and end indices within the
  header that correspond to the given range. This is essential for slicing
  data.
- `generateSubDataframe(dataframe, range)`: Creates a new `DataFrame`
  containing only the data within the specified index range of the original
  `DataFrame`.
- `mergeAnomaly(anomaly1, ...anomalies)`: Merges multiple `AnomalyMap` objects
  into a single one. If anomalies exist for the same trace and commit, the
  later ones in the arguments list will overwrite earlier ones. It always
  returns a non-null `AnomalyMap`.
- `removeAnomaly(anomalies, id)`: Creates a new `AnomalyMap` excluding any
  anomalies with the specified `id`. This is used when an anomaly is moved or
  re-triaged on the backend, and the old entry needs to be cleared.
- `findAnomalyInRange(allAnomaly, range)`: Filters an `AnomalyMap` to include
  only anomalies whose commit positions fall within the given commit range.
- `mergeColumnHeaders(a, b)`: Merges two arrays of `ColumnHeader` objects,
  producing a new sorted array of unique headers. It also returns mapping
  objects (`aMap`, `bMap`) that indicate the new index of each header from the
  original arrays. This is fundamental for the `join` operation.
  - **Why map objects?** When merging traces from two DataFrames, the data
    points need to be placed at the correct positions in the newly merged
    header. The maps provide this correspondence.
- `join(a, b)`: Combines two `DataFrame` objects into a new one.
  1.  It first merges their headers using `mergeColumnHeaders`.
  2.  Then, it creates a new `traceset`. For each trace in the original
      DataFrames, it uses the `aMap` and `bMap` to place the trace data points
      into the correct slots in the new, longer trace arrays, filling gaps
      with `MISSING_DATA_SENTINEL`.
  3.  It also merges the `paramset` from both DataFrames.
  4.  **Purpose:** This is useful when combining data from different sources
      or different time periods that might not perfectly align.
- `buildParamSet(d)`: Reconstructs the `paramset` of a `DataFrame` based on
  the keys present in its `traceset`. This ensures the `paramset` accurately
  reflects the data.
- `timestampBounds(df)`: Returns the earliest and latest timestamps present in
  the `DataFrame`'s header.

### `traceset.ts`

This file provides utility functions for extracting and formatting information
from the trace keys within a `DataFrame` or `DataTable`. Trace keys are strings
that encode various parameters (e.g.,
`",benchmark=Speedometer,test=MotionMark,"`).

**Key Functions:**

- `getAttributes(df)`: Extracts all unique attribute keys (e.g., "benchmark",
  "test") present across all trace keys in a `DataFrame`.
- `getTitle(dt)`: Identifies the common key-value pairs across all trace
  labels in a `DataTable`. These common pairs form the "title" of the chart,
  representing what all displayed traces have in common.
  - **Why `DataTable` input?** This function is often used directly with the
    `DataTable` that feeds a chart, as column labels in the `DataTable` are
    typically the trace keys.
- `getLegend(dt)`: Identifies the key-value pairs that are _not_ common across
  all trace labels in a `DataTable`. These differing parts form the "legend"
  for each trace, distinguishing them from one another.
  - It ensures that all legend objects have the same set of keys (sorted
    alphabetically), filling in missing values with `"untitled_key"` for
    consistency in display.
- `titleFormatter(title)`: Formats the output of `getTitle` (an object) into a
  human-readable string, typically by joining values with '/'.
- `legendFormatter(legend)`: Formats the output of `getLegend` (an array of
  objects) into an array of human-readable strings.
- `getLegendKeysTitle(label)`: Takes a legend object (for a single trace) and
  creates a string by joining its keys, often used as a title for the legend
  section.
- `isSingleTrace(dt)`: Checks if a `DataTable` contains data for only a single
  trace (i.e., has 3 columns: domain, commit position/date, and one trace).
- `findTraceByLabel(dt, legendTraceId)`: Finds the column label (trace key) in
  a `DataTable` that matches the given `legendTraceId`.
- `findTracesForParam(dt, paramKey, paramValue)`: Finds all trace labels in a
  `DataTable` that contain a specific key-value pair.
- `removeSpecialFunctions(key)`: A helper used internally to strip function
  wrappers (like `norm(...)`) from trace keys before processing, ensuring that
  the underlying parameters are correctly parsed.

**Design Rationale for Title/Legend Generation:** When multiple traces are
plotted, the title should reflect what's common among them (e.g.,
"benchmark=Speedometer"), and the legend should highlight what's different
(e.g., "test=Run1" vs. "test=Run2"). These functions automate this process by
analyzing the trace keys.

## Workflows

### Initial Data Load and Display

```
1. User navigates to a page or submits a query.
   |
   V
2. <explore-simple-sk> (or similar component) determines initial time range and ParamSet.
   |
   V
3. Calls `dataframeRepository.resetTraces(initialRange, initialParamSet)`.
   |
   V
4. DataFrameRepository:
   a. Sets `loading = true`.
   b. Constructs `FrameRequest`.
   c. POSTs to `/_/frame/start`.
   d. Receives `FrameResponse` (containing DataFrame and AnomalyMap).
   e. Updates its internal `_header`, `_traceset`, `anomaly`.
   f. Calls `setDataFrame()`:
      i. Updates `this.dataframe` (triggers `dataframeContext`).
      ii. Converts DataFrame to `google.visualization.DataTable`.
      iii. Updates `this.data` (triggers `dataTableContext`).
   g. Updates `this.anomaly` (triggers `dataframeAnomalyContext`).
   h. Sets `loading = false`.
   |
   V
5. Charting components (consuming `dataTableContext`) re-render with the new data.
   |
   V
6. Other UI elements (consuming `dataframeContext`, `dataframeAnomalyContext`) update.
```

### Extending Time Range (e.g., Scrolling)

```
1. User action triggers a request to load more data (e.g., scrolls near edge of chart).
   |
   V
2. UI component calls `dataframeRepository.extendRange(offsetInSeconds)`.
   |
   V
3. DataFrameRepository:
   a. Sets `loading = true`.
   b. Calculates the new time range (`deltaRange`).
   c. Slices the new range into chunks if `offsetInSeconds` is large (`sliceRange`).
   d. For each chunk:
      i. Constructs `FrameRequest`.
      ii. POSTs to `/_/frame/start`.
   e. `Promise.all` awaits all chunk responses.
   f. Filters out empty/error responses and sorts responses by timestamp.
   g. Merges `header` and `traceset` from sorted responses into existing `_header` and `_traceset`.
      - For traceset: pads with `MISSING_DATA_SENTINEL` if a trace is missing in a new chunk.
   h. Merges `anomalymap` from sorted responses into existing `anomaly`.
   i. Calls `setDataFrame()` (as in initial load).
   j. Sets `loading = false`.
   |
   V
4. Charting components and other UI elements update.
```

### Displaying Chart Title and Legend

```
1. Charting component (e.g., <perf-explore-sk>) has access to the `DataTable` via `dataTableContext`.
   |
   V
2. It calls `getTitle(dataTable)` and `getLegend(dataTable)` from `traceset.ts`.
   |
   V
3. It then uses `titleFormatter` and `legendFormatter` to get displayable strings.
   |
   V
4. Renders these strings as the chart title and legend series.
```

## Testing

- `dataframe_context_test.ts`: Tests the `DataFrameRepository` class. It uses
  `fetch-mock` to simulate API responses from `/_/frame/start` and
  `/_/user_issues/`. Tests cover initialization, data loading (`resetTraces`),
  range extension (`extendRange`) with and without chunking, anomaly merging,
  and user issue fetching/updating.
- `index_test.ts`: Tests the utility functions in `index.ts`, such as
  `mergeColumnHeaders`, `join`, `findSubDataframe`, `mergeAnomaly`, etc. It
  uses manually constructed `DataFrame` objects to verify the logic of these
  data manipulation functions.
- `traceset_test.ts`: Tests the functions in `traceset.ts` for extracting
  titles and legends from trace keys. It generates `DataFrame` objects with
  various key combinations, converts them to `DataTable` (requiring Google
  Chart API to be loaded), and then asserts the output of `getTitle`,
  `getLegend`, etc.
- `test_utils.ts`: Provides helper functions for tests, notably:
  - `generateFullDataFrame`: Creates mock `DataFrame` objects with specified
    structures, which is invaluable for setting up consistent test
    scenarios.
  - `generateAnomalyMap`: Creates mock `AnomalyMap` objects linked to a
    `DataFrame`.
  - `mockFrameStart`: A utility to easily mock the `/_/frame/start` endpoint
    with `fetch-mock`, returning parts of a provided full `DataFrame` based
    on the request's time range.
  - `mockUserIssues`: Mocks the `/_/user_issues/` endpoint.

The testing strategy relies heavily on creating controlled mock data and API
responses to ensure that the data processing and fetching logic behaves as
expected under various conditions.

# Module: /modules/day-range-sk

The `day-range-sk` module provides a custom HTML element for selecting a date
range. It allows users to pick a "begin" and "end" date, which is a common
requirement in applications that deal with time-series data or event logging.

The primary goal of this module is to offer a user-friendly way to define a time
interval. It achieves this by composing two `calendar-input-sk` elements, one
for the start date and one for the end date. This design choice leverages an
existing, well-tested component for date selection, promoting code reuse and
consistency.

**Key Components and Responsibilities:**

- **`day-range-sk.ts`**: This is the core file defining the `DayRangeSk`
  custom element.

  - **Why**: It encapsulates the logic for managing the begin and end dates,
    handling user interactions, and emitting an event when the range
    changes.
  - **How**:
  - It extends `ElementSk`, a base class for custom elements, providing
    lifecycle callbacks and rendering capabilities.
  - It uses the `lit-html` library for templating, rendering two
    `calendar-input-sk` elements labeled "Begin" and "End".
  - The `begin` and `end` dates are stored as attributes (and corresponding
    properties) representing Unix timestamps in seconds. This is a common
    and unambiguous way to represent points in time.
  - When either `calendar-input-sk` element fires an `input` event
    (signifying a date change), the `DayRangeSk` element updates its
    corresponding `begin` or `end` attribute and then dispatches a custom
    event named `day-range-change`.
  - The `day-range-change` event's `detail` object contains the `begin` and
    `end` timestamps, allowing parent components to easily consume the
    selected range.
  - Default values for `begin` and `end` are set if not provided: `begin`
    defaults to 24 hours before the current time, and `end` defaults to the
    current time. This provides a sensible initial state.
  - The `connectedCallback` and `attributeChangedCallback` are used to
    ensure the element renders correctly when added to the DOM or when its
    attributes are modified.

- **`day-range-sk.scss`**: This file contains the styling for the
  `day-range-sk` element.

  - **Why**: To provide a consistent visual appearance and integrate with
    the application's theming.
  - **How**: It imports common theme variables (`themes.scss`) and defines
    specific styles for the labels and input fields within the
    `day-range-sk` component, ensuring they adapt to light and dark modes.

- **`day-range-sk-demo.html` and `day-range-sk-demo.ts`**: These files provide
  a demonstration page for the `day-range-sk` element.

  - **Why**: To showcase the element's functionality, allow for interactive
    testing, and serve as an example of how to use it.
  - **How**:
  - The HTML file includes instances of `day-range-sk` with different
    initial `begin` and `end` attributes.
  - The TypeScript file listens for the `day-range-change` event from these
    instances and displays the event details in a `<pre>` tag, demonstrating
    how to retrieve the selected date range.

- **`day-range-sk_puppeteer_test.ts`**: This file contains Puppeteer tests for
  the `day-range-sk` element.

  - **Why**: To ensure the element renders correctly and behaves as expected
    in a browser environment.
  - **How**: It uses the `loadCachedTestBed` utility to set up a testing
    environment, navigates to the demo page, and takes screenshots for
    visual regression testing. It also performs a basic smoke test to
    confirm the element is present on the page.

**Key Workflows:**

1.  **Initialization:** `User HTML` -> `day-range-sk (attributes: begin, end)`
    `day-range-sk.connectedCallback()` `IF begin/end not set` `Set default begin
(now - 24h), end (now)` `_render()` `Create two <calendar-input-sk> elements
with initial dates`

2.  **User Selects a New "Begin" Date:** `User interacts with "Begin"
<calendar-input-sk>` `<calendar-input-sk> fires "input" event (with new
Date)` `day-range-sk._beginChanged(event)` `Update this.begin (convert Date
to timestamp)` `this._sendEvent()` `Dispatch "day-range-change" event with {
begin: new_begin_timestamp, end: current_end_timestamp }`

3.  **User Selects a New "End" Date:** `User interacts with "End"
<calendar-input-sk>` `<calendar-input-sk> fires "input" event (with new
Date)` `day-range-sk._endChanged(event)` `Update this.end (convert Date to
timestamp)` `this._sendEvent()` `Dispatch "day-range-change" event with {
begin: current_begin_timestamp, end: new_end_timestamp }`

4.  **Parent Component Consumes Date Range:** `Parent Component` `Listen for
"day-range-change" on <day-range-sk>` `On event:` `Access
event.detail.begin` `Access event.detail.end` `Perform actions with the new
date range`

The conversion between `Date` objects (used by `calendar-input-sk`) and numeric
timestamps (used by `day-range-sk`'s attributes and events) is handled
internally by the `dateFromTimestamp` utility function and by using
`Date.prototype.valueOf() / 1000`. This design ensures that the `day-range-sk`
element exposes a simple, numeric API for its date range while leveraging a more
complex date object-based component for the UI.

# Module: /modules/domain-picker-sk

The `domain-picker-sk` module provides a custom HTML element
`<domain-picker-sk>` that allows users to select a data domain. This domain can
be defined in two ways: either as a specific date range or as a number of data
points (commits) preceding a chosen end date. This flexibility is crucial for
applications that need to visualize or analyze time-series data where users
might want to focus on a specific period or view the most recent N data points.

The core design choice is to offer these two distinct modes of domain selection,
catering to different user needs. The "Date Range" mode is useful when users
know the specific start and end dates they are interested in. The "Dense" mode
is more suitable when users want to see a fixed amount of recent data,
regardless of the specific start date.

The component's state is managed internally and can also be set externally via
the `state` property. This `state` object, defined by the `DomainPickerState`
interface, holds the `begin` and `end` timestamps (in Unix seconds), the
`num_commits` (for "Dense" mode), and the `request_type` which indicates the
current selection mode (0 for "Date Range" - `RANGE`, 1 for "Dense" - `DENSE`).

**Key Files and Their Responsibilities:**

- **`domain-picker-sk.ts`**: This is the heart of the module. It defines the
  `DomainPickerSk` class, which extends `ElementSk`.

  - **Why**: It encapsulates all the logic for rendering the UI, handling
    user interactions, and managing the component's state.
  - **How**:
  - It uses the `lit-html` library for templating, allowing for efficient
    updates to the DOM when the state changes. The `template` static method
    defines the basic structure, and `_showRadio` and `_requestType` static
    methods conditionally render different parts of the UI based on the
    current `request_type` and the `force_request_type` attribute.
  - It manages the `_state` object. Initial default values are set in the
    constructor (e.g., end date is now, begin date is 24 hours ago, default
    `num_commits` is 50).
  - Event handlers like `typeRange`, `typeDense`, `beginChange`,
    `endChange`, and `numChanged` update the internal `_state` and then call
    `render()` to reflect these changes in the UI.
  - The `force_request_type` attribute (`'range'` or `'dense'`) allows the
    consuming application to lock the picker into a specific mode, hiding
    the radio buttons that would normally allow the user to switch. This is
    useful when the application context dictates a specific type of domain
    selection. The `attributeChangedCallback` and the getter/setter for
    `force_request_type` handle this.
  - It leverages other custom elements: `radio-sk` for mode selection and
    `calendar-input-sk` for date picking, promoting modularity and reuse.

- **`domain-picker-sk.scss`**: This file contains the SASS styles for the
  component.

  - **Why**: It separates the presentation from the logic, making the
    component easier to style and maintain.
  - **How**: It defines styles for the layout of controls (e.g., using
    flexbox to align items), descriptive text, input fields, and the
    calendar input. It also imports shared styles from
    `elements-sk/modules/styles` for consistency (e.g., buttons, colors).

- **`index.ts`**: A simple entry point that imports and registers the
  `domain-picker-sk` custom element.

  - **Why**: This is a common pattern for web components, making it easy for
    other parts of the application to import and use the component.
  - **How**: It executes `import './domain-picker-sk';` which ensures the
    `DomainPickerSk` class is defined and registered with the browser's
    `CustomElementRegistry` via the `define` function call within
    `domain-picker-sk.ts`.

- **`domain-picker-sk-demo.html` and `domain-picker-sk-demo.ts`**: These files
  provide a demonstration page for the component.

  - **Why**: They allow developers to see the component in action, test its
    different states and attributes, and serve as a basic example of how to
    use it.
  - **How**: `domain-picker-sk-demo.html` includes instances of
    `<domain-picker-sk>`, some with the `force_request_type` attribute set.
    `domain-picker-sk-demo.ts` initializes the `state` of these demo
    instances with sample data.

- **`domain-picker-sk_puppeteer_test.ts`**: Contains Puppeteer tests for the
  component.

  - **Why**: To ensure the component renders correctly and behaves as
    expected in a browser environment.
  - **How**: It uses the `puppeteer-tests/util` library to load the demo
    page and take screenshots, verifying the visual appearance of the
    component in its default state.

**Key Workflows/Processes:**

1.  **Initialization and Rendering:**

    - `<domain-picker-sk>` element is added to the DOM.
    - `connectedCallback` is invoked.
    - Properties like `state` and `force_request_type` are upgraded (if set as
      attributes before the element was defined).
    - Default `_state` is established (e.g., end = now, begin = 24h ago,
      mode = RANGE).
    - `render()` is called:
      - It checks `force_request_type`. If set, it overrides
        `_state.request_type`.
      - The main template is rendered.
      - `_showRadio` decides whether to show mode selection radio buttons.
      - `_requestType` renders either the "Begin" date input (for RANGE
        mode) or the "Points" number input (for DENSE mode).

    ```
    [DOM Insertion] -> connectedCallback() -> _upgradeProperty('state')
                                           -> _upgradeProperty('force_request_type')
                                           -> render()
                                                |
                                                V
                                          [UI Displayed]
    ```

2.  **User Changes Mode (if `force_request_type` is not set):**

    - User clicks on "Date Range" or "Dense" radio button.
    - `@change` event triggers `typeRange()` or `typeDense()`.
    - `_state.request_type` is updated.
    - `render()` is called.
    - The UI updates to show the relevant inputs (Begin date vs. Points).

    ```
    [User clicks radio] -> typeRange()/typeDense() -> _state.request_type updated
                                                 -> render()
                                                      |
                                                      V
                                                [UI Updates]
    ```

3.  **User Changes Date/Number of Commits:**

    - User interacts with `<calendar-input-sk>` (for Begin/End dates) or the
      `<input type="number">` (for Points).
    - `@input` (for calendar) or `@change` (for number input) event triggers
      `beginChange()`, `endChange()`, or `numChanged()`.
    - The corresponding part of `_state` (e.g., `_state.begin`, `_state.end`,
      `_state.num_commits`) is updated.
    - `render()` is called (though in the case of date changes, the
      `<calendar-input-sk>` handles its own visual update for the date
      display, and `render()` here ensures the parent component is aware and
      can re-render if other parts depend on these values, although in the
      current implementation, `render()` on the parent might be redundant for
      just date changes if no other part of _this_ component's template
      changes directly).

    ```
    [User changes input] -> beginChange()/endChange()/numChanged()
                            |
                            V
                        _state updated
                            |
                            V
                          render()  // Potentially re-renders the component
                            |
                            V
                      [UI reflects new value]
    ```

The component emits no custom events itself but relies on the events from its
child components (`radio-sk`, `calendar-input-sk`) to trigger internal state
updates and re-renders. Consumers of `domain-picker-sk` would typically read the
`state` property to get the user's selection.

# Module: /modules/errorMessage

The `errorMessage` module provides a wrapper around the `errorMessage` function
from the `elements-sk` library. Its primary purpose is to offer a more
convenient way to display persistent error messages to the user.

**Core Functionality and Design Rationale:**

The key differentiation of this module lies in its default behavior for message
display duration. While the `elements-sk` `errorMessage` function requires a
duration to be specified for how long a message (often referred to as a "toast")
remains visible, this module defaults the duration to `0` seconds.

This design choice is intentional: a duration of `0` typically signifies that
the error message will _not_ automatically close. This is particularly useful in
scenarios where an error is critical or requires user acknowledgment, and an
auto-dismissing message might be missed. By defaulting to a persistent display,
the module prioritizes ensuring the user is aware of the error.

**Responsibilities and Key Components:**

The module exposes a single function: `errorMessage`.

- **`errorMessage(message: string | { message: string } | { resp: Response } |
object, duration: number = 0): void`**:
  - This function is responsible for displaying an error message to the
    user.
  - It accepts the same flexible `message` parameter as the underlying
    `elements-sk` function. This means it can handle plain strings, objects
    with a `message` property, objects containing a `Response` object (from
    which an error message can often be extracted), or generic objects.
  - The crucial aspect is the `duration` parameter. If not explicitly
    provided by the caller, it defaults to `0`. This default triggers the
    persistent display behavior mentioned above.
  - Internally, this function simply calls `elementsErrorMessage` from the
    `elements-sk` library, passing along the provided `message` and the
    (potentially defaulted) `duration`.

**Workflow:**

The typical workflow for using this module is straightforward:

1.  **Import:** The `errorMessage` function is imported from this module.
2.  **Invocation:** When an error condition occurs that needs to be communicated
    to the user persistently, the `errorMessage` function is called with the
    error details.
    - `errorMessage("A critical error occurred.")` -> Displays "A critical
      error occurred." indefinitely.
    - `errorMessage("Something went wrong.", 5000)` -> Displays "Something
      went wrong." for 5 seconds (overriding the default).

Essentially, this module acts as a thin convenience layer, promoting a specific
error display pattern (persistent messages) by changing the default behavior of
a more general utility. This reduces boilerplate for common use cases where
persistent error notification is desired.

# Module: /modules/existing-bug-dialog-sk

The `existing-bug-dialog-sk` module provides a user interface element for
associating performance anomalies with existing bug reports in a bug tracking
system (like Monorail). It's designed to be used within a larger performance
monitoring application where users need to triage and manage alerts generated by
performance regressions.

The core purpose of this module is to simplify the workflow of linking one or
more detected anomalies to a pre-existing bug. Instead of manually navigating to
the bug tracker and updating the bug, users can do this directly from the
performance monitoring interface. This reduces context switching and streamlines
the bug management process.

**Key Components and Responsibilities:**

- **`existing-bug-dialog-sk.ts`**: This is the heart of the module, defining
  the custom HTML element `existing-bug-dialog-sk`.

  - **Why**: It encapsulates the entire UI and logic for the dialog. This
    includes displaying a form for entering a bug ID, a dropdown for
    selecting the bug tracking project (though currently hardcoded to
    'chromium'), and a list of already associated bugs for the selected
    anomalies.
  - **How**:
  - It uses Lit for templating and rendering the dialog's HTML structure.
  - It manages the dialog's visibility (`open()`, `closeDialog()`).
  - It handles form submission:
    - Takes the entered bug ID and the list of selected anomalies
      (`_anomalies`).
    - Makes an HTTP POST request to a backend endpoint
      (`/_/triage/associate_alerts`) to create the association.
    - Upon success, it opens the bug page in a new tab and dispatches a
      custom event `anomaly-changed`. This event signals other parts of
      the application (e.g., charts or lists displaying anomalies) that
      the anomaly data has been updated (specifically, the `bug_id` field)
      and they might need to re-render.
    - Handles potential errors by displaying an error message toast.
  - It fetches and displays a list of bugs already associated with the
    anomalies in the current group. This involves:
    - Making a POST request to `/_/anomalies/group_report` to get details
      of anomalies in the same group, including their associated
      `bug_id`s. This endpoint might return a `sid` (state ID) if the
      report generation is asynchronous, requiring a follow-up request.
    - Once the list of associated bug IDs is retrieved, it makes another
      POST request to `/_/triage/list_issues` to fetch the titles of these
      bugs. This provides more context to the user than just showing bug
      IDs.
  - The `setAnomalies()` method is crucial for initializing the dialog with
    the relevant anomaly data when it's about to be shown.
  - It relies on `window.perf.bug_host_url` to construct links to the bug
    tracker.

- **`existing-bug-dialog-sk.scss`**: This file contains the SASS/CSS styles
  for the dialog.

  - **Why**: It ensures the dialog has a consistent look and feel with the
    rest of the application, using shared theme variables
    (`--on-background`, `--background`, etc.).
  - **How**: It defines styles for the dialog container, input fields,
    buttons, close icon, and the list of associated bugs. It also includes
    specific styling for the loading spinner and selected items.

- **`index.ts`**: This is a simple entry point that imports and registers the
  `existing-bug-dialog-sk` custom element, making it available for use in
  HTML.

**Workflow for Associating Anomalies with an Existing Bug:**

1.  **User Action**: The user selects one or more anomalies in the main
    application interface and chooses an option to associate them with an
    existing bug.
2.  **Dialog Initialization**: The application calls `setAnomalies()` on an
    `existing-bug-dialog-sk` instance, passing the selected anomalies.
3.  **Dialog Display**: The application calls `open()` on the dialog instance.
    `Application existing-bug-dialog-sk | | | -- setAnomalies(anomalies) --> | |
| | ------ open() ---------> | | | | -- fetch_associated_bugs() --> Backend
API (/anomalies/group_report) | | | <-- (Associated Bug IDs) -- | | | | --
fetch_bug_titles() ---> Backend API (/triage/list_issues) | | | <--- (Bug
Titles) -------- | | | | -- Renders Dialog with form & associated bugs list
--`
4.  **User Interaction**:
    - The user sees the dialog.
    - If there are other anomalies in the same group already linked to bugs,
      these bugs (ID and title) are listed.
    - The user enters a Bug ID into the input field.
    - The user clicks the "Submit" button.
5.  **Form Submission and Backend Communication**: `existing-bug-dialog-sk |
| -- (User Submits Form) --> | | | | -- _spinner.active = true --> (UI
Update: Show spinner) | | | -- fetch('/_/triage/associate_alerts', POST,
{bug_id, keys}) --> Backend API | | | <---- (Success/Failure) ---- | | |
| -- _spinner.active = false -> (UI Update: Hide spinner) | | | -- IF
Success: | | | -- closeDialog() ------> (UI Update: Hide dialog) | | | |
| -- window.open(bug_url) -> (Opens bug in new tab) | | | | | --
dispatchEvent('anomaly-changed') --> Application (Notifies other components)
| | | -- IF Failure: | | | -- errorMessage(msg) -> (UI Update: Show error
toast)`
6.  **Outcome**:
    - **Success**: The anomalies are linked to the specified bug in the
      backend. The dialog closes, the bug page opens in a new tab, and other
      parts of the UI (listening for `anomaly-changed`) update to reflect the
      new association.
    - **Failure**: An error message is shown, and the dialog remains open,
      allowing the user to try again or correct the input.

The design prioritizes a clear and focused user experience for a common task in
performance alert triaging. By integrating directly with the backend API for bug
association and fetching related bug information, it aims to be an efficient
tool for developers and SREs. The use of custom events allows for loose coupling
with other components in the larger application.

# Module: /modules/explore-multi-sk

## explore-multi-sk Module

### Overview

The `explore-multi-sk` module provides a user interface for displaying and
interacting with multiple performance data graphs simultaneously. This is
particularly useful when users need to compare different metrics,
configurations, or time ranges side-by-side. The core idea is to leverage the
functionality of individual `explore-simple-sk` elements, which represent single
graphs, and manage their states and interactions within a unified multi-graph
view.

### Key Design Decisions and Implementation Choices

**State Management:** A central `State` object within `explore-multi-sk` manages
properties that are common across all displayed graphs. These include the time
range (`begin`, `end`), display options (`showZero`, `dots`), and pagination
settings (`pageSize`, `pageOffset`). This approach simplifies the overall state
management and keeps the URL from becoming overly complex, as only a limited set
of shared parameters need to be reflected in the URL.

Each individual graph (`explore-simple-sk` instance) maintains its own specific
state related to the data it displays (formulas, queries, selected keys).
`explore-multi-sk` stores an array of `GraphConfig` objects, where each object
corresponds to an `explore-simple-sk` instance and holds its unique
configuration.

The `stateReflector` utility is used to synchronize the shared `State` with the
URL, allowing for bookmarking and sharing of multi-graph views.

**Dynamic Graph Addition and Removal:** Users can dynamically add new graphs to
the view. When a new graph is added, an empty `explore-simple-sk` instance is
created and the user can then configure its data source (query or formula).

If the `useTestPicker` option is enabled (often determined by backend defaults),
instead of a simple "Add Graph" button, a `test-picker-sk` element is displayed.
This component provides a more structured way to select tests and parameters,
and upon selection, a new graph is automatically generated and populated.

Graphs can also be removed. Event listeners are in place to handle
`remove-explore` custom events, which are typically dispatched by the individual
`explore-simple-sk` elements when a user closes them in a "Multiview" context
(where `useTestPicker` is active).

**Pagination:** To handle potentially large numbers of graphs, pagination is
implemented using the `pagination-sk` element. This allows users to view a
subset of the total graphs at a time, improving performance and usability. The
`pageSize` and `pageOffset` are part of the shared state.

**Graph Manipulation (Split and Merge):**

- **Split Graph:** If a single graph displaying multiple traces is present,
  the "Split Graph" functionality allows the user to create separate graphs
  for each of those traces. This is useful for focusing on individual trends
  that were previously combined.
- **Merge Graphs:** Conversely, the "Merge Graphs" functionality takes all
  currently displayed graphs and combines their traces into a single graph.
  This can be helpful for seeing an aggregated view.

These operations primarily involve manipulating the `graphConfigs` array and
then re-rendering the graphs.

**Shortcuts:** The module supports saving and loading multi-graph configurations
using shortcuts. When the configuration of graphs changes (traces added/removed,
graphs split/merged), `updateShortcutMultiview` is called. This function
communicates with a backend service (`/_/shortcut/get` and a corresponding save
endpoint invoked by `updateShortcut` from `explore-simple-sk`) to store or
retrieve the `graphConfigs` associated with a unique shortcut ID. This ID is
then reflected in the URL, allowing users to share specific multi-graph setups.

**Synchronization of Interactions:**

- **X-Axis Label:** When the x-axis label (e.g., switching between commit
  number and date) is toggled on one graph, a custom event `x-axis-toggled` is
  dispatched. `explore-multi-sk` listens for this and updates the x-axis on
  all other visible graphs to maintain consistency.
- **Chart Selection (Plot Summary):** While not explicitly detailed in
  `explore-multi-sk.ts`, the `explore-simple-sk` component likely has
  mechanisms for plot selection. If the `plotSummary` feature is active,
  selections on one graph might influence others, though the provided code for
  `explore-multi-sk` doesn't directly show this cross-graph selection
  synchronization logic, but it does have `syncChartSelection` which would
  handle this.

**Defaults and Configuration:** The component fetches default configurations
from a `/_/defaults/` endpoint. These defaults can influence various aspects,
such as: - Whether to use `test-picker-sk` (`useTestPicker`). - Default
parameters and their order for `test-picker-sk` (`include_params`,
`default_param_selections`). This allows for instance-specific customization of
the Perf UI.

### Responsibilities and Key Components

- **`explore-multi-sk.ts`**:

  - **Responsibilities**: This is the main TypeScript file defining the
    `ExploreMultiSk` custom element. It is responsible for:
  - Managing the overall state of the multi-graph view (shared properties
    like time range, pagination).
  - Handling the addition, removal, and configuration of individual
    `explore-simple-sk` graph elements.
  - Interacting with the `stateReflector` to update the URL based on the
    shared state.
  - Implementing the "Split Graph" and "Merge Graphs" functionalities.
  - Managing pagination for the displayed graphs.
  - Fetching and applying default configurations.
  - Coordinating interactions between graphs (e.g., synchronizing x-axis
    labels).
  - Interacting with the `test-picker-sk` if enabled.
  - Handling user authentication status for features like "Add to
    Favorites".
  - Managing shortcuts for saving and loading multi-graph configurations.
  - **Key Interactions**:
  - Creates and manages instances of `explore-simple-sk`.
  - Uses `pagination-sk` for displaying graphs in pages.
  - Uses `test-picker-sk` for adding graphs when `useTestPicker` is true.
  - Uses `favorites-dialog-sk` to allow users to save graph configurations.
  - Communicates with backend services for shortcuts and default
    configurations.

- **`explore-multi-sk.html` (Inferred from the Lit `html` template in
  `explore-multi-sk.ts`)**:

  - **Responsibilities**: Defines the structure of the `explore-multi-sk`
    element. This includes:
  - A menu section with buttons for "Add Graph", "Split Graph", "Merge
    Graphs", and "Add to Favorites".
  - The `test-picker-sk` element (conditionally visible).
  - `pagination-sk` elements for navigating through graph pages.
  - A container (`#graphContainer`) where the individual `explore-simple-sk`
    elements are dynamically rendered.
  - **Key Components**:
  - `<button>` elements for user actions.
  - `<test-picker-sk>` for test selection.
  - `<pagination-sk>` for graph pagination.
  - `<favorites-dialog-sk>` for saving favorites.
  - A `div` (`#graphContainer`) to hold the `explore-simple-sk` instances.

- **`explore-multi-sk.scss`**:

  - **Responsibilities**: Provides the styling for the `explore-multi-sk`
    element and its children. It ensures that the layout is appropriate for
    displaying multiple graphs and their controls.
  - **Key Aspects**:
  - Styles the `#menu` and `#pagination` areas.
  - Defines the height of embedded `explore-simple-sk` plots.
  - Handles the conditional visibility of elements like `#test-picker` and
    `#add-graph-button`.

### Key Workflows

**1. Initial Load and State Restoration:**

```
User navigates to URL with explore-multi-sk
    |
    V
explore-multi-sk.connectedCallback()
    |
    V
Fetch defaults from /_/defaults/
    |
    V
stateReflector() is initialized
    |
    V
State is read from URL (or defaults if URL is empty)
    |
    V
IF state.shortcut is present:
    Fetch graphConfigs from /_/shortcut/get using the shortcut ID
    |
    V
ELSE (or after fetching):
    For each graphConfig (or if starting fresh, often one empty graph is implied or added):
        Create/configure explore-simple-sk instance
        Set its state based on graphConfig and shared state
    |
    V
Add graphs to the current page based on pagination settings
    |
    V
Render the component
```

**2. Adding a Graph (without Test Picker):**

```
User clicks "Add Graph" button
    |
    V
explore-multi-sk.addEmptyGraph() is called
    |
    V
A new ExploreSimpleSk instance is created
A new empty GraphConfig is added to this.graphConfigs
    |
    V
explore-multi-sk.updatePageForNewExplore()
    |
    V
IF current page is full:
    Increment pageOffset (triggering pageChanged)
ELSE:
    Add new graph to current page
    |
    V
The new explore-simple-sk element might open its query dialog for the user
```

**3. Adding a Graph (with Test Picker):**

```
TestPickerSk is visible (due to defaults or state)
    |
    V
User interacts with TestPickerSk, selects tests/parameters
    |
    V
User clicks "Plot" button in TestPickerSk
    |
    V
TestPickerSk dispatches 'plot-button-clicked' event
    |
    V
explore-multi-sk listens for 'plot-button-clicked'
    |
    V
explore-multi-sk.addEmptyGraph(unshift=true) is called (new graph at the top)
    |
    V
explore-multi-sk.addGraphsToCurrentPage() updates the view
    |
    V
TestPickerSk.createQueryFromFieldData() gets the query
    |
    V
The new ExploreSimpleSk instance has its query set
```

**4. Splitting a Graph:**

```
User has one graph with multiple traces and clicks "Split Graph"
    |
    V
explore-multi-sk.splitGraph()
    |
    V
this.getTracesets() retrieves traces from the first (and only) graph
    |
    V
this.clearGraphs() removes the existing graph configuration
    |
    V
FOR EACH trace in the retrieved traceset:
    this.addEmptyGraph()
    A new GraphConfig is created for this trace (e.g., config.queries = [queryFromKey(trace)])
    |
    V
this.updateShortcutMultiview() (new shortcut reflecting multiple graphs)
    |
    V
this.state.pageOffset is reset to 0
    |
    V
this.addGraphsToCurrentPage() renders the new set of individual graphs
```

**5. Saving/Updating a Shortcut:**

```
Graph configuration changes (e.g., trace added/removed, graph split/merged, new graph added)
    |
    V
explore-multi-sk.updateShortcutMultiview() is called
    |
    V
Calls exploreSimpleSk.updateShortcut(this.graphConfigs)
    |
    V
(Inside updateShortcut)
IF graphConfigs is not empty:
    POST this.graphConfigs to backend (e.g., /_/shortcut/new or /_/shortcut/update)
    Backend returns a new or existing shortcut ID
    |
    V
explore-multi-sk.state.shortcut is updated with the new ID
    |
    V
this.stateHasChanged() is called, triggering stateReflector to update the URL
```

# Module: /modules/explore-simple-sk

The `explore-simple-sk` module provides a custom HTML element for exploring and
visualizing performance data. It allows users to query, plot, and analyze
traces, identify anomalies, and interact with commit details. This element is a
core component of the Perf application's data exploration interface.

**Core Functionality:**

The element's primary responsibility is to provide a user interface for:

1.  **Querying Data:** Users can construct queries to select specific traces
    based on various parameters.
2.  **Plotting Traces:** Selected traces are rendered on a graph, allowing for
    visual inspection of performance trends.
3.  **Analyzing Data:** Users can interact with the plot to zoom, pan, and
    select individual data points for detailed inspection.
4.  **Anomaly Detection:** The element integrates with anomaly detection
    services to highlight and manage performance regressions or improvements.
5.  **Commit Details:** Information about the commits associated with data
    points can be displayed, linking performance changes to specific code
    modifications.

**Key Design Decisions and Implementation Choices:**

- **State Management:** The element's state (e.g., current query, time range,
  plot settings) is managed internally and reflected in the URL. This allows
  users to share specific views of the data and enables bookmarking. The
  `State` class in `explore-simple-sk.ts` defines the structure of this state.
- **Data Fetching:** Data is fetched asynchronously from the backend using the
  `/frame/start` endpoint. The `requestFrame` method handles initiating these
  requests and processing the responses. The `FrameRequest` and
  `FrameResponse` types define the communication contract with the server.
- **Plotting Library:** The module supports two plotting libraries:
  `plot-simple-sk` (a custom canvas-based plotter) and `plot-google-chart-sk`
  (which wraps Google Charts). The choice of plotter can be configured.
- **Component-Based Architecture:** The UI is built using a collection of
  smaller, specialized custom elements (e.g., `query-sk` for query input,
  `paramset-sk` for displaying parameters, `commit-detail-panel-sk` for commit
  information). This promotes modularity and reusability.
- **Event-Driven Communication:** Components communicate with each other and
  with the main `explore-simple-sk` element through custom events. For
  example, when a query changes in `query-sk`, it emits a `query-change` event
  that `explore-simple-sk` listens to.
- **Caching and Optimization:** To improve performance, the element employs
  strategies like incremental data loading when panning and caching commit
  details.

**Key Files and Components:**

- **`explore-simple-sk.ts`:** This is the main TypeScript file that defines
  the `ExploreSimpleSk` custom element. It handles:
  - State management and URL reflection.
  - Data fetching and processing.
  - Rendering the UI template.
  - Event handling and coordination between child components.
  - Interaction logic for plotting, zooming, selecting points, etc.
- **`explore-simple-sk.html` (embedded in `explore-simple-sk.ts`):** This
  Lit-html template defines the structure of the element's UI. It includes
  placeholders for various child components and dynamic content.
- **`explore-simple-sk.scss`:** This SCSS file provides the styling for the
  element and its components.
- **Child Components (imported in `explore-simple-sk.ts`):**
  - `query-sk`: For constructing and managing queries.
  - `paramset-sk`: For displaying and interacting with parameter sets.
  - `plot-simple-sk` / `plot-google-chart-sk`: For rendering the plots.
  - `commit-detail-panel-sk`: For displaying commit information.
  - `anomaly-sk`: For displaying and managing anomalies.
  - Many other components for specific UI elements like dialogs, buttons,
    and icons.

**Workflow Example: Plotting a Query**

1.  **User Interaction:** The user interacts with the `query-sk` element to
    define a query.
2.  **Event Emission:** `query-sk` emits a `query-change` event with the new
    query.
3.  **State Update:** `explore-simple-sk` listens for this event, updates its
    internal state (specifically the `queries` array in the `State` object), and
    triggers a re-render.
4.  **Data Request:** `explore-simple-sk` constructs a `FrameRequest` based on
    the updated state and calls `requestFrame` to fetch data from the server.
    `User Input (query-sk) -> Event (query-change) -> State Update
(ExploreSimpleSk) -> Data Request (requestFrame)`
5.  **Data Processing:** Upon receiving the `FrameResponse`, `explore-simple-sk`
    processes the data, updates its internal `_dataframe` object, and prepares
    the data for plotting.
6.  **Plot Rendering:** `explore-simple-sk` passes the processed data to the
    `plot-simple-sk` or `plot-google-chart-sk` element, which then renders the
    traces on the graph. `Server Response (FrameResponse) -> Data Processing
(ExploreSimpleSk) -> Plot Update (plot-simple-sk/plot-google-chart-sk) ->
Visual Output`
7.  **URL Update:** The state change is reflected in the URL, allowing the user
    to bookmark or share the current view.

This workflow illustrates the reactive nature of the element, where user
interactions trigger state changes, which in turn lead to data fetching and UI
updates.

# Module: /modules/explore-sk

The `explore-sk` module serves as the primary user interface for exploring and
analyzing performance data within the Perf application. It provides a
comprehensive view for users to query, visualize, and interact with performance
traces.

The core functionality of `explore-sk` is built upon the `explore-simple-sk`
element. `explore-sk` acts as a wrapper, enhancing `explore-simple-sk` with
additional features like user authentication integration, default configuration
loading, and the optional `test-picker-sk` for more guided query construction.

**Key Responsibilities and Components:**

- **`explore-sk.ts`**: This is the main TypeScript file defining the
  `ExploreSk` custom element.

  - **Why**: It orchestrates the interaction between various sub-components
    and manages the overall state of the exploration page.
  - **How**:
  - It initializes by fetching default configurations (e.g., query
    parameters, display settings) from a backend endpoint (`/_/defaults/`).
    This ensures that the exploration view is pre-configured with sensible
    starting points.
  - It integrates with `alogin-sk` to determine the logged-in user's status.
    This information is used to enable features like "favorites" if a user
    is logged in.
  - It utilizes `stateReflector` to persist and restore the state of the
    underlying `explore-simple-sk` element in the URL. This allows users to
    share specific views or bookmark their current exploration state.
  - It conditionally initializes and displays `test-picker-sk`. If the
    `use_test_picker_query` flag is set in the state (often via URL
    parameters or defaults), the `test-picker-sk` component is shown,
    providing a structured way to build queries based on available parameter
    keys and values.
  - It listens for events from `test-picker-sk` (e.g.,
    `plot-button-clicked`, `remove-all`, `populate-query`) and translates
    these into actions on the `explore-simple-sk` element, such as adding
    new traces based on the selected test parameters or clearing the view.
  - It provides buttons like "View in multi-graph" and "Toggle Chart Style"
    which directly interact with methods exposed by `explore-simple-sk`.

- **`explore-simple-sk` (imported module)**: This is a fundamental building
  block that handles the core trace visualization, querying logic, and
  interaction with the graph.

  - **Why**: Encapsulates the complex logic of fetching trace data,
    rendering graphs, and handling user interactions like zooming, panning,
    and selecting traces.
  - **How**: `explore-sk` delegates most of the heavy lifting related to
    data exploration to this component. It passes down the initial state,
    default configurations, and user-specific settings.

- **`test-picker-sk` (imported module)**: A component that allows users to
  build queries by selecting from available test parameters and their values.

  - **Why**: Simplifies the query construction process, especially when
    dealing with a large number of possible parameters. It provides a more
    user-friendly alternative to manually typing complex query strings.
  - **How**: When active, it presents a UI for selecting dimensions and
    values. Upon user action (e.g., clicking a "plot" button), it emits an
    event with the constructed query, which `explore-sk` then uses to fetch
    and display the corresponding traces via `explore-simple-sk`. It can
    also be populated based on a highlighted trace, allowing users to
    quickly refine queries based on existing data.

- **`favorites-dialog-sk` (imported module)**: Enables users to save and
  manage their favorite query configurations.

  - **Why**: Provides a convenient way for users to quickly return to
    frequently used or important exploration views.
  - **How**: Integrated into `explore-simple-sk` and its functionality is
    enabled by `explore-sk` based on the user's login status.

- **State Management (`stateReflector`)**:

  - **Why**: To make the exploration state shareable and bookmarkable.
    Changes in the exploration view (queries, zoom levels, etc.) are
    reflected in the URL.
  - **How**: `explore-sk` uses `stateReflector` to listen for state changes
    in `explore-simple-sk`. When the state changes, `stateReflector` updates
    the URL. Conversely, when the page loads or the URL changes,
    `stateReflector` parses the URL and applies the state to
    `explore-simple-sk`.

**Workflow Example: Initial Page Load with Test Picker**

1.  `explore-sk` element is connected to the DOM.
2.  `connectedCallback` is invoked:
    - Renders its initial template.
    - Fetches default configurations from `/_/defaults/`.
    - `stateReflector` is initialized. If the URL contains state for
      `explore-simple-sk`, it's applied.
    - The state might indicate `use_test_picker_query = true`.
3.  If `use_test_picker_query` is true:
    - `initializeTestPicker()` is called.
    - `test-picker-sk` element is made visible.
    - `test-picker-sk` is initialized with parameters from the defaults (e.g.,
      `include_params`, `default_param_selections`) or from existing queries
      in the state.
4.  User interacts with `test-picker-sk` to select desired test parameters.
5.  User clicks the "Plot" button within `test-picker-sk`.
6.  `test-picker-sk` emits a `plot-button-clicked` event.
7.  `explore-sk` listens for this event:
    - It retrieves the query constructed by `test-picker-sk`.
    - It calls `exploreSimpleSk.addFromQueryOrFormula()` to add the new traces
      to the graph.
8.  `explore-simple-sk` fetches the data, renders the traces, and emits a
    `state_changed` event.
9.  `stateReflector` captures this `state_changed` event and updates the URL to
    reflect the new query.

This workflow illustrates how `explore-sk` acts as a central coordinator,
integrating various specialized components to provide a cohesive data
exploration experience. The design emphasizes modularity, with
`explore-simple-sk` handling the core plotting and `test-picker-sk` offering an
alternative query input mechanism, all managed and presented by `explore-sk`.

# Module: /modules/favorites-dialog-sk

The `favorites-dialog-sk` module provides a custom HTML element that displays a
modal dialog for users to add or edit "favorites." Favorites, in this context,
are likely user-defined shortcuts or bookmarks to specific views or states
within the application, identified by a name, description, and a URL.

**Core Functionality and Design:**

The primary purpose of this module is to present a user-friendly interface for
managing these favorites. It's designed as a modal dialog to ensure that the
user's focus is on the task of adding or editing a favorite without distractions
from the underlying page content.

**Key Components:**

- **`favorites-dialog-sk.ts`**: This is the heart of the module, defining the
  `FavoritesDialogSk` custom element.

  - **Why**: It encapsulates the logic for displaying the dialog, handling
    user input, and interacting with a backend service to persist favorite
    data.
  - **How**:
  - It extends `ElementSk`, a base class for custom elements in the Skia
    infrastructure, providing a common foundation.
  - It uses the Lit library (`lit/html.js`) for templating, allowing for
    declarative and efficient rendering of the dialog's UI.
  - The `open()` method is the public API for triggering the dialog. It
    accepts optional parameters for pre-filling the form when editing an
    existing favorite. Crucially, it returns a `Promise`. This promise-based
    approach is a key design choice. It resolves when the favorite is
    successfully saved and rejects if the user cancels the dialog. This
    allows the calling code (likely a parent component managing the list of
    favorites) to react appropriately, for instance, by re-fetching the
    updated list of favorites only when a change has actually occurred.
  - Input fields for "Name," "Description," and "URL" capture the necessary
    information. The "Name" and "URL" fields are mandatory.
  - The `confirm()` method handles the submission logic. It performs basic
    validation (checking for empty name and URL) and then makes an HTTP POST
    request to either `/_/favorites/new` or `/_/favorites/edit` depending on
    whether a new favorite is being created or an existing one is being
    modified.
  - A `spinner-sk` element is used to provide visual feedback to the user
    during the asynchronous operation of saving the favorite.
  - Error handling is implemented using `errorMessage` to display issues to
    the user, such as network errors or validation failures from the
    backend.
  - The `dismiss()` method handles the cancellation of the dialog, rejecting
    the promise returned by `open()`.
  - Input event handlers (`filterName`, `filterDescription`, `filterUrl`)
    update the component's internal state as the user types, and trigger
    re-renders via `this._render()`.

- **`favorites-dialog-sk.scss`**: This file contains the SASS styles for the
  dialog.

  - **Why**: It separates the presentation concerns from the JavaScript
    logic, making the component more maintainable.
  - **How**: It defines styles for the `<dialog>` element, input fields,
    labels, and buttons, ensuring a consistent look and feel within the
    application's theme (as indicated by `@import
'../themes/themes.scss';`).

- **`favorites-dialog-sk-demo.html` / `favorites-dialog-sk-demo.ts`**: These
  files provide a demonstration page for the `favorites-dialog-sk` element.

  - **Why**: This allows developers to see the component in isolation, test
    its functionality, and understand how to integrate it.
  - **How**: The HTML sets up a basic page with buttons to trigger the
    dialog in "new favorite" and "edit favorite" modes. The TypeScript file
    wires up event listeners on these buttons to call the `open()` method of
    the `favorites-dialog-sk` element with appropriate parameters.

**Workflow: Adding/Editing a Favorite**

A typical workflow involving this dialog would be:

1.  **User Action**: The user clicks a button (e.g., "Add Favorite" or an "Edit"
    icon next to an existing favorite) in the main application UI.
2.  **Dialog Invocation**: The event handler for this action calls the `open()`
    method of an instance of `favorites-dialog-sk`.

    - If adding a new favorite, `open()` might be called with minimal or no
      arguments, defaulting the URL to the current page.
    - If editing, `open()` is called with the `favId`, `name`, `description`,
      and `url` of the favorite to be edited.

    ```
    User clicks "Add New"  --> favoritesDialog.open('', '', '', 'current.page.url')
                                        |
                                        V
                                  Dialog Appears
                                        |
                                        V
    User fills form, clicks "Save" --> confirm() is called
                                        |
                                        V
                                 POST /_/favorites/new
                                        |
                                        V (Success)
                                  Dialog closes, open() Promise resolves
                                        |
                                        V
                             Calling component re-fetches favorites

    -------------------------------- OR ---------------------------------

    User clicks "Edit Favorite" --> favoritesDialog.open('id123', 'My Fav', 'Desc', 'fav.url.com')
                                        |
                                        V
                                  Dialog Appears (pre-filled)
                                        |
                                        V
    User modifies form, clicks "Save" --> confirm() is called
                                        |
                                        V
                                 POST /_/favorites/edit (with 'id123')
                                        |
                                        V (Success)
                                  Dialog closes, open() Promise resolves
                                        |
                                        V
                             Calling component re-fetches favorites

    -------------------------------- OR ---------------------------------

    User clicks "Cancel" or Close Icon --> dismiss() is called
                                        |
                                        V
                                  Dialog closes, open() Promise rejects
                                        |
                                        V
                             Calling component does nothing (no re-fetch)
    ```

3.  **User Interaction**: The user fills in or modifies the "Name,"
    "Description," and "URL" fields in the dialog.

4.  **Submission/Cancellation**:

    - **Save**: The user clicks the "Save" button.
      - The `confirm()` method is invoked.
      - Input validation (name and URL not empty) is performed.
      - A `fetch` request is made to the backend API (`/_/favorites/new` or
        `/_/favorites/edit`).
      - A spinner is shown during the API call.
      - Upon successful completion, the dialog closes, and the `Promise`
        returned by `open()` resolves.
      - If the API call fails, an error message is displayed, and the dialog
        remains open (or the promise might reject depending on specific
        error handling in `confirm`).
    - **Cancel**: The user clicks the "Cancel" button or the close icon.
      - The `dismiss()` method is invoked.
      - The dialog closes.
      - The `Promise` returned by `open()` rejects.

5.  **Post-Dialog Action**: The component that initiated the dialog (e.g., a
    `favorites-sk` list component) uses the resolved/rejected state of the
    `Promise` to decide whether to refresh its list of favorites. This is a key
    aspect of the design – it avoids unnecessary re-fetches if the user simply
    cancels the dialog.

The design prioritizes a clear separation of concerns, using custom elements for
UI encapsulation, SASS for styling, and a promise-based API for asynchronous
operations and communication with parent components. This makes the
`favorites-dialog-sk` a reusable and well-defined piece of UI for managing user
favorites.

# Module: /modules/favorites-sk

The `favorites-sk` module provides a user interface element for displaying and
managing a user's "favorites". Favorites are essentially bookmarked URLs,
categorized into sections. This module allows users to view their favorited
links, edit their details (name, description, URL), and delete them.

**Core Functionality & Design:**

The primary responsibility of `favorites-sk` is to fetch favorite data from a
backend endpoint (`/_/favorites/`) and render it in a user-friendly way. It also
handles interactions for modifying these favorites, such as editing and
deleting.

- **Data Fetching and Rendering:**

  - Upon connection to the DOM (`connectedCallback`), the element attempts
    to fetch the favorites configuration from the backend.
  - The fetched data, expected to be in a `Favorites` JSON format (defined
    in `perf/modules/json`), is stored in the `favoritesConfig` property.
  - The `_render()` method is called to update the display.
  - The rendering logic iterates through sections and then links within each
    section, generating an HTML table for display.
  - A key design choice is to distinguish "My Favorites" from other
    sections. "My Favorites" are displayed with "Edit" and "Delete" buttons,
    implying user ownership and modifiability. Other sections are presented
    as read-only.

- **Favorite Management:**

  - **Deletion:**
  - When a user clicks the "Delete" button for a favorite in the "My
    Favorites" section, the `deleteFavoriteConfirm` method is invoked.
  - This method displays a standard browser confirmation dialog
    (`window.confirm`) to prevent accidental deletions.
  - If confirmed, `deleteFavorite` sends a POST request to
    `/_/favorites/delete` with the ID of the favorite to be removed.
  - After a successful deletion, the favorites list is re-fetched to reflect
    the change.
  - **Editing:**
  - Clicking the "Edit" button calls the `editFavorite` method.
  - This method interacts with a `favorites-dialog-sk` element (defined in
    `perf/modules/favorites-dialog-sk`).
  - The `favorites-dialog-sk` is responsible for presenting a modal dialog
    where the user can modify the favorite's name, description, and URL.
  - Upon successful editing (dialog submission), the favorites list is
    re-fetched.

- **Error Handling:**

  - Network errors or non-OK responses during fetch operations (fetching
    favorites, deleting favorites) are caught.
  - An error message is displayed to the user via the `errorMessage` utility
    (from `elements-sk/modules/errorMessage`).

**Key Components/Files:**

- **`favorites-sk.ts`:** This is the heart of the module. It defines the
  `FavoritesSk` custom element, extending `ElementSk`. It contains the logic
  for fetching, rendering, deleting, and initiating the editing of favorites.
  - `constructor()`: Initializes the element with its Lit-html template.
  - `deleteFavorite()`: Handles the asynchronous request to the backend for
    deleting a favorite.
  - `deleteFavoriteConfirm()`: Provides a confirmation step before actual
    deletion.
  - `editFavorite()`: Manages the interaction with the `favorites-dialog-sk`
    for editing.
  - `template()`: The static Lit-html template function that defines the
    overall structure of the element.
  - `getSectionsTemplate()`: A helper function that dynamically generates
    the HTML for displaying sections and their links based on
    `favoritesConfig`. It specifically adds edit/delete controls for the "My
    Favorites" section.
  - `fetchFavorites()`: Fetches the favorites data from the backend and
    triggers a re-render.
  - `connectedCallback()`: A lifecycle method that ensures favorites are
    fetched when the element is added to the page.
- **`favorites-sk.scss`:** Provides the styling for the `favorites-sk`
  element, defining its layout, padding, colors for links, and table
  appearance.
- **`index.ts`:** A simple entry point that imports and registers the
  `favorites-sk` custom element, making it available for use in HTML.
- **`favorites-sk-demo.html` & `favorites-sk-demo.ts`:** These files provide a
  demonstration page for the `favorites-sk` element. The HTML includes an
  instance of `<favorites-sk>` and a `<pre>` tag to display events. The
  TypeScript file simply imports the element and sets up an event listener
  (though no custom events are explicitly dispatched by `favorites-sk` in the
  provided code).

**Workflow: Deleting a Favorite**

```
User Clicks "Delete" Button (for a link in "My Favorites")
    |
    V
favorites-sk.ts: deleteFavoriteConfirm(id, name)
    |
    V
window.confirm("Deleting favorite: [name]. Are you sure?")
    |
    +-- User clicks "Cancel" --> Workflow ends
    |
    V User clicks "OK"
favorites-sk.ts: deleteFavorite(id)
    |
    V
fetch('/_/favorites/delete', { method: 'POST', body: {id: favId} })
    |
    +-- Network Error/Non-OK Response --> errorMessage() is called, display error
    |
    V Successful Deletion
favorites-sk.ts: fetchFavorites()
    |
    V
fetch('/_/favorites/')
    |
    V
Parse JSON response, update this.favoritesConfig
    |
    V
this._render() // Re-renders the component with the updated list
```

**Workflow: Editing a Favorite**

```
User Clicks "Edit" Button (for a link in "My Favorites")
    |
    V
favorites-sk.ts: editFavorite(id, name, desc, url)
    |
    V
Get reference to <favorites-dialog-sk id="fav-dialog">
    |
    V
favorites-dialog-sk.open(id, name, desc, url) // Opens the edit dialog
    |
    +-- User cancels dialog --> Promise rejects (potentially with undefined, handled)
    |
    V User submits changes in dialog
Promise resolves
    |
    V
favorites-sk.ts: fetchFavorites() // Re-fetches and re-renders the list
    |
    V
fetch('/_/favorites/')
    |
    V
Parse JSON response, update this.favoritesConfig
    |
    V
this._render()
```

The design relies on Lit for templating and rendering, which provides efficient
updates to the DOM when the `favoritesConfig` data changes. The separation of
concerns is evident: `favorites-sk` handles the list display and top-level
actions, while `favorites-dialog-sk` manages the intricacies of the editing
form.

# Module: /modules/graph-title-sk

## Graph Title (`graph-title-sk`)

The `graph-title-sk` module provides a custom HTML element designed to display
titles for individual graphs in a structured and informative way. Its primary
goal is to present key-value pairs of metadata associated with a graph in a
visually clear and space-efficient manner.

### Responsibilities and Key Components

The core of this module is the `GraphTitleSk` custom element
(`graph-title-sk.ts`). Its main responsibilities are:

1.  **Data Reception and Storage:** It receives a `Map<string, string>` where
    keys represent parameter names (e.g., "bot", "benchmark") and values
    represent their corresponding values (e.g., "linux-perf", "Speedometer2").
    This map, along with the number of traces in the graph, is provided via the
    `set()` method.

2.  **Dynamic Rendering:** Based on the provided data, the element dynamically
    generates HTML to display the title. It iterates through the key-value pairs
    and renders them in a columnar layout. Each pair is displayed with the key
    (parameter name) in a smaller font above its corresponding value.

3.  **Handling Empty or Generic Titles:**

    - If a key or its corresponding value is an empty string, that particular
      entry is omitted from the displayed title. This ensures that the title
      remains concise and only shows relevant information.
    - If the input `titleEntries` map is empty but `numTraces` is greater than
      zero, it displays a generic title like "Multi-trace Graph (X traces)" to
      indicate a graph with multiple data series without specific shared
      parameters.

4.  **Space Management and Truncation:**

        - The title entries are arranged in a flexible, wrapping layout (`display:

    flex; flex-wrap: wrap;`) using CSS (`graph-title-sk.scss`). This allows
    the title to adapt to different screen widths.

    - To prevent overcrowding, especially when there are many parameters, the
      component implements a "show more" functionality. If the number of title
      entries exceeds a predefined limit (`MAX_PARAMS`, currently 8), it
      initially displays only the first `MAX_PARAMS` entries. A "Show Full
      Title" button (`<md-text-button class="showMore">`) is then provided,
      allowing the user to expand the view and see all title entries.
      Conversely, a "Show Short Title" mechanism is implied (though not
      explicitly shown as a button in the current code, `showShortTitles()` method exists) to revert to the truncated view.
    - Individual values that are very long are visually truncated in the
      display, but the full value is available as a tooltip when the user
      hovers over the text. This is achieved by setting the`title`attribute
      of the`div` containing the value.

### Design Decisions and Implementation Choices

- **Custom Element (`ElementSk`):** The component is built as a custom element
  extending `ElementSk`. This aligns with the Skia infrastructure's approach
  to building reusable UI components and allows for easy integration into Skia
  applications.
- **Lit Library for Templating:** The HTML structure is generated using the
  `lit` library's `html` template literal tag. This provides a declarative and
  efficient way to define the component's view and update it when data
  changes. The `_render()` method, inherited from `ElementSk`, is called to
  trigger re-rendering when the internal state (`_titleEntries`, `numTraces`,
  `showShortTitle`) changes.
- **CSS for Styling:** Styling is handled through a dedicated SCSS file
  (`graph-title-sk.scss`). This separates presentation concerns from the
  component's logic. CSS variables (e.g., `var(--primary)`) are used for
  theming, allowing the component's appearance to be consistent with the
  overall application theme.
- **`set()` Method for Data Input:** Instead of relying solely on HTML
  attributes for complex data like a map, a public `set()` method is provided.
  This is a common pattern for custom elements when dealing with non-string
  data or when updates need to trigger specific internal logic beyond simple
  attribute reflection.
- **Conditional Rendering for Title Brevity:** The decision to truncate the
  number of displayed parameters by default (when exceeding `MAX_PARAMS`) and
  provide a "Show Full Title" option is a user experience choice. It
  prioritizes a clean initial view for complex graphs while still allowing
  users to access all details if needed.

### Key Workflows

**1. Initial Rendering with Data:**

```
User/Application Code          GraphTitleSk Element
---------------------          --------------------
calls set(titleData, numTraces) -->
                                  stores titleData & numTraces
                                  calls _render()
                                  |
                                  V
                                  getTitleHtml() is invoked
                                  |
                                  V
                                  Iterates titleData:
                                    - Skips empty keys/values
                                    - If entries > MAX_PARAMS & showShortTitle is true:
                                      - Renders first MAX_PARAMS entries
                                      - Renders "Show Full Title" button
                                    - Else:
                                      - Renders all entries
                                  |
                                  V
                                  HTML template is updated with generated content
                                  Browser renders the title
```

**2. Toggling Full/Short Title Display (when applicable):**

```
User Interaction                  GraphTitleSk Element
----------------                  --------------------
Clicks "Show Full Title" button -->
                                  onClick handler (showFullTitle) executes
                                  |
                                  V
                                  this.showShortTitle = false
                                  calls _render()
                                  |
                                  V
                                  getTitleHtml() is invoked
                                  |
                                  V
                                  Now renders ALL title entries because showShortTitle is false
                                  |
                                  V
                                  HTML template is updated
                                  Browser re-renders the title to show all entries
```

A similar flow occurs if a mechanism to call `showShortTitles()` is implemented
and triggered.

The demo page (`graph-title-sk-demo.html` and `graph-title-sk-demo.ts`)
showcases various states of the `graph-title-sk` element, including:

- A "good" example with several valid entries.
- A "partial" example where some entries have empty keys or values.
- A "generic" example where an empty map is provided, resulting in the
  "Multi-trace Graph" title.
- An "empty" example (though the demo code doesn't explicitly create a state
  where `numTraces` is 0 and the map is also empty, which would result in no
  title being displayed).

# Module: /modules/ingest-file-links-sk

## Module: ingest-file-links-sk

**Overview:**

The `ingest-file-links-sk` module provides a custom HTML element,
`<ingest-file-links-sk>`, designed to display a list of relevant links
associated with a specific data point in the Perf performance monitoring system.
These links are retrieved from the `ingest.Format` data structure, which can be
generated by various ingestion processes. The primary purpose is to offer users
quick access to related resources, such as Swarming task runs, Perfetto traces,
or bot information, directly from the Perf UI.

**Why:**

Performance analysis often requires context beyond the raw data. Understanding
the environment in which a test ran (e.g., specific bot configuration), or
having direct access to detailed trace files, can be crucial for debugging
performance regressions or understanding improvements. This module centralizes
these relevant links in a consistent and easily accessible manner, improving the
efficiency of performance investigations.

**How:**

The `<ingest-file-links-sk>` element fetches link data asynchronously. When its
`load()` method is called with a `CommitNumber` (representing a specific point
in time or version) and a `traceID` (identifying the specific data series), it
makes a POST request to the `/_/details/?results=false` endpoint. This endpoint
is expected to return a JSON object conforming to the `ingest.Format` structure.

The element then parses this JSON response. It specifically looks for the
`links` field within the `ingest.Format`. If `links` exist and the `version`
field in the `ingest.Format` is present (indicating a modern format), the
element dynamically renders a list of these links.

Key design considerations and implementation details:

- **Asynchronous Loading:** Link fetching is an asynchronous operation to
  avoid blocking the UI. A `spinner-sk` element is displayed while data is
  being loaded.
- **URL vs. Text:** The module intelligently differentiates between actual
  URLs and plain text values within the `links` object. If a value is a valid
  URL, it's rendered as an `<a>` tag. Otherwise, it's displayed as "Key:
  Value".
- **Markdown Link Handling:** The element includes logic to parse and convert
  Markdown-style links (e.g., `[Link Text](url)`) into standard HTML anchor
  tags. This allows ingestion processes to provide links in a more
  human-readable format if desired.
- **Sorted Display:** Links are displayed in alphabetical order by their keys
  for consistent presentation.
- **Error Handling:** If the fetch request fails or the response is not in the
  expected format, an error message is displayed, and the spinner is hidden.
- **Legacy Format Compatibility:** The element checks for the `version` field
  in the response. If it's missing, it assumes a legacy data format that
  doesn't support these links and gracefully avoids displaying anything.

**Responsibilities and Key Components:**

- **`ingest-file-links-sk.ts`:** This is the core file defining the
  `IngestFileLinksSk` custom element.
  - It handles the fetching of link data from the backend API.
  - It manages the rendering of the link list based on the fetched data.
  - It includes the logic for differentiating between URLs and plain text,
    and for parsing Markdown links.
  - It manages the display of a loading spinner and error messages.
  - The `load(cid: CommitNumber, traceid: string)` method is the public API
    for triggering the data fetching and rendering process.
  - The `displayLinks` static method is responsible for generating the
    `TemplateResult` array for rendering the list items.
  - The `isUrl` and `removeMarkdown` helper functions provide utility for
    link processing.
- **`ingest-file-links-sk.scss`:** This file contains the SASS styles for the
  custom element, defining its appearance, including list styling and spinner
  positioning.
- **`ingest-file-links-sk-demo.html` and `ingest-file-links-sk-demo.ts`:**
  These files provide a demonstration page for the element. The demo page uses
  `fetch-mock` to simulate the backend API response, allowing developers to
  see the element in action and test its functionality in isolation.
- **`ingest-file-links-sk_test.ts`:** This file contains unit tests for the
  `IngestFileLinksSk` element. It uses `fetch-mock` to simulate various API
  responses and asserts the element's behavior, such as correct link
  rendering, spinner state, and error handling.
- **`ingest-file-links-sk_puppeteer_test.ts`:** This file contains
  Puppeteer-based end-to-end tests. These tests load the demo page in a
  headless browser and verify the element's visual rendering and basic
  functionality.

**Key Workflow: Loading and Displaying Links**

```
User Action/Page Load -> Calls ingest-file-links-sk.load(commit, traceID)
                             |
                             V
ingest-file-links-sk:  Show spinner-sk
                             |
                             V
                       Make POST request to /_/details/?results=false
                       (with commit and traceID in request body)
                             |
                             V
Backend API:          Processes request, retrieves links for the
                       given commit and trace
                             |
                             V
ingest-file-links-sk:  Receives JSON response (ingest.Format)
                             |
                             +----------------------+
                             |                      |
                             V                      V
                       Response OK?           Response Error?
                             |                      |
                             V                      V
                       Parse links            Display error message
                       Hide spinner             Hide spinner
                       Render link list
```

# Module: /modules/json

## JSON Module Documentation

This module defines TypeScript interfaces and types that represent the structure
of JSON data used throughout the Perf application. It essentially acts as a
contract between the Go backend and the TypeScript frontend, ensuring data
consistency and type safety.

**Why:**

The primary motivation for this module is to leverage TypeScript's strong typing
capabilities. By defining these interfaces, we can catch potential data
inconsistencies and errors at compile time rather than runtime. This is
particularly crucial for a data-intensive application like Perf, where the
frontend relies heavily on JSON responses from the backend.

Furthermore, these definitions are **automatically generated** from Go struct
definitions. This ensures that the frontend and backend data models remain
synchronized. Any changes to the Go structs will trigger an update to these
TypeScript interfaces, reducing the likelihood of manual errors and
inconsistencies.

**How:**

The `index.ts` file contains all the interface and type definitions. These are
organized into a flat structure for simplicity, with some nested namespaces
(e.g., `pivot`, `progress`, `ingest`) where logical grouping is beneficial.

A key design choice is the use of **nominal typing** for certain primitive types
(e.g., `CommitNumber`, `TimestampSeconds`, `Trace`). This is achieved by
creating type aliases that are branded with a unique string literal type. For
example:

```typescript
export type CommitNumber = number & {
  _commitNumberBrand: 'type alias for number';
};

export function CommitNumber(v: number): CommitNumber {
  return v as CommitNumber;
}
```

This prevents accidental assignment of a generic `number` to a `CommitNumber`
variable, even though they are structurally identical at runtime. This adds an
extra layer of type safety, ensuring that, for example, a timestamp is not
inadvertently used where a commit number is expected. Helper functions (e.g.,
`CommitNumber(v: number)`) are provided for convenient type assertion.

**Key Components/Files/Submodules:**

- **`index.ts`**: This is the sole file in this module and contains all the
  TypeScript interface and type definitions. It serves as the single source of
  truth for JSON data structures used in the frontend.
  - **Interfaces (e.g., `Alert`, `DataFrame`, `FrameRequest`,
    `Regression`)**: These define the shape of complex JSON objects. For
    instance, the `Alert` interface describes the structure of an alert
    configuration, including its query, owner, and various detection
    parameters. The `DataFrame` interface represents the core data structure
    for displaying traces, including the actual trace data (`traceset`),
    column headers (`header`), and associated parameter sets (`paramset`).
  - **Type Aliases (e.g., `ClusterAlgo`, `StepDetection`, `Status`)**: These
    define specific allowed string values for certain properties, acting
    like enums. For example, `ClusterAlgo` can only be `'kmeans'` or
    `'stepfit'`, ensuring that only valid clustering algorithms are
    specified.
  - **Nominally Typed Aliases (e.g., `CommitNumber`, `TimestampSeconds`,
    `Trace`, `ParamSet`)**: As explained above, these provide stronger type
    checking for primitive types that have specific semantic meaning within
    the application. `TraceSet`, for example, is a map where keys are trace
    identifiers (strings) and values are `Trace` arrays (nominally typed
    `number[]`).
  - **Namespaced Interfaces (e.g., `pivot.Request`, `ingest.Format`)**: Some
    interfaces are grouped under namespaces to organize related data
    structures. For example, `pivot.Request` defines the structure for
    requesting pivot table operations, including grouping criteria and
    aggregation operations. The `ingest.Format` interface defines the
    structure of data being ingested into Perf, including metadata like Git
    hash and the actual performance results.
  - **Utility/Generic Types (e.g., `ReadOnlyParamSet`, `AnomalyMap`)**:
    These represent common data patterns. `ReadOnlyParamSet` is a map of
    parameter names to arrays of their possible string values, marked as
    read-only to reflect its typical usage. `AnomalyMap` is a nested map
    structure used to associate anomalies with specific commits and traces.

**Workflow Example: Requesting and Displaying Trace Data**

A common workflow involves the frontend requesting trace data from the backend
and then displaying it.

1.  **Frontend (Client) prepares a `FrameRequest`:**

    ```
    Client Code --> Creates `FrameRequest` object:
                  {
                    begin: 1678886400, // Start timestamp
                    end: 1678972800,   // End timestamp
                    queries: ["config=gpu&name=my_test_trace"],
                    // ... other properties
                  }
    ```

2.  **Frontend sends the `FrameRequest` to the Backend (Server).**

3.  **Backend processes the request and generates a `FrameResponse`:**

    ```
    Server Logic --> Processes `FrameRequest`
                 --> Fetches data from database/cache
                 --> Constructs `FrameResponse` object:
                     {
                       dataframe: {
                         traceset: { "config=gpu&name=my_test_trace": [10.1, 10.5, 9.8, ...Trace] },
                         header: [ { offset: 12345, timestamp: 1678886400 }, ...ColumnHeader[] ],
                         paramset: { "config": ["gpu", "cpu"], "name": ["my_test_trace"] }
                       },
                       skps: [0, 5, 10], // Indices of significant points
                       // ... other properties like msg, display_mode, anomalymap
                     }
    ```

4.  **Backend sends the `FrameResponse` (as JSON) back to the Frontend.**

5.  **Frontend receives the JSON and parses it, expecting it to conform to the
    `FrameResponse` interface:** ``Client Code --> Receives JSON --> Parses JSON
into a `FrameResponse` typed object --> Uses
`frameResponse.dataframe.traceset` to render charts --> Uses
`frameResponse.dataframe.header` to display commit information``

This typed interaction ensures that if the backend, for example, renamed
`traceset` to `trace_data` in its Go struct, the automatic generation would
update the `DataFrame` interface. The TypeScript compiler would then flag an
error in the frontend code trying to access `frameResponse.dataframe.traceset`,
preventing a runtime error and guiding the developer to update the frontend code
accordingly.

# Module: /modules/json-source-sk

The `json-source-sk` module provides a custom HTML element, `<json-source-sk>`,
designed to display the raw JSON data associated with a specific data point in a
trace. This is particularly useful in performance analysis and debugging
scenarios where understanding the exact input data ingested by the system is
crucial.

The core responsibility of this module is to fetch and present JSON data in a
user-friendly dialog. It aims to simplify the process of inspecting the source
data for a given commit and trace identifier.

The key component is the `JSONSourceSk` class, defined in `json-source-sk.ts`.
This class extends `ElementSk`, a base class for custom elements in the Skia
infrastructure.

**How it Works:**

1.  **Initialization and Properties:**

    - The element requires two primary properties to be set:
      - `cid`: The Commit ID (represented as `CommitNumber`), which
        identifies a specific version or point in time.
      - `traceid`: A string identifier for the specific trace being
        examined.
    - When these properties are set, the element renders itself. If `traceid`
      is not a valid key (checked by `validKey` from
      `perf/modules/paramtools`), the control buttons are hidden.

2.  **User Interaction and Data Fetching:**

    - The element displays two buttons: "View Json File" and "View Short Json
      File".
    - Clicking either button triggers the `_loadSource` or `_loadSourceSmall`
      methods, respectively.
    - These methods internally call `_loadSourceImpl`. This implementation
      detail allows for sharing the core fetching logic while differentiating
      the request URL.
    - `_loadSourceImpl` constructs a `CommitDetailsRequest` object containing
      the `cid` and `traceid`.
    - It then makes a POST request to the `/_/details/` endpoint.
      - If "View Short Json File" was clicked (`isSmall` is true), the URL
        includes `?results=false`, indicating to the backend that a
        potentially truncated or summarized version of the JSON is
        requested.
      - A `spinner-sk` element is activated to provide visual feedback
        during the fetch operation.
    - The response from the server is parsed as JSON using `jsonOrThrow`. If
      the request is successful, the JSON data is formatted with indentation
      and stored in the `_json` private property.
    - The element is then re-rendered to display the fetched JSON.
    - If an error occurs during fetching or parsing, `errorMessage` (from
      `perf/modules/errorMessage`) is used to display an error notification to
      the user.

3.  **Displaying the JSON:**

    - The fetched JSON data is displayed within a `<dialog>` element
      (`#json-dialog`).
    - The `jsonFile()` method in the template is responsible for rendering the
      `<pre>` tag containing the formatted JSON string, but only if `_json` is
      not empty.
    - The dialog is shown using `showModal()`, providing a modal interface for
      viewing the JSON.
    - A close button (`#closeIcon` with a `close-icon-sk`) allows the user to
      dismiss the dialog. Closing the dialog also clears the `_json` property.

**Design Rationale:**

- **Dedicated Element:** Creating a dedicated custom element encapsulates the
  functionality of fetching and displaying JSON, making it reusable across
  different parts of the application where such inspection is needed.
- **Asynchronous Fetching:** The use of `async/await` and `fetch` allows for
  non-blocking data retrieval, ensuring the UI remains responsive while
  waiting for the server.
- **Error Handling:** Incorporating error handling via `jsonOrThrow` and
  `errorMessage` provides a better user experience by informing users about
  issues during data retrieval.
- **Clear Visual Feedback:** The `spinner-sk` element clearly indicates when
  data is being loaded.
- **Modal Dialog:** Using a modal dialog (`<dialog>`) for displaying the JSON
  helps focus the user's attention on the data without cluttering the main
  interface.
- **Option for Short JSON:** The "View Short Json File" option caters to
  scenarios where the full JSON might be excessively large, providing a way to
  quickly inspect a summary or a smaller subset of the data. This can improve
  performance and readability for very large JSON files.
- **Styling and Theming:** The SCSS file (`json-source-sk.scss`) provides
  basic styling and leverages existing button styles
  (`//elements-sk/modules/styles:buttons_sass_lib`). It also includes
  considerations for dark mode by using CSS variables like `--on-background`
  and `--background`.

**Workflow Example: Viewing JSON Source**

```
User Sets Properties       Element Renders         User Clicks Button      Fetches Data             Displays JSON
--------------------       ---------------         ------------------      ------------             -------------
[json-source-sk          -> [Buttons visible]  ->  ["View Json File"] -> POST /_/details/  ->   <dialog>
  .cid = 123                                                                {cid, traceid}      <pre>{json}</pre>
  .traceid = ",foo=bar,"]                                                                  </dialog>
                                                                        (spinner active)
                                                                              |
                                                                              V
                                                                        Response Received
                                                                        (spinner inactive)
```

The demo page (`json-source-sk-demo.html` and `json-source-sk-demo.ts`)
illustrates how to use the `<json-source-sk>` element. It sets up mock data
using `fetchMock` to simulate the backend endpoint and programmatically clicks
the button to demonstrate the JSON loading functionality.

The Puppeteer test (`json-source-sk_puppeteer_test.ts`) ensures the element
renders correctly and performs basic visual regression testing.

# Module: /modules/new-bug-dialog-sk

The `new-bug-dialog-sk` module provides a user interface element for filing new
bugs related to performance anomalies. It aims to streamline the bug reporting
process by pre-filling relevant information and integrating with the Buganizer
issue tracker.

**Core Functionality:**

The primary responsibility of this module is to display a dialog that allows
users to input details for a new bug. This dialog is populated with information
derived from one or more selected `Anomaly` objects. The user can then review
and modify this information before submitting the bug.

**Key Design Decisions and Implementation Choices:**

- **Pre-population of Bug Details:** To reduce manual effort and ensure
  consistency, the dialog attempts to intelligently pre-fill fields like the
  bug title, labels, and components.
  - The bug title is generated based on the nature (regression/improvement),
    magnitude (percentage change), and affected revision range of the
    anomalies. This logic, found in `getBugTitle()`, mimics the behavior of
    the legacy Chromeperf UI to maintain familiarity for users.
  - Labels and components are aggregated from all selected anomalies. Unique
    labels are presented as checkboxes (defaulting to checked), and unique
    components are presented as radio buttons (with the first one selected
    by default). This is handled by `getLabelCheckboxes()` and
    `getComponentRadios()`.
- **Dynamic UI Generation:** The dialog's content, specifically the label
  checkboxes and component radio buttons, is dynamically generated based on
  the provided `Anomaly` data. This ensures that only relevant options are
  presented to the user. Lit-html's templating capabilities are used for this
  dynamic rendering.
- **User Contextualization:** The dialog attempts to automatically CC the
  logged-in user on the new bug. This is achieved by fetching the user's login
  status via `/alogin-sk`.
- **Asynchronous Bug Filing:** The actual bug filing process is asynchronous.
  When the user submits the form, a POST request is made to the
  `/_/triage/file_bug` endpoint.
  - A spinner (`spinner-sk`) is displayed during this operation to provide
    visual feedback.
  - Upon successful bug creation, the user is redirected to the newly
    created bug page in a new tab, and an `anomaly-changed` event is
    dispatched to notify other components (like `explore-simple-sk` or
    `chart-tooltip-sk`) that the anomalies have been updated with the new
    bug ID.
  - If an error occurs, an error message is displayed using
    `error-toast-sk`, and the dialog remains open, allowing the user to
    retry or correct information.
- **Standard HTML Dialog Element:** The core dialog functionality leverages
  the native `<dialog>` HTML element, which provides built-in accessibility
  and modal behavior.

**Workflow: Filing a New Bug**

1.  **Initialization:** An external component (e.g., a chart displaying
    anomalies) invokes the `setAnomalies()` method on `new-bug-dialog-sk`,
    passing the relevant `Anomaly` objects and associated trace names.
2.  **Opening the Dialog:** The external component calls the `open()` method.
    `User Action (e.g., click "File Bug" button) | V External Component
--[setAnomalies(anomalies, traceNames)]--> new-bug-dialog-sk | V External
Component --[open()]--> new-bug-dialog-sk`
3.  **Dialog Population:** - `new-bug-dialog-sk` fetches the current user's login status to pre-fill
    the CC field. - The `_render()` method is called, which uses the Lit-html template. - `getBugTitle()` generates a suggested title. - `getLabelCheckboxes()` and `getComponentRadios()` create the UI for
    selecting labels and components based on the input anomalies. - The dialog (`<dialog id="new-bug-dialog">`) is displayed modally.
    ``new-bug-dialog-sk.open() | V [Fetch Login Status] --> Updates `_user`
| V _render() |--> getBugTitle() --> Populates Title Input |-->
getLabelCheckboxes() --> Creates Label Checkboxes |-->
getComponentRadios() --> Creates Component Radios | V Dialog is
displayed to the user``
4.  **User Interaction:** The user reviews and potentially modifies the
    pre-filled information (title, description, labels, component, assignee,
    CCs).
5.  **Submission:** The user clicks the "Submit" button. `User clicks "Submit" |
V Form Submit Event | V new-bug-dialog-sk.fileNewBug()`
6.  **Bug Filing Process:** - The `fileNewBug()` method is invoked. - The spinner is activated, and form buttons are disabled. - Form data (title, description, selected labels, selected component,
    assignee, CCs, anomaly keys, trace names) is collected. - A POST request is sent to `/_/triage/file_bug` with the collected data.
    `fileNewBug() | V [Activate Spinner, Disable Buttons] | V [Extract Form
Data] | V fetch('/_/triage/file_bug', {POST, body: jsonData})`
7.  **Response Handling:** - **Success:** - The server responds with a JSON object containing the `bug_id`. - The spinner is deactivated, and buttons are re-enabled. - The dialog is closed. - A new browser tab is opened to the URL of the created bug (e.g.,
    `https://issues.chromium.org/issues/BUG_ID`). - The `bug_id` is updated in the local `_anomalies` array. - An `anomaly-changed` custom event is dispatched with the updated
    anomalies and bug ID. - **Failure:** - The server responds with an error. - The spinner is deactivated, and buttons are re-enabled. - An error message is displayed to the user via `errorMessage()`. The
    dialog remains open. `fetch Response | +-- Success (HTTP 200, valid
JSON with bug_id) | | | V | [Deactivate Spinner, Enable Buttons] | |
| V | closeDialog() | | | V | window.open(bugUrl, '_blank') | | | V
| Update local _anomalies with bug_id | | | V |
dispatchEvent('anomaly-changed', {anomalies, bugId}) | +-- Failure
(HTTP error or invalid JSON) | V [Deactivate Spinner, Enable
Buttons] | V errorMessage(errorMsg) --> Displays error toast`

**Key Files:**

- **`new-bug-dialog-sk.ts`**: This is the core file containing the
  `NewBugDialogSk` class definition, which extends `ElementSk`. It includes
  the Lit-html template for the dialog, the logic for populating form fields
  based on `Anomaly` data, handling form submission, interacting with the
  backend API to file the bug, and managing the dialog's visibility and state.
- **`new-bug-dialog-sk.scss`**: This file defines the styles for the dialog,
  ensuring it integrates visually with the rest of the application and themes.
  It styles the dialog container, input fields, buttons, and the close icon.
- **`new-bug-dialog-sk-demo.ts` and `new-bug-dialog-sk-demo.html`**: These
  files provide a demonstration page for the `new-bug-dialog-sk` element. The
  `.ts` file sets up mock data (`Anomaly` objects) and mock fetch responses to
  simulate the bug filing process, allowing for isolated testing and
  development of the dialog. The `.html` file includes the `new-bug-dialog-sk`
  element and a button to trigger its opening.
- **`index.ts`**: This file simply imports `new-bug-dialog-sk.ts` to ensure
  the custom element is defined and available for use.

The module relies on several other elements and libraries:

- `alogin-sk`: To determine the logged-in user for CC'ing.
- `close-icon-sk`: For the dialog's close button.
- `spinner-sk`: To indicate activity during bug filing.
- `error-toast-sk` (via `errorMessage` utility): To display error messages.
- `lit`: For templating and component rendering.
- `jsonOrThrow`: A utility for parsing JSON responses and throwing errors on
  failure.

# Module: /modules/paramtools

The `paramtools` module provides a TypeScript implementation of utility
functions for manipulating parameter sets and structured keys. It mirrors the
functionality found in the Go module `/infra/go/paramtools`, which is the
primary source of truth for these operations. The decision to replicate this
logic in TypeScript is to enable client-side applications to perform these
common tasks without needing to make server requests for simple transformations
or validations. This approach improves performance and reduces server load for
UI-driven interactions.

The core responsibility of this module is to provide robust and consistent ways
to:

1.  **Create and parse structured keys:** Structured keys are a fundamental
    concept for identifying specific data points (e.g., traces in performance
    data).
2.  **Manipulate `ParamSet` objects:** `ParamSet`s are used to represent
    collections of possible parameter values, often used for filtering or
    querying data.

Key functionalities and their "why" and "how":

- **`makeKey(params: Params | { [key: string]: string }): string`**:

  - **Why**: To create a canonical string representation of a set of
    key-value parameters. This canonical form is essential for consistent
    identification and comparison of data points. The keys within the
    structured key are sorted alphabetically to ensure that the same set of
    parameters always produces the same key, regardless of the order in
    which they were provided.
  - **How**: It takes a `Params` object (a dictionary of string key-value
    pairs). It first checks if the `params` object is empty, throwing an
    error if it is, as a key must represent at least one parameter. Then, it
    sorts the keys of the `params` object alphabetically. Finally, it
    constructs the string by joining each key-value pair with `=` and then
    joining these pairs with `,`, prefixing and suffixing the entire string
    with a comma. `Input: { "b": "2", "a": "1", "c": "3" } | V Sort keys: [
"a", "b", "c" ] | V Format pairs: "a=1", "b=2", "c=3" | V Join and wrap:
",a=1,b=2,c=3,"`

- **`fromKey(structuredKey: string, attribute?: string): Params`**:

  - **Why**: To convert a structured key string back into a `Params` object,
    making it easier to work with the individual parameters
    programmatically. It also handles the removal of special functions that
    might be embedded in the key (e.g., `norm(...)` for normalization).
  - **How**: It first calls `removeSpecialFunctions` to strip any function
    wrappers from the key. Then, it splits the key string by the comma
    delimiter. Each resulting segment (if not empty) is then split by the
    equals sign to separate the key and value. These key-value pairs are
    collected into a new `Params` object. An optional `attribute` parameter
    allows excluding a specific key from the resulting `Params` object,
    which can be useful in scenarios where certain attributes are metadata
    and not part of the core parameters.

- **`removeSpecialFunctions(key: string): string`**:

  - **Why**: Structured keys can sometimes include functional wrappers
    (e.g., `norm(...)`, `avg(...)`) or special markers (e.g.,
    `special_zero`). This function is designed to strip these away,
    returning the "raw" underlying key. This is important when you need to
    work with the base parameters without the context of the applied
    function or special condition.
  - **How**: It uses regular expressions to detect if the key matches a
    pattern like `function_name(,param1=value1,...)`. If a match is found,
    it extracts the content within the parentheses. The extracted string (or
    the original key if no function was found) is then processed by
    `extractNonKeyValuePairsInKey`.
  - **`extractNonKeyValuePairsInKey(key: string): string`**: This helper
    function further refines the key string. It splits the string by commas
    and filters out any segments that do not represent a valid `key=value`
    pair. This helps to remove extraneous parts like `special_zero` that
    might be comma-separated but aren't true parameters. The valid pairs are
    then re-joined and wrapped with commas.

- **`validKey(key: string): boolean`**:

  - **Why**: To provide a simple client-side check to determine if a string
    is a "valid" basic structured key, meaning it's not a key representing a
    calculation (like `avg(...)`) or other special trace types. This is a
    lightweight validation, as the server performs more comprehensive
    checks.
  - **How**: It checks if the key string starts and ends with a comma. This
    is a convention for simple, non-functional structured keys.

- **`addParamsToParamSet(ps: ParamSet, p: Params): void`**:

  - **Why**: To add a new set of parameters (from a `Params` object) to an
    existing `ParamSet`. `ParamSet`s store unique values for each parameter
    key. This function ensures that when new parameters are added, only new
    values are appended to the existing lists for each key, maintaining
    uniqueness.
  - **How**: It iterates through the key-value pairs of the input `Params`
    object (`p`). For each key, it retrieves the corresponding array of
    values from the `ParamSet` (`ps`). If the key doesn't exist in `ps`, a
    new array is created. If the value from `p` is not already present in
    the array, it's added.

- **`paramsToParamSet(p: Params): ParamSet`**:

  - **Why**: To convert a single `Params` object (representing one specific
    combination of parameters) into a `ParamSet`. In a `ParamSet`, each key
    maps to an array of values, even if there's only one value.
  - **How**: It creates a new, empty `ParamSet`. Then, for each key-value
    pair in the input `Params` object, it creates a new entry in the
    `ParamSet` where the key maps to an array containing just that single
    value.

- **`addParamSet(p: ParamSet, ps: ParamSet | ReadOnlyParamSet): void`**:

  - **Why**: To merge one `ParamSet` (or `ReadOnlyParamSet`) into another.
    This is useful for combining sets of available parameter options, for
    example, when aggregating data from multiple sources.
  - **How**: It iterates through the keys and their associated value arrays
    in the source `ParamSet` (`ps`). If a key from `ps` is not present in
    the target `ParamSet` (`p`), the entire key and its value array (cloned)
    are added to `p`. If the key already exists in `p`, it iterates through
    the values in the source array and adds any values that are not already
    present in the target array for that key.

- **`toReadOnlyParamSet(ps: ParamSet): ReadOnlyParamSet`**:

  - **Why**: To provide a type assertion that casts a mutable `ParamSet` to
    an immutable `ReadOnlyParamSet`. This is useful for signaling that a
    `ParamSet` should not be modified further, typically when passing it to
    components or functions that expect read-only data.
  - **How**: It performs a type assertion. No actual data transformation
    occurs; it's a compile-time type hint.

- **`queryFromKey(key: string): string`**:

  - **Why**: To convert a structured key into a URL query string format
    (e.g., `a=1&b=2&c=3`). This is specifically useful for frontend
    applications, like `explore-simple-sk`, where state or filters are often
    represented in the URL.
  - **How**: It first uses `fromKey` to parse the structured key into a
    `Params` object. Then, it leverages the `URLSearchParams` browser API to
    construct a query string from these parameters. This ensures proper URL
    encoding of keys and values. `Input Key: ",a=1,b=2,c=3," | V fromKey ->
Params: { "a": "1", "b": "2", "c": "3" } | V URLSearchParams -> Query
String: "a=1&b=2&c=3"`

The design choice to have these functions operate with less stringent validation
than their server-side Go counterparts is deliberate. The server remains the
ultimate authority on data validity. These client-side functions prioritize ease
of use and performance for UI interactions, assuming that the data they operate
on has either originated from or will eventually be validated by the server.

The `index_test.ts` file provides comprehensive unit tests for these functions,
ensuring their correctness and robustness across various scenarios, including
handling empty inputs, duplicate values, and special key formats. This focus on
testing is crucial for maintaining the reliability of these foundational utility
functions.

# Module: /modules/perf-scaffold-sk

The `perf-scaffold-sk` module provides a consistent layout and navigation
structure for all pages within the Perf application. It acts as a wrapper,
ensuring that common elements like the title bar, navigation sidebar, and error
notifications are present and behave uniformly across different sections of
Perf.

**Core Responsibilities:**

- **Layout Management:** Establishes the primary visual structure, dividing
  the page into a header, a collapsible sidebar for navigation, and a main
  content area.
- **Navigation:** Provides a standardized set of navigation links in the
  sidebar, allowing users to easily access different Perf features (e.g., New
  Query, Favorites, Alerts).
- **Global Elements:** Hosts globally relevant components like the login
  status (`alogin-sk`), theme chooser (`theme-chooser-sk`), and error/toast
  notifications (`error-toast-sk`).
- **Dynamic Content Injection:** Handles the placement of page-specific
  content into the `main` content area and allows for specific content (like
  help text) to be injected into the sidebar.

**Key Components and Design Decisions:**

- **`perf-scaffold-sk.ts`:** This is the heart of the module, defining the
  `PerfScaffoldSk` custom element.

  - **Why:** Encapsulating the scaffold logic within a custom element
    promotes reusability and modularity. It allows any Perf page to adopt
    the standard layout simply by including this element.
  - **How:** It uses Lit for templating and rendering the structure
    (`<app-sk>`, `header`, `aside#sidebar`, `main`, `footer`).
  - **Content Redistribution:** A crucial design choice is how it handles
    child elements. Since it doesn't use Shadow DOM for the main content
    area (to allow global styles to apply easily to the page content), it
    programmatically moves children of `<perf-scaffold-sk>` into the
    `<main>` section.

  - **Process:**

    1.  When `connectedCallback` is invoked, existing children of
        `<perf-scaffold-sk>` are temporarily moved out.
    2.  The scaffold's own template (header, sidebar, etc.) is rendered.
    3.  The temporarily moved children are then appended to the newly
        rendered `<main>` element.
    4.  A `MutationObserver` is set up to watch for any new children added
        to `<perf-scaffold-sk>` and similarly move them to `<main>`.

  - **Sidebar Content:** An exception is made for elements with the specific
    ID `SIDEBAR_HELP_ID`. These are moved into the `#help` div within the
    sidebar. This allows pages to provide context-specific help information
    directly within the scaffold.

    ```
    <perf-scaffold-sk>
        <!-- This will go into <main> -->
        <div>Page specific content</div>

        <!-- This will go into <aside>#help -->
        <div id="sidebar_help">Contextual help</div>
    </perf-scaffold-sk>
    ```

  - **Configuration via `window.perf`:** The scaffold reads various
    configuration options from the global `window.perf` object. This allows
    instances of Perf to customize links (help, feedback, chat), behavior
    (e.g., `show_triage_link`), and display information (e.g., instance URL,
    build tag). This makes the scaffold adaptable to different Perf
    deployments.

  - For example, the `_helpUrl` and `_reportBugUrl` are initialized with
    defaults but can be overridden by `window.perf.help_url_override` and
    `window.perf.feedback_url` respectively.

  - The visibility of the "Triage" link is controlled by
    `window.perf.show_triage_link`.

  - **Build Information:** It displays the current application build tag,
    fetching it via `getBuildTag()` from
    `//perf/modules/window:window_ts_lib` and linking it to the
    corresponding commit in the buildbot git repository.

  - **Instance Title:** It can display the name of the Perf instance,
    extracted from `window.perf.instance_url`.

- **`perf-scaffold-sk.scss`:** Defines the styles for the scaffold.

  - **Why:** Separates styling concerns from the element's logic.
  - **How:** It uses SASS and imports common themes from
    `//perf/modules/themes:themes_sass_lib`. It defines the layout,
    including the sidebar width and the main content area's width (using
    `calc(99vw - var(--sidebar-width))` to avoid horizontal scrollbars
    caused by `100vw` including the scrollbar width). It also styles the
    navigation links and other elements within the scaffold.

- **`perf-scaffold-sk-demo.html` & `perf-scaffold-sk-demo.ts`:** Provide a
  demonstration page for the scaffold.

  - **Why:** Allows developers to see the scaffold in action and test its
    appearance and behavior in isolation.
  - **How:** `perf-scaffold-sk-demo.ts` initializes a mock `window.perf`
    object with various settings and then injects an instance of
    `<perf-scaffold-sk>` with some placeholder content (including a `div`
    with `id="sidebar_help"`) into the `perf-scaffold-sk-demo.html` page.

**Workflow: Initializing and Rendering a Page with the Scaffold**

1.  A Perf page (e.g., the "New Query" page) includes `<perf-scaffold-sk>` as
    its top-level layout element. `html <!-- new_query_page.html --> <body>
<perf-scaffold-sk> <!-- Content specific to the New Query page -->
<query-composer-sk></query-composer-sk> <div id="sidebar_help"> <p>Tips for
creating new queries...</p> </div> </perf-scaffold-sk> </body>`
2.  The `PerfScaffoldSk` element's `connectedCallback` fires.
3.  `perf-scaffold-sk.ts`: - Temporarily moves `<query-composer-sk>` and `<div
id="sidebar_help">...</div>` out of `perf-scaffold-sk`. - Renders its own internal template (header with title, login, theme
    chooser; sidebar with nav links; empty main area; footer with error
    toast). `<app-sk> <header>...</header> <aside id=sidebar> <div
      id=links>...</div> <div id=help></div> <-- Placeholder for sidebar help
      ... </aside> <main></main> <-- Placeholder for main content
      <footer>...</footer> </app-sk>` - The `redistributeAddedNodes` function is called: - `<query-composer-sk>` (since it doesn't have `id="sidebar_help"`) is
    appended to the `<main>` element. - `<div id="sidebar_help">...</div>` is appended to the `<div
id="help">` element within the `<aside>` sidebar. - A `MutationObserver` starts listening for any further children added
    directly to `<perf-scaffold-sk>`.

The final rendered structure (simplified) would look something like:

```
perf-scaffold-sk
  └── app-sk
      ├── header
      │   ├── h1.name (Instance Title)
      │   ├── div.spacer
      │   ├── alogin-sk
      │   └── theme-chooser-sk
      ├── aside#sidebar
      │   ├── div#links
      │   │   ├── a (New Query)
      │   │   ├── a (Favorites)
      │   │   └── ... (other nav links)
      │   ├── div#help
      │   │   └── div#sidebar_help (Content from original page)
      │   │       └── <p>Tips for creating new queries...</p>
      │   └── div#chat
      ├── main
      │   └── query-composer-sk (Content from original page)
      └── footer
          └── error-toast-sk
```

# Module: /modules/picker-field-sk

The `picker-field-sk` module provides a custom HTML element that serves as a
stylized text input field with an associated dropdown menu for selecting from a
predefined list of options. This component is designed to offer a user-friendly
way to pick a single value from potentially many choices, enhancing the user
experience in forms or selection-heavy interfaces.

**Core Functionality and Design:**

The primary goal of `picker-field-sk` is to present a familiar text input that,
upon interaction (focus or click), reveals a filterable list of valid options.
This addresses the need for a compact and efficient way to select an item,
especially when the number of options is large.

The implementation leverages the Vaadin ComboBox component (`@vaadin/combo-box`)
for its underlying dropdown and filtering capabilities. This choice was made to
utilize a well-tested and feature-rich component, avoiding the need to
reimplement complex dropdown logic, keyboard navigation, and accessibility
features. `picker-field-sk` then wraps this Vaadin component, applying custom
styling and providing a simplified API tailored to its specific use case.

**Key Responsibilities and Components:**

- **`picker-field-sk.ts`**: This is the heart of the module, defining the
  `PickerFieldSk` custom element which extends `ElementSk`.

  - **Properties:**
  - `label`: A string that serves as both the visual label above the input
    field and the placeholder text within it when empty. This provides
    context to the user about the expected input.
  - `options`: An array of strings representing the valid choices the user
    can select from. The component dynamically adjusts the width of the
    dropdown overlay to accommodate the longest option, ensuring
    readability.
  - `helperText`: An optional string displayed below the input field,
    typically used for providing additional guidance or information to the
    user.
  - **Events:**
  - `value-changed`: This custom event is dispatched whenever the selected
    value in the combo box changes. This includes selecting an item from the
    dropdown, typing a value that matches an option (due to `autoselect`),
    or clearing the input. The new value is available in
    `event.detail.value`. This event is crucial for parent components to
    react to user selections.
  - **Methods:**
  - `focus()`: Programmatically sets focus to the input field.
  - `openOverlay()`: Programmatically opens the dropdown list of options.
    This is useful for guiding the user or for integrating with other UI
    elements.
  - `disable()`: Makes the input field read-only, preventing user
    interaction.
  - `enable()`: Removes the read-only state, allowing user interaction.
  - `clear()`: Clears the current value in the input field.
  - `setValue(val: string)`: Programmatically sets the value of the input
    field.
  - `getValue()`: Retrieves the current value of the input field.
  - **Rendering:** Uses `lit-html` for templating. The template renders a
    `<vaadin-combo-box>` element and binds its properties and events to the
    `PickerFieldSk` element's state.
  - **Overlay Width Calculation:** The `calculateOverlayWidth()` private
    method dynamically adjusts the `--vaadin-combo-box-overlay-width` CSS
    custom property. It iterates through the `options` to find the longest
    string and sets the overlay width to be slightly larger than this
    string, ensuring all options are fully visible without truncation. This
    is a key usability enhancement. `User provides options -->
PickerFieldSk.options setter | V calculateOverlayWidth() | V Find max
option length | V Set --vaadin-combo-box-overlay-width CSS property`

- **`picker-field-sk.scss`**: Contains the SASS styles for the component.

  - It primarily targets the underlying `vaadin-combo-box` and its shadow
    parts (e.g., `::part(label)`, `::part(input-field)`, `::part(items)`) to
    customize its appearance to match the application's theme (including
    dark mode support).
  - CSS custom properties like `--vaadin-field-default-width`,
    `--vaadin-combo-box-overlay-width`, and `--lumo-text-field-size` are
    used to control the dimensions and sizing of the Vaadin component.
  - Dark mode styles are applied by targeting `.darkmode picker-field-sk`,
    adjusting colors for labels, helper text, and input fields to ensure
    proper contrast and visual integration.

- **`index.ts`**: A simple entry point that imports and thereby registers the
  `picker-field-sk` custom element, making it available for use in HTML.

- **`picker-field-sk-demo.html` & `picker-field-sk-demo.ts`**: These files
  create a demonstration page for the `picker-field-sk` component.

  - `picker-field-sk-demo.html` includes instances of the `picker-field-sk`
    element and buttons to trigger its various functionalities (focus, fill,
    open overlay, disable/enable).
  - `picker-field-sk-demo.ts` contains JavaScript to initialize the demo
    elements with sample data (a large list of "speedometer" options to
    showcase performance with many items) and to wire up the buttons to the
    corresponding methods of the `PickerFieldSk` instances. This allows
    developers to visually inspect and interact with the component.

**Workflow Example: User Selects an Option**

1.  **Initialization**: A parent component instantiates `<picker-field-sk>` and
    sets its `label` and `options` properties. `<picker-field-sk .label="Fruit"
.options=${['Apple', 'Banana', 'Cherry']}></picker-field-sk>`
2.  **User Interaction**: The user clicks on or focuses the `picker-field-sk`
    input. `User clicks/focuses input --> vaadin-combo-box internally handles
focus/click | V vaadin-combo-box displays dropdown with options`
3.  **Filtering (Optional)**: The user types into the input field. The
    `vaadin-combo-box` filters the displayed options based on the typed text.
4.  **Selection**: The user clicks an option from the dropdown or presses Enter
    when an option is highlighted. `User selects "Banana" --> vaadin-combo-box
updates its internal value | V vaadin-combo-box emits 'value-changed' event`
5.  **Event Propagation**: - The `vaadin-combo-box` within `picker-field-sk` emits its native
    `value-changed` event. - The `onValueChanged` method in `PickerFieldSk` catches this event. - `PickerFieldSk` then dispatches its own `value-changed` custom event,
    with the selected value in `event.detail.value`.
    `picker-field-sk.onValueChanged(vaadinEvent) | V Dispatch new
CustomEvent('value-changed', { detail: { value: vaadinEvent.detail.value
}})`
6.  **Parent Component Reaction**: The parent component, listening for the
    `value-changed` event on the `<picker-field-sk>` element, receives the event
    and can act upon the selected value. `Parent component listens for
'value-changed' --> Accesses event.detail.value | V Update application
state`

This layered approach, building upon the Vaadin ComboBox, provides a robust and
themeable selection component while abstracting away the complexities of the
underlying library for the consumers of `picker-field-sk`.

# Module: /modules/pinpoint-try-job-dialog-sk

## Pinpoint Try Job Dialog (`pinpoint-try-job-dialog-sk`)

The `pinpoint-try-job-dialog-sk` module provides a user interface element for
initiating Pinpoint A/B try jobs.

**Purpose:**

The primary reason for this module's existence within the Perf application is to
allow users to request additional trace data for specific benchmark runs. While
Pinpoint itself supports a wider range of try job use cases, this dialog is
specifically tailored for this trace generation scenario. It's important to note
that this component is considered a legacy feature, and future development
should favor the newer Pinpoint frontend.

**How it Works:**

The dialog is designed to gather the necessary parameters from the user to
construct and submit a Pinpoint A/B try job request. This process involves:

1.  **Initialization:** The dialog can be pre-populated with initial values such
    as the test path, base commit, and end commit. This often happens when a
    user interacts with a chart tooltip and wants to investigate a specific data
    point further.
2.  **User Input:** The user can modify the pre-filled values or enter new ones.
    Key inputs include:
    - **Base Commit:** The starting commit hash for the A/B comparison.
    - **Experiment Commit:** The ending commit hash for the A/B comparison.
    - **Tracing Arguments:** A string specifying the categories and options
      for the trace generation. A default value is provided, and a link to
      Chromium source documentation offers more details on available options.
3.  **Authentication:** The dialog uses `alogin-sk` to determine the logged-in
    user. The user's email is included in the try job request.
4.  **Submission:** Upon submission, the dialog constructs a
    `CreateLegacyTryRequest` object. This object encapsulates all the necessary
    information for the Pinpoint backend.
    - The `testPath` (e.g., `master/benchmark_name/story_name`) is parsed to
      extract the configuration (e.g., `benchmark_name`) and the benchmark
      (e.g., `story_name`).
    - The `story` is typically the last segment of the `testPath`.
    - The `extra_test_args` field is formatted to include the user-provided
      tracing arguments.
5.  **API Interaction:** The dialog sends a POST request to the `/_/try/`
    endpoint with the JSON payload.
6.  **Response Handling:**
    - **Success:** If the request is successful, the Pinpoint backend responds
      with a JSON object containing the `jobUrl` for the newly created
      Pinpoint job. This URL is then displayed to the user, allowing them to
      navigate to the Pinpoint UI to monitor the job's progress.
    - **Error:** If an error occurs during the request or processing, an error
      message is displayed to the user.

**Workflow:**

```
User Interaction (e.g., click on chart tooltip)
    |
    V
Dialog Pre-populated with context (testPath, commits)
    |
    V
pinpoint-try-job-dialog-sk.open()
    |
    V
User reviews/modifies input fields (Base Commit, Exp. Commit, Trace Args)
    |
    V
User clicks "Send to Pinpoint"
    |
    V
[pinpoint-try-job-dialog-sk]
  - Gathers input values
  - Retrieves logged-in user via alogin-sk
  - Constructs `CreateLegacyTryRequest` JSON
  - Sends POST request to /_/try/
    |
    V
[Backend Pinpoint Service]
  - Processes the request
  - Creates A/B try job
  - Returns jobUrl (success) or error
    |
    V
[pinpoint-try-job-dialog-sk]
  - Displays spinner during request
  - On Success:
    - Displays link to the created Pinpoint job (jobUrl)
    - Hides spinner
  - On Error:
    - Displays error message
    - Hides spinner
```

**Key Components/Files:**

- **`pinpoint-try-job-dialog-sk.ts`:** This is the core TypeScript file that
  defines the custom element's logic.
  - **`PinpointTryJobDialogSk` class:** Extends `ElementSk` and manages the
    dialog's state, user input, and interaction with the Pinpoint API.
  - **`template`:** Defines the HTML structure of the dialog using
    `lit-html`. This includes input fields for commits and tracing
    arguments, a submit button, a spinner for loading states, and a link to
    the created Pinpoint job.
  - **`connectedCallback()`:** Initializes the dialog, sets up event
    listeners (e.g., for form submission, closing the dialog on outside
    click), and fetches the logged-in user's information.
  - **`setTryJobInputParams(params: TryJobPreloadParams)`:** Allows external
    components to pre-fill the dialog's input fields. This is crucial for
    integrating the dialog with other parts of the Perf UI, like chart
    tooltips.
  - **`open()`:** Displays the modal dialog.
  - **`closeDialog()`:** Closes the modal dialog.
  - **`postTryJob()`:** This is the central method for handling the job
    submission. It reads values from the input fields, constructs the
    `CreateLegacyTryRequest` payload, and makes the `fetch` call to the
    Pinpoint API. It also handles the UI updates based on the API response
    (showing the job URL or an error message).
  - **`TryJobPreloadParams` interface:** Defines the structure for the
    parameters used to pre-populate the dialog.
- **`pinpoint-try-job-dialog-sk.scss`:** Contains the SASS/CSS styles for the
  dialog, ensuring it aligns with the application's visual theme. It styles
  the input fields, buttons, and the overall layout of the dialog.
- **`index.ts`:** A simple entry point that imports and registers the
  `pinpoint-try-job-dialog-sk` custom element.
- **`BUILD.bazel`:** Defines the build rules for the module, specifying its
  dependencies (e.g., `elements-sk` components like `select-sk`, `spinner-sk`,
  `alogin-sk`, and Material Web components) and how it should be compiled.

**Design Decisions:**

- **Based on `bisect-dialog-sk`:** The dialog's structure and initial
  functionality were adapted from an existing bisect dialog. This likely
  accelerated development by reusing common patterns for dialog interactions
  and API calls.
- **Legacy Component:** The explicit note to avoid building on top of this
  dialog indicates a strategic decision to migrate towards a newer Pinpoint
  frontend. This suggests that this component is maintained for existing
  functionality but is not the target for future enhancements related to
  Pinpoint interactions.
- **Specific Use Case:** The dialog is narrowly focused on requesting
  additional traces. This simplifies the UI and the request payload, making it
  easier for users to achieve this specific task.
- **Client-Side Request Construction:** The `CreateLegacyTryRequest` object is
  fully constructed on the client-side before being sent to the backend. This
  gives the frontend more control over the request parameters.
- **Standard HTML Dialog:** The use of the `<dialog>` HTML element provides
  built-in modal behavior, simplifying the implementation of showing and
  hiding the dialog.
- **Error Handling:** The dialog includes basic error handling by displaying
  messages returned from the API, improving the user experience when things go
  wrong.
- **Spinner for Feedback:** The `spinner-sk` component provides visual
  feedback to the user while the API request is in progress.

This component serves as a bridge for users of the Perf application to leverage
Pinpoint's capabilities for generating detailed trace information, even as the
broader Pinpoint tooling evolves.

# Module: /modules/pivot-query-sk

The `pivot-query-sk` module provides a custom HTML element for users to
configure and interact with pivot table requests. Pivot tables are a powerful
data summarization tool, and this element allows users to define how data should
be grouped, what aggregate operations should be performed, and what summary
statistics should be displayed.

The core of the module is the `PivotQuerySk` class, which extends `ElementSk`.
This class manages the state of the pivot request and renders the UI for user
interaction. It leverages other custom elements like `multi-select-sk` and
`select-sk` to provide intuitive input controls.

**Key Design Choices and Implementation Details:**

- **Event-Driven Updates:** The element emits a custom event, `pivot-changed`,
  whenever the user modifies any part of the pivot request. This allows
  consuming applications to react to changes in real-time. The event detail
  (`PivotQueryChangedEventDetail`) contains the updated `pivot.Request` object
  or `null` if the current configuration is invalid. This decouples the UI
  component from the application logic that processes the pivot request.
- **Data Binding and Rendering:** The `PivotQuerySk` element uses Lit's `html`
  templating for rendering. It maintains internal state for the
  `_pivotRequest` (the current pivot configuration) and `_paramset` (the
  available options for grouping). When these properties are set or updated,
  the `_render()` method is called to re-render the component, ensuring the UI
  reflects the current state.
- **Handling Null Pivot Requests:** The `createDefaultPivotRequestIfNull()`
  method ensures that if `_pivotRequest` is initially `null`, it's initialized
  with a default valid structure before any user interaction attempts to
  modify it. This prevents errors and provides a sensible starting point.
- **Dynamic Option Generation:** The options for "group by" and "summary" are
  dynamically generated based on the provided `_paramset` and the existing
  `_pivotRequest`. The `allGroupByOptions()` method is particularly noteworthy
  as it ensures that even if the `_paramset` changes, any currently selected
  `group_by` keys in the `_pivotRequest` are still displayed as options. This
  prevents accidental data loss during `_paramset` updates. It achieves this
  by concatenating keys from both sources, sorting, and then filtering out
  duplicates.
- **Input Validation:** The `pivotRequest` getter includes a call to
  `validatePivotRequest` (from `pivotutil`). This ensures that the component
  only returns a valid `pivot.Request` object. If the current configuration is
  invalid, it returns `null`. This promotes data integrity.

**Responsibilities and Key Components:**

- **`pivot-query-sk.ts`**: This is the main file defining the `PivotQuerySk`
  custom element.

  - **`PivotQuerySk` class**:
  - Manages the `pivot.Request` object, which defines the grouping,
    operation, and summary statistics for a pivot table.
  - Takes a `ParamSet` as input, which provides the available keys for the
    "group by" selection. This `ParamSet` likely originates from the dataset
    being analyzed.
  - Renders UI controls (multi-selects and a select dropdown) for users to
    specify:
    - **Group By Keys**: Which parameters to use for grouping data rows
      (e.g., 'config', 'os'). This uses `multi-select-sk`.
    - **Operation**: The primary aggregate function to apply (e.g., 'avg',
      'sum', 'count'). This uses a standard `select` element.
    - **Summary Statistics**: Optional additional aggregate functions to
      calculate for each group (e.g., 'stddev', 'percentile'). This also
      uses `multi-select-sk`.
  - Emits a `pivot-changed` custom event when the user modifies the pivot
    request.
  - **`PivotQueryChangedEventDetail` type**: Defines the structure of the
    data passed in the `pivot-changed` event.
  - **`PivotQueryChangedEventName` constant**: The string name of the custom
    event.
  - **Event Handlers (`groupByChanged`, `operationChanged`,
    `summaryChanged`)**: These methods are triggered by user interactions
    with the respective UI elements. They update the internal
    `_pivotRequest` and then call `emitChangeEvent`.
  - **`emitChangeEvent()`**: Constructs and dispatches the `pivot-changed`
    event.
  - **Property Getters/Setters (`pivotRequest`, `paramset`)**: Provide
    controlled access to the element's core data, triggering re-renders when
    set.

- **`pivot-query-sk.scss`**: Contains the styling for the `pivot-query-sk`
  element. It ensures a consistent look and feel, leveraging styles from
  `themes_sass_lib` and `select_sass_lib`. The layout is primarily flex-based
  to arrange the different selection components.

- **`pivot-query-sk-demo.html` and `pivot-query-sk-demo.ts`**: These files
  provide a demonstration page for the `pivot-query-sk` element.

  - The HTML sets up a basic page structure and includes an instance of
    `pivot-query-sk`.
  - The TypeScript initializes the demo element with sample `pivot.Request`
    data and a `ParamSet`. It also includes an event listener for
    `pivot-changed` to display the selected pivot configuration as JSON,
    illustrating how to consume the element's output.

**Workflow for User Interaction and Event Emission:**

1.  **Initialization:**

    - The `pivot-query-sk` element is created.
    - The consuming application sets the `paramset` (available grouping keys)
      and optionally an initial `pivotRequest`.
    - The element renders its initial state based on these inputs.

2.  **User Modifies a Selection (e.g., changes a "group by" option):**

    - `multi-select-sk` (for "group by") emits a `selection-changed` event.
    - `PivotQuerySk.groupByChanged()` is called.
    - `createDefaultPivotRequestIfNull()` ensures `_pivotRequest` is not null.
    - `_pivotRequest.group_by` is updated based on the new selection.
    - `emitChangeEvent()` is called.

3.  **Event Emission:**

    - `emitChangeEvent()`:
      - Retrieves the current `pivotRequest` (which might be `null` if
        invalid).
      - Creates a new `CustomEvent` named `pivot-changed`.
      - The `detail` of the event is the current (potentially validated)
        `pivotRequest`.
      - The event is dispatched, bubbling up the DOM.

4.  **Application Responds:**

    - The consuming application, listening for `pivot-changed` events on the
      `pivot-query-sk` element or one of its ancestors, receives the event.
    - The application can then use the `event.detail` (the `pivot.Request`) to
      update its data display, fetch new data, or perform other actions.

This flow can be visualized as:

```
User Interaction (e.g., click on multi-select)
      |
      v
Internal element event (e.g., @selection-changed from multi-select-sk)
      |
      v
PivotQuerySk Event Handler (e.g., groupByChanged)
      |
      v
Update internal _pivotRequest state
      |
      v
PivotQuerySk.emitChangeEvent()
      |
      v
Dispatch "pivot-changed" CustomEvent (with pivot.Request as detail)
      |
      v
Consuming Application's Event Listener
      |
      v
Application processes the new pivot.Request
```

# Module: /modules/pivot-table-sk

The `pivot-table-sk` module provides a custom HTML element, `<pivot-table-sk>`,
designed to display pivoted data in a tabular format. This element is
specifically for DataFrames that have been pivoted and contain summary values,
as opposed to summary traces (which would be displayed in a plot).

**Core Functionality and Design**

The primary purpose of `pivot-table-sk` is to present complex, multi-dimensional
data in an understandable and interactive table. The "why" behind its design is
to offer a user-friendly way to explore summarized data that arises from
pivoting operations.

Key design considerations include:

- **Data Input:** It takes a `DataFrame` (from
  `//perf/modules/json:index_ts_lib`) and a `pivot.Request` (also from
  `//perf/modules/json:index_ts_lib`) as input. The `pivot.Request` is crucial
  as it dictates how the `DataFrame` was originally pivoted, including the
  `group_by` keys, the main `operation`, and the `summary` operations.
- **Display:** The data is rendered as an HTML table. The table headers are
  derived from the `group_by` keys and the `summary` operations.
- **Interactivity (Sorting):** Users can sort the table by clicking on column
  headers. The sorting mechanism is designed to be intuitive, mimicking
  spreadsheet behavior where subsequent sorts on different columns break ties
  from previous sorts.
- **Query Context:** The element also displays the query parameters, the
  "group by" keys, the primary operation, and the summary operations that led
  to the current view of the data. This provides context to the user.
- **Validation:** It includes a mechanism to validate if the provided
  `pivot.Request` is suitable for display as a pivot table (using
  `validateAsPivotTable` from `//perf/modules/pivotutil:index_ts_lib`). This
  prevents rendering errors or confusing displays if the input data structure
  isn't appropriate.

**Key Components and Files**

- **`pivot-table-sk.ts`**: This is the heart of the module, defining the
  `PivotTableSk` custom element.

  - **`PivotTableSk` class:**
  - Extends `ElementSk` (from `//infra-sk/modules/ElementSk:index_ts_lib`).
  - Manages the input `DataFrame` (`df`), `pivot.Request` (`req`), and the
    original `query` string.
  - **`KeyValues` type and `keyValuesFromTraceSet` function:** This is a
    critical internal data structure. `KeyValues` is an object where keys
    are trace keys (e.g., `',arch=x86,config=8888,'`) and values are arrays
    of strings. These string arrays represent the values of the parameters
    specified in `req.group_by`, in the same order. For example, if
    `req.group_by` is `['config', 'arch']`, then for the trace
    `',arch=arm,config=8888,'`, the corresponding `KeyValues` entry would be
    `['8888', 'arm']`. This transformation is performed by
    `keyValuesFromTraceSet` and is essential for rendering the "key" columns
    of the table and for sorting by these keys.
  - **`SortSelection` class:** Represents the sorting state of a single
    column. It stores:
    - `column`: The index of the column.
    - `kind`: Whether the column represents 'keyValues' (from `group_by`)
      or 'summaryValues' (from `summary` operations).
    - `dir`: The sort direction ('up' or 'down').
    - It provides methods to `toggleDirection`, `buildCompare` (to create
      a JavaScript sort comparison function based on its state), and
      `encode`/`decode` for serialization.
  - **`SortHistory` class:** Manages the overall sorting state of the table.
    - It holds an array (`history`) of `SortSelection` objects.
    - The "spreadsheet-like" multi-column sorting is achieved here. When a
      user clicks a column to sort, that column's `SortSelection` is moved
      to the _front_ of the `history` array, and its direction is toggled.
    - `buildCompare` in `SortHistory` creates a composite comparison
      function that iterates through the `SortSelection` objects in
      `history`. The first `SortSelection` determines the primary sort
      order. If it results in a tie, the second `SortSelection` is used to
      break the tie, and so on. This creates the effect of a stable sort
      across multiple user interactions without needing a true stable sort
      algorithm for each click.
    - It also provides `encode`/`decode` methods to serialize the entire
      sort history (e.g., for persisting sort state in a URL).
  - **`set()` method:** The primary way to provide data to the component. It
    initializes `keyValues`, `sortHistory`, and the main `compare` function.
    It can also accept an `encodedHistory` string to restore a previous sort
    state.
  - **Rendering Logic (Templates):** Uses `lit-html` for templating.
    - `queryDefinition()`: Renders the contextual information about the
      query and pivot operations.
    - `tableHeader()`, `keyColumnHeaders()`, `summaryColumnHeaders()`:
      Generate the table header row, including sort icons.
    - `sortArrow()`: Dynamically displays the correct sort icon (up arrow,
      down arrow, or neutral sort icon) based on the current
      `SortHistory`.
    - `tableRows()`, `keyRowValues()`, `summaryRowValues()`: Generate the
      data rows of the table, applying the current sort order.
    - `displayValue()`: Formats numerical values for display, converting a
      special sentinel value (`MISSING_DATA_SENTINEL` from
      `//perf/modules/const:const_ts_lib`) to '-'.
  - **Event Emission:** Emits a `change` event when the user sorts the
    table. The event detail (`PivotTableSkChangeEventDetail`) is the encoded
    `SortHistory` string. This allows parent components to react to sort
    changes and potentially persist the state.
  - **Dependencies:**
  - Relies on `paramset-sk` to display the query parameters.
  - Uses various icon elements (`arrow-drop-down-icon-sk`,
    `arrow-drop-up-icon-sk`, `sort-icon-sk`) for the sort indicators.
  - `//perf/modules/json:index_ts_lib` for `DataFrame`, `TraceSet`,
    `pivot.Request` types.
  - `//perf/modules/pivotutil:index_ts_lib` for `operationDescriptions` and
    `validateAsPivotTable`.
  - `//perf/modules/paramtools:index_ts_lib` for `fromKey` (to parse trace
    keys into parameter sets).
  - `//infra-sk/modules:query_ts_lib` for `toParamSet` (to convert a query
    string into a `ParamSet`).

- **`pivot-table-sk.scss`**: Provides the styling for the `pivot-table-sk`
  element, including table borders, padding, text alignment, and cursor styles
  for interactive elements. It leverages themes from
  `//perf/modules/themes:themes_sass_lib`.

- **`index.ts`**: A simple entry point that imports and thereby registers the
  `pivot-table-sk` custom element.

- **`pivot-table-sk-demo.html` & `pivot-table-sk-demo.ts`**:

  - These files set up a demonstration page for the `pivot-table-sk`
    element.
  - `pivot-table-sk-demo.ts` creates sample `DataFrame` and `pivot.Request`
    objects and uses them to populate instances of `pivot-table-sk` on the
    demo page. This is crucial for development and visual testing. It
    demonstrates valid use cases, cases with invalid pivot requests, and
    cases with null DataFrames to ensure the component handles these
    scenarios gracefully.

- **Test Files (`pivot-table-sk_test.ts`,
  `pivot-table-sk_puppeteer_test.ts`)**:

  - **`pivot-table-sk_test.ts` (Karma test):** Contains unit tests for the
    `PivotTableSk` element and its internal logic, particularly the
    `SortSelection` and `SortHistory` classes. It verifies:
  - Correct initialization and rendering.
  - The sorting behavior when column headers are clicked (e.g., sort
    direction changes, correct sort icons appear, `change` event is emitted
    with the correct encoded history).
  - The `buildCompare` functions in `SortSelection` and `SortHistory`
    produce the correct sorting results for various data types and sort
    directions.
  - The `encode` and `decode` methods for `SortSelection` and `SortHistory`
    work correctly, allowing for round-tripping of sort state.
  - The `keyValuesFromTraceSet` function correctly transforms `TraceSet`
    data based on the `pivot.Request`.
  - **`pivot-table-sk_puppeteer_test.ts` (Puppeteer test):** Performs
    end-to-end tests by loading the demo page in a headless browser.
  - It checks if the elements render correctly on the page (smoke test).
  - It takes screenshots of the rendered component for visual regression
    testing.

**Workflow Example: User Sorting the Table**

1.  **Initial State:**

    - The `pivot-table-sk` element is initialized with a `DataFrame`, a
      `pivot.Request`, and an optional initial `encodedHistory` string.
    - `pivot-table-sk` creates a `SortHistory` object. If `encodedHistory` is
      provided, `SortHistory.decode()` is called. Otherwise, a default sort
      order is established (usually based on the order of summary columns,
      then key columns, all initially 'up').
    - `SortHistory.buildCompare()` generates the initial comparison function.
    - The table is rendered, sorted according to this initial comparison
      function. Each column header shows a default `sort-icon-sk`.

2.  **User Clicks a Column Header (e.g., "config" key column):**

        - `changeSort(columnIndex, 'keyValues')` is called within
          `pivot-table-sk`.
        - `this.sortHistory.selectColumnToSortOn(columnIndex, 'keyValues')` is
          invoked:
          - The `SortSelection` for the "config" column is found in
            `this.sortHistory.history`.
          - It's removed from its current position.
          - Its `direction` is toggled (e.g., from 'up' to 'down').
          - This updated `SortSelection` is prepended to
            `this.sortHistory.history`. `Before: [SummaryCol0(up),

    SummaryCol1(up), KeyCol0(config, up), KeyCol1(arch, up)] Click on
    KeyCol0 (config): After: [KeyCol0(config, down), SummaryCol0(up),
    SummaryCol1(up), KeyCol1(arch, up)]`    -`this.compare = this.sortHistory.buildCompare(...)`is called. A new
    composite comparison function is generated. Now, rows will primarily be
    sorted by "config" (descending). Ties will be broken by "SummaryCol0"
    (ascending), then "SummaryCol1" (ascending), and finally "KeyCol1"
    (ascending).

    - A`CustomEvent('change')`is dispatched. The`event.detail`contains
      `this.sortHistory.encode()`, which is a string representation of the new
      sort order (e.g., "dk0-su0-su1-ku1").
    - `this.\_render()`is called, re-rendering the table with the new sort
      order. The "config" column header now shows an
      `arrow-drop-down-icon-sk`.

3.  **User Clicks Another Column Header (e.g., "avg" summary column):**

        - The process repeats. The `SortSelection` for the "avg" column is moved
          to the front of `this.sortHistory.history` and its direction is toggled.
          `Before: [KeyCol0(config, down), SummaryCol0(avg, up), SummaryCol1(sum,

    up), KeyCol1(arch, up)] Click on SummaryCol0 (avg): After:
    [SummaryCol0(avg, down), KeyCol0(config, down), SummaryCol1(sum, up),
    KeyCol1(arch, up)]` - The table is re-rendered, now primarily sorted by "avg" (descending),
    with ties broken by "config" (descending), then "sum" (ascending), then
    "arch" (ascending).

This multi-level sorting, driven by the `SortHistory` maintaining the sequence
of user sort actions, is a key aspect of the "how" behind the `pivot-table-sk`'s
user experience. It aims to provide a powerful yet familiar way to analyze
pivoted data.

# Module: /modules/pivotutil

The `pivotutil` module provides utility functions and constants for working with
pivot table requests. Its primary purpose is to ensure the validity and
integrity of pivot requests before they are processed, and to offer
human-readable descriptions for pivot operations. This centralization of
pivot-related logic helps maintain consistency and simplifies the handling of
pivot table configurations across different parts of the application.

### Key Components and Responsibilities

**`index.ts`**: This is the core file of the module and contains all the
exported functionalities.

- **`operationDescriptions`**:

  - **Why**: Pivot operations are often represented by short, cryptic
    identifiers (e.g., `avg`, `std`). To improve user experience and make
    UIs more understandable, a mapping to human-readable names is necessary.
  - **How**: This is a simple JavaScript object (dictionary) where keys are
    the `pivot.Operation` enum values (imported from `../json`) and values
    are their corresponding descriptive strings (e.g., "Mean", "Standard
    Deviation"). This allows for easy lookup and display of operation names.

- **`validatePivotRequest(req: pivot.Request | null): string`**:

  - **Why**: Before attempting to process a pivot request, it's crucial to
    ensure that the request is structurally sound and contains the minimally
    required information. This prevents runtime errors and provides early
    feedback to the user or calling code if the request is malformed.
  - **How**: This function performs basic validation checks on a
    `pivot.Request` object.

  * It first checks if the request itself is `null`. If so, it returns an
    error message.
  * It then verifies that the `group_by` property is present and is an array
    with at least one element. A pivot table fundamentally relies on
    grouping data, so this is a mandatory field.
  * If all checks pass, it returns an empty string, indicating a valid
    request. Otherwise, it returns a string describing the specific
    validation error.

  - **Workflow**: `Input: pivot.Request | null | V Is request null?
--(Yes)--> Return "Pivot request is null." | (No) V Is req.group_by null
or empty? --(Yes)--> Return "Pivot must have at least one GroupBy." |
(No) V Return "" (Valid)`

- **`validateAsPivotTable(req: pivot.Request | null): string`**:

  - **Why**: Some contexts specifically require a pivot _table_ that
    displays summary values, not just a pivot _plot_ which might only group
    traces without performing summary calculations. This function enforces
    the presence of summary operations.
  - **How**:

  * It first calls `validatePivotRequest` to ensure the basic structure of
    the request is valid. If `validatePivotRequest` returns an error, that
    error is immediately returned.
  * If the basic validation passes, it then checks if the `summary` property
    of the request is present and is an array with at least one element.
    Summary operations (like sum, average, etc.) are essential for
    generating the aggregated values displayed in a pivot table. Without
    them, the request might be valid for plotting individual traces grouped
    by some criteria, but not for a typical pivot table with summarized
    data.
  * If the `summary` array is missing or empty, an error message is
    returned. Otherwise, an empty string is returned.

  - **Workflow**: `Input: pivot.Request | null | V Call
validatePivotRequest(req) --> invalidMsg | V Is invalidMsg not empty?
--(Yes)--> Return invalidMsg | (No) V Is req.summary null or empty?
--(Yes)--> Return "Must have at least one Summary operation." | (No) V
Return "" (Valid for pivot table)`

**`index_test.ts`**: This file contains unit tests for the functions in
`index.ts`.

- **Why**: To ensure the validation logic correctly identifies valid and
  invalid pivot requests under various conditions. This maintains the
  reliability of the `pivotutil` module.
- **How**: It uses the `chai` assertion library to define test cases.
  - For `validatePivotRequest`, it tests scenarios like:
  - `null` request.
  - `group_by` being `null`.
  - `group_by` being an empty array.
  - A completely valid request.
  - For `validateAsPivotTable`, it builds upon the `validatePivotRequest`
    checks and adds tests for: _ `summary` being `null`. _ `summary` being
    an empty array. \* A valid request with at least one summary operation.
    Each test asserts whether the validation functions return an empty
    string (for valid inputs) or a non-empty error message string (for
    invalid inputs) as expected.

The design decision to separate `validatePivotRequest` and
`validateAsPivotTable` allows for more granular validation. Some parts of an
application might only need the basic validation (e.g., ensuring data can be
grouped), while others specifically require summary operations for display in a
tabular format. This separation provides flexibility. The use of descriptive
error messages aids in debugging and user feedback.

# Module: /modules/plot-google-chart-sk

The `plot-google-chart-sk` module provides a custom element for rendering
interactive time-series charts using Google Charts. It is designed to display
performance data, including anomalies and user-reported issues, and allows users
to interact with the chart through panning, zooming, and selecting data points.

**Key Responsibilities:**

- **Data Visualization:** Renders line charts based on `DataTable` objects,
  which are consumed from a Lit context (`dataTableContext`). This `DataTable`
  typically contains time-series data where the first column is a commit
  identifier (e.g., revision number or timestamp), the second is a date
  object, and subsequent columns represent different data traces.
- **Interactivity:**
  - **Panning:** Allows users to pan the chart horizontally by clicking and
    dragging.
  - **Zooming:** Supports both horizontal and vertical zooming. Users can
    Ctrl-click and drag to select a region to zoom into. A reset button
    allows returning to the original view.
  - **Delta Calculation:** Enables users to Shift-click and drag vertically
    to measure the difference (both raw and percentage) between two Y-axis
    values.
  - **Tooltip Display:** Shows detailed information about a data point when
    the user hovers over it.
  - **Data Point Selection:** Allows users to click on a data point to
    select it, which can trigger other actions in the application.
- **Anomaly and Issue Display:** Overlays icons on the chart to indicate
  anomalies (regressions, improvements, untriaged, ignored) and user-reported
  issues at specific data points. These are also consumed from Lit contexts
  (`dataframeAnomalyContext` and `dataframeUserIssueContext`).
- **Legend and Trace Management:** Includes a side panel (`side-panel-sk`)
  that displays a legend for the plotted traces. Users can toggle the
  visibility of individual traces using checkboxes in the side panel.
- **Dynamic Updates:** Responds to changes in data, selected ranges, and other
  properties by redrawing or updating the chart view.

**Design Decisions and Implementation Choices:**

- **Google Charts Integration:** Leverages the
  `@google-web-components/google-chart` library for the core charting
  functionality. This provides a robust and feature-rich charting engine.
- **LitElement and Context API:** Built as a LitElement custom element, making
  it easy to integrate into modern web applications. It utilizes Lit's Context
  API to consume shared data like the `DataTable`, anomaly information, and
  loading states from parent components or a centralized data store. This
  promotes a decoupled architecture.
- **Modular Sub-components:**
  - `v-resizable-box-sk`: A dedicated component for the vertical selection
    box used in the "deltaY" mode. It calculates and displays the difference
    between the start and end points of the drag.
  - `drag-to-zoom-box-sk`: Handles the visual representation of the
    selection box during the drag-to-zoom interaction. It manages the
    display and dimensions of the box as the user drags.
  - `side-panel-sk`: Encapsulates the legend and trace visibility controls.
    This separation of concerns keeps the main chart component focused on
    plotting.
- **Event-Driven Communication:** Emits custom events (e.g.,
  `selection-changed`, `plot-data-mouseover`, `plot-data-select`) to notify
  parent components of user interactions and chart state changes. This allows
  for integration with other parts of an application.
- **Overlay for Anomalies and Issues:** Anomalies and user issues are rendered
  as absolutely positioned `md-icon` elements on top of the chart. Their
  positions are calculated based on the chart's layout and the data point
  coordinates. This approach avoids modifying the Google Chart's internal
  rendering and allows for more flexible styling and interaction with these
  markers.
- **Caching and Performance:**
  - Caches the Google Chart object (`this.chart`) and chart layout
    information (`this.cachedChartArea`) to avoid redundant lookups.
  - Maintains a `removedLabelsCache` to efficiently hide and show traces
    without reconstructing the entire `DataView` each time.
- **Separate Interaction Modes:** The `navigationMode` property (`pan`,
  `deltaY`, `dragToZoom`) manages the current mouse interaction state. This
  simplifies event handling by directing mouse events to the appropriate logic
  based on the active mode.
- **Dynamic Y-Axis Title:** The `determineYAxisTitle` method attempts to
  create a meaningful Y-axis title by examining the `unit` and
  `improvement_direction` parameters from the trace names. It displays these
  only if they are consistent across all visible traces.

**Key Components/Files:**

- **`plot-google-chart-sk.ts`:** The core component that orchestrates the
  chart display and interactions.
  - Manages the Google Chart instance.
  - Handles mouse events for panning, zooming, delta calculation, and data
    point interactions.
  - Consumes data (`DataTable`, `AnomalyMap`, `UserIssueMap`) via Lit
    context.
  - Renders anomaly and user issue icons as overlays.
  - Communicates with `side-panel-sk` to manage trace visibility.
  - Dispatches custom events for user interactions.
- **`side-panel-sk.ts`:** Implements the side panel containing the legend and
  checkboxes for toggling trace visibility.
  - Generates legend entries based on the `DataTable`.
  - Manages the checked state of traces and communicates changes to
    `plot-google-chart-sk`.
  - Can display the calculated delta values from the `v-resizable-box-sk`.
- **`v-resizable-box-sk.ts`:** A custom element for the vertical resizable
  selection box used during the delta calculation (Shift-click + drag).
  - Displays the selection box and calculates the raw and percentage
    difference between the Y-values at the start and end of the drag.
- **`drag-to-zoom-box-sk.ts`:** A custom element for the selection box used
  during the drag-to-zoom interaction (Ctrl-click + drag).
  - Draws a semi-transparent rectangle indicating the area to be zoomed.
- **`plot-google-chart-sk-demo.ts` and `plot-google-chart-sk-demo.html`:**
  Provide a demonstration page showcasing the `plot-google-chart-sk` element
  with sample data. This is crucial for development and testing.
- **`index.ts`:** Serves as the entry point for the module, importing and
  registering all the custom elements defined within.

**Key Workflows:**

1.  **Initial Chart Rendering:** `DataTable` (from context) ->
    `plot-google-chart-sk` -> `updateDataView()` -> Creates
    `google.visualization.DataView` -> Sets columns based on `domain`
    (commit/date) and visible traces -> `updateOptions()` configures chart
    appearance (colors, axes, view window) -> `plotElement.value.view = view`
    and `plotElement.value.options = options` -> Google Chart renders. ->
    `onChartReady()`: -> Caches chart object. -> Calls `drawAnomaly()`,
    `drawUserIssues()`, `drawXbar()`.

2.  **Panning:** User mousedown (not Shift or Ctrl) -> `onChartMouseDown()`:
    `navigationMode = 'pan'` User mousemove -> `onWindowMouseMove()`: ->
    Calculates deltaX based on mouse movement and current domain. -> Updates
    `this.selectedRange`. -> Calls `updateOptions()` to update chart's
    horizontal view window. -> Dispatches `selection-changing` event. User
    mouseup -> `onWindowMouseUp()`: -> Dispatches `selection-changed` event. ->
    `navigationMode = null`.

3.  **Drag-to-Zoom:** User Ctrl + mousedown -> `onChartMouseDown()`:
    `navigationMode = 'dragToZoom'` -> `zoomRangeBox.value.initializeShow()`:
    Displays the drag box. User mousemove -> `onWindowMouseMove()`: ->
    `zoomRangeBox.value.handleDrag()`: Updates the drag box dimensions. User
    mouseup -> `onChartMouseUp()`: -> Calculates zoom boundaries based on drag
    box and `isHorizontalZoom`. -> `zoomRangeBox.value.hide()`. ->
    `showResetButton = true`. -> `updateBounds()`: Updates chart's
    `hAxis.viewWindow` or `vAxis.viewWindow`. -> `navigationMode = null`.

4.  **Delta Calculation (Shift-Click):** User Shift + mousedown ->
    `onChartMouseDown()`: `navigationMode = 'deltaY'` ->
    `deltaRangeBox.value.show()`: Displays the vertical resizable box. User
    mousemove -> `onWindowMouseMove()`: ->
    `deltaRangeBox.value.updateSelection()`: Updates box height and calculates
    delta. -> Updates `sidePanel.value` with delta values. User Shift +
    mousedown (again) or regular mousedown -> `onChartMouseDown()`: -> Toggles
    `deltaRangeOn`. If finishing, `sidePanel.value.showDelta = true`. User
    mouseup (after dragging) -> `onChartMouseUp()`: -> Updates `sidePanel.value`
    with final delta values. -> `navigationMode = null`.

5.  **Toggling Trace Visibility:** User clicks checkbox in `side-panel-sk` ->
    `side-panel-sk` dispatches `side-panel-selected-trace-change`.
    `plot-google-chart-sk` listens (`sidePanelCheckboxUpdate()`): -> Updates
    `this.removedLabelsCache`. -> Calls `updateDataView()`: -> Recreates
    `DataView`, hiding/showing columns based on `removedLabelsCache`. -> Updates
    chart.

6.  **Anomaly/Issue Display:** `anomalyMap` or `userIssues` (from context)
    changes -> `plot-google-chart-sk.willUpdate()` ->
    `plotElement.value.redraw()` (if chart already rendered). Chart redraw
    triggers `onChartReady()`: -> `drawAnomaly()` / `drawUserIssues()`: ->
    Iterates through anomalies/issues for visible traces. -> Calculates screen
    coordinates (x, y) using `chart.getChartLayoutInterface().getXLocation()`
    and `getYLocation()`. -> Clones template `md-icon` elements from slots. ->
    Positions the icons absolutely within `anomalyDiv` or `userIssueDiv`.

This detailed explanation should provide a solid understanding of the
`plot-google-chart-sk` module's purpose, architecture, and key functionalities.

# Module: /modules/plot-simple-sk

The `plot-simple-sk` module provides a custom HTML element for rendering 2D line
graphs. It's designed to be interactive, allowing users to zoom, inspect
individual data points, and highlight specific traces.

**Core Functionality and Design:**

The primary goal of `plot-simple-sk` is to display time-series data or any data
that can be represented as a set of (x, y) coordinates. Key design
considerations include:

1.  **Performance:** To handle potentially large datasets and maintain a smooth
    user experience, the element employs several optimization techniques:

    - **Dual Canvases:** It uses two `<canvas>` elements stacked on top of
      each other.
      - The bottom canvas (`traces`) is for drawing the static parts of the
        plot: the lines, axes, and dots representing data points. These are
        pre-rendered into `Path2D` objects for efficient redrawing.
      - The top canvas (`overlay`) is for dynamic elements that change
        frequently, such as crosshairs, zoom selection rectangles, and hover
        highlights. This separation prevents unnecessary redrawing of the
        entire plot.
    - **`Path2D` Objects:** Trace lines and data point dots are converted into
      `Path2D` objects. This allows the browser to optimize their rendering,
      leading to faster redraws compared to repeatedly issuing drawing
      commands.
    - **k-d Tree for Point Proximity:** For features like displaying
      information on mouse hover or selecting the nearest data point on click,
      a k-d tree (`kd.ts`) is used. This data structure allows for efficient
      searching of the closest point in a 2D space, crucial for interactivity
      with potentially many data points.
    - **Debounced Redraws and Calculations:** Operations like rebuilding the
      k-d tree (`recalcSearchTask`) or redrawing after a zoom (`zoomTask`) are
      often scheduled using `window.setTimeout`. This prevents these
      potentially expensive operations from blocking the main thread and
      ensures they only happen when necessary, improving responsiveness.
      `requestAnimationFrame` is used for mouse movement updates to
      synchronize with the browser's repaint cycle.

2.  **Interactivity:**

    - **Zooming:**
      - **Summary and Detail Views:** The plot can optionally display a
        "summary" area above the main "detail" area. The summary shows an
        overview of all data, and users can drag a region on the summary to
        zoom the detail view to that specific x-axis range.
      - **Detail View Zoom:** Users can also drag a rectangle directly on
        the detail view to zoom into a specific x and y range.
      - **Zoom Stack:** The element maintains a stack of zoom levels
        (`detailsZoomRangesStack`), allowing users to progressively zoom in
        and potentially (though not explicitly stated as a current feature
        for _out_) navigate back through zoom levels.
    - **Hover and Selection:**
      - Moving the mouse near a trace highlights the closest data point and
        emits a `trace_focused` event.
      - Clicking on a trace selects the closest data point and emits a
        `trace_selected` event.
    - **Crosshairs:** When the shift key is held, crosshairs are displayed,
      indicating the mouse's current x and y position on the plot.
    - **Highlighting Traces:** Specific traces can be programmatically
      highlighted, making them stand out.
    - **X-Bar and Bands:** Vertical lines (`xbar`) or regions (`bands`) can be
      drawn on the plot to mark specific x-axis values or ranges.
    - **Anomalies and User Issues:** The plot can display markers for
      anomalies (regressions, improvements) and user-reported issues at
      specific data points.

3.  **Appearance and Theming:**

    - **Responsive Sizing:** The plot adapts to the `width` and `height`
      attributes of the custom element and uses `ResizeObserver` to redraw
      when its dimensions change.
    - **Device Pixel Ratio:** It accounts for `window.devicePixelRatio` to
      render crisply on high-DPI displays by drawing to a larger canvas and
      then scaling it down with CSS transforms.
    - **CSS Variables for Theming:** The element is designed to integrate with
      `elements-sk/themes` and uses CSS variables for colors (e.g.,
      `--on-background`, `--success`, `--failure`), allowing its appearance to
      be customized by the surrounding application's theme. It listens for
      `theme-chooser-toggle` events to redraw when the theme changes.

**Key Files and Responsibilities:**

- **`plot-simple-sk.ts`:** This is the heart of the module, defining the
  `PlotSimpleSk` custom element.

  - **Rendering Logic:** Contains all the drawing code for the traces, axes,
    labels, summary view, detail view, crosshairs, zoom indicators,
    anomalies, etc. It manages the two canvas contexts (`ctx` for traces,
    `overlayCtx` for overlays).
  - **State Management:** Manages the internal state, including the
    `lineData` (traces and their pre-rendered paths), `labels` (x-axis tick
    information), current `_zoom` state, `detailsZoomRangesStack` for detail
    view zooms, `hoverPt`, `crosshair`, `highlighted` traces, `_xbar`,
    `_bands`, and `_anomalyDataMap`.
  - **Event Handling:** Sets up event listeners for mouse interactions
    (move, down, up, leave, click) to handle zooming, hovering, and
    selection. It also listens for `theme-chooser-toggle` and
    `ResizeObserver` events.
  - **API Methods:** Exposes methods like `addLines`, `deleteLines`,
    `removeAll`, and properties like `highlight`, `xbar`, `bands`, `zoom`,
    `anomalyDataMap`, `userIssueMap`, and `dots` to control the plot's
    content and appearance.
  - **Coordinate Transformations:** Uses `d3-scale` (specifically
    `scaleLinear`) to map data coordinates (domain) to canvas pixel
    coordinates (range) and vice-versa. Functions like `rectFromRange` and
    `rectFromRangeInvert` handle these transformations for rectangular
    regions.
  - **Path and Search Builders:**
  - `PathBuilder`: A helper class to construct `Path2D` objects for trace
    lines and dots based on the current scales and data.
  - `SearchBuilder`: A helper class to prepare the data points for the
    `KDTree` by converting source coordinates to canvas coordinates.
  - **Drawing Areas:** Defines `SummaryArea` and `DetailArea` interfaces and
    manages their respective rectangles, axes, and scaling ranges.

- **`kd.ts`:** Implements a k-d tree.

  - **Purpose:** Provides an efficient way (`O(log n)` on average for
    search) to find the nearest data point to a given mouse coordinate on
    the canvas. This is crucial for interactivity like mouse hovering and
    clicking to identify specific points on traces.
  - **Implementation:** It's a trimmed-down version of an existing k-d tree
    library, specifically tailored for finding the single closest 2D point.
    It takes an array of points (each with `x` and `y` properties), a
    distance metric function, and the dimensions to consider (`['x', 'y']`).
    The `nearest()` method is the primary interface used by
    `plot-simple-sk.ts`.

- **`ticks.ts`:** Responsible for generating appropriate tick marks and labels
  for the time-based x-axis.

  - **Purpose:** Given an array of `Date` objects representing the x-axis
    values, it determines a sensible set of tick positions and their
    corresponding formatted string labels (e.g., "Jul", "Mon, 8 AM", "10:30
    AM").
  - **Logic:** It considers the total duration spanned by the dates and
    selects an appropriate time granularity (e.g., months, days, hours,
    minutes) for the labels using `Intl.DateTimeFormat`. It aims for a
    reasonable number of ticks (`MIN_TICKS` to `MAX_TICKS`) and uses a
    `fixTicksLength` function to thin out the ticks if too many are
    generated.
  - **Output:** The `ticks()` function returns an array of objects, each
    with an `x` (index in the original data) and a `text` (formatted label).

- **`plot-simple-sk.scss`:** Contains the SASS/CSS styles for the
  `plot-simple-sk` element.

  - **Layout:** Defines the positioning of the canvas elements (absolute
    positioning for the overlay on top of the trace canvas).
  - **Theming Integration:** Imports `themes.scss` and uses CSS variables
    (e.g., `var(--on-background)`, `var(--background)`) to ensure the plot's
    colors match the application's theme.

- **`index.ts`:** A simple entry point that imports `plot-simple-sk.ts` to
  ensure the custom element is defined and registered with the browser.

- **Demo Files (`plot-simple-sk-demo.html`, `plot-simple-sk-demo.ts`,
  `plot-simple-sk-demo.scss`):**

  - Provide a live demonstration of the `plot-simple-sk` element's
    capabilities.
  - The HTML sets up the plot elements and buttons to trigger various
    actions.
  - The TypeScript file (`plot-simple-sk-demo.ts`) contains the logic to
    interact with the plot, such as adding random trace data, highlighting
    traces, zooming, clearing the plot, and displaying anomaly markers. It
    also logs events emitted by the plot.

**Key Workflows:**

1.  **Initialization and Rendering:** `ElementSk constructor` ->
    `connectedCallback` -> `render` `render` -> `_render` (lit-html template
    instantiation) -> `canvas.getContext` -> `updateScaledMeasurements` ->
    `updateScaleRanges` -> `recalcDetailPaths` -> `recalcSummaryPaths` ->
    `drawTracesCanvas`

2.  **Adding Data (`addLines`):** `addLines` -> Convert `MISSING_DATA_SENTINEL`
    to `NaN` -> Store in `this.lineData` -> `updateScaleDomains` ->
    `recalcSummaryPaths` -> `recalcDetailPaths` -> `drawTracesCanvas`
    `recalcDetailPaths` / `recalcSummaryPaths` -> For each line: `PathBuilder`
    creates `linePath` and `dotsPath`. `recalcDetailPaths` -> `recalcSearch`
    (schedules `recalcSearchImpl`) `recalcSearchImpl` -> `SearchBuilder`
    populates points -> `new KDTree`

3.  **Mouse Hover and Focus:** `mousemove` event -> `this.mouseMoveRaw` updated
    `raf` loop -> checks `this.mouseMoveRaw` -> `eventToCanvasPt` -> If
    `this.pointSearch`: `this.pointSearch.nearest(pt)` -> updates `this.hoverPt`
    -> dispatches `trace_focused` event -> Updates `this.crosshair` (based on
    shift key and `hoverPt`) -> `drawOverlayCanvas`

4.  **Zooming via Summary Drag:** `mousedown` on summary -> `this.inZoomDrag =
'summary'` -> `this.zoomBegin` set `mousemove` (while dragging) -> `raf`
    loop: -> `eventToCanvasPt` -> `clampToRect` (summary area) ->
    `this.summaryArea.range.x.invert(pt.x)` to get source x -> `this.zoom =
[min_x, max_x]` (triggers `_zoomImpl` via setter task) `_zoomImpl` (after
    timeout) -> `updateScaleDomains` -> `recalcDetailPaths` ->
    `drawTracesCanvas` `mouseup` / `mouseleave` -> dispatches `zoom` event ->
    `this.inZoomDrag = 'no-zoom'`

5.  **Zooming via Detail Area Drag:** `mousedown` on detail ->
    `this.inZoomDrag = 'details'` -> `this.zoomRect` initialized `mousemove`
    (while dragging) -> `raf` loop: -> `eventToCanvasPt` -> `clampToRect`
    (detail area) -> Updates `this.zoomRect.width/height` -> `drawOverlayCanvas`
    (to show the dragging rectangle) `mouseup` / `mouseleave` ->
    `dispatchZoomEvent` -> `doDetailsZoom` `doDetailsZoom` -> If zoom box is
    large enough: `this.detailsZoomRangesStack.push(rectFromRangeInvert(...))`
    -> `_zoomImpl`

6.  **Drawing Process:**

    - `drawTracesCanvas()`:

      1.  Clears the appropriate part of the main trace canvas (`this.ctx`).
      2.  Draws detail area:
          - Saves context, clips to detail rect.
          - Calls `drawXAxis` (for detail).
          - Iterates `this.lineData`: draws `line.detail.linePath` and
            `line.detail.dotsPath` if `this.dots` is true.
          - Restores context.
          - Calls `drawXAxis` again (to draw labels outside the clipped
            region).
      3.  If `this.summary` and not dragging zoom:
          - Draws summary area similarly.
      4.  Calls `drawYAxis` (for detail).
      5.  Calls `drawOverlayCanvas()`.

    - `drawOverlayCanvas()`:

      1.  Clears the entire overlay canvas (`this.overlayCtx`).
      2.  If `this.summary`:
          - Saves context, clips to summary rect.
          - Calls `drawXBar`, `drawBands`.
          - Draws detail zoom indicator box if `detailsZoomRangesStack` is
            not empty.
          - Draws summary zoom bars and shaded regions based on
            `this._zoom`.
          - Restores context.
      3.  Clips to detail rect:
          - Calls `drawXBar`, `drawBands`.
          - Draws highlighted lines.
          - Draws hovered line/dots.
          - Calls `drawUserIssues`, `drawAnomalies`.
          - If dragging zoom in detail: draws `this.zoomRect` (dashed).
          - If not dragging: draws crosshairs and hover label.
          - Restores context.

This structured approach allows `plot-simple-sk` to be both feature-rich and
performant for visualizing and interacting with 2D data plots.

# Module: /modules/plot-summary-sk

The `plot-summary-sk` module provides a custom HTML element,
`<plot-summary-sk>`, designed to display a summary plot of performance data and
allow users to select a range within that plot. This is particularly useful for
visualizing trends over time or commit ranges and enabling interactive
exploration of the data.

At its core, `plot-summary-sk` leverages the Google Charts library to render an
area chart. It's designed to work with a `DataFrame`, a data structure commonly
used in Perf for holding timeseries data. The element can display data based on
either commit offsets or timestamps (`domain` attribute).

Key Responsibilities:

- **Data Visualization**: Renders an area chart representing performance data
  over a specified domain (commit or date).
- **Range Selection**: Allows users to interactively select a range on the
  plot. This selection can be initiated by dragging on the chart or by
  programmatically setting the selection.
- **Event Emission**: Emits a `summary_selected` custom event when the user
  makes or changes a selection. This event carries details about the selected
  range (start, end, value, and domain).
- **Dynamic Data Loading**: Optionally, it can display controls to load more
  data in either direction (earlier or later), integrating with a
  `DataFrameRepository` to fetch and append new data.
- **Theming**: Adapts to theme changes (e.g., dark mode) by redrawing the
  chart with appropriate styles.
- **Responsiveness**: The chart redraws itself when its container is resized,
  ensuring it remains visually correct.

Key Components/Files:

- **`plot-summary-sk.ts`**: This is the main file defining the `PlotSummarySk`
  LitElement.
  - **Why**: It encapsulates the logic for chart rendering, user
    interaction, data handling, and event emission.
  - **How**:
  - It consumes `DataFrame` data (from `dataTableContext`) and renders it
    using `<google-chart>`.
  - It manages the display of single or all traces based on the
    `selectedTrace` property.
  - It uses an internal `h-resizable-box-sk` element to provide the visual
    selection rectangle and handles the mouse events for drawing and
    resizing this selection.
  - It translates between the visual coordinates of the selection box and
    the data values (commit offsets or timestamps) of the underlying chart.
  - It listens for `google-chart-ready` events to ensure operations like
    setting a selection programmatically happen after the chart is fully
    initialized.
  - It provides `controlTemplate` for optional "load more data" buttons,
    which interact with a `DataFrameRepository` (consumed via
    `dataframeRepoContext`).
  - It uses a `ResizeObserver` to detect when the element is resized and
    triggers a chart redraw.
  - It manages colors for different traces to ensure consistent
    visualization.
- **`h-resizable-box-sk.ts`**: This file defines the `HResizableBoxSk`
  LitElement, a reusable component for creating a horizontally resizable and
  draggable selection box.
  - **Why**: To decouple the complex UI interaction logic of drawing,
    moving, and resizing a selection rectangle from the main
    `plot-summary-sk` component. This promotes reusability and simplifies
    the main component's logic.
  - **How**:
  - It renders a `div` (`.surface`) that represents the selection.
  - It listens for `mousedown` events on its container to initiate an
    action: 'draw' (if clicking outside the existing selection), 'drag' (if
    clicking inside the selection), 'left' (if clicking on the left edge),
    or 'right' (if clicking on the right edge).
  - It listens for `mousemove` events on the `window` to update the
    selection's position and size during an action. This ensures interaction
    continues even if the mouse moves outside the element's bounds.
  - It listens for `mouseup` events on the `window` to finalize the action
    and emits a `selection-changed` event with the new range.
  - It uses CSS to style the selection box and provide visual cues for
    dragging and resizing (e.g., `cursor: move`, `cursor: ew-resize`).
  - The `selectionRange` property (getter and setter) allows programmatic
    control and retrieval of the selection, defined by `begin` and `end`
    pixel offsets relative to the component.
- **`plot-summary-sk.css.ts`**: Contains the CSS styles for the
  `plot-summary-sk` element, defined as a Lit `css` tagged template literal.
  - **Why**: To encapsulate the visual styling, ensuring the plot and its
    controls are laid out correctly and are visually consistent with the
    application's theme.
  - **How**: It uses flexbox for layout, positions the selection box
    (`h-resizable-box-sk`) absolutely over the chart, and styles the
    optional loading buttons and loading indicator.
- **`plot-summary-sk-demo.ts` and `plot-summary-sk-demo.html`**: Provide a
  demonstration page for the `plot-summary-sk` element.
  - **Why**: To allow developers to see the component in action, test its
    features, and understand how to integrate it.
  - **How**: The HTML sets up multiple instances of `plot-summary-sk` with
    different configurations (e.g., `domain`, `selectionType`). The
    TypeScript file generates sample `DataFrame` objects, converts them to
    Google DataTable format, and populates the plot elements. It also
    listens for `summary_selected` events and displays their details.
- **Test Files (`*.test.ts`, `*_puppeteer_test.ts`)**:
  - **Why**: To ensure the component functions as expected and to prevent
    regressions.
  - **How**:
  - Unit tests (`plot-summary-sk_test.ts`, `h_resizable_box_sk_test.ts`)
    verify individual component logic, such as programmatic selection and
    state changes. They often mock dependencies like the Google Chart
    library or use test utilities to generate data.
  - Puppeteer tests (`plot-summary-sk_puppeteer_test.ts`) perform end-to-end
    testing by interacting with the component in a real browser environment.
    They simulate user actions like mouse drags and verify the emitted event
    details and visual output (via screenshots).

Key Workflows:

1.  **Initialization and Data Display**:

    ```
    [DataFrame via context or property]
           |
           v
    plot-summary-sk
           |
           v
    [willUpdate/updateDataView] --> Converts DataFrame to Google DataTable
           |
           v
    <google-chart> --> Renders area chart
           |
           v
    [google-chart-ready event] --> plot-summary-sk may apply cached selection
    ```

2.  **User Selecting a Range by Drawing**:

    ```
    User mousedowns on <plot-summary-sk> (outside existing selection in h-resizable-box-sk)
           |
           v
    h-resizable-box-sk (action = 'draw')
           |
           v
    User moves mouse (mousemove on window)
           |
           v
    h-resizable-box-sk --> Updates selection box dimensions
           |
           v
    User mouseups (mouseup on window)
           |
           v
    h-resizable-box-sk --> Emits 'selection-changed' (with pixel coordinates)
           |
           v
    plot-summary-sk (onSelectionChanged)
           |
           v
    Converts pixel coordinates to data values (commit/timestamp)
           |
           v
    Emits 'summary_selected' (with data values)
    ```

3.  **User Resizing/Moving an Existing Selection**:

    ```
    User mousedowns on <h-resizable-box-sk> (on edge for resize, or middle for drag)
           |
           v
    h-resizable-box-sk (action = 'left'/'right'/'drag')
           |
           v
    User moves mouse (mousemove on window)
           |
           v
    h-resizable-box-sk --> Updates selection box position/dimensions
           |
           v
    User mouseups (mouseup on window)
           |
           v
    h-resizable-box-sk --> Emits 'selection-changed'
           |
           v
    plot-summary-sk (onSelectionChanged) --> Converts & Emits 'summary_selected'
    ```

4.  **Programmatic Selection**: `Application calls
plotSummarySkElement.Select(beginHeader, endHeader) OR Application sets
plotSummarySkElement.selectedValueRange = { begin: val1, end: val2 } | v
plot-summary-sk | v Caches selectedValueRange (important if chart not ready)
| v [If chart ready] --> Converts data values to pixel coordinates | v Sets
selectionRange on <h-resizable-box-sk>` If the chart is not ready when
    `selectedValueRange` is set, the conversion and setting of the
    `h-resizable-box-sk` selection is deferred until the `google-chart-ready`
    event fires.

The design separates the concerns of data plotting (Google Charts), interactive
range selection UI (`h-resizable-box-sk`), and the overall orchestration and
data conversion logic (`plot-summary-sk`). This makes the system more modular
and easier to maintain. The use of LitElement and contexts allows for a reactive
programming model and clean integration with other parts of the Perf
application.

# Module: /modules/point-links-sk

The `point-links-sk` module is a custom HTML element designed to display links
associated with specific data points in a performance analysis context. These
links often originate from ingestion files and can include commit details, build
logs, or other relevant resources.

The primary purpose of this module is to provide users with quick access to
contextual information related to a data point. It achieves this by:

1.  **Fetching and Displaying Links:** The module fetches link data from a
    backend API based on a commit ID and a trace ID. It then renders these links
    as clickable anchor elements.
2.  **Generating Commit Range Links:** A key feature is its ability to generate
    links representing the range of commits between two data points. This is
    particularly useful for understanding changes that might have occurred
    between two performance measurements.
    - If the commit hashes for a given key (e.g., "V8 Git Hash") are different
      between the current and previous data points, it constructs a URL that
      shows the log of commits between those two specific commit hashes.
    - If the commit hashes are the same, it simply links to the individual
      commit, indicating no change in that specific dependency.
3.  **Caching:** To optimize performance and avoid redundant API calls, the
    module can utilize a provided cache of previously loaded commit links. If
    the links for a specific commit and trace ID are already in the cache, it
    will use those instead of re-fetching.
4.  **User-Friendly Presentation:** Links are presented in a list format, with a
    "copy to clipboard" button for each link, enhancing usability.

**Key Responsibilities and Components:**

- **`point-links-sk.ts`**: This is the core file defining the `PointLinksSk`
  custom element.
  - It extends `ElementSk` from `infra-sk`.
  - **`load()` method**: This is the main public method responsible for
    initiating the process of fetching and displaying links. It takes the
    current commit ID, the previous commit ID, a trace ID, and arrays of
    keys to identify which links should be treated as commit ranges and
    which are general "useful links". It handles the logic for checking the
    cache, fetching data from the API, processing commit ranges, and
    updating the display.
  - **`getLinksForPoint()` and `invokeLinksForPointApi()` methods**: These
    private methods handle the actual API interaction to retrieve link data.
    `getLinksForPoint` attempts to fetch from `/_/links/` first and falls
    back to `/_/details/?results=false` if the initial attempt fails. It
    also includes workarounds for specific data inconsistencies (e.g., V8
    and WebRTC URLs).
  - **`renderPointLinks()` and `renderRevisionLink()` methods**: These
    methods, along with the static `template`, use `lit-html` to generate
    the HTML structure for displaying the links.
  - **Helper methods (`getCommitIdFromCommitUrl`, `getRepoUrlFromCommitUrl`,
    `getFormattedCommitRangeText`, `extractUrlFromStringForFuchsia`)**:
    These provide utility functions for parsing URLs and formatting text.
  - **Data properties (`commitPosition`, `displayUrls`, `displayTexts`)**:
    These store the state of the component, such as the current commit and
    the links to be displayed.
- **`point-links-sk.scss`**: Provides the styling for the `point-links-sk`
  element, ensuring a consistent look and feel, including styling for Material
  Design icons and buttons.
- **`index.ts`**: A simple entry point that imports and thereby registers the
  `point-links-sk` custom element.
- **`point-links-sk-demo.html` & `point-links-sk-demo.ts`**: These files set
  up a demonstration page for the `point-links-sk` element. The
  `point-links-sk-demo.ts` file uses `fetch-mock` to simulate the backend API,
  allowing developers to test the component's behavior in isolation. It
  demonstrates how to instantiate and use the `point-links-sk` element with
  different configurations.

**Workflow for Loading and Displaying Links:**

The typical workflow when the `load()` method is called can be visualized as:

```
Caller invokes pointLinksSk.load(currentCID, prevCID, traceID, rangeKeys, usefulKeys, cachedLinks)
    |
    V
Check if links for (currentCID, traceID) exist in `cachedLinks`
    |
    +-- YES --> Use cached links
    |           |
    |           V
    |           Render links
    |
    +-- NO ---> Fetch links for `currentCID` from API (`getLinksForPoint`)
                |
                V
                If `rangeKeys` are provided:
                |   Fetch links for `prevCID` from API (`getLinksForPoint`)
                |   For each key in `rangeKeys`:
                |       Extract current commit hash from `currentCID` links
                |       Extract previous commit hash from `prevCID` links
                |       If hashes are different:
                |           Generate "commit range" URL (e.g., .../+log/prevHash..currentHash)
                |       Else (hashes are same):
                |           Use current commit URL
                |       Add to `displayUrls` and `displayTexts`
                |
                V
                If `usefulKeys` are provided:
                |   For each key in `usefulKeys`:
                |       Add corresponding link from `currentCID` links to `displayUrls`
                |
                V
                Update cache with newly fetched/generated links for (currentCID, traceID)
                |
                V
                Render links
```

This module is designed to be flexible, allowing the consuming application to
specify which types of links should be processed for commit ranges and which
should be displayed as direct links. The inclusion of error handling (via
`errorMessage`) and the fallback mechanism in API calls (`/_/links/` then
`/_/details/`) make it more robust.

# Module: /modules/progress

The `progress` module provides a mechanism for initiating and monitoring the
status of long-running tasks on the server. This is crucial for user experience,
as it allows the client to display progress information and avoid appearing
unresponsive during lengthy operations.

The core of this module is the `startRequest` function. This function is
designed to handle asynchronous server-side processes that might take a
significant amount of time to complete.

**How `startRequest` Works:**

1.  **Initiation:**

    - It begins by sending an initial POST request to a specified
      `startingURL` with a given `body`. This request typically triggers the
      long-running task on the server.
    - If a `spinner-sk` element is provided, it's activated to visually
      indicate that a process is underway.

2.  **Polling:**

    - The server's response to the initial request (and subsequent polling
      requests) is expected to be a JSON object of type
      `progress.SerializedProgress`. This object contains:
      - `status`: Indicates whether the task is "Running" or "Finished" (or
        potentially other states like "Error").
      - `messages`: An array of key-value pairs providing more detailed
        information about the current state of the task (e.g., current step,
        progress percentage).
      - `url`: If the `status` is "Running", this URL is used for the next
        polling request to get updated progress.
      - `results`: If the `status` is "Finished", this field contains the
        final output of the long-running process.
    - If the `status` is "Running", `startRequest` will schedule a
      `setTimeout` to make a GET request to the `url` provided in the response
      after a specified `period`. This creates a polling loop.

3.  **Callback and Completion:**

    - An optional `callback` function can be provided. This function is
      invoked after each successful fetch (both the initial request and every
      polling update), receiving the `progress.SerializedProgress` object.
      This allows the UI to update with the latest progress information.
    - The polling continues until the server responds with a `status` that is
      not "Running" (e.g., "Finished").
    - Once the task is complete, the Promise returned by `startRequest`
      resolves with the final `progress.SerializedProgress` object.
    - If a `spinner-sk` was provided, it is deactivated.

4.  **Error Handling:**

    - If any network request fails (e.g., non-2xx HTTP status), the Promise
      returned by `startRequest` is rejected with an error.
    - The spinner (if provided) is also deactivated in case of an error.

**Workflow Diagram:**

```
Client UI                 startRequest Function                 Server
----------                ---------------------                 ------
   |                          |
   | -- Call startRequest --> |
   |                          | -- POST to startingURL (body) --> |
   |                          |                                   |
   |                          | <-- Response (SerializedProgress) -- |
   |                          |
   | -- (Optional) Activate -- |
   |      Spinner            |
   |                          |
   |                          | -- If status is "Running": --------> Schedule setTimeout(period)
   |                          |     |
   |                          |     V
   |                          | -- GET to progress.url -----------> |
   |                          |                                   |
   |                          | <-- Response (SerializedProgress) -- |
   |                          |     |
   |                          |     --- (Invoke callback) ---------> Client UI (Update progress)
   |                          |     |
   |                          |     --- Loop back to "If status is 'Running'"
   |                          |
   |                          | -- If status is "Finished": -------> Resolve Promise
   |                          |                                     |
   | -- (Optional) Deactivate | <-----------------------------------
   |      Spinner            |
   |                          |
   | <-- Promise Resolves ---- |
   |     (SerializedProgress)  |
```

**Key Files:**

- **`progress.ts`**:

  - **Responsibilities**: Implements the core logic for initiating requests,
    polling for status updates, handling responses, and managing callbacks
    and promises. It also provides utility functions for formatting progress
    messages.
  - **Key Components**:
  - `startRequest`: The primary function that orchestrates the entire
    progress monitoring flow. It encapsulates the logic for making the
    initial POST request and subsequent GET requests for polling. The use of
    a single `processFetch` internal function is a design choice to reduce
    code duplication, as the response handling logic is identical for both
    the initial and polling fetches.
  - `messagesToErrorString`: A utility function designed to extract a
    user-friendly error message from the `messages` array within
    `SerializedProgress`. It prioritizes messages with the key "Error" but
    falls back to concatenating all messages if no specific error message is
    found. This ensures that some form of feedback is available even if the
    server doesn't explicitly flag an error.
  - `messagesToPreString`: Formats messages for display, typically within a
    `<pre>` tag, by putting each key-value pair on a new line. This is
    useful for presenting detailed progress logs.
  - `messageByName`: Allows retrieval of a specific message's value by its
    key from the `messages` array, with a fallback if the key is not found.
    This is useful for extracting specific pieces of information from the
    progress updates (e.g., the current step number).
  - **Dependencies**:
  - `elements-sk/modules/spinner-sk`: Used to visually indicate that a
    background task is in progress.
  - `perf/modules/json`: Provides the `progress.SerializedProgress` type
    definition, ensuring consistency in how progress information is
    structured between the client and server.

- **`progress_test.ts`**:

  - **Responsibilities**: Contains unit tests for the `progress.ts` module.
  - **Key Focus**:
  - Verifies that `startRequest` correctly handles different server response
    scenarios: immediate completion, one or more polling steps, and network
    errors.
  - Ensures that the optional callback is invoked correctly during the
    polling process.
  - Tests the behavior of the message formatting utility functions
    (`messagesToErrorString`, `messageByName`) with various inputs.
  - **Methodology**: Uses `fetch-mock` to simulate server responses,
    allowing for controlled testing of the asynchronous network interactions
    without relying on an actual backend. This is crucial for creating
    reliable and fast unit tests.

The design of this module prioritizes a clear separation of concerns.
`startRequest` focuses on the communication and polling logic, while the utility
functions provide convenient ways to interpret and display the progress
information received from the server. The use of Promises simplifies handling
asynchronous operations, and the optional callback provides flexibility for
updating the UI in real-time.

# Module: /modules/query-chooser-sk

## Query Chooser Element (`query-chooser-sk`)

The `query-chooser-sk` module provides a user interface element for selecting
and modifying query parameters. It's designed to offer a compact way to display
the currently active query and provide a mechanism to change it through a
dialog.

### Core Functionality and Design

The primary goal of `query-chooser-sk` is to present a summarized view of the
current query and allow users to edit it in a more detailed interface. This is
achieved by:

1.  **Displaying a summary:** The current query is displayed in a concise format
    using the `paramset-sk` element. This gives users a quick overview of the
    active filters.
2.  **Providing an "Edit" button:** This button triggers the display of a
    dialog.
3.  **Embedding `query-sk` in a dialog:** The dialog contains a `query-sk`
    element. This is where the user can interactively build or modify their
    query by selecting values for different parameters.
4.  **Showing query match count:** Alongside the `query-sk` element,
    `query-count-sk` is used to display how many items match the currently
    constructed query. This provides immediate feedback to the user as they
    refine their selection.
5.  **Event propagation:** `query-chooser-sk` listens for `query-change` events
    from the embedded `query-sk` element. When a change occurs,
    `query-chooser-sk` updates its own `current_query` and re-renders,
    effectively propagating the change. It also emits its own `query-change`
    event, allowing parent components to react to query modifications.

This design separates the concerns of displaying the current state from the more
complex interaction of query building. The dialog provides a focused environment
for query modification without cluttering the main UI.

### Key Components and Files

- **`query-chooser-sk.ts`**: This is the core TypeScript file defining the
  `QueryChooserSk` custom element.
  - It manages the visibility of the editing dialog.
  - It orchestrates the interaction between the summary display
    (`paramset-sk`), the query editing interface (`query-sk`), and the match
    count display (`query-count-sk`).
  - It defines properties like `current_query`, `paramset`, `key_order`, and
    `count_url` which are essential for its operation and for configuring
    its child elements.
  - The `_editClick` and `_closeClick` methods handle the opening and
    closing of the dialog.
  - The `_queryChange` method is crucial for reacting to changes in the
    embedded `query-sk` element and updating the `current_query`.
- **`query-chooser-sk.html` (template within `query-chooser-sk.ts`)**: This
  Lit HTML template defines the structure of the element.
  - It includes a `div` with class `row` to display the "Edit" button and
    the `paramset-sk` summary.
  - Another `div` with id `dialog` acts as the container for `query-sk`,
    `query-count-sk`, and the "Close" button. The visibility of this dialog
    is controlled by adding/removing the `display` class.
- **`query-chooser-sk.scss`**: This file provides the styling for the element.
  It ensures proper layout of the button, summary, and the dialog content. It
  also includes theming support.
- **`index.ts`**: A simple entry point that imports and registers the
  `query-chooser-sk` custom element.
- **`query-chooser-sk-demo.html` / `query-chooser-sk-demo.ts`**: These files
  provide a demonstration page for the element, showcasing its usage with
  sample data and event handling. `fetchMock` is used in the demo to simulate
  the `count_url` endpoint.
- **`query-chooser-sk_puppeteer_test.ts`**: Contains Puppeteer tests to verify
  the rendering and basic functionality of the element.

### Workflow: Editing a Query

The typical workflow for a user interacting with `query-chooser-sk` is as
follows:

```
User sees current query summary & "Edit" button
  |
  | (User clicks "Edit")
  V
Dialog appears, showing:
  - `query-sk` (for selecting parameters/values)
  - `query-count-sk` (displaying number of matches)
  - "Close" button
  |
  | (User interacts with `query-sk`, changing selections)
  V
`query-sk` emits "query-change" event
  |
  V
`query-chooser-sk` (_queryChange method):
  - Updates its `current_query` property/attribute
  - Re-renders to reflect new `current_query` in summary & `query-count-sk`
  - Emits its own "query-change" event (for parent components)
  |
  | (User is satisfied with the new query)
  V
User clicks "Close"
  |
  V
Dialog is hidden
  |
  V
`query-chooser-sk` displays the updated query summary.
```

The `paramset` attribute is crucial as it provides the available keys and values
that `query-sk` will use to render its selection interface. The `key_order`
attribute influences the order in which parameters are displayed within
`query-sk`. The `count_url` is passed directly to `query-count-sk` to fetch the
number of matching items for the current query.

# Module: /modules/query-count-sk

The `query-count-sk` module provides a custom HTML element designed to display
the number of results matching a given query. Its primary purpose is to offer a
dynamic and responsive way to inform users about the scope of their queries in
real-time, without requiring a full page reload or complex UI updates. This is
particularly useful in applications where users frequently refine search
criteria and need immediate feedback on the impact of those changes.

The core functionality revolves around the `QueryCountSk` class, which extends
`ElementSk`. This class manages the state of the displayed count, handles
asynchronous data fetching, and updates the UI accordingly.

**Key Components and Design Decisions:**

- **`query-count-sk.ts`:** This is the heart of the module.
  - **Asynchronous Data Fetching:** When the `current_query` or `url`
    attributes change, the element initiates a POST request to the specified
    `url`.
  - The request body includes the `current_query`, and a default time window
    of the last 24 hours (`begin` and `end` timestamps). This design choice
    implies that the element is typically used for querying recent data.
  - To prevent race conditions and unnecessary network requests, any ongoing
    fetch operation is aborted if a new query is initiated. This is achieved
    using an `AbortController`. This is a crucial design decision for
    performance and responsiveness, especially when users rapidly change
    query parameters.
  - The component expects a JSON response with a `count` (number of matches)
    and a `paramset` (a read-only representation of parameters related to
    the query).
  - **State Management:** The `_count` property stores the fetched count as
    a string, and `_requestInProgress` is a boolean flag indicating whether
    a fetch operation is currently active. This flag is used to show/hide a
    loading spinner (`spinner-sk`).
  - **Rendering:** The component uses `lit-html` for efficient template
    rendering. The template displays the `_count` and the `spinner-sk`
    conditionally.
  - **Event Emission:** Upon successful data retrieval, a `paramset-changed`
    custom event is dispatched. This event carries the `paramset` received
    from the server. This allows other components on the page to react to
    changes in the available parameters based on the current query results.
    This decoupling is a key design aspect for building modular UIs.
  - **Error Handling:** Network errors or non-OK HTTP responses are caught,
    and an error message is displayed to the user via the `errorMessage`
    utility (likely from `perf/modules/errorMessage`). AbortErrors are
    handled gracefully by simply stopping the current operation without
    displaying an error, as this usually means the user initiated a new
    action.
- **`query-count-sk.scss`:** Provides styling for the element, ensuring the
  count and spinner are displayed appropriately. The `display: inline-block`
  and flexbox layout for the internal `div` are chosen for simple alignment of
  the count and spinner.
- **`query-count-sk-demo.html` and `query-count-sk-demo.ts`:** These files
  provide a demonstration and testing environment for the `query-count-sk`
  element.
  - The demo sets up a `fetch-mock` to simulate server responses, allowing
    for isolated testing of the component's behavior.
  - It showcases how to instantiate the element and interact with its
    attributes (`url`, `current_query`).
  - The presence of `<error-toast-sk>` in the demo suggests that this is the
    intended mechanism for displaying errors surfaced by `errorMessage`.
- **`index.ts`:** A simple entry point that imports and registers the
  `query-count-sk` custom element, making it available for use in an HTML
  page.

**Workflow for Displaying Query Count:**

1.  **Initialization:**

    - The `query-count-sk` element is added to the DOM.
    - The `url` attribute (pointing to the backend endpoint) is set.

    ```
    Page         query-count-sk
     |                |
     |--(Set url)---->|
    ```

2.  **Query Update:**

    - The `current_query` attribute is set or updated (e.g., by user input in
      another part of the application).

    ```
    Page         query-count-sk
     |                |
     |--(Set current_query)-->|
    ```

3.  **Data Fetching:**

    - The `attributeChangedCallback` (or `connectedCallback` on initial load)
      triggers the `_fetch()` method.
    - If a previous fetch is in progress, it's aborted.
    - `_requestInProgress` is set to `true`, and the spinner becomes visible.
    - A POST request is made to `this.url` with the `current_query` and time
      range.

    ```
    query-count-sk                           Server
        |                                      |
        |--(Set _requestInProgress=true)------>| (Spinner shows)
        |                                      |
        |----(POST / {q: current_query, ...})-->|
    ```

4.  **Response Handling:**

    - **Success:**

      - The server responds with JSON: `{ count: N, paramset: {...} }`.
      - `_count` is updated with `N`.
      - `_requestInProgress` is set to `false` (spinner hides).
      - The component re-renders to display the new count.
      - A `paramset-changed` event is dispatched with the `paramset`.

      ```
      query-count-sk                           Server
          |                                      |
          |<----(HTTP 200, {count, paramset})----|
          |                                      |
          |--(Update _count, _requestInProgress=false)-->| (Spinner hides, count updates)
          |                                      |
          |--(Dispatch 'paramset-changed')------>| (Other components may react)
      ```

    - **Error (e.g., network issue, server error):**

      - `_requestInProgress` is set to `false` (spinner hides).
      - An error message is displayed (e.g., via `error-toast-sk`).

      ```
      query-count-sk                           Server
          |                                      |
          |<----(HTTP Error or Network Error)----|
          |                                      |
          |--(Set _requestInProgress=false)------>| (Spinner hides)
          |                                      |
          |--(Display error message)------------>|
      ```

    - **Abort:**

      - If the fetch was aborted (e.g., new query initiated before
        completion), the `catch` block for `AbortError` is entered.
      - No UI update for count or error display happens; the new fetch
        operation takes precedence.

The design emphasizes responsiveness by aborting stale requests and provides a
clear visual indication of ongoing activity (the spinner). The
`paramset-changed` event promotes loose coupling between components, allowing
other parts of the application to adapt based on the query results without
direct dependencies on `query-count-sk`'s internal implementation.

# Module: /modules/regressions-page-sk

The `regressions-page-sk` module provides a user interface for viewing and
managing performance regressions. It allows users to select a "subscription"
(often representing a team or area of ownership, like "Sheriff Config 1") and
then displays a list of detected performance anomalies (regressions or
improvements) associated with that subscription.

The core functionality revolves around fetching and displaying this data in a
user-friendly way.

**Key Responsibilities and Components:**

- **`regressions-page-sk.ts`**: This is the main TypeScript file that defines
  the `RegressionsPageSk` custom HTML element.

  - **State Management (`State` interface, `stateReflector`)**: The
    component maintains its UI state (selected subscription, whether to show
    triaged items or improvements, and a flag for using a Skia-specific
    backend) in the `state` object. The `stateReflector` utility is crucial
    here. It synchronizes this internal state with the URL query parameters.
    This means a user can bookmark a specific view (e.g., a particular
    subscription with improvements shown) and share it, or refresh the page
    and return to the same state.
  - _Why `stateReflector`?_ It provides a clean way to manage application
    state that needs to be persistent across page loads and shareable via
    URLs, without manually parsing and updating the URL.
  - **Data Fetching (`fetchRegressions`, `init`)**:
  - The `init` method is called during component initialization and whenever
    the state changes significantly (like selecting a new subscription). It
    fetches the list of available subscriptions (sheriff lists) from either
    a legacy endpoint (`/_/anomalies/sheriff_list`) or a Skia-specific one
    (`/_/anomalies/sheriff_list_skia`) based on the `state.useSkia` flag.
    The fetched subscriptions are then sorted alphabetically for display in
    a dropdown.
  - The `fetchRegressions` method is responsible for fetching the actual
    anomaly data. It constructs a query based on the current `state`
    (selected subscription, filters for triaged/improvements, and a cursor
    for pagination). It also chooses between legacy and Skia-specific
    anomaly list endpoints. The fetched anomalies are then appended to the
    `cpAnomalies` array, and if a cursor is returned, a "Show More" button
    is made visible.
  - _Why two sets of endpoints (legacy vs. Skia)?_ This suggests a migration
    path or different data sources/backends being supported, allowing the
    component to adapt based on configuration.
  - **Rendering (`template`, `_render`)**: The component uses `lit-html` for
    templating. The `template` static method defines the HTML structure,
    which includes:
  - A dropdown (`<select id="filter">`) to choose a subscription.
  - Buttons to toggle the display of triaged items and improvements.
  - A `<subscription-table-sk>` to display details about the selected
    subscription and its associated alerts.
  - An `<anomalies-table-sk>` to display the list of anomalies/regressions.
  - Spinners (`spinner-sk`) to indicate loading states.
  - A "Show More" button for paginating through anomalies.
  - The `_render()` method (implicitly called by `ElementSk` when properties
    change) re-renders the component with the latest data.
  - **Event Handling (`filterChange`, `triagedChange`,
    `improvementChange`)**: These methods handle user interactions like
    selecting a subscription or toggling filters. They update the
    component's `state`, trigger `stateHasChanged` (which in turn updates
    the URL and can re-fetch data), and then explicitly call
    `fetchRegressions` and `_render` to reflect the changes.
  - **Legacy Regression Display (`getRegTemplate`, `regRowTemplate`)**:
    There's also code related to displaying `regressions` directly in a
    table within this component (the `regressions` property and
    `getRegTemplate`). However, the primary display of anomalies seems to be
    delegated to `anomalies-table-sk`. This older regression display logic
    might be for a previous version or a specific use case not currently
    active in the demo. The `isRegressionImprovement` static method
    determines if a given regression object represents an improvement based
    on direction and cluster type.

- **`anomalies-table-sk` (external dependency)**: This component is
  responsible for rendering the detailed table of anomalies.
  `regressions-page-sk` fetches the anomaly data and then passes it to
  `anomalies-table-sk` for display. This promotes modularity, separating data
  fetching/management from presentation.

- **`subscription-table-sk` (external dependency)**: This component displays
  information about the currently selected subscription, including any
  configured alerts. Similar to `anomalies-table-sk`, it receives data from
  `regressions-page-sk`.

- **`regressions-page-sk.scss`**: Provides styling for the
  `regressions-page-sk` component, including colors for positive/negative
  changes and styles for spinners and buttons.

- **`regressions-page-sk-demo.html` and `regressions-page-sk-demo.ts`**: These
  files set up a demonstration page for the `regressions-page-sk` component.

  - `regressions-page-sk-demo.ts` is particularly important for
    understanding how the component is intended to be used and tested. It
    initializes a global `window.perf` object with configuration settings
    that the main component might rely on (though direct usage isn't evident
    in `regressions-page-sk.ts` itself, it's a common pattern in Perf).
  - It uses `fetchMock` to simulate API responses for `/users/login/status`,
    `/_/subscriptions`, and `/_/regressions` (which seems to be an older
    endpoint pattern compared to what `regressions-page-sk.ts` uses). This
    mocking is crucial for creating a standalone demo environment.
  - _Why `fetchMock`?_ It allows developers to work on and test the UI
    component without needing a live backend, ensuring predictable data and
    behavior for demos and tests.

**Workflow for Displaying Regressions:**

1.  **Initialization (`connectedCallback`, `init`)**:

    - `regressions-page-sk` element is added to the DOM.
    - `stateReflector` is set up to read initial state from URL or use
      defaults.
    - `init()` is called:
      - Fetches the list of available subscriptions (e.g., "Sheriff Config
        1", "Sheriff Config 2").
      - Populates the subscription dropdown (`<select id="filter">`).

2.  **User Selects a Subscription (`filterChange`)**:

    - User selects "Sheriff Config 2" from the dropdown.
    - `filterChange("Sheriff Config 2")` is triggered.
    - `state.selectedSubscription` is updated to "Sheriff Config 2".
    - `cpAnomalies` is cleared, `anomalyCursor` is reset.
    - `stateHasChanged()` is called, updating the URL (e.g.,
      `?selectedSubscription=Sheriff%20Config%202`).
    - `fetchRegressions()` is called.

3.  **Fetching Anomalies (`fetchRegressions`)**:

    - An API request is made to
      `/_/anomalies/anomaly_list?sheriff=Sheriff%20Config%202` (or the Skia
      equivalent).
    - A loading spinner is shown.
    - The server responds with a list of anomalies and potentially a cursor
      for pagination.

4.  **Displaying Anomalies**:

    - The fetched anomalies are appended to `this.cpAnomalies`.
    - The `subscriptionTable` is updated with subscription details and alerts
      from the response.
    - The `anomaliesTable` (the `anomalies-table-sk` instance) is populated
      with `this.cpAnomalies`.
    - If a cursor was returned, the "Show More" button becomes visible.
    - Loading spinner is hidden.
    - The component re-renders.

    ```
    User Action                      Component State                     API Interaction                      UI Update
    -----------                      ---------------                     ---------------                      ---------
    Page Load
      |
      V
    regressions-page-sk.init()
      |                              state = {selectedSubscription:''}
      V
    fetch('/_/anomalies/sheriff_list') -> ["Sheriff1", "Sheriff2"]
      |                              subscriptionList = ["Sheriff1", "Sheriff2"]
      V
                                                                                                       Populate dropdown
                                                                                                       Disable filter buttons

    Selects "Sheriff1"
      |
      V
    regressions-page-sk.filterChange("Sheriff1")
      |                              state = {selectedSubscription:'Sheriff1', ...}
      |                              (URL updates via stateReflector)
      V
    regressions-page-sk.fetchRegressions()
      |                              anomaliesLoadingSpinner = true
      V
    fetch('/_/anomalies/anomaly_list?sheriff=Sheriff1') -> {anomaly_list: [...], anomaly_cursor: 'cursor123'}
      |                              cpAnomalies = [...], anomalyCursor = 'cursor123', showMoreAnomalies = true
      |                              anomaliesLoadingSpinner = false
      V
                                                                                                       Update anomaliesTable
                                                                                                       Update subscriptionTable
                                                                                                       Show "Show More" button
                                                                                                       Enable filter buttons
    Clicks "Show More"
      |
      V
    regressions-page-sk.fetchRegressions() (called by button click)
      |                              showMoreLoadingSpinner = true
      V
    fetch('/_/anomalies/anomaly_list?sheriff=Sheriff1&anomaly_cursor=cursor123') -> {anomaly_list: [more...], anomaly_cursor: null}
      |                              cpAnomalies = [all...], anomalyCursor = null, showMoreAnomalies = false
      |                              showMoreLoadingSpinner = false
      V
                                                                                                       Update anomaliesTable (append)
                                                                                                       Hide "Show More" button
    ```

5.  **Toggling Filters (e.g., "Show Triaged", `triagedChange`)**:

    - User clicks "Show Triaged".
    - `triagedChange()` is triggered.
    - `state.showTriaged` is toggled.
    - Button text updates (e.g., to "Hide Triaged").
    - `stateHasChanged()` updates the URL (e.g.,
      `?selectedSubscription=Sheriff%20Config%202&showTriaged=true`).
    - `fetchRegressions()` is called again, this time with `triaged=true` in
      the query.
    - The UI updates with the newly filtered list of anomalies.

The design separates concerns: `regressions-page-sk` handles overall page logic,
state, and orchestration of data fetching, while specialized components like
`anomalies-table-sk` and `subscription-table-sk` handle the rendering of
specific data views. The use of `stateReflector` ensures the UI state is
bookmarkable and shareable. The demo files with `fetchMock` are critical for
isolated development and testing of the UI component.

# Module: /modules/report-page-sk

The `report-page-sk` module is designed to display a detailed report page for
performance anomalies. Its primary purpose is to provide users with a
comprehensive view of selected anomalies, including their associated graphs and
commit information, facilitating the analysis and understanding of performance
regressions or improvements.

At its core, the `report-page-sk` element orchestrates the display of several
key pieces of information. It fetches anomaly data from a backend endpoint
(`/_/anomalies/group_report`) based on URL parameters (like revision, anomaly
IDs, bug ID, etc.). This data is then used to populate an `anomalies-table-sk`
element, which presents a tabular view of the anomalies.

A crucial design decision is the use of an `AnomalyTracker` class. This class is
responsible for managing the state of each anomaly, including whether it's
selected (checked) by the user, its associated graph, and the relevant time
range for graphing. This separation of concerns keeps the main `ReportPageSk`
class cleaner and focuses its responsibilities on rendering and user
interaction.

When an anomaly is selected in the table, `report-page-sk` dynamically generates
and displays an `explore-simple-sk` graph for that anomaly. The
`explore-simple-sk` element is configured to show data around the anomaly's
occurrence, typically a week before and after, to provide context. If multiple
anomalies are selected, their graphs are displayed, and their heights are
adjusted to fit the available space. A key feature is the synchronized X-axis
across all displayed graphs, ensuring a consistent time scale for comparison.

The page also attempts to identify and display common commits related to the
selected anomalies. It fetches commit details using the `lookupCids` function
and highlights commits that appear to be "roll" commits (e.g., "Roll repo from
hash to hash"). For these roll commits, it provides a link to the underlying
commit or the parent commit if the roll pattern is not directly parseable from
the commit message, which can be helpful for developers to trace the source of a
change.

**Key Components and Responsibilities:**

- **`report-page-sk.ts`**: This is the main TypeScript file defining the
  `ReportPageSk` custom element.

  - **`ReportPageSk` class**:
  - **Initialization**: Fetches default configurations (`/_/defaults/`) and
    then anomaly data based on URL parameters.
  - **Anomaly Management**: Uses an `AnomalyTracker` instance to manage the
    state of individual anomalies (selected, graphed, time range).
  - **Rendering**: Dynamically renders the `anomalies-table-sk` and
    `explore-simple-sk` graphs based on user interactions and fetched data.
    It uses the `lit-html` library for templating.
  - **Event Handling**: Listens for `anomalies_checked` events from the
    `anomalies-table-sk` to update the displayed graphs. It also handles
    `x-axis-toggled` events from `explore-simple-sk` to synchronize the
    x-axis across multiple graphs.
  - **Graph Generation**: When an anomaly is selected, it creates an
    `explore-simple-sk` instance, configures its query based on the
    anomaly's test path, and sets the appropriate time range.
  - **Commit Information**: Fetches commit details relevant to the anomalies
    and displays a list of common commits, with special handling for "roll"
    commits.
  - **Spinner**: Shows a loading spinner (`spinner-sk`) during data fetching
    operations.
  - **`AnomalyTracker` class**:
  - **State Management**: Stores `AnomalyDataPoint` objects, each containing
    an `Anomaly`, its checked status, its associated `ExploreSimpleSk` graph
    instance (if any), and its `Timerange`.
  - **Loading Data**: Populates its internal tracker from a list of
    anomalies and their corresponding time ranges.
  - **Accessors**: Provides methods to get individual anomaly data,
    set/unset graphs, and retrieve lists of all or selected anomalies. This
    abstraction is key to decoupling the graph display logic from the raw
    anomaly data.
  - **`AnomalyDataPoint` interface**: Defines the structure for storing
    information about a single anomaly within the `AnomalyTracker`.

- **`report-page-sk.scss`**: Contains the SASS/CSS styles for the
  `report-page-sk` element, including styling for the common commits section
  and the dialog for displaying all commits (though the dialog itself is not
  fully implemented in the provided `showAllCommitsTemplate`).

- **Data Fetching Workflow**:

  1.  `ReportPageSk` element is connected to the DOM.
  2.  URL parameters (e.g., `rev`, `anomalyIDs`, `bugID`) are read.
  3.  `fetchAnomalies()` is called.
      - POST request to `/_/anomalies/group_report` with URL parameters in
        the body.
      - Backend responds with `anomaly_list`, `timerange_map`, and
        `selected_keys`.
  4.  `AnomalyTracker` is loaded with this data.
  5.  `anomalies-table-sk` is populated.
  6.  Graphs for initially selected anomalies are rendered.

- **User Interaction Workflow (Selecting an Anomaly)**:

  1.  User checks/unchecks an anomaly in `anomalies-table-sk`.
  2.  `anomalies-table-sk` fires an `anomalies_checked` custom event with the
      anomaly and its checked state.
  3.  `ReportPageSk` listens for this event.
  4.  `updateGraphs()` is called:
      - If checked and no graph exists:
      - `addGraph()` is called.
      - A new `explore-simple-sk` instance is created and configured.
      - The graph is added to the DOM.
      - The `AnomalyTracker` is updated with the new graph instance.
      - If unchecked and a graph exists:
      - The graph is removed from the DOM.
      - The `AnomalyTracker` is updated to remove the graph reference.
  5.  `updateChartHeights()` is called to adjust the height of all visible
      graphs.

The design emphasizes dynamic content loading and interactive exploration. By
using separate custom elements for the table (`anomalies-table-sk`) and graphs
(`explore-simple-sk`), the module maintains a good separation of concerns and
leverages reusable components. The `AnomalyTracker` further enhances this by
encapsulating the state and logic related to individual anomalies.

# Module: /modules/revision-info-sk

The `revision-info-sk` custom HTML element is designed to display information
about anomalies detected around a specific revision. This is particularly useful
for understanding the impact of a code change on performance metrics.

The core functionality revolves around fetching and presenting `RevisionInfo`
objects. A `RevisionInfo` object contains details like the benchmark, bot, bug
ID, start and end revisions of an anomaly, the associated test, and links to
explore the anomaly further.

**Key Components and Workflow:**

1.  **`revision-info-sk.ts`**: This is the main TypeScript file defining the
    `RevisionInfoSk` element.

    - **State Management**: The element maintains its state in a `State`
      object, primarily storing the `revisionId`. It utilizes `stateReflector`
      from `infra-sk/modules/statereflector` to keep the URL in sync with the
      element's state. This allows users to share links that directly open to
      a specific revision's information.
      - `URL change` -> `stateReflector updates State.revisionId` ->
        `getRevisionInfo() is called`
      - `User types revision ID and clicks "Get Revision Information"` ->
        `State.revisionId updated` -> `stateReflector updates URL` ->
        `getRevisionInfo() is called`
    - **Data Fetching (`getRevisionInfo`)**: When a revision ID is provided
      (either via URL or user input), this method is triggered.
      - It displays a spinner (`spinner-sk`) to indicate loading.
      - It makes a `fetch` request to the `/_/revision/?rev=<revisionId>`
        endpoint.
      - The JSON response, an array of `RevisionInfo` objects, is parsed
        using `jsonOrThrow`.
      - The fetched `revisionInfos` are stored, and the UI is re-rendered to
        display the information.
    - **Rendering (`template`, `getRevInfosTemplate`, `revInfoRowTemplate`)**:
      Lit-html is used for templating.
      - The main template (`template`) includes an input field for the
        revision ID, a button to trigger fetching, a spinner, and a
        container for the revision information.
      - `getRevInfosTemplate` generates an HTML table if `revisionInfos` is
        populated. This table includes a header row with a "select all"
        checkbox and columns for bug ID, revision range, master, bot,
        benchmark, and test.
      - `revInfoRowTemplate` renders each individual `RevisionInfo` as a row
        in the table. Each row has a checkbox for selection, a link to the
        bug (if any), a link to explore the anomaly, and the other relevant
        details.
    - **Multi-Graph Functionality**: The element allows users to select
      multiple detected anomaly ranges and view them together on a multi-graph
      page.

      - **Selection**: Checkboxes (`checkbox-sk`) are provided for each
        revision info row and a "select all" checkbox. The `toggleSelectAll`
        method handles the logic for the master checkbox.
      - **`updateMultiGraphStatus`**: This method is called whenever a
        checkbox state changes. It checks if any revisions are selected and
        enables/disables the "View Selected Graph(s)" button accordingly. It
        also updates the `selectAll` state if no individual revisions are
        checked.
      - **`getGraphConfigs`**: This helper function takes an array of
        selected `RevisionInfo` objects and transforms them into an array of
        `GraphConfig` objects. Each `GraphConfig` contains the query string
        associated with the anomaly.
      - **`getMultiGraphUrl`**: This asynchronous method constructs the URL
        for the multi-graph view.

      * It calls `getGraphConfigs` to get the configurations for the
        selected revisions.
      * It calls `updateShortcut` (from `explore-simple-sk`) to generate a
        shortcut ID for the combined graph configurations. This typically
        involves a POST request to `/_/shortcut/update`.
      * It determines the overall time range (`begin` and `end` timestamps)
        encompassing all selected anomalies.
      * It gathers all unique `anomaly_ids` from the selected revisions to
        highlight them on the multi-graph page.
      * It constructs the final URL, including the `begin`, `end`
        timestamps, the `shortcut` ID, the `totalGraphs`, and
        `highlight_anomalies` parameters.

      - **`viewMultiGraph`**: This method is called when the "View Selected
        Graph(s)" button is clicked.

      * It gathers all checked `RevisionInfo` objects.
      * It calls `getMultiGraphUrl` to generate the redirect URL.
      * If a URL is successfully generated, it navigates the current window
        (`window.open(url, '_self')`) to the multi-graph page. If not, it
        displays an error message.

    - **Styling (`revision-info-sk.scss`)**: Provides basic styling for the
      element, such as left-aligning table headers and styling the spinner.

2.  **`index.ts`**: Simply imports and thereby registers the `revision-info-sk`
    custom element.

3.  **Demo Page (`revision-info-sk-demo.html`, `revision-info-sk-demo.ts`,
    `revision-info-sk-demo.scss`)**:

    - Provides a simple HTML page to showcase the `revision-info-sk` element.
    - The `revision-info-sk-demo.ts` file uses `fetch-mock` to mock the
      `/_/revision/` API endpoint. This is crucial for demonstrating the
      element's functionality without needing a live backend. When the demo
      page loads and the user interacts with the element (e.g., enters a
      revision ID '12345'), the mocked response is returned.

**Design Decisions and Rationale:**

- **Custom Element**: Encapsulating this functionality as a custom element
  (`<revision-info-sk>`) promotes reusability across different parts of the
  Perf application or potentially other Skia web applications.
- **State Reflection**: Using `stateReflector` enhances user experience by
  allowing direct navigation to a revision's details via URL and updating the
  URL as the user interacts with the element. This makes sharing and
  bookmarking specific views straightforward.
- **Lit-html for Templating**: Lit-html is chosen for its efficiency and
  declarative approach to building UIs, making the rendering logic concise and
  maintainable.
- **Asynchronous Operations**: Data fetching and shortcut generation are
  asynchronous operations. The use of `async/await` makes the code easier to
  read and manage compared to traditional Promise chaining.
- **Dedicated Multi-Graph URL Generation**: The logic for constructing the
  multi-graph URL is encapsulated in `getMultiGraphUrl`. This separates
  concerns and makes the process of generating the complex URL clearer. It
  relies on the `explore-simple-sk` module's `updateShortcut` function,
  promoting reuse of existing shortcut generation logic.
- **Error Handling**: `jsonOrThrow` is used to simplify error handling for
  fetch requests. The `viewMultiGraph` method also includes basic error
  handling if the URL generation fails.
- **Clear Separation of Concerns**: The element focuses on displaying revision
  information and providing navigation to related views (bug tracker, explore
  page, multi-graph view). It doesn't concern itself with the details of how
  anomalies are detected or how the multi-graph page itself functions.

**Workflow for Displaying Revision Information:**

```
User Interaction / URL Change
       |
       v
[revision-info-sk] stateReflector updates internal 'state.revisionId'
       |
       v
[revision-info-sk] getRevisionInfo() called
       |
       +--------------------------------+
       |                                |
       v                                v
[revision-info-sk] shows spinner     [revision-info-sk] makes fetch request to `/_/revision/?rev=<ID>`
       |                                |
       |                                v
       |                          [Backend] processes request, returns RevisionInfo[]
       |                                |
       |                                v
       +------------------> [revision-info-sk] receives JSON response, parses with jsonOrThrow
                                        |
                                        v
                         [revision-info-sk] stores 'revisionInfos', hides spinner
                                        |
                                        v
                         [revision-info-sk] re-renders using Lit-html templates to display table
```

**Workflow for Viewing Multi-Graph:**

```
User selects one or more revision info rows (checkboxes)
       |
       v
[revision-info-sk] updateMultiGraphStatus() enables "View Selected Graph(s)" button
       |
       v
User clicks "View Selected Graph(s)" button
       |
       v
[revision-info-sk] viewMultiGraph() called
       |
       v
[revision-info-sk] collects selected RevisionInfo objects
       |
       v
[revision-info-sk] calls getMultiGraphUrl(selectedRevisions)
       |
       +------------------------------------------------------+
       |                                                      |
       v                                                      v
[getMultiGraphUrl] calls getGraphConfigs() to create GraphConfig[] [getMultiGraphUrl] calls updateShortcut(GraphConfig[])
       |                                                      | (makes POST to /_/shortcut/update)
       |                                                      v
       |                                                [Backend] returns shortcut ID
       |                                                      |
       +-------------------------------------> [getMultiGraphUrl] constructs final URL (with begin, end, shortcut, anomaly IDs)
                                                              |
                                                              v
                                     [viewMultiGraph] receives the multi-graph URL
                                                              |
                                                              v
                                     [Browser] navigates to the generated multi-graph URL
```

# Module: /modules/split-chart-menu-sk

The `split-chart-menu-sk` module provides a user interface element for selecting
an attribute by which to split a chart. This is particularly useful in data
visualization scenarios where users need to break down aggregated data into
smaller, more specific views. For example, in a performance monitoring
dashboard, a user might want to see performance metrics split by benchmark,
specific test case (story), or sub-component (subtest).

The core functionality revolves around presenting a list of available attributes
to the user in a dropdown menu. These attributes are dynamically derived from
the underlying data. When an attribute is selected, the component emits an
event, allowing other parts of the application to react and update the chart
display accordingly.

**Key Components and Design:**

- **`split-chart-menu-sk.ts`**: This is the main TypeScript file that defines
  the `SplitChartMenuSk` LitElement.

  - **Data Consumption:** The component utilizes the Lit `context` API
    (`@consume`) to access data from two sources: `dataframeContext` and
    `dataTableContext`.
  - `dataframeContext` provides the `DataFrame` (from
    `//perf/modules/json:index_ts_lib` and
    `//perf/modules/dataframe:dataframe_context_ts_lib`). The `DataFrame` is
    the source from which the list of available attributes for splitting is
    derived. This design decouples the menu from the specifics of data
    fetching and management, allowing it to focus solely on the UI aspect of
    attribute selection. The `getAttributes` function (from
    `//perf/modules/dataframe:traceset_ts_lib`) is used to extract these
    attributes.
  - `dataTableContext` provides `DataTable` (also from
    `//perf/modules/dataframe:dataframe_context_ts_lib`). While consumed,
    its direct usage within this specific component's rendering logic isn't
    immediately apparent in the provided `render` method, but it might be
    used by other parts of the application or for future enhancements.
  - **User Interaction:**
  - A Material Design outlined button (`<md-outlined-button>`) labeled
    "Split By" serves as the trigger to open the menu.
  - The menu itself is a Material Design menu (`<md-menu>`), which is
    populated with `<md-menu-item>` elements, one for each attribute
    retrieved from the `DataFrame`.
  - The `menuOpen` state property controls the visibility of the menu.
    Clicking the button toggles this state. The menu also closes itself via
    the `@closed` event.
  - **Event Emission:** When a user clicks on a menu item, the
    `bubbleAttribute` method is called. This method dispatches a custom
    event named `split-chart-selection`.
  - The event detail (`SplitChartSelectionEventDetails`) contains the
    selected `attribute` (a string).
  - The event is configured to bubble (`bubbles: true`) and pass through
    shadow DOM boundaries (`composed: true`), making it easy for ancestor
    elements to listen and react to the selection. This event-driven
    approach is crucial for decoupling the menu from the chart component or
    any other component that needs to know about the selected split
    attribute.
  - **Styling:** Styles are imported from `split-chart-menu-sk.css.ts`
    (`style`). This keeps the component's presentation concerns separate
    from its logic. The styles ensure the component is displayed as an
    inline block and sets a default background color, also styling the
    Material button.

- **`split-chart-menu-sk.css.ts`**: This file defines the CSS styles for the
  component using Lit's `css` tagged template literal. The primary styling
  focuses on the host element's positioning and background, and customizing
  the Material Design button's border radius.

- **`index.ts`**: This file simply imports and registers the
  `split-chart-menu-sk` custom element, making it available for use in HTML.

**Workflow: Selecting a Split Attribute**

1.  **Initialization:**

    - The `split-chart-menu-sk` component is rendered.
    - It consumes the `DataFrame` from the `dataframeContext`.
    - The `getAttributes()` method is called (implicitly via the render
      method's map function) to populate the list of attributes for the menu.

2.  **User Interaction:**

    - User clicks the "Split By" button.
    - `menuClicked` handler is invoked -> `this.menuOpen` becomes `true`.
    - The `<md-menu>` component becomes visible, displaying the list of
      attributes.

    ```
    User          split-chart-menu-sk        DataFrame
    |                    |                       |
    |---Clicks "Split By"->|                       |
    |                    |---Toggles menuOpen=true-->|
    |                    |                       |
    |                    |<--Displays Menu-------|
    |                    |                       |
    ```

3.  **Attribute Selection:**

        - User clicks on an attribute in the menu (e.g., "benchmark").
        - The `click` handler on `<md-menu-item>` calls
          `this.bubbleAttribute("benchmark")`.
        - `bubbleAttribute` creates a `CustomEvent('split-chart-selection', {

    detail: { attribute: "benchmark" } })`. - The event is dispatched.

        ```
        User          split-chart-menu-sk        (Parent Component)
        |                    |                       |
        |---Clicks "benchmark"->|                       |
        |                    |---Calls bubbleAttribute("benchmark")-->|
        |                    |                       |
        |                    |---Dispatches "split-chart-selection" event--> (Listens for event)
        |                    |                       |                       |
        |                    |                       |                       |---Handles event, updates chart
        ```

4.  **Menu Closes:**

    - The `<md-menu>` component emits a `closed` event.
    - The `menuClosed` handler is invoked -> `this.menuOpen` becomes `false`.

This design ensures that `split-chart-menu-sk` is a self-contained, reusable UI
component whose sole responsibility is to provide a way to select a splitting
attribute and communicate that selection to the rest of the application via a
well-defined event. The use of context for data consumption and custom events
for output makes it highly decoupled and easy to integrate.

The demo page (`split-chart-menu-sk-demo.html` and
`split-chart-menu-sk-demo.ts`) demonstrates how to use the component and listen
for the `split-chart-selection` event. The Puppeteer test
(`split-chart-menu-sk_puppeteer_test.ts`) provides a basic smoke test and a
visual regression test by taking a screenshot.

# Module: /modules/subscription-table-sk

The `subscription-table-sk` module provides a custom HTML element designed to
display information about a "subscription" and its associated "alerts". This is
particularly useful in contexts where users need to understand the configuration
of automated monitoring or alerting systems.

The core functionality is encapsulated within the `subscription-table-sk.ts`
file, which defines the `SubscriptionTableSk` custom element. This element is
built using Lit, a library for creating fast, lightweight web components.

**Why and How:**

The primary goal is to present complex subscription and alert data in a
user-friendly and interactive manner. Instead of a static display, this
component allows for toggling the visibility of the detailed alert
configurations. This design choice avoids overwhelming the user with too much
information upfront, providing a cleaner initial view focused on the
subscription summary.

The `SubscriptionTableSk` element takes `Subscription` and `Alert[]` objects as
input. The `Subscription` object contains general information like name, contact
email, revision, bug tracking details (component, hotlists, priority, severity,
CC emails). The `Alert[]` array holds detailed configurations for individual
alerts, including their query parameters, step algorithm, radius, and other
specific settings.

**Key Responsibilities and Components:**

- **`subscription-table-sk.ts`**:
  - **`SubscriptionTableSk` class**: This is the heart of the module. It
    extends `ElementSk`, a base class for Skia custom elements.
  - **Data Handling**: It stores the `subscription` and `alerts` data
    internally.
  - **Rendering Logic (`template` static method)**: It uses Lit's `html`
    tagged template literal to define the structure and content of the
    element. It conditionally renders the subscription details and the
    alerts table based on the available data and the `showAlerts` state.
    - Subscription details are always visible if a subscription is loaded.
    - The alerts table is only rendered if `showAlerts` is true. This
      state is toggled by a button.
  - **`load(subscription: Subscription, alerts: Alert[])` method**: This
    public method is the primary way to feed data into the component. It
    updates the internal state and triggers a re-render.
  - **`toggleAlerts()` method**: This method flips the `showAlerts` boolean
    flag and triggers a re-render, effectively showing or hiding the alerts
    table.
  - **`formatRevision(revision: string)` method**: A helper function to
    display the revision string as a clickable link, pointing to a specific
    configuration file URL. This improves usability by allowing users to
    quickly navigate to the source of the configuration.
  - **`paramset-sk` integration**: For displaying the alert `query`, it
    utilizes the `paramset-sk` element. The `toParamSet` utility function
    (from `infra-sk/modules/query`) is used to convert the query string into
    a format suitable for `paramset-sk`, which then renders it as a
    structured set of key-value pairs. This enhances readability of complex
    query strings.
  - **Styling (`subscription-table-sk.scss`)**: This file defines the visual
    appearance of the element. It uses SCSS and imports styles from shared
    libraries (`themes_sass_lib`, `buttons_sass_lib`, `select_sass_lib`) to
    maintain a consistent look and feel with other Skia elements. The styles
    focus on clear presentation of information, with distinct sections for
    subscription details and the alerts table.

**Workflow: Displaying Subscription and Alerts**

1.  **Initialization**: An instance of `subscription-table-sk` is added to the
    DOM. `<subscription-table-sk></subscription-table-sk>`
2.  **Data Loading**: External code (e.g., in a demo page or a larger
    application) calls the `load()` method on the element instance, passing in
    the `Subscription` object and an array of `Alert` objects.
    `element.load(mySubscriptionData, myAlertsData);`
3.  **Initial Render**:
    - The `SubscriptionTableSk` element updates its internal `subscription`
      and `alerts` properties.
    - `showAlerts` is set to `false` by default upon loading new data.
    - The `_render()` method is called (implicitly by Lit or explicitly).
    - The `template` function generates the HTML:
      - Subscription details (name, email, revision, etc.) are displayed.
      - A button labeled "Show [N] Alert Configurations" is displayed.
      - The alerts table is _not_ rendered yet.
4.  **User Interaction (Toggling Alerts)**: - The user clicks the "Show [N] Alert Configurations" button. - The `click` event triggers the `toggleAlerts()` method. - `showAlerts` becomes `true`. - `_render()` is called again. - The `template` function now also renders the `<table
id="alerts-table">`. - The table header is displayed. - For each `Alert` object in `ele.alerts`: - A table row (`<tr>`) is created. - Cells (`<td>`) display various alert properties (step algorithm,
    radius, k, etc.). - The alert `query` is passed to a `<paramset-sk>` element for
    structured display. - The button label changes to "Hide Alert Configurations".
5.  **Further Toggling**: Clicking the button again will hide the table, and the
    label will revert.

**Diagram: Data Flow and Rendering**

```
External Code ---> subscriptionTableSkElement.load(subscription, alerts)
                     |
                     V
SubscriptionTableSk Internal State:
  - this.subscription = subscription
  - this.alerts = alerts
  - this.showAlerts = false (initially or after load)
                     |
                     V
_render() ------> Lit Template Evaluation
                     |
     -------------------------------------
    |                                     |
    V (if this.subscription is not null)  V (if this.showAlerts is true)
Render Subscription Details             Render Alerts Table
  - Name, Email, Revision (formatted link)  - Iterate through this.alerts
  - Bug info, Hotlists, CCs                 - For each alert:
  - "Show/Hide Alerts" Button                 - Display properties in <td>
                                              - Use <paramset-sk> for query
```

**Demo Page (`subscription-table-sk-demo.html`,
`subscription-table-sk-demo.ts`)**

The demo page serves as an example and testing ground.

- `subscription-table-sk-demo.html`: Sets up the basic HTML structure,
  including instances of `subscription-table-sk` (one for light mode, one for
  dark mode to test theming) and buttons to interact with them. It also
  includes an `error-toast-sk` for displaying potential errors.
- `subscription-table-sk-demo.ts`: Contains JavaScript to:
  - Import and register the `subscription-table-sk` element.
  - Define sample `Subscription` and `Alert` data.
  - Add event listeners to the "Populate Tables" button, which calls the
    `load()` method on the `subscription-table-sk` instances with the sample
    data.
  - Add event listeners to the "Toggle Alerts Table" button, which calls the
    `toggleAlerts()` method on the instances.

This setup allows developers to see the component in action and verify its
functionality with predefined data.

# Module: /modules/test-picker-sk

The `test-picker-sk` module provides a custom HTML element, `<test-picker-sk>`,
designed to guide users in selecting a valid trace or test for plotting. It
achieves this by presenting a series of dependent input fields, where the
options available in each field are dynamically updated based on selections made
in previous fields. This ensures that users can only construct valid
combinations of parameters.

**Core Functionality and Design:**

The primary goal of `test-picker-sk` is to simplify the process of selecting a
specific data series (a "trace" or "test") from a potentially large and complex
dataset. This is often necessary in performance analysis tools where data is
categorized by multiple parameters (e.g., benchmark, bot, specific test,
sub-test variations).

The design enforces a specific order for filling out these parameters. This
hierarchical approach is crucial because the valid options for a parameter often
depend on the values chosen for its preceding parameters.

**Key Components and Responsibilities:**

- **`test-picker-sk.ts`**: This is the heart of the module, defining the
  `TestPickerSk` custom element.

  - **`FieldInfo` class**: This internal class is a simple data structure
    used to manage the state of each individual input field within the
    picker. It stores a reference to the `PickerFieldSk` element, the
    parameter name (e.g., "benchmark", "bot"), and the currently selected
    value.
  - **Dynamic Field Generation (`addChildField`)**: When a value is selected
    in a field, and if there are more parameters in the hierarchy, a new
    `PickerFieldSk` input is dynamically added to the UI. The options for
    this new field are fetched from the backend. This progressive disclosure
    prevents overwhelming the user with too many options at once.
  - **Backend Communication (`callNextParamList`)**: The element interacts
    with a backend endpoint (`/_/nextParamList/`). This endpoint is
    responsible for:

  * Providing the list of valid options for the _next_ input field based on
    the current selections.
  * Returning a count of how many unique traces/tests match the current
    partial or complete selection.

  - **State Management (`_fieldData`, `_currentIndex`)**: The `_fieldData`
    array holds `FieldInfo` objects for each parameter field.
    `_currentIndex` tracks which field is currently active or the next to be
    added.
  - **Event Handling (`value-changed`, `plot-button-clicked`)**:
  - It listens for `value-changed` events from its child `picker-field-sk`
    elements. When a value changes, it triggers logic to update subsequent
    fields and the match count.
  - It emits a `plot-button-clicked` custom event when the user clicks the
    "Add Graph" button. This event includes the fully constructed query
    string representing the selected trace.
  - **Query Population (`populateFieldDataFromQuery`)**: This method allows
    the picker to be initialized with a pre-existing query string. It will
    populate the fields sequentially based on the query parameters. If a
    parameter in the hierarchy is missing or empty in the query, the
    population stops at that point.
  - **Plotting Logic (`onPlotButtonClick`, `PLOT_MAXIMUM`)**: The "Add
    Graph" button is enabled only when the number of matching traces is
    within a manageable range (greater than 0 and less than or equal to
    `PLOT_MAXIMUM`). This prevents users from attempting to plot an
    overwhelming number of traces.
  - **Rendering and UI Updates**: The component uses Lit library for
    templating and re-renders itself when its internal state changes (e.g.,
    new fields added, count updated, request in progress). It also manages
    the enabled/disabled state of input fields during backend requests.

- **`picker-field-sk` (Dependency)**: While not part of this module,
  `test-picker-sk` heavily relies on the `picker-field-sk` element. Each
  parameter in the test picker is represented by an instance of
  `picker-field-sk`. This child component is responsible for displaying a
  label, an input field, and a dropdown menu of selectable options.

- **`test-picker-sk.scss`**: Defines the visual styling for the
  `test-picker-sk` element and its internal components, ensuring a consistent
  look and feel. It styles the layout of the fields, the match count display,
  and the plot button.

**Workflow: User Selecting a Test**

1.  **Initialization (`initializeTestPicker`)**:

    - `test-picker-sk` is given an ordered list of parameter names (e.g.,
      `['benchmark', 'bot', 'test']`) and optional default parameters.
    - `test-picker-sk` -> Backend (`/_/nextParamList/`): Requests options for
      the _first_ parameter (e.g., "benchmark") with an empty query.

    ```
    User Interface:                 Backend:
    [test-picker-sk]
        |
        initializeTestPicker(['benchmark', 'bot', 'test'], {})
        |
        ---> POST /_/nextParamList/ (q="")
                                        |
                                        (Processes request, queries data source)
                                        |
             <--- {paramset: {benchmark: ["b1", "b2"]}, count: 100}
        |
        (Renders first PickerFieldSk for "benchmark" with options "b1", "b2")
        [Benchmark: [select ▼]] [Matches: 100] [Add Graph (disabled)]
    ```

2.  **User Selects a Value**:

    - The user selects "b1" for "benchmark".
    - The `picker-field-sk` for "benchmark" emits a `value-changed` event.
    - `test-picker-sk` -> Backend: Requests options for the _next_ parameter
      ("bot"), now including the selection `benchmark=b1` in the query.

    ```
    User Interface:
    [Benchmark: [b1      ▼]]
        | (value-changed: {value: "b1"})
    [test-picker-sk]
        |
        ---> POST /_/nextParamList/ (q="benchmark=b1")
                                        |
                                        (Processes request, filters based on benchmark=b1)
                                        |
             <--- {paramset: {bot: ["botX", "botY"]}, count: 20}
        |
        (Renders PickerFieldSk for "bot" with options "botX", "botY")
        [Benchmark: [b1      ▼]] [Bot: [select ▼]] [Matches: 20] [Add Graph (disabled)]
    ```

3.  **Process Repeats**: This continues for each parameter in the hierarchy.

4.  **Final Selection and Plotting**:

    - Once all necessary parameters are selected (or the user chooses to
      stop), the match count reflects the number of specific traces.
    - If the count is within the `PLOT_MAXIMUM`, the "Add Graph" button
      enables.
    - User clicks "Add Graph".
    - `test-picker-sk` emits `plot-button-clicked` with the final query (e.g.,
      `benchmark=b1&bot=botX&test=testZ`).

    ```
    User Interface:
    [Benchmark: [b1      ▼]] [Bot: [botX    ▼]] [Test: [testZ   ▼]] [Matches: 5] [Add Graph (enabled)]
        | (User clicks "Add Graph")
    [test-picker-sk]
        |
        emits 'plot-button-clicked' (detail: {query: "benchmark=b1&bot=botX&test=testZ"})
    ```

**Why this Approach?**

- **Guided Selection**: Prevents users from creating invalid or non-existent
  trace combinations.
- **Performance**: By fetching options incrementally, the backend doesn't need
  to return massive lists of all possible values for all parameters at once.
  Queries to the backend are progressively filtered.
- **User Experience**: The interface is less cluttered as fields appear only
  when needed. The match count provides immediate feedback on the specificity
  of the selection.

The `test-picker-sk-demo.html` and `test-picker-sk-demo.ts` files provide a
runnable example of the component, mocking the backend `/_/nextParamList/`
endpoint to showcase its functionality without needing a live backend. This is
essential for development and testing. The Puppeteer and Karma tests
(`test-picker-sk_puppeteer_test.ts`, `test-picker-sk_test.ts`) ensure the
component behaves as expected under various conditions.

# Module: /modules/themes

The `/modules/themes` module is responsible for defining the visual styling and
theming for the application. It builds upon the base theming provided by
`infra-sk` and introduces application-specific overrides and additions.

**Why and How:**

The primary goal of this module is to establish a consistent and branded look
and feel across the application. Instead of defining all styles from scratch, it
leverages the `infra-sk` theming library as a foundation. This promotes code
reuse and ensures that common UI elements have a familiar appearance.

The approach taken is to:

1.  **Import Base Styles:** The `themes.scss` file begins by importing the core
    styles from `../../../infra-sk/themes`. This brings in the foundational
    design system, including color palettes, typography, spacing, and component
    styles.
2.  **Import External Resources:** It also imports the Material Icons font
    library directly from Google Fonts
    (`https://fonts.googleapis.com/icon?family=Material+Icons`). This makes a
    wide range of standard icons readily available for use within the
    application's UI.
3.  **Define Application-Specific Overrides and Additions:** The core principle
    is to only define _deltas_ from the base `infra-sk` theme and global changes
    from `elements-sk` components. This means that `themes.scss` focuses on
    styling aspects that are unique to this specific application or require
    modifications to the default `infra-sk` appearance.

**Key Components and Files:**

- **`themes.scss`**: This is the central SCSS (Sassy CSS) file for the module.

  - **Responsibility:** It orchestrates the application's theme by importing
    base styles, external resources, and defining application-specific
    styling rules.
  - **Implementation Details:**
  - `@import '../../../infra-sk/themes';`: This line incorporates the
    foundational theme from the `infra-sk` library. The relative path
    indicates that `infra-sk` is expected to be a sibling or ancestor
    directory in the project structure.
  - `@import
url('https://fonts.googleapis.com/icon?family=Material+Icons');`: This
    directive pulls in the Material Icons font stylesheet, enabling the use
    of standard Google Material Design icons throughout the application.
  - `body { margin: 0; padding: 0; }`: This is an example of a global
    override. It resets the default browser margins and padding on the
    `<body>` element, providing a cleaner baseline for layout. This is a
    common practice to ensure consistent spacing across different browsers.
    Other application-specific styles would follow this pattern, targeting
    specific elements or defining new CSS classes.

- **`BUILD.bazel`**: This file defines how the `themes.scss` file is processed
  and made available to the rest of the application.

  - **Responsibility:** It uses the `sass_library` rule (defined in
    `//infra-sk:index.bzl`) to compile the SCSS into CSS and declare it as a
    reusable library.
  - **Implementation Details:**
  - `load("//infra-sk:index.bzl", "sass_library")`: Imports the necessary
    Bazel rule for handling SASS compilation.
  - `sass_library(name = "themes_sass_lib", ...)`: Defines a SASS library
    target named `themes_sass_lib`.
    - `srcs = ["themes.scss"]`: Specifies that `themes.scss` is the source
      file for this library.
    - `visibility = ["//visibility:public"]`: Makes this compiled CSS
      library accessible to any other part of the project.
    - `deps = ["//infra-sk:themes_sass_lib"]`: Declares a dependency on
      the `infra-sk` SASS library. This is crucial because `themes.scss`
      imports styles from `infra-sk`. The build system needs to know about
      this dependency to ensure `infra-sk` styles are available during the
      compilation of `themes.scss`.

**Workflow (Styling Application):**

```
Browser Request --> HTML Document
                     |
                     v
                 Link to Compiled CSS (from themes_sass_lib)
                     |
                     v
Application of Styles:
  1. Base browser styles
  2. infra-sk/themes.scss styles (imported)
  3. Material Icons styles (imported)
  4. modules/themes/themes.scss overrides & additions (applied last, taking precedence)
                     |
                     v
                 Rendered Page with Application-Specific Theme
```

In essence, this module provides a layered approach to theming. It starts with a
robust base, incorporates external resources like icon fonts, and then applies
specific customizations to achieve the desired visual identity for the
application. The `BUILD.bazel` file ensures that these SASS files are correctly
processed and made available as CSS to the application during the build process.

# Module: /modules/trace-details-formatter

This module provides a mechanism for formatting trace details and converting
trace strings into query strings. The core idea is to offer a flexible way to
represent and interpret trace information, accommodating different formatting
conventions, particularly for Chrome-specific trace structures.

The "why" behind this module stems from the need to handle various trace
formats. Different systems or parts of the application might represent trace
identifiers (which are essentially a collection of parameters) in distinct ways.
This module centralizes the logic for translating between these representations.
For example, a compact string representation of a trace might be used in URLs or
displays, while a more structured `ParamSet` is needed for querying data.

The "how" is achieved through an interface `TraceFormatter` and concrete
implementations. This allows for different formatting strategies to be plugged
in as needed. The `GetTraceFormatter()` function acts as a factory, returning
the appropriate formatter based on the application's configuration
(`window.perf.trace_format`).

**Key Components/Files:**

- **`traceformatter.ts`**: This is the central file containing the core logic.

  - **`TraceFormatter` interface**: Defines the contract for all trace
    formatters. It mandates two primary methods:
  - `formatTrace(params: Params): string`: Takes a `Params` object (a
    key-value map representing trace parameters) and returns a string
    representation of the trace. This is useful for displaying trace
    identifiers in a user-friendly or system-specific format.
  - `formatQuery(trace: string): string`: Takes a string representation of a
    trace and converts it into a query string (e.g.,
    "key1=value1&key2=value2"). This is crucial for constructing API
    requests to fetch data related to a specific trace.
  - **`DefaultTraceFormatter` class**: Provides a basic implementation of
    `TraceFormatter`.
  - Its `formatTrace` method generates a string like "Trace ID:
    ,key1=value1,key2=value2,...". This is a generic way to represent the
    trace parameters.
  - Its `formatQuery` method currently returns an empty string, indicating
    that this default formatter doesn't have a specific logic for converting
    its trace string representation back into a query.
  - **`ChromeTraceFormatter` class**: Implements `TraceFormatter`
    specifically for traces originating from Chrome's performance
    infrastructure.
  - **Why `ChromeTraceFormatter`?** Chrome's performance data often uses a
    hierarchical, slash-separated string to identify traces (e.g.,
    `master/bot/benchmark/test/subtest_1`). This formatter handles this
    specific convention.
  - **`keys` array**: This private property (`['master', 'bot', 'benchmark',
'test', 'subtest_1', 'subtest_2', 'subtest_3']`) defines the expected
    order of parameters in the Chrome-style trace string. This order is
    significant for both formatting and parsing.
  - **`formatTrace(params: Params): string`**: It iterates through the
    predefined `keys` and constructs a slash-separated string from the
    corresponding values in the input `params`. `Input Params: { master:
"m", bot: "b", benchmark: "bm", test: "t" } keys: [ "master", "bot",
"benchmark", "test", ... ] Output String: "m/b/bm/t"`
  - **`formatQuery(trace: string): string`**: This is the inverse operation.
    It takes a slash-separated trace string, splits it, and maps the parts
    back to the predefined `keys` to build a `ParamSet`. It then converts
    this `ParamSet` into a standard URL query string. - **Handling Statistics (Ad-hoc logic for Chromeperf/Skia bridge)**: A
    special piece of logic exists within `formatQuery` related to
    `window.perf.enable_skia_bridge_aggregation`. If a trace's 'test'
    value ends with a known statistic suffix (e.g., `_avg`, `_count`),
    this suffix is used to determine the `stat` parameter in the output
    query, and the suffix is removed from the 'test' parameter. If no
    such suffix is found, a default `stat` value of 'value' is added.
    This logic is a temporary measure to bridge formatting differences
    between Chromeperf and Skia systems and is intended to be removed
    once Chromeperf is deprecated. `Input Trace String
(enable_skia_bridge_aggregation = true):
"master/bot/benchmark/test_name_max/subtest" Splits into: ["master",
"bot", "benchmark", "test_name_max", "subtest"] Processed ParamSet:
{ master: ["master"], bot: ["bot"], benchmark: ["benchmark"], test:
["test_name"], stat: ["max"], subtest_1: ["subtest"] } Output Query:
"master=master&bot=bot&benchmark=benchmark&test=test_name&stat=max&subtest_1=subtest"`
  - **`STATISTIC_SUFFIX_TO_VALUE_MAP`**: A map used by
    `ChromeTraceFormatter` to translate common statistic suffixes (like
    "avg", "count") found in test names to their corresponding "stat"
    parameter values (like "value", "count").
  - **`traceFormatterRecords`**: A record (object map) that associates
    `TraceFormat` enum values (like `''` for default, `'chrome'` for
    Chrome-specific) with their corresponding `TraceFormatter` instances.
    This acts as a registry for available formatters.
  - **`GetTraceFormatter()` function**: This is the public entry point for
    obtaining a trace formatter. It reads `window.perf.trace_format` (a
    global configuration setting) and returns the appropriate formatter
    instance from `traceFormatterRecords`. If the format is not found, it
    defaults to `DefaultTraceFormatter`.

  ```
  Global Config: window.perf.trace_format = "chrome"
       |
       v
  GetTraceFormatter()
       |
       v
  traceFormatterRecords["chrome"]
       |
       v
  Returns new ChromeTraceFormatter() instance
  ```

- **`traceformatter_test.ts`**: Contains unit tests for the
  `ChromeTraceFormatter`, specifically focusing on the `formatQuery` method
  and its logic for handling statistic suffixes under different configurations
  of `window.perf.enable_skia_bridge_aggregation`.

This module depends on:

- `infra-sk/modules:query_ts_lib`: For the `fromParamSet` function, used to
  convert a `ParamSet` object into a URL query string.
- `perf/modules/json:index_ts_lib`: For type definitions like `Params`,
  `ParamSet`, and `TraceFormat`.
- `perf/modules/paramtools:index_ts_lib`: For the `makeKey` function, used by
  `DefaultTraceFormatter` to create a string representation of a `Params`
  object.
- `perf/modules/window:window_ts_lib`: To access global configuration values
  like `window.perf.trace_format` and
  `window.perf.enable_skia_bridge_aggregation`.

# Module: /modules/triage-menu-sk

The `triage-menu-sk` module provides a user interface element for managing and
triaging anomalies in bulk. It's designed to streamline the process of handling
multiple performance regressions or improvements detected in data.

The core purpose of this module is to allow users to efficiently take action on
a set of selected anomalies. Instead of interacting with each anomaly
individually, this menu provides centralized controls for common triage
operations. This is crucial for workflows where many anomalies might be
identified simultaneously, requiring a quick and consistent way to categorize or
address them.

Key responsibilities and components:

- **`triage-menu-sk.ts`**: This is the heart of the module, defining the
  `TriageMenuSk` custom element.

  - **Anomaly Aggregation**: It receives a list of `Anomaly` objects and
    associated `trace_names`. This allows it to operate on multiple
    anomalies at once.
  - **Action Buttons**: It renders buttons for common triage actions:
  - **"New Bug"**: Triggers the `new-bug-dialog-sk` element, allowing the
    user to create a new bug report associated with the selected anomalies.
  - **"Existing Bug"**: Triggers the `existing-bug-dialog-sk` element,
    enabling the user to link the selected anomalies to an already existing
    bug.
  - **"Ignore"**: Marks the selected anomalies as "Ignored". This is useful
    for anomalies that are deemed not actionable or are false positives.
  - **Nudging Functionality**:
  - The `NudgeEntry` class and related logic (`generateNudgeButtons`,
    `nudgeAnomaly`, `makeNudgeRequest`) allow users to adjust the perceived
    start and end points of an anomaly. This is a subtle but important
    feature for refining the automated anomaly detection. The UI presents a
    set of buttons (e.g., -2, -1, 0, +1, +2) that shift the anomaly's
    boundaries.
  - The `_allowNudge` flag controls whether the nudge buttons are visible,
    allowing for contexts where nudging might not be appropriate (e.g., when
    multiple, disparate anomalies are selected).
  - **State Management**: It maintains the state of the selected anomalies
    (`_anomalies`, `_trace_names`) and the nudge options (`_nudgeList`).
  - **Communication with Backend**: The `makeEditAnomalyRequest` and
    `makeNudgeRequest` methods handle sending HTTP POST requests to the
    `/_/triage/edit_anomalies` endpoint. This endpoint is responsible for
    persisting the triage decisions (bug associations, ignore status, nudge
    adjustments) in the backend database.
  - The `editAction` parameter in `makeEditAnomalyRequest` can take values
    like `IGNORE`, `RESET` (to de-associate bugs), or implicitly associate
    with a bug ID when called from the bug dialogs.
  - **Event Emission**: It emits an `anomaly-changed` custom event. This
    event signals to parent components (likely a component displaying a list
    or plot of anomalies) that one or more anomalies have been modified and
    their representation needs to be updated. The event detail includes the
    affected `traceNames`, the `editAction` performed, and the updated
    `anomalies`.

- **Integration with Dialogs**:

  - It directly embeds and interacts with `new-bug-dialog-sk` and
    `existing-bug-dialog-sk`. When the user clicks "New Bug" or "Existing
    Bug", this element calls the respective `open()` methods on these dialog
    components.
  - It passes the currently selected anomalies and trace names to these
    dialogs using their `setAnomalies` methods, so the dialogs know which
    anomalies the bug report will be associated with.

- **`triage-menu-sk.html` (Implicit via Lit template in `.ts`)**: Defines the
  visual structure of the menu, including the layout of the action buttons and
  the nudge buttons. The rendering is dynamic based on the number of selected
  anomalies and whether nudging is allowed.

- **`triage-menu-sk.scss`**: Provides the styling for the menu, ensuring it
  integrates visually with the surrounding application.

**Key Workflow Example (Ignoring Anomalies):**

1.  **User Selects Anomalies**: In a parent component (e.g., a plot or a list),
    the user selects one or more anomalies.
2.  **`triage-menu-sk` Receives Data**: The parent component calls
    `triageMenuSkElement.setAnomalies(selectedAnomalies,
correspondingTraceNames, nudgeOptions)`.
3.  **Menu Updates**: `triage-menu-sk` re-renders, enabling the "Ignore" button
    (and potentially others). `User Action (Selects Anomalies) --> Parent
Component | v triage-menu-sk.setAnomalies() | v UI Renders (Buttons
enabled)`
4.  **User Clicks "Ignore"**: `User Click ("Ignore") -->
triage-menu-sk.ignoreAnomaly() | v makeEditAnomalyRequest(anomalies, traces,
"IGNORE") | v POST /_/triage/edit_anomalies | (Backend processes) v HTTP 200
OK | v Dispatch "anomaly-changed" event`
5.  **Backend Interaction**: `makeEditAnomalyRequest` is called. It constructs a
    JSON payload with the anomaly keys, trace names, and the action "IGNORE".
    This payload is sent to `/_/triage/edit_anomalies`.
6.  **Event Notification**: Upon a successful response from the backend,
    `triage-menu-sk` updates the local state of the anomalies (setting `bug_id`
    to -2 for ignored anomalies) and dispatches the `anomaly-changed` event.
7.  **Parent Component Reacts**: The parent component listens for
    `anomaly-changed` and updates its display to reflect that the anomalies are
    now ignored (e.g., by changing their color, removing them from an active
    list).

The design decision to have `triage-menu-sk` orchestrate calls to the backend
and then emit a generic `anomaly-changed` event decouples it from the specifics
of how anomalies are displayed. Parent components only need to know that
anomalies have changed and can react accordingly. The use of dedicated dialog
components (`new-bug-dialog-sk`, `existing-bug-dialog-sk`) encapsulates the
complexity of bug reporting, keeping the triage menu itself focused on
initiating these actions.

# Module: /modules/triage-page-sk

## Triage Page (`triage-page-sk`)

The `triage-page-sk` module provides the user interface for viewing and triaging
regressions in performance data. It allows users to filter regressions based on
time range, commit status (all, regressions, untriaged), and alert
configurations. The primary goal is to present a clear overview of regressions
and facilitate the process of identifying their cause and impact.

### Responsibilities and Key Components

The module is responsible for:

- **Fetching and Displaying Regression Data:** It communicates with a backend
  endpoint (`/_/reg/`) to retrieve regression information for a specified time
  range and filter criteria. This data is then rendered in a tabular format,
  showing commits along with any associated regressions.
- **State Management:** The component's state (selected time range, filters)
  is reflected in the URL. This allows users to bookmark specific views or
  share links to particular triage scenarios. The `stateReflector` utility
  from `infra-sk/modules/statereflector` is used for this purpose.
- **User Interaction for Filtering:** It provides UI elements (select
  dropdowns, date range pickers) for users to define what data they want to
  see. Changes to these filters trigger new data fetches.
- **Triage Workflow:** When a user initiates a triage action on a specific
  regression, a dialog (`<dialog>`) containing the `cluster-summary2-sk`
  element is displayed. This dialog allows the user to view details of the
  regression and assign a triage status (e.g., "positive", "negative",
  "acknowledged").
- **Communicating Triage Decisions:** Once a triage status is submitted, the
  module sends this information to a backend endpoint (`/_/triage/`) to
  persist the decision.
- **Displaying Triage Status:** Each regression in the table is visually
  represented by a `triage-status-sk` element, which shows its current triage
  state.

### Key Files

- **`triage-page-sk.ts`**: This is the core TypeScript file defining the
  `TriagePageSk` custom element.
  - **Why**: It encapsulates all the logic for data fetching, rendering,
    state management, and handling user interactions. It leverages Lit for
    templating and rendering the UI.
  - **How**:
  - It defines a `State` interface to manage the component's configuration
    (begin/end timestamps, subset filter, alert filter).
  - The `connectedCallback` initializes the `stateReflector` to synchronize
    the component's state with the URL.
  - `updateRange()` is a crucial method that fetches regression data from
    the `/_/reg/` endpoint whenever the state changes (e.g., date range or
    filter selection). It uses the `fetch` API for network requests.
  - The `template` function (using `lit/html`) defines the HTML structure of
    the component, including the filter controls, the main table displaying
    regressions, and the triage dialog.
  - Event handlers like `commitsChange`, `filterChange`, `rangeChange`,
    `triage_start`, and `triaged` manage user input and interactions with
    child components.
  - The `triage_start` method is triggered when a user wants to triage a
    specific regression. It prepares the data for the `cluster-summary2-sk`
    element and displays the triage dialog.
  - The `triaged` method is called when the user submits a triage decision
    from the `cluster-summary2-sk` dialog. It sends a POST request to
    `/_/triage/` with the triage information.
  - Helper methods like `stepUpAt`, `stepDownAt`, `alertAt`, etc., are used
    to determine how to render cells in the regression table based on the
    data received.
  - `calc_all_filter_options` dynamically generates the list of available
    alert filters based on categories returned from the backend.
- **`triage-page-sk.scss`**: Contains the SASS/CSS styles for the
  `triage-page-sk` element.
  - **Why**: To ensure the component has a consistent and appropriate visual
    appearance within the application.
  - **How**: It defines styles for the layout of the header, filter
    sections, the regression table, and the triage dialog. It imports shared
    styles for buttons, selects, and theming.
- **`triage-page-sk-demo.html` / `triage-page-sk-demo.ts`**: Provide a
  demonstration page for the `triage-page-sk` element.
  - **Why**: To allow developers to see the component in action and test its
    basic functionality in isolation.
  - **How**: The HTML file includes an instance of `<triage-page-sk>`. The
    TypeScript file simply imports the main component to register it.

### Key Workflows

**1. Initial Page Load and Data Fetch:**

```
User navigates to page / URL with state parameters
      |
      V
triage-page-sk.connectedCallback()
      |
      V
stateReflector initializes state from URL (or defaults)
      |
      V
triage-page-sk.updateRange()
      |
      V
FETCH /_/reg/ with current state (begin, end, subset, alert_filter)
      |
      V
Backend responds with RegressionRangeResponse (header, table, categories)
      |
      V
triage-page-sk.reg is updated
      |
      V
triage-page-sk.calc_all_filter_options() (if categories present)
      |
      V
triage-page-sk._render() displays the regression table
```

**2. User Changes Filter or Date Range:**

```
User interacts with <select> (commits/filter) or <day-range-sk>
      |
      V
Event handler (e.g., commitsChange, filterChange, rangeChange) updates this.state
      |
      V
this.stateHasChanged() (triggers stateReflector to update URL)
      |
      V
triage-page-sk.updateRange()
      |
      V
FETCH /_/reg/ with new state
      |
      V
Backend responds with updated RegressionRangeResponse
      |
      V
triage-page-sk.reg is updated
      |
      V
triage-page-sk._render() re-renders the regression table with new data
```

**3. User Initiates Triage:**

```
User clicks on a regression in the table (within a <triage-status-sk> element)
      |
      V
<triage-status-sk> emits 'start-triage' event with details (alert, full_summary, cluster_type)
      |
      V
triage-page-sk.triage_start(event)
      |
      V
this.dialogState is populated with event.detail
      |
      V
this._render() (updates the <cluster-summary2-sk> properties within the dialog)
      |
      V
this.dialog.showModal() (displays the triage dialog)
```

**4. User Submits Triage:**

```
User interacts with <cluster-summary2-sk> in the dialog and clicks "Save" (or similar)
      |
      V
<cluster-summary2-sk> emits 'triaged' event with details (columnHeader, triage status)
      |
      V
triage-page-sk.triaged(event)
      |
      V
Constructs TriageRequest body (cid, triage, alert, cluster_type)
      |
      V
this.dialog.close()
      |
      V
this.triageInProgress = true; this._render() (shows spinner)
      |
      V
FETCH POST /_/triage/ with TriageRequest
      |
      V
Backend responds (e.g., with a bug link if applicable)
      |
      V
this.triageInProgress = false; this._render() (hides spinner)
      |
      V
(Optional) If json.bug exists, window.open(json.bug)
      |
      V
(Implicit) The <triage-status-sk> for the triaged item may update its display, or a full data refresh might be triggered if necessary to show the updated status.
```

### Design Decisions

- **State Reflection in URL:** The decision to reflect the component's state
  (date range, filters) in the URL is crucial for shareability and
  bookmarking. It allows users to return to a specific view of regressions or
  share it with colleagues.
- **Component-Based Architecture:** The page is built using custom elements
  (`triage-page-sk`, `commit-detail-sk`, `day-range-sk`, `triage-status-sk`,
  `cluster-summary2-sk`). This promotes modularity, reusability, and
  separation of concerns. Each component handles a specific piece of
  functionality.
- **Asynchronous Operations:** Data fetching and triage submissions are
  asynchronous operations handled using `fetch` and Promises. Spinners
  (`spinner-sk`) are used to provide visual feedback to the user during these
  operations.
- **Dedicated Triage Dialog:** Instead of inline editing, a modal dialog
  (`<dialog>`) is used for the triage process. This provides a focused
  interface for the user to review cluster details and make a triage decision
  without cluttering the main regression table.
- **Dynamic Filter Options:** The "Which alerts to display" filter options are
  dynamically populated based on the categories returned from the backend.
  This ensures that the filter options are relevant to the current dataset.
- **Use of Lit for Templating:** Lit is used for its efficient rendering and
  declarative templating, making it easier to manage the UI structure and
  updates.

The `triage-page-sk` serves as the central hub for users to actively engage with
and manage performance regressions, making it a critical component in the
performance monitoring workflow.

# Module: /modules/triage-status-sk

The `triage-status-sk` module provides a custom HTML element designed to
visually represent and interact with the triage status of a "cluster" within the
Perf application. A cluster, in this context, likely refers to a group of
related performance measurements or anomalies that require user attention and
classification (triaging).

**Core Functionality & Design:**

The primary purpose of this element is to offer a concise and interactive way
for users to understand the current triage state of a cluster and to initiate
the triaging process.

1.  **Visual Indication:** The element displays a button. The appearance of this
    button (specifically, an icon within it) changes based on the cluster's
    triage status: "positive," "negative," or "untriaged." This provides an
    immediate visual cue to the user.

    - **Why:** Direct visual feedback is crucial for quickly assessing the
      state of many items in a list or dashboard. Instead of reading text,
      users can rely on familiar icons.
    - **How:** It leverages the `tricon2-sk` element to display the
      appropriate icon based on the `triage.status` property. The styling for
      these states is defined in `triage-status-sk.scss`, ensuring visual
      consistency with the application's theme (including dark mode).

2.  **Initiating Triage:** Clicking the button does not directly change the
    triage status within this element. Instead, it emits a custom event named
    `start-triage`.

    - **Why:** This follows a common pattern in web components where
      individual components are responsible for a specific piece of UI and
      interaction, but delegate more complex actions or state management to
      parent components or application-level logic. This keeps the
      `triage-status-sk` element focused and reusable. The actual triaging
      process likely involves a dialog or a more complex UI, which is beyond
      the scope of this simple button.
    - **How:** The `_start_triage` method is invoked on button click. This
      method constructs a `detail` object containing all relevant information
      about the cluster (`full_summary`, current `triage` status, `alert`
      configuration, `cluster_type`, and a reference to the element itself)
      and dispatches the `start-triage` `CustomEvent`.

**Key Components & Files:**

- **`triage-status-sk.ts`:** This is the heart of the module, defining the
  `TriageStatusSk` custom element class which extends `ElementSk`.
  - **Properties:** It manages several key pieces of data as properties:
  - `triage`: An object of type `TriageStatus` (defined in
    `perf/modules/json`) holding the `status` ('positive', 'negative',
    'untriaged') and a `message` string. This is the primary driver for the
    element's appearance.
  - `full_summary`: Potentially detailed information about the cluster, of
    type `FullSummary`.
  - `alert`: Information about any alert configuration associated with the
    cluster, of type `Alert`.
  - `cluster_type`: A string ('high' or 'low'), likely indicating the
    priority or type of the cluster.
  - **Rendering:** It uses `lit-html` for templating
    (`TriageStatusSk.template`). The template renders a `<button>`
    containing a `tricon2-sk` element. The `class` of the button and the
    `value` of the `tricon2-sk` are bound to `ele.triage.status`,
    dynamically changing the appearance.
  - **Event Dispatch:** The `_start_triage` method is responsible for
    creating and dispatching the `start-triage` event.
- **`triage-status-sk.scss`:** Defines the visual styling for the
  `triage-status-sk` element. It includes specific styles for the different
  triage states (`.positive`, `.negative`, `.untriaged`) and their hover
  states, ensuring they integrate with the application's themes (including
  dark mode variables like `--positive`, `--negative`, `--surface`).
- **`index.ts`:** A simple entry point that imports and thereby registers the
  `triage-status-sk` custom element, making it available for use in HTML.
- **`triage-status-sk-demo.html` & `triage-status-sk-demo.ts`:** These files
  provide a demonstration page for the `triage-status-sk` element.
  - The HTML sets up instances of the element in different theme contexts
    (default and dark mode).
  - The TypeScript file demonstrates how to listen for the `start-triage`
    event and how to programmatically set the `triage` property of the
    element. This is crucial for developers to understand how to integrate
    and use the component.
- **`BUILD.bazel`:** Defines how the module is built and its dependencies. It
  specifies `tricon2-sk` as a UI dependency and includes necessary SASS and
  TypeScript libraries.
- **`triage-status-sk_puppeteer_test.ts`:** Contains Puppeteer-based tests to
  ensure the element renders correctly and behaves as expected in a browser
  environment. This is important for maintaining code quality and preventing
  regressions.

**Workflow Example: User Initiates Triage**

```
User sees a triage-status-sk button (e.g., showing an 'untriaged' icon)
    |
    V
User clicks the button
    |
    V
[triage-status-sk.ts] _start_triage() method is called
    |
    V
[triage-status-sk.ts] Creates a 'detail' object with:
                     - triage: { status: 'untriaged', message: '...' }
                     - full_summary: { ... }
                     - alert: { ... }
                     - cluster_type: 'low' | 'high'
                     - element: (reference to itself)
    |
    V
[triage-status-sk.ts] Dispatches a 'start-triage' CustomEvent with the 'detail' object
    |
    V
[Parent Component/Application Logic] Listens for 'start-triage' event
    |
    V
[Parent Component/Application Logic] Receives event.detail
    |
    V
[Parent Component/Application Logic] Uses the received data to:
                                     - Open a triage dialog
                                     - Populate the dialog with cluster details
                                     - Allow user to select a new triage status
                                     - (Potentially) update the original triage-status-sk element's
                                       'triage' property after the dialog interaction is complete.
```

This design allows `triage-status-sk` to be a focused, presentational component,
while the more complex logic of handling the triage process itself is managed
elsewhere in the application. This promotes separation of concerns and
reusability.

# Module: /modules/triage2-sk

The `triage2-sk` module provides a custom HTML element for selecting a triage
status. This element is designed to be a simple, reusable UI component for
indicating whether a particular item is "positive", "negative", or "untriaged".
Its primary purpose is to offer a standardized way to represent and interact
with triage states across different parts of the Perf application.

The core of the module is the `triage2-sk` custom element, defined in
`triage2-sk.ts`. This element leverages the Lit library for templating and
rendering. It presents three buttons, each representing one of the triage
states:

- **Positive:** Indicated by a check circle icon (`<check-circle-icon-sk>`).
- **Negative:** Indicated by a cancel icon (`<cancel-icon-sk>`).
- **Untriaged:** Indicated by a help icon (`<help-icon-sk>`).

The "why" behind this design is to provide a clear visual representation of the
current triage status and an intuitive way for users to change it. By using
distinct icons and styling for each state, the element aims to reduce ambiguity.

**Key Implementation Details:**

- **`triage2-sk.ts`:** This is the main TypeScript file defining the
  `TriageSk` class, which extends `ElementSk`.

  - **State Management:** The current triage state is managed by the `value`
    attribute (and corresponding property). It can be one of "positive",
    "negative", or "untriaged". If no value is provided, it defaults to
    "untriaged".
  - **Event Emission:** When the user clicks a button to change the triage
    state, the element dispatches a custom event named `change`. The
    `detail` property of this event contains the new triage status as a
    string (e.g., "positive"). This allows parent components to react to
    changes in the triage status. `User clicks "Positive" button | V
triage2-sk sets its 'value' attribute to "positive" | V triage2-sk
dispatches a 'change' event with detail: "positive"`
  - **Rendering:** The `template` static method uses Lit's `html` tagged
    template literal to define the structure of the element. It dynamically
    sets the `selected` attribute on the appropriate button based on the
    current `value`.
  - **Attribute Observation:** The element observes the `value` attribute.
    When this attribute changes (either programmatically or through user
    interaction), the `attributeChangedCallback` is triggered, which
    re-renders the component and dispatches the `change` event.
  - **Type Safety:** The `isStatus` function ensures that the `value`
    property is always one of the allowed `Status` types, defaulting to
    "untriaged" if an invalid value is encountered. This contributes to the
    robustness of the component.

- **`triage2-sk.scss`:** This file contains the SASS styles for the
  `triage2-sk` element.

  - **Theming:** It defines styles for both a legacy color scheme and a
    theme-based color scheme (including dark mode). This ensures the
    component integrates visually with the rest of the application,
    regardless of the active theme. The styling differentiates the selected
    button and provides hover effects for better user feedback. The fill
    colors of the icons change based on the triage state (e.g., green for
    positive, red for negative).

- **`index.ts`:** This file serves as the entry point for the module,
  exporting the `TriageSk` class and ensuring the custom element is defined.

- **Demo and Testing:**

  - `triage2-sk-demo.html` and `triage2-sk-demo.ts`: Provide a simple
    demonstration page showcasing the element in various states and how to
    listen for the `change` event. This is useful for manual testing and
    visual inspection.
  - `triage2-sk_test.ts`: Contains Karma unit tests that verify the event
    emission and value changes of the component.
  - `triage2-sk_puppeteer_test.ts`: Includes Puppeteer-based end-to-end
    tests that check the rendering of the component in a browser environment
    and capture screenshots for visual regression testing.

The design choice of using custom elements and Lit allows for a modular and
maintainable component that can be easily integrated into larger applications.
The clear separation of concerns (logic in TypeScript, styling in SASS, and
structure in the template) follows common best practices for web component
development.

# Module: /modules/tricon2-sk

The `tricon2-sk` module provides a custom HTML element `<tricon2-sk>` designed
to visually represent triage states. This component is crucial for user
interfaces where quick identification of an item's status (e.g., in a bug
tracker, code review system, or monitoring dashboard) is necessary.

The core idea is to offer a standardized, reusable icon that clearly
communicates whether an item is "positive," "negative," or "untriaged." This
avoids inconsistencies and reduces cognitive load for users who frequently
interact with such systems.

**Key Components and Responsibilities:**

- **`tricon2-sk.ts`**: This is the heart of the module. It defines the
  `TriconSk` class, which extends `ElementSk` (a base class for custom
  elements in the Skia infrastructure).

  - **Purpose:** To render one of three specific icons based on its `value`
    attribute.
  - **Implementation:**
  - It utilizes the `lit-html` library for templating, allowing for
    efficient rendering and updates.
  - A `static template` function determines which icon to display
    (`check-circle-icon-sk` for "positive", `cancel-icon-sk` for "negative",
    and `help-icon-sk` for "untriaged" or any other value). This design
    centralizes the icon selection logic.
  - The `value` attribute is the primary interface for controlling the
    displayed icon. Changes to this attribute trigger a re-render via
    `attributeChangedCallback` and `_render()`.
  - The `connectedCallback` ensures that the `value` property is properly
    initialized if set before the element is attached to the DOM.
  - **Dependencies:** It imports specific icon components
    (`check-circle-icon-sk`, `cancel-icon-sk`, `help-icon-sk`) from the
    `elements-sk` module, promoting modularity and reuse of existing icon
    assets.

- **`tricon2-sk.scss`**: This file handles the styling of the `tricon2-sk`
  element and its internal icons.

  - **Purpose:** To define the colors of the icons based on their state and
    to ensure they adapt correctly to different themes (e.g., light and dark
    mode).
  - **Implementation:**
  - It uses SASS for more organized and maintainable styles.
  - Crucially, it defines CSS variables (e.g., `--green`, `--red`,
    `--brown`) for the icon fill colors. This allows themes (defined in
    `themes.scss`) to override these colors easily.
  - Specific styles are also provided for when the element is within a
    `.body-sk` context and when `.darkmode` is applied to `.body-sk`. This
    ensures the icons maintain appropriate contrast and visibility across
    different UI themes. The fallback hardcoded colors (`#388e3c`, etc.)
    provide a default styling if CSS variables are not defined by a theme.

- **`index.ts`**: This file serves as the main entry point for the module when
  it's imported. Its sole responsibility is to import `tricon2-sk.ts`, which
  in turn registers the `<tricon2-sk>` custom element. This is a common
  pattern for organizing custom element definitions.

- **`tricon2-sk-demo.html` and `tricon2-sk-demo.ts`**: These files create a
  demonstration page for the `<tricon2-sk>` element.

  - **Purpose:** To showcase the different states of the `tricon2-sk`
    element and how it appears in various theming contexts (default, with
    `colors.css` theming, and with `themes.css` in both light and dark
    modes). This is invaluable for development, testing, and documentation.
  - **How it works:** The HTML file directly uses the `<tricon2-sk>` element
    with different `value` attributes. The accompanying TypeScript file
    simply imports the `index.ts` of the `tricon2-sk` module to ensure the
    custom element is defined before the browser tries to render it.

- **`tricon2-sk_puppeteer_test.ts`**: This file contains automated UI tests
  for the `tricon2-sk` element using Puppeteer.

  - **Purpose:** To verify that the element renders correctly in different
    states and to capture screenshots for visual regression testing.
  - **How it works:** It loads the demo page (`tricon2-sk-demo.html`) in a
    headless browser, checks if the expected number of `tricon2-sk` elements
    are present (a basic smoke test), and then takes a screenshot of the
    page. This ensures that changes to the component's appearance are caught
    early.

**Workflow: Displaying a Triage Icon**

1.  **Usage:** An application includes the `<tricon2-sk>` element in its HTML,
    setting the `value` attribute:

    ```html
    <tricon2-sk value="positive"></tricon2-sk>
    ```

2.  **Element Initialization (`tricon2-sk.ts`):**

    - The `TriconSk` class is instantiated.
    - `connectedCallback` is called, ensuring the `value` property is
      synchronized with the attribute.
    - `_render()` is called.

3.  **Template Selection (`tricon2-sk.ts`):**

    - The `static template` function is invoked.
    - Based on `this.value` (e.g., "positive"), it returns the corresponding
      HTML template: `html<check-circle-icon-sk></check-circle-icon-sk>`.

4.  **Icon Rendering:**

    - The selected icon component (e.g., `<check-circle-icon-sk>`) renders
      itself.

5.  **Styling (`tricon2-sk.scss`):**

                  - CSS rules are applied. For example, if the value is "positive":
                    `tricon2-sk { check-circle-icon-sk { fill: var(--green); // Initially

            attempts to use the CSS variable } }`    - If themes are active (e.g.,`.body-sk.darkmode`), more specific rules

        might override the fill color: `.body-sk.darkmode tricon2-sk {

    check-circle-icon-sk { fill: #4caf50; // Specific dark mode color } }`

**Diagram: Attribute Change leading to Icon Update**

```
[User/Application sets/changes 'value' attribute on <tricon2-sk>]
       |
       v
[<tricon2-sk> element]
       |
       +---------------------+
       | attributeChangedCallback() is triggered |
       +---------------------+
              |
              v
       [this._render()]
              |
              v
       [TriconSk.template(this)]  <-- Reads current 'this.value'
              |
              +-------------+-------------+
              | (value is   | (value is   | (value is other)
              | "positive") | "negative") |
              v             v             v
      [Returns    [Returns    [Returns
       <check-...>] <cancel-...>] <help-...>]
              |
              v
[lit-html updates the DOM with the new icon template]
              |
              v
[Browser renders the new icon with appropriate CSS styles]
```

The design decision to use distinct, imported icon components
(`check-circle-icon-sk`, etc.) rather than, for example, a single SVG sprite or
dynamically generating SVG paths, promotes better separation of concerns. Each
icon can be managed and updated independently. The use of CSS variables for
theming is a standard and flexible approach, allowing consuming applications to
easily adapt the icon colors to their specific look and feel without modifying
the component's core logic or styles directly.

# Module: /modules/trybot

The `trybot` module provides utilities for processing and analyzing results from
Perf trybots. Trybots are automated systems that run performance tests on code
changes before they are submitted. This module focuses on calculating and
presenting metrics that help developers understand the performance impact of
their changes.

The core functionality revolves around aggregating and averaging `stddevRatio`
values across different parameter combinations. The `stddevRatio` is a key
metric representing the change in performance relative to the standard deviation
of the baseline. A positive `stddevRatio` generally indicates a performance
regression, while a negative value suggests an improvement.

The primary goal is to help developers quickly identify which aspects of their
change (represented by key-value parameters like `model=GCE` or
`test=MyBenchmark`) are contributing most significantly to performance changes,
both positive and negative. By grouping results by these parameters and
calculating average `stddevRatio`, the module provides a summarized view that
highlights potential problem areas or confirms expected improvements.

### Key Components and Files:

- **`calcs.ts`**: This file contains the logic for performing calculations on
  trybot results.

  - **`byParams(res: TryBotResponse): AveForParam[]`**: This is the central
    function of the module.

  - **Why**: Developers need a way to understand the overall performance
    impact of their changes across various configurations (e.g., different
    devices, tests, or operating systems). Simply looking at individual
    trace results can be overwhelming. This function provides a summarized
    view by grouping results by their parameters.

  - **How**:

    1.  It takes a `TryBotResponse` object, which contains a list of
        individual test results (`res.results`). Each result includes a
        `stddevRatio` and a set of `params` (key-value pairs describing the
        test configuration).
    2.  It iterates through each result and then through each key-value pair
        within that result's `params`.
    3.  For each unique `key=value` string (e.g., "model=GCE"), it maintains
        a running total of `stddevRatio` values, the count of traces
        contributing to this total (`n`), and counts of traces with positive
        (`high`) or negative (`low`) `stddevRatio`. This aggregation happens
        in the `runningTotals` object.

        ```
        Input TryBotResponse.results:
        [
          { params: {arch: "arm", os: "android"}, stddevRatio: 1.5 },
          { params: {arch: "x86", os: "linux"}, stddevRatio: -0.5 },
          { params: {arch: "arm", os: "ios"}, stddevRatio: 2.0 }
        ]

        -> runningTotals intermediate state (simplified):
           "arch=arm": { totalStdDevRatio: 3.5, n: 2, high: 2, low: 0 }
           "os=android": { totalStdDevRatio: 1.5, n: 1, high: 1, low: 0 }
           "arch=x86": { totalStdDevRatio: -0.5, n: 1, high: 0, low: 1 }
           "os=linux": { totalStdDevRatio: -0.5, n: 1, high: 0, low: 1 }
           "os=ios": { totalStdDevRatio: 2.0, n: 1, high: 1, low: 0 }
        ```

    4.  After processing all results, it calculates the average
        `stddevRatio` for each `key=value` pair by dividing
        `totalStdDevRatio` by `n`.

    5.  It constructs an array of `AveForParam` objects. Each object
        represents a `key=value` parameter and includes its calculated
        average `stddevRatio`, the total number of traces (`n`) that matched
        this parameter, and the counts of high and low `stddevRatio` traces.

    6.  Finally, it sorts this array in descending order based on the
        `aveStdDevRatio`. This crucial step brings the parameters associated
        with the largest (potentially negative) performance regressions to
        the top, making them easy to identify.

  - **`AveForParam` interface**: Defines the structure for the output of
    `byParams`. It holds the aggregated average `stddevRatio` for a specific
    `keyValue` pair, along with counts of traces.

  - **`runningTotal` interface**: An internal helper interface used during
    the aggregation process within `byParams` to keep track of sums and
    counts before the final average is computed.

- **`calcs_test.ts`**: This file contains unit tests for the functions in
  `calcs.ts`.

  - **Why**: To ensure the correctness of the calculation logic, especially
    for edge cases (e.g., empty input) and the core averaging and sorting
    functionality.
  - **How**: It uses `chai` for assertions. Tests cover scenarios like:
  - Empty input to `byParams` should return an empty list.
  - Correct calculation of average `stddevRatio` for multiple traces sharing
    common parameters. For example, if two traces have `test=1`, their
    `stddevRatio` values should be averaged for the `test=1` entry in the
    output.
  - Ensuring the output is correctly sorted by `aveStdDevRatio` in
    descending order.

### Key Workflows/Processes:

**Calculating Average StdDevRatio by Parameter:**

```
TryBotResponse
     |
     v
byParams(response)
     |
     | 1. Initialize `runningTotals` (empty map)
     |
     | 2. For each `result` in `response.results`:
     |    |
     |    |-> For each `param` (key-value pair) in `result.params`:
     |         |
     |         |--> Generate `runningTotalsKey` (e.g., "model=GCE")
     |         |--> Retrieve or create `runningTotal` entry for `runningTotalsKey`
     |         |--> Update `totalStdDevRatio`, `n`, `high`, `low` in the entry
     |
     | 3. Initialize `ret` (empty array of AveForParam)
     |
     | 4. For each `runningTotalKey` in `runningTotals`:
     |    |
     |    |-> Calculate `aveStdDevRatio` = `runningTotal.totalStdDevRatio` / `runningTotal.n`
     |    |-> Create `AveForParam` object
     |    |-> Push to `ret`
     |
     | 5. Sort `ret` by `aveStdDevRatio` (descending)
     |
     v
Array of AveForParam
```

This workflow allows users to quickly pinpoint which configuration parameters
(like specific device models, operating systems, or test names) are associated
with the most significant average performance changes in a given trybot run. The
sorting ensures that the most impactful parameters are immediately visible.

# Module: /modules/trybot-page-sk

The `trybot-page-sk` module provides a user interface for analyzing performance
regressions. It allows users to select either a specific commit from the
repository or a trybot run (representing a potential code change) and then
analyze performance metrics associated with that selection. The core purpose is
to help developers identify and understand performance impacts before or after
code submission.

**Key Responsibilities and Components:**

- **User Input and Selection:**

  - The page is organized into two main tabs: "Commit" and "TryBot". This
    separation allows users to focus on either analyzing historical
    performance data or evaluating the impact of pending changes.
  - **Commit Analysis:** Users can select a specific commit using the
    `commit-detail-picker-sk` element. This allows them to investigate
    performance regressions that might have been introduced by a particular
    code change.
  - **TryBot Analysis:** (The "TryBot" tab is present in the UI template but
    its functionality for selecting trybot runs, CLs, and patch numbers is
    not fully detailed in the provided `trybot-page-sk.ts`. It appears to be
    a planned feature or a more complex interaction than commit selection.)
    The underlying `TryBotRequest` interface includes fields like `cl` and
    `patch_number`, indicating the intent to support this.
  - Once a commit (or eventually a trybot run) is selected, users define the
    scope of the analysis by specifying a query using `query-sk`. This query
    filters the performance traces to be considered (e.g., focusing on
    specific benchmarks, configurations, or architectures).
  - The `paramset-sk` and `query-count-sk` elements provide feedback on the
    current query, showing the matching parameters and the number of traces
    that fit the criteria. This helps users refine their query to target the
    relevant data.

- **Data Fetching and Processing:**

  - When the user clicks the "Run" button, the `run` method is invoked. This
    method constructs a `TryBotRequest` object based on the user's
    selections (commit number, query, or eventually CL/patch details).
  - It sends this request to the `/_/trybot/load/` backend endpoint. This
    endpoint is responsible for fetching the relevant performance data
    (trace values, headers, parameter sets) for the specified commit/trybot
    and query. The `startRequest` utility handles the asynchronous request
    and displays progress using a `spinner-sk`.
  - The response (`TryBotResponse`) contains the performance data,
    including:
  - `results`: An array of individual trace results, each containing
    parameter values (`params`), actual metric values (`values`), and a
    `stddevRatio` (how many standard deviations the trace's value is from
    the median of its historical data).
  - `paramset`: The complete set of parameters found across all returned
    traces.
  - `header`: Information about the data points in each trace, likely
    including timestamps.
  - The received data is then processed. Notably, the `byParams` function
    (from `../trybot/calcs`) is used to aggregate results by parameter
    key-value pairs, calculating average standard deviation ratios, counts,
    and high/low values for each group. This helps identify which parameters
    are most strongly correlated with performance changes.

- **Results Display and Visualization:**

  - The results are presented in two tabs: "Individual" and "By Params".
  - **Individual Tab:**
    - Lists individual traces that match the query, showing their
      parameters, standard deviation ratio, and an option to plot them.
    - To avoid overwhelming the user, only the head and tail of long lists
      are displayed.
    - Clicking the plot icon (`timeline-icon-sk`) for a trace renders its
      values over time on a `plot-simple-sk` element. Users can CTRL-click
      to plot multiple traces on the same graph for comparison.
    - The table intelligently displays parameter values, showing "〃" if a
      value is the same as the row above it and "∅" if a parameter doesn't
      exist for a trace.
  - **By Params Tab:**
    - Displays the aggregated results from the `byParams` calculation. For
      each parameter key-value pair (e.g., "config=gles"), it shows the
      average standard deviation ratio, the number of traces (N) in that
      group, and the highest/lowest individual trace values.
    - This view helps quickly identify which specific parameter values are
      associated with significant performance deviations.
    - Similar to the individual tab, users can click a plot icon to
      visualize a group of traces. Up to `maxByParamsPlot` traces from the
      selected group (sorted by `stddevRatio`) are plotted on a separate
      `plot-simple-sk`.
    - When a trace is focused on the "By Params" plot (e.g., by hovering),
      its full trace ID and its parameter set are displayed below the plot
      using `by-params-traceid` and `by-params-paramset` respectively.
      `paramset-sk` is used to display the parameters, highlighting the
      ones belonging to the focused trace.

- **State Management:**

  - The component uses `stateReflector` to synchronize its internal state
    (`this.state`, which is a `TryBotRequest` object) with the URL. This
    means that the selected commit, query, and analysis type ("commit" or
    "trybot") are reflected in the URL query parameters. This allows users
    to bookmark or share specific analysis views.
  - Changes to the commit selection, query, or tab selection trigger
    `stateHasChanged()`, which updates the URL via `stateReflector` and
    re-renders the component.

- **Styling and Structure:**

  - The `trybot-page-sk.scss` file defines the visual appearance and layout
    of the component, including styles for the query section, results
    tables, and plot areas.
  - The component is built using Lit templates, enabling reactive updates to
    the DOM when the underlying state changes.

**Workflow Example (Commit Analysis):**

1.  **User Selects Tab:** User ensures the "Commit" tab is selected. `[tabs-sk]
--selects index 0--> [TrybotPageSk.tabSelected] --> state.kind = "commit"
--> stateHasChanged()`
2.  **User Selects Commit:** User interacts with `commit-detail-picker-sk`.
    `[commit-detail-picker-sk] --commit-selected event-->
[TrybotPageSk.commitSelected] --> state.commit_number =
selected_commit_offset --> stateHasChanged() --> _render() (UI updates to
show query section)`
3.  **User Enters Query:** User types into `query-sk`.

    ```
    [query-sk] --query-change event--> [TrybotPageSk.queryChange]
        --> state.query = new_query_string
        --> stateHasChanged()
        --> _render() (paramset-sk summary updates)

    [query-sk] --query-change-delayed event--> [TrybotPageSk.queryChangeDelayed]
        --> [query-count-sk].current_query = new_query_string (triggers count update)
    ```

4.  **User Clicks "Run":** `[Run Button] --click--> [TrybotPageSk.run] -->
spinner-sk.active = true --> startRequest('/_/trybot/load/', state, ...) -->
HTTP POST to backend with { kind: "commit", commit_number: X, query: "Y" }
<-- Backend responds with TryBotResponse (trace data, paramset, header) -->
results = TryBotResponse --> byParams = byParams(results) -->
spinner-sk.active = false --> _render() (results tables and plot areas
become visible and populated)`

5.  **User Interacts with Results:**

        - **Plotting Individual Trace:** `[Timeline Icon in Individual Table]

    --click--> [TrybotPageSk.plotIndividualTrace(event, index)] -->
    individualPlot.addLines(...) --> displayedTrace = true --> \_render()
    (individual plot becomes visible)`    - **Plotting By Params Group:**`[Timeline Icon in By Params Table]
    --click--> [TrybotPageSk.plotByParamsTraces(event, index)] --> Filters
    results.results for matching key=value --> byParamsPlot.addLines(...)
    --> byParamsParamSet.paramsets = [ParamSet of plotted traces] -->
    displayedByParamsTrace = true --> \_render() (by params plot and its
    paramset become visible)`    - **Focusing Trace on By Params Plot:**`[by-params-plot] --trace_focused
    event--> [TrybotPageSk.byParamsTraceFocused] -->
    byParamsTraceID.innerText = focused_trace_name -->
    byParamsParamSet.highlight = fromKey(focused_trace_name) --> \_render()
    (updates highlighted params in by-params-paramset)`

The design emphasizes providing both a high-level overview of potential
regression areas (via "By Params") and the ability to drill down into individual
trace performance. The use of `stddevRatio` as a primary metric helps quantify
the significance of observed changes.

# Module: /modules/user-issue-sk

## User Issue Management Element (`user-issue-sk`)

The `user-issue-sk` module provides a custom HTML element for associating and
managing Buganizer issues with specific data points in the Perf application.
This allows users to directly link performance regressions or anomalies to their
corresponding bug reports, enhancing traceability and collaboration.

**Why:** Tracking issues related to performance data is crucial for effective
debugging and resolution. This element centralizes the issue linking process
within the Perf UI, providing a seamless experience for users to add, view, and
remove bug associations.

**How:**

The core functionality revolves around the `UserIssueSk` LitElement class. This
class manages the display and interaction logic for associating a Buganizer
issue with a data point identified by its `trace_key` and `commit_position`.

**Key Responsibilities and Components:**

- **User Authentication:** The element first checks if a user is logged in
  using `alogin-sk`. This is essential because only logged-in users can add or
  remove issue associations. If a user is not logged in, they can only view
  existing issue links.
- **State Management:**
  - `bug_id`: This property determines the element's display.
  - `bug_id === 0`: Indicates no Buganizer issue is associated with the data
    point. The element will display an "Add Bug" button (if the user is
    logged in).
  - `bug_id > 0`: An existing Buganizer issue is linked. The element will
    display a link to the bug and, if the user is logged in, a "close" icon
    to remove the association.
  - `bug_id === -1`: This is a special state where the element renders
    nothing, effectively hiding itself. This might be used in scenarios
    where issue linking is not applicable.
  - `_text_input_active`: A boolean flag that controls the visibility of the
    input field for entering a new bug ID.
- **Rendering Logic:** The `render()` method dynamically chooses between two
  main templates based on the `bug_id` and login status:
  - `addIssueTemplate()`: Shown when `bug_id === 0` and the user is logged
    in. It initially displays an "Add Bug" button. Clicking this button
    reveals an input field for the bug ID and confirm/cancel icons.
  - `showLinkTemplate()`: Shown when `bug_id > 0`. It displays a formatted
    link to the Buganizer issue (using `AnomalySk.formatBug`). If the user
    is logged in, a "close" icon is also displayed to allow removal of the
    issue link.
- **API Interaction:**
  - `addIssue()`: Triggered when a user submits a new bug ID. It makes a
    POST request to the `/_/user_issue/save` endpoint with the `trace_key`,
    `commit_position`, and the new `issue_id`.
  - `removeIssue()`: Triggered when a logged-in user clicks the "close" icon
    next to an existing bug link. It makes a POST request to the
    `/_/user_issue/delete` endpoint with the `trace_key` and
    `commit_position`.
- **Event Dispatching:** After successfully adding or removing an issue, the
  element dispatches a custom event named `user-issue-changed`. This event
  bubbles up and carries a `detail` object containing the `trace_key`,
  `commit_position`, and the new `bug_id`. This allows parent components or
  other parts of the application to react to changes in issue associations
  (e.g., by refreshing a list of user-reported issues).
- **Error Handling:** Uses the `errorMessage` utility from
  `perf/modules/errorMessage` to display feedback to the user in case of API
  errors or invalid input.

**Key Files:**

- **`user-issue-sk.ts`**: This is the heart of the module. It defines the
  `UserIssueSk` LitElement, including its properties, styles, templates, and
  logic for interacting with the backend API and handling user input. The
  design focuses on conditional rendering based on the `bug_id` and user login
  status. The API calls are standard `fetch` requests.
- **`index.ts`**: A simple entry point that imports and registers the
  `user-issue-sk` custom element, making it available for use in HTML.
- **`BUILD.bazel`**: Defines the build dependencies for the element, including
  `alogin-sk` for authentication, `anomaly-sk` for bug link formatting, icon
  elements for the UI, and Lit libraries for web component development.

**Workflows:**

1.  **Adding a New Issue:** User (logged in) sees "Add Bug" button User clicks
    "Add Bug" -> `activateTextInput()` is called -> `_text_input_active` becomes
    `true` -> Element re-renders to show input field, check icon, close icon
    User types bug ID into input field -> `changeHandler()` updates `_input_val`
    User clicks check icon -> `addIssue()` is called -> Input validation (is
    `_input_val` > 0?) -> POST request to `/_/user_issue/save` with trace_key,
    commit_position, input_val -> On success: -> `bug_id` is updated with
    `_input_val` -> `_input_val` reset to 0 -> `_text_input_active` set to
    `false` -> `user-issue-changed` event is dispatched -> Element re-renders to
    show the new bug link and remove icon -> On failure: -> `errorMessage` is
    displayed -> `hideTextInput()` is called (resets state)

2.  **Viewing an Existing Issue:** Element is initialized with `bug_id > 0` ->
    `render()` calls `showLinkTemplate()` -> A link to `perf.bug_host_url +
bug_id` is displayed. -> If user is logged in, a "close" icon is also
    displayed.

3.  **Removing an Existing Issue:** User (logged in) sees bug link and "close"
    icon User clicks "close" icon -> `removeIssue()` is called -> POST request
    to `/_/user_issue/delete` with trace_key, commit_position -> On success: ->
    `bug_id` is set to 0 -> `_input_val` reset to 0 -> `_text_input_active` set
    to `false` -> `user-issue-changed` event is dispatched -> Element re-renders
    to show "Add Bug" button -> On failure: -> `errorMessage` is displayed

The design prioritizes a clear separation of concerns: display logic is handled
by LitElement's templating system, state is managed through properties, and
backend interactions are encapsulated in dedicated asynchronous methods. The use
of custom events allows for loose coupling with other components that might need
to react to changes in issue associations.

# Module: /modules/window

The `window` module is designed to provide utility functions related to the
browser's `window` object, specifically focusing on parsing and interpreting
configuration data embedded within it. This approach centralizes the logic for
accessing and processing global configurations, making it easier to manage and
test.

A key responsibility of this module is to extract and process build tag
information. This information is often embedded in the `window.perf.image_tag`
global variable, which is expected to be an `SkPerfConfig` object (defined in
`//perf/modules/json:index_ts_lib`). The `getBuildTag` function is the primary
component for this task.

The `getBuildTag` function takes an image tag string as input (or defaults to
`window.perf?.image_tag`). Its core purpose is to parse this string and
categorize the build tag. The function employs a specific parsing logic based on
the structure of the image tag:

1.  **Initial Validation**:

    - The function first splits the input tag string by the `@` character.
    - If there are fewer than two parts (i.e., no `@` or `@` is the first/last
      character), it's considered an invalid tag.
    - It then checks if the second part (after `@`) starts with `tag:`. If
      not, it's also an invalid tag.

    ```
    Input Tag String
          |
          V
    Split by '@'
          |
          V
    Check for at least 2 parts AND second part starts with "tag:"
          |
          +-- No --> Invalid Tag
          |
          V
    Proceed to type determination
    ```

2.  **Tag Type Determination**: Based on the prefix of the raw tag (the part
    after `tag:`):

                          - **Git Tag**: If the raw tag starts with `tag:git-`, it's classified as a
                            'git' type. The function extracts the first 7 characters of the Git
                            hash. `rawTag starts with "tag:git-" | V Type: 'git' Tag: First 7 chars

                    of Git hash`    - **Louhi Build Tag**: If the raw tag has a specific length (>= 38

                characters) and contains`louhi`at a particular position (substring
                from index 25 to 30), it's classified as a 'louhi' type. The function
                extracts a 7-character identifier (substring from index 31 to 38) which
                typically represents a hash or version.`rawTag length >= 38 AND

            rawTag[25:30] == "louhi" | V Type: 'louhi' Tag: rawTag[31:38]` - **Regular Tag**: If neither of the above conditions is met, it's

        considered a generic 'tag' type. The function returns the portion of the
        string after`tag:`. `Neither Git nor Louhi | V Type: 'tag' Tag: rawTag

    after "tag:"`

This structured approach ensures that different build tag formats can be
reliably identified and their relevant parts extracted. The decision to
differentiate between 'git', 'louhi', and generic 'tag' types allows downstream
consumers of this information to handle them appropriately. For instance, a
'git' tag might be used to link to a specific commit, while a 'louhi' tag might
indicate a specific build from an internal CI system.

The module also extends the global `Window` interface to declare the `perf:
SkPerfConfig` property. This is a TypeScript feature that provides type safety
when accessing `window.perf`, ensuring that developers are aware of its expected
structure.

The `window_test.ts` file provides unit tests for the `getBuildTag` function,
covering various scenarios including valid git tags, Louhi build tags, arbitrary
tags, and different forms of invalid tags. These tests are crucial for verifying
the correctness of the parsing logic and ensuring that changes to the function
do not introduce regressions. The use of `chai` for assertions is a standard
practice for testing in this environment.

# Module: /modules/word-cloud-sk

The `word-cloud-sk` module provides a custom HTML element designed to visualize
key-value pairs and their relative frequencies. This is particularly useful for
displaying data from clusters or other datasets where understanding the
distribution of different attributes is important.

The core idea is to present this frequency information in an easily digestible
format, combining textual representation with a simple bar graph for each item.
This allows users to quickly grasp the prevalence of certain key-value pairs
within a dataset.

**Key Components and Responsibilities:**

- **`word-cloud-sk.ts`**: This is the heart of the module, defining the
  `WordCloudSk` custom element which extends `ElementSk`.

  - **Why**: It encapsulates the logic for rendering the word cloud. By
    extending `ElementSk`, it leverages common functionalities provided by
    the `infra-sk` library for custom elements.
  - **How**: It uses the `lit-html` library for templating. The `items`
    property, an array of `ValuePercent` objects (defined in
    `//perf/modules/json:index_ts_lib`), is the primary input. Each
    `ValuePercent` object contains a `value` (the key-value string) and a
    `percent` (its frequency).
  - The rendering logic iterates through the `items` and creates a table row
    for each. Each row displays the key-value string, its percentage as
    text, and a horizontal bar whose width is proportional to the
    percentage.
  - The `connectedCallback` ensures that if the `items` property is set
    before the element is fully connected to the DOM, it's properly upgraded
    and the element is rendered.
  - The `_render()` method is called whenever the `items` property changes,
    ensuring the display is updated.

- **`word-cloud-sk.scss`**: This file contains the SASS styles for the
  `word-cloud-sk` element.

  - **Why**: It provides the visual appearance of the word cloud, ensuring
    it's readable and visually distinct.
  - **How**: It defines styles for the table, table cells, and the
    percentage bar. It uses CSS variables for theming (e.g., `--light-gray`,
    `--on-surface`, `--primary`), allowing the component to adapt to
    different themes (like light and dark mode) defined in
    `//perf/modules/themes:themes_sass_lib` and
    `//elements-sk/modules:colors_sass_lib`.
  - Specific styles are applied for font family, size, padding, borders, and
    the background color and height of the percentage bar.

- **`word-cloud-sk-demo.html` and `word-cloud-sk-demo.ts`**: These files
  provide a demonstration page for the `word-cloud-sk` element.

  - **Why**: They serve as a live example of how to use the component and
    allow for easy visual testing and development.
  - **How**: `word-cloud-sk-demo.html` includes multiple instances of the
    `<word-cloud-sk>` tag, some within sections with different theming
    (e.g., dark mode). `word-cloud-sk-demo.ts` then selects these instances
    and populates their `items` property with sample data. This demonstrates
    how the component can be instantiated and how data is passed to it.

- **`index.ts`**: This file simply imports and thereby registers the
  `word-cloud-sk` custom element.

  - **Why**: It acts as the entry point for the element, ensuring it's
    defined when the module is imported.

**Workflow: Data Display**

The primary workflow involves providing data to the `word-cloud-sk` element and
its subsequent rendering:

1.  **Instantiation**: An instance of `<word-cloud-sk>` is created in HTML.

    ```
    <word-cloud-sk></word-cloud-sk>
    ```

2.  **Data Provision**: The `items` property of the element is set with an array
    of `ValuePercent` objects.

    ```
    // In JavaScript/TypeScript:
    const wordCloudElement = document.querySelector('word-cloud-sk');
    wordCloudElement.items = [
      { value: 'arch=x86', percent: 100 },
      { value: 'config=565', percent: 60 },
      // ... more items
    ];
    ```

3.  **Rendering (`_render()` called in `word-cloud-sk.ts`)**:

    - The `WordCloudSk` element iterates through the `_items` array.
    - For each item:
      - A table row (`<tr>`) is generated.
      - The `item.value` is displayed in the first cell (`<td>`).
      - The `item.percent` is displayed as text (e.g., "60%") in the second
        cell.
      - A `<div>` element is created in the third cell. Its `width` style is
        set to `item.percent` pixels, creating a visual bar representation
        of the percentage.

    The overall structure rendered looks like this (simplified):

    ```
    <table>
      <tr> <!-- For item 1 -->
        <td class="value">[item1.value]</td>
        <td class="textpercent">[item1.percent]%</td>
        <td class="percent">
          <div style="width: [item1.percent]px"></div>
        </td>
      </tr>
      <tr> <!-- For item 2 -->
        <td class="value">[item2.value]</td>
        <td class="textpercent">[item2.percent]%</td>
        <td class="percent">
          <div style="width: [item2.percent]px"></div>
        </td>
      </tr>
      <!-- ... more rows -->
    </table>
    ```

This process ensures that whenever the input data changes, the visual
representation of the word cloud is automatically updated. The use of CSS
variables for styling allows the component to seamlessly integrate into
applications with different visual themes.

# Module: /nanostat

## Nanostat

`nanostat` is a command-line tool designed to compare and analyze the results of
Skia's nanobench benchmark. It takes two JSON files generated by nanobench as
input, representing "old" and "new" benchmark runs, and provides a statistical
summary of the performance changes between them. This is particularly useful for
developers to understand the performance impact of their code changes.

### Why it exists

When making changes to a codebase, especially one as performance-sensitive as a
graphics library like Skia, it's crucial to measure the impact on performance.
Nanobench produces detailed raw data, but interpreting this data directly can be
cumbersome. `nanostat` was created to:

1.  **Automate Statistical Analysis:** Apply statistical tests (Mann-Whitney U
    test or Welch's T-test) to determine if observed differences in benchmark
    results are statistically significant or likely due to random variation.
2.  **Summarize Changes:** Present a concise, human-readable summary of
    performance changes, highlighting significant regressions or improvements.
3.  **Facilitate Quick Comparisons:** Enable developers to quickly compare
    benchmark runs before and after a code change, streamlining the performance
    analysis workflow.
4.  **Provide Filtering and Sorting:** Offer options to filter out insignificant
    changes, remove outliers, and sort results based on various criteria (e.g.,
    by the magnitude of change or by test name).

### How it works

The core workflow of `nanostat` involves several steps:

1.  **Input:** It accepts two file paths as command-line arguments, pointing to
    the "old" and "new" nanobench JSON output files.

    ```
    nanostat [options] old.json new.json
    ```

2.  **Parsing:** The `loadFileByName` function in `main.go` is responsible for
    opening and parsing these JSON files. It uses the
    `perf/go/ingest/format.ParseLegacyFormat` function to interpret the
    nanobench output structure and then
    `perf/go/ingest/parser.GetSamplesFromLegacyFormat` to extract the raw sample
    values for each benchmark test. Each file's data is converted into a
    `parser.SamplesSet`, which is a map where keys are test identifiers and
    values are slices of performance measurements (samples).

3.  **Statistical Analysis:** The `samplestats.Analyze` function (from the
    `perf/go/samplestats` module) is the heart of the comparison. It takes the
    two `parser.SamplesSet` (before and after samples) and a
    `samplestats.Config` object as input. The configuration includes:

    - `Alpha`: The significance level (default 0.05). A p-value below alpha
      indicates a significant difference.
    - `IQRR`: A boolean indicating whether to apply the Interquartile Range
      Rule to remove outliers from the sample data before analysis.
    - `All`: A boolean determining if all results (significant or not) should
      be displayed.
    - `Test`: The type of statistical test to perform (Mann-Whitney U test or
      Welch's T-test).
    - `Order`: The function used to sort the output rows.

    For each common benchmark test found in both input files,
    `samplestats.Analyze` calculates statistics for both sets of samples (mean,
    percentage deviation) and then performs the chosen statistical test to
    compare the two distributions. This yields a p-value.

4.  **Filtering and Sorting:** Based on the `config`, `samplestats.Analyze`
    filters out rows where the change is not statistically significant (if
    `config.All` is false). The remaining rows are then sorted according to
    `config.Order`.

5.  **Output Formatting:** The `formatRows` function in `main.go` takes the
    analyzed and sorted `samplestats.Row` data and prepares it for display.

    - It identifies "important keys" from the benchmark parameters (e.g.,
      `config`, `name`, `test`). These are keys whose values differ across the
      benchmark results, helping to distinguish them.
    - It constructs a header line for the output table.
    - For each row of results, it formats the old and new means, standard
      deviations, the percentage delta, the p-value, sample sizes, and the
      important key values.
    - If a change is not significant (p-value > alpha), the delta is shown as
      "~" unless the `--all` flag is used.
    - The formatted strings are then printed to `stdout` using
      `text/tabwriter` to create a well-aligned table.

    Example output line:

    ```
             old          new  delta     stats            name
      2.15 ±  5%   2.00 ±  2%   -7%   (p=0.001, n=10+ 8)  tabl_digg.skp
    ```

### Key Components and Files

- **`main.go`**: This is the entry point of the application.

  - **Responsibilities**:
  - Parses command-line arguments and flags (`-alpha`, `-sort`, `-iqrr`,
    `-all`, `-test`).
  - Validates user input and displays usage information if necessary.
  - Calls `loadFileByName` to load and parse the input JSON files.
  - Constructs the `samplestats.Config` based on the provided flags.
  - Invokes `samplestats.Analyze` to perform the statistical comparison.
  - Calls `formatRows` to format the results for display.
  - Uses `text/tabwriter` to print the formatted output to the console.
  - **Key functions**:
  - `actualMain(stdout io.Writer)`: Contains the main logic, allowing
    `stdout` to be replaced for testing.
  - `loadFileByName(filename string) parser.SamplesSet`: Reads a nanobench
    JSON file, parses it, and extracts the performance samples. It leverages
    `perf/go/ingest/format` and `perf/go/ingest/parser`.
  - `formatRows(config samplestats.Config, rows []samplestats.Row)
[]string`: Takes the analysis results and formats them into a slice of
    strings, ready for tabular display. It intelligently includes relevant
    parameter keys in the output.

- **`main_test.go`**: Contains unit tests for `nanostat`.

  - **Responsibilities**:
  - Ensures that `nanostat` produces the expected output for various
    command-line flag combinations and input files.
  - Uses golden files (`testdata/*.golden`) to compare actual output against
    expected output.
  - **Key functions**:
  - `TestMain_DifferentFlags_ChangeOutput(t *testing.T)`: The main test
    function that sets up different test cases.
  - `check(t *testing.T, name string, args ...string)`: A helper function
    that runs `nanostat` with specified arguments, captures its output, and
    compares it against a corresponding golden file.

- **`README.md`**: Provides user-facing documentation on how to install and
  use `nanostat`, including examples and descriptions of command-line options.

- **`Makefile`**: Contains targets for building, testing, and regenerating
  test data (golden files). The `regenerate-testdata` target is crucial for
  updating the golden files when the tool's output format or logic changes.

- **`BUILD.bazel`**: Defines how to build and test the `nanostat` binary and
  its library using the Bazel build system. It lists dependencies on other
  Skia modules, such as:

  - `//go/paramtools`: Used in `formatRows` to work with parameter sets from
    benchmark results.
  - `//perf/go/ingest/format`: Used for parsing the legacy nanobench JSON
    format.
  - `//perf/go/ingest/parser`: Used to extract sample data from the parsed
    format.
  - `//perf/go/samplestats`: Provides the core statistical analysis
    functions (`samplestats.Analyze`, `samplestats.Order`,
    `samplestats.Test`).

### Dependencies and Design Choices

- **`perf/go/samplestats`**: `nanostat` heavily relies on this module for the
  actual statistical computations. This promotes code reuse and separation of
  concerns, keeping `nanostat` focused on command-line parsing, file I/O, and
  output formatting.
- **`perf/go/ingest/format` and `perf/go/ingest/parser`**: These modules
  handle the complexities of interpreting the nanobench JSON structure,
  abstracting this detail away from `nanostat`'s main logic.
- **Command-line Flags**: The tool offers a range of flags to customize its
  behavior (`-alpha`, `-iqrr`, `-all`, `-sort`, `-test`). This flexibility
  allows users to tailor the analysis to their specific needs. For example,
  the `-iqrr` flag allows for more robust analysis by removing potential
  outlier data points that could skew results. The `-test` flag allows users
  to choose between parametric (T-test) and non-parametric (U-test)
  statistical tests, depending on the assumptions they are willing to make
  about their data's distribution.
- **Tabular Output**: Using `text/tabwriter` provides a clean, aligned, and
  easy-to-read output format, which is essential for quickly scanning and
  understanding the performance changes.
- **Golden File Testing**: The use of golden files in `main_test.go` is a good
  practice for testing command-line tools. It makes it easy to verify that
  changes to the code don't unintentionally alter the output format or the
  results of the analysis. The `Makefile` target `regenerate-testdata`
  simplifies updating these files when intended changes occur.

# Module: /pages

The `/pages` module is responsible for defining the HTML structure and initial
JavaScript and CSS for all the user-facing pages of the Skia Performance
application. Each page represents a distinct view or functionality within the
application, such as viewing alerts, exploring performance data, or managing
regressions.

The core design philosophy is to keep the HTML files minimal and delegate the
rendering and complex logic to custom HTML elements (Skia Elements). This
promotes modularity and reusability of UI components.

**Key Components and Responsibilities:**

- **HTML Files (e.g., `alerts.html`, `newindex.html`):**
  - These files serve as the entry point for each page.
  - They define the basic HTML structure (`<head>`, `<body>`).
  - Crucially, they include a `perf-scaffold-sk` custom element. This
    element acts as a common layout wrapper for all pages, providing
    consistent navigation, header, footer, and potentially other shared UI
    elements.
  - Inside the `perf-scaffold-sk`, they embed the primary custom element
    specific to that page's functionality (e.g., `<alerts-page-sk>`,
    `<explore-sk>`).
  - They include Go template placeholders like `{%- template
"googleanalytics" . -%}` and `{% .Nonce %}` for server-side rendering of
    common snippets and security nonces.
  - A `window.perf = {%.context %};` script tag is used to pass initial data
    or configuration from the server (Go backend) to the client-side
    JavaScript. This context likely contains information needed by the
    page-specific custom element to initialize itself.
- **TypeScript Files (e.g., `alerts.ts`, `newindex.ts`):**
  - These files are the JavaScript entry points for each page.
  - Their primary responsibility is to import the necessary custom elements.
    This ensures that the browser knows how to render elements like
    `<perf-scaffold-sk>` and the page-specific custom element (e.g.,
    `../modules/alerts-page-sk`).
  - By importing these elements, their associated JavaScript logic is
    executed, making them functional.
- **SCSS Files (e.g., `alerts.scss`, `newindex.scss`):**
  - These files provide page-specific styling.
  - Currently, they all primarily `@import 'body';`, which means they
    inherit base body styles from `body.scss`.
  - If a page required unique styling beyond what the custom elements or
    `body.scss` provide, those styles would be defined here.
- **`body.scss`:**
  - This file defines global, minimal styles for the `<body>` element, such
    as removing default margins and padding. This ensures a consistent
    baseline across all pages.
- **`BUILD.bazel`:**
  - This file defines how each page is built using the `sk_page` rule from
    `//infra-sk:index.bzl`.
  - For each page, it specifies:
  - `html_file`: The entry HTML file.
  - `ts_entry_point`: The entry TypeScript file.
  - `scss_entry_point`: The entry SCSS file.
  - `sk_element_deps`: A list of dependencies on other modules that provide
    the custom HTML elements used by the page. This is crucial for ensuring
    that elements like `perf-scaffold-sk` and page-specific elements (e.g.,
    `alerts-page-sk`) are compiled and available.
  - `sass_deps`: Dependencies for SCSS, typically including `:body_sass_lib`
    which refers to the `body.scss` file.
  - Other build-related configurations like `assets_serving_path`, `nonce`,
    and `production_sourcemap`.

**Workflow for a Page Request:**

1.  User navigates to a URL (e.g., `/alerts`).
2.  The server (Go backend) maps this URL to the corresponding HTML file (e.g.,
    `alerts.html`).
3.  The Go backend processes the HTML template, injecting data for `{% .context
%}`, the `{% .Nonce %}`, and other templates like "googleanalytics" and
    "cookieconsent".
4.  The processed HTML is sent to the browser. `User Request ----> Go Backend
----> Template Processing (alerts.html + context) ----> HTML Response (URL
Routing) (Injects window.perf data, nonce)`
5.  The browser parses the HTML.
6.  When the browser encounters `<script src="alerts.js"></script>` (or the
    equivalent generated by the build system), it fetches and executes
    `alerts.ts`.
7.  `alerts.ts` imports `../modules/perf-scaffold-sk` and
    `../modules/alerts-page-sk`. This registers these custom elements with the
    browser. `Browser Receives HTML -> Parses HTML -> Encounters <script> for
alerts.ts | -> Fetches and Executes alerts.ts | -> import
'../modules/perf-scaffold-sk'; -> import '../modules/alerts-page-sk';
(Custom elements are now defined)`
8.  The browser then renders the custom elements (`<perf-scaffold-sk>` and
    `<alerts-page-sk>`). The JavaScript logic within these custom elements takes
    over, potentially fetching more data via AJAX using the initial
    `window.perf` context if needed, and populating the page content. `Custom
Elements Registered -> Browser renders <perf-scaffold-sk> and
<alerts-page-sk> | -> JavaScript within these elements executes (e.g., reads
window.perf, makes AJAX calls, builds UI)`
9.  The SCSS file (`alerts.scss`) is also linked in the HTML (via the build
    system), and its styles (including those from `body.scss`) are applied.

This structure allows for a clean separation of concerns:

- HTML provides the basic skeleton and server-side data injection points.
- TypeScript/JavaScript (via custom elements) handles all dynamic behavior, UI
  rendering, and interaction logic.
- SCSS handles the styling.

The `help.html` page is slightly different as it directly embeds more static
content (help text and examples) within its HTML structure using Go templating
(`{% range ... %}`). However, it still utilizes the `perf-scaffold-sk` for
consistent page layout and imports its JavaScript for any scaffold-related
functionalities.

The `newindex.html` and `multiexplore.html` pages additionally include a `div`
with `id="sidebar_help"` within the `perf-scaffold-sk`. This suggests that the
`perf-scaffold-sk` might have a designated area or slot where page-specific help
content can be injected, or that the page-specific JavaScript (`explore-sk.ts`
or `explore-multi-sk.ts`) might dynamically populate or interact with this
sidebar content.

# Module: /res

## Resource Module (`/res`)

### High-Level Overview

The `/res` module serves as a centralized repository for static assets required
by the application. Its primary purpose is to provide a consistent and organized
location for resources such as images, icons, and potentially other static files
that are part of the user interface or overall application branding. By
co-locating these assets, the module simplifies resource management, facilitates
easier updates, and ensures that all parts of the application can reliably
access necessary visual or static elements.

### Design Decisions and Implementation Choices

The decision to have a dedicated `/res` module stems from the need to separate
static content from dynamic code. This separation offers several benefits:

1.  **Organization:** Grouping all static assets in one place makes the project
    structure cleaner and easier to navigate. Developers know exactly where to
    look for or add new resources.
2.  **Maintainability:** When assets need to be updated (e.g., a new logo, a
    changed icon), modifications are localized to this module, reducing the risk
    of inadvertently affecting other parts of the codebase.
3.  **Build Process Optimization:** Build tools can often be configured to
    handle static assets differently (e.g., copying them directly to the output
    directory, optimizing images). Having a dedicated module simplifies the
    configuration of such processes.
4.  **Caching and Delivery:** Web servers and content delivery networks (CDNs)
    can be more effectively configured to cache and serve static assets when
    they are located in a well-defined directory.

The internal structure of `/res` is designed to categorize different types of
assets. For instance, images are placed within a dedicated `img` subdirectory.
This categorization aids in discoverability and allows for type-specific
processing or handling if needed in the future.

### Key Components/Files/Submodules

- **`/res/img` (Submodule/Directory):**
  - **Responsibility:** This submodule is dedicated to storing all image
    assets used by the application. This includes logos, icons, background
    images, and any other visual elements that are not dynamically
    generated.
  - **Why:** Separating images into their own directory within `/res` keeps
    the root of the resource module clean and allows for specific
    image-related build optimizations or management strategies. For example,
    image compression tools or sprite generation scripts could target this
    directory specifically.
  - **Key Files:**
  - **`/res/img/favicon.ico`:**
    - **Responsibility:** This specific file provides the "favorite icon"
      or "favicon" for the application. Web browsers display this icon in
      various places, such as the browser tab, bookmarks bar, and address
      bar history. It's a small but important branding element that helps
      users quickly identify the application among many open tabs or saved
      links.
    - **Why:** The `.ico` format is the traditional and most widely
      supported format for favicons, ensuring compatibility across
      different browsers and platforms. Placing it directly in the `img`
      directory makes it easily discoverable by build tools and web
      servers, which often look for `favicon.ico` in standard locations.
      Its presence here ensures that the application has a visual
      identifier in browser contexts.

### Workflows and Processes

A typical workflow involving the `/res` module might look like this:

1.  **Asset Creation/Acquisition:** A designer creates a new icon or a new
    version of the application logo.

    ```
    Designer           Developer
       |                   |
    [New Image Asset] --> [Receives Asset]
    ```

2.  **Asset Placement:** The developer places the new image file (e.g.,
    `new_icon.png`) into the appropriate subdirectory within `/res`, likely
    `/res/img/`.

    ```
    Developer
       |
    [Places new_icon.png into /res/img/]
    ```

3.  **Referencing the Asset:** Application code (e.g., HTML, CSS, JavaScript)
    that needs to display this icon will reference it using a path relative to
    how the assets are served.

    ```
    Application Code (e.g., HTML)
       |
    <img src="/path/to/res/img/new_icon.png">
    ```

    _(Note: The exact `/path/to/` depends on how the web server or build system
    exposes the `/res` directory.)_

4.  **Build Process:** During the application build, files from the `/res`
    module are typically copied to a public-facing directory in the build
    output.

    ```
    Build System
       |
    [Reads /res/img/new_icon.png] --> [Copies to /public_output/img/new_icon.png]
    ```

5.  **Client Request:** When a user accesses the application, their browser
    requests the asset. `User's Browser Web Server | | [Requests
/public_output/img/new_icon.png] ----> [Serves new_icon.png] | | [Displays
new_icon.png] <------------------------+`

This workflow highlights how the `/res` module acts as the source of truth for
static assets, which are then processed and served to the end-user. The
`favicon.ico` follows a similar, often more implicit, path as browsers
automatically request it from standard locations.

# Module: /samplevariance

The `samplevariance` module is a command-line tool designed to analyze the
variance of benchmark samples, specifically those generated by nanobench and
stored in Google Cloud Storage (GCS). Nanobench typically produces multiple
samples (e.g., 10) for each benchmark execution. This tool facilitates the
examination of these samples across a large corpus of historical benchmark runs.

The primary motivation for this tool is to identify benchmarks exhibiting high
variance in their results. High variance can indicate instability in the
benchmark itself, the underlying system, or the measurement process. By
calculating statistics like the ratio of the median to the minimum value for
each set of samples, `samplevariance` helps pinpoint traces that warrant further
investigation.

The core workflow involves:

1.  **Initialization**: Parsing command-line flags to determine the GCS location
    of benchmark data, output destination (stdout or a file), filtering criteria
    for traces, and the number of top results to display.
2.  **File Discovery**: Listing all relevant JSON files from the specified GCS
    bucket and prefix.
3.  **Data Processing (Concurrent)**: Distributing the discovered filenames to a
    pool of worker goroutines. Each worker:
    - Downloads a JSON file from GCS.
    - Parses the legacy nanobench format to extract benchmark results.
    - Filters traces based on the user-provided criteria.
    - For each matching trace, calculates the median and minimum of its
      samples.
    - Computes the ratio of median to minimum.
    - Stores this information as a `sampleInfo` struct.
4.  **Aggregation and Sorting**: Collecting all `sampleInfo` structs from the
    workers and sorting them in descending order based on the calculated
    median/min ratio. This brings the traces with the highest variance to the
    top.
5.  **Output**: Writing the sorted results to a CSV file (or stdout), including
    the trace identifier, minimum value, median value, and the median/min ratio.

```
[Flags] -> initialize() -> (ctx, bucket, objectPrefix, traceFilter, outputWriter)
                               |
                               v
filenamesFromBucketAndObjectPrefix(ctx, bucket, objectPrefix) -> [filenames]
                               |
                               v
samplesFromFilenames(ctx, bucket, traceFilter, [filenames])
  |
  |--> [gcsFilenameChannel] -> Worker Goroutine 1 -> traceInfoFromFilename() -> [sampleInfo] --\
  |                                                                                           |
  |--> [gcsFilenameChannel] -> Worker Goroutine 2 -> traceInfoFromFilename() -> [sampleInfo] ----> [aggregatedSamples] (mutex protected)
  |                                                                                           |
  |--> ... (up to workerPoolSize)                                                             |
  |                                                                                           |
  |--> [gcsFilenameChannel] -> Worker Goroutine N -> traceInfoFromFilename() -> [sampleInfo] --/
                               |
                               v
                             Sort([aggregatedSamples])
                               |
                               v
                             writeCSV([sortedSamples], topN, outputWriter) -> CSV Output
```

Key components and their responsibilities:

- **`main.go`**: This is the entry point of the application and orchestrates
  the entire process.

  - `main()`: Drives the overall workflow: initialization, fetching
    filenames, processing samples, sorting, and writing the output.
  - `initialize()`: Handles command-line argument parsing. It sets up the
    GCS client, determines the input GCS path (defaulting to yesterday's
    data if not specified), parses the trace filter query, and configures
    the output writer (stdout or a specified file). The choice to default to
    yesterday's data provides a convenient way to monitor recent benchmark
    stability without requiring explicit date specification.
  - `filenamesFromBucketAndObjectPrefix()`: Interacts with GCS to list all
    object names (filenames) under the specified bucket and prefix. It uses
    GCS client library features to efficiently retrieve only the names,
    minimizing data transfer.
  - `samplesFromFilenames()`: Manages the concurrent processing of benchmark
    files. It creates a channel (`gcsFilenameChannel`) to distribute
    filenames to a pool of worker goroutines (`workerPoolSize`). An
    `errgroup` is used to manage these goroutines and propagate any errors.
    A mutex protects the shared `samples` slice where results from workers
    are aggregated. This concurrent design is crucial for performance when
    dealing with a large number of benchmark files.
  - `traceInfoFromFilename()`: This function is executed by each worker
    goroutine. It takes a single GCS filename, reads the corresponding
    object from the bucket, parses the JSON content using
    `format.ParseLegacyFormat` (from `perf/go/ingest/format`) and
    `parser.GetSamplesFromLegacyFormat` (from `perf/go/ingest/parser`). For
    each trace that matches the `traceFilter` (a `query.Query` object from
    `go/query`), it sorts the sample values, calculates the median (using
    `stats.Sample.Quantile` from `go-moremath/stats`) and minimum, and then
    computes their ratio. The use of established libraries for parsing and
    statistical calculation ensures correctness and leverages existing,
    tested code.
  - `writeCSV()`: Formats the processed `sampleInfo` data into CSV format
    and writes it to the designated output writer. It includes a header row
    and then iterates through the `sampleInfo` slice, writing each entry. It
    also handles the `--top` flag to limit the number of output rows.
  - `sampleInfo`: A simple struct to hold the calculated statistics (trace
    ID, median, min, ratio) for a single benchmark trace's samples.
  - `sampleInfoSlice`: A helper type that implements `sort.Interface` to
    allow sorting `sampleInfo` slices by the `ratio` field in descending
    order. This is key to presenting the most variant traces first.

- **`main_test.go`**: Contains unit tests for the `writeCSV` function. These
  tests verify that the CSV output is correctly formatted under different
  conditions, such as when writing all samples, a limited number of top
  samples, or when the number of samples is less than the requested top N.
  This ensures the output formatting logic is robust.

The design decision to use a worker pool (`workerPoolSize`) for processing files
in parallel significantly speeds up the analysis, especially when dealing with
numerous benchmark result files often found in GCS. The use of
`golang.org/x/sync/errgroup` simplifies error handling in concurrent operations.
Filtering capabilities (via the `--filter` flag and `go/query`) allow users to
narrow down the analysis to specific subsets of benchmarks, making the tool more
flexible and targeted. The output as a CSV file makes it easy to import the
results into spreadsheets or other data analysis tools for further examination.

# Module: /scripts

The `/scripts` module provides tooling to support the data ingestion pipeline
for Skia Perf. The primary focus is on automating the process of transferring
processed data to the designated cloud storage location for further analysis and
visualization within the Skia performance monitoring system.

The key responsibility of this module is to ensure reliable and timely delivery
of performance data. This is achieved by interacting with Google Cloud Storage
(GCS) using the `gsutil` command-line tool.

The main component within this module is the `upload_extracted_json_files.sh`
script.

**`upload_extracted_json_files.sh`**

This shell script is responsible for uploading JSON files, which are assumed to
be the output of a preceding data extraction or processing phase, to a specific
Google Cloud Storage bucket (`gs://skia-perf/nano-json-v1/`).

**Design Rationale and Implementation Details:**

- **Why a shell script?** Shell scripting is a straightforward and widely
  available tool for automating command-line operations, making it suitable
  for tasks like file transfers to cloud storage. It avoids the need for more
  complex programming language environments for this specific, relatively
  simple task.
- **Why `gsutil`?** `gsutil` is the standard command-line tool for interacting
  with Google Cloud Storage. It provides robust features for uploading,
  downloading, and managing data in GCS buckets.
- **Why `-m` (parallel uploads)?** The `-m` flag in `gsutil cp` enables
  parallel uploads. This is a crucial performance optimization, especially
  when dealing with a potentially large number of JSON files. By uploading
  multiple files concurrently, the overall time taken for the transfer is
  significantly reduced.
- **Why `cp -r` (recursive copy)?** The `-r` flag ensures that the entire
  directory structure under `downloads/` is replicated in the destination GCS
  path. This is important for maintaining the organization of the data and
  potentially for downstream processing that might rely on the file paths.
- **Why the specific GCS path structure (`gs://skia-perf/nano-json-v1/$(date
-u --date +1hour +%Y/%m/%d/%H)`)?**
  - `gs://skia-perf/nano-json-v1/`: This is the base path in the GCS bucket
    designated for "nano" format JSON files, version 1. This structured
    naming helps in organizing different types and versions of data within
    the bucket.
  - `$(date -u --date +1hour +%Y/%m/%d/%H)`: This part dynamically generates
    a timestamped subdirectory structure.
  - `date -u`: Ensures the date is in UTC, providing a consistent timezone
    regardless of where the script is run.
  - `--date +1hour`: This is a deliberate choice to place the data into the
    _next_ hour's ingestion slot. This likely provides a buffer, ensuring
    that all data generated within a given hour is reliably captured and
    processed for that hour, even if the script runs slightly before or
    after the hour boundary. It helps prevent data from being missed or
    attributed to the wrong time window due to minor timing discrepancies in
    script execution.
  - `+%Y/%m/%d/%H`: Formats the date and time into a hierarchical path
    (e.g., `2023/10/27/15`). This organization is beneficial for:
    - **Data partitioning:** Makes it easy to query or process data for
      specific time ranges.
    - **Data lifecycle management:** Facilitates policies for archiving or
      deleting older data based on these time-based folders.
    - **Browseability:** Improves human readability and navigation within
      the GCS bucket.

**Workflow:**

The script executes a simple, linear workflow:

1.  **Source:** Identifies the `downloads/` directory in the current working
    directory as the source of JSON files. `[Local Filesystem] | ./downloads/
(contains *.json files)`
2.  **Destination Path Generation:** Dynamically constructs the target GCS path
    using the current UTC time, advanced by one hour, and formatted as
    `YYYY/MM/DD/HH`. `date command ---> YYYY/MM/DD/HH (e.g., 2023/10/27/15) |
Target GCS Path: gs://skia-perf/nano-json-v1/YYYY/MM/DD/HH/`
3.  **Upload:** Uses `gsutil` to recursively copy all contents from `downloads/`
    to the generated GCS path, utilizing parallel uploads for efficiency.
    `./downloads/* ---(gsutil -m cp -r)--->
gs://skia-perf/nano-json-v1/YYYY/MM/DD/HH/`

This script assumes that the `downloads/` directory exists in the location where
the script is executed and contains the JSON files ready for upload. It also
presumes that the user running the script has the necessary `gsutil` tool
installed and configured with appropriate permissions to write to the specified
GCS bucket.

# Module: /secrets

The `/secrets` module is responsible for managing the creation and configuration
of secrets required for various Skia Perf services to operate. These secrets
primarily involve Google Cloud service accounts and OAuth credentials for email
sending. The scripts in this module automate the setup of these credentials,
ensuring that services have the necessary permissions to interact with Google
Cloud APIs and other resources.

The design philosophy emphasizes secure and automated credential management.
Instead of manual creation and configuration of secrets, these scripts provide a
repeatable and version-controlled way to provision them. This reduces the risk
of human error and ensures that services are configured with the principle of
least privilege. For instance, service accounts are granted only the specific
roles they need to perform their tasks.

### Key Components and Scripts:

**1. Service Account Creation Scripts:**

- **`create-flutter-perf-service-account.sh`**: This script provisions a
  Google Cloud service account specifically for the Flutter Perf instance. It
  leverages a common script (`../../kube/secrets/add-service-account.sh`) to
  handle the underlying `gcloud` commands.

  - **Why**: Flutter Perf needs its own identity to interact with Google
    Cloud services like Pub/Sub (for message queuing) and Cloud Trace (for
    application performance monitoring). Separating this into its own
    service account adheres to the principle of least privilege and allows
    for more granular permission management.
  - **How**: It calls the `add-service-account.sh` script, passing in
    parameters like the project ID, the desired service account name
    ("flutter-perf-service-account"), a descriptive display name, and the
    necessary IAM roles (`roles/pubsub.editor`, `roles/cloudtrace.agent`).

- **`create-perf-cockroachdb-backup-service-account.sh`**: This script creates
  a dedicated service account for the Perf CockroachDB backup cronjob.

  - **Why**: The backup process requires permissions to write data to Google
    Cloud Storage. A dedicated service account ensures that only the backup
    job has these specific permissions, enhancing security. If the backup
    job's credentials were compromised, the blast radius would be limited to
    storage object administration.
  - **How**: Similar to the Flutter Perf service account, it utilizes
    `../../kube/secrets/add-service-account.sh`. It specifies the service
    account name ("perf-cockroachdb-backup") and the
    `roles/storage.objectAdmin` role, which grants permissions to manage
    objects in Cloud Storage buckets.

- **`create-perf-ingest-sa.sh`**: This script is responsible for creating the
  `perf-ingest` service account. This account is used by the Perf ingestion
  service, which processes and stores performance data.

  - **Why**: The ingestion service needs to publish messages to Pub/Sub
    topics, send trace data to Cloud Trace, and read data from specific
    Google Cloud Storage buckets (`gs://skia-perf`,
    `gs://cluster-telemetry-perf`). A dedicated service account with these
    precise permissions is crucial for security and operational clarity. It
    also leverages Workload Identity, a more secure way for Kubernetes
    workloads to access Google Cloud services.
  - **How**:

  * It sources configuration (`../kube/config.sh`) and utility functions
    (`../bash/ramdisk.sh`) for environment setup.
  * Creates the service account (`perf-ingest`) using `gcloud iam
service-accounts create`.
  * Assigns necessary IAM roles:
    - `roles/pubsub.editor`: To publish messages to Pub/Sub.
    - `roles/cloudtrace.agent`: To send trace data.
  * Configures Workload Identity by binding the Kubernetes service account
    (`default/perf-ingest` in the `skia-public` namespace) to the Google
    Cloud service account. This allows pods running as `perf-ingest` in
    Kubernetes to impersonate the `perf-ingest` Google Cloud service account
    without needing to mount service account key files directly. `Kubernetes
Pod (default/perf-ingest) ----> Impersonates ----> Google Cloud SA
(perf-ingest@skia-public.iam.gserviceaccount.com) | +----> Accesses GCP
Resources (Pub/Sub, Cloud Trace, GCS)`
  * Grants `objectViewer` permissions on specific GCS buckets using `gsutil
iam ch`.
  * Creates a JSON key file for the service account (`perf-ingest.json`).
  * Creates a Kubernetes secret named `perf-ingest` from this key file using
    `kubectl create secret generic`. This secret can then be used by
    deployments that might not be able to use Workload Identity directly or
    for other specific use cases.
  * Operations are performed in a temporary ramdisk (`/tmp/ramdisk`) to
    avoid leaving sensitive key files on persistent storage.

- **`create-perf-sa.sh`**: This script creates the primary `skia-perf` service
  account. This is a general-purpose service account for the main Perf
  application.

  - **Why**: The main Perf application requires permissions for Pub/Sub,
    Cloud Trace, and reading from the `gs://skia-perf` bucket. Similar to
    `perf-ingest`, this service account uses Workload Identity for enhanced
    security when running within Kubernetes.
  - **How**: The process is very similar to `create-perf-ingest-sa.sh`:

  * Sources configuration and sets up a ramdisk.
  * Creates the `skia-perf` service account.
  * Assigns `roles/cloudtrace.agent` and `roles/pubsub.editor`.
  * Configures Workload Identity, binding the Kubernetes service account
    (`default/skia-perf`) to the `skia-perf` Google Cloud service account.
  * Grants `objectViewer` on the `gs://skia-perf` GCS bucket.
  * Creates a JSON key and stores it as a Kubernetes secret named
    `skia-perf`.

**2. Email Secrets Creation:**

- **`create-email-secrets.sh`**: This script facilitates the creation of
  Kubernetes secrets necessary for Perf to send emails via Gmail. This
  typically involves an OAuth 2.0 flow.
  - **Why**: Perf needs to send email notifications (e.g., for alerts).
    Using Gmail programmatically requires proper authentication, which is
    achieved through OAuth 2.0. Storing these credentials as Kubernetes
    secrets makes them securely available to the Perf application pods.
  - **How**: This script guides the user through a semi-automated process:
  * It takes the email address to be authenticated as an argument (e.g.,
    `alertserver@skia.org`).
  * It converts the email address into a Kubernetes-friendly secret name
    format (e.g., `alertserver-skia-org`).
  * It prompts the user to download the `client_secret.json` file (obtained
    from the Google Cloud Console after enabling the Gmail API and creating
    OAuth 2.0 client credentials) to `/tmp/ramdisk`.
  * It then instructs the user to run the `three_legged_flow` Go program
    (which must be built and installed separately from
    `../go/email/three_legged_flow`). This program initiates the OAuth 2.0
    three-legged authentication flow. `User Action: Run three_legged_flow
--> Browser opens for Google Auth --> User authenticates as specified
email | v three_legged_flow generates client_token.json`
  * Once `client_token.json` (containing the authorization token and refresh
    token) is generated in `/tmp/ramdisk`, the script uses `kubectl create
secret generic` to create a Kubernetes secret named
    `perf-${EMAIL}-secrets`. This secret contains both `client_secret.json`
    and `client_token.json`.
  * Crucially, it then removes the `client_token.json` file from the local
    filesystem because it contains a sensitive refresh token. The source of
    truth for this token becomes the Kubernetes secret.
  * The use of `/tmp/ramdisk` ensures that sensitive downloaded and
    generated files are stored in memory and are less likely to be
    inadvertently persisted.

The common pattern across these scripts is the use of `gcloud` for Google Cloud
resource management and `kubectl` for interacting with Kubernetes to store the
secrets. The use of a ramdisk for temporary storage of sensitive files like
service account keys and OAuth tokens is a security best practice. Workload
Identity is preferred for service accounts running in GKE, reducing the need to
manage and distribute service account key files.