blob: 38d46dcd0b9aa8ef8b807f083348d45028192346 [file] [log] [blame] [view]
# Gold instance configurations
## json5 files
These are the configurations for each Gold instance. They are used by various
Gold services, built into the container, and passed as arguments in the yaml
files in the k8s-config repo.
## CockroachDB data retention policy files
These are SQL files that need to be executed manually to apply data retention
policies on certain tables. We use
[Row-Level TTL](https://www.cockroachlabs.com/docs/stable/row-level-ttl)
to automatically delete expired data.
Data retention policies are implemented as configurations in the data layer
rather than part of the application schema because:
- The policies may vary for each instance.
- Gold does not depend on the policies to function, and should not use those
columns (`expire_at`).
- This separation of concern allows us to evolve the implementation of data
retention policies in the future.
Guidelines when defining policies for a specific instance.
- Keep data at least 2x the size of the commits sliding window.
- Define proper policies based on the purpose of a table. For example, the
`ValuesAtHead` table is a "caching" table that requires a policy similar to
LRU cache; the `Expectations` table is a "core entity" that should retain all
triaged records indefinitely and only delete overdue un-triaged ones.
- Refer to existing policies of other instances for techniques and consistency.