[perf] versioned SQL migrations Design doc: go/versioned-sql-migrations-proposal Implement versioned SQL migrations and readiness validation Transition the database schema migrations and validation from the previous single-step JSON catalog diffing model to a sequential, version-controlled SQL migrations system. The old diff-based approach forced pods to crash-loop on startup if a deployment was delayed or batched with multiple schema changes. This change introduces the following modifications: - Replaces the legacy, static JSON-diff comparisons with sequential DDL scripts managed in a migrations directory. - Adds a lightweight VerifySchemaVersion check that verifies if the database is fully migrated based on a schema_migrations logging table. - Decouples connection pool boot checks from production startup using the KUBERNETES_SERVICE_HOST env variable. Local runs still fail-fast. - Integrates VerifySchemaVersion into the /readiness endpoint to hold pods in a NotReady status (returning 503) instead of crash-looping until the migration job finishes. - Spawns a background goroutine to periodically log migration wait messages during server startup in production. - Cleans up obsolete previous schema JSON catalogs, LoadPrev(), and unused spanner DDL variables. Change-Id: I213f91448db7cbba84e8daffce32ec53c65ec41f Reviewed-on: https://skia-review.googlesource.com/c/buildbot/+/1249317 Reviewed-by: Anri Sidorov <ansid@google.com> Commit-Queue: Sergei Rudenkov <sergeirudenkov@google.com>
This repo contains infrastructure code for Skia.
The infrastructure code is generally built to run on x86 linux. Running on other platforms may be possible but is not officially supported and success will vary depending on the command.
The main source code repository is a Git repository hosted at https://skia.googlesource.com/buildbot.git. It is possible to check out this repository directly with git clone or via go get.
Using git clone allows you to work in whatever directory you want. You will still need to set GOPATH in order to build some apps (recommended to put this in a cache dir). E.g.:
$ cd ${WORKDIR}
$ git clone https://skia.googlesource.com/buildbot.git
$ export GOPATH=${HOME}/.cache/gopath/$(basename ${WORKDIR})
$ mkdir $GOPATH
$ cd buildbot
Almost all applications are built with Bazel, and bazelisk is the recommended tool to ensure you have the right version of bazel installed:
go install github.com/bazelbuild/bazelisk@latest go install github.com/bazelbuild/buildtools/buildifier@latest go install github.com/kisielk/errcheck@latest go install golang.org/x/tools/cmd/goimports@latest go install github.com/mikefarah/yq/v4@latest go install go.chromium.org/luci/client/cmd/...@latest
export PATH=$PATH:$(go env GOPATH)/bin
You will need Node.js and npm installed to run web infrastructure tests and linters. We recommend using nvm to manage Node versions.
After installing Node.js, run the following command to install repository dependencies (including linter tools):
npm install
sudo apt-get install jq
bazelisk build --config=mayberemote //...
This step might be an optional step, but some test requires these enviornment variables.
Runs
./scripts/run_emulators/run_emulators start
The following are example of environment variables.
Emulators started. Set environment variables as follows: export DATASTORE_EMULATOR_HOST=localhost:8891 export BIGTABLE_EMULATOR_HOST=localhost:8892 export PUBSUB_EMULATOR_HOST=localhost:8893 export FIRESTORE_EMULATOR_HOST=localhost:8894 export COCKROACHDB_EMULATOR_HOST=localhost:8895
And stores these environment variables to ~/.bashrc file.
bazelisk test --config=mayberemote //...
To update generated code run the following in any directory:
go generate ./...
Install Cloud SDK.
Use this command to run the presubmit tests:
./run_unittests --small
Getting AI code review for a local commit:
bazelisk run //cmd/autoreview
Read more at cmd/autoreview/README.md