[public's a subset] trace visibility promoter core & CLI

Key changes:
- Implements a state-based batch pagination sweep
  in promoter.go to perform OOM-safe, timeout-proof trace promotions.
- Robustness Fixes: Corrects sub-context propagation in
  util.ChunkIterParallelPool, returns context errors on
  cancellation/timeout, and validates rows iteration via rows.Err().
- Standalone CLI Subcommand: Adds the 'visibility promote' command
  to perf-tool with custom '--timeout' flags.
- TraceParamStore APIs: Adds bulk UpdateVisibility and
  GetInternalTraceIDsForParam methods to the SQL storage interfaces.

Bug: 504536191
Change-Id: I362acc217bd737c53c62b2b0973429e9ea735a57
Reviewed-on: https://skia-review.googlesource.com/c/buildbot/+/1218116
Reviewed-by: Anri Sidorov <ansid@google.com>
Commit-Queue: Sergei Rudenkov <sergeirudenkov@google.com>
19 files changed
tree: 16f6ac0f85a011f9a4399d184fc0ad04198c8aab
  1. .agents/
  2. am/
  3. android_ingest/
  4. android_stats/
  5. api/
  6. attest/
  7. autogardener/
  8. autoroll/
  9. bash/
  10. bazel/
  11. blamer/
  12. bugs-central/
  13. cabe/
  14. cbb/
  15. cd/
  16. cdn/
  17. cherrypick-watcher/
  18. cmd/
  19. codesize/
  20. comment_rag/
  21. comments/
  22. cq_watcher/
  23. ct/
  24. datahopper/
  25. debugger-app/
  26. demos/
  27. docker/
  28. docs/
  29. docsyserver/
  30. ds/
  31. elements-sk/
  32. external/
  33. fiddlek/
  34. firestore/
  35. get_service_account/
  36. gitsync/
  37. go/
  38. gold-client/
  39. golden/
  40. helloworld/
  41. infra/
  42. infra-sk/
  43. jsdoc/
  44. jsfiddle/
  45. k8s-checker/
  46. k8s-deployer/
  47. kube/
  48. leasing/
  49. licenses/
  50. machine/
  51. make/
  52. mcp/
  53. modules/
  54. named-fiddles/
  55. new_element/
  56. npm-audit-mirror/
  57. perdiff/
  58. perf/
  59. periodic-trigger/
  60. pinpoint/
  61. proberk/
  62. promk/
  63. puppeteer-tests/
  64. rag/
  65. sa-keys-checker/
  66. scrap/
  67. scripts/
  68. shaders/
  69. sk/
  70. skbug/
  71. skcq/
  72. skfe/
  73. skolo/
  74. skottie/
  75. static_server/
  76. status/
  77. task_driver/
  78. task_scheduler/
  79. temporal/
  80. test-service/
  81. tool/
  82. tools/
  83. tree_status/
  84. trybot_updater/
  85. .bazelignore
  86. .bazelrc
  87. .bazelversion
  88. .eslintrc.js
  89. .gitattributes
  90. .gitignore
  91. .golangci.yml
  92. .mockery.yaml
  93. .npmrc
  94. .prettierignore
  95. .prettierrc.json
  96. .puppeteerrc.js
  97. .pylintrc
  98. .vpython
  99. BAZEL_CHEATSHEET.md
  100. BUILD.bazel
  101. build_infra_prod.sh
  102. CDB.md
  103. codereview.settings
  104. DATASTORE.md
  105. demopage.sh
  106. DEPS
  107. elements.d.ts
  108. go.mod
  109. go.sum
  110. karmatest.sh
  111. launch.md
  112. LICENSE
  113. LINT.md
  114. Makefile
  115. MODULE.bazel
  116. MODULE.bazel.lock
  117. OWNERS
  118. package-lock.json
  119. package.json
  120. pnpm-lock.yaml
  121. PRESUBMIT.py
  122. PRIVACY_POLICY.md
  123. puppeteerrc.bzl
  124. puppeteerrc.js.tpl
  125. README.md
  126. requirements.txt
  127. run_lint.sh
  128. STYLEGUIDE.md
  129. tools.go
  130. tsconfig.json
  131. whitespace.txt
  132. workflowcheck.yaml
README.md

Skia-Buildbot Repository

This repo contains infrastructure code for Skia.

Supported Infrastucture Platforms

The infrastructure code is generally built to run on x86 linux. Running on other platforms may be possible but is not officially supported and success will vary depending on the command.

Getting the Source Code

The main source code repository is a Git repository hosted at https://skia.googlesource.com/buildbot.git. It is possible to check out this repository directly with git clone or via go get.

Using git clone allows you to work in whatever directory you want. You will still need to set GOPATH in order to build some apps (recommended to put this in a cache dir). E.g.:

$ cd ${WORKDIR}
$ git clone https://skia.googlesource.com/buildbot.git
$ export GOPATH=${HOME}/.cache/gopath/$(basename ${WORKDIR})
$ mkdir $GOPATH
$ cd buildbot

Install dependencies

Almost all applications are built with Bazel, and bazelisk is the recommended tool to ensure you have the right version of bazel installed:

go install github.com/bazelbuild/bazelisk@latest
go install github.com/bazelbuild/buildtools/buildifier@latest
go install github.com/kisielk/errcheck@latest
go install golang.org/x/tools/cmd/goimports@latest
go install github.com/mikefarah/yq/v4@latest
go install go.chromium.org/luci/client/cmd/...@latest

Add bazelisk to path

export PATH=$PATH:$(go env GOPATH)/bin

Install Node.js and npm

You will need Node.js and npm installed to run web infrastructure tests and linters. We recommend using nvm to manage Node versions.

After installing Node.js, run the following command to install repository dependencies (including linter tools):

npm install

Install other dependencies:

sudo apt-get install jq

Build ~everything

bazelisk build --config=mayberemote //...

Test everything

Set up environ variables

This step might be an optional step, but some test requires these enviornment variables.

Runs

./scripts/run_emulators/run_emulators start

The following are example of environment variables.

Emulators started. Set environment variables as follows:
export DATASTORE_EMULATOR_HOST=localhost:8891
export BIGTABLE_EMULATOR_HOST=localhost:8892
export PUBSUB_EMULATOR_HOST=localhost:8893
export FIRESTORE_EMULATOR_HOST=localhost:8894
export COCKROACHDB_EMULATOR_HOST=localhost:8895

And stores these environment variables to ~/.bashrc file.

Execute tests

bazelisk test --config=mayberemote //...

Generated Code

To update generated code run the following in any directory:

go generate ./...

Running unit tests

Install Cloud SDK.

Use this command to run the presubmit tests:

./run_unittests --small

AI Code Review

Getting AI code review for a local commit:

bazelisk run //cmd/autoreview

Read more at cmd/autoreview/README.md