Regression detection query optimization round 2

- In https://skia-review.googlesource.com/c/buildbot/+/817735 we separated out the event driven from continous mode. Also a change was made to update the query of the alert config to search only for the incoming trace and not the entire query in the config, when the config was running in Individual mode.
- Considering there are a lot more traces coming in an ingestion event as compared to possibly matched configs, it will involve less db queries if we simply merge the queries for multiple traces into the commonly matched config.
- There is still a slim chance that a lot of traces match a single alert config and the query can become long, but that should be acceptable since it will be a subset of the original query in the config so should be faster than running the entire original query.
- This change also reduces the time complexity of the query hits from O(num_configs*no_traces) to O(num_configs).

Bug: b/326128508
Change-Id: Ie84631123c29439373e9fc502d46e213fed11fa5
Reviewed-on: https://skia-review.googlesource.com/c/buildbot/+/821645
Reviewed-by: Joe Gregorio <jcgregorio@google.com>
Commit-Queue: Ashwin Verleker <ashwinpv@google.com>
3 files changed
tree: 0db00f945968ae6234b2f58e87e9fab8c9ce709a
  1. am/
  2. android_ingest/
  3. android_stats/
  4. api/
  5. autoroll/
  6. bash/
  7. bazel/
  8. bisection/
  9. blamer/
  10. bugs-central/
  11. cabe/
  12. cd/
  13. cherrypick-watcher/
  14. cmd/
  15. codereview-watcher/
  16. codesize/
  17. comments/
  18. cq_watcher/
  19. ct/
  20. datahopper/
  21. debugger-app/
  22. demos/
  23. docker/
  24. docker_pushes_watcher/
  25. docs/
  26. docsyserver/
  27. ds/
  28. elements-sk/
  29. email/
  30. external/
  31. fiddlek/
  32. firestore/
  33. get_service_account/
  34. gitsync/
  35. go/
  36. gold-client/
  37. golden/
  38. helloworld/
  39. infra/
  40. infra-sk/
  41. jsdoc/
  42. jsfiddle/
  43. k8s-checker/
  44. k8s-deployer/
  45. kube/
  46. leasing/
  47. licenses/
  48. machine/
  49. make/
  50. modules/
  51. named-fiddles/
  52. new_element/
  53. npm-audit-mirror/
  54. perdiff/
  55. perf/
  56. periodic-trigger/
  57. pinpoint/
  58. proberk/
  59. promk/
  60. puppeteer-tests/
  61. sa-keys-checker/
  62. scrap/
  63. scripts/
  64. shaders/
  65. sk/
  66. skbug/
  67. skcq/
  68. skfe/
  69. skolo/
  70. skottie/
  71. static_server/
  72. status/
  73. task_driver/
  74. task_scheduler/
  75. temporal/
  76. test-service/
  77. tool/
  78. tools/
  79. tree_status/
  80. trybot_updater/
  81. .bazelignore
  82. .bazelrc
  83. .bazelversion
  84. .eslintrc.js
  85. .gitattributes
  86. .gitignore
  87. .mockery.yaml
  88. .npmrc
  89. .prettierignore
  90. .prettierrc.json
  91. .puppeteerrc.js
  92. .vpython
  93. BAZEL_CHEATSHEET.md
  94. BUILD.bazel
  95. build_infra_prod.sh
  96. CDB.md
  97. cipd.ensure
  98. codereview.settings
  99. DATASTORE.md
  100. demopage.sh
  101. DEPS
  102. go.mod
  103. go.sum
  104. go_repositories.bzl
  105. karmatest.sh
  106. launch.md
  107. LICENSE
  108. Makefile
  109. OWNERS
  110. package-lock.json
  111. package.json
  112. pnpm-lock.yaml
  113. PRESUBMIT.py
  114. PRIVACY_POLICY.md
  115. README.md
  116. STYLEGUIDE.md
  117. tools.go
  118. tsconfig.json
  119. whitespace.txt
  120. WORKSPACE
README.md

Skia-Buildbot Repository

This repo contains infrastructure code for Skia.

Supported Infrastucture Platforms

The infrastructure code is generally built to run on x86 linux. Running on other platforms may be possible but is not officially supported and success will vary depending on the command.

Getting the Source Code

The main source code repository is a Git repository hosted at https://skia.googlesource.com/buildbot.git. It is possible to check out this repository directly with git clone or via go get.

Using git clone allows you to work in whatever directory you want. You will still need to set GOPATH in order to build some apps (recommended to put this in a cache dir). E.g.:

$ cd ${WORKDIR}
$ git clone https://skia.googlesource.com/buildbot.git
$ export GOPATH=${HOME}/.cache/gopath/$(basename ${WORKDIR})
$ mkdir $GOPATH
$ cd buildbot

Install dependencies

Almost all applications are built with Bazel, and bazelisk is the recommended tool to ensure you have the right version of bazel installed:

go install github.com/bazelbuild/bazelisk@latest
go install github.com/bazelbuild/buildtools/buildifier@latest
go install github.com/kisielk/errcheck@latest
go install golang.org/x/tools/cmd/goimports@latest
go install github.com/mikefarah/yq/v4@latest
go install go.chromium.org/luci/client/cmd/...@latest

Add bazelisk to path

export PATH=$PATH:$(go env GOPATH)/bin

Install other dependencies:

sudo apt-get install jq

Build ~everything

bazelisk build --config=mayberemote //...

Test everything

bazelisk test --config=mayberemote //...

Generated Code

To update generated code run the following in any directory:

go generate ./...

Running unit tests

Install Cloud SDK.

Use this command to run the presubmit tests:

./run_unittests --small