Google Git
Sign in
skia / buildbot / 54a384e9fe25d9e0aa1498317b57723900189cdc
commit54a384e9fe25d9e0aa1498317b57723900189cdc[log] [tgz]
authorKevin Lubick <kjlubick@google.com>Tue Aug 03 11:17:43 2021 -0400
committerKevin Lubick <kjlubick@google.com>Tue Aug 03 15:21:31 2021 +0000
tree6f39123b56d9d19f0c5c0882f9a32852563aac5d
parent7e15eb551c79528302680296ec1cd13261d9c7a9 [diff]
[gold] Add high contention mode for calculating diffs

I don't know if we only want one of these two modes, so
I'm making it opt-in for now. The chrome instance needs it
but all other instances were doing fine with the other way.

The chrome instance has a lot of CLs, and thus a lot of
rows in the SecondaryBranchDiffCalculationWork table, which
caused finding new work to take 10+ seconds, mostly because
there were a lot of retries as that table was being modified
from periodictasks, as well as other diffcalculators.

Bug: skia:12221
Change-Id: I6d15c1c1b4206accd7fe0f7d9af7698ffcc83d7b
Reviewed-on: https://skia-review.googlesource.com/c/buildbot/+/436007
Reviewed-by: Kevin Lubick <kjlubick@google.com>
13 files changed
tree: 6f39123b56d9d19f0c5c0882f9a32852563aac5d
  1. am/
  2. android_ingest/
  3. android_stats/
  4. api/
  5. autoroll/
  6. bash/
  7. bazel/
  8. blamer/
  9. bugs-central/
  10. codereview-watcher/
  11. comments/
  12. contestk/
  13. cq_watcher/
  14. ct/
  15. datahopper/
  16. debugger-app/
  17. demos/
  18. docker/
  19. docker_pushes_watcher/
  20. docs/
  21. docsyserver/
  22. dot/
  23. ds/
  24. external/
  25. fiddlek/
  26. firestore/
  27. get_service_account/
  28. git_cookie_authdaemon/
  29. gitsync/
  30. go/
  31. gold-client/
  32. golden/
  33. hashtag/
  34. html-template-minifier/
  35. infra/
  36. infra-sk/
  37. jsdoc/
  38. jsfiddle/
  39. k8s-checker/
  40. kube/
  41. leasing/
  42. licenses/
  43. machine/
  44. make/
  45. modules/
  46. named-fiddles/
  47. new_element/
  48. particles/
  49. perdiff/
  50. perf/
  51. periodic-trigger/
  52. power/
  53. proberk/
  54. promk/
  55. pulld/
  56. puppeteer-tests/
  57. push/
  58. scrap/
  59. scripts/
  60. shaders/
  61. sk/
  62. sk8s/
  63. skbug/
  64. skcq/
  65. skfe/
  66. skolo/
  67. skottie/
  68. status/
  69. switchboard/
  70. task_driver/
  71. task_scheduler/
  72. tools/
  73. tree_status/
  74. trybot_updater/
  75. velero/
  76. .bazelignore
  77. .bazelrc
  78. .bazelversion
  79. .eslintrc.js
  80. .gitattributes
  81. .gitignore
  82. .mocharc.json
  83. BAZEL_CHEATSHEET.md
  84. BUILD.bazel
  85. build_infra_prod.sh
  86. cipd.ensure
  87. codereview.settings
  88. DATASTORE.md
  89. DEPS
  90. elements-sk-scss.bzl
  91. go.mod
  92. go.sum
  93. go_repositories.bzl
  94. launch.md
  95. LICENSE
  96. Makefile
  97. package-lock.json
  98. package.json
  99. PRESUBMIT.py
  100. PRIVACY_POLICY.md
  101. README.md
  102. run_unittests
  103. run_unittests.go
  104. STYLEGUIDE.md
  105. tools.go
  106. tsconfig.json
  107. whitespace.txt
  108. WORKSPACE
README.md

Skia-Buildbot Repository

This repo contains infrastructure code for Skia.

Getting the Source Code

The main source code repository is a Git repository hosted at https://skia.googlesource.com/buildbot.git. It is possible to check out this repository directly with git clone or via go get.

Using git clone allows you to work in whatever directory you want. You will still need to set GOPATH in order to build some apps (recommended to put this in a cache dir). E.g.:

$ cd ${WORKDIR}
$ git clone https://skia.googlesource.com/buildbot.git
$ export GOPATH=${HOME}/.cache/gopath/$(basename ${WORKDIR})
$ mkdir $GOPATH
$ cd buildbot

Using go get will fetch the repository into your GOPATH directory along with all the Go dependencies. You will need to set GOPATH and GO111MODULE=on. E.g.:

$ export GOPATH=${WORKDIR}
$ export GO111MODULE=on
$ go get -u -t go.skia.org/infra/...
$ cd ${GOPATH}/src/go.skia.org/infra/

Note: go.skia.org is a custom import path and will only work if used like the examples here.

Install Node.js (not as root) and add the bin dir to your path. Optionally run npm install npm -g, as suggested by the npm getting started doc.

Install other dependencies:

$ sudo apt-get install \
    jq \
    python-django
$ go get -u \
    github.com/kisielk/errcheck \
    golang.org/x/tools/cmd/goimports \
    go.chromium.org/luci/client/cmd/isolate \
    go get github.com/mikefarah/yq/v4

Build ~everything:

$ make all

Generated Code

Some code is generated using go generate with external binaries. First, install the version of protoc referenced in the asset creation script and ensure it is on your PATH before other versions of protoc.

Install the necessary go packages:

$ go get -u \
  github.com/golang/protobuf/protoc-gen-go \
  golang.org/x/tools/cmd/stringer \
  google.golang.org/grpc \

Get mockery and add it to your PATH.

To generate code run in this directory:

$ go generate ./...

Running unit tests

Install Cloud SDK.

Use this command to run the presubmit tests:

$ ./run_unittests --small