Google Git
Sign in
skia / buildbot / 8d9cef29be25
commit8d9cef29be251a057208e409f2f3120ee470927d[log] [tgz]
authorKevin Lubick <kjlubick@google.com>Thu Oct 05 14:20:49 2023 -0400
committerSkCQ <skcq-be@skia-corp.google.com.iam.gserviceaccount.com>Thu Oct 05 18:45:47 2023 +0000
treefdcb5c9a3d670f8bc410ea57a6269dfb07cde2aa
parent53536344718309665e67af9b3fb236963b7412f8 [diff]
[task_scheduler] Add more tracing to RBE CAS code and warning if it takes longer than usual

This outage [1] took a while to figure out, due to a lack of logs
(no errors or warnings were thrown). This aims to address that by
making such anomalous behavior be visible in the logs.

As of writing, a healthy mergeCASInputs takes place in <10s.
The anomalous behavior saw that take many minutes.

It also removes the (mostly useless) calls to metrics2.FuncTimer
as those do not show up in the cloud logs and are harder to use
than the tracing spans that are used in their place.

[1] https://groups.google.com/a/google.com/g/luci-outages/c/tvtxib74WAo/m/omAz0QBOAgAJ

Change-Id: I83eae14f05f76077807a5c444c66553a40e7a986
Reviewed-on: https://skia-review.googlesource.com/c/buildbot/+/763357
Commit-Queue: Joe Gregorio <jcgregorio@google.com>
Reviewed-by: Joe Gregorio <jcgregorio@google.com>
Auto-Submit: Kevin Lubick <kjlubick@google.com>
Commit-Queue: Kevin Lubick <kjlubick@google.com>
3 files changed
tree: fdcb5c9a3d670f8bc410ea57a6269dfb07cde2aa
  1. am/
  2. android_ingest/
  3. android_stats/
  4. api/
  5. autoroll/
  6. bash/
  7. bazel/
  8. bisection/
  9. blamer/
  10. bugs-central/
  11. cabe/
  12. cd/
  13. cherrypick-watcher/
  14. cmd/
  15. codereview-watcher/
  16. codesize/
  17. comments/
  18. comp-ui/
  19. cq_watcher/
  20. ct/
  21. datahopper/
  22. debugger-app/
  23. demos/
  24. docker/
  25. docker_pushes_watcher/
  26. docs/
  27. docsyserver/
  28. ds/
  29. elements-sk/
  30. email/
  31. external/
  32. fiddlek/
  33. firestore/
  34. get_service_account/
  35. gitsync/
  36. go/
  37. gold-client/
  38. golden/
  39. helloworld/
  40. infra/
  41. infra-sk/
  42. jsdoc/
  43. jsfiddle/
  44. k8s-checker/
  45. k8s-deployer/
  46. kube/
  47. leasing/
  48. licenses/
  49. machine/
  50. make/
  51. modules/
  52. named-fiddles/
  53. new_element/
  54. npm-audit-mirror/
  55. perdiff/
  56. perf/
  57. periodic-trigger/
  58. proberk/
  59. promk/
  60. puppeteer-tests/
  61. sa-keys-checker/
  62. scrap/
  63. scripts/
  64. shaders/
  65. sk/
  66. skbug/
  67. skcq/
  68. skfe/
  69. skolo/
  70. skottie/
  71. static_server/
  72. status/
  73. task_driver/
  74. task_scheduler/
  75. test-service/
  76. tools/
  77. tree_status/
  78. trybot_updater/
  79. .bazelignore
  80. .bazelrc
  81. .bazelversion
  82. .eslintrc.js
  83. .gitattributes
  84. .gitignore
  85. .npmrc
  86. .prettierignore
  87. .prettierrc.json
  88. .puppeteerrc.js
  89. .vpython
  90. BAZEL_CHEATSHEET.md
  91. BUILD.bazel
  92. build_infra_prod.sh
  93. CDB.md
  94. cipd.ensure
  95. codereview.settings
  96. DATASTORE.md
  97. demopage.sh
  98. DEPS
  99. go.mod
  100. go.sum
  101. go_repositories.bzl
  102. karmatest.sh
  103. launch.md
  104. LICENSE
  105. Makefile
  106. OWNERS
  107. package-lock.json
  108. package.json
  109. PRESUBMIT.py
  110. PRIVACY_POLICY.md
  111. README.md
  112. STYLEGUIDE.md
  113. tools.go
  114. tsconfig.json
  115. whitespace.txt
  116. WORKSPACE
README.md

Skia-Buildbot Repository

This repo contains infrastructure code for Skia.

Getting the Source Code

The main source code repository is a Git repository hosted at https://skia.googlesource.com/buildbot.git. It is possible to check out this repository directly with git clone or via go get.

Using git clone allows you to work in whatever directory you want. You will still need to set GOPATH in order to build some apps (recommended to put this in a cache dir). E.g.:

$ cd ${WORKDIR}
$ git clone https://skia.googlesource.com/buildbot.git
$ export GOPATH=${HOME}/.cache/gopath/$(basename ${WORKDIR})
$ mkdir $GOPATH
$ cd buildbot

Install dependencies

Almost all applications are built with Bazel, and bazelisk is the recommended tool to ensure you have the right version of bazel installed:

go install github.com/bazelbuild/bazelisk@latest
go install github.com/bazelbuild/buildtools/buildifier@latest
go install github.com/kisielk/errcheck@latest
go install golang.org/x/tools/cmd/goimports@latest
go install github.com/mikefarah/yq/v4@latest
go install go.chromium.org/luci/client/cmd/...@latest

Install other dependencies:

sudo apt-get install jq

Build ~everything

bazelisk build --config=mayberemote //...

Test everything

bazelisk test --config=mayberemote //...

Generated Code

To update generated code run the following in any directory:

go generate ./...

Running unit tests

Install Cloud SDK.

Use this command to run the presubmit tests:

./run_unittests --small