[autoroll] Log fewer errors during find-and-replace process

We have at least one autoroller with a lot of transitive deps which also
uses find-and-replace. In this case, we're find-and-replacing in
submodule links, which each contain a single dependency reference. The
current code will log an error for each transitive dep which isn't found
in every submodule link, which is undesirable.

Additionally, we have code in updateSingleDep which automatically
detects whether we have a submodule link that needs to be updated. That
gets done before the find-and-replace occurs, which means the find-and-
replace has no effect in some cases. Only log an error if nothing has
produced a change in that file.

Finally, fix a potential issue where we fetch the file from Gitiles
instead of checking our local changes first during find-and-replace. I
have not seen evidence that this has caused a real problem, but it could
be relevant if the same file appears multiple times between
find-and-replace configs and the primary dependency file.

Change-Id: I9a67a5806b37637f9701f930d9b65a1869f4437a
Reviewed-on: https://skia-review.googlesource.com/c/buildbot/+/868816
Commit-Queue: Eric Boren <borenet@google.com>
Auto-Submit: Eric Boren <borenet@google.com>
Reviewed-by: Kaylee Lubick <kjlubick@google.com>
Commit-Queue: Kaylee Lubick <kjlubick@google.com>
2 files changed
tree: 2c07fa970b10d2eaf12647f0b01f44a753da9ab0
  1. am/
  2. android_ingest/
  3. android_stats/
  4. api/
  5. autoroll/
  6. bash/
  7. bazel/
  8. bisection/
  9. blamer/
  10. bugs-central/
  11. cabe/
  12. cd/
  13. cherrypick-watcher/
  14. cmd/
  15. codereview-watcher/
  16. codesize/
  17. comments/
  18. cq_watcher/
  19. ct/
  20. datahopper/
  21. debugger-app/
  22. demos/
  23. docker/
  24. docker_pushes_watcher/
  25. docs/
  26. docsyserver/
  27. ds/
  28. elements-sk/
  29. email/
  30. external/
  31. fiddlek/
  32. firestore/
  33. get_service_account/
  34. gitsync/
  35. go/
  36. gold-client/
  37. golden/
  38. helloworld/
  39. infra/
  40. infra-sk/
  41. jsdoc/
  42. jsfiddle/
  43. k8s-checker/
  44. k8s-deployer/
  45. kube/
  46. leasing/
  47. licenses/
  48. machine/
  49. make/
  50. modules/
  51. named-fiddles/
  52. new_element/
  53. npm-audit-mirror/
  54. perdiff/
  55. perf/
  56. periodic-trigger/
  57. pinpoint/
  58. proberk/
  59. promk/
  60. puppeteer-tests/
  61. sa-keys-checker/
  62. scrap/
  63. scripts/
  64. shaders/
  65. sk/
  66. skbug/
  67. skcq/
  68. skfe/
  69. skolo/
  70. skottie/
  71. static_server/
  72. status/
  73. task_driver/
  74. task_scheduler/
  75. temporal/
  76. test-service/
  77. tool/
  78. tools/
  79. tree_status/
  80. trybot_updater/
  81. .bazelignore
  82. .bazelrc
  83. .bazelversion
  84. .eslintrc.js
  85. .gitattributes
  86. .gitignore
  87. .mockery.yaml
  88. .npmrc
  89. .prettierignore
  90. .prettierrc.json
  91. .puppeteerrc.js
  92. .vpython
  93. BAZEL_CHEATSHEET.md
  94. BUILD.bazel
  95. build_infra_prod.sh
  96. CDB.md
  97. cipd.ensure
  98. codereview.settings
  99. DATASTORE.md
  100. demopage.sh
  101. DEPS
  102. go.mod
  103. go.sum
  104. go_repositories.bzl
  105. karmatest.sh
  106. launch.md
  107. LICENSE
  108. Makefile
  109. OWNERS
  110. package-lock.json
  111. package.json
  112. pnpm-lock.yaml
  113. PRESUBMIT.py
  114. PRIVACY_POLICY.md
  115. README.md
  116. STYLEGUIDE.md
  117. tools.go
  118. tsconfig.json
  119. whitespace.txt
  120. WORKSPACE
README.md

Skia-Buildbot Repository

This repo contains infrastructure code for Skia.

Supported Infrastucture Platforms

The infrastructure code is generally built to run on x86 linux. Running on other platforms may be possible but is not officially supported and success will vary depending on the command.

Getting the Source Code

The main source code repository is a Git repository hosted at https://skia.googlesource.com/buildbot.git. It is possible to check out this repository directly with git clone or via go get.

Using git clone allows you to work in whatever directory you want. You will still need to set GOPATH in order to build some apps (recommended to put this in a cache dir). E.g.:

$ cd ${WORKDIR}
$ git clone https://skia.googlesource.com/buildbot.git
$ export GOPATH=${HOME}/.cache/gopath/$(basename ${WORKDIR})
$ mkdir $GOPATH
$ cd buildbot

Install dependencies

Almost all applications are built with Bazel, and bazelisk is the recommended tool to ensure you have the right version of bazel installed:

go install github.com/bazelbuild/bazelisk@latest
go install github.com/bazelbuild/buildtools/buildifier@latest
go install github.com/kisielk/errcheck@latest
go install golang.org/x/tools/cmd/goimports@latest
go install github.com/mikefarah/yq/v4@latest
go install go.chromium.org/luci/client/cmd/...@latest

Add bazelisk to path

export PATH=$PATH:$(go env GOPATH)/bin

Install other dependencies:

sudo apt-get install jq

Build ~everything

bazelisk build --config=mayberemote //...

Test everything

bazelisk test --config=mayberemote //...

Generated Code

To update generated code run the following in any directory:

go generate ./...

Running unit tests

Install Cloud SDK.

Use this command to run the presubmit tests:

./run_unittests --small