| commit | 988474ac727f1bd5cb6fba34f44fc3068428b904 | [log] [tgz] |
|---|---|---|
| author | Kevin Lubick <kjlubick@google.com> | Wed May 06 07:38:29 2020 -0400 |
| committer | Kevin Lubick <kjlubick@google.com> | Wed May 06 13:35:58 2020 +0000 |
| tree | 821c23aa7661844b0bc3a9499f8df9f2c75f5d6d | |
| parent | 45585440714ba277b3ba45ba11e5d0be5990ae04 [diff] |
[gold] Implement Changelist indexing The summary of CL indexing is that we scan for open CLs that have been "updated" (i.e. Gold has seen data from them) in the last N days. From these CLs, we query the patchsets that we've ingested data. For the most recent PS, we re-query all data uploaded to Firestore for that PS and then filter out the stuff we care about in our index (currently, just the untriaged results). We add that to the index. For earlier PS that we have cached, we don't re-fetch data, but we will re-filter it based on the latest expectations, in an effort to keep these older indices up to date. Eventually, CLs will be closed or (effectively) abandoned, at which point we will stop updating them in our cache and they will expire. This expiration is handled by a TTL cache. Bug: skia:9710 Change-Id: Ie692e72d136b5711fcebcf8370d6a018f0de6973 Reviewed-on: https://skia-review.googlesource.com/c/buildbot/+/287882 Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
This repo contains infrastructure code for Skia.
The main source code repository is a Git repository hosted at https://skia.googlesource.com/buildbot.git. It is possible to check out this repository directly with git clone or via go get.
Using git clone allows you to work in whatever directory you want. You will still need to set GOPATH in order to build some apps (recommended to put this in a cache dir). E.g.:
$ cd ${WORKDIR}
$ git clone https://skia.googlesource.com/buildbot.git
$ export GOPATH=${HOME}/.cache/gopath/$(basename ${WORKDIR})
$ mkdir $GOPATH
$ cd buildbot
Using go get will fetch the repository into your GOPATH directory along with all the Go dependencies. You will need to set GOPATH and GO111MODULE=on. E.g.:
$ export GOPATH=${WORKDIR}
$ export GO111MODULE=on
$ go get -u -t go.skia.org/infra/...
$ cd ${GOPATH}/src/go.skia.org/infra/
Note: go.skia.org is a custom import path and will only work if used like the examples here.
Install Node.js (not as root) and add the bin dir to your path. Optionally run npm install npm -g, as suggested by the npm getting started doc.
Install other dependencies:
$ sudo apt-get install python-django $ go get -u github.com/kisielk/errcheck \ golang.org/x/tools/cmd/goimports \ go.chromium.org/luci/client/cmd/isolate $ npm install -g polylint bower
Build ~everything:
$ make all
Some code is generated using go generate with external binaries. First, install the version of protoc referenced in the asset creation script and ensure it is on your PATH before other versions of protoc.
Install the necessary go packages:
$ go get -u \ github.com/golang/protobuf/protoc-gen-go \ golang.org/x/tools/cmd/stringer \ google.golang.org/grpc \ github.com/vektra/mockery/...
To generate code run in this directory:
$ go generate ./...
Install Cloud SDK.
Use this command to run the presubmit tests:
$ ./run_unittests --small