| commit | 166d95c5f9d0918982a2b89a01f9dcecad843e98 | [log] [tgz] |
|---|---|---|
| author | Kevin Lubick <kjlubick@google.com> | Thu Oct 24 11:16:32 2019 -0400 |
| committer | Skia Commit-Bot <skia-commit-bot@chromium.org> | Thu Oct 24 18:45:19 2019 +0000 |
| tree | 06f6ce767da3861b68856e375f2bf73c37ac9cd8 | |
| parent | ac86ad9ba0220dafa3f83ffd077e5b7e466a7c05 [diff] |
[gold] Performance enhancements for /digests Wrote two benchmarks to show 1 easy win and 1 hard win (pre-caching Test() saves ~20ms in GetDigestDetails and probably less than that elsewhere; If I were to re-write lots of things to have a slice of traces instead of a map, it would save about 15ms of overhead on the range) Implemented a few small improvements, the aformentioned pre-caching and being more efficient about looking up if a trace had a given digest. I hardened the endpoint against garbage data, in that we bail out sooner if there's an unknown test or digest (the previous impl did not do the right thing in this case). The rest is cleanup that I found while scouring through. In the end, based on my extensive testing (of two random data samples [1], [2]), the /json/details endpoint response dropped from ~2 seconds to ~250 milliseconds (on some large cases, up to 1/3 of that time is gzipping the response, I think). With that worst-case scenario addressed, I feel confident in upping the QPS on this endpoint. [1] https://gold.skia.org/detail?test=blurredclippedcircle&digest=412961706b174f726893ab4a054a658a [2] https://gold.skia.org/detail?test=hslcolorfilter&digest=daf52d9be7a334fbfbb38e56d51017cd Bug: skia:9557,skia:9080 Change-Id: Ifaf14917d8fa3fa9b5a22c85b3ab9b2bbef914a4 Reviewed-on: https://skia-review.googlesource.com/c/buildbot/+/250320 Reviewed-by: Ben Wagner aka dogben <benjaminwagner@google.com> Commit-Queue: Kevin Lubick <kjlubick@google.com>
This repo contains infrastructure code for Skia.
The main source code repository is a Git repository hosted at https://skia.googlesource.com/buildbot.git. It is possible to check out this repository directly with git clone or via go get.
Using git clone allows you to work in whatever directory you want. You will still need to set GOPATH in order to build some apps (recommended to put this in a cache dir). E.g.:
$ cd ${WORKDIR}
$ git clone https://skia.googlesource.com/buildbot.git
$ export GOPATH=${HOME}/.cache/gopath/$(basename ${WORKDIR})
$ mkdir $GOPATH
$ cd buildbot
Using go get will fetch the repository into your GOPATH directory along with all the Go dependencies. You will need to set GOPATH and GO111MODULE=on. E.g.:
$ export GOPATH=${WORKDIR}
$ export GO111MODULE=on
$ go get -u -t go.skia.org/infra/...
$ cd ${GOPATH}/src/go.skia.org/infra/
Note: go.skia.org is a custom import path and will only work if used like the examples here.
Install Node.js (not as root) and add the bin dir to your path. Optionally run npm install npm -g, as suggested by the npm getting started doc.
Install other dependencies:
$ sudo apt-get install python-django $ go get -u github.com/kisielk/errcheck \ golang.org/x/tools/cmd/goimports \ go.chromium.org/luci/client/cmd/isolate $ npm install -g polylint bower
Build ~everything:
$ make all
Some code is generated using go generate with external binaries. First, install the version of protoc referenced in the asset creation script and ensure it is on your PATH before other versions of protoc.
Install the necessary go packages:
$ go get -u \ github.com/golang/protobuf/protoc-gen-go \ golang.org/x/tools/cmd/stringer \ google.golang.org/grpc \ github.com/vektra/mockery/...
To generate code run in this directory:
$ go generate ./...
Install Cloud SDK.
Use this command to run the presubmit tests:
$ ./run_unittests --small