Gold is used in automated tests to answer “Did my code draw the right thing?” It has the following major features:
There are two parts to Gold, the “server part” and the “test part”. The “server part”, also known as the “Gold instance”, runs on a web server and ingests the data from an executable run alongside the tests.
If you are part of a Google team, the Skia infra team can host your instance for you. File a bug and we'll get back to you.
Otherwise, you'll need approximately the following steps. We use Google Cloud and Kubernetes (k8s) and the following assumes you do too.
grouping_param_keys_by_corpus field (example here).
The primary way to integrate with your tests is to use
goldctl (pronounced “gold-control”), a helper binary that speaks to the Gold Frontend and uploads data to the GCS bucket.
goldctl from source, make sure you a have a recent version of Go installed. Then it can be installed with:
$ go get -u go.skia.org/infra/gold-client/cmd/goldctl
If you are part of a Google team that uses CIPD, goldctl is available via CIPD.
A typical workflow for using Gold in a post-submit fashion is:
# Set up authentication (Google folks, there's also the --luci option, which may be useful) goldctl auth --work-dir ./tmp --service-account /path/to/serviceaccount.json # The keys.json is a JSON file that has key-value pairs describing how these inputs # got drawn. Commonly, you might specify the OS, the GPU, etc. For instances owned by the # Skia team, the --instance is a nice shortcut (assigned when the instance is created), # For other clients, goldctl has a --url to specify the webserver endpoint. goldctl imgtest init --work-dir ./tmp --keys-file ./keys.json --instance my-instance # Run a test named "cute-dog", assume it outputs /out/foo.png # This command will upload the image to the bucket and store an entry for test-name linking # to that image. goldctl imgtest add --work-dir ./tmp --test-name "cute-dog" --png-file /out/foo.png # Run many more tests # Upload a JSON file to the GCS bucket that includes all the test entries from this run. goldctl imgtest finalize --work-dir ./tmp
Using Gold in pass/fail mode (i.e. useful for presubmits), looks very similar but has a few small differences:
--passfail (and optionally
--failure-file) to the
goldctl imgtest init call.
goldctl imgtest finalize call at the end. In pass/fail mode, individual JSON files are uploaded in a “streaming” fashion instead of one big file at the end.
Of note, the
goldctl imgtest init call is optional; it just makes the future calls less verbose by specifying things once instead of multiple times.
For more, try adding
--help to the various