tree 8e79b71b75b87330538957d16fff4cf293c6a124
parent b195d18f8e107466942c7d7c0160b74ee2833456
author Ravi Mistry <rmistry@google.com> 1565271665 -0400
committer Skia Commit-Bot <skia-commit-bot@chromium.org> 1565352792 +0000

Add ability to bring up multiple Android compile servers

* Separates FE and BE and creates new FE server.
  * Creates new AndroidCompileInstances Kind in Datastore that keeps track of when mirror was last synced and if mirror should be force synced.
  * Changed UI to account for and display multiple backends. Screenshot: https://screenshot.googleplex.com/E9fA5A9QNPU
  * The force mirror sync button will now signal to all instances to update their mirror and so will behave similarly as before.

* Support for running multiple backends
  * Multiple instances satisfy new requests by Ack'ing and Nack'ing pubsub storage notifications.
  * Care has been taken to make sure duplicate tasks are not triggered in case pubsub sends the same message again (At-least-once delivery from https://cloud.google.com/pubsub/docs/subscriber).
  * In case a backend is restarted it will run all tasks that were interrupted before picking up new tasks.

* Other improvements
  * Changed startup to only sync checkouts that do not exist. Otherwise it syncs all checkouts during startup which is unnecessary because checkouts are synced before running tasks anyway.
  * Removed UpdateInfraFailureMetric. The alert was only triggered for merge errors and the error is already exposed in the recipe output anyway.


Bug: skia:9302
Change-Id: I662f26c7fc0c841b6b36e39b20f06d50289f7eaa
Reviewed-on: https://skia-review.googlesource.com/c/buildbot/+/231096
Commit-Queue: Ravi Mistry <rmistry@google.com>
Reviewed-by: Eric Boren <borenet@google.com>