tree 2d0dd657622df73788edee43a68808965cc59bcc
parent 9b561311b7603253889c6e1cddeb6bb2f88ba7a0
author Kevin Lubick <kjlubick@google.com> 1614365698 -0500
committer Kevin Lubick <kjlubick@google.com> 1614367352 +0000

[gold] Add gitilesfollower

This is a simplified version of gitsync that works directly
with the SQL database and gitiles. The idea here is that
this process will make a mapping of commit_id and git_hash
in the GitCommits table, so when we see those git_hash
during ingestion, we can derive a commit_id and use that
for the actual ordering of data.

This version takes a lesson from Perf by only querying data
from the most recent commit in the DB and the latest on
the tree to make Gold resilient to merged/changed
history (e.g. go/skia-infra-pm-007)

As a result, I've made the commit_id field required for
the GitCommits table.

Suggested review order:
 - sql.go and tables.go
 - gitilesfollower_manual_test.go
 - gittilesfollower_test.go
 - gitilesfollower.go
 - Everything else

Bug: skia:10582, skia:11367
Change-Id: I0b6d3e89c91ff38aeef99f11b02775f644ed819d
Reviewed-on: https://skia-review.googlesource.com/c/buildbot/+/375664
Reviewed-by: Leandro Lovisolo <lovisolo@google.com>