Add genpromcrd.

A command-line application for producing CRDs for Managed Prometheus.

$ go run ./promk/go/genpromcrd  --help
usage: genpromcrd --directory=[k8s-config checkout dir] [options]
options:
  -directory string
        The directory that contains a checkout of k8s-config.
  -dryrun
        If true then just print the names of the files that would be written.
  -logtostdout
        If true then write logging on stdout.

The genpromcrd cmd runs over all Deployments and StatefulSets and
writes out Managed Prometheus CRDs for both scraping and alerting.
For example, given the following file in the git repo that contains
all the cluster config:

        k8s-config/
        ├── monitoring
        │   └── appgroups
        │       └── perf.yml
        └── skia-infra-public
            └── perf.yml

All the Rules files for alerts to run for all Deployments and
StatefulSets are held under /monitoring/appgroups and the name
of the file before the '.yml' corresponds to an appgroup label.

Since perf.yaml resides inside a directory associated with a
cluster, the Deployment there runs in the namespace 'somenamespace',
and has .template.label.appgroup=perf, a new file will be written to:

   skia-infra-public/perf_somenamespace_appgroup_alerts.yml

which is a modified version of /monitoring/appgroups/perf.yaml, updated
to scrape the deployment in the correct namespace, and it will also
contain 'absent()' alerts for all the alerts defined in 'perf.yml'.

The list of directories processed are defined in:

    //kube/clusters/config.json



Bug: skia:13542
Change-Id: If4c1f6b281a236a3613ca4b65c6279d0be90e3d8
Reviewed-on: https://skia-review.googlesource.com/c/buildbot/+/561420
Reviewed-by: Ravi Mistry <rmistry@google.com>
Commit-Queue: Joe Gregorio <jcgregorio@google.com>
22 files changed
tree: f97f4fcac93a998dd6fbab433b1b78665a535155
  1. am/
  2. android_ingest/
  3. android_stats/
  4. api/
  5. autoroll/
  6. bash/
  7. bazel/
  8. blamer/
  9. bugs-central/
  10. cd/
  11. cherrypick-watcher/
  12. codereview-watcher/
  13. codesize/
  14. comments/
  15. comp-ui/
  16. contestk/
  17. cq_watcher/
  18. ct/
  19. datahopper/
  20. debugger-app/
  21. demos/
  22. docker/
  23. docker_pushes_watcher/
  24. docs/
  25. docsyserver/
  26. dot/
  27. ds/
  28. email/
  29. external/
  30. fiddlek/
  31. firestore/
  32. get_service_account/
  33. gitsync/
  34. go/
  35. gold-client/
  36. golden/
  37. hashtag/
  38. infra/
  39. infra-sk/
  40. jsdoc/
  41. jsfiddle/
  42. k8s-checker/
  43. k8s-deployer/
  44. kube/
  45. leasing/
  46. licenses/
  47. machine/
  48. make/
  49. modules/
  50. named-fiddles/
  51. new_element/
  52. npm-audit-mirror/
  53. particles/
  54. perdiff/
  55. perf/
  56. periodic-trigger/
  57. proberk/
  58. promk/
  59. puppeteer-tests/
  60. sa-keys-checker/
  61. scrap/
  62. scripts/
  63. shaders/
  64. sk/
  65. skbug/
  66. skcq/
  67. skfe/
  68. skolo/
  69. skottie/
  70. static_server/
  71. status/
  72. switchboard/
  73. task_driver/
  74. task_scheduler/
  75. test-service/
  76. third_party/
  77. tools/
  78. tree_status/
  79. trybot_updater/
  80. .bazelignore
  81. .bazelrc
  82. .bazelversion
  83. .eslintrc.js
  84. .gitattributes
  85. .gitignore
  86. .mocharc.json
  87. .npmrc
  88. .vpython
  89. BAZEL_CHEATSHEET.md
  90. BUILD.bazel
  91. build_infra_prod.sh
  92. cipd.ensure
  93. codereview.settings
  94. DATASTORE.md
  95. demopage.sh
  96. DEPS
  97. elements-sk-scss.bzl
  98. go.mod
  99. go.sum
  100. go_repositories.bzl
  101. launch.md
  102. LICENSE
  103. Makefile
  104. OWNERS
  105. package-lock.json
  106. package.json
  107. PRESUBMIT.py
  108. PRIVACY_POLICY.md
  109. README.md
  110. run_unittests
  111. run_unittests.go
  112. STYLEGUIDE.md
  113. tools.go
  114. tsconfig.json
  115. whitespace.txt
  116. WORKSPACE
README.md

Skia-Buildbot Repository

This repo contains infrastructure code for Skia.

Getting the Source Code

The main source code repository is a Git repository hosted at https://skia.googlesource.com/buildbot.git. It is possible to check out this repository directly with git clone or via go get.

Using git clone allows you to work in whatever directory you want. You will still need to set GOPATH in order to build some apps (recommended to put this in a cache dir). E.g.:

$ cd ${WORKDIR}
$ git clone https://skia.googlesource.com/buildbot.git
$ export GOPATH=${HOME}/.cache/gopath/$(basename ${WORKDIR})
$ mkdir $GOPATH
$ cd buildbot

Using go get will fetch the repository into your GOPATH directory along with all the Go dependencies. You will need to set GOPATH and GO111MODULE=on. E.g.:

$ export GOPATH=${WORKDIR}
$ export GO111MODULE=on
$ go get -u -t go.skia.org/infra/...
$ cd ${GOPATH}/src/go.skia.org/infra/

Note: go.skia.org is a custom import path and will only work if used like the examples here.

Install Node.js (not as root) and add the bin dir to your path. Optionally run npm install npm -g, as suggested by the npm getting started doc.

Install other dependencies:

$ sudo apt-get install \
    jq \
    python-django
$ go get -u \
    github.com/kisielk/errcheck \
    golang.org/x/tools/cmd/goimports \
    go.chromium.org/luci/client/cmd/isolate \
    go get github.com/mikefarah/yq/v4

Build ~everything:

$ make all

Generated Code

Some code is generated using go generate with external binaries. First, install the version of protoc referenced in the asset creation script and ensure it is on your PATH before other versions of protoc.

Install the necessary go packages:

$ go get -u \
  github.com/golang/protobuf/protoc-gen-go \
  golang.org/x/tools/cmd/stringer \
  google.golang.org/grpc \

To generate code run in this directory:

$ go generate ./...

Running unit tests

Install Cloud SDK.

Use this command to run the presubmit tests:

$ ./run_unittests --small