Switch over all the Ansible scripts for switchboard to use roles.
Also update README.md and hosts.ini.
Change-Id: I9eda6cf2b2aa6fc9d195d17fd9528a0f9dc29dad
Reviewed-on: https://skia-review.googlesource.com/c/buildbot/+/430696
Reviewed-by: Kevin Lubick <kjlubick@google.com>
diff --git a/skolo/ansible/Makefile b/skolo/ansible/Makefile
index 3a43ea9..1ad1967 100644
--- a/skolo/ansible/Makefile
+++ b/skolo/ansible/Makefile
@@ -7,82 +7,8 @@
# You will need the "-f 1" because ssh over ${SSH_WRAP} with passwords seems to
# fail with a higher number of parallel forks.
-# Push a fresh copy of 'authorized_hosts' to the jumphosts.
-push_authorized_keys_to_jumphosts:
- ${SSH_WRAP} ansible-playbook push_authorized_keys_to_linux.yml --extra-vars variable_hosts=jumphosts
+install_test_machine_monitor_to_linux_incremental:
+ ${SSH_WRAP} ansible-playbook ./switchboard/install_test_machine_monitor_linux.yml --extra-vars variable_hosts=rack2_linux -l skia-e-linux-207 --ask-pass
-push_authorized_keys_to_routers:
- ${SSH_WRAP} ansible-playbook push_authorized_keys_to_router.yml --extra-vars variable_hosts=routers -f 1
-
-# Rack 1
-push_authorized_keys_to_rack1_rpis_incremental:
- ${SSH_WRAP} ansible-playbook push_authorized_keys_to_linux.yml --extra-vars variable_hosts=rack1_rpis_incremental_rollout -f 1 --ask-pass
-
-push_authorized_keys_to_rack1_rpis_full:
- ${SSH_WRAP} ansible-playbook push_authorized_keys_to_linux.yml --extra-vars variable_hosts=rack1_rpis -f 1 --ask-pass
-
-# Rack 2
-push_authorized_keys_to_rack2_linux_incremental:
- ${SSH_WRAP} ansible-playbook push_authorized_keys_to_linux.yml --extra-vars variable_hosts=rack2_linux -l skia-e-linux-207
-
-push_authorized_keys_to_rack2_linux_full:
- ${SSH_WRAP} ansible-playbook push_authorized_keys_to_linux.yml --extra-vars variable_hosts=rack2_linux
-
-push_authorized_keys_to_rack2_mac_incremental:
- ${SSH_WRAP} ansible-playbook push_authorized_keys_to_mac.yml --extra-vars variable_hosts=rack2_mac -l skia-e-mac-220 --ask-pass
-
-push_authorized_keys_to_rack2_mac_full:
- ${SSH_WRAP} ansible-playbook push_authorized_keys_to_mac.yml --extra-vars variable_hosts=rack2_mac -f 1 --ask-pass
-
-push_authorized_keys_to_rack2_win_incremental:
- ${SSH_WRAP} ansible-playbook push_authorized_keys_to_win.yml --extra-vars variable_hosts=rack2_win -l skia-e-win-201 --ask-pass
-
-push_authorized_keys_to_rack2_win_full:
- ${SSH_WRAP} ansible-playbook push_authorized_keys_to_win.yml --extra-vars variable_hosts=rack2_win -f 1 --ask-pass
-
-
-# Rack 3
-push_authorized_keys_to_rack3_linux_incremental:
- ${SSH_WRAP} ansible-playbook push_authorized_keys_to_linux.yml --extra-vars variable_hosts=rack3_linux -l skia-e-linux-307 --ask-pass
-
-push_authorized_keys_to_rack3_linux_full:
- ${SSH_WRAP} ansible-playbook push_authorized_keys_to_linux.yml --extra-vars variable_hosts=rack3_linux -f 1 --ask-pass
-
-push_authorized_keys_to_rack3_mac_incremental:
- ${SSH_WRAP} ansible-playbook push_authorized_keys_to_mac.yml --extra-vars variable_hosts=rack3_mac -l skia-e-mac-330 --ask-pass
-
-push_authorized_keys_to_rack3_mac_full:
- ${SSH_WRAP} ansible-playbook push_authorized_keys_to_mac.yml --extra-vars variable_hosts=rack3_mac -f 1 --ask-pass
-
-push_authorized_keys_to_rack3_win_incremental:
- ${SSH_WRAP} ansible-playbook push_authorized_keys_to_win.yml --extra-vars variable_hosts=rack3_win -l skia-e-win-302
-
-push_authorized_keys_to_rack3_win_full:
- ${SSH_WRAP} ansible-playbook push_authorized_keys_to_win.yml --extra-vars variable_hosts=rack3_win -f 1 --ask-pass
-
-
-# Rack 4
-push_authorized_keys_to_rack4_rpis_incremental:
- ${SSH_WRAP} ansible-playbook push_authorized_keys_to_linux.yml --extra-vars variable_hosts=rack4_rpis_incremental_rollout
-
-push_authorized_keys_to_rack4_rpis_full:
- ${SSH_WRAP} ansible-playbook push_authorized_keys_to_linux.yml --extra-vars variable_hosts=rack4_rpis
-
-add_chrome_bot_to_sudoers_rpis_rack4:
- ${SSH_WRAP} ansible-playbook add-chrome-bot-to-sudoers.yaml --extra-vars variable_hosts=rack4_rpis --ask-become-pass
-
-get_rack4_rpi_k3s_versions:
- ${SSH_WRAP} ansible-playbook rack4/get-k3s-version.yaml --extra-vars variable_hosts=rack4_rpis
-
-# Rack 5
-push_authorized_keys_to_rack5_linux_incremental:
- ${SSH_WRAP} ansible-playbook push_authorized_keys_to_linux.yml --extra-vars variable_hosts=rack5_linux -l skia-i-gapid-001 --ask-pass
-
-push_authorized_keys_to_rack5_linux_full:
- ${SSH_WRAP} ansible-playbook push_authorized_keys_to_linux.yml --extra-vars variable_hosts=rack5_linux -f 1 --ask-pass
-
-push_authorized_keys_to_all:
- -${SSH_WRAP} ansible-playbook push_authorized_keys_to_linux.yml --extra-vars variable_hosts=all_linux
- -${SSH_WRAP} ansible-playbook push_authorized_keys_to_router.yml --extra-vars variable_hosts=routers
- -${SSH_WRAP} ansible-playbook push_authorized_keys_to_mac.yml --extra-vars variable_hosts=all_mac
- -${SSH_WRAP} ansible-playbook push_authorized_keys_to_win.yml --extra-vars variable_hosts=all_win
+distrubute_authorized_keys_to_rpis:
+ ${SSH_WRAP} ansible-playbook ./switchboard/update-authorized-keys.yml --extra-vars variable_hosts=rpi
diff --git a/skolo/ansible/README.md b/skolo/ansible/README.md
index 9a31628..fb52f91 100644
--- a/skolo/ansible/README.md
+++ b/skolo/ansible/README.md
@@ -11,6 +11,17 @@
Runs might fail for a small number of hosts, you can re-run a script for a
specific host by passing `-l (hostname)` to the `ansible-playbook` command.
-You can target a server not in hosts by appending a comma to the name:
+You can target a machine not in the skolo by referring to it by IP address,
+presuming the IP address is in the range 192.168.0.0/16. This is defined in
+`hosts.ini` as the `[local]` group of machines.
- $ ansible-playbook ./add-chrome-bot-to-sudoers.yaml --extra-vars variable_hosts=192.168.1.157,
+ $ ansible-playbook ./switchboard/rpi.yml --extra-vars variable_hosts=192.168.1.157
+
+## Notes
+
+See `./group_vars/all.yml` for variables that are defined everywhere.
+
+See `hosts.ini` for all the hosts and groups of hosts you can target when
+running an Ansible script.
+
+See `ssh.cfg` for the SSH configuration that Ansible uses when running.
diff --git a/skolo/ansible/hosts.ini b/skolo/ansible/hosts.ini
index 0b82015..6759d45 100644
--- a/skolo/ansible/hosts.ini
+++ b/skolo/ansible/hosts.ini
@@ -27,6 +27,9 @@
rack3_linux
rack5_linux
+[switchboard_rpis]
+skia-rpi2-rack4-shelf1-[001:008]
+
[rpis:children]
rack4_rpis
rack1_rpis
@@ -252,4 +255,4 @@
# This allows targeting devices not in the lab for development.
[local]
-192.168.[1:255].[1:255]
\ No newline at end of file
+192.168.[1:255].[1:255]
diff --git a/skolo/ansible/switchboard/README.md b/skolo/ansible/switchboard/README.md
new file mode 100644
index 0000000..507bd3e
--- /dev/null
+++ b/skolo/ansible/switchboard/README.md
@@ -0,0 +1,6 @@
+# Switchboard Ansible Scripts
+
+Thse are scripts for managing devices in the skolo that are setup for
+Switchboard.
+
+Always run these script from the parent directory: //infra/skolo/ansible.
diff --git a/skolo/ansible/switchboard/install_test_machine_monitor.yml b/skolo/ansible/switchboard/install_test_machine_monitor.yml
new file mode 100644
index 0000000..93364f3
--- /dev/null
+++ b/skolo/ansible/switchboard/install_test_machine_monitor.yml
@@ -0,0 +1,7 @@
+# Installs test_machine_monitor.
+- hosts: "{{ variable_hosts | default('rpis') }}"
+ user: chrome-bot
+ gather_facts: true
+
+ roles:
+ - { role: install_test_machine_monitor, start_swarming: true }
diff --git a/skolo/ansible/switchboard/linux.yml b/skolo/ansible/switchboard/linux.yml
new file mode 100644
index 0000000..7c6953a
--- /dev/null
+++ b/skolo/ansible/switchboard/linux.yml
@@ -0,0 +1,18 @@
+# Configures a Linux machine with everything it needs to run under switchboard.
+- hosts: "{{ variable_hosts | default('linux') }}"
+ user: root
+ gather_facts: yes
+
+ pre_tasks:
+ - name: Install applications.
+ become: yes
+ apt:
+ pkg:
+ - rsync
+
+ roles:
+ - copy_authorized_keys
+ - copy_service_account_key
+ - install_collectd
+ - swarming_needs
+ - { role: install_test_machine_monitor, start_swarming: false }
diff --git a/skolo/ansible/switchboard/prepare-linux-for-ansible.yml b/skolo/ansible/switchboard/prepare-linux-for-ansible.yml
new file mode 100644
index 0000000..4cb8e37
--- /dev/null
+++ b/skolo/ansible/switchboard/prepare-linux-for-ansible.yml
@@ -0,0 +1,23 @@
+# Does the absolute minimum for a linux test machine.
+#
+# Once this is run then the 'linux.yml' playbook can be run to finish/update the
+# machine.
+- hosts: '{{ variable_hosts }}'
+ user: root
+ gather_facts: no
+
+ pre_tasks:
+ # First install python3 so we can use more issue more than 'raw' commands.
+ - name: install python3
+ raw:
+ 'apt update && apt -y install sudo python3 python3-apt python3-distutils'
+
+ # We can gather facts now that python3 is installed.
+ # Needed for create_chrome_bot_user.
+ - name: gather facts
+ setup:
+
+ roles:
+ - set_root_password
+ - create_chrome_bot_user
+ - copy_authorized_keys
diff --git a/skolo/ansible/switchboard/prepare-rpi-for-ansible.yml b/skolo/ansible/switchboard/prepare-rpi-for-ansible.yml
new file mode 100644
index 0000000..59ab8e8
--- /dev/null
+++ b/skolo/ansible/switchboard/prepare-rpi-for-ansible.yml
@@ -0,0 +1,24 @@
+# Does the absolute minimum for an RPI with an SD card that's been configured
+# via //infra/skolo/rasperry-pi/switchboard/configure-image.sh
+#
+# Once this is run then the 'rpi.yml' playbook can be run to finish/update the
+# machine.
+- hosts: '{{ variable_hosts }}'
+ user: root
+ gather_facts: no
+
+ pre_tasks:
+ # First install python3 so we can use more issue more than 'raw' commands.
+ - name: install python3
+ raw:
+ 'apt update && apt -y install sudo python3 python3-apt python3-distutils'
+
+ # We can gather facts now that python3 is installed.
+ # Needed for create_chrome_bot_user.
+ - name: gather facts
+ setup:
+
+ roles:
+ - set_root_password
+ - create_chrome_bot_user
+ - copy_authorized_keys
diff --git a/skolo/ansible/switchboard/rpi.yml b/skolo/ansible/switchboard/rpi.yml
new file mode 100644
index 0000000..6b1cb43
--- /dev/null
+++ b/skolo/ansible/switchboard/rpi.yml
@@ -0,0 +1,21 @@
+# Configures an RPi with everything it needs to run under switchboard.
+- hosts: "{{ variable_hosts | default('rpis') }}"
+ user: chrome-bot
+ gather_facts: yes
+
+ pre_tasks:
+ - name: Install applications.
+ become: yes
+ apt:
+ pkg:
+ - adb
+ - libimobiledevice-utils
+ - rsync
+
+ roles:
+ - copy_authorized_keys
+ - copy_adbkey
+ - copy_service_account_key
+ - install_collectd
+ - swarming_needs
+ - { role: install_test_machine_monitor, start_swarming: true }
diff --git a/skolo/ansible/switchboard/update-authorized-keys.yml b/skolo/ansible/switchboard/update-authorized-keys.yml
new file mode 100644
index 0000000..6d7755c
--- /dev/null
+++ b/skolo/ansible/switchboard/update-authorized-keys.yml
@@ -0,0 +1,5 @@
+# Distributes authorized_keys to the hosts.
+- hosts: "{{ variable_hosts | default('all') }}"
+ gather_facts: yes
+ roles:
+ - copy_authorized_keys
diff --git a/skolo/raspberry-pi/switchboard/README.md b/skolo/raspberry-pi/switchboard/README.md
index b24377a..ff95025 100644
--- a/skolo/raspberry-pi/switchboard/README.md
+++ b/skolo/raspberry-pi/switchboard/README.md
@@ -11,68 +11,33 @@
https://raspi.debian.net/verified/20210629_raspi_4_bullseye.img.xz
-Download that and burn it to an SD card. After it has been burned, reload the SD
-card and run:
+This image is backed up at:
+
+ gs://skia-skolo/skolo-images/switchboard/
+
+Download that and burn it to an SD card.
+
+[Balena Etcher](https://www.balena.io/etcher/) is a nice GUI application that
+runs on all platforms that allows you to burn multiple SD cards at the same
+time.
+
+After it has been burned, reload the SD card and run:
./configure-image.sh <machine-name>
Once the SD card has been placed in an RPi and is running in the lab:
-1. Add the hostname to //skolo/ansible/hosts.ini.
-2. Run:
+1. Add the hostname to //skolo/ansible/hosts.ini, making sure it ends up as part
+ of `switchboard_rpis`.
+2. Then run the ansible scripts to configure the running RPi:
```bash
$ cd //skolo/ansible/
- $ ansible-playbook ./switchboard/setup-switchboard-rpi.yml \
+ $ ansible-playbook ./switchboard/prepare-rpi-for-ansible.yml \
--extra-vars variable_hosts=<machine-name>
-
- $ cd //machine
- $ make build_test_machine_monitor_rpi
- $ TARGET=<machine-name> make push_test_machine_monitor_rpi
+ $ ansible-playbook ./switchboard/rpi.yml \
+ --extra-vars variable_hosts=<machine-name>
```
Now the RPi should be fully setup with adb, idevice-\*, a recent copy of
authorized_keys, and running test_machine_monitor.
-
-## Testing outside the skolo.
-
-_This only applies to testing RPis outside the skolo._
-
-Outside the skolo the RPi will not have a metadata server to talk to, so
-`test_machine_monitor` will fail to run. You can get `test_machine_monitor` to
-run by supplying another form of Google Application Credentials. One way is to
-create a service account key, copy it over the RPI, and then set an Env variable
-in the `test_machine_monitor.service` file.
-
-1. Use an existing service account key, or visit the cloud console page and
- generate a new key.
-2. Copy that key over to the RPi:
-
- scp key.json chrome-bot@192.168.1.107:/home/chrome-bot/key.json
-
-3. Update the `test_machine_monitor.service` to add the
- `GOOGLE_APPLICATION_CREDENTIALS` environment variable that points to the
- service account key.
-
- [Unit]
- Description=test_machine_monitor
- After=syslog.target network.target
-
- [Service]
- Type=simple
- User=chrome-bot
- Environment=GOOGLE_APPLICATION_CREDENTIALS=/home/chrome-bot/key.json
- ExecStart=/usr/local/bin/test_machine_monitor \
- --start_switchboard
- Restart=always
-
- [Install]
- WantedBy=multi-user.target
-
-4. Force the new config to be loaded:
-
- systemctl daemon-reload
-
-5. Restart the service
-
- systemctl restart test_machine_monitor
