This file documents things that don't belong to a specific service.
Items below here should include target links from alerts.
This means a given disk on one of our machines has a low disk. Running out of disk space causes problems, so we try to keep a healthy buffer (which varies depending on the total disk size). For machines running Swarming, this can cause issues when trying to download a task from Isolate, which has been a problem before ().
To fix, connect to the machine, and use
df -h or a similar command to identify which disk(s) are low.
du -hd 2 can be a useful tool for identifying which folders are taking up a lot of space.
sudo apt-get clean
/b/s/*_cachefolders have gotten very large. If so, stop swarming, delete the folders, and reboot.
sudo docker system prune -fa.
If many machines are experiencing this, you may want to use the run_on_swarming_bots script to fix them all at once.
Key metrics: collectd_df_df_complex