regtest: Add an option to exit after n failures

Note that running jobs are not cancelled when max failures is reached,
so we usually end up getting more failures than the maximum.
4 files changed