Cherry pick from 34b0e9037f: Drop default number of reruns to 1