Better handling of cached testconfigs (I think)