Fixes #20139 Refresh does not support dry-run, so don't run it in check mode. Also add a test for this case.