If mkdocs is running in parallel, it can get upset if one of these
files exists when it lists the directory, but doesn't exist when it
goes to generate the output. Fundamentally this is a problem in mkdocs
more than in our code, but we might as well avoid it.
This got... long... and complicated. But I think it's a really good
demonstration of getting redo to do complicated things elegantly. At
least, I hope it is.