I reprovisioned the host. It generated a new host key.
Should we keep host keys with other secrets? Maybe. But we'll have to connect over SSH to deploy the persistent host key so ...