I’ve made a POC branch named crawl which you can test and enchance. If it works, we’ll put it behind a site setting that will be off by default.
Note: I’ve not tested the branch but happy to help
I would be happy to accept a PR here. Not familiar with multisite stuff yet