OK so I had a look through the previous commits and realized that what we initially wanted to do is to validate the format of the URL. That means that there is no need for us to care about whether the URL would return a valid HTTP response as that would require a different solution if we wanted to verify that.
However, I did noticed that our UrlValidator wasn’t handling a bunch of cases properly and that has been fixed in
https://github.com/discourse/discourse/commit/6ecf37c482603cba27510e41723fdd1909d0a135
I’ve also switched the validation for UserProfile#website to use our UrlValidator which removes the Regexp that has been quite problematic for us
https://meta.discourse.org/t/cannot-update-preferences-website-is-invalid/39162