Can you make sure your site is configured to serve everything in https? Passkey challenge verification requires that all requests go through https. Also the domain between browser and server must match exactly. If there is a mismatch somewhere, the verification will fail.
We have a setting for this, force_https, you can try that, it may help (though be careful, it might also lock you out if server isn’t configured properly).