Inline-onebox doesn't use SiteSetting.onebox_user_agent

Bug
1

Hello,

we have recently deployed Anubis on our server to protect against resource-intense AI crawlers. It sends a static page with title “Making sure you’re not a bot!” to requests sent with browser-like user agents, does a javascript proof-of-work and then redirects to the proper page. User agents that don’t have Mozilla in the name can directly access the page.

Now the discourse onebox link previews are broken, but only for inline links.

Looking at server logs, discourse sends two different types of user agents:

  • Discourse Forum Onebox v3.4.1 for big onebox link previews (see lib/onebox/helpers.rb#L217-L226 in 03f3e84ce520d375aa1c66bb0e95cc5ed9832776)

  • Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0 Safari/605.1.15 for inline link previews (seelib/final_destination.rb#L40 in 03f3e84ce520d375aa1c66bb0e95cc5ed9832776)

I’ve tried “Force custom user agent hosts” in the admin settings, which claims “Hosts for which to use the custom onebox user agent on all requests.”, but it didn’t make a difference.

I’ve also tested changing the onebox_user_agent hidden setting, but it only changes the regular onebox link preview user agent, not the inline one.

This looks like a bug to me, wouldn’t it make more sense to use the same configurable Discourse Forum Onebox user agent for inline link previews too?

Thanks!