Summary
If your Discourse site looks like plain, unstyled HTML when accessed via a link in the Facebook app on iPhone — no CSS, no JavaScript, no functionality — the cause is Discourse’s crawler detection incorrectly identifying Facebook’s in-app browser as a bot.
The fix is a one-line change via the Rails console.
The symptom
Users clicking links to your Discourse site from the Facebook app on iPhone see a stripped-down, unstyled HTML page — essentially the crawler/noscript layout. No JavaScript runs, so features like image grids, lightboxes, and media players don’t work. The same links work correctly in Safari, Chrome, and Facebook’s in-app browser on Android and iPad.
The cause
Facebook’s in-app browser on iPhone identifies itself with a user agent string containing the word facebook, for example:
Mozilla/5.0 (iPhone; CPU iPhone OS 18_7 like Mac OS X) AppleWebKit/605.1.15
(KHTML, like Gecko) Mobile/23D8133 Safari/604.1 MetaIAB Facebook
Discourse’s crawler detection (CrawlerDetection) checks user agents against the crawler_user_agents site setting, whose default value includes facebook:
rss|bot|spider|crawler|facebook|archive|wayback|ping|...
This causes Discourse to serve Facebook’s in-app browser the static crawler layout instead of the full Ember application — even though it’s a real browser being used by a real person.
You can confirm this is happening by checking your nginx access log for requests from this browser and noting the response payload size is dramatically smaller than normal (typically ~5KB vs ~35KB for a full topic page).
The fix
Add MetaIAB to the crawler_check_bypass_agents site setting. This setting is specifically designed to exempt user agents from crawler treatment even when they match the crawler list.
Note: crawler_check_bypass_agents is a hidden site setting and does not appear in the standard admin UI. The Rails console is required.
Via the Rails console
SiteSetting.crawler_check_bypass_agents = "MetaIAB"
If the setting already has a value (e.g. cubot), append with a pipe separator:
SiteSetting.crawler_check_bypass_agents = "cubot|MetaIAB"
The change takes effect immediately — no restart required.
Why this works
The CrawlerDetection.crawler? method uses three settings in combination:
non_crawler_user_agents— if the UA matches this list, it might be a real browsercrawler_user_agents— if it also matches this list, it’s treated as a crawlercrawler_check_bypass_agents— if it matches this list, it’s exempted from crawler treatment regardless
Facebook’s in-app browser UA contains Safari, which matches non_crawler_user_agents. It also contains facebook, which matches crawler_user_agents. Adding MetaIAB — a string unique to Facebook’s in-app browser UA — to crawler_check_bypass_agents causes Discourse to serve it the full application.
Why Android and iPad are unaffected
- Android: Facebook’s in-app browser on Android sends a different user agent that does not contain
facebook, so it passes crawler detection without issue. - iPad: Facebook’s in-app browser on iPad also triggers the crawler layout, but because the iPad reports a wide
window.innerWidth(~1180px), Discourse serves the desktop layout which happens to render adequately. The iPhone’s narrow viewport (~414px) triggers the mobile layout, which in crawler mode is completely non-functional.
Additional note: meta-webindexer flooding
Separately, Facebook’s web indexer (meta-webindexer/1.1) may send a very high volume of requests to your site — potentially thousands per hour — all targeting the homepage. Unlike meta-externalagent (which handles OG link previews and should remain unblocked), meta-webindexer appears to serve no useful purpose for most Discourse installations.
If you observe this traffic in your logs, you can block it at the nginx level by adding it to your bot block configuration:
"~*meta-webindexer" 1;
meta-externalagent should remain allowed, as it is responsible for scraping OG metadata when links are shared on Facebook.