It turns out the two issues are related. Some emojis include the Unicode U+FE0F character, which is the VARIATION SELECTOR-16 selector, an “invisible codepoint which specifies that the preceding character should be displayed with emoji presentation”. Since the preceding code point is turned into an image, Safari gets confused, and it displays this rectangle instead.
https://github.com/discourse/discourse/pull/6456
Note that the PR trims U+FE0F when replacing unicode emojis but I wonder whether it breaks emojis when they get converted back to Unicode in push notifications and the like.