If you look at the .json output of a user profile, you’ll see the user_fields record (like https://meta.discourse.org/u/pfaffman.json, except there are no custom fields here). That’s where the user fields that you add in the UX are (they are actually a special kind of user_custom_field. The user_custom_fields do not seem to be in the user record in a topic stream. I’m not sure if they were once there and are now gone (this would probably be a bug). You need to make sure that your user fields are visible in those settings (show on user card, maybe?).
From what I see now, you need to create a plugin that will add those custom fields to the serializer.
If you could use groups instead of your custom user field, you might solve your problem.