This does happen. If the topic belongs to a restricted category we add invited user access to those allowed groups.
But the user was logged in, right?
So if the user is logged in (I assume so because the second invite was via username) add the user to the required group manually and they should be able to view the topic.