I did a quick repro of sorts which can probably help solve the issue
- create
authors - Topic A, assign steaky.
- Topic B, assign omaoma.
- Reply in topic B, assign chi.west to the post.
The highlighted “Assigned (2)” goes briefly from “Assigned (3)” then back to “Assigned (2)”. I think it’s fighting itself 
I think we can go about this based on “total number of assignments” rather than “how many topics were assigned”.
Let’s see if I can get this in…