Okay yeah definitely doesn’t sound like that many people would be toggling in that time 
Are you able to do a screen recording to show what’s happening?
I’d also advise you take a look at the Network tab in Chrome Developer tools. It should show 2 requests like this after you save it (first one is the one to show the prompt of XX users, second one is to actually save the setting and do the backfill):
The second one needs to have a 200 status, if it doesn’t and you navigate away then it will not have backfilled for existing users.