Hi @lee-dohm 
I couldn’t reproduce this issue locally with the steps you provided. After looking at the code, I confirmed that the Priority order is ascending, and Priority (reverse) descending. Also, when two items have the same priority, we order them by the created_at value in descending order.
Could you check if the something else flags have a higher score than the off-topic one? The formula takes different things into account to calculate the score.
Let me know and I’ll keep investigating if necessary!