Ah, I thought it might be too optimistic. 
The site settings that need a rake task running after enabling/disabling often have a hint in the setting description (eg. Add rel nofollow to user content or spoiler enabled). If you do need the rake task for this one then adding that to the setting description like the others may be a useful addition.