Uploads do not have source information associated with them, the only protection available is “prevent anons from downloading files”. And being told “you must sign in to download this file” is one of the most annoying things about traditional PHP forums.
Adding category source information to uploads is problematic - what if you uploaded something in Staff then want to re-share it publicly? What if you want to take it back? You’d have your hands full trying to fully specify the behavior here.
UPDATE: Secure Media Uploads