Could the attachment not have a different hashed URL for each post it appears in so indeed the permissions follow from the post? And only allow access to it if the user also has access to the related post?
This is a fairly serious issue because the presumption is that the attached file has the same security settings as the post it goes in… this is not true at the moment and may result in unpleasant surprises. In the meantime and until this issue is solved, how about adding a warning note to the upload interface along the lines of “Note: attachments to posts in this forum, even in secure categories, are available to any user with the attachment URL.”
On a related note, will google be spidering the attachments? It would be a shame and a surprise to users if uploaded attachments to private groups appear in google searches.