There are some situations in which people want to have uploads kept in Discourse that are not strictly associated with existing Discourse models and properties that support uploads, i.e. “whitelisted” orphaned uploads.
For example some people want to use the Custom Wizard plugin to upload content associated with a topic (not a post per se).
The issue is that the clean_up_uploads job removes all orphaned uploads. It’s exceptions are limited to urls in specific site settings.
You could of course turn off clean_up_uploads, or increase the clean_orphan_uploads_grace_period_hours, which is what I’m sometimes forced to do. But this is not ideal.
Maybe there’s another way of “whitelisting” orphaned uploads from a plugin that I’m not seeing, but if not, it’d be great to add some mechanism of doing that into the upload cleanup process.
I’d be happy to prepare a PR along these lines, but I’m wondering if this is something others see as an issue.
For reviewables @eviltrout introduced a general mechanism where arbitrary objects could be linked to a reviewable.
Moving to a similar system for uploads would be like moving multiple mountains. It would be a nice mountain to move though cause this current system is pretty fragile.
Easiest workaround you have for now is to create a hidden site setting with a data_type for :upload. Then put it in the site settings table.
Yeah, I was thinking this. I didn’t love it as a solution as I’d need to create different a setting for each upload, which feels like an abuse of the site_settings table.
As an interim solution, before the mountain is moved, would you mind if made a PR to add an extra exception to the big “exceptions” query that uses plugin_store_row with a plugin_name like whitelisted_orphan_upload_id?
This query is already SOOOO slow, I worry that another join here is going to make it very painful.
I am not sure but maybe we start by preparing the new table in core for UploadRefrence(object_id, object_type, upload_id) and joining into that, at least it is cheap and then we can move stuff to it. There are quite a few easy moves, only post uploads would be tricky..er.
I think this is work I would prefer the team here to take, it is very fiddly.
Are you talking about Rails’ single table inheritance? It’s a really good fit for reviewables because there is core logic to a reviewable that they all share, but then extra fields.
In this case though, it seems like the problem is we’re putting upload ids all over the place, and not deleting the uploads when they’re cleared out?
Is it unsafe to delete an upload if a user removes it from their profile, because the same upload sha1 could be used elsewhere in the app? If that’s the case I think @sam’s suggestion of an UploadReference table is a good solution.
If uploads are never shared this way, I think a better solution would be to delete when those fields are NULLd out.
That’s exactly why we need the UploadReference table
I actually think that’s a very good task to learn more about Discourse internals. Might be a good task for @cvx, starting early October. We’ll do some pair programming to get things started and properly split into several “small” tasks.
UploadReference 테이블에 대해 매우 관심이 있습니다. 저희는 플러그인 내에 고아 업로드(orphan uploads)로 간주될 수 있는 업로드가 많이 존재하는 상황에 처해 있습니다. 현재 저희의 해결책은 단순히 clean_up_uploads를 끄는 것입니다. 며칠 전 확인해 보니, 이제 플러그인에서 참조하고 있는지 여부까지 파악해야 하는 스크립트로 수동으로 제거해야 할 10GB 분량의 고아 업로드가 쌓여 있습니다. 여기서 논의된 것과 같은 해결책이 있다면 저희에게 큰 도움이 될 것입니다.
또한, 이 주제에 대해 말씀드리자면. 우리는 경우에 따라 일부 업로드를 단순히 HTML img 태그로 참조하고 있습니다. 이렇게 하면 해당 게시물의 베이킹(baking)이 중단되는 것 같고, 하이퍼링크 같은 다른 요소들도 처리되지 않습니다. 업로드가 일반적으로 upload://.png 형식으로 참조되는 것을 확인했습니다. 제 질문은, 이 문자열을 어디서 찾을 수 있는가입니다. 제가 보기에는 Upload 객체에 저장되어 있지 않은 것 같습니다.
