The user hnaseri is experiencing high and constant network traffic on their NAS storage, which hosts upload files for a Discourse forum using GlusterFS. Initially, they suspected a Discourse job related to image color lookup, but after investigation, they found that the issue stems from the bulk reports section in the admin panel. When accessing this section, Discourse attempts to calculate the used space on the NAS, which is slow due to the large number of files. This process causes significant network activity and timeouts, as seen in the error log: 'hijack admin/reports bulk' is still running after 90 seconds. The database is on a fast SSD, so the bottleneck is clearly the NAS. Falco suggests using Linux inspection tools to monitor system activity, while Sam notes that background jobs run via Sidekiq. hnaseri confirms that du -s is slow on the NAS, and Falco explains that the report result is cached but fails to complete due to the slow network share. The user asks if the disk size calculation can be disabled, similar to how S3 uploads are handled, to avoid this performance issue.
I am hosting all of my upload files on a NAS Storage (glusterfs).
Recently I found that there is a huge and constant network traffic on the NAS. And traced it down to discourse requesting for optimized images. Is there a job that constantly lookup these images? why? and How can I turn it off?
It is working on 25 images per 15 minutes. yes? this should be very negligible. I am seeing thousands of files being lookup every minutes.
and also looking at the bandwidth from 6 month ago, I see the same behaviour. So I think it should be something else.
However I’m pretty much sure its being done by a discourse job or somthing similar. cause when I stop discourse app, the bandwidth go away. However when I just stop discourse nginx app, the bandwidth still remains.
All our background logic happens on Sidekiq jobs. If no job is running and you still have high disk I/O it may be users visiting your website and images being served by nginx ?
First I restarted the discourse app so that the constant traffic go away. Then I went to admin panel and went to the section for bulk reports. Its been a long time that reports dont show properly here: