I’ve just been setting up s3 backups for a discourse installation, and have run into problems. the site is now giving me 500 errors for all pages.
I’d entered at least most of the s3 setup, but the backups weren’t getting uploaded, and I was looking for why that was. I had entered bucket details for the backups, but not bucket details for general file uploads.
I’m a bit hazy on the sequence of events. I wound up doing a container restart and even a rebuild at one point, which hasn’t fixed things, but I’m fairly sure the error had started occurring before that.
In the rails/production.log file, I see:
Job exception: undefined method `every' for Jobs::CreateBackup:Class
Job exception: undefined method `every' for Jobs::CreateBackup:Class
Started GET "/" for 161.202.72.162 at 2015-09-16 02:35:11 +0000
Processing by ListController#latest as HTML
Rendered list/list.erb within layouts/application (1.6ms)
Rendered layouts/_head.html.erb (3.7ms)
Rendered common/_special_font_face.html.erb (0.6ms)
Rendered common/_discourse_stylesheet.html.erb (0.3ms)
Rendered application/_header.html.erb (0.3ms)
Rendered common/_discourse_javascript.html.erb (2.3ms)
Completed 500 Internal Server Error in 75ms (ActiveRecord: 22.1ms)
ActionView::Template::Error (s3_upload_bucket)
/var/www/discourse/lib/file_store/s3_store.rb:95:in `s3_bucket'
Job exception: undefined method `every' for Jobs::CreateBackup:Class
Job exception: undefined method `every' for Jobs::CreateBackup:Class
The Job exception lines repeat at around one such error per second. The bit in between is the output for a single web page request.
Any suggestions on how I should proceed? I’m guessing I might need to go in through the rails console to adjust settings. What are the names of the settings involved?
‘git pull’ and ‘./launcher rebuild app’ doesn’t fix the problem, and it potentially could have been part of causing the problem. i.e. the problem is not simply that discourse is out of date, but it could be that something in an older database hasn’t been updated to be compatible with the modern code.
I’ve just become involved with this instance of discourse. I’m not entirely sure when it was originally built, but I have noticed that the containers/app.yml file had some differences from what is in https://github.com/discourse/discourse_docker/blob/master/samples/standalone.yml . e.g. /var/log didn’t have a volume. I presume this is a reflection of this discourse instance having been around a while. Maybe there’s some hang-over from an earlier time in either the config outside the container, or in the database.
I expect the bit about “every” in the “Jobs::CreateBackup” class is not the primary issue. It goes back before the current failure to render pages at all, and before there was any s3 configuration in place. I’m obviously more concerned with the complete unavailability of the site, and unless you think it’s important I’ll ignore the Jobs::CreateBackup issue until the other is resolved.
Looking at the error above, I see the reference to /var/www/discourse/lib/file_store/s3_store.rb:95, which is the 3rd line below:
def s3_bucket
return @s3_bucket if @s3_bucket
raise Discourse::SiteSettingMissing.new("s3_upload_bucket") if SiteSetting.s3_upload_bucket.blank?
@s3_bucket = SiteSetting.s3_upload_bucket.downcase
end
It seems to me that:
it’s odd that the s3 code is being invoked for rendering pages, but perhaps this relates to some sort of poor man’s cron arrangement with job execution hanging off page rendering calls?
either s3 should only be invoked if the requisite settings are there, or the exception should be graciously handled and other parts of the request allowed to proceed.
fixing the s3_upload_bucket setting might help
fixing some other setting might cause this code not to be invoked in the first place, but I don’t yet know where to look for that.
Could it be that the problem arises because I have set SiteSetting.s3_backup_bucket, but not SiteSetting.s3_upload_bucket? That seems reasonable to do in that I’m curently concerned with off-site backups, not storing uploads off-site, but perhaps it doesn’t work to do this?
SiteSetting.s3_upload_bucket = SiteSetting.s3_backup_bucket does indeed bring the site back up. It’s not really what I want to do with uploaded files though.
Looks like I had SiteSetting.enable_s3_uploads set, which is quite understandably incompatible with not setting SiteSetting.s3_upload_bucket, but it should be handled more gracefully.