Automatic daily backups do not work


(Daniela) #1

Last backup is dated 11 December.
Backup settings:

Backup page:

Backup logs: empty

Generally when backups stop working, the server is out of storage (we must delete the old docker images).

But now the space not seem to be the problem

Any help is appreciated, thanks.

Edit: Mmm…I see a lot of errors log that we never had before (a lot of warning but no errors)


Backup issue - undefined method `every' for Jobs::CreateBackup:Class
I can't get a good backup - it fails after after_create_hook
(Daniela) #2

We arrived at about 40.000 errors.

/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/i18n-0.7.0/lib/i18n/backend/base.rb:37:in `throw'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/i18n-0.7.0/lib/i18n/backend/base.rb:37:in `translate'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/i18n-0.7.0/lib/i18n/backend/fallbacks.rb:38:in `translate'
/var/www/discourse/lib/freedom_patches/translate_accelerator.rb:128:in `translate'
/var/www/discourse/lib/email/message_builder.rb:94:in `body'
/var/www/discourse/lib/email/message_builder.rb:103:in `build_args'
/var/www/discourse/lib/email/message_builder.rb:9:in `build_email'
/var/www/discourse/app/mailers/user_notifications.rb:323:in `send_notification_email'
/var/www/discourse/app/mailers/user_notifications.rb:223:in `notification_email'
/var/www/discourse/app/mailers/user_notifications.rb:128:in `user_posted'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.2.4/lib/abstract_controller/base.rb:198:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.2.4/lib/abstract_controller/callbacks.rb:20:in `block in process_action'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.2.4/lib/active_support/callbacks.rb:88:in `__run_callbacks__'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.2.4/lib/active_support/callbacks.rb:778:in `_run_process_action_callbacks'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.2.4/lib/active_support/callbacks.rb:81:in `run_callbacks'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.2.4/lib/abstract_controller/callbacks.rb:19:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.2.4/lib/abstract_controller/base.rb:137:in `process'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionview-4.2.4/lib/action_view/rendering.rb:30:in `process'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionmailer-4.2.4/lib/action_mailer/base.rb:596:in `block in process'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.2.4/lib/active_support/notifications.rb:164:in `block in instrument'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.2.4/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.2.4/lib/active_support/notifications.rb:164:in `instrument'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionmailer-4.2.4/lib/action_mailer/base.rb:593:in `process'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionmailer-4.2.4/lib/action_mailer/base.rb:584:in `initialize'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionmailer-4.2.4/lib/action_mailer/message_delivery.rb:25:in `new'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionmailer-4.2.4/lib/action_mailer/message_delivery.rb:25:in `__getobj__'
/usr/local/lib/ruby/2.0.0/delegate.rb:70:in `method_missing'
/var/www/discourse/lib/email/sender.rb:26:in `send'
/var/www/discourse/app/jobs/regular/user_email.rb:69:in `execute'
/var/www/discourse/app/jobs/base.rb:154:in `block (2 levels) in perform'

No one has any idea?


(Chris Saenz) #3

When I had this happen it was indeed the storage space, even though it appeared to be plenty. Running the process uses a lot more space than the finished zip file. Eventually we moved to a larger drive.

Try deleting one of the older backups and give it another shot.


(Allen - Watchman Monitoring) #4

At first my host was low on disk space, but a full rebuild with 10+ gb free hasn’t changed anything


(Jeff Atwood) #5

No repro on my instances on Digital Ocean, on latest code, backups are working fine.


(Daniela) #6

Just for knowledge, when your backups do not work, do you get the email notifications?

We will try this night.
I am hoping for better results than @watchmanmonitor :pray:

Thanks anyway.


(Chris Saenz) #7

No. As far as I know there’s no notification for failed backups. It’s on my wishlist…


(Daniela) #8

I do not know what happened, but since yesterday all fatal errors are gone except this one:

RuntimeError (can't add a new key into hash during iteration) /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/message_bus-1.0.16/lib/message_bus/connection_manager.rb:102:in[]=’`

/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/message_bus-1.0.16/lib/message_bus/connection_manager.rb:102:in `[]='
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/message_bus-1.0.16/lib/message_bus/connection_manager.rb:102:in `subscribe_client'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/message_bus-1.0.16/lib/message_bus/connection_manager.rb:82:in `block in add_client'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/message_bus-1.0.16/lib/message_bus/connection_manager.rb:81:in `each'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/message_bus-1.0.16/lib/message_bus/connection_manager.rb:81:in `add_client'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/message_bus-1.0.16/lib/message_bus/rack/middleware.rb:159:in `add_client_with_timeout'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/message_bus-1.0.16/lib/message_bus/rack/middleware.rb:121:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/rack-1.6.4/lib/rack/session/abstract/id.rb:225:in `context'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/rack-1.6.4/lib/rack/session/abstract/id.rb:220:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.2.4/lib/action_dispatch/middleware/cookies.rb:560:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.2.4/lib/active_record/query_cache.rb:36:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.2.4/lib/active_record/connection_adapters/abstract/connection_pool.rb:653:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.2.4/lib/action_dispatch/middleware/callbacks.rb:29:in `block in call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.2.4/lib/active_support/callbacks.rb:88:in `__run_callbacks__'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.2.4/lib/active_support/callbacks.rb:778:in `_run_call_callbacks'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.2.4/lib/active_support/callbacks.rb:81:in `run_callbacks'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.2.4/lib/action_dispatch/middleware/callbacks.rb:27:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.2.4/lib/action_dispatch/middleware/remote_ip.rb:78:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.2.4/lib/action_dispatch/middleware/debug_exceptions.rb:17:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.2.4/lib/action_dispatch/middleware/show_exceptions.rb:30:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/logster-1.0.1/lib/logster/middleware/reporter.rb:31:in `call'
/var/www/discourse/config/initializers/silence_logger.rb:24:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.2.4/lib/action_dispatch/middleware/request_id.rb:21:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/rack-1.6.4/lib/rack/methodoverride.rb:22:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/rack-1.6.4/lib/rack/runtime.rb:18:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/rack-1.6.4/lib/rack/sendfile.rb:113:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/rack-mini-profiler-0.9.7/lib/mini_profiler/profiler.rb:170:in `call'
/var/www/discourse/lib/middleware/request_tracker.rb:73:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/railties-4.2.4/lib/rails/engine.rb:518:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/railties-4.2.4/lib/rails/application.rb:165:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/railties-4.2.4/lib/rails/railtie.rb:194:in `public_send'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/railties-4.2.4/lib/rails/railtie.rb:194:in `method_missing'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/rack-1.6.4/lib/rack/urlmap.rb:66:in `block in call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/rack-1.6.4/lib/rack/urlmap.rb:50:in `each'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/rack-1.6.4/lib/rack/urlmap.rb:50:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/unicorn-4.9.0/lib/unicorn/http_server.rb:580:in `process_client'
/var/www/discourse/lib/scheduler/defer.rb:85:in `process_client'
/var/www/discourse/lib/middleware/unicorn_oobgc.rb:95:in `process_client'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/unicorn-4.9.0/lib/unicorn/http_server.rb:674:in `worker_loop'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/unicorn-4.9.0/lib/unicorn/http_server.rb:529:in `spawn_missing_workers'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/unicorn-4.9.0/lib/unicorn/http_server.rb:140:in `start'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/unicorn-4.9.0/bin/unicorn:126:in `<top (required)>'
/var/www/discourse/vendor/bundle/ruby/2.0.0/bin/unicorn:23:in `load'
/var/www/discourse/vendor/bundle/ruby/2.0.0/bin/unicorn:23:in `<main>'

Backups are still not working but my sysadmin is out, so until January we can not update to beta 7 or delete old backups.


(Sam Saffron) #9

Not following, why can’t you run:

cd /var/discourse
git pull
./launcher rebuild app

(Daniela) #10

Yes I know, but I do not have access to the server :stuck_out_tongue_winking_eye:, only sysadmin can do it.


(Sam Saffron) #11

You going to need a sysadmin then :slight_smile: