Discourse failed to backup, how to debug?

Okay I’ve spent over a day on this now. No solution but have narrowed it down and I think @sam needs to look at it. I believe the bug is in mini_racer, which seems to be crashing randomly during transpilation.

To reproduce the bug in development mode:

  1. rm -rf tmp
  2. redis-cli flushall
  3. Create a backup in /admin/backups

It should crash on “Notifying ‘eviltrout’ of the end of the backup…”

The process will be using 100% CPU, and you need to kill it before testing again.

Notes:

  • The backup takes place in a fork from unicorn which is forked. I think this is important as running it from a rails console does not create the same issue. If you recall, we were able to crash discourse altogether when we were fiddling with PrettyText warming up before forking. I think miniracer is a little delicate when being forked in our app.

  • The file it crashes while transpiling changes. Sometimes it’s the first file, sometimes it’s the fourth file, etc.

  • Because some files will succeed, if you don’t rm -rf tmp the site will eventually start working as it will have cached all the files it needs to transpile. This is why it took forever to debug, because it would eventually fix itself!

  • The mini racer is supposed to have a 15s timeout on eval, but even if you wait 15s it never continues.

6 Likes