Back in the days I worked at Stack Overflow, my day always started the same way.
- Navigate to the errors page
- Look through the errors we got overnight
- Sort them out
- Continue with my day
We used an error logging solution that was based off Google Code Archive - Long-term storage for Google Code Project Hosting. over time we rewrote most of it. Life was good. I even blogged about some of our extensions to it.
Fast forward a few year, I am working with Discourse and was very much dismayed to find there is no such thing for Rails apps.
Instead, a pretty disturbing pattern has emerged in the community. People are outsourcing errors or simply leaving it in a impossible to understand enormous local log.
In the outsource-this-camp you have http://www.exceptional.io/ , https://airbrake.io/ and the open source GitHub - errbit/errbit: The open source error catcher that's Airbrake API compliant that we use.
There is also the built in text rails log, it has no web interface, and logs so much data it is very hard to make any sense of it.
###Why I strongly oppose the outsourcing approach for Discourse
- Discourse should stand on its own without dependencies on third party services
We need the ability to quickly troubleshoot installs, pick up on various errors early and address them. If we expect users installing Discourse to go about installing and configuring yet another service, just to get something basic like logging, we have lost the game.
- We need the service to be “close” to the application.
We have the technology to notify administrators when stuff is going pear shape. We already have an admin panel that has room for this. We need to remove all barriers from getting access to failures. When a request fails we want to know which user had the failure.
###What I want?
It has always been our goal to help Ruby and Rails the ecosystem. I would really like our error log to be engineered externally to our app as Rack middleware, with hooks to notify the main application as needed on failure.
At a very basic level we want a row for each HTTP request that fails and each jobs that fails.
###Absolute musts for v0.1
- Implemented as rack middleware, with minimal new gem dependencies (if any)
- No new storage system dependencies (no mongo, no hadoop clusters etc.)
- A flexible storage engine (file / postgres / redis)
- View backtraces per error
- Group the same errors in a row
- Ships with Discourse
Before I go about reinventing wheels, is there a standalone rack middleware that already fill this need?