Hey Discourse Team,
This marks the 1-year-and-1-month anniversary of the Rails 5 release, it looks like 5.1.3 will be out soon (second release candidate lands today). Wondering if there have been any thoughts on upgrading Discourse to Rails 5.x anytime soon? Why or why not?
I think our main problem is that Active Record gets slower in each release? Not sure, but we worry much more about “keeping up with the Joneses” for Ember (client side JS) releases than Rails on the server side, which is quite mature.
It is on my list to get us to the latest Rails but it isn’t our priority at the moment because none of the new features are useful for us. Right now, we’re working with a GSOC student to get our Rails benchmarks back in order on https://rubybench.org to have an idea of how performance is going to be like on Rails 5.
I think upgrading to Rails 5 would be a good idea since the Action Cable could be useful especially for getting notifications…
We already use a much better library to handle all the notifications/updates 
This has kind of changed because of Automatic encoding of parsed URL params , if the upgrade is not to painful and there is no perf regression we should go ahead.
I am in for a long long ride…
undefined method `xhr' for #<RSpec::ExampleGroups::AdminDashboardController::WhileLoggedInAsAnAdmin::Index::VersionCheckingIsDisabled:0x00564a2c54c820>
Ok I have tests green going for the main code base in
https://github.com/discourse/discourse/pull/5122
When I thought I was done, I realized I’ll have to go through all our plugins as well otherwise 
Will grind through those next week!
Rails 5.1 is now deployed on meta. I’m seeing performance regressed on the topics and categories page ATM so we can’t roll it out immediately to everyone. Another notable change is that Activerecord’s pluck is notably slower which I’ve submit a PR for in
https://github.com/rails/rails/pull/30524
Note that we actually override the default ActiveRecord#pluck and use our own implementation of fast_pluck. However, they code path being fixed here is shared between both implementation.
I’ll continue to profile the topics and categories page to see where the extra time is being spent on.
Can you please specify how much, in percent?
Your Results: (note for timings- percentile is first, duration is second in millisecs)
---
categories_admin:
50: 18
75: 18
90: 19
99: 27
home_admin:
50: 23
75: 23
90: 24
99: 29
topic_admin:
50: 18
75: 18
90: 19
99: 25
categories:
50: 43
75: 43
90: 48
99: 50
home:
50: 46
75: 48
90: 53
99: 71
topic:
50: 19
75: 20
90: 21
99: 26
Your Results: (note for timings- percentile is first, duration is second in millisecs)
---
categories_admin:
50: 21 # 16.6%
75: 21
90: 24
99: 32
home_admin:
50: 25 # 8.7% slower
75: 25
90: 26
99: 31
topic_admin:
50: 21 # 16.6% slower
75: 22
90: 23
99: 28
categories:
50: 44 # 2% slower
75: 46
90: 50
99: 73
home:
50: 48 # 4.3% slower
75: 53
90: 54
99: 80
topic:
50: 22 # 15.7% slower
75: 23
90: 25
99: 32
I ran the bench again this morning and included the results and regression in percentage below.
Topic is an absolute showstopper, that is a core page. Categories can be too, if it is the homepage.
OK @tgxworld says this is ActiveRecord getting slower and slower in each new version, such that even the simple / basic queries are contributing a lot to general slowdown 
In a particular kick in the pants, even our “fast pluck” @sam wrote got slower because it relied on parts of Rails internals that also got substantially slower!
Here are our priorities for the Rails 5 upgrade:
We need to bypass ActiveRecord since it keeps getting slower, especially on the topic page. This is the most important page in our application! Let’s target a 20% speed improvement on the topic page, by bypassing ActiveRecord and doing raw queries.
Make sure our https://rubybench.org/ reflects this loss in performance so we can (cough) encourage people to take action on this.
The categories page, and other key pages, should be at roughly the same speed as Rails 4, within 5%.
We created this particular bench just for this particular case RubyBench - Long Running Ruby Benchmark
That sounds true, but it also seems like a huge drag. I’m still (almost?) a Rails novice, but it seems like ActiveRecord is sort of the basis of the whole endeavor. And then writing around it will make things all the more fragile for future updates. Yuck.
Unavoidable since the performance is dismal, if you ever care about performance. Also not quite true, it won’t be fragile – the database is the model, not the code.
These SQL-to-ORM things are always a mixed bag. You give up a lot for what you get, and it’s almost a religious issue, like many things in programming. See Object-Relational Mapping is the Vietnam of Computer Science
Some updates on this, it turns out that our profile script used an incorrect topic_id in the URL which means that we’ve been profiling against a private message page with one post instead of a public topics page with multiple posts. I’ve fixed it in
https://github.com/discourse/discourse/commit/4142bed1af8cf6d5ca4accd2205d3f7ff5277398
and the following is the correct benchmark for our topics page:
# Rails 4.2.9
Your Results: (note for timings- percentile is first, duration is second in millisecs)
---
topic:
50: 53
75: 54
90: 60
99: 76
timings:
load_rails: 1188
ruby-version: 2.4.1-p111
rss_kb: 186964
pss_kb: 123467
virtual: physical
architecture: amd64
operatingsystem: Ubuntu
memorysize: 15.59 GB
kernelversion: 4.10.0
physicalprocessorcount: 1
processor0: Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz
rss_kb_3476: 350996
pss_kb_3476: 285127
rss_kb_3557: 349488
pss_kb_3557: 284004
rss_kb_3792: 349364
pss_kb_3792: 283873
I noticed an inefficiency in our code base today and managed to recover performance with
https://github.com/discourse/discourse/commit/5d4221fbe1cd2000ac10088c5ea8a8016eb64ad8
About a 22% increase for the 50th percentile. Note that this isn’t Rails 5.1 related, just a performance regression that we’ve been carrying around for some time.
On the Rails 5 front, I’ve basically been adding more benchmarks to RubyBench so that I can use those metrics to work with the Rails team and get us back up to speed.
I’ve just deployed meta back on Rails 5.1 after we reverted it thinking that it caused Can't change "default locale". Let me know if you notice any new bugs 
@codinghorror We’re not going to be able to bypass Active Record entirely and drop straight to raw SQL since we have alot of logic baked in to our serializers that we use to form the json payload. So for, I’ve skipped AR in favor of SQL on the topics code path whenever possible.
Did we meet our goal of 20% speed improvement on topics page and no regression in performance on categories?
Not quite… I’m going to use the results I’m getting from profiling with ruby-prof since the results I’m getting from our profiling script varies quite between each run and between each day I run the benchmarks. @sam I’m not sure if you noticed this previously?
100 requests took: 13.31 seconds
PERF commits applied100 requests took: 12.61 seconds ~5.2% improvement compared to 1
PERF commits applied100 requests took 13.54 seconds ~7.4% slower compared to 2
100 requests took 12.89 seconds ~2.2% slower compared to 2
Not sure about this yet as I’ve only been profiling the topics page.
