将 Discourse 升级到 Rails 6

大家好,

Rails 6.0.0 已于 25 天前发布,我认为现在是更新 Discourse 的时候了 :slight_smile: 为此我完成了一些必要的步骤。

  1. 修复损坏的测试规范
  • lib/mini_sql_multisite_connection.rb 中添加空方法 trigger_transactional_callbacks?
  • UrlHelper 默认加载的是 ActionView::Helpers::UrlHelper 而非 lib/UrlHelper。我通过在前面添加 :: 解决了这个问题,不过,你们觉得是否应该更改该类的名称?
  • 在 Rails 5.2.3 中,MigrationContext 接受一个参数,而在 6.0.0 中则需要额外的 schema 参数。
  1. 修复已弃用的方法
  1. 在引入 Zeitwerk 之前,首先使用经典自动加载器。
  2. 修复 Rails 6.0.0 上的迁移问题 - 最新版本的 Rails 不允许旧迁移中的错误(例如不能定义已存在的列 ‘integer’)。

我进行了冒烟测试,确认 Discourse 按预期工作。此外,我还运行了性能测试以确保没有回归(我使用了默认的 500 次迭代)。

测试 Rails 5.2.3 Rails 6.0.0 百分比
categories-50 27 24 88.89%
categories-75 31 26 83.87%
categories-90 36 37 102.78%
categories-99 52 50 96.15%
home-50 27 26 96.30%
home-75 30 28 93.33%
home-90 39 38 97.44%
home-99 53 55 103.77%
topic-50 35 27 77.14%
topic-75 36 29 80.56%
topic-90 37 39 105.41%
topic-99 56 50 89.29%
categories_admin-50 47 47 100.00%
categories_admin-75 54 59 109.26%
categories_admin-90 64 66 103.13%
categories_admin-99 132 116 87.88%
home_admin-50 47 46 97.87%
home_admin-75 51 56 109.80%
home_admin-90 63 64 101.59%
home_admin-99 110 97 88.18%
topic_admin-50 50 49 98.00%
topic_admin-75 58 59 101.72%
topic_admin-90 65 67 103.08%
topic_admin-99 113 86 76.11%
load_rails 2593 2618 100.96%
rss_kb 318800 287332 90.13%
pss_kb 306913 275378 89.73%
平均 89.31%

我将创建一个包含上述所有更改的拉取请求。如果您希望我对任何内容进行调整,或需要我进行额外测试以确保一切正常运行,请随时告知。

PR - DEV: Upgrading Discourse to Rails 6 by KrisKotlarek · Pull Request #8083 · discourse/discourse · GitHub

此致,
Kris

This is awesome :confetti_ball:

Can you put this in a markdown table with percent change? Cursory look is showing not much has changes which is awesome.

Plugin wise, we have a rake task that installs all official plugins, can you run that and ensure the plugin specs pass on Rails 6? (rake plugin:spec should do the trick)

I updated the original post to display table. Thank you for pointing me plugins specs. I see that 2 specs failed on Travis, I will take a look and fix them.

There are 2 numbers here that I am finding very interesting:

RSS on 6.0 is almost 10% better.

Topic (median time) - which is our most common route is 22% faster.

This is some seriously improved perf, are you able to consistently measure 22% faster on topic-50? Can you confirm the actual page is rendering right?

I run benchmark 3 times and this time, results are less spectacular. My flow is to type ruby script/bench.rb in correct branch master or rails6, hit enter and don’t touch keyboard to not affect results

topic-50 RSS
5.2.3 50 322852
5.2.3 50 309684
5.2.3 50 346376
Average 50 326304
6.0.0 49 328844
6.0.0 49 321824
6.0.0 49 283584
Average 49 311417

I also connected my development server to performance database to ensure that topic page looks correct. Screenshot below looks fine to me

I would like to get your opinion about one fix.
I downloaded all plugins, however one new spec is failing comparing to master (./plugins/discourse-data-explorer/spec/controllers/queries_controller_spec.rb:32)

  1) DataExplorer::QueryController when disabled denies every request
     Failure/Error: render 'default/empty'

     ActionView::Template::Error:
       wrong number of arguments (given 2, expected 1)

That is fixed in master rspec-rails https://github.com/rspec/rspec-rails/blob/4-0-dev/lib/rspec/rails/view_rendering.rb
by changing
def self.call(_template) to def self.call(_template, _source = nil)

I can monkey patch rspec-rails with a new file in lib/freedom_patched/rspec-rails.rb but wanted to ensure that this is the best approach.

I think that this is the last change which is blocking Rails 6 from being merged.

In addition, I noticed that this spec is broken, however, it is broken on master as well, I can try to fix it (./plugins/discourse-calendar/spec/jobs/update_holiday_usernames_spec.rb:14)

 Failure/Error: expect(DiscourseCalendar.users_on_holiday).to eq([post.user.username])
       expected: ["bruce1"]
            got: []

Finally, there are deprecated methods in plugins which I can easily fix tomorrow.

What is you opinon about rspec-rails?

Oh my, I guess we monkey patch until rspec-rails 4 is released, can not think of any cleaner fix here.

Or… maybe … use the beta gem for now if everything is working?

Got you, let my try install beta tonight and see how it goes. It might be easy and smooth update.

I made few additional fixes.

First of all, I found why one spec was failing for me in both master and rails6 branch - FIX: Freezed time used in update_holiday_usernames_spec.rb should be UTC by KrisKotlarek · Pull Request #3 · discourse/discourse-calendar · GitHub

I also created pull requests for deprecated methods in various plugins:

I rebased latest master into rails6 branch

Finally, I updated rspec-rails to version 4.0.0.beta2 and it works fine on my local machine. Travis got some problems, however I see same problems in other pull requests so I don’t think this is correlated to upgrade of rspec-rails.

This is now merged :confetti_ball: :confetti_ball: :confetti_ball:

Going to keep a close eye on it today, thanks heaps for this work.

And a big thanks to the Rails team for making this such a pleasant upgrade!!

Will report back on this topic with a few pretty graphs.

Upgrade is looking quite uneventful, which is great. Performance is even and remains incredibly similar.

Memory and CPU look remarkably similar.

My only concern (and something I would like to get to the bottom of) it appears we get “runaway” threads for a few seconds regularly on the web workers.

So somehow, some requests are causing a huge number of threads to be spawned and then go away.

Will continue to investigate this, we need to get backtraces when the number is high, so we can find the culprit.

Given everything else is looking really good I will not be reverting the upgrade.

This should be fixed per:

This is a result of new code in Rails 6 that protects access to thread bound variable determining if you can use prepared statements or not.

At Discourse we do not use prepared statements at all, so this patch is not something we need.

See more at:

And … confirmed … my fix gets rid of the large amount of thread spikes

Also worth noting… this is how I debugged it:

  1. I wrote this little class
# frozen_string_literal: true

class Thread
  attr_accessor :origin
end

class ThreadDetective
  def self.test_thread
    Thread.new { sleep 1 }
  end
  def self.start(max_threads)
    @thread ||= Thread.new do
      self.new.monitor(max_threads)
    end

    @trace = TracePoint.new(:thread_begin) do |tp|
      Thread.current.origin = Thread.current.inspect
    end
    @trace.enable
  end

  def self.stop
    @thread&.kill
    @thread = nil
    @trace&.disable
    @trace.stop
  end

  def monitor(max_threads)
    STDERR.puts "Monitoring threads in #{Process.pid}"

    while true
      threads = Thread.list

      if threads.length > max_threads
        str = +("-" * 60)
        str << "#{threads.length} found in Process #{Process.pid}!\n"

        threads.each do |thread|
          str << "\n"
          if thread.origin
            str << thread.origin
          else
            str << thread.inspect
          end
          str << "\n"
        end
        str << ("-" * 60)

        STDERR.puts str
      end
      sleep 1
    end
  end

end
  1. I then wired in unicorn after_fork a require of this class and ran ThreadDetective.start(14)

  2. The class diligently watched every time a thread was created using a TracePoint and placed a tiny frame on the thread called origin to help me track where it came from. Once a large number of threads were observed it dumped stuff to STDERR. This can be tracked in /var/www/discourse/logs/unicorn.stderr.log

Once I knew that 100 threads were all coming from a single spot, it was very easy to isolate the root cause.

I noticed that I can no longer use dev.local as my hostname in development mode in Rails 6, so I’ve added an ENV variable to configure that whitelist:

We should not need to keep this monkey patch long term cause we just landed a fix in Rails.

你好,

感谢你为将 Rails 6 引入 Discourse 所做的努力!我想冒昧地问一下,这预计何时会在 Discourse 中发布?或者它是否已经包含在 2.4.0.beta 中?我只是想了解一下,这是否可能会破坏用户实例上已安装的任何插件。

此致,
Andreas。

对于所有使用默认发布渠道的用户,此功能自 9 月起已正式上线。它首次在 2.4.0.beta5 版本中亮相。

好的,非常感谢。祝2020年一切顺利,也感谢你们在此所做的一切。