Mise à niveau de Discourse vers Rails 6

Bonjour l’équipe,

Rails 6.0.0 a été publié il y a 25 jours, je pense donc qu’il est temps de mettre à jour Discourse :slight_smile: J’ai dû effectuer quelques étapes pour que cela fonctionne.

  1. Corriger les spécifications cassées
  • Ajouter une méthode vide trigger_transactional_callbacks? dans lib/mini_sql_multisite_connection.rb
  • UrlHelper charge par défaut ActionView::Helpers::UrlHelper au lieu de lib/UrlHelper. Je l’ai résolu en ajoutant :: devant, mais que pensez-vous de changer le nom de cette classe ?
  • Dans Rails 5.2.3, MigrationContext acceptait un argument, tandis que dans 6.0.0, un schéma supplémentaire est requis.
  1. Corriger les méthodes obsolètes
  • update_attributes! a été remplacé par update!
  • Content_type contiendra désormais le jeu de caractères ; il faut utiliser le type de média à la place - rails/guides/source/upgrading_ruby_on_rails.md at main · rails/rails · GitHub
  • Corriger l’avertissement concernant la constante déjà initialisée (TRADITIONAL_ESCAPED_CHAR, RFC_5987_ESCAPED_CHAR). Avant de les supprimer, j’ai confirmé que les valeurs dans ActionPack étaient identiques.
  1. Utiliser le chargeur automatique classique comme première étape avant Zeitwerk.
  2. Corriger les migrations sur Rails 6.0.0 - la version la plus récente de Rails ne permet plus l’erreur présente dans les anciennes migrations (vous ne pouvez pas définir une colonne déjà définie ‘integer’).

J’ai effectué des tests de fumée pour vérifier que Discourse fonctionne comme prévu. De plus, j’ai exécuté des tests de performance pour m’assurer qu’il n’y a pas de régression (j’ai utilisé 500 itérations par défaut).

Test Rails 5.2.3 Rails 6.0.0 Pourcentage
categories-50 27 24 88,89%
categories-75 31 26 83,87%
categories-90 36 37 102,78%
categories-99 52 50 96,15%
home-50 27 26 96,30%
home-75 30 28 93,33%
home-90 39 38 97,44%
home-99 53 55 103,77%
topic-50 35 27 77,14%
topic-75 36 29 80,56%
topic-90 37 39 105,41%
topic-99 56 50 89,29%
categories_admin-50 47 47 100,00%
categories_admin-75 54 59 109,26%
categories_admin-90 64 66 103,13%
categories_admin-99 132 116 87,88%
home_admin-50 47 46 97,87%
home_admin-75 51 56 109,80%
home_admin-90 63 64 101,59%
home_admin-99 110 97 88,18%
topic_admin-50 50 49 98,00%
topic_admin-75 58 59 101,72%
topic_admin-90 65 67 103,08%
topic_admin-99 113 86 76,11%
load_rails 2593 2618 100,96%
rss_kb 318800 287332 90,13%
pss_kb 306913 275378 89,73%
Moyenne 89,31%

Je vais créer une pull request avec tous les changements mentionnés ci-dessus. Veuillez me faire savoir si vous souhaitez que j’ajuste quelque chose ou que j’effectue des tests supplémentaires pour garantir que tout fonctionne comme prévu.

PR - DEV: Upgrading Discourse to Rails 6 by KrisKotlarek · Pull Request #8083 · discourse/discourse · GitHub

Cordialement,
Kris

23 « J'aime »

This is awesome :confetti_ball:

Can you put this in a markdown table with percent change? Cursory look is showing not much has changes which is awesome.

Plugin wise, we have a rake task that installs all official plugins, can you run that and ensure the plugin specs pass on Rails 6? (rake plugin:spec should do the trick)

13 « J'aime »

I updated the original post to display table. Thank you for pointing me plugins specs. I see that 2 specs failed on Travis, I will take a look and fix them.

8 « J'aime »

There are 2 numbers here that I am finding very interesting:

RSS on 6.0 is almost 10% better.

Topic (median time) - which is our most common route is 22% faster.

This is some seriously improved perf, are you able to consistently measure 22% faster on topic-50? Can you confirm the actual page is rendering right?

13 « J'aime »

I run benchmark 3 times and this time, results are less spectacular. My flow is to type ruby script/bench.rb in correct branch master or rails6, hit enter and don’t touch keyboard to not affect results

topic-50 RSS
5.2.3 50 322852
5.2.3 50 309684
5.2.3 50 346376
Average 50 326304
6.0.0 49 328844
6.0.0 49 321824
6.0.0 49 283584
Average 49 311417

I also connected my development server to performance database to ensure that topic page looks correct. Screenshot below looks fine to me

8 « J'aime »

I would like to get your opinion about one fix.
I downloaded all plugins, however one new spec is failing comparing to master (./plugins/discourse-data-explorer/spec/controllers/queries_controller_spec.rb:32)

  1) DataExplorer::QueryController when disabled denies every request
     Failure/Error: render 'default/empty'

     ActionView::Template::Error:
       wrong number of arguments (given 2, expected 1)

That is fixed in master rspec-rails https://github.com/rspec/rspec-rails/blob/4-0-dev/lib/rspec/rails/view_rendering.rb
by changing
def self.call(_template) to def self.call(_template, _source = nil)

I can monkey patch rspec-rails with a new file in lib/freedom_patched/rspec-rails.rb but wanted to ensure that this is the best approach.

I think that this is the last change which is blocking Rails 6 from being merged.

In addition, I noticed that this spec is broken, however, it is broken on master as well, I can try to fix it (./plugins/discourse-calendar/spec/jobs/update_holiday_usernames_spec.rb:14)

 Failure/Error: expect(DiscourseCalendar.users_on_holiday).to eq([post.user.username])
       expected: ["bruce1"]
            got: []

Finally, there are deprecated methods in plugins which I can easily fix tomorrow.

What is you opinon about rspec-rails?

4 « J'aime »

Oh my, I guess we monkey patch until rspec-rails 4 is released, can not think of any cleaner fix here.

Or… maybe … use the beta gem for now if everything is working?

5 « J'aime »

Got you, let my try install beta tonight and see how it goes. It might be easy and smooth update.

5 « J'aime »

I made few additional fixes.

First of all, I found why one spec was failing for me in both master and rails6 branch - FIX: Freezed time used in update_holiday_usernames_spec.rb should be UTC by KrisKotlarek · Pull Request #3 · discourse/discourse-calendar · GitHub

I also created pull requests for deprecated methods in various plugins:

I rebased latest master into rails6 branch

Finally, I updated rspec-rails to version 4.0.0.beta2 and it works fine on my local machine. Travis got some problems, however I see same problems in other pull requests so I don’t think this is correlated to upgrade of rspec-rails.

12 « J'aime »

This is now merged :confetti_ball: :confetti_ball: :confetti_ball:

Going to keep a close eye on it today, thanks heaps for this work.

And a big thanks to the Rails team for making this such a pleasant upgrade!!

Will report back on this topic with a few pretty graphs.

18 « J'aime »

Upgrade is looking quite uneventful, which is great. Performance is even and remains incredibly similar.

Memory and CPU look remarkably similar.

My only concern (and something I would like to get to the bottom of) it appears we get “runaway” threads for a few seconds regularly on the web workers.

So somehow, some requests are causing a huge number of threads to be spawned and then go away.

Will continue to investigate this, we need to get backtraces when the number is high, so we can find the culprit.

Given everything else is looking really good I will not be reverting the upgrade.

17 « J'aime »

This should be fixed per:

This is a result of new code in Rails 6 that protects access to thread bound variable determining if you can use prepared statements or not.

At Discourse we do not use prepared statements at all, so this patch is not something we need.

See more at:

18 « J'aime »

And … confirmed … my fix gets rid of the large amount of thread spikes

Also worth noting… this is how I debugged it:

  1. I wrote this little class
# frozen_string_literal: true

class Thread
  attr_accessor :origin
end

class ThreadDetective
  def self.test_thread
    Thread.new { sleep 1 }
  end
  def self.start(max_threads)
    @thread ||= Thread.new do
      self.new.monitor(max_threads)
    end

    @trace = TracePoint.new(:thread_begin) do |tp|
      Thread.current.origin = Thread.current.inspect
    end
    @trace.enable
  end

  def self.stop
    @thread&.kill
    @thread = nil
    @trace&.disable
    @trace.stop
  end

  def monitor(max_threads)
    STDERR.puts "Monitoring threads in #{Process.pid}"

    while true
      threads = Thread.list

      if threads.length > max_threads
        str = +("-" * 60)
        str << "#{threads.length} found in Process #{Process.pid}!\n"

        threads.each do |thread|
          str << "\n"
          if thread.origin
            str << thread.origin
          else
            str << thread.inspect
          end
          str << "\n"
        end
        str << ("-" * 60)

        STDERR.puts str
      end
      sleep 1
    end
  end

end
  1. I then wired in unicorn after_fork a require of this class and ran ThreadDetective.start(14)

  2. The class diligently watched every time a thread was created using a TracePoint and placed a tiny frame on the thread called origin to help me track where it came from. Once a large number of threads were observed it dumped stuff to STDERR. This can be tracked in /var/www/discourse/logs/unicorn.stderr.log

Once I knew that 100 threads were all coming from a single spot, it was very easy to isolate the root cause.

27 « J'aime »

I noticed that I can no longer use dev.local as my hostname in development mode in Rails 6, so I’ve added an ENV variable to configure that whitelist:

11 « J'aime »

We should not need to keep this monkey patch long term cause we just landed a fix in Rails.

5 « J'aime »

Hi there,

thanks for your efforts on bringing Rails 6 to Discourse! May I humbly ask when this is expected to land in Discourse? Or is it already in 2.4.0.beta? I am just asking about the possibility whether this might break any plugin people have installed on their instances.

With kind regards,
Andreas.

This is live since September for everyone using the default release channel. It was first featured in the 2.4.0.beta5.

8 « J'aime »

All right, thank you so much. Best wishes for 2020 and thanks for everything you are doing here.

6 « J'aime »