Backend 502/504 Timeout when saving major edits to long posts (~100k words) due to Diff bottleneck

Ondersteuning
1

When editing and saving extremely long posts (around 100,000 characters), we encountered a backend timeout issue. The server becomes unresponsive during the save operation, resulting in 502/504 errors. The frontend console shows the following error stack:

ajax-error.js:36:15
l ajax-error.js:36
u ajax-error.js:75
d ajax-error.js:84
Ember 41
update rest.js:72
update rest.js:72
save rest.js:115
editPost composer.js:1147
Ember 6

I performed some comparative tests which suggest the bottleneck is in the diff calculation between old and new versions:

  • Directly editing a long post A into long post B and saving consistently triggers a 502/504 timeout.
  • Clearing the long post first, saving a short placeholder (e.g. 5 characters), then pasting the full new content B and saving completes quickly.

It seems the current Diff engine struggles with extreme cases involving very long text combined with a high percentage of changes. Would it be possible to add a performance fallback mechanism? For example, when the text is extremely long and the modification ratio is high, the system could treat it as a “Complete Rewrite” instead of performing a detailed line-by-line diff.

Does the team have any plans to optimize the Diff handling for large posts, or introduce some kind of graceful degradation / protection for such scenarios? Another idea is to allow the save to succeed first, and compute the diff asynchronously afterwards.

2

Meanwhile, the server-side Unicorn logs captured the exact point of the timeout, confirming that the worker was killed while processing the markdown diff:

Unicorn worker received USR2 signal indicating it is about to timeout, dumping backtrace for main thread
config/unicorn.conf.rb:204:in `backtrace'
config/unicorn.conf.rb:204:in `block (2 levels) in reload'
/var/www/discourse/lib/discourse_diff.rb:172:in `[]'
/var/www/discourse/lib/discourse_diff.rb:172:in `tokenize_markdown'
/var/www/discourse/lib/discourse_diff.rb:115:in `side_by_side_markdown'
/var/www/discourse/app/serializers/post_revision_serializer.rb:128:in `body_changes'
...