Hello, have you been able to find it? It would be great to parameterize the timeout into app.yml ENV or discourse site settings for us with low memory…
Maybe a dumb question?
When you move a lot of posts like this is this process managed by Sidekiq?
Sorry, did not search the code…
Took a quick peek at the Ruby code base and yes when the move posts function is called, the jobs are queued with the
Not being a discourse developer, it would appear to a casual code-browser, observer that issues related to delays, errors or timeouts related to moving posts would be directly related to
sidekiq performance and configuration.
I tried studying how discourse uses sidekiq at the system level a number of days ago, but I was not successful in finding the “cliff notes” version for dummies.
So, I went to the sidekiq web site to try to get a better feel for what is going on under the cover and noticed there were three different offerings, got confused and moved on because I could not understand, within my short attention span and need for immediate gradification, what version of sidekiq discourse uses, what are the exact features and switches which can be configured…
Being a novice in this area, I’m interested to know exactly the
sidekiq architecture, features, switches, environmentals, available in
discourse … but so far I “still haven’t found, what I’m looking for” - sung to the tune of our fav U2 song.
All answers spring from curiosity…
On a tip from a leader in another thread, we turned off all plug-ins except “who’s online” and now, no issues with moves lately.
So cautious optimism here. Will update if things change.
Thanks to everyone who has lent assistance on this issue!
Which plugins specifically did you disable?
Ideally, I would have done each on their own to check which one(s) were causing problems, but I didn’t expect it to work so switched them all off in one go.
Very likely babble, I am guessing it has hooks that fire on post move.
Good to know. Thanks. And props to @featheredtoast for the fix.
My community recently started experiencing the 502 error issue when moving posts, especially between large threads. I had no custom plug-ins installed. Following the advice from another Discourse thread, I increased the
unicorn_workers to 10 and
db_shared_buffers to 4096MB, but that did not ameliorate the situation. Below is our forum’s
./discourse-doctor log. Hoping to get some pointers. Thank you!
==================== DOCKER INFO ==================== DOCKER VERSION: Docker version 17.10.0-ce, build f4ffd25 DOCKER PROCESSES (docker ps -a) CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES ddfb2222fd64 local_discourse/app "/sbin/boot" 10 days ago Up 10 days 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp app ddfb2222fd64 local_discourse/app "/sbin/boot" 10 days ago Up 10 days 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp app Discourse container app is running ==================== PLUGINS ==================== - git clone https://github.com/discourse/docker_manager.git - git clone https://github.com/discourse/discourse-solved.git No non-official plugins detected. See https://github.com/discourse/discourse/blob/master/lib/plugin/metadata.rb for the official list. ======================================== Discourse 2.6.0.beta2 Discourse version at localhost: Discourse 2.6.0.beta2 ==================== MEMORY INFORMATION ==================== RAM (MB): 16434 total used free shared buff/cache available Mem: 16048 5605 919 4255 9523 5850 Swap: 2047 437 1610 ==================== DISK SPACE CHECK ==================== ---------- OS Disk Space ---------- Filesystem Size Used Avail Use% Mounted on /dev/disk/by-label/DOROOT 315G 132G 168G 45% / /dev/disk/by-label/DOROOT 315G 132G 168G 45% /var/lib/docker/aufs /dev/disk/by-label/DOROOT 315G 132G 168G 45% / /dev/disk/by-label/DOROOT 315G 132G 168G 45% /var/lib/docker/plugins /dev/disk/by-label/DOROOT 315G 132G 168G 45% / ---------- Container Disk Space ---------- unknown shorthand flag: 'w' in -w See 'docker exec --help'. ==================== DISK INFORMATION ==================== Disk /dev/vda: 320 GiB, 343597383680 bytes, 671088640 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: gpt Disk identifier: 29B528BA-16C4-402E-BEE9-53555C8B6F10 Device Start End Sectors Size Type /dev/vda1 2048 671086591 671084544 320G Linux filesystem ==================== END DISK INFORMATION ====================
Hi, I encounter the same issue. Can’t split megatopics because of this.
Also tried in safe-mode, didn’t change anything.
No issue in my development Discourse installation (same version 2.6.0.beta2) though.
And nothing in the logs.
I’m getting this 502 errors for an year ;(
I don’t think we’ve asked you: what plugins are you running?
I disable all plugins to check is it plugin related bug. It looks like it repeated stable with long threads.
10 posts were merged into an existing topic: 502 errors when splitting topic, moving or renaming categories
This most annoying bug is still with us.
Could you give me instructions to nail the problem?
Unfortunately we are still running into this issue from time to time. I managed to collect some data about a recent problem where moving 27 posts into an existing topic (2013 posts) failed:
Beginning move at 2021-05-25 20:54:57 +0000 create_temp_table 0.000134 0.000027 0.000161 ( 0.002810) delete_invalid_post_timings 0.000317 0.000065 0.000382 ( 0.306869) move_incoming_emails 0.000056 0.000006 0.000062 ( 0.000478) move_notifications 0.000074 0.000008 0.000082 ( 0.005584) update_reply_counts 0.000088 0.000010 0.000098 ( 0.001058) update_quotes 0.000079 0.000008 0.000087 ( 0.002426) move_first_post_replies 0.000071 0.000008 0.000079 ( 0.000695) delete_post_replies 0.000062 0.000006 0.000068 ( 0.000834) copy_first_post_timings 0.000064 0.000007 0.000071 ( 0.002660) move_post_timings 0.000082 0.000009 0.000091 ( 0.008195) copy_topic_users 0.000398 0.000043 0.000441 ( 0.043640) move_each_post 4.345220 0.132652 4.477872 ( 3.447047) create_moderator_post_in_original_topic 0.163962 0.000253 0.164215 ( 0.207101) update_statistics 0.217266 0.051660 0.268926 ( 45.258070) update_user_actions 0.000934 0.000101 0.001035 ( 0.178894) update_last_post_stats 0.001589 0.000171 0.001760 ( 0.003156) update_upload_security_status 0.000030 0.000003 0.000033 ( 0.000034) update_bookmarks 0.000492 0.000054 0.000546 ( 0.003832) Finished move at 2021-05-25 20:55:47 +0000
Most methods finish real quick, but
update_statistics took over 45 seconds in this case. All that time is spent on the following method call:
Maybe we defer the update_statistics work to sidekiq?
Yes, probably. But I’m going to take a hard look at how complicated it would be to move the whole process into a Sidekiq job and let the UI wait until it’s finished. We are kinda playing whac-a-mole here by always moving bits and pieces into jobs.
Thank you for your hard work!
I also get this 502 when trying to merge topics. Not every time, but a lot of the time.
I’m doing this when someone has started a topic on a subject that already exists, so typically I’m trying to move only a very small number of posts (no more than five).
I haven’t read this entire topic, but I’m understanding that this is a known issue that you’re working on and there isn’t anything I can do myself?