Moving posts returns 502 bad gateway

Hello, have you been able to find it? It would be great to parameterize the timeout into app.yml ENV or discourse site settings for us with low memory…


Maybe a dumb question?


When you move a lot of posts like this is this process managed by Sidekiq?

Sorry, did not search the code…


Update


Took a quick peek at the Ruby code base and yes when the move posts function is called, the jobs are queued with the enqueue_jobs().

https://github.com/discourse/discourse/blob/master/app/models/post_mover.rb

Not being a discourse developer, it would appear to a casual code-browser, observer that issues related to delays, errors or timeouts related to moving posts would be directly related to sidekiq performance and configuration.

I tried studying how discourse uses sidekiq at the system level a number of days ago, but I was not successful in finding the “cliff notes” version for dummies.

So, I went to the sidekiq web site to try to get a better feel for what is going on under the cover and noticed there were three different offerings, got confused and moved on :slight_smile: because I could not understand, within my short attention span and need for immediate gradification, what version of sidekiq discourse uses, what are the exact features and switches which can be configured…

Being a novice in this area, I’m interested to know exactly the sidekiq architecture, features, switches, environmentals, available in discourse … but so far I “still haven’t found, what I’m looking for” - sung to the tune of our fav U2 song.

All answers spring from curiosity…

4 Likes

Update:

On a tip from a leader in another thread, we turned off all plug-ins except “who’s online” and now, no issues with moves lately.

So cautious optimism here. Will update if things change.

Thanks to everyone who has lent assistance on this issue!

3 Likes

Which plugins specifically did you disable?

3 Likes

Ideally, I would have done each on their own to check which one(s) were causing problems, but I didn’t expect it to work so switched them all off in one go.

1 Like

Very likely babble, I am guessing it has hooks that fire on post move.

4 Likes

Good to know. Thanks. And props to @featheredtoast for the fix.

2 Likes

My community recently started experiencing the 502 error issue when moving posts, especially between large threads. I had no custom plug-ins installed. Following the advice from another Discourse thread, I increased the unicorn_workers to 10 and db_shared_buffers to 4096MB, but that did not ameliorate the situation. Below is our forum’s ./discourse-doctor log. Hoping to get some pointers. Thank you!

==================== DOCKER INFO ====================
DOCKER VERSION: Docker version 17.10.0-ce, build f4ffd25

DOCKER PROCESSES (docker ps -a)

CONTAINER ID        IMAGE                 COMMAND             CREATED             STATUS              PORTS                                      NAMES
ddfb2222fd64        local_discourse/app   "/sbin/boot"        10 days ago         Up 10 days          0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp   app

ddfb2222fd64        local_discourse/app   "/sbin/boot"        10 days ago         Up 10 days          0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp   app

Discourse container app is running


==================== PLUGINS ====================
          - git clone https://github.com/discourse/docker_manager.git
          - git clone https://github.com/discourse/discourse-solved.git

No non-official plugins detected.

See https://github.com/discourse/discourse/blob/master/lib/plugin/metadata.rb for the official list.

========================================
Discourse 2.6.0.beta2
Discourse version at localhost: Discourse 2.6.0.beta2 


==================== MEMORY INFORMATION ====================
RAM (MB): 16434

              total        used        free      shared  buff/cache   available
Mem:          16048        5605         919        4255        9523        5850
Swap:          2047         437        1610

==================== DISK SPACE CHECK ====================
---------- OS Disk Space ----------
Filesystem                 Size  Used Avail Use% Mounted on
/dev/disk/by-label/DOROOT  315G  132G  168G  45% /
/dev/disk/by-label/DOROOT  315G  132G  168G  45% /var/lib/docker/aufs
/dev/disk/by-label/DOROOT  315G  132G  168G  45% /
/dev/disk/by-label/DOROOT  315G  132G  168G  45% /var/lib/docker/plugins
/dev/disk/by-label/DOROOT  315G  132G  168G  45% /

---------- Container Disk Space ----------
unknown shorthand flag: 'w' in -w
See 'docker exec --help'.


==================== DISK INFORMATION ====================
Disk /dev/vda: 320 GiB, 343597383680 bytes, 671088640 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 29B528BA-16C4-402E-BEE9-53555C8B6F10

Device     Start       End   Sectors  Size Type
/dev/vda1   2048 671086591 671084544  320G Linux filesystem

==================== END DISK INFORMATION ====================
3 Likes

Hi, I encounter the same issue. Can’t split megatopics because of this.
Also tried in safe-mode, didn’t change anything.

No issue in my development Discourse installation (same version 2.6.0.beta2) though.
And nothing in the logs.

3 Likes

I’m getting this 502 errors for an year ;(

1 Like

I don’t think we’ve asked you: what plugins are you running?

2 Likes

I disable all plugins to check is it plugin related bug. It looks like it repeated stable with long threads.

2 Likes

10 posts were merged into an existing topic: 502 errors when splitting topic, moving or renaming categories

This most annoying bug is still with us.
Could you give me instructions to nail the problem?

1 Like

Unfortunately we are still running into this issue from time to time. I managed to collect some data about a recent problem where moving 27 posts into an existing topic (2013 posts) failed:

Beginning move at 2021-05-25 20:54:57 +0000
create_temp_table
  0.000134   0.000027   0.000161 (  0.002810)

delete_invalid_post_timings
  0.000317   0.000065   0.000382 (  0.306869)

move_incoming_emails
  0.000056   0.000006   0.000062 (  0.000478)

move_notifications
  0.000074   0.000008   0.000082 (  0.005584)

update_reply_counts
  0.000088   0.000010   0.000098 (  0.001058)

update_quotes
  0.000079   0.000008   0.000087 (  0.002426)

move_first_post_replies
  0.000071   0.000008   0.000079 (  0.000695)

delete_post_replies
  0.000062   0.000006   0.000068 (  0.000834)

copy_first_post_timings
  0.000064   0.000007   0.000071 (  0.002660)

move_post_timings
  0.000082   0.000009   0.000091 (  0.008195)

copy_topic_users
  0.000398   0.000043   0.000441 (  0.043640)

move_each_post
  4.345220   0.132652   4.477872 (  3.447047)

create_moderator_post_in_original_topic
  0.163962   0.000253   0.164215 (  0.207101)

update_statistics
  0.217266   0.051660   0.268926 ( 45.258070)

update_user_actions
  0.000934   0.000101   0.001035 (  0.178894)

update_last_post_stats
  0.001589   0.000171   0.001760 (  0.003156)

update_upload_security_status
  0.000030   0.000003   0.000033 (  0.000034)

update_bookmarks
  0.000492   0.000054   0.000546 (  0.003832)

Finished move at 2021-05-25 20:55:47 +0000

Most methods finish real quick, but update_statistics took over 45 seconds in this case. All that time is spent on the following method call:

9 Likes

Maybe we defer the update_statistics work to sidekiq?

1 Like

Yes, probably. But I’m going to take a hard look at how complicated it would be to move the whole process into a Sidekiq job and let the UI wait until it’s finished. We are kinda playing whac-a-mole here by always moving bits and pieces into jobs.

7 Likes

Thank you for your hard work!

2 Likes

I also get this 502 when trying to merge topics. Not every time, but a lot of the time.

I’m doing this when someone has started a topic on a subject that already exists, so typically I’m trying to move only a very small number of posts (no more than five).

I haven’t read this entire topic, but I’m understanding that this is a known issue that you’re working on and there isn’t anything I can do myself?

3 Likes

I’m also experiencing this issue on our forum. A number of us are unable to edit and/or create new posts. We’re getting the following error message:

1 Like