Rebuild hanging

Installation
1

Been stuck on this for over an hour:

I, [2024-04-17T09:57:04.110084 #1] INFO -- : > cd /var/www/discourse && su discourse -c 'bundle exec rake assets:precompile:build'
97:M 17 Apr 2024 10:01:01.012 * 100 changes in 300 seconds. Saving...
97:M 17 Apr 2024 10:01:01.012 * Background saving started by pid 3733
3733:C 17 Apr 2024 10:01:01.026 * DB saved on disk
3733:C 17 Apr 2024 10:01:01.027 * Fork CoW for RDB: current 1 MB, peak 1 MB, average 0 MB
97:M 17 Apr 2024 10:01:01.112 * Background saving terminated with success
97:M 17 Apr 2024 10:56:01.848 * Replication backlog freed after 3600 seconds without connected replicas.

The server has 64GB of memory so I don’t think it’s a memory issue, although in the container I had specified db_shared_buffers: "4096MB" as per the set-up recommendations.

Any ideas what’s going on? How to troubleshoot? Fix?

2

That’s strange. I think I would control-c and try again.

2 Likes
3

Thanks Jay, I waited two hours and did that (it’s an archived forum so wasn’t too bothered about the downtime).

Only thing I did different was I had added - git clone https://github.com/discourse/discourse-calendar but noticed it was without the .git on the end - not sure if that made any difference though.

2 Likes
4

Ever since this issue we’re now experiencing problems with the server. When this first happened we noticed other Ruby sites on the server could not be reached. That happened twice several days apart and where a reboot sorted it (those sites use discourse auth). It just happened again but this time two of the discourse forums were getting 504 Gateway Time-out as well.

I noticed others have had a similar issue of rebuild hanging and am wondering whether something changed in Discourse recently that would be linked to this? Does Discourse alter anything outside of the containers, such as system Ruby perhaps? It’s very odd :confused:

5

Yesterday there was a fix that allows low ram servers to rebuild much more weirdly/quickly, but I think it might not work since it tests for 2gb and your issue is likely that you have more than 2gb but it’s all taken up by other stuff on the server.

My guess is just that you need more ram.

1 Like
6

The server has 64GB or RAM Jay, and each DC instance is configured with db_shared_buffers: "4096MB".

Also these further issues haven’t been on rebuild, but seem to be the remnants of that original issue.

I will clean up docker ./launcher cleanup to see if that helps, but if you or anyone else has any other ideas in the meantime that will be greatly appreciated.

Edit: just noticed something weird after running docker container ls -a, will post a new thread about it.

7

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.