# No errors, instance not coming back up after rebuild

**URL:** https://meta.discourse.org/t/no-errors-instance-not-coming-back-up-after-rebuild/257515
**Category:** Self-hosting
**Created:** [March 8, 2023, 5:08pm UTC](https://meta.discourse.org/t/no-errors-instance-not-coming-back-up-after-rebuild/257515 "2023-03-08T17:08:07Z")
**Posts on this page:** 19
**Page:** 1

<div class="post-metadata">

### Author: ![MarcP](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/marcp/32/160184_2.png) [@MarcP](https://meta.discourse.org/u/MarcP)
#### Post date: [March 8, 2023, 5:08pm UTC](https://meta.discourse.org/t/no-errors-instance-not-coming-back-up-after-rebuild/257515/1 "2023-03-08T17:08:08Z")

</div>

For the first time in years after a rebuild my instance is not coming back up. My last successful rebuild was yesterday.

Is there maybe a fresh commit that may be the reason for this?

---

<div class="post-metadata">

### Author: ![pfaffman](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/pfaffman/32/120154_2.png) [@pfaffman](https://meta.discourse.org/u/pfaffman)
#### Post date: [March 8, 2023, 5:12pm UTC](https://meta.discourse.org/t/no-errors-instance-not-coming-back-up-after-rebuild/257515/2 "2023-03-08T17:12:38Z")

</div>

What are the symptoms?

What do you see if you

```
tail -f /var/discourse/shared/standalone/log/rails/production.log

```

---

<div class="post-metadata">

### Author: ![MarcP](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/marcp/32/160184_2.png) [@MarcP](https://meta.discourse.org/u/MarcP)
#### Post date: [March 8, 2023, 5:16pm UTC](https://meta.discourse.org/t/no-errors-instance-not-coming-back-up-after-rebuild/257515/3 "2023-03-08T17:16:22Z")

</div>

```plaintext
Done in 89.60s.
Downloading MaxMindDB...
Compressing Javascript and Generating Source Maps

I, [2023-03-08T17:11:55.996529 #1] INFO -- : File > /usr/local/bin/discourse chmod: +x chown: 
I, [2023-03-08T17:11:55.999205 #1] INFO -- : File > /usr/local/bin/rails chmod: +x chown: 
I, [2023-03-08T17:11:56.001899 #1] INFO -- : File > /usr/local/bin/rake chmod: +x chown: 
I, [2023-03-08T17:11:56.004616 #1] INFO -- : File > /usr/local/bin/rbtrace chmod: +x chown: 
I, [2023-03-08T17:11:56.007306 #1] INFO -- : File > /usr/local/bin/stackprof chmod: +x chown: 
I, [2023-03-08T17:11:56.010710 #1] INFO -- : File > /etc/update-motd.d/10-web chmod: +x chown: 
I, [2023-03-08T17:11:56.012746 #1] INFO -- : File > /etc/logrotate.d/rails chmod: chown: 
I, [2023-03-08T17:11:56.014343 #1] INFO -- : File > /etc/logrotate.d/nginx chmod: chown: 
I, [2023-03-08T17:11:56.017963 #1] INFO -- : File > /etc/runit/1.d/00-ensure-links chmod: +x chown: 
I, [2023-03-08T17:11:56.020609 #1] INFO -- : File > /etc/runit/1.d/01-cleanup-web-pids chmod: +x chown: 
I, [2023-03-08T17:11:56.023663 #1] INFO -- : File > /root/.bash_profile chmod: 644 chown: 
I, [2023-03-08T17:11:56.026021 #1] INFO -- : File > /usr/local/etc/ImageMagick-7/policy.xml chmod: chown: 
I, [2023-03-08T17:11:56.026795 #1] INFO -- : Replacing (?-mix:server.+{) with limit_req_zone $binary_remote_addr zone=flood:10m rate=$reqs_per_secondr/s;
limit_req_zone $binary_remote_addr zone=bot:10m rate=$reqs_per_minuter/m;
limit_req_status 429;
limit_conn_zone $binary_remote_addr zone=connperip:10m;
limit_conn_status 429;
server {
 in /etc/nginx/conf.d/discourse.conf
I, [2023-03-08T17:11:56.026984 #1] INFO -- : Replacing (?-mix:location @discourse {) with location @discourse {
  limit_conn connperip $conn_per_ip;
  limit_req zone=flood burst=$burst_per_second nodelay;
  limit_req zone=bot burst=$burst_per_minute nodelay; in /etc/nginx/conf.d/discourse.conf
I, [2023-03-08T17:11:56.029658 #1] INFO -- : File > /etc/runit/1.d/remove-old-socket chmod: +x chown: 
I, [2023-03-08T17:11:56.032272 #1] INFO -- : File > /etc/runit/3.d/remove-old-socket chmod: +x chown: 
I, [2023-03-08T17:11:56.032398 #1] INFO -- : Replacing (?-mix:listen 80;) with listen unix:/shared/nginx.http.sock;
set_real_ip_from unix:;
 in /etc/nginx/conf.d/discourse.conf
I, [2023-03-08T17:11:56.032577 #1] INFO -- : Replacing (?-mix:listen 443 ssl http2;) with listen unix:/shared/nginx.https.sock ssl http2;
set_real_ip_from unix:; in /etc/nginx/conf.d/discourse.conf
I, [2023-03-08T17:11:56.035350 #1] INFO -- : File > /tmp/add-cloudflare-ips chmod: +x chown: 
I, [2023-03-08T17:11:56.035435 #1] INFO -- : > /tmp/add-cloudflare-ips
I, [2023-03-08T17:11:56.359453 #1] INFO -- : CloudFlare IPs:
set_real_ip_from 173.245.48.0/20; set_real_ip_from 103.21.244.0/22; set_real_ip_from 103.22.200.0/22; set_real_ip_from 103.31.4.0/22; set_real_ip_from 141.101.64.0/18; set_real_ip_from 108.162.192.0/18; set_real_ip_from 190.93.240.0/20; set_real_ip_from 188.114.96.0/20; set_real_ip_from 197.234.240.0/22; set_real_ip_from 198.41.128.0/17; set_real_ip_from 162.158.0.0/15; set_real_ip_from 104.16.0.0/13; set_real_ip_from 104.24.0.0/14; set_real_ip_from 172.64.0.0/13; set_real_ip_from 131.0.72.0/22; set_real_ip_from 2400:cb00::/32; set_real_ip_from 2606:4700::/32; set_real_ip_from 2803:f800::/32; set_real_ip_from 2405:b500::/32; set_real_ip_from 2405:8100::/32; set_real_ip_from 2a06:98c0::/29; set_real_ip_from 2c0f:f248::/32;

I, [2023-03-08T17:11:56.359655 #1] INFO -- : > rm /tmp/add-cloudflare-ips
I, [2023-03-08T17:11:56.361599 #1] INFO -- : 
I, [2023-03-08T17:11:56.361818 #1] INFO -- : > echo "Beginning of custom commands"
I, [2023-03-08T17:11:56.363535 #1] INFO -- : Beginning of custom commands

I, [2023-03-08T17:11:56.367829 #1] INFO -- : File > /etc/service/monerochan_merchant_rpc/run chmod: +x chown: 
I, [2023-03-08T17:11:56.368034 #1] INFO -- : > echo "End of custom commands"
I, [2023-03-08T17:11:56.369958 #1] INFO -- : End of custom commands

I, [2023-03-08T17:11:56.370117 #1] INFO -- : Terminating async processes
I, [2023-03-08T17:11:56.370225 #1] INFO -- : Sending INT to HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/13/bin/postmaster -D /etc/postgresql/13/main pid: 42
I, [2023-03-08T17:11:56.370261 #1] INFO -- : Sending TERM to exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf pid: 103
2023-03-08 17:11:56.370 UTC [42] LOG: received fast shutdown request
103:signal-handler (1678295516) Received SIGTERM scheduling shutdown...
2023-03-08 17:11:56.372 UTC [42] LOG: aborting any active transactions
2023-03-08 17:11:56.374 UTC [42] LOG: background worker "logical replication launcher" (PID 51) exited with exit code 1
2023-03-08 17:11:56.375 UTC [46] LOG: shutting down
2023-03-08 17:11:56.392 UTC [42] LOG: database system is shut down
103:M 08 Mar 2023 17:11:56.469 # User requested shutdown...
103:M 08 Mar 2023 17:11:56.469 * Saving the final RDB snapshot before exiting.
103:M 08 Mar 2023 17:11:56.570 * DB saved on disk
103:M 08 Mar 2023 17:11:56.570 # Redis is now ready to exit, bye bye...
sha256:422bd26e098f3af0623647ebce02770ac1608bfac07260aeb5469ab975696363
a0b91a9cc45e8666352e172143854705faa97b38208fcfe0650ea929989b8570
Removing old container
+ /usr/bin/docker rm app
app

```

```plaintext
➜ discourse git:(main) ✗ tail -f /var/discourse/shared/standalone/log/rails/production.log

Bye!
Deprecation notice: (siwe) full_screen_login is now forced. The full_screen_login parameter can be removed from the auth_provider. (removal in Discourse 2.9.0) 
At /var/www/discourse/lib/plugin/instance.rb:763:in `public_send`
Migrating to MakeChatMentionNotificationIdNullable (20230227172543)
Migrating to DropBadgeGrantedTitleColumn (20230228105851)
Migrating to AddExternalToSidebarUrls (20230303015952)
Theme setting type has changed but cannot be converted. 

 #<ThemeSettingsManager::Upload:0x00007f60ff714e38 @name=:background_image, @default="", @theme=#<Theme id: 31, name: "Search Banner", user_id: 2, created_at: "2021-08-03 16:38:09.042735000 +0000", updated_at: "2021-08-03 17:01:41.329058000 +0000", compiler_version: 0, user_selectable: false, hidden: false, color_scheme_id: nil, remote_theme_id: 29, component: true, enabled: true, auto_update: true>, @opts={:description=>"background image for the banner", :textarea=>false, :json_schema=>nil, :refresh=>false}, @types={:integer=>0, :float=>1, :string=>2, :bool=>3, :list=>4, :enum=>5, :upload=>6}>
Bye!
Deprecation notice: (siwe) full_screen_login is now forced. The full_screen_login parameter can be removed from the auth_provider. (removal in Discourse 2.9.0) 
At /var/www/discourse/lib/plugin/instance.rb:763:in `public_send`
Migrating to MakeChatMentionNotificationIdNullable (20230227172543)
Migrating to DropBadgeGrantedTitleColumn (20230228105851)
Migrating to AddExternalToSidebarUrls (20230303015952)
Theme setting type has changed but cannot be converted. 

 #<ThemeSettingsManager::Upload:0x00007f60ff714e38 @name=:background_image, @default="", @theme=#<Theme id: 31, name: "Search Banner", user_id: 2, created_at: "2021-08-03 16:38:09.042735000 +0000", updated_at: "2021-08-03 17:01:41.329058000 +0000", compiler_version: 0, user_selectable: false, hidden: false, color_scheme_id: nil, remote_theme_id: 29, component: true, enabled: true, auto_update: true>, @opts={:description=>"background image for the banner", :textarea=>false, :json_schema=>nil, :refresh=>false}, @types={:integer=>0, :float=>1, :string=>2, :bool=>3, :list=>4, :enum=>5, :upload=>6}>

```

I’m not seeing anything out of the ordinary. Perhaps the nginx lines are troubling? (my nginx setup is from official discourse maintenance page tutorial).

---

<div class="post-metadata">

### Author: ![pfaffman](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/pfaffman/32/120154_2.png) [@pfaffman](https://meta.discourse.org/u/pfaffman)
#### Post date: [March 8, 2023, 5:19pm UTC](https://meta.discourse.org/t/no-errors-instance-not-coming-back-up-after-rebuild/257515/4 "2023-03-08T17:19:38Z")

</div>

What happens if you visit your site? Can you share the URL?

---

<div class="post-metadata">

### Author: ![MarcP](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/marcp/32/160184_2.png) [@MarcP](https://meta.discourse.org/u/MarcP)
#### Post date: [March 8, 2023, 5:21pm UTC](https://meta.discourse.org/t/no-errors-instance-not-coming-back-up-after-rebuild/257515/5 "2023-03-08T17:21:26Z")

</div>

Rather not in public, but I’ll shoot you a DM.

Edit: rerouted nginx to ignore 502 errors etc. Just a bad gateway error.

 ![image](https://global.discourse-cdn.com/meta/original/4X/f/6/7/f679658f8a50e99bf189a1bbf76dc89881e9c938.png)

---

<div class="post-metadata">

### Author: ![pfaffman](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/pfaffman/32/120154_2.png) [@pfaffman](https://meta.discourse.org/u/pfaffman)
#### Post date: [March 8, 2023, 5:57pm UTC](https://meta.discourse.org/t/no-errors-instance-not-coming-back-up-after-rebuild/257515/6 "2023-03-08T17:57:31Z")

</div>

You’ll need to look at the log file when something tries to load the site and see what the error is.

If cloudflare (or whatever) is stopping a browser from accessing the site then maybe you can look through the logs to look for a 500 error to see what the issue is.

You might be able to `curl localhost` from inside the container.

---

<div class="post-metadata">

### Author: ![MarcP](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/marcp/32/160184_2.png) [@MarcP](https://meta.discourse.org/u/MarcP)
#### Post date: [March 8, 2023, 5:58pm UTC](https://meta.discourse.org/t/no-errors-instance-not-coming-back-up-after-rebuild/257515/7 "2023-03-08T17:58:34Z")

</div>

That is some good feedback. Will try that and get back.

EDIT: Hmm, I don’t think anything happens in the logs, but I’m seeing the following.

```plaintext
➜ ~ tail -f /var/discourse/shared/standalone/log/rails/production.log
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/subscribe.rb:14:in `subscribe'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis.rb:288:in `_subscription'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/commands/pubsub.rb:20:in `block in subscribe'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis.rb:265:in `block in synchronize'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis.rb:265:in `synchronize'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis.rb:265:in `synchronize'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/commands/pubsub.rb:19:in `subscribe'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/message_bus-4.3.2/lib/message_bus/backends/redis.rb:302:in `global_subscribe'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/message_bus-4.3.2/lib/message_bus.rb:768:in `global_subscribe_thread'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/message_bus-4.3.2/lib/message_bus.rb:739:in `block in new_subscriber_thread'

```

In my nginx I’m seeing:

```plaintext
023/03/08 17:58:38 [crit] 115962#115962: *448 connect() to unix:/var/discourse/shared/standalone/nginx.http.sock failed (2: No such file or directory) while connecting to upstream, client: IP.XXX server: [domain.com](http://domain.com), request: "GET /service-worker.js HTTP/2.0", upstream: "http://unix:/var/discourse/shared/standalone/nginx.http.sock:/service-worker.js", host: "domain.com", referrer: "https://domain.com/service-worker.js"
2023/03/08 17:59:32 [notice] 318573#318573: signal process started

```

Does this help? Or should I completely revert the nginx setup for this?

---

<div class="post-metadata">

### Author: ![Falco](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/falco/32/179432_2.png) [@Falco](https://meta.discourse.org/u/Falco)
#### Post date: [March 8, 2023, 6:19pm UTC](https://meta.discourse.org/t/no-errors-instance-not-coming-back-up-after-rebuild/257515/9 "2023-03-08T18:19:39Z")

</div>

> [@MarcP](#):
>
> `➜ ~ tail -f /var/discourse/shared/standalone/log/rails/production.log`

If you share the full backtrace (a few more line of that file) we can help you.

---

<div class="post-metadata">

### Author: ![MarcP](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/marcp/32/160184_2.png) [@MarcP](https://meta.discourse.org/u/MarcP)
#### Post date: [March 8, 2023, 6:30pm UTC](https://meta.discourse.org/t/no-errors-instance-not-coming-back-up-after-rebuild/257515/10 "2023-03-08T18:30:55Z")

</div>

```plaintext
Completed 200 OK in 25ms (Views: 0.1ms | ActiveRecord: 0.0ms | Allocations: 17734)
Started GET "/categories_and_latest" for [ip removed] at 2023-03-08 16:20:41 +0000
Processing by CategoriesController#categories_and_latest as JSON
  Rendered text template (Duration: 0.0ms | Allocations: 1)
Completed 200 OK in 112ms (Views: 0.2ms | ActiveRecord: 0.0ms | Allocations: 63242)
Started GET "/" for [ip removed] at 2023-03-08 16:21:00 +0000
Processing by CategoriesController#index as HTML
  Rendered categories/index.html.erb within layouts/crawler (Duration: 1.4ms | Allocations: 1135)
  Rendered layout layouts/crawler.html.erb (Duration: 6.7ms | Allocations: 3536)
Completed 200 OK in 75ms (Views: 7.7ms | ActiveRecord: 0.0ms | Allocations: 41712)
Started GET "/notifications?limit=30&recent=true&bump_last_seen_reviewable=true" for [ip removed] at 2023-03-08 16:21:35 +0000
Processing by NotificationsController#index as JSON
  Parameters: {"limit"=>"30", "recent"=>"true", "bump_last_seen_reviewable"=>"true"}
Completed 200 OK in 60ms (Views: 0.1ms | ActiveRecord: 0.0ms | Allocations: 30822)
Started GET "/" for [ip removed] at 2023-03-08 16:22:00 +0000
Processing by CategoriesController#index as HTML
  Rendered categories/index.html.erb within layouts/crawler (Duration: 1.1ms | Allocations: 1135)
  Rendered layout layouts/crawler.html.erb (Duration: 5.4ms | Allocations: 3536)
Completed 200 OK in 86ms (Views: 6.1ms | ActiveRecord: 0.0ms | Allocations: 41842)
Shutting down
Terminating quiet threads
Scheduler exiting...
Error fetching job: Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL)
Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL) subscribe failed, reconnecting in 1 second. Call stack /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:398:in `rescue in establish_connection'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:379:in `establish_connection'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:115:in `block in connect'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:344:in `with_reconnect'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:114:in `connect'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:409:in `ensure_connected'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:269:in `block in process'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:356:in `logging'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:268:in `process'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:175:in `block in call_loop'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:331:in `with_socket_timeout'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:174:in `call_loop'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/subscribe.rb:44:in `subscription'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/subscribe.rb:14:in `subscribe'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis.rb:288:in `_subscription'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/commands/pubsub.rb:20:in `block in subscribe'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis.rb:265:in `block in synchronize'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis.rb:265:in `synchronize'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis.rb:265:in `synchronize'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/commands/pubsub.rb:19:in `subscribe'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/message_bus-4.3.2/lib/message_bus/backends/redis.rb:302:in `global_subscribe'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/message_bus-4.3.2/lib/message_bus.rb:768:in `global_subscribe_thread'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/message_bus-4.3.2/lib/message_bus.rb:739:in `block in new_subscriber_thread'
Error fetching job: Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL)
Error fetching job: Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL)
Error fetching job: Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL)
Job exception: Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL)

Error fetching job: Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL)
Job exception: Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL)

Job exception: Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL)

Job exception: Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL)

Job exception: Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL)

Job exception: Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL)

Failed to process job: Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL) ["/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:398:in `rescue in establish_connection'", "/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:379:in `establish_connection'", "/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:115:in `block in connect'", "/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:344:in `with_reconnect'", "/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:114:in `connect'", "/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:409:in `ensure_connected'", "/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:269:in `block in process'", "/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:356:in `logging'", "/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:268:in `process'", "/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:161:in `call'", "/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rack-mini-profiler-3.0.0/lib/mini_profiler/profiling_methods.rb:85:in `block in profile_method'", "/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis.rb:270:in `block in send_command'", "/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis.rb:269:in `synchronize'", "/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis.rb:269:in `send_command'", "/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/commands/scripting.rb:110:in `_eval'", "/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/commands/scripting.rb:97:in `evalsha'", "/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/message_bus-4.3.2/lib/message_bus/backends/redis.rb:463:in `cached_eval'", "/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/message_bus-4.3.2/lib/message_bus/backends/redis.rb:150:in `publish'", "/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/message_bus-4.3.2/lib/message_bus.rb:391:in `publish'", "/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/message_bus-4.3.2/lib/message_bus.rb:751:in `block in new_subscriber_thread'", "/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/message_bus-4.3.2/lib/message_bus/timer_thread.rb:117:in `do_work'", "/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/message_bus-4.3.2/lib/message_bus/timer_thread.rb:95:in `block (2 levels) in queue'"]
Pausing to allow jobs to finish...
heartbeat: Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL)
Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL) subscribe failed, reconnecting in 1 second. Call stack /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:398:in `rescue in establish_connection'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:379:in `establish_connection'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:115:in `block in connect'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:344:in `with_reconnect'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:114:in `connect'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:409:in `ensure_connected'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:269:in `block in process'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:356:in `logging'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:268:in `process'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:161:in `call'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rack-mini-profiler-3.0.0/lib/mini_profiler/profiling_methods.rb:85:in `block in profile_method'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis.rb:270:in `block in send_command'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis.rb:269:in `synchronize'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis.rb:269:in `send_command'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/commands/strings.rb:191:in `get'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/message_bus-4.3.2/lib/message_bus/backends/redis.rb:401:in `process_global_backlog'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/message_bus-4.3.2/lib/message_bus/backends/redis.rb:286:in `block in global_subscribe'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/message_bus-4.3.2/lib/message_bus/backends/redis.rb:299:in `global_subscribe'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/message_bus-4.3.2/lib/message_bus.rb:768:in `global_subscribe_thread'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/message_bus-4.3.2/lib/message_bus.rb:739:in `block in new_subscriber_thread'
Job exception: Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL)

Bye!
Deprecation notice: (siwe) full_screen_login is now forced. The full_screen_login parameter can be removed from the auth_provider. (removal in Discourse 2.9.0) 
At /var/www/discourse/lib/plugin/instance.rb:763:in `public_send`
Migrating to MakeChatMentionNotificationIdNullable (20230227172543)
Migrating to DropBadgeGrantedTitleColumn (20230228105851)
Migrating to AddExternalToSidebarUrls (20230303015952)
Theme setting type has changed but cannot be converted. 

 #<ThemeSettingsManager::Upload:0x00007f60ff714e38 @name=:background_image, @default="", @theme=#<Theme id: 31, name: "Search Banner", user_id: 2, created_at: "2021-08-03 16:38:09.042735000 +0000", updated_at: "2021-08-03 17:01:41.329058000 +0000", compiler_version: 0, user_selectable: false, hidden: false, color_scheme_id: nil, remote_theme_id: 29, component: true, enabled: true, auto_update: true>, @opts={:description=>"background image for the banner", :textarea=>false, :json_schema=>nil, :refresh=>false}, @types={:integer=>0, :float=>1, :string=>2, :bool=>3, :list=>4, :enum=>5, :upload=>6}>
Deprecation notice: (siwe) full_screen_login is now forced. The full_screen_login parameter can be removed from the auth_provider. (removal in Discourse 2.9.0) 
At /var/www/discourse/lib/plugin/instance.rb:763:in `public_send`
Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL) subscribe failed, reconnecting in 1 second. Call stack /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:398:in `rescue in establish_connection'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:379:in `establish_connection'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:115:in `block in connect'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:344:in `with_reconnect'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:114:in `connect'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:409:in `ensure_connected'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:269:in `block in process'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:356:in `logging'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:268:in `process'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:175:in `block in call_loop'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:331:in `with_socket_timeout'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:174:in `call_loop'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/subscribe.rb:44:in `subscription'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/subscribe.rb:14:in `subscribe'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis.rb:288:in `_subscription'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/commands/pubsub.rb:20:in `block in subscribe'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis.rb:265:in `block in synchronize'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis.rb:265:in `synchronize'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis.rb:265:in `synchronize'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/commands/pubsub.rb:19:in `subscribe'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/message_bus-4.3.2/lib/message_bus/backends/redis.rb:302:in `global_subscribe'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/message_bus-4.3.2/lib/message_bus.rb:768:in `global_subscribe_thread'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/message_bus-4.3.2/lib/message_bus.rb:739:in `block in new_subscriber_thread'

```

This is from the moment before the “fatal” rebuild, up until after a few tries more with removed plugins etc.

---

<div class="post-metadata">

### Author: ![MarcP](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/marcp/32/160184_2.png) [@MarcP](https://meta.discourse.org/u/MarcP)
#### Post date: [March 8, 2023, 9:19pm UTC](https://meta.discourse.org/t/no-errors-instance-not-coming-back-up-after-rebuild/257515/11 "2023-03-08T21:19:15Z")

</div>

I deleted the following plugins, and now my instance is booting up again.

- [Automation](https://meta.discourse.org/t/discourse-automation/195773)
- [Fingerprint](https://meta.discourse.org/t/discourse-fingerprint-browser-fingerprinting-plugin/114890)
- [Translator](https://meta.discourse.org/t/discourse-translator/32630/1)

So it’s probably related to recent commits of one of those.

---

<div class="post-metadata">

### Author: ![pfaffman](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/pfaffman/32/120154_2.png) [@pfaffman](https://meta.discourse.org/u/pfaffman)
#### Post date: [March 8, 2023, 9:27pm UTC](https://meta.discourse.org/t/no-errors-instance-not-coming-back-up-after-rebuild/257515/12 "2023-03-08T21:27:53Z")

</div>

The error doesn’t get into the log because it happens rails cranks up:

```plaintext
root@test1-web-only:/var/www/discourse# rails c                                                                                                                                               
bundler: failed to load command: pry (/var/www/discourse/vendor/bundle/ruby/3.2.0/bin/pry)                                                                                                    
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/zeitwerk-2.6.7/lib/zeitwerk/loader/helpers.rb:135:in `const_get': uninitialized constant PluginInstance (NameError)                          
                                                                                                                                                                                              
    parent.const_get(cname, false)                                                                                                                                                            
          ^^^^^^^^^^                                                                                                                                                                          
Did you mean? PluginStore                       

```

I have replicated the error on another install. I’m trying to figure out which plugin it is right now.

---

<div class="post-metadata">

### Author: ![Falco](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/falco/32/179432_2.png) [@Falco](https://meta.discourse.org/u/Falco)
#### Post date: [March 8, 2023, 9:44pm UTC](https://meta.discourse.org/t/no-errors-instance-not-coming-back-up-after-rebuild/257515/13 "2023-03-08T21:44:30Z")

</div>

Can you please share the plugin list and the each plugin commit?

---

<div class="post-metadata">

### Author: ![davidkingham](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/davidkingham/32/119528_2.png) [@davidkingham](https://meta.discourse.org/u/davidkingham)
#### Post date: [March 8, 2023, 9:45pm UTC](https://meta.discourse.org/t/no-errors-instance-not-coming-back-up-after-rebuild/257515/14 "2023-03-08T21:45:14Z")

</div>

I have figured out it’s a conflict of Automation and another plugin. It will rebuild with only automation, but not with other plugins enabled. I’m getting close to finding the other culprit…

---

<div class="post-metadata">

### Author: ![Falco](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/falco/32/179432_2.png) [@Falco](https://meta.discourse.org/u/Falco)
#### Post date: [March 8, 2023, 9:46pm UTC](https://meta.discourse.org/t/no-errors-instance-not-coming-back-up-after-rebuild/257515/15 "2023-03-08T21:46:34Z")

</div>

Actually we had a bug in both automation and assign, and just fixed both. So rebuild _now_ will fix it.

cc @pfaffman

---

<div class="post-metadata">

### Author: ![pfaffman](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/pfaffman/32/120154_2.png) [@pfaffman](https://meta.discourse.org/u/pfaffman)
#### Post date: [March 8, 2023, 9:47pm UTC](https://meta.discourse.org/t/no-errors-instance-not-coming-back-up-after-rebuild/257515/16 "2023-03-08T21:47:02Z")

</div>

Someone else just told me that

> [@](#):
>
> This morning I updated my instance via /upgrade which I thought was pretty safe, but it killed the site. I then went in and manually disabled each plugin, of course it was the last one I tried! (Discourse Automation)

Here’s the list of plugins (plus one more that’s private)

```plaintext
          - git clone https://github.com/discourse/docker_manager.git
            #- git clone https://github.com/discourse/discourse-docs.git
            #- git clone https://github.com/discourse/discourse-solved.git
            #- git clone https://github.com/discourse/discourse-voting.git
            #- git clone https://github.com/discourse/discourse-reactions.git
            #- git clone https://github.com/discourse/discourse-canned-replies.git
            #- git clone https://github.com/jomaxro/discourse-plugin-site-setting-override.git
            #- git clone https://github.com/discourse/discourse-automation.git

```

I was able to get the

---

<div class="post-metadata">

### Author: ![pfaffman](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/pfaffman/32/120154_2.png) [@pfaffman](https://meta.discourse.org/u/pfaffman)
#### Post date: [March 8, 2023, 9:47pm UTC](https://meta.discourse.org/t/no-errors-instance-not-coming-back-up-after-rebuild/257515/17 "2023-03-08T21:47:44Z")

</div>

> [@Falco](#):
>
> So rebuild _now_ will fix it.

Aha. So apparently my test that was supposed to confirm the broken plugin confirms the fix.

---

<div class="post-metadata">

### Author: ![davidkingham](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/davidkingham/32/119528_2.png) [@davidkingham](https://meta.discourse.org/u/davidkingham)
#### Post date: [March 8, 2023, 9:48pm UTC](https://meta.discourse.org/t/no-errors-instance-not-coming-back-up-after-rebuild/257515/18 "2023-03-08T21:48:44Z")

</div>

Same here haha, how’s that for timing! Thanks Falco.

---

<div class="post-metadata">

### Author: ![MarcP](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/marcp/32/160184_2.png) [@MarcP](https://meta.discourse.org/u/MarcP)
#### Post date: [March 8, 2023, 9:55pm UTC](https://meta.discourse.org/t/no-errors-instance-not-coming-back-up-after-rebuild/257515/19 "2023-03-08T21:55:51Z")

</div>

I will try to rebuild now and restore the removed plugins. Just in case, this is the rest of the plugins:

EDIT: Working with the 3 removed plugins back installed. Thanks team!

| discourse 12436d05 | Up to date |
| --- | --- |
| docker\_manager e90c8f55 | Up to date |
| discourse-adplugin bfd4438b | Up to date |
| discourse-calendar adca3f65 | Up to date |
| discourse-category-lockdown 2cf5f064 | Up to date |
| discourse-chat-integration 75cf4136 | Up to date |
| discourse-chatbot eb9c50ae | Up to date |
| discourse-data-explorer 389b8e15 | Up to date |
| discourse-docs 63bb4629 | Up to date |
| discourse-encrypt 0f3c612b | Up to date |
| discourse-formatting-toolbar d99f3c6d | Up to date |
| discourse-gamification a842e183 | Up to date |
| discourse-patreon 778829aa | Up to date |
| discourse-policy b86d520c | Up to date |
| discourse-pushover-notifications 30711ac7 | Up to date |
| discourse-siwe 752687c8 | Up to date |
| discourse-solved 2c1c64af | Up to date |
| discourse-staff-alias 10ae5329 | Up to date |
| discourse-telegram-notifications d9886998 | Up to date |
| discourse-whos-online aeee51e4 | Up to date |

---

<div class="post-metadata">

### Author: ![system](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/system/32/443519_2.png) [@system](https://meta.discourse.org/u/system)
#### Post date: [April 7, 2023, 10:07pm UTC](https://meta.discourse.org/t/no-errors-instance-not-coming-back-up-after-rebuild/257515/21 "2023-04-07T22:07:54Z")

</div>

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.