White screen after deleting custom emoji

Support
1

Hi everyone,

I have a problem with a Discourse install where the screen is completely blank after I deleted a custom emoji from the admin panel.

I have tried running ./launcher rebuild app and it didn’t work, I’m still getting the white screen :frowning:

In the nginx log I got a whole series of errors like these as the last entries before fail:

[error] 212#212: *9 connect() failed (111: Connection refused) while connecting to upstream, client: 555.555.555.55, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:3000/", host: xxx

Does anyone know what’s going on please?

Any help would be very much appreciated!

Thanks!!

p.s. I am running Discourse Docker on a Digital Ocean Droplet

2

Disk full? Out of memory? Swap file set up? Docker installed properly? I think you have some other problem. Check disk, memory, and docker info.

3

Thanks, I’ve checked and all seems ok, talking to Digital Ocean to see if they can spot anything wrong.

I will report back :slight_smile:

4

Update:

Docker seems fine:

# docker info
Containers: 1
 Running: 1
 Paused: 0
 Stopped: 0
Images: 4
Server Version: 17.03.0-ce
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 7
 Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins: 
 Volume: local
 Network: bridge host macvlan null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 977c511eda0925a723debdc94d09459af49d082a
runc version: a01dafd48bc1c7cc12bdb01206f9fea7dd6feb70
init version: 949e6fa
Security Options:
 apparmor
 seccomp
  Profile: default
Kernel Version: 4.4.0-75-generic
Operating System: Ubuntu 16.04.2 LTS
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 1.953 GiB
Name: ubuntu-o
ID: Q34X:7PTI:SI67:4XJX:4YGR:EJGC:MR74:N3DT:SWBY:6WKX:YMJK:SHDO
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
WARNING: No swap limit support
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

Docker stats:

CONTAINER           CPU %               MEM USAGE / LIMIT       MEM %               NET I/O             BLOCK I/O           PIDS
e97b6427d2cf        5.98%               1.017 GiB / 1.953 GiB   52.07%              161 kB / 216 kB     144 MB / 15.6 MB    65

Memory usage:

# df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            981M     0  981M   0% /dev
tmpfs           201M  3.2M  197M   2% /run
/dev/vda1        30G  9.9G   20G  34% /
tmpfs          1001M  596K 1000M   1% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs          1001M     0 1001M   0% /sys/fs/cgroup
none             30G  9.9G   20G  34% /var/lib/docker/aufs/mnt/99c0f2807a ... 2
shm              64M  4.0K   64M   1% /var/lib/docker/containers/e ... a/shm
tmpfs           201M     0  201M   0% /run/user/0

I’m really tired and can’t see anything wrong and I’m not sure how to test that the swap file is set up, it should be though! As I say, everything was working fine and nothing was changed except deleting the emoji.

From my local machine:

$ curl -i <myip>
HTTP/1.1 301 Moved Permanently
Server: nginx/1.11.6
Date: Wed, 26 Apr 2017 00:03:36 GMT
Content-Type: text/html
Content-Length: 185
Location: https://<myserver>
Connection: keep-alive

<html>
<head><title>301 Moved Permanently</title></head>
<body bgcolor="white">
<center><h1>301 Moved Permanently</h1></center>
<hr><center>nginx/1.11.6</center>
</body>
</html>

Does this help any?

Thanks again!! :slight_smile:

5

OK, a bit more awake this morning, here’s the log where the 500 errors started and as I said, it was literally after I deleted a custom emoji, of course there still could be some other problem somewhere that is causing this, but now I am pretty convinced this was the trigger event:

Started GET "/admin/customize/emojis.json?_=1493157113083" for 213.205.253.56 at 2017-04-25 21:52:21 +0000
Processing by Admin::EmojisController#index as JSON
  Parameters: {"_"=>"1493157113083"}
Completed 200 OK in 2ms (Views: 0.1ms | ActiveRecord: 0.0ms)
Started DELETE "/admin/customize/emojis/ord_88" for 213.205.253.56 at 2017-04-25 21:52:25 +0000
Processing by Admin::EmojisController#destroy as */*
  Parameters: {"id"=>"ord_88"}
Completed 200 OK in 80ms (Views: 4.7ms | ActiveRecord: 13.5ms)
Started GET "/latest.json?order=default&_=1493157113084" for 213.205.253.56 at 2017-04-25 21:52:28 +0000
Processing by ListController#latest as JSON
  Parameters: {"order"=>"default", "_"=>"1493157113084"}
Completed 200 OK in 341ms (Views: 0.6ms | ActiveRecord: 45.9ms)
Started GET "/" for 213.205.253.56 at 2017-04-25 21:52:30 +0000
Processing by ListController#latest as HTML
Completed 500 Internal Server Error in 55ms (ActiveRecord: 15.1ms)
NoMethodError (undefined method `url' for nil:NilClass)
/var/www/discourse/app/models/emoji.rb:98:in `block (2 levels) in load_custom'
Started GET "/" for <myip> at 2017-04-25 21:52:34 +0000
Processing by ListController#latest as HTML
Completed 500 Internal Server Error in 85ms (ActiveRecord: 14.8ms)
NoMethodError (undefined method `url' for nil:NilClass)
/var/www/discourse/app/models/emoji.rb:98:in `block (2 levels) in load_custom'

From this moment onwards there have been problems and the site is unusable :frowning:

Does anyone know what has happened here and how to fix it please?

Thank you!!

6

OK problem fixed following the advice given here:

In short there was a broken emoji, deleting the custom emoji and clearing the emoji cache solved the problem.

Thanks everyone, we’re back up and running again, hope this helps others :slight_smile:

2 Likes
8

I’ve have a look through the code path again to see if I missed something. You shouldn’t have to go through all the trouble just to remove a custom emoji.

4 Likes
9

Thanks for this - please let me know if you need any further information or have any questions and I will try and help :slight_smile:

10

I can’t really tell what the root cause is here but I’ve added preventive measures to not blow up the whole site just because one custom emoji is broken.

https://github.com/discourse/discourse/commit/e61959e6a8c8c7095ac69ec4a96486a7dcb8b511

4 Likes
11

That’s good enough for me - thank you for your efforts!!

All the best
Kel

1 Like
12

This topic was automatically closed after 3 days. New replies are no longer allowed.