Installed OK. Working fine for one day. Suddenly stopped working

Installation
1

Hi All,

I’ve recently moved the discourse forum from one server to another - an old Ubuntu (13?) to version 16.06. To the best of my understanding, it is running in a docker container with an apache/nginx front-end and is using version: 2.1.0.beta1 +282.

To accomplish this move, I upgraded to the latest version on the existing server, took a full backup, then installed the latest version on the target and imported the data. All seemed fine for one day.

Today, I was able to access the server (I was still logged in) but other people couldn’t. Then, I lost my access, meaning I got a blank screen, which is what other people are experiencing. One user did say that it worked on her phone, but not her main PC, but I suspect it is not working on either now.

Looking in Chrome dev tools, I have a host of errors of the form:

(index):32 Uncaught SyntaxError: Invalid or unexpected token
_ember_jquery-27e777857b8c0730dacfe09cb11711365d21a5db4f9ee0b85d494e4259cf6cda.js:59785 Uncaught Error: Could not find module `preload-store` imported from `discourse/models/topic`
    at o (_ember_jquery-27e777857b8c0730dacfe09cb11711365d21a5db4f9ee0b85d494e4259cf6cda.js:59785)
    at i (_ember_jquery-27e777857b8c0730dacfe09cb11711365d21a5db4f9ee0b85d494e4259cf6cda.js:59763)
    at _ember_jquery-27e777857b8c0730dacfe09cb11711365d21a5db4f9ee0b85d494e4259cf6cda.js:59820
    at e (_ember_jquery-27e777857b8c0730dacfe09cb11711365d21a5db4f9ee0b85d494e4259cf6cda.js:59687)
    at require (_ember_jquery-27e777857b8c0730dacfe09cb11711365d21a5db4f9ee0b85d494e4259cf6cda.js:59819)
    at _application-fad5e5f87c66e66d6c184629209e8cc05bc33467507b76e7013a3ff871328b38.js:12551
_ember_jquery-27e777857b8c0730dacfe09cb11711365d21a5db4f9ee0b85d494e4259cf6cda.js:59791 Uncaught Error: Could not find module preload-store
    at s (_ember_jquery-27e777857b8c0730dacfe09cb11711365d21a5db4f9ee0b85d494e4259cf6cda.js:59791)
    at require (_ember_jquery-27e777857b8c0730dacfe09cb11711365d21a5db4f9ee0b85d494e4259cf6cda.js:59806)
    at (index):307
    at (index):307
_ember_jquery-27e777857b8c0730dacfe09cb11711365d21a5db4f9ee0b85d494e4259cf6cda.js:59791 **Uncaught Error: Could not find module preload-store**
    at s (_ember_jquery-27e777857b8c0730dacfe09cb11711365d21a5db4f9ee0b85d494e4259cf6cda.js:59791)
    at require (_ember_jquery-27e777857b8c0730dacfe09cb11711365d21a5db4f9ee0b85d494e4259cf6cda.js:59806)
    at (index):313
    at (index):313
**(index):317 Uncaught SyntaxError: Invalid or unexpected token**
_ember_jquery-27e777857b8c0730dacfe09cb11711365d21a5db4f9ee0b85d494e4259cf6cda.js:29446 **TypeError: Cannot read property 'startRouting' of undefined**
    at startRouting (_ember_jquery-27e777857b8c0730dacfe09cb11711365d21a5db4f9ee0b85d494e4259cf6cda.js:15321)
    at s.didBecomeReady (_ember_jquery-27e777857b8c0730dacfe09cb11711365d21a5db4f9ee0b85d494e4259cf6cda.js:16290)
    at invokeWithOnError (_ember_jquery-27e777857b8c0730dacfe09cb11711365d21a5db4f9ee0b85d494e4259cf6cda.js:12554)
    at a.flush (_ember_jquery-27e777857b8c0730dacfe09cb11711365d21a5db4f9ee0b85d494e4259cf6cda.js:12613)
    at u.flush (_ember_jquery-27e777857b8c0730dacfe09cb11711365d21a5db4f9ee0b85d494e4259cf6cda.js:12737)
    at p.end (_ember_jquery-27e777857b8c0730dacfe09cb11711365d21a5db4f9ee0b85d494e4259cf6cda.js:12807)
    at p.run (_ember_jquery-27e777857b8c0730dacfe09cb11711365d21a5db4f9ee0b85d494e4259cf6cda.js:12921)
    at p.join (_ember_jquery-27e777857b8c0730dacfe09cb11711365d21a5db4f9ee0b85d494e4259cf6cda.js:12952)
    at Function.u.join (_ember_jquery-27e777857b8c0730dacfe09cb11711365d21a5db4f9ee0b85d494e4259cf6cda.js:34030)
    at HTMLDocument.<anonymous> (_ember_jquery-27e777857b8c0730dacfe09cb11711365d21a5db4f9ee0b85d494e4259cf6cda.js:34093)
u @ _ember_jquery-27e777857b8c0730dacfe09cb11711365d21a5db4f9ee0b85d494e4259cf6cda.js:29446
_ember_jquery-27e777857b8c0730dacfe09cb11711365d21a5db4f9ee0b85d494e4259cf6cda.js:29446 TypeError: Cannot read property 'startRouting' of undefined
    at startRouting (_ember_jquery-27e777857b8c0730dacfe09cb11711365d21a5db4f9ee0b85d494e4259cf6cda.js:15321)
    at s.didBecomeReady (_ember_jquery-27e777857b8c0730dacfe09cb11711365d21a5db4f9ee0b85d494e4259cf6cda.js:16290)
    at invokeWithOnError (_ember_jquery-27e777857b8c0730dacfe09cb11711365d21a5db4f9ee0b85d494e4259cf6cda.js:12554)
    at a.flush (_ember_jquery-27e777857b8c0730dacfe09cb11711365d21a5db4f9ee0b85d494e4259cf6cda.js:12613)
    at u.flush (_ember_jquery-27e777857b8c0730dacfe09cb11711365d21a5db4f9ee0b85d494e4259cf6cda.js:12737)
    at p.end (_ember_jquery-27e777857b8c0730dacfe09cb11711365d21a5db4f9ee0b85d494e4259cf6cda.js:12807)
    at p.run (_ember_jquery-27e777857b8c0730dacfe09cb11711365d21a5db4f9ee0b85d494e4259cf6cda.js:12921)
    at p.join (_ember_jquery-27e777857b8c0730dacfe09cb11711365d21a5db4f9ee0b85d494e4259cf6cda.js:12952)
    at Function.u.join (_ember_jquery-27e777857b8c0730dacfe09cb11711365d21a5db4f9ee0b85d494e4259cf6cda.js:34030)
    at HTMLDocument.<anonymous> (_ember_jquery-27e777857b8c0730dacfe09cb11711365d21a5db4f9ee0b85d494e4259cf6cda.js:34093)
u @ _ember_jquery-27e777857b8c0730dacfe09cb11711365d21a5db4f9ee0b85d494e4259cf6cda.js:29446

Things I have tried include:

  • restarting Apache
  • restarting NGINX
  • stopping the app in the container/docker
  • restarting the app in the container/docker
  • I’ve rebuilt the app.
  • I’ve read this: How do I manually update Discourse and Docker image to latest?
  • – as a consequence, I did the git pull, rebuild step - I already had the latest code.
  • – I also tried the safe mode - that seemed to get me in once or twice, but now no longer works.

Looking at the code, I see that it is unhappy with this on the main page (view > source)

<script>//<![CDATA[
e��ø�[�)q�h��i�$�5�D4�����z��J�;cmঀq`����\z��,��[��Ы��m��>J���!%>��R	�R�<i���L��(䔽�$��b\1�_�kH�#�BF;*�>7�\V��T�K��D���B�bK�*���aޢ-��u7�Q��e��b�fۺ�QC��V�]���a�3U��0�.k��LD��U�0˙�e.=1�.�>�H:��������*���(=i�)%kk^9d>�p���,���8��yH	:�Y3����G�/���kQ����:p���G+�1XF���}] �ʘ���x���
//]]></script>

And looking through the code in the docker container, I can see where that is inserted (in file: /app/views/layouts/application.html.erb):

<%= preload_script "ember_jquery" %>
------>    <%= preload_script "preload-store" %>
<%= preload_script "vendor" %>

Does anyone have any ideas as to why this might have occurred and how I could resolve it?

Thanks, Steve

2

If I’m not going miserably off topic, You shouldn’t have either of those installed for the vanilla 30 minutes docker install. Could you please do the following steps:

sudo apt-get purge apache2
sudo apt-get purge nginx
sudo apt-get autoclean
cd /var/discourse
git pull
./launcher rebuild app
1 Like
3

Hi @itsbhanusharma

Thanks for the reply. This forum is running on a server which hosts other applications - WordPress, api etc. I may not need the NGINX (my ignorance of that doesn’t help) but I am pretty sure I don’t want to remove Apache.

I can post the config files I am using for Apache if that is helpful plus my app.yml file. As I mentioned, why this would work for a day and then suddenly stop is what is even more intriguing. Is it cache related?

4

If You’re pretty sure that you can’t remove apache2, my simplest recommendation will be to move to a server dedicated to your forum. However, if you have to have forum & other applications on the same server then please search this forum for topics about configuring access to discourse with apache2 There have been some success cases but IMO those don’t really work well.

There is a lot that can happen and a lot more that may have been borked with time. You took a very nuclear decision and should have done some research before attempting such a complicated setup.

1 Like
5

This is not a complicated setup… but it requires some work and documentation.

3 Likes
6

first, some research should have been done.
Second, He’s talking of apache2 if I’m not mistaken. Article linked above contains nginx which plays very well with discourse.
Also, not to forget, there are 3 services fighting for port 80 & 443 above.

EDIT:
Found a howto though

Reason why it doesn’t works well is well documented by sam in comment below that tutorial. Please read it.

1 Like
7

I would recommend OP to drop Apache totally and use nginx outside of the Discourse container instead.

8

They have their API and website already running on apache2 dependency thing maybe? He mentioned that he can’t get rid of apache!

9

Hi @_Vincent and @itsbhanusharma

We’re already running some web apps under Apache so ideally, I would like to leave that as is - but I note your suggestions above @_Vincent and will definitely consider those (am also reading them now, too).

My app.yml file asks the container to listen on port 3000 and fwd to 80.

templates:
  - "templates/postgres.template.yml"
  - "templates/redis.template.yml"
  - "templates/web.template.yml"
  - "templates/web.ratelimited.template.yml"
## Uncomment these two lines if you wish to add Lets Encrypt (https)
  #- "templates/web.ssl.template.yml"
  #- "templates/web.letsencrypt.ssl.template.yml"

## which TCP/IP ports should this container expose?
## If you want Discourse to share a port with another webserver like Apache or nginx,
## see https://meta.discourse.org/t/17247 for details
expose:
  - "3000:80"   # http
#  - "443:443" # https

I’ve also included the templates being used as they were mentioned in the other posts.

My Apache config for the site is as follows:

<IfModule mod_ssl.c>
<VirtualHost _default_:443>
        ServerAdmin me@server-name-we-use.org
        ServerName server-name-we-use.org

        ProxyPreserveHost On
        ProxyRequests Off
        ProxyPass / http://localhost:3000/
        ProxyPassReverse / http://localhost:3000/
        RequestHeader set X-Forwarded-Proto "https"
        Header always set Strict-Transport-Security "max-age=31536000"

[chopped for brevity]

As you can see, there is a SSL certificate in place, too.

Assuming I have replicated the setup and not made a mistake somewhere, this was working on another server as-is so I know it ought to be possible. I didn’t set this up originally though hence my gaps in understanding.

The original docs I have did say about installing NGINX but I can’t see how that was hooked in and it seems to be routing traffic in part.

Why would this work for a day though? Any ideas what might be the reason for that? Nothing changed (by me) in between.

Thank you for your assistance, Steve

10

It can be either luck or something that self updated. bothered digging logs to find out what was working and had since stopped working?

There are some problems with apache that will not let long polling function as expected. This heavily limits the functioning of discourse. Hence, it is not a recommended setup.

One more thing I would recommend though is looking at the rest of your apache config so to be sure that there are no default or global parameters that may be interfering with discourse.

11

I have a bit more information.

Somehow, suddenly, the site started working yesterday. I am not sure why and it was after a small while, so not obviously immediately after I had done something.

However, it stayed up for about 18 hours. I logged in this morning and all was well. I replied to a post and saw that some other people had successfully logged on.

A little while later, it went down again.

I restarted Apache. No change.
I stopped and started the docker app. No change.
I rebuilt the app and watched it run. Still, no change.
I tried safe mode with all options, no change.

Looking at the production log file, I did see this which happened a few times before the site dropped.

Job exception: Cannot assign requested address - connect(2) for [::1]:6379

Job exception: Cannot assign requested address - connect(2) for [::1]:6379

Job exception: Error connecting to Redis on localhost:6379 (Errno::ECONNREFUSED)

Error connecting to Redis on localhost:6379 (Errno::ECONNREFUSED) subscribe failed, reconnecting in 1 second. Call stack ["/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/redis-4.0.1/lib/redis/client.rb:344:in `rescue in establish_connection'", "/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/redis-4.0.1/lib/redis/client.rb:328:in `establish_connection'", "/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/redis-4.0.1/lib/redis/client.rb:99:in `block in connect'", "/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/redis-4.0.1/lib/redis/client.rb:291:in `with_reconnect'", "/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/redis-4.0.1/lib/redis/client.rb:98:in `connect'", "/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/redis-4.0.1/lib/redis/client.rb:363:in `ensure_connected'", "/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/redis-4.0.1/lib/redis/client.rb:219:in `block in process'", "/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/redis-4.0.1/lib/redis/client.rb:304:in `logging'", "/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/redis-4.0.1/lib/redis/client.rb:218:in `process'", "/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/redis-4.0.1/lib/redis/client.rb:118:in `call'", "/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/redis-4.0.1/lib/redis.rb:889:in `block in get'", "/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/redis-4.0.1/lib/redis.rb:45:in `block in synchronize'", "/usr/local/lib/ruby/2.5.0/monitor.rb:226:in `mon_synchronize'", "/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/redis-4.0.1/lib/redis.rb:45:in `synchronize'", "/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/redis-4.0.1/lib/redis.rb:888:in `get'", "/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/message_bus-2.1.5/lib/message_bus/backends/redis.rb:285:in `process_global_backlog'", "/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/message_bus-2.1.5/lib/message_bus/backends/redis.rb:321:in `block in global_subscribe'", "/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/message_bus-2.1.5/lib/message_bus/backends/redis.rb:334:in `global_subscribe'", "/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/message_bus-2.1.5/lib/message_bus.rb:530:in `global_subscribe_thread'", "/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/message_bus-2.1.5/lib/message_bus.rb:478:in `block in new_subscriber_thread'"]
Migrating to AllowNullIpSearchLog (20180521184439)

Could it be connected?

I’ve been checking the Apache configs as suggested and nothing seems an issue so far. Anything in particular you think I should look for/be worried about?

Thanks, Steve

12

Seems like your redis is failing! perhaps a RAM issue?

13

I’ve ruled out RAM as being the cause. I’ve been monitoring the memory and doubled the VMs allocation to 16GB, so that is fine.

I now think this is related to SSL certificates because when I connect to the site without them through the Apache conf, I can get in.

I cannot log in however because in the prior site, it had HTTPS enabled. So, is there a way to disable that without logging in as an admin (because I can’t for the reason just mentioned).

Thanks for any assistance you can give,

Steve

ps - I’m going through this site and have found a few posts about how to enable SSL in NGINX (in the container) which I am attempting, but that isn’t working out just yet, so need a plan B.

14

You can change that site setting from rails console, Query can be found somewhere on this forum.

2 Likes
15

@itsbhanusharma that’s a big help - found a post here: How to change a site setting via the Console – see the last entry.

Thank you,

Right - now onto fixing the SSL :slight_smile:

2 Likes