Upgrade / Rebuilds Fail due to MaxMind DB EOL

Guys,

Hope somebody can help here. Ive not had a problem for over 12 months with upgrades to my forum that is running in a docker container on EC2

Ive tried:

  • launcher cleanup but no issues with disk space
  • latest git pull
  • multiple app rebuilds
  • Even tried a discourse-setup and was going to try a DB restore :frowning:
  • Server reboot
  • apt-get update; apt-get upgrade

Here is the tail of the log

Done compressing all JS files : 88.88 secs
rake aborted!
SocketError: Failed to open TCP connection to geolite.maxmind.com:443 (getaddrinfo: No address associated with hostname)
/var/www/discourse/lib/final_destination.rb:411:in `safe_session'
/var/www/discourse/lib/final_destination.rb:362:in `safe_get'
/var/www/discourse/lib/final_destination.rb:131:in `get'
/var/www/discourse/lib/file_helper.rb:51:in `download'
/var/www/discourse/lib/discourse_ip_info.rb:30:in `mmdb_download'
/var/www/discourse/lib/tasks/assets.rake:220:in `block (3 levels) in <top (required)>'
/var/www/discourse/lib/tasks/assets.rake:219:in `each'
/var/www/discourse/lib/tasks/assets.rake:219:in `block (2 levels) in <top (required)>'

Caused by:

SocketError: getaddrinfo: No address associated with hostname

/var/www/discourse/lib/final_destination.rb:411:in `safe_session'
/var/www/discourse/lib/final_destination.rb:362:in `safe_get'
/var/www/discourse/lib/final_destination.rb:131:in `get'
/var/www/discourse/lib/file_helper.rb:51:in `download'
/var/www/discourse/lib/discourse_ip_info.rb:30:in `mmdb_download'
/var/www/discourse/lib/tasks/assets.rake:220:in `block (3 levels) in <top (required)>'
/var/www/discourse/lib/tasks/assets.rake:219:in `each'
/var/www/discourse/lib/tasks/assets.rake:219:in `block (2 levels) in <top (required)>'

Tasks: TOP => assets:precompile

(See full trace by running task with --trace)

I, [2019-12-30T18:56:27.608998 #1] INFO -- : Downloading MaxMindDB...
Compressing Javascript and Generating Source Maps
I, [2019-12-30T18:56:27.633923 #1] INFO -- : Terminating async processes
I, [2019-12-30T18:56:27.637383 #1] INFO -- : Sending INT to HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/10/bin/postmaster -D /etc/postgresql/10/main pid: 49
I, [2019-12-30T18:56:27.639770 #1] INFO -- : Sending TERM to exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf pid: 166
166:signal-handler (1577732187) Received SIGTERM scheduling shutdown...
2019-12-30 18:56:27.639 UTC [49] LOG: received fast shutdown request
2019-12-30 18:56:27.654 UTC [49] LOG: aborting any active transactions
166:M 30 Dec 2019 18:56:27.666 # User requested shutdown...
166:M 30 Dec 2019 18:56:27.694 * Saving the final RDB snapshot before exiting.
2019-12-30 18:56:27.702 UTC [49] LOG: worker process: logical replication launcher (PID 58) exited with exit code 1
2019-12-30 18:56:27.711 UTC [53] LOG: shutting down
2019-12-30 18:56:27.819 UTC [49] LOG: database system is shut down
166:M 30 Dec 2019 18:56:27.885 * DB saved on disk
166:M 30 Dec 2019 18:56:27.886 # Redis is now ready to exit, bye bye...

FAILED
--------------------
Pups::ExecError: cd /var/www/discourse && su discourse -c 'bundle exec rake assets:precompile' failed with return #<Process::Status: pid 507 exit 1>
Location of failure: /pups/lib/pups/exec_command.rb:112:in `spawn'
exec failed with the params {"cd"=>"$home", "hook"=>"assets_precompile", "cmd"=>["su discourse -c 'bundle exec rake assets:precompile'"]}
fcf17d793c27c4e87616420ead222cc3f4a9fc163f239a5542b1c9a092579b30

** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one.
./discourse-doctor may help diagnose the problem.

Really hope somebody can shed some light on this.

Thanks all.

6 Likes

In case anyone wants to continue using the last files published under the old license, I have saved all of them to archive.org.

All files are created by MaxMind and distributed under the Creative Commons Attribution-ShareAlike 4.0 International License.

archive.org URL MD5 Hash Last-Modified Header
GeoLite2-City.tar.gz 7ba2c58b4e0eac6c08c6399aebdae26c Tue, 24 Dec 2019 17:46:00 GMT
GeoLite2-Country.tar.gz dc6224c648350d90f344a0c5c3ca5474 Tue, 24 Dec 2019 17:40:42 GMT
GeoLite2-ASN.tar.gz f3c9c5775fd226db6e8098675ff65861 Tue, 24 Dec 2019 14:16:58 GMT
GeoLite2-City-CSV.zip 89e5bf01a970b5668d74cf29a242d546 Tue, 24 Dec 2019 17:45:52 GMT
GeoLite2-Country-CSV.zip f50b518341de54fe48c5e34f13e24e99 Tue, 24 Dec 2019 17:40:41 GMT
GeoLite2-ASN-CSV.zip 463ccd73c104c52547fa50a84fc2f86e Tue, 24 Dec 2019 14:16:58 GMT

Archive of info-page (Screenshot)

10 Likes

We now support the officially supported way of obtaining databases per:

https://github.com/discourse/discourse/commit/a8ffb6949c76ea16e96ec4d45e6524cd8cc42c2d

If you wish to get regular MaxMind updates you will have to head to GeoLite2 Sign Up | MaxMind to register an account and generate a license key.

Then you would amend your container to include it in the env: section:

env:
   DISCOURSE_MAXMIND_LICENSE_KEY: ...key here...

The fix ensures there is no time bomb anymore. If we have no license key we will never try to download the files.

We are discussing with MaxMind the possibility of bundling the updated DBs in our new base images. There are 2 alternatives here for self hosters depending on the outcome:

  1. No IP resolution on user profile page and admin page until you add a license

  2. Stale IP resolution for up to N days since you got a base image (aka did ./launcher rebuild app )

It is very likely (1) will be the self-hoster outcome.

We are also investigating other alternatives here, but our hands are pretty tied.

I am pretty against adding functionality to core that does IP lookups via a web service cause that exposes information to a third party. There are only 2 real big alternatives out there and both require registration.

29 Likes

@sam I have a small feature request, can a setting be added to admin panel to add the maxmind license key?

4 Likes

Unlikely in the near future, this is a global setting, per site in a Multisite makes no sense

6 Likes

Anybody else getting this error when trying to sign up to maxmind account?

I can confirm that we were able to sign up for an account without issue. Youā€™ll need to reach out to MaxMind for support with account issues, nothing we can help with here.

6 Likes

Iā€™m trying to upgrade from 2.3.8 o 2.3.9 but nothing works.

The relevant error, as far as I can tell, is the one discussed in this topic.

rake aborted!
SocketError: Failed to open TCP connection to geolite.maxmind.com:443 (getaddrinfo: Name or service not known)

In my app.yml Iā€™ve tried values 30, 100, and 0 for that env flag:

DISCOURSE_REFRESH_MAXMIND_DB_DURING_PRECOMPILE_DAYS: 0

But it never seems to make any difference (I wonder if itā€™s being properly recognized).

What else could I try?

1 Like

Thanks to everyone who worked on the quick fix the past few weeks. Had to do the same thing with Matomo, which was much more painful (manual).

That brings me to my question: Where all is the MaxMind data used? Just in cases of admins looking at usersā€™ IP addresses? Something else Iā€™m not thinking of?

Obviously already have the MM account but wondering if itā€™s worth the effort to do with any urgency.

3 Likes

It does geographical lookups of ip numbers. Iā€™ve done a bunch of installs and upgrades lately without setting an API key and rebuilds work fine. (current version is beta 10)

Also itā€™s not that hard to get a key to download the database.

Iā€™m hoping to update discourse-setup to all for the key next week.

4 Likes

Correct, it is used for user IP lookup by admins. Also for the ā€œrecently used devicesā€ list in user preferences, and for admin alerts when a new signin is detected to their account from a different location.

3 Likes

Sorry for the intrusion, but I wanted to ask whether it is planned to keep the ability to have the IP lookup disabled. It seems to me that forcing admins to subscribe to a third party service is not a good idea. Myself I already have a license key for other uses, so I am speaking in a general sense here.

If you donā€™t enter a license key, then it will be disabled. There is nothing forcing admins to set it up.

Also, just in case there is any confusion, Discourse never sends IP addresses to a third party. Discourse downloads an entire database of IP locations from MaxMind, and then does the lookup internally.

9 Likes

Great, thanks for the clarification!

4 Likes

Revising this ā€¦ this was backported to stable.

12 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.