Fail to download maxmind db with valid key & can download the DB from inside container

The build failed because of this. I removed the ENV variable and rebuild without it. Then went to debugging inside the container. where I went and added the key to discourse.conf, it’s still failing.

(Thankfully I did this before telling the client that they’d given me an invalid key!)

I added a puts to the rake task to print the URL that it’s trying to download the database like this:

    url =
      "https://download.maxmind.com/app/geoip_download?license_key=#{GlobalSetting.maxmind_license_key}&edition_id=#{name}&suffix=tar.gz"

    gz_file =
      FileHelper.download(
        url,
        max_file_size: 100.megabytes,
        tmp_file_name: "#{name}.gz",
        validate_uri: false,
        follow_redirect: false,
      )

    if gz_file.nil?
      puts "no gzfile"
    end
    puts "Got file #{gz_file} for #{url}"
    filename = File.basename(gz_file.path)

Now I get this:

root@web-only:/var/www/discourse# rake maxminddb:get
Plugin name is 'discourse-topic-voting', but plugin directory is named 'discourse-voting'
Downloading MaxMindDb's GeoLite2-City...
no gzfile
Got file  for https://download.maxmind.com/app/geoip_download?license_key=VALID_KEY_IS_HERE&edition_id=GeoLite2-City&suffix=tar.gz
rake aborted!
NoMethodError: undefined method `path' for nil:NilClass (NoMethodError)

    filename = File.basename(gz_file.path)
                                    ^^^^^
/var/www/discourse/lib/discourse_ip_info.rb:52:in `mmdb_download'
/var/www/discourse/lib/tasks/maxminddb.rake:9:in `block (2 levels) in <main>'
/var/www/discourse/lib/tasks/maxminddb.rake:7:in `each'
/var/www/discourse/lib/tasks/maxminddb.rake:7:in `block in <main>'
/usr/local/bin/bundle:25:in `load'
/usr/local/bin/bundle:25:in `<main>'
Tasks: TOP => maxminddb:get
(See full trace by running task with --trace)

There was another topic where the problem was that the site was blocked by Maxmind, so I tried, and successfully downloaded that URL with wget from inside the container on the server in question.

It seems to me that FileHelper.download isn’t much of a Helper after all and is silently returning nil instead of a file. The file is only 44MB, so that’s not it. When I check the full headers with wget I don’t see a redirect.

Any ideas what the deal could be?

Is there a reason not to catch this error and let a build complete with an invalid key? Maybe just print a warning?

1 Like

Does it work better with follow_redirect: true?

As per a recent change, MaxMind changed their URL to download, but you can keep the original one as long you allow redirection.

2 Likes

I wonder whether there was a miscommunication and it’s not really May 1 that this starts. :thinking:

2 Likes

OMG. Yes. I don’t know why I didn’t just change that and think that I knew that it wasn’t redirecting.

4 Likes

Cool, merged it, let us know how it goes please!

3 Likes

Tried putting our forum’s maxmind key back into use, we still get the original error when trying to rebuild.

1 Like

Could you check if you use the latest Discourse version? It works for Jay; I can’t see any reasons it would not work for you, especially when it follows MaxMind’s recommendation. :thinking:

1 Like

Not sure how it wouldn’t when rebuilding the app

You’re right. You never know, though. :smile: What is the hash version displayed? We can quickly check, at the very least.

I reread the MaxMind announcement, but I can’t see anything besides changing the URL or allowing redirections. :thinking:

1 Like

Are you running tests-passed or latest-release?

We’re on tests-passed, at least afaik.

What’s the url or get the commit when you view source.

End of the logs from the crashed build:

Downloading MaxMindDB...
Compressing Javascript and Generating Source Maps
I, [2024-03-26T13:27:38.305597 #1] INFO -- : Terminating async processes
I, [2024-03-26T13:27:38.307312 #1]  INFO -- : Sending INT to HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/13/bin/postmaster -D /etc/postgresql/13/main pid: 42
I, [2024-03-26T13:27:38.307891 #1]  INFO -- : Sending TERM to exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf pid: 111
111:signal-handler (1711459658) Received SIGTERM scheduling shutdown...
2024-03-26 13:27:38.307 UTC [42] LOG:  received fast shutdown request
2024-03-26 13:27:38.314 UTC [42] LOG:  aborting any active transactions
2024-03-26 13:27:38.321 UTC [42] LOG:  background worker "logical replication launcher" (PID 51) exited with exit code 1
2024-03-26 13:27:38.322 UTC [46] LOG:  shutting down
111:M 26 Mar 2024 13:27:38.353 # User requested shutdown...
111:M 26 Mar 2024 13:27:38.353 * Saving the final RDB snapshot before exiting.
2024-03-26 13:27:38.415 UTC [42] LOG:  database system is shut down
111:M 26 Mar 2024 13:27:39.896 * DB saved on disk
111:M 26 Mar 2024 13:27:39.896 # Redis is now ready to exit, bye bye...
FAILED
--------------------
Pups::ExecError: cd /var/www/discourse && su discourse -c 'bundle exec rake themes:update assets:precompile' failed with return #<Process::Status: pid 3889 exit 1>
Location of failure: /usr/local/lib/ruby/gems/3.2.0/gems/pups-1.2.1/lib/pups/exec_command.rb:132:in `spawn'
exec failed with the params {"cd"=>"$home", "hook"=>"assets_precompile", "cmd"=>["su discourse -c 'bundle exec rake themes:update assets:precompile'"]}
bootstrap failed with exit code 1
** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one.
./discourse-doctor may help diagnose the problem.
fe4eec7726fdbe08aaa2691e806bb17423a8f4eaedb8414397e3097d725e04c7

Commit: https://github.com/discourse/discourse/commits/f350cea5d2

As a later side note, I didn’t link to the site this occurred on, because one of the staff members was testing a kernel bug in a separate container on the same server, and crashed the whole server by mistake, so linking to the site wouldn’t have helped at all.

1 Like

Does it still happen @Firepup650?

I’ve been meaning to rebuild for a bit, so I’ll run one and see.

1 Like

I can confirm, the issue no longer occurs. Our forum now rebuilds fine.

4 Likes