Missing system avatar after migrating to s3

A couple weeks ago we have migrated from local storage to S3. Posts and user avatars were successfully migrated, except for the custom system avatar.

It seems the url rewrite failed as the S3_CDN_URL gets appended after the forum url.

I havent checked anything in the database yet but it seems that “https:{S3_URL}” is the relative global url for the avatar?

https://forum.tosdr.org/https://tosdr-forum.s3.eu-west-2.jbcdn.net/original/1X/3662711e5030823983d25ac319ce6b9a7ad030e6.png

Note that this only happened with the system avatar, the file exists on S3 as well, its just the relative url thats bugged.

Changing avatars does not work either. Gravatar results in the same issue, as well as letter images.

Is there any solution to this? Note: We have already reinstalled the forum to an older backup and migrated the site again.

1 Like

I checked the database and all uploads were totally fine. (The avatar upload in question was valid)

Could it be this line somehow?

https://github.com/discourse/discourse/blob/5e93730375011e282ee5f7da47732f3f75742dfa/lib/file_store/s3_store.rb#L135-L137

s3_cdn_url is present though, I cant explain it any other way.

It doesn’t seem to update the avatars either regardless of a 200 OK

Started PUT "/u/system/preferences/avatar/pick" for 87.78.131.160 at 2021-04-10 12:55:46 +0000
Processing by UsersController#pick_avatar as */*
  Parameters: {"upload_id"=>"", "type"=>"system", "username"=>"system"}
Completed 200 OK in 15ms (Views: 0.1ms | ActiveRecord: 0.0ms | Allocations: 4976)
Started PUT "/u/system/preferences/avatar/pick" for 87.78.131.160 at 2021-04-10 12:55:02 +0000
Processing by UsersController#pick_avatar as */*
  Parameters: {"upload_id"=>"1", "type"=>"gravatar", "username"=>"system"}
Completed 200 OK in 15ms (Views: 0.2ms | ActiveRecord: 0.0ms | Allocations: 5061)
Started PUT "/u/system/preferences/avatar/pick" for 87.78.131.160 at 2021-04-10 12:54:13 +0000
Processing by UsersController#pick_avatar as */*
  Parameters: {"upload_id"=>"695", "type"=>"uploaded", "username"=>"system"}
Completed 200 OK in 17ms (Views: 0.2ms | ActiveRecord: 0.0ms | Allocations: 4982)

All above result in the query to the invalid base url:

Started GET "/u/system/preferences/https://tosdr-forum.s3.eu-west-2.jbcdn.net/original/1X/3662711e5030823983d25ac319ce6b9a7ad030e6.png" for 87.78.131.160 at 2021-04-10 12:54:15 +0000
Processing by UserBadgesController#username as JSON
  Parameters: {"username"=>"system"}
ActionController::RoutingError (No route matches [GET] "/u/system/preferences/https:/tosdr-forum.s3.eu-west-2.jbcdn.net/original/1X/3662711e5030823983d25ac319ce6b9a7ad030e6.png")
config/initializers/100-quiet_logger.rb:23:in `call'
config/initializers/100-silence_logger.rb:31:in `call'
lib/middleware/enforce_hostname.rb:23:in `call'
lib/middleware/request_tracker.rb:177:in `call'
  Rendering exceptions/not_found.html.erb within layouts/no_ember
  Rendered exceptions/not_found.html.erb within layouts/no_ember (Duration: 0.3ms | Allocations: 105)
  Rendered layouts/_head.html.erb (Duration: 1.2ms | Allocations: 262)
  Rendered common/_discourse_stylesheet.html.erb (Duration: 3.5ms | Allocations: 1094)
Completed 200 OK in 19ms (Views: 0.2ms | ActiveRecord: 0.0ms | Allocations: 4961)