### Author:  [@Daniel\_Lopes](https://meta.discourse.org/u/Daniel_Lopes)
#### Post date: [October 12, 2018, 7:56pm UTC](https://meta.discourse.org/t/problems-with-avatar-uploads-due-to-s3-changes/99400/1 "2018-10-12T19:56:11Z")
Hi everyone,
Somehow I keep having issues with avatars. I’m on the last version ([v2.2.0.beta3 +3](https://github.com/discourse/discourse/compare/2178f7768fe7cfebd9ee1ad8aa024e11f1e11bc6...tests-passed)).
This is something noticed a few weeks ago but I’m only having the time to debug now. At first, I thought it could be a problem with the customizations I did through my theme plugin, but I don’t think that’s the case.
Some images show up but some others don’t. Has anyone seem something like that?
Here’s an example of an image that should be showing up (both urls seem correct):
`https://app.thewatercooler.io/user_avatar/app.thewatercooler.io/tammycravit/45/1089_1.png`
And here’s another one that works:
`https://app.thewatercooler.io/user_avatar/app.thewatercooler.io/robertwilliams/45/1456_1.png`
Here are all my S3 settings:
---
### Author:  [@Daniel\_Lopes](https://meta.discourse.org/u/Daniel_Lopes)
#### Post date: [October 12, 2018, 8:05pm UTC](https://meta.discourse.org/t/problems-with-avatar-uploads-due-to-s3-changes/99400/2 "2018-10-12T20:05:48Z")
I think the problem might be related to Gravatar. I’m checking the database and looks like the users where the images are not showing up are the ones that Discourse is trying to download from Gravatar (with no success).
```plaintext
[31] pry(main)> u.user_avatar.update_gravatar!
=> nil
[32] pry(main)> u.refresh_avatar
=> nil
[33] pry(main)> u.user_avatar
=> #
[34] pry(main)> u.small_avatar_url
=> "https://app.thewatercooler.io/user_avatar/app.thewatercooler.io/valedeoro/45/1282_1.png"
[35] pry(main)> u.refresh_avatar
=> nil
[36] pry(main)>
```
And I can confirm that the Gravatar file exists in the S3 bucket by checking the associated object and searching the listed file on Amazon:
```plaintext
[38] pry(main)> u.user_avatar.gravatar_upload
```
I’m also noticing a really long delay to render some of the images that display correctly.
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [October 12, 2018, 8:47pm UTC](https://meta.discourse.org/t/problems-with-avatar-uploads-due-to-s3-changes/99400/3 "2018-10-12T20:47:22Z")
It is already the weekend for him but @tgxworld can check when it is his Monday
---
### Author:  [@Daniel\_Lopes](https://meta.discourse.org/u/Daniel_Lopes)
#### Post date: [October 12, 2018, 9:33pm UTC](https://meta.discourse.org/t/problems-with-avatar-uploads-due-to-s3-changes/99400/4 "2018-10-12T21:33:20Z")
Thank you Jeff! No rush at all. I’m running the rake tasks to refresh the avatars now, and will keep this post updated 🙂
---
### Author:  [@Daniel\_Lopes](https://meta.discourse.org/u/Daniel_Lopes)
#### Post date: [October 15, 2018, 11:03pm UTC](https://meta.discourse.org/t/problems-with-avatar-uploads-due-to-s3-changes/99400/5 "2018-10-15T23:03:21Z")
Running the rake tasks had no results. I will try to debug more tomorrow.
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [October 16, 2018, 1:59am UTC](https://meta.discourse.org/t/problems-with-avatar-uploads-due-to-s3-changes/99400/6 "2018-10-16T01:59:22Z")
Can you be sure to update to latest, I did notice some issues with one of our once off jobs did play up on meta [FIX: reduce amount of work onceoff does · discourse/discourse@8d06731 · GitHub](https://github.com/discourse/discourse/commit/8d06731484f84ee680d12d355c3b82ddc1191140)
---
### Author:  [@tgxworld](https://meta.discourse.org/u/tgxworld)
#### Post date: [October 16, 2018, 2:39pm UTC](https://meta.discourse.org/t/problems-with-avatar-uploads-due-to-s3-changes/99400/8 "2018-10-16T14:39:11Z")
Does `Upload.find(1282)` return an upload record?
---
### Author:  [@Daniel\_Lopes](https://meta.discourse.org/u/Daniel_Lopes)
#### Post date: [October 16, 2018, 3:21pm UTC](https://meta.discourse.org/t/problems-with-avatar-uploads-due-to-s3-changes/99400/9 "2018-10-16T15:21:39Z")
> [@tgxworld](#):
>
> Upload.find(1282)
Thanks for replying. Yes, it does and the file exists in the bucket too.
```plaintext
[1] pry(main)> Upload.find(1282)
=> #
```
@sam updating from Beta to Stable now and I’ll post back here in a bit. Should I run the rake tasks to refresh everything again?
---
### Author:  [@tgxworld](https://meta.discourse.org/u/tgxworld)
#### Post date: [October 16, 2018, 3:24pm UTC](https://meta.discourse.org/t/problems-with-avatar-uploads-due-to-s3-changes/99400/10 "2018-10-16T15:24:09Z")
Interesting, what about `upload.optimized_images`?
---
### Author:  [@Daniel\_Lopes](https://meta.discourse.org/u/Daniel_Lopes)
#### Post date: [October 16, 2018, 3:25pm UTC](https://meta.discourse.org/t/problems-with-avatar-uploads-due-to-s3-changes/99400/11 "2018-10-16T15:25:53Z")
> [@tgxworld](#):
>
> upload.optimized\_images
That returns an empty array:
```
[3] pry(main)> upload.optimized_images
=> []
[4] pry(main)>
```
---
### Author:  [@tgxworld](https://meta.discourse.org/u/tgxworld)
#### Post date: [October 16, 2018, 3:27pm UTC](https://meta.discourse.org/t/problems-with-avatar-uploads-due-to-s3-changes/99400/12 "2018-10-16T15:27:46Z")
Hmm that is the problem here. I’ll have a look at this tomorrow to see if I can reproduce with the given image.
---
### Author:  [@Daniel\_Lopes](https://meta.discourse.org/u/Daniel_Lopes)
#### Post date: [October 16, 2018, 3:29pm UTC](https://meta.discourse.org/t/problems-with-avatar-uploads-due-to-s3-changes/99400/13 "2018-10-16T15:29:58Z")
Thank you so much! I’m now updating to get Sam’s commit and will run the rake tasks to refresh again, and see what happens.
---
### Author:  [@Daniel\_Lopes](https://meta.discourse.org/u/Daniel_Lopes)
#### Post date: [October 16, 2018, 9:30pm UTC](https://meta.discourse.org/t/problems-with-avatar-uploads-due-to-s3-changes/99400/14 "2018-10-16T21:30:38Z")
The problem seems to be widespread actually. Now I’m on tests-passed branch, and I ran both refresh and clean rake tasks, but no luck. The weird thing is that I’m also noticing the same issue with some users that have custom\_uploads as well. The ones with custom uploads have optimized\_images, but deleting them has no effect either.
I also noticed that the reference that discourse has for the file is not the same URL that I can get from the bucket itself. For example, this the link I get from S3 directly (using their UI):
```plaintext
https://s3-us-west-2.amazonaws.com/thewatercooler/original/2X/d/de05564186252ab863d84df88dc83bfa45b0dd70.jpeg
```
But this is what I get from the Discourse’s database:
```plaintext
//thewatercooler.s3.dualstack.us-east-1.amazonaws.com/original/2X/d/de05564186252ab863d84df88dc83bfa45b0dd70.jpeg
```
But that’s not the S3 endpoint I have set on my settings page (I have the default). I just changed my s3 endpoint to [https://s3-us-west-2.amazonaws.com](https://s3-us-west-2.amazonaws.com) but no luck either.
If you think having some Gravatar images might help, here’s a list of the one that I noticed the issue:
```
https://www.gravatar.com/avatar/a65aa3101e9ae2cae094ff045527cf4f.png?s=360&d=404
https://www.gravatar.com/avatar/b19d019cab25762612252799f44a2b96.png?s=360&d=404
https://www.gravatar.com/avatar/1fc4324f4e1fe24c6587f900ff795229.png?s=360&d=404
https://www.gravatar.com/avatar/186f9af07d20464bae0e5be5ca060a17.png?s=360&d=404
https://www.gravatar.com/avatar/bcf814a93c75a521821ceebb74e6df90.png?s=360&d=404
https://www.gravatar.com/avatar/1face0cac8f8d45e23699ec830c04954.png?s=360&d=404
https://www.gravatar.com/avatar/2822ffe52dff31cb708edd4469704d2f.png?s=360&d=404
https://www.gravatar.com/avatar/bcf814a93c75a521821ceebb74e6df90.png?s=360&d=404
https://www.gravatar.com/avatar/5d04a3641d34189806ac548cf24b282e.png?s=360&d=404
https://www.gravatar.com/avatar/600c032e2bcbe65a04676c33add4b8d2.png?s=360&d=404
https://www.gravatar.com/avatar/add7942f5530b8d9a3bb06f9aac1baf6.png?s=360&d=404
```
---
### Author:  [@tgxworld](https://meta.discourse.org/u/tgxworld)
#### Post date: [October 17, 2018, 1:31am UTC](https://meta.discourse.org/t/problems-with-avatar-uploads-due-to-s3-changes/99400/15 "2018-10-17T01:31:51Z")
Did you happen to reconfigure `SiteSetting.s3_region` recently? It looks like that site setting has been configured to be `us-east-1` when it should be `us-west-2`.
[https://thewatercooler.s3.dualstack.us-west-2.amazonaws.com/original/2X/d/de05564186252ab863d84df88dc83bfa45b0dd70.jpeg](https://thewatercooler.s3.dualstack.us-west-2.amazonaws.com/original/2X/d/de05564186252ab863d84df88dc83bfa45b0dd70.jpeg) works while [https://thewatercooler.s3.dualstack.us-east-1.amazonaws.com/original/2X/d/de05564186252ab863d84df88dc83bfa45b0dd70.jpeg](https://thewatercooler.s3.dualstack.us-east-1.amazonaws.com/original/2X/d/de05564186252ab863d84df88dc83bfa45b0dd70.jpeg) doesn’t.
---
### Author:  [@Daniel\_Lopes](https://meta.discourse.org/u/Daniel_Lopes)
#### Post date: [October 17, 2018, 6:53am UTC](https://meta.discourse.org/t/problems-with-avatar-uploads-due-to-s3-changes/99400/16 "2018-10-17T06:53:51Z")
Yeah, I had to change after one of the last updates (I explained what happened here [Avatar upload failing](https://meta.discourse.org/t/avatar-upload-failing/96880))
I’m gonna write a rake task to update all the images. I wasn’t sure if changing the region could have affected it since it always worked before the other thread.
I will post my gist with my script to update the images in a bit to maybe help others in the future.
Thanks a lot!
---
### Author:  [@tgxworld](https://meta.discourse.org/u/tgxworld)
#### Post date: [October 17, 2018, 7:06am UTC](https://meta.discourse.org/t/problems-with-avatar-uploads-due-to-s3-changes/99400/17 "2018-10-17T07:06:49Z")
Does that mean the region was incorrectly set initially before it was corrected? You can remap the URLs with `bundle exec script/discourse remap //thewatercooler.s3.dualstack.us-east-1.amazonaws.com //thewatercooler.s3.dualstack.us-west-2.amazonaws.com`. Please take a backup before running that in case it doesn’t remap correctly.
---
### Author:  [@Daniel\_Lopes](https://meta.discourse.org/u/Daniel_Lopes)
#### Post date: [October 17, 2018, 7:12am UTC](https://meta.discourse.org/t/problems-with-avatar-uploads-due-to-s3-changes/99400/18 "2018-10-17T07:12:19Z")
> [@tgxworld](#):
>
> bundle exec script/discourse remap [//thewatercooler.s3.dualstack.us-east-1.amazonaws.com](https://thewatercooler.s3.dualstack.us-east-1.amazonaws.com) [//thewatercooler.s3.dualstack.us-west-2.amazonaws.com](https://thewatercooler.s3.dualstack.us-west-2.amazonaws.com)
Awesome, will run the script now. I don’t understand what happened. I never touched the bucket settings on Discourse and my bucket was always west. For some reason it worked before. What I’m noticing is that images stored under 1x don’t use the dualstack url format and all my old uploads are 1x. Don’t know if something changed in the code recently that could have affected that.
Here’s an example:
```
"//thewatercooler.s3.amazonaws.com/original/1X/03eb07a5e93b9e2cb80c63d3fd86a310fb18968c.png"
"//thewatercooler.s3.amazonaws.com/original/1X/073fe33269897de4d0d7973b7f43e0c61a355f4f.png"
"//thewatercooler.s3.amazonaws.com/original/1X/8a34f55c0451703d672181f46cc9c9c4e8b9b457.png"
"//thewatercooler.s3.amazonaws.com/original/1X/9ee647f24b1d3d32653facb9da5d56469d22b6f9.jpeg"
"//thewatercooler.s3.amazonaws.com/original/1X/6f96e830f8f68e1b3232b33934569707f5406b1b.jpg"
"//thewatercooler.s3.dualstack.us-east-1.amazonaws.com/original/2X/8/868a566f2f56afe1a3e6fd16997e5a021a535a8b.jpeg"
"//thewatercooler.s3.dualstack.us-east-1.amazonaws.com/original/2X/9/9f175016fdd217566c4803d45bc8f99d996fe9b9.jpeg"
"//thewatercooler.s3.dualstack.us-east-1.amazonaws.com/original/2X/c/c3cd9870fa16a4875583183b833462cc8b1d40ff.jpeg"
"//thewatercooler.s3.dualstack.us-east-1.amazonaws.com/original/2X/4/40ef5880701b79c2968a284840925e632b4a9ad2.jpeg"
"//thewatercooler.s3.dualstack.us-east-1.amazonaws.com/original/2X/c/c90759b41f603fd18b75a41221850d40f0245ea9.jpeg"
```
All the 1X urls work independent of the region but the 2X ones have the region on the url.
---
### Author:  [@tgxworld](https://meta.discourse.org/u/tgxworld)
#### Post date: [October 17, 2018, 7:26am UTC](https://meta.discourse.org/t/problems-with-avatar-uploads-due-to-s3-changes/99400/19 "2018-10-17T07:26:26Z")
Does `OnceoffLog.where(job_name: "CorrectMissingDualstackUrls")` return anything for you?
---
### Author:  [@Daniel\_Lopes](https://meta.discourse.org/u/Daniel_Lopes)
#### Post date: [October 17, 2018, 7:31am UTC](https://meta.discourse.org/t/problems-with-avatar-uploads-due-to-s3-changes/99400/20 "2018-10-17T07:31:15Z")
> [@tgxworld](#):
>
> `OnceoffLog.where(job_name: "CorrectMissingDualstackUrls")`
I just ran Remap, but checking OnceOfflLog shows this:
```plaintext
[#]
```
---
### Author:  [@Daniel\_Lopes](https://meta.discourse.org/u/Daniel_Lopes)
#### Post date: [October 17, 2018, 7:55am UTC](https://meta.discourse.org/t/problems-with-avatar-uploads-due-to-s3-changes/99400/21 "2018-10-17T07:55:56Z")
Running Remap and the rake tasks to refresh and clean the avatars solved the problem.
Thank you so much for all the help!!
[Next page](https://meta.discourse.org/t/problems-with-avatar-uploads-due-to-s3-changes/99400.md?page=2)