Avatar images and posted images


(Victoria Hedenmalm) #1

Hello!

We migrated our forum to docker about two weeks ago and set it up with HTTPS.

Yesterday I noticed that we are having some trouble with images in threads not showing up and avatar images as well. We are using aws with ss3 and tried fixing the first issue by setting up a bucket with no forbidden characters. For some reason the image url when inspecting is still the same as before and the image does not load. I have tried rake posts:rebake and rebuilt as well but it does not seem to help.

I found a solution to the above problem with posted images in this thread.

As for the avatar issue I tried to do a git pull and rake avatars:refresh as well as clean. The images are still missing. I get this warning:

convert: delegate library support not built-in `/tmp/gravatar20160627-22484-y43mul.png' (LCMS) @ warning/profile.c/ProfileImage/814.

I have also tried enabling LCMS and that did nothing. All users (except one, the one below, and lettered avatar images) who previously had a profile pic now look like this:

<img src="//discourse-meta.s3-us-west-1.amazonaws.com/original/3X/7/5/754166bce894c532359421642679f5b1445e0ea6.png" width="325"height=“54”>

In a thread the avatar is looking okay, when sized 45x45 px. I have also tried to destroy optimized images in rails and then rebake, which I suppose is included in clean and refresh. Also not working.

I don’t know if these issues are related or not but I figured they might be so I put both in the same thread. I have now run out of ideas to try to fix these issues. Does anyone have any suggestions?

Regards,
Victoria


Broken picture after changing hostname
(fearlessfrog) #2

We’ve started to get the same. Lot’s of our users avatars at various sizes are now going grey/not found.

Is this just the case that imagemagick is unwell with Discourse?


(Sam Saffron) #3

Can you upload some of the problem images here so we can test them?

Can you confirm you are on the latest image?


(fearlessfrog) #4

Logs of output for us:

rake avatars:refresh
Refreshing avatars

convert: delegate library support not built-in `/tmp/gravatar20160708-25792-1nc2zb1.png' (LCMS) @ warning/profile.c/ProfileImage/814.
convert: delegate library support not built-in `/tmp/gravatar20160708-25792-1j76w66.png' (LCMS) @ warning/profile.c/ProfileImage/814.
convert: delegate library support not built-in `/tmp/gravatar20160708-25792-678mia.png' (LCMS) @ warning/profile.c/ProfileImage/814.
convert: delegate library support not built-in `/tmp/gravatar20160708-25792-1j9ki6s.png' (LCMS) @ warning/profile.c/ProfileImage/814.
convert: delegate library support not built-in `/tmp/gravatar20160708-25792-cdgype.png' (LCMS) @ warning/profile.c/ProfileImage/814.

(repeats about 100 times at least…)

Looks related to an issue to tweak IM here from @sam and closed by @codinghorror

https://github.com/discourse/discourse_docker/issues/224


(Sam Saffron) #5

Possibly, I just need the failing images :slight_smile: A bug I cannot repro is :crab: to fix.


(fearlessfrog) #6

It’s spooky that both us and @victoria moved to S3 recently (spooky music). :ghost:

Ok, env and a test case:

docker_manager (e215aab) Up to date
with v1.6.0.beta10 +9

A broken user with gravatar here:

puts u
email: "beachav8r@mudspike.com",
https://www.gravatar.com/avatar/71319852e571324ae83cd56878add78a

Broken optimized image: (click through, as your discourse downloading local might make it work)
https://forums.mudspike.com/user_avatar/forums.mudspike.com/beachav8r/45/6_1.png

A good optimized image (same user) just different size:
https://forums.mudspike.com/user_avatar/forums.mudspike.com/beachav8r/25/6_1.png

Let me know what else would be useful. I’m going to start to dig through user_avatar.rb upwards too.


(fearlessfrog) #7

Looking at this from the issue on github:

I don’t think the syscall on the convert failing would bubble up the exception and stop anything anyway?

If it is working fine for you guys, then my best guess is something related to S3 as the target, although I think (not clear, just started looking) that the avatar optimize stuff gets all treated to local storage anyway?


(fearlessfrog) #8

I’m experimenting with including lcms and lcms2 in the im6 build. The image wget for ImageMagick will need updating, as this is 404. The next one up past that version is 6.9.5-0.

@sam Could you recreate this with the gravatar info I gave?


(fearlessfrog) #9

I think it’s S3 related. I updated the convert line that optimizes the avatars retrieved from gravatar to not use a color profile and the warning goes away, but the images don’t get pulled through. Our forum looks like this now the CDN cache is drying up:

@zogstrip helped out before when our files filled up, want to take a look? :slight_smile:


(fearlessfrog) #10

So looking in our S3 bucket, is it correct that we just have two top level ‘folder/objects’ of

optimized
original

…in the bucket? There are no _optimized or _emoji folder (or numerical ones, or tombstone etc) like in local storage at this top level.

If anyone uses S3 here (which I’m starting to regret, but we only went to it when a bug caused our disk to fill) could they check please as it might provide us with some clues.


(fearlessfrog) #11

What’s the best way forward here?

Is S3 dropped from being supported, as in we should go back to local storage to be more like how the Discourse team hosts and we’d have less of these types of issues?


(Jeff Atwood) #12

Hard to say – did you want to purchase a hosting contract? It’d be easier for us to support you in great detail if you were a customer.


(fearlessfrog) #13

Understood. We’re not that large a community, so Discourse might not be the right choice for us given the business you guys are trying to build.


(Jeff Atwood) #14

It is currently a weekend, so unless you are a paid customer, you might need to wait for others to have time to get to your free support request. I apologize if this is inconvenient for you.


(fearlessfrog) #15

Thanks for that. I didn’t expect a free reply today.


(Sam Saffron) #16

Try

  1. Rebuilding with latest
  2. Changing one of the problem avatars (editing 1 pixel) and seeing if it corrects said user

Is there anything custom about your setup?


(fearlessfrog) #17

Will do - thanks @sam. I’ll report back.

I’ve traced through the user_avatar/show logic and if anything I think it may be due to the crazy s3 -> Cloudfront -> Cloudflare cache we have set up, as that’s the only custom thing we do and may be at the root of this.

The image rendering of the avatars is deliberately cache header’d, so it looks like the rake task is completing ok (as in I use a console, find a user with a bad avatar user, find their user_avatar object, and then see the upload then the url and origin fields do look ok) but it’s not busting the cache.


(fearlessfrog) #18

Ok, so I did this:

  • Removed the S3 cache setting from the Admin/Files area to have less moving parts. The setting ‘Enable S3 for Uploads’ is still on. Turned off Cloudflare (by putting it into dev mode where all calls go back to origin).

  • git pull, launcher rebuild app. Updated ok. Now on v1.6.0.beta10 +93

  • ./launcher enter app, rake avatars:clean, rake avatars:refresh.

I still get a

'..convert: delegate library support not built-in `/tmp/gravatar20160710-883-1m8auir.png' (LCMS) @ warning/profile.c/ProfileImage/814.' 

…warning per user on the rake avatars:refresh, although I don’t know if that actually stops anything other than ignores the ImageMagick convert color profile.

After doing that, any of our users that use an avatar image from gravatar appear as blank grey icons. It’s about half our users. Users with avatars they uploaded to our forum themselves are ok, as our users with just the default letter avatars. This issue is just with gravatar originated images.

Tracing through a user with a blank optimized avatar, in hope I can Stanford Teddy bear this:

[1] pry(main)> u = User.find_by_username("BeachAV8R")

[2] pry(main)> avatar = u.user_avatar
=> #<UserAvatar:0x007f355cbfcc80
 id: 4,
 user_id: 3,
 custom_upload_id: nil,
 gravatar_upload_id: 32907,
 last_gravatar_download_attempt: Sun, 10 Jul 2016 23:01:18 UTC +00:00,
 created_at: Sun, 17 May 2015 05:27:23 UTC +00:00,
 updated_at: Sun, 10 Jul 2016 23:01:25 UTC +00:00>

[4] pry(main)> upload = Upload.find(32907)
=> #<Upload:0x007f355d19ddb0
 id: 32907,
 user_id: 3,
 original_filename: "gravatar.png",
 filesize: 153856,
 width: 360,
 height: 360,
 url:
  "//mudspikefiles.s3.amazonaws.com/original/3X/3/0/300bdaac902161296c149598be0e2a53f726aa4d.png",
 created_at: Sun, 10 Jul 2016 23:01:24 UTC +00:00,
 updated_at: Sun, 10 Jul 2016 23:01:25 UTC +00:00,
 sha1: "300bdaac902161296c149598be0e2a53f726aa4d",
 origin:
  "http://www.gravatar.com/avatar/71319852e571324ae83cd56878add78a.png?s=360&d=404",
 retain_hours: nil>

So these originals look good from the above user:

FileOnS3

FileOnGravatar

Here’s the optimized links that are broken from that same user above:

https://forums.mudspike.com/user_avatar/forums.mudspike.com/beachav8r/25/6_1.png

BrokeOptimized

So my best guess is still that the combination of S3 and the optimization of the gravatar locally downloaded images with the rake task to refresh them is probably the bug.

Earlier today I also tried turning off S3 and going back to local storage and then re-running the rake avatars:refresh, but as I was using the caches I probably need to go try that again to see if I can narrow this down.

Thanks for any help (not just the discourse team Jeff, I mean just generally anyone :slight_smile: )


(Mittineague) #19

The
URL doesn’t look so good to me

[FileOnS3](http:////mudspikefiles.s3.amazonaws.com/original/3X/3/0/300bdaac902161296c149598be0e2a53f726aa4d.png)

(fearlessfrog) #20

That was just me typing http:// on to the link to save people have to type from the pry output above - thanks, and fixed.