Our image uploads seem to be not showing the lightbox expansion, so that when you click them you see a higher resolution image. This has happened after updating to .beta8 yesterday. Test image, as curious to see if the same here: [IMG_3695] Our settings (that worked ok before). [image] Any ideas of things to try or to get more diagnostics? For topics with these images there are no browser console errors or /admin log ember errors.

[image] fearlessfrog: Is there something in the beta6 to beta8 that recooks all posts or something? Yep, see Discourse 2.2.0.beta8 Release Notes [image]Discourse 2.2.0.beta8 Release Notes Important Note: Expect high CPU usage for a few hours after this upgrade due to image re-optimization. Sites with large numbers of images may see high CPU usage for a longer period of time.

It worked here, so I’ll see if our sidekiq is poorly with a /sidekiq/queues?poll=true

Is there something in the beta6 to beta8 that recooks all posts or something? We have about 3,500 enqueued Jobs:ProcessPosts being done at the moment (with only 5 busy), which would explain why it is so behind. I guess we wait?

Ah, got it - thanks! I should have looked there first. :blush: We’re a busy forum on an undersized box, so it’s taking a while to catch up.

If it helps anyone else, we’ve found that for our hardware of a dual proc DigitalOcean with 4GB seems to fall behind with a default ‘rebake old posts’ set to 80. We’ve been going for a few years and have a lot of image intensive posts, so YMMV. Our sidekiq was clear of PostProcess jobs about 12 hours ago, but it’s crept up to about 4,500 enqueued again this morning, causing delays in new post processing. I’ve set the rebake_old_posts_count value to ‘20’ and it seems to be slowly making headway now. The setting can be found via admin here: admin/site_settings/category/all_results?filter=rebake%20old @sam even though this is an unusual rebake, being especially CPU intensive (and I imagine some S3 back/forth as well), is there a way that this rebake value could be backed off if sidekiq is backlogged so much? I’m not meaning anything too complex an algorithm, just something that could automatically half it if the system is increasingly getting behind?

Yeah we can probably skip queueing if more than N jobs are in the low queue

I know you guys are (rightfully) hosting tech-agnostic with Discourse, but the ideal for AWS S3 image hosted people would be moving the image processing AWS Lambda side. I’ve done that on Rails systems when it was javascript and imageMagick (moving it away from background work on the Rails side) and it works pretty well (as S3 can trigger a function on ObjectCreate completely on the AWS side, build a hierarchy of resized images etc). Theoretically :slight_smile: it’s got even easier now with AWS Lambda Ruby official support. You might be able to use the same image processing code on the Lamda side. The downside would be the AWS config API cruft to set up the trigger events etc. Not that great an idea, but I thought I’d throw that out there for long term chewing.

Yeah, it is an idea I have thought about in the past but we would need a custom image for this do line up all the right versions of image magick and so on. We may get there one day but odds are this is going to be in the “mega enterprise” type of setup.

In theory you can run an AWS AutoScalingGroup that runs the Discourse image, with 0 unicorns, some sidekiqs, and autoscale based in Sidekiq queue size. We almost did this for a customer, but managed another cheaper way. Running stuff with the same base image instead of a custom lambda setup (that adheres to the lambda function size restriction) allows us to have only one code path to keep working, instead of two different ways to do the same, debug, update, keep in sync, etc.

[image] Falco: We almost did this for a customer, but managed another cheaper way. I think pretty much anything else is cheaper than AWS, but you know how the saying has evolved - ‘No-one got fired for picking IBM, Microsoft, AWS’. :slight_smile: In my use of Lambda on S3 images, we just used the base AWS Lambda node.js image and a const gm = require('gm').subClass({ imageMagick: true }); ..and it fitted what we needed, which was async processing of uploaded or edited images, but with processing just on the AWS side. It allowed us to go browser → S3 direct, and then by naming convention (or object path in terms of S3) we could rely on the optimization/resizes used by convert without worrying the Rails server side queues with work. I think what you do with Discourse needs to cover much more feature points, and I agree with you that duplicating that side to support those with, or without, AWS would be a dev pain, and probably not worth it. I just like the general approach as it is one of the few genuine approaches where a Lambda function platform could be useful for surged scale stuff.

[image] fearlessfrog: I think what you do with Discourse needs to cover much more feature points, and I agree with you that duplicating that side to support those with, or without, AWS would be a dev pain, and probably not worth it. Yep the tradeoff is exactly this. We would be building a new path for 0.01% of the sites, that would increase dev complexity by 100%. Something we may pursue in custom plugins for [image] sam: the “mega enterprise” type of setup. [image] fearlessfrog: one of the few genuine approaches where a Lambda function platform could be useful for surged scale stuff. I was toying with a project for this but for doing videos. Transcoding those is out of reach for most VPSs out there, and a more acceptable trade off, for a way better user experience, with multiple resolutions, codecs, etc.

Image lightbox stopped appearing on beta 8 update

Product Support

jomaxro (Joshua Rosenfeld) January 15, 2019, 8:49pm 4

Yep, see Discourse 2.2.0.beta8 Release Notes

Topic		Replies	Views
User profile images loading slowly, failing to load Product Support	30	2467	February 23, 2020
My journey into a massive posts rebake job Self-hosting	35	5228	July 10, 2024
Images are disappearing off of s3! Bug	56	6158	September 20, 2018
"convert" process eating CPU Self-hosting	14	2669	March 7, 2019
Rebake after CDN change missed old posts Product Support	2	605	March 7, 2021

Image lightbox stopped appearing on beta 8 update

Related topics