Topic List Previews Theme Component

That’s a good spec! You must have a particularly busy site?

Are you sure there was no rebake occuring?

How many pictures are being posted per minute?

Preparation of Thumbnails is mostly core code and requires mandatory additional image processing.

This would be the case for any plugin or theme component of this nature.

3 Likes

We turned off the plugin for about 24hr, then turned it back on for 24hr.
Below I’m pasting notes from our other admin.


The plugin is causing jobs being scheduled to slow.
The more jobs are in the queue, the more the CPU struggles to keep up.
so what happens is the server sees a lot of traffic, but things seem normal for a while until the jobs start backing up.
Eventually the CPU struggles to keep up and usage spikes up and down as it completes backlogged jobs.

That’s why we saw earlier there was 3000+ backlogged jobs.
under normal circumstances there’s never more than 5 jobs. but almost always there are 0 because no jobs should be backlogged, they should be processed immediately.

Image attached shows the current jobs getting backlogged. Right now it’s hovering 30-35 backlogged jobs.
All the jobs are new from the last post in the sidecar.
I can’t tell why yet but this only happens when the plugin is on.

Last hour of the resources on CPU

last 24 hours, pointing out approx. where the plugin was enabled (See the up trend after the spike)

It seems like the pattern is longer than 24 hours to notice but the plugin causes general resource usage in the 80-90+% usage most of the time.

Once we turn it off, we’ll start noticing that over the next 24 hours the server will average 60-75% and the jobs won’t get backlogged.

Also when we want to, I changed the app.yml to have 16 unicorn workers. when we want to rebuild I think we should disable the plugin, and have 16 unicorn workers. We observe the server’s performance over that period of time, and adjust the workers value to what we think is best.

Overthe last 7 days
red = enabled
blue = disabled

After turning on the plugin again, the CPU is spiking. I’m less confident in the jobs part being a major thing. I noticed the job counts go up with more unicorn workers. I think the number is unrelated.

I do still see the issue about the plugin holding up resources though.

Still not 100% on it but im very confident the plugin is an issue.

plugin + 16 workers: pins the server
16 workers no plugin - working well
plugin + 8 workers - sluggish but it works

graph after turning off the plugin

1 Like

Hi,

I’m the other admin that made those notes.
I don’t see the jobs as being part of the issue any more.
unfortunately I’m not knowledgable about discourses internals very much to know what could be the cause. I just know what I see currently.

It can be summarized as the site is generally sluggish when enabled with 8 workers and nearly goes down with 16 workers
With the plugin off the site works perfectly fine and with enough workers it’s very fast.

This leads me to believe there is something with the plugin that is slowing the site either by resources or something being held up in IO or async operations perhaps.

1 Like

Please have a look at your sidekiq queue. Do you see lots of jobs queued and what are the job names?

I’m sure it’s just a backlog of thumbnail production which will settle down once all images have been processed.

That’s a core process.

You can safely remove the sidecar plugin if you don’t need its additional features and just use the Theme Component.

Can you confirm the issue persists with the Theme Component active but not the sidecar plugin?

If you’ve never had the sidecar plugin installed then the issue must be in core.

If you need help with your analysis you can hire Pavilion.

I welcome independent reports of this issue.

4 Likes

Just confirming - an increase in high-CPU image-resizing jobs is expected when you enable any kind of topic-list thumbnailing theme component :+1:. Core generates the thumbnails ‘on demand’ for the first user to view a given topic in a topic list.

As @merefield mentioned, it should settle down once all thumbnails have been generated for frequently-listed topics.

Increasing the number of workers is probably not a good idea - trying to do more work in parallel on a CPU-constrained machine will make the symptoms worse. Better to keep the worker numbers as normal so that the jobs can queue up and be processed without overloading the machine.

In the sidekiq UI, you probably want to be looking at the “Busy” and “Queued” tabs. “Scheduled” jobs are the ones scheduled for a specific time in the future, and so they are very unlikely to be causing performance issues.

4 Likes

It looks great on preview, but in reality it’s completely different. What am I doing wrong?

Theme: FKB Pro - Social theme - #268 by Don

1 Like

Many “total conversion” themes are not supported. I suspect FKB Pro has its own overrides that clash with this component. I’m afraid I can’t support this scenario.

This Theme Component is meant to be used as a building block within a theme that doesn’t try to cover all the bases.

If you want a right sidebar with Topic List Previews, try Right Side Blocks in combo.

3 Likes

There is small CSS/spacing issue between topic header and featured link icon thingy.

2 Likes

Added an experimental feature:

1 Like

This is working quite well now, still behind a setting:

This is responsive to width, so only appears in Tiles (masonry) mode when the list area is wide enough.

Demo here: https://www.starzen.space/

NB The excess padding on the bottom right is a necessary known issue due to the masonry renderer which for performance reasons (it’s fast!) renders things to the nearest grid row. Once (!) they implement native masonry in CSS grid I’ll refactor this to make it nicer.

NB#2 The screenshot also contains some controls from Discourse Bars theme component (the sidebar is hidden). Topic List Previews works really well with Discourse Bars.

1 Like

An image for YouTube appears on the home page.

Could it be for Twitter too?

2 Likes

It is not in any way restricted to youtube.

It exposes whatever the thumbnail is. If the Twitter embed creates a thumbnail, then yes.

3 Likes

Could you please tell me what settings I should choose to make it like in this picture?

I want it to be in the form of a list of thumbnails like in this picture.

image left aligned

55

1 Like

Remove routes from this setting:

Otherwise they will show as “Tiles”. If you want zero routes as “Tiles” format, remove all of these lines, but you might want to consider Tiles on mobile (in which case keep those with *-mobile).

2 Likes

It’s perfect now, thank you for that :pray:

2 Likes

Fonts come out too much, how do I reduce the words?

1 Like

There’s a setting in the accompanying sidecar plugin (listed in the OP):

image

But without the plugin you might want to try the native settings:

I haven’t looked at this in a while! :sweat_smile:

1 Like

I’m playing with the settings you mentioned, but it doesn’t work.

Interestingly, there are too few summaries on one topic and too much on the other.

The character must be within a certain character limit.

88

1 Like

Install the sidecar plugin, use that setting.

2 Likes

I would guess that you have to fire a post rebake to redo the topic summary.

1 Like