Multiple repeated summary mail entries

[ecki (Bernd)]

My idle test user just received its weekly summary mail and it contains 5 times the same topic (it’s a topic which was edited and pinned a few times, but still it should only show up once, especially if it shows the exact same text not something like “have been pinned” or such.

From the 3 popular posts 2 have been duplicates.

Also the “new for you” table contained a post 3 times:

With that lots of duplication nearly no information was conveyed (even though other posts would have been available)

can I add some db query results to guide the troubleshooting in this bug report?

[ecki (Bernd)]

I also excluded a tag, but the preview function still shows me “Popular Post” answers under a topic with an excluded tag. Not sure if this exclude is not working for “Popular posts” or if just the pre-view function is not recalculating the conditions. (I will see in the next mail on Friday:)

Please let me know how I can troubleshoot that a bit better. Are you otherwise happy with the summary mails? odd that I should be the first one to have such problems.

[bigfudge (Ben)]

I’m having the same issue. I have posts repeated 4 times in the same email, and the email header says “4 new topics”

[ecki (Bernd)]

The problem still exists with stable 3.3.0+3 as I just noticed in one of my notification mails.

[nat (Natalie T)]

Thanks folks, we’ll have a look at this.

[JammyDodger]

Cross-linking another recent case of this -

Duplicate topics in email activity summary

I’m not sure if this is a bug or a problem with my settings. Lately some of the email digests contain duplicate topics. Here’s a screenshot of one of them with the same topic repeated three times in a row. Multiple people are receiving digests with duplicate topics, but not all of them are like this. Does anyone know what it might be? [Discourse duplicate topics in email digest]

[j127]

If you have trouble reproducing it and want more details, let me know. I’m not sure if there is a query log somewhere, but if I go to /admin/email/preview-digest the problem is there in the preview.

[j127]

I’m not sure if I should post this here or in a new topic, but posts with tags that are in the digest suppress tags list are being featured in the email digest too. The main featured topic at the moment has a suppressed tag on it.

[ecki (Bernd)]

yes confirmed was also the case for me.

[ToddZ]

I’m seeing the same behavior: “Popular Posts” and “New for you” topics being repeated.

In this case one Popular Post is tripled, another is doubled:

Self-hosted standard installation here, current at 3.4.0.beta2-dev (f2121cc437).

I see this report dates back to Feb. – anything happening? Any further details I could provide to help?

EDIT: if anything it’s getting worse. I’m now seeing the lead Popular Topic tripled. It’s really not a good look – I’d love some suggestions for pursuing troubleshooting this.

[ToddZ]

@ecki , @bigfudge , @j127 – are you still having this issue? I’m seeing digests go out looking just ridiculous, and I’m not sure how to draw attention to this report.

[j127]

It’s still a problem here.

[ecki (Bernd)]

Yes the problem still exists, I have just upgraded to STABLE * v3.3.1 +5 and the preview mails still show the behavior.

@ToddZ

if anything it’s getting worse

I helped myself by reducing the number of entries in each of the 3 categories (1,1,3), but of course this makes the email less helpful. Its just less embarrassing.

[ToddZ]

Thanks for confirming I’m not alone! It’s frustrating to keep seeing it with my digests, but not in any digests I get from various other Discourse forums.

I read elsewhere that the Team was away last week for a company retreat. Maybe @nat can pop in here as they get caught up on things…

[Moin]

The meetup is this week, so you need a bit more patience

In the meantime, is there anything your forums have in common? A plugin you all use or a setting that all of you have configured differently than the default?

[j127]

At the moment, these extra plugins are enabled:

/discourse-math.git
/discourse-data-explorer.git
/discourse-graphviz
/discourse-perspective-api
/discourse-adplugin.git
/discourse-user-notes.git
/discourse-affiliate.git
/discourse-categories-suppressed.git
/discourse-templates.git

Edit: all of the included plugins are also enabled, like footnote and templates.

Email settings that aren’t default:

• post excerpts in emails – checked
• personal email time window seconds – 60
• suppress digest email after days – 1095
• digest suppress categories – a few categories are here
• digest suppress tags – one private tag (that ends up in the digests anyway)
• email accent bg color – custom color
• log mail processing failures – checked
• delete email logs after days – 10
• mailgun api key – filled out

[ecki (Bernd)]

On my system I am using STABLE Docker with seperate Redis and Data containers.

The following plugins are enabled in the app.yaml:

• docker_manager.git
• discourse-rss-polling.git

And the following in UI

• automation true
• footnote true
• rss polling true
• spoiler alert true

We use a theme cloned from the default theme, created on an older version.

The following is overwritten in email settings:

• contact_email
• blocked_email_domains
• normalize_emails true
• auto approve email domains
• hide email address token: true
• personal email time window seconds: 120
• digest topics: 1 (workaround)
• digest posts: 1 (workaround)
• digest other topics: 3
• supress digest email after days: 600
• digest suppress categories: forum help
• digest suppress tags: rss, test
• show topicsd featured link in digest: true
• email subject: prefix %{optional_pm}%{optional_cat}%{topic_title}
• default email message level: only when away
• default email list mailinglist mode frequency: send en amail for every except own
• inclde thumbnails in backup: true
• embed set canonical url: true

There are no job customisations I know of. We are using a private smtp server, no mailgun api. (But the problem exists before sending its also visible in previews, anyway).

[ToddZ]

Thanks for the update, Moin. Here’s what I have:

Active plugins:

• Automation (w/one action: “notify when user changes group”)
• Footnote
• Spoiler Alert
• Staff Alias

Email settings that aren’t default:

• personal email time window seconds – 20
• suppress digest email after days – 365
• digest suppress categories – 2 categories are here
• digest suppress tags – one private tag (seems to be correctly suppressed from the digest)
• log mail processing failures – checked
• delete email logs after days – 90
• apply custom styles to digest - unchecked
• default include tl0 in digests - checked
• contact email - custom

Email settings that aren’t default but don’t seem relevant to digest:

• reply by email enabled - checked
• manual polling enabled - checked
• email in - checked
• email in allowed groups - some groups specified
• blocked email domains - custom
• unsubscribe via email footer - checked
• disallow reply by email after days - 0
• forwarded emails behaviour - create_replies
• default email mailing list mode frequency - Send an email for every new post except my own
• default email previous replies - unless previously sent

Default theme, with these Active Components:

• Category Headers
• Custom CSS
• Custom Header Links
• Extended About Page
• Unanswered Filter
• User Card Directory
• Welcome Link Banner

Self-hosted standard installation, nothing fancy. Test-passed branch, current at 3.4.0.beta2-dev (f2121cc437).

Using MailerSend.net for SMTP relay, but the problem exists before sending (visible in previews.)

[simon]

It might also be helpful to see details about a specific topic that’s triggering the issue for a given user. For example, to know if the topic has multiple tags, if the user has muted any of those tags, etc.

I’ve gone line by line through the method that’s used to put together the digest email topics: discourse/app/models/topic.rb at main · discourse/discourse · GitHub. I’m not seeing any obvious problems, but the method has multiple join statements. It’s possible there’s a condition that can cause a topic to end up in the list multiple times.

[ToddZ]

Thanks for the interest, Simon!

The digest attached is typical of what I’ve been seeing – though I haven’t analyzed them for tag count before.

It was generated for a user:

• last seen 2024-09-09
• Digest frequency: weekly
• “Include content from new users in summary emails”: checked
• no relevant tags muted

Popular Topics:

“Past SciFi tales…” – repeated 3 times. Topic has 3 tags, 15 replies, 1 (staff) like.
“Citizen Science Fiction” – repeated 2 times. Topic has 2 tags, 2 replies, 1 (staff) like.

New for You:

“Cyborgs and Centaurs…” – repeated 3 times. Topic has 3 tags, 2 replies, 0 likes.