White_listed_spam_host_domains setting not being honoured for New Users?

[Richie]

I’m not sure if this is a bug or if I’m missing a setting somewhere.

The following four items landed in my Moderators Inbox earlier today:

Clicking on the bottom one in the list, New user martincook posts blocked due to repeated links, says:

This is an automated message.

The new user martincook tried to create multiple posts with links to dronescene.co.uk, www.youtube.com, but those posts were blocked to avoid spam. The user is still able to create new posts that do not link to dronescene.co.uk, www.youtube.com.

Please review the user.

This can be modified via the newuser_spam_host_threshold and white_listed_spam_host_domains site settings. Consider adding dronescene.co.uk, www.youtube.com to the whitelist if they should be exempt.

Which is fine.

However, if I click on the top post in the list, Llyn Y Fan Fach Reservoir & Drive 21 Sept 2019” was flagged as spam the message contains:

Llyn Y Fan Fach Reservoir & Drive 21 Sept 2019 - Videos by Drone - Grey Arrows Drone Club UK

This new user tried to create multiple posts with links to the same domain (www.youtube.com). See the newuser_spam_host_threshold site setting.

Each of the other messages also referenced the youtube domain as being the culprit.

What’s confusing me here is that I have already whitelisted the youtube domain.

And various sub-domains for it too:

The other mentioned setting, that of newuser_spam_host_threshold is currently set to 3:

Which incidentally, is how many YouTube links this new user posted before all these notifications arrived in the Moderators Inbox.

If it helps, the new user made the following posts at these times:

• 08:21 post contained a link to:
  – https://www.youtube.com/watch?v=5SE0AQl7VI8

• 08:56 post contained links to:
  – https://dronescene.co.uk/?place=513
  – https://www.youtube.com/watch?v=LL1bdlr23qg

• 09:06 post contained links to:
  – https://dronescene.co.uk/?place=515
  – https://www.youtube.com/watch?v=5SE0AQl7VI8

As you can see, they posted 3 links to www.youtube.com and 2 links to dronescene.co.uk so 5 links in total, across three posts.

Is this whitelist not being honoured?

Am I missing another setting somewhere that may be related?

Have I misunderstood how this works?

Any other suggestions?

I’m happy to spend some time trying to repro over on try.discourse.org if someone can let me know if any domains are whitelisted over there and what that newuser_spam_host_threshold value is set to.

I’m running Discourse v2.4.0.beta5

[codinghorror (Jeff Atwood)]

Can we repro this for new users @tshenry? The whitelisting should work.

[tshenry (Taylor)]

Ok, below are my findings using the same site settings.

I made three posts that contained the links that were listed in the OP. All three went through without issue, so it was not the YouTube links that triggered the user’s posts to enter the review queue. The whitelisting worked.

I tried posting one more dronescene.co.uk link and was presented with a modal:

Dismissing the message returns the user to the composer. They will see a greyed out version of their post. It doesn’t actually get posted, nor does it get sent to the review queue.

Abandoning the post and refreshing the page gets rid of the greyed out post.

The moment the user received the “Sorry, you can’t post a link to that host” message, it appears all of the user’s posts that contain links get sent to the review queue to make sure they aren’t spamming various links throughout the site.

There is some questionable behavior here in the messaging of the reviewable item:

This new user tried to create multiple posts with links to the same domain (www.youtube.com). See the newuser_spam_host_threshold site setting.

It’s pointing out the www.youtube.com, when it should be pointing out dronescene.co.uk instead as that link was not whitelisted and was what resulted in the user being blocked from submitting their post.

I tried the whole test a second time, adding a additional post with a single link included to wikipedia (no dronescene.co.uk included). The post was also included in the review queue with the message:

This new user tried to create multiple posts with links to the same domain (en.wikipedia.org). See the newuser_spam_host_threshold site setting.

As you can see it’s just using whatever link is included, (probably last posted when multiple links are included). This is definitely a little confusing since the user only posted a wikipedia link once.

Based on the current behavior, the copy might be more accurate as:

This new user tried to create multiple posts with links to the same domain (offending-domain.com). All posts from this user containing links should be reviewed. See the newuser_spam_host_threshold site setting.

[Richie]

@tshenry Thank you, sincerely, for the amount of time you’ve spent trying to repro this, I really appreciate it

It’s pointing out the www.youtube.com , when it should be pointing out dronescene.co.uk instead as that link was not whitelisted and was what resulted in the user being blocked from submitting their post.

Interesting!

And yes, would also explain our confusion as YouTube was already whitelisted on our Discourse.

Based on the current behavior, the copy might be more accurate as:

This new user tried to create multiple posts with links to the same domain ( offending-domain.com ).

Would that present us with the actual offending domain name too? Or would it use the last domain name they used, regardless of its whitelist status?

[tshenry (Taylor)]

Would that present us with the actual offending domain name too? Or would it use the last domain name they used, regardless of its whitelist status?

That section of text is used to describe why a post ended up in the review queue, so my initial feeling is to only include the one offending domain that was responsible for flagging all of the user’s posts that contained links. The individual links can be reviewed within the post itself.

I’m curious if @eviltrout has thoughts on all this?

[Richie]

my initial feeling is to only include the one offending domain that was responsible for flagging

That would be ideal.

It’d solve all confusion issues too

[eviltrout (Robin Ward)]

It does seem like a bug if the wrong domain is showing up there. I suspect it is somewhat tricky for the code to know which one it is which is why the wrong one is showing up.

Still, we should fix it at some point.

[Richie]

Maybe adjust the copy in the meantime, to save future confusion?

[codinghorror (Jeff Atwood)]

Yes can you adjust the copy @tshenry for now? Let’s get this to a resolved state.

[tshenry (Taylor)]

Alright, this has been adjusted with the following:

https://github.com/discourse/discourse/pull/8203

The description for this type of reviewable will now say:

This new user tried to create multiple posts with links to the same domain. All posts from this user that include links should be reviewed. See the newuser_spam_host_threshold site setting.

[Richie]

Great stuff, thanks again for your time on this one @tshenry