Embedding, but don't automatically create a topic until a user chooses to comment

[avorio]

@eviltrout

Do you actually have millions of blog posts that would be created as topics? Because that would definitely create a lot of data and performance would be a concern.

Yes, we do indeed have about 2 million articles that could receive comments.

That’s right — besides the UX problem discussed above, there’s clearly a performance concern here, too.

[eviltrout (Robin Ward)]

It’s unfortunately a somewhat unique situation for us. Most large blogs we’ve attached comments to have several thousand articles, not millions.

I imagine we could do it like this:

• Instead of linking to the topic, the first commenter would link to a new “create topic” route.

• It would display text such as “You are the first to comment on this article” and the composer would pop up.

• Upon submitting the post, they would see a spinner until the topic is imported with the post as a reply. We’d have to use the message bus or other process to be notified when the post goes through.

I think this could work within our constraints, but would be at least a day or two of work for us. I unfortunately can’t see us prioritizing this any time soon as the typical use case is fine for most people who don’t have millions of articles.

[elSpectre (Neil Baptista)]

We might fall into the somewhat unique category as well, but would definitely hugely benefit from this flow, as we’ll have millions of potential topics (i.e. books).

We’re using Discourse for our forum already (amazing stuff!) and have been wrangling with how to do comments on our site. We implemented Disqus but there were a lot of shortcomings, so we’ve backed that out, started to think about rolling our own and then this update happened.

Hoping to keep our community all connected within Discourse by adding discussions to books and books lists using this approach. But, we wouldn’t want to create a topic for every book in our database, only needing topics as the discussions start.

[avorio]

@elSpectre

We might fall into the somewhat unique category as well, but would definitely hugely benefit from this flow, as we’ll have millions of potential topics (i.e. books).

We’re using Discourse for our forum already (amazing stuff!) and have been wrangling with how to do comments on our site. We implemented Disqus but there were a lot of shortcomings, so we’ve backed that out, started to think about rolling our own and then this update happened.

Hoping to keep our community all connected within Discourse by adding discussions to books and books lists using this approach. But, we wouldn’t want to create a topic for every book in our database, only needing topics as the discussions start.

I’m happy to know that you’d benefit from this approach too, @elSpectre!

What are your thoughts on the specs?

[elSpectre (Neil Baptista)]

@eviltrout

I imagine we could do it like this:

Instead of linking to the topic, the first commenter would link to a new “create topic” route.
It would display text such as “You are the first to comment on this article” and the composer would pop up.
Upon submitting the post, they would see a spinner until the topic is imported with the post as a reply. We’d have to use the message bus or other process to be notified when the post goes through.

I think this would work for us perfectly. Then, the only concern would be how is performance once we have 100K plus Topics created. I really like the composer from Discourse so having that pop-up or ideally, embed on our page would be great. The perfect solution would be that we could have an embedded composer that simulates in-place commenting like Disqus. We tried integrating their service, but it didn’t go well, as they want to take users on their own sign-up path to become global Disqus users.

[Falco]

@eviltrout

I imagine we could do it like this:

Instead of linking to the topic, the first commenter would link to a new “create topic” route.
It would display text such as “You are the first to comment on this article” and the composer would pop up.
Upon submitting the post, they would see a spinner until the topic is imported with the post as a reply. We’d have to use the message bus or other process to be notified when the post goes through.

Hey eviltrout thanks for the attack plan.

I’m arguing to move my current in-house comment system to discourse, and this is what I need to get the mind share so Discourse wins.

Do you have any other points to highlight about this feature? I’ll try to implement this when we start evaluation of the comment system again (probably on March / April).

[avorio]

Keep us posted of your progress, @Falco!

[eviltrout (Robin Ward)]

Not that I can think of!

It’s quite a big amount of work so take your time

[avorio]

Hey @Falco! Any update so far?

[Heather_Miller]

For what it’s worth, we were plagued with this issue – flood of new topics when people visit our pages due to this behavior of embedded Discourse. It’s caused us lots of problems.

We made a bug report here:

Flood of new topics from Embedded Discourse

We’re an open source software project with a lot of documentation pages. On each of our documentation pages, we decided to replace Disqus for commenting with (embedded) Discourse. After all, we already have a community of users using Discourse – the idea of merging the two places beginners ask questions seemed like a great idea. However, it seems that whenever someone reloads one of our documentation pages (without actually attempting to comment – they’re just visiting the pages…) a new Discour…

Our bug report was closed and redirected here. (For the record, this behavior seems to us like more of a bug than a feature.)

[sam (Sam Saffron)]

@Heather_Miller

(For the record, this behavior seems to us like more of a bug than a feature.)

Its certainly buggy under your use case.

I am open to changing this so the comment link goes to an interstitial webpage, which would initiate the crawlback if needed and then take you to the topic.

So:

• User clicks “http://discourse.mysite.com/comments?url=https://mainsite.com&user-click=true” : note user-click=true must be appended via JavaScript to avoid crawlers creating topic.

• If the Discourse topic already exists, discourse redirects to it.

• If the Discourse topic does not exist (and user-click is true), initiate crawlback, if successful and topic is created, redirect to it.

There would be a possible few second delay on “show comments” click (we could display a glorious hamster on a treadmill creating the topic or something along those lines when stuff is held up), JavaScript crawlers could mess stuff up (though they are quite rare and the /comments path or whatever should be excluded in robots.txt)

@eviltrout

I think this could work within our constraints, but would be at least a day or two of work for us. I unfortunately can’t see us prioritizing this any time soon as the typical use case is fine for most people who don’t have millions of articles.

I agree, as a team we just can’t prioritize this at the moment, there is about 1-2 days work in nutting out the change I proposed bit I think my “mini” spec is sound (@eviltrout ? )

I am putting pr-welcome on this in case the community feels like contributing.

I also am not sure we need a “switch” for this mode, it should just be the default behavior, it is far safer. (maybe if people need to know for sure the topic exists on discourse they could turn off the new behavior)

[eviltrout (Robin Ward)]

It could work. There are things to figure out with regard to whether the user is logged in or not. For example if the user is not logged in should they be able to trigger the topic creation? But it’s still not that hard, it just has to be done.

[sam (Sam Saffron)]

@eviltrout

For example if the user is not logged in should they be able to trigger the topic creation?

We could display a “login to comment” screen there, and finally redirect back to http://discourse.mysite.com/comments?url=https://mainsite.com&user-click=true after login. It is workable, but yeah, anon requires more jumps.

[Stephen]

We encountered the excess post load problem, but the other way around. Having WordPress poll ~15k posts for updated comment counts via the plugin caused crazy scaling issues on the remote host.

That said, we found value in having the stubs in Discourse, and they didn’t create substantial noise as the category article comments fell into was suppressed from the front page using the ‘Suppress this category from the homepage’ option. The titles were usually descriptive enough that search would find the stubs, and direct the users back to pages on the content site.

@sam

it should just be the default behavior, it is far safer

It may be that for knowledge sites this setting is preferable, but for content entities the stubs almost certainly have value, particularly when the plugin embeds the excerpt and link back to WP. @simon has some approaches in mind to tackle the issue of load, at least on the WordPress side.

[simon]

@steve_pd

@Simon_Cossar has some approaches in mind to tackle the issue of load

Thanks! I’m working on that right now.

[Martijn_Hoekstra]

I really appreciate that we’ve gotten to the point where the behaviour is acknowledged as a problem and something to be fixed , and isn’t trivialized as that’s just how Discourse works, and if you don’t like it you’re the problem

I can imagine this issue mostly beeing a bootstrapping issue, and that having stubs created when a post is first read is the desirable behaviour in the “steady state” where either Discourse has been enabled from the start or all posts already have stubs, so it doesn’t flood the interface (whether mailing list mode or normal), but stub threads for new posts are shown on the front page.

I wouldn’t immediately have a suggestion for the most desirable behaviour, but it’s good to see people are thinking towards solutions now.

[codinghorror (Jeff Atwood)]

@Martijn_Hoekstra

if you don’t like it you’re the problem

More along the lines of, if you don’t like it, pay us as a customer to fund the work, because the current behavior works for a LOT of sites.

[hnaseri (Hosein Naseri)]

@codinghorror

pay us as a customer to fund the work

How is this possible? Maybe if there was a fundrising system or something like that to fund some of the feature requests, that would have been a great Idea.

[codinghorror (Jeff Atwood)]

Pretty easy, go to https://discourse.org/buy and push the big buttons!

[tobiaseigen]

I just had another idea. What if there were a category setting to hide topics that have no replies? They would not show up anywhere until there is a reply. Then the embed code could place the new topics in that category as it does now but no spamming would occur. Once someone clicks through to add a reply the topic would show up as usual.

This us not an answer to the use case with tens of thousands of posts but works fine for most sites.