Embed blacklist selector behavior

probus · July 30, 2015, 9:13pm

When embedding discourse comments in static pages the blacklist/whitelist css functionality is broken.

The blacklist only seems to work initially if the discussion topic is created during the first user visit to the static page. If you wait for the topic to be created from the rss feed the blacklist/whitelist doesn’t have any effect. Even if you get the correct content in the opening post by visiting the static page first, after an hour the opening post updates itself and reverts to a state like it would have been with empty blacklist/whitelist.

It seems when the system checks for updates to the static page the whitelist/blacklist gets bypassed.

Steps to reproduce:

Embed Discourse comments in your blog
Write “p” in settings>embed blacklist selector
Write a blog post and don’t visit the page
Wait a few minutes for the comment topic to get created from rss feed
The opening post has full contents of (every p element in) the page and blacklist doesn’t have any effect

Or

Embed Discourse comments in your blog
Write “p” in settings>embed blacklist selector
Write a blog post and visit the page
Notice a comment topic with mostly empty first post being created immediately like it should
Wait an hour
Watch the topic opening post update itself with again all the content from the static page

edit. updated with more information

codinghorror · March 24, 2015, 5:04am

This is @sam’s feature and is highly specific to embedding and the Show Full Post button, at least, I think that is what you are referring to.

probus · March 24, 2015, 9:02am

I haven’t tested if the Show Full Post button/setting has any effect. After quick testing, it doesn’t seem to.

Basically I’m saying the whitelist/blacklist is bust. You don’t have any way to influence what gets picked up on the opening post of the comment topic. It will always revert to default behaviour. It’s impossible to have, as an example, empty first post with no blog content in it.

After playing around with the Show Full Post button enabled, it seems like the fist paragraph that is initially shown is from the unfiltered contents and after you click it, it shows the white/blacklisted version. Weird.

riking · March 25, 2015, 3:12am

Ah! So you just need to get a smarter excerpt.

probus · March 25, 2015, 10:45am

I don’t understand.

Perhaps it would help if someone explained what the whitelist/blacklist is supposed to do. Right now, it doesn’t seem to do anything useful and the experience is at best inconsistent.

sam · March 25, 2015, 10:46pm

Its supposed to hint at the GitHub - cantino/ruby-readability: Port of arc90's readability project to Ruby to tell it what to select or not to select.

See:

github.com

cantino/ruby-readability/blob/master/lib/readability.rb#L54


    @input = GuessHtmlEncoding.encode(@input, @options[:html_headers]) unless @options[:do_not_guess_encoding]
    @options[:encoding] = @input.encoding.to_s
  end


  @input = @input.gsub(REGEXES[:replaceBrsRe], '</p><p>').gsub(REGEXES[:replaceFontsRe], '<\1span>')
  @remove_unlikely_candidates = @options[:remove_unlikely_candidates]
  @weight_classes = @options[:weight_classes]
  @clean_conditionally = @options[:clean_conditionally]
  @best_candidate_has_image = true
  make_html
  handle_exclusions!(@options[:whitelist], @options[:blacklist])
end


def prepare_candidates
  @html.css("script, style").each { |i| i.remove }
  remove_unlikely_candidates! if @remove_unlikely_candidates
  transform_misused_divs_into_paragraphs!


  @candidates     = score_paragraphs(options[:min_text_length])
  @best_candidate = select_best_candidate(@candidates)
end

probus · March 26, 2015, 2:08pm

Well, I assume that it’s supposed to hint that every time ruby-readability is used. Since it only has effect on very limited cases and otherwise the ruby-readability seems to be working on default settings, the implementation seems buggy.

Just to be clear:

I’d expect it to have some effect when embed truncate is not used (i.e. no button).
I’d expect the effect to be the same before and after the Show Full Post button is clicked
I’d expect the opening post not to change if there has been no edits to the blog post

Right now, none of these conditions are true.

sam · March 26, 2015, 10:12pm

The options are always passed, I am very confused here. Embedding always goes to the main site. I have no idea about the embed truncate thing

You are going to have to help fix this up if you want to use it, we have very very few spots using this feature and for those spots it works adequately.

probus · March 27, 2015, 8:51am

Me too, since clearly they are not always passed. Unfortunately I don’t understand the code well enough to see where the problem is. I only tried the truncate after @codinghorror suggested it has something to do with this.

You can try this on your own blog if you want. Set “.post” in the blacklist and see what happens. When you write a new blog post the created comment topic opening post will temporarily be empty until the Sidekiq job PollFeed runs and updates the full contents there without passing on the blacklist.

probus · April 14, 2015, 9:02am

I’m trying one more time, since we are nearing release.

Is there a way to have anything else than the full content of the blog post in the opening post? Empty post would be fine. If not, please tell me how I can help you to fix it.

probus · April 20, 2015, 9:11am

I’ve found a workaround.

If you disable rss feed polling, the whitelist/blacklist seems to work correctly. The post content also doesn’t revert back after a while.

The downsides are, that the opening post contents won’t update any more automatically and the comment topic gets created only when the first user loads the blog post. So when you create a new blog post you have to visit the url to create the comment topic and if you then change the blog contents the content of the opening post doesn’t update automatically anymore.

system · September 27, 2018, 10:33am