Optionally threading posts to parent topic in slack integration

At work, we use slack a lot. There’s some content that needs to be accessible on slack but should be organized and searchable, both of which I’ve seen Discourse shine at. We’ve tried a few wiki-like options and they all go stale, in part from lack of connection to slack. So I’m driving an evaluation of Discourse to run in parallel to and connected to our existing slack, since I already run a community Discourse forum. :slight_smile:

However, we have a lot of channels which use threads aggressively. This includes the highest priority slack channel to bidirectionally (watch and post) integrate between Discourse and slack.

Would it do violence to the chat-integrations plugin to have a non-default mode that posts new topics as messages to slack, stores the association of the slack message ID and topic, and then new comments on that topic are sent to slack as threaded messages? For my use case, that might be the main differentiator between success and failure.

Update: For my use case, this would be an option to watch not a site-wide slack chat configuration, because “prefer threads” or “prefer not threads” is a per channel preference based on the purpose (and typical participants) of a slack channel.

3 Likes

This sounds like a cool idea! Adding that to the plugin would certainly be #pr-welcome

Just checking… by “post” do you mean the existing transcript posting feature? Or are you trying to get a slack thread to 'synchronize" with a discourse topic. I think the latter is quite complicated to get right, and probably wouldn’t fit in the chat-integration plugin.

1 Like

No, not trying to bidirectionally sync a single thread so that you can post on either side and have it reflected on the other. I’ve been responsible for implementing fully bidirectional synchronization for active-active synchronization across two very active domains and I’m fully aware of the opportunities to get things wrong there. :slight_smile:

Our scenario is a channel in which:

  • Threaded answers are the norm because the channel tends to collect simultaneous conversations on unrelated topics (the channel is dedicated to a business purpose not a topic; the impractical alternative would be to invite half of engineering to new temporary channels created a dozen times a day)
  • Many but not all of those threaded conversations should be identified as answering questions that someone else will later want the answer to, and the integration used to promote them as trustworthy to the associated category in Discourse (someone like me actively curating useful answers)
  • New topics in that associated category in Discourse should result in slack posts to that slack channel
    • Additional comments on those new topics should be posted to Slack in a thread following the initial new slack post
  • Our users are directed to search Discourse first before asking in Slack

Additionally, you asking this question makes me think that as a related feature, it would be great to have the ability when summarizing a Slack thread into a Discourse post, for the integration to post a pointer to the new Discourse topic to the end of the original Slack thread that was just summarized. This would be helpful for us to keep conversation in one place.

This is actually tied into the original feature idea because if we had both, then it would make sense that instead of posting a new slack message when summarizing, it would post the message to the summarized thread, and note that thread as the place to post additional Discourse comments. It would not summarize additional slack comments in that thread to Discourse; it would be intended to redirect conversation to Discourse on that topic. This seems logical to me:

  • GIVEN: there is a place to store the slack message ID associated with a topic
  • AND: new comments on a topic are posted as comments to the stored slack message
  • THEN: set the slack message ID for a topic when using the post thread slackbot integration feature, causing new Discourse comments on the topic to be announced in that thread, including links back to the Discourse topic/comment

That would really help socialize the “check the Discourse first before asking” concept.

5 Likes

That all sounds good to me!

:100:

3 Likes

As a visual aid, does this look like the Slack side UX you want to promote?

… oh my, the year is dropped from the thread starter. any guesses?

@riking A bit, but with some differences.

The announce wouldn’t quite look like that; it would be posted by the watch side of the integration. It wouldn’t say that it was moved. Here’s an example of the current (unthreaded) interface, from the slack integration as I implemented for makerforums:

In the threaded model, the first of those posts would start a slack thread, and Nedman’s comment would instead be in a threaded slack response. When post thread :thread_url is used, the :thread_url would be stored with the new topic and the watch side of the chat integration would post the initial topic post and all further comments to that slack thread, but the content would be the same.

The watch thread integration sends you to discourse where you have to write the title and have the opportunity to edit the new topic post before posting it. So the announcement that is posted to slack already credits the person running post thread as the author, then has a line with the topic title as the link text which links to discourse, and then has a pull quote from the beginning of the post. The change I propose here is that with a non-default and per-channel configuration (so a watch option, not site-wide chat integration plugin configuration), it would post that content to the thread rather than to the channel. (Not “also send to channel” — which would defeat the purpose of the change for us.) Further comments on that new topic in discourse would also go into that same slack thread.

And, since you ask for guesses, 2017, the year that slack rolled out threads?

2 Likes

I’ve been thinking about this and reading the chat.postMessage slack API docs, and I believe I can boil down my wall of words to something much simpler.

Only watch and not follow has the ability to choose threading responses, by a mechanism I’m still trying to determine. Alternatively, @david what would you think of a new thread rule filter with precedence mute thread watch follow, and plumbing the rule through trigger_notification to enable rule-sensitive behavior?

  1. If watch is configured to thread (alternatively, a thread rule is defined), sending a new post notification to a slack channel, if the post topic has a slack ts associated with it, post to that slack thread by setting thread_ts to the provided ts value from Slack.

  2. When sending a new post notification to a slack channel, and the post topic does not have a ts associated with it, store the returned response ts for the topic (so that future posts on the topic can be threaded if watch is configured for threads).

  3. When using the post thread :thread_url command, store the thread ts in the topic that is created, which will be used only by threaded watch rules.

Here are my current thoughts and concerns:

  1. How to determine whether to post to threads on a per-rule basis. A new filter feels easiest to me at the moment, but maybe I’m missing something.

  2. Flowing the original slack post URL and thread ID through the transcript flow is what is most opaque to me right now. This looks like I really need to add a per-provider thread ID somewhere and preserve it until saving the post. I would implement it only for slack ts but presumably it won’t be the only chat integration with threads.

  3. For posting, I think I need to store the slack ts in a slack-specific custom field on Topic, not a general DiscourseChat custom thread field.

1 Like

Does the ‘threading’ boolean really need to be per-rule? Why not make a new site setting

chat_integration_slack_thread_responses

If that is enabled, and we have a record of the topic’s thread ID, then we send subsequent notifications to the slack thread.

Yeah, I think that’s totally fine. Just use a slack-specific topic custom field. If we feel the need to implement this for other providers later, we can make the solution more generic.

Yeah, this is very tricky. As a v1, I would be tempted to include something like

<!--SLACK_THREAD_ID=abcdefg-->

at the top of the post. Then the plugin can check for posts which start with this string, and assign the custom field. It’s not perfect, but maybe a good starting point?

2 Likes

Thanks much!

Last night, I implemented and have passing tests all through the stack (though I haven’t yet tested live) my first two ACs, but not yet the transcript flow, using a custom field on Topic and a thread rule. So making good progress toward being able to show code in at least a draft PR.

I also have a separate commit to remove the unused pstore_get from the slack provider. Since that’s the only use of pstore in the whole stack, would you like me to also remove all the pstore_* from app/initializers/discourse_chat.rb in that cleanup commit?

That’s exactly where I started, and it sure would have been easier!

However, at least for my use case (and I’ve heard this from people at other companies, we’re not unique) different channels have different preferences for whether threads should be used. There are channels which are meant to be used with threads (because most people care only about some topics) and channels which are very much anti-thread (because threads bury important information).

I’ve seen two essentially unrelated aspects driving this preference:

  • Membership: Channels with most members having actively used IRC for decades are used to mentally disambiguating intermingled conversational threads in a single channel and don’t want to click into threads to see important content, whereas channels with most members having lived in email expect conversations to be threaded and to click into conversations they care about.
  • Purpose: Channels with an announcement business purpose tend to be aggressively threaded, but channels with research or collaboration purposes are often intentionally not threaded because threads hide information unless you notice them and click into them.

If you’d like to have some common syntax for rule options, I’d think that Rule could be generally expanded so that it has an :options value which, like :tags, is an array. I suppose it might make sense to take a page from command-line shells, so that /discourse [watch|follow|mute] -something -here sets :options to ['something', 'here'] — reserving a leading - to introduce an option. Then it would be /discourse watch my-category-slug #tagged-foo -threaded. I don’t know how much more work that would be over what I already did. Do you have strong feelings one way or the other here? I can see arguments both ways: adding another value to Rule makes it more complicated to write a chat provider; adding another filter value makes another slack-specific feature (like posting a transcript) that makes the interface more complicated for non-slack users. Of course I might be missing something that would make it harder than I think; for example, if a category slug could start with - and it is suddenly ambiguous; the shell uses -- to mean “everything after this is not an argument” but perhaps we could invert it to -- means “everything after this is an option.” If you’d like me to look into this, please give me some direction on syntax that you would find appropriate for specifying rule options. But just adding thread seems simpler to me, so absent specific direction to add a general “options” feature, I’ll wait to make any change in that direction.

[Edit: Changing from a rule filter to an option would definitely complicate the UI which would have to grow to general support for general options and that doesn’t feel like a great choice to me. So now, having looked at UI, I’d prefer to stick with the filter; I think it’s cleaner.]

For the post transcript flow, thanks for the HTML comment idea. For my use case it’s not going to hurt me any; I’m honestly expecting to be the primary user anyway, at least initially. That’s a nice simple hack.

I have a separate idea, not to embed in this PR, that it would be nice to have a mapping from the channel for the message being posted to the category in which the resulting discourse post should be made. For that to happen, I think that the transcript would best change to have an envelope or sidecar, at which point the envelope or sidecar could have both the category and the slack ID. That looks like a substantial amount of learning for me, since I’m still in the “advanced googling the problem” stage of learning Ruby on Rails. :wink:

2 Likes

One other idea for gold-plating this feature:

What if there were another setting, cross_post_to_channel_after_hours (default 48?) or something, where if the time since the last reply is greater than x hours, the post is sent the the thread with the “also send to channel” flag.

Don’t take my suggestion too seriously right now if you just want to get things working without it first (probably a better strategy!). Just food for thought…

Not at all a crazy idea but I’m not planning to implement it, since it would break my main use case and I have no need for it anywhere else. :slight_smile: I totally see that other folks would want it though!

If it were a normal setting rather than a per-channel option, it would be chat_integration_slack_copy_threads_to_channel_after_hours which would default to 0 (don’t send) but you can set it to any value. It is conceptually fairly simple but requires more work because you have to go grab the thread (using the code initially written for transcripts, and I don’t know off the top of my head whether it would require refactoring) in order to make a decision about setting a simple flag.

But if you want to work on it, I hope that what I’m doing would provide a good framework. I would just ask that you not turn it on by default because if that hit in an upgrade and my users were “spammed” by me curating content to discourse, I’d have unhappy users.

1 Like

To simplify things, I think you could just check the date of the previous post in the Discourse topic instead.

(It would behave a bit differently in some cases, but could be a fine place to start)

@david I have a draft PR up that passes unit tests as I’ve written them. I haven’t yet tried running it live against slack so I wouldn’t want to merge it until that’s done. But at least all the unit tests pass for me, and I tried to add meaningful tests for them.

I also put the ID comment at the end instead of the beginning of the post because it felt more likely to survive and less likely to be mangled there, and I tried to be conservative about parsing it out.

Thanks for your work on this plugin!

https://github.com/discourse/discourse-chat-integration/pull/38

I fixed my initial linting failures, but now tests I didn’t touch are failing. I’m based off latest master, and tests pass locally. Do you have insight into the failing tests?

4 Likes

I’ve cleaned up the PR a lot but it’s not quite yet ready to go. I have two or three things so far causing me trouble that I don’t yet know how to fix.

  1. I’m trying to use fa-arrow-circle-o-right as my thread icon, and it is showing up empty in the UI on my live site. (I’ve been running su discourse -c 'bundle exec rake assets:precompile' && sv restart unicorn after checking out my branch on my live site to test on the live server.) I’ve added it to plugin.rb as well as referencing it, so I’m lost on what next steps are. Is there a list of fontawesome icons that are approved for use in Discourse? Found lib/svg_sprite/svg_sprite.rb and chevron-right looks great for this use.

  2. Tests are all passing for me locally, but in Travis I’m getting consistent errors that appear unrelated to my changes, and naturally that’s hard for me to investigate or reason about. 13 failures with a 404 instead of some other expected (e.g. 200) in spec/lib/discourse_chat/provider/slack/slack_command_controller_spec.rb Fixed by not cargo-coding isolate_namespace and now I know about rake routes

I’ve successfully posted:

There might be more to clean up, but I think this works.

After this merges, I’ll update Chatroom Integration Plugin (discourse-chat-integration) appropriately.

2 Likes

I keep finding new opportunities to learn here. Now I know how to run yarn prettier, next I get to learn how to update front end tests…

Error: Assertion Failed: You must use set() to set the `channel` property (of <(unknown):ember3806>) to `<(unknown):ember3799>`.

Thanks so much for all your work here @mcdanlj - this is now merged:

https://github.com/discourse/discourse-chat-integration/commit/da9106127afea45a72b8dcc8b944b8cbfc1b8fd1

4 Likes

I appreciate your very helpful reviews! I updated the official integration wiki post as promised. If you’d prefer to call out the changes in other ways I’m cool with that, no pride of authorship or anything like that. :slight_smile:

3 Likes

Could threading please be enabled within chat-integration Settings for Slack ala

Enable threading of Posts to Slack

You mean, you want to be able to configure the plug-in to refuse to honor the thread option in the integration?

We updated to current 2.6 beta1a tests-pass few days ago, but as far as I can tell no slack x-posts are threaded since the upgrrade. We still see topic posts and replies individually displayed on Slack channels.