Moving chat messages to a new channel jumbled them all up

I moved 28 messages from one channel to a new one and they all went out of order:

Hmmm. Is it just me, or do these seem out of order now? :thinking:

I've had another look at the out-of-order posts in https://meta.discourse.org/chat/channel/147/chat?messageId=2644, and it seems they all have the same timestamp.

5 Likes

Thanks for reporting this, I did try to take this into account with the feature but I think I got a little too lucky with the testing I did :sweat_smile: So we do indeed set all the timestamps to the same thing here:

The problem is that we do not want to interleave the moved messages between the existing channel messages, and it gets harder and harder the more messages your move.

A question before I dive too deep on this – can you remember and identify which messages are out of place? Is there just a couple, or are they completely out of order? I think what has possibly caused the discrepancy is that when we are fetching messages for a channel, we do it by ID order (which we order DESC and then reverse in most cases):

Whereas in the message mover I am ordering by created_at to maintain order, which could cause some small discrepancies:

I have some ideas on how we might be able to address this (maybe changing the message mover to order by ID or changing the controller to order by created_at would be enough, preferring the latter because I think it would make more sense), but I would like to hear just how messed up the order is if it’s possible to tell.

4 Likes

I un-deleted them from the original channel after I noticed they were jumbled in the new one. I should be able to quote them in order here: :crossed_fingers:

Original order
Originally sent in The Chat Feedback topic
JammyDodger

I'm looking for ways we can capture the essence of the new chat flow as a demonstration of how chat can be the seeds of bigger discussion

Has anyone got any ideas on how we can do that from where we are now with the chat-testing here on Meta?

The feedback points are great, and I think they will be calved off into their own proper topics shortly too, but I was hoping for something that would work as a great exemplar for anyone just joining the @chat-testers. Something that people can just look at and go 'Aaaah. Yes, I wasn't sure at first, but I see how can chat can be the pre-cursor to in depth discussion'

I may be asking too much :slightly_smiling_face:

RGJ

TBH I don't see how chat can be the pre-cursor to in depth discussion :thinking:

But that might be because I'm old

JammyDodger

So we're looking for an example that will help change @RGJ's mind too :slightly_smiling_face:

Moin

I like the example here. But that example does not fit here. Maybe the topic could be about a feature you miss just now. Something you wouldn't start a topic because for example you don’t have time to check whether there is a similar feature request or you think no one else will be interested into that.

JammyDodger

@Moin, your search skills are always a blessing :slightly_smiling_face:

I was looking for one about seeds and trees, but couldn't find it on here

But, yes. Some kind of relaxed/friendly/informal chat in which an idea forms through the course of the easy back-and-forth, and then inspires a proper discussion topic

oshyan

This is 100% my interest and use-case for chat. But can you clarify exactly what you mean by an example of this. Are you interested in, say, samples of a chat (regardless of whether it happened in Discourse chat) that could/should/would have (obviously) led to deeper discussion and/or did lead to deeper discussion but in an arguably less ideal medium for it (chat vs. forum)? If that's the case I may need a little to find some, but I absolutely have good examples from my productivity community. If you're looking for examples in Discourse chat, specifically, that's going to be harder to find. But I absolutely think this is a big value of chat in Discourse, and depending on the community, it will play a bigger or smaller role.

Jonathan_Poyer

I think showcase a discussion around a new feature is actually a nice way to demo it at least during the first moment of that idea. For some people, this spark of discussion is during the time of development or about to begin. There are always more things to discuss about and referring to a topic (or multiple) makes sense.

oshyan

As a perhaps more conceptual example of a situation where chat can (and should) quickly turn into a topic, even mid-way through a discussion, this is something that often happens in software dev management communities I'm part of, as well as my productivity community:

  • New person joins chat and asks a seemingly simple or innocuous question
  • Responses from highly informed and/or impassioned regulars quickly grow to 10s of lines of text, paragraph breaks start happening, and that chat channel becomes saturated only with discussion on this one question (topic)
  • Everything becomes difficult to respond to as each "message" includes lots of points and ideas and no select-to-quote/reply feature makes parsing and responding to each thing difficult
  • These conversations are also often valuable debates that would quickly disappear in the subsequent flow of a chat, so even retroactively moving them to a Topic is potentially very valuable
JammyDodger

I think I was initially looking for something we could provide as an example topic/chat here on Meta to demonstrate to people new to Discourse Chat of how it could gel nicely with pre-existing ideas of the 'long-form paragraph' view of Discourse

So even something that we created to neatly demonstrate the principle

Though it sounds like you have lots of examples that would make a great discussion topic :slightly_smiling_face:

I think anything that can help people easily visualise where in their forum structure chat can slot into would be useful. I am open to all ideas :+1:

agungor

I feel that this conversation is becoming such an example per se. :smiley:

Jonathan_Poyer

Feels like something that would become a thread on other platforms would either need to turn into another chat or a split off topic. But at the same time, topic feel also more like long term discussion vs one-off as here?

jimkleiber

Maybe one way is to quote the initial chat message that started the idea into a topic: How can chat seed topic discussions?

This can give visibility to people who aren't in the chat, especially if the topic starts with a question

but ironically I'm answering here lol

Hmm, I just tried to quote my replies here into that same topic but I think I only have the option to quote into a new topic, not an existing one

JammyDodger

I was just mulling that over. :slightly_smiling_face: I was working out if I could make a topic with each person's chat as a reply somehow, as there were no suitable chat channels to slide them to. But now you've made it I can see that it could have its own chat channel, and we could move this conversation there :+1:

jimkleiber

Ah yes, almost like creating the topic that will create the new chat channel

and then the topic can be filled with just quotes from the chat, pulling the highlights

Jumbled
oshyan

This is 100% my interest and use-case for chat. But can you clarify exactly what you mean by an example of this. Are you interested in, say, samples of a chat (regardless of whether it happened in Discourse chat) that could/should/would have (obviously) led to deeper discussion and/or did lead to deeper discussion but in an arguably less ideal medium for it (chat vs. forum)? If that's the case I may need a little to find some, but I absolutely have good examples from my productivity community. If you're looking for examples in Discourse chat, specifically, that's going to be harder to find. But I absolutely think this is a big value of chat in Discourse, and depending on the community, it will play a bigger or smaller role.

Jonathan_Poyer

I think showcase a discussion around a new feature is actually a nice way to demo it at least during the first moment of that idea. For some people, this spark of discussion is during the time of development or about to begin. There are always more things to discuss about and referring to a topic (or multiple) makes sense.

oshyan

As a perhaps more conceptual example of a situation where chat can (and should) quickly turn into a topic, even mid-way through a discussion, this is something that often happens in software dev management communities I'm part of, as well as my productivity community:

  • New person joins chat and asks a seemingly simple or innocuous question
  • Responses from highly informed and/or impassioned regulars quickly grow to 10s of lines of text, paragraph breaks start happening, and that chat channel becomes saturated only with discussion on this one question (topic)
  • Everything becomes difficult to respond to as each "message" includes lots of points and ideas and no select-to-quote/reply feature makes parsing and responding to each thing difficult
  • These conversations are also often valuable debates that would quickly disappear in the subsequent flow of a chat, so even retroactively moving them to a Topic is potentially very valuable
JammyDodger

I think I was initially looking for something we could provide as an example topic/chat here on Meta to demonstrate to people new to Discourse Chat of how it could gel nicely with pre-existing ideas of the 'long-form paragraph' view of Discourse

So even something that we created to neatly demonstrate the principle

I think anything that can help people easily visualise where in their forum structure chat can slot into would be useful. I am open to all ideas :+1:

I'm looking for ways we can capture the essence of the new chat flow as a demonstration of how chat can be the seeds of bigger discussion

Though it sounds like you have lots of examples that would make a great discussion topic :slightly_smiling_face:

Has anyone got any ideas on how we can do that from where we are now with the chat-testing here on Meta?

RGJ

TBH I don't see how chat can be the pre-cursor to in depth discussion :thinking:

agungor

I feel that this conversation is becoming such an example per se. :smiley:

Jonathan_Poyer

Feels like something that would become a thread on other platforms would either need to turn into another chat or a split off topic. But at the same time, topic feel also more like long term discussion vs one-off as here?

JammyDodger

The feedback points are great, and I think they will be calved off into their own proper topics shortly too, but I was hoping for something that would work as a great exemplar for anyone just joining the @chat-testers. Something that people can just look at and go 'Aaaah. Yes, I wasn't sure at first, but I see how can chat can be the pre-cursor to in depth discussion'

jimkleiber

Maybe one way is to quote the initial chat message that started the idea into a topic: How can chat seed topic discussions?

This can give visibility to people who aren't in the chat, especially if the topic starts with a question

JammyDodger

I think, at least. :slightly_smiling_face: I'll double-check my thinking before I do anything

I may be asking too much :slightly_smiling_face:

jimkleiber

but ironically I'm answering here lol

RGJ

But that might be because I'm old

JammyDodger

I was just mulling that over. :slightly_smiling_face: I was working out if I could make a topic with each person's chat as a reply somehow, as there were no suitable chat channels to slide them to. But now you've made it I can see that it could have its own chat channel, and we could move this conversation there :+1:

jimkleiber

Ah yes, almost like creating the topic that will create the new chat channel

JammyDodger

So we're looking for an example that will help change @RGJ's mind too :slightly_smiling_face:

jimkleiber

and then the topic can be filled with just quotes from the chat, pulling the highlights

Moin

I like the example here. But that example does not fit here. Maybe the topic could be about a feature you miss just now. Something you wouldn't start a topic because for example you don’t have time to check whether there is a similar feature request or you think no one else will be interested into that.

JammyDodger

@Moin, your search skills are always a blessing :slightly_smiling_face:

I was looking for one about seeds and trees, but couldn't find it on here

But, yes. Some kind of relaxed/friendly/informal chat in which an idea forms through the course of the easy back-and-forth, and then inspires a proper discussion topic

1 Like

I wonder when it’s preferable to move messages rather than quote them. Perhaps it depends whether there is an existing topic or not? Not sure. In what scenarios would one of the following be better to nudge people towards?

  1. Quote chat messages in existing topic
  2. Move chat messages to an existing topic
  3. Quote chat messages to a new topic
  4. Move chat messages to a new topic

Since strings of chat messages are, erm, more “chatty” than topics, I have some sense that we may want to encourage quoting more than moving, in general.

Are there cases folks have observed or have in mind where you’re like, “nope, quoting would not be good here. definitely need to move them instead.”?

1 Like

If you just quote the discussion can be continued in two places.

@Moin are you suggesting that moving messages would be preferable when you really want to avoid that?

Thanks for doing that – that’s completely jumbled up! I will have to do some local testing on larger message sets. I think at the very least this will be needed:

I am generally uneasy about ordering by ID though because of weird inconsistencies, I think ordering messages by created_at would be better generally for the channels. @joffreyjaffeux or @mcwumbly what are your thoughts on this? If we decide to do that then the message mover might need to artificially space out the created_at values by 10ms each or so for consistent ordering.

I think in general if they are totally irrelevant to the current channel it would be better to move to a more appropriate channel. We used this many times previously internally when we used Mattermost. For example, a bunch of incident response in the general channel that should be moved into the incident channel for better record keeping. Or, idle chatter in a channel that is better to be in the random channel.

I don’t think there is any value in those cases of quoting and leaving old cruft behind, and as Moin says, things can then be confusing, where the discussion is continued in two different places.

Keep in mind these two options do not currently exist. We removed “Move to Topic” because in the initial implementation it was making one post per chat message, and also not deleting the original messages in the channel. If in future we want to make this again it needs to:

  • a) quote batches of messages together (say 100 per post) using the chat quote feature and
  • b) delete the originals in the channel to avoid duplication.
4 Likes

I’ll abstain from commenting on the implementation for ordering posts and let @joffreyjaffeux comment on that aspect of things.

Ah, yeah. I wasn’t asking about moving chat messages within chat, but I can see how that can be useful, and it doesn’t have the problem of trying to convert short-form into ling-form (or vice versa) “in post”.

Makes sense. I like the general form of quoting as more of a “transcript” like this because I think it’s likely going to read that way anyway. In the past, when I used the Slack transcript feature, I often found myself wrapping it in [details] as well, and summarizing things in the main post body.

Another thought I had along those lines might be to have a fancier “expand context” feature, so you could quote a single message, but then load additional messages inline on demand to view more context from chat without leaving the topic.

I’m skeptical about this part being necessary or valuable when referencing discussions across the slow lane/fast lane boundary.

3 Likes

It’s only if you’re choosing Move to Topic that this would happen, why keep things around in the channel if your intention was to move it? We have had some discussions around this internally already. Sure just a normal quote of messages into a topic would not delete anything.

Some trivia for you, the class that generates the quotes is actually called ChatTranscriptService :slight_smile:

This is interesting, we actually do have something similar to this with our topic quotes (you probably already have seen that). It would probably be useful to get a little more context without having to actually visit the channel.

2 Likes

I would say the use case for moving is:

  • We have a channel dedicated to discussion about “Whales”
  • A bunch of people start having a an involved discussion about “Penguins” cause they forgot to click “#penguin” and stuff got heated
  • Mod steps in and :foot: kicks the Penguin talk to the Penguin channel.

I guess the fundemental thing here is the re-sequencing.

I would say “fudge created_at” is about the only sane solution here cause you want everything moved in one block? Plus it technically is created at the point of time it is moved.

4 Likes

Yeah, I think I’m wondering whether it’s needed, or if quote/transcribe is the thing to focus on making work really well.

2 Likes

Yes I would 100% do this if our normal GET messages route for a channel was ordered by created_at, that’s what I want to sort out, was just wondering if Joffrey had some historical knowledge about that. If not will change both things at once.

2 Likes

Yes Im 100% with sam and you😁 Moving everything in one go and giving it the created_at of the time of the move is the only sane approach IMO. Otherwise it opens a gigantic can of worms… how do I know where to find it? receiving unread notifications for things created before my last read? nope nope nope

3 Likes

Nice, I will tweak the move tool to the moved messages be in :sparkles: the future :sparkles: with a small spaced out increment, and make the chat messages order by created_at instead of ID in the general controller :+1:

3 Likes

Just merged this to hopefully address the issue:

I didn’t do anything to artificially space out created_at for now into the future, so let’s see how this goes first.

4 Likes

This topic was automatically closed after 11 days. New replies are no longer allowed.