Collude - a collaborative text editor for Discourse

Collude is a collaborative text editor for Discourse, written using EasySync, the same underlying algorithm used in Etherpad :nerd_face:

Give it a look

Check out a video demo, or give it a whirl on my test instance.

Give it a try

NOTE: This project is decidedly beta at the moment, and I am posting here in order to garner some willing souls to help me test it out and let me know whatā€™s rough and what I should improve next.

To install, follow the normal plugin installation instructions.

As an admin, create a post, and click ā€˜Allow collaborative editingā€™ under the wrench menu

Now, users with sufficient trust level (default is 2, configurable in the settings), will be able to click ā€˜Collaborateā€™ and get an editor to modify the post in real time.

Configurable settings:

Learn more, help out, or give feedback

Visit the github repo, or ping me @gdpelican here on meta.

:tulip::tulip::tulip:

51 Likes

This looks pretty cool! I had a quick skim of the repo and canā€™t see any external dependencies - is that correct? It just implements the ā€˜EasySyncā€™ algorithm in Discourse?

1 Like

There arenā€™t any external dependencies, no. The linked PDF is essentially a fairly detailed technical description of how Etherpad does their collaborative editing, which Iā€™ve implemented in the plugin, the guts being here on the frontend, and here on the backend. (The paper does an excellent job of obscuring how simple the implementation actually is, but I couldnā€™t find a better one, and of course Etherpadā€™s source has a bunch of other stuff going on, so itā€™s not the greatest thing to look at for an MVP type implementation).

Itā€™s probably not perfectly to the letter of the algorithm, notably the bits about merging otherwise incompatible diffs, but I kinda found those to be somewhat rare corner cases, which could be pretty easily remedied by the users involved even if it doesnā€™t follow what should happen exactly. For example, the paper suggests that if two users submit diffs like this at the same time:

(baseball -> below) + (baseball -> basil) == besiow

whereas my implementation might output

(baseball -> below) + (baseball -> basil) == belowasil

ĀÆ\_(惄)_/ĀÆ

16 Likes

Awesome! I love this. Thanks for creating it! :sunflower:

Some suggestions:

  • when editing collaboratively in the composer, it is not clear how to stop editing. itā€™s the ā€œminimize the composer panelā€ button in the top right corner of the composer which in this case makes it go away instead of minimizing. Maybe change the helper text to say ā€œstop editingā€. Or add a button where REPLY/cancel normally would be to say STOP EDITING or some such.
  • it would be helpful to know who has collaboratively edited a particular post, as well as who is currently in there collaboratively editing right now. Could the existing ā€œreplyingā€¦ā€ feature potentially be utilized for this? e.g, display avatars and ā€œeditingā€¦ā€ in the composer and below the post.
  • it would also be helpful to know who contributed what to the collaboratively edited text. Not sure how to handle this - I liked how etherpad used to let you choose a color for yourself, which would identify your contributions. Google docs lets you see whoā€™s editing and where their cursor is in the doc.
  • I think the ā€œcollaborateā€ button could be changed to ā€œeditā€ but using the same icon and helper textā€¦ I think itā€™s clearer and in many cases there may not be more than one person around to edit anyway.
  • a different and more scalable way to allow/disallow collaboration may be to add the collaborate icon to the post menu, with ā€œstart collaboratingā€ (not enabled) and ā€œstop collaboratingā€ (enabled) as options. Only visible to those allowed to start/stop collaboration.
  • a category setting to enable collaborative editing might be preferable, along the lines of wiki.
8 Likes

Great, thanks for the feedback! Some responses:

Iā€™ve added this (It was a bug that it wasnā€™t there, heh :sweat_smile:
image

I agree with this a lot and will be thinking hard over the next wee while on the best way to implement it; itā€™s tricky, but essential in my opinion. Displaying cursors feels fragile and crazy to me (I think you have to absolutely position an indicator div based on a cursor position sent over the wire), so Iā€™m currently thinking on the best way to store information in the diff structure about the author of each fragment (which I think should allow Etherpad-style background coloring), while trying to keep it acceptably scalable. The answer to ā€˜can the existing replying functionality be usedā€™ is no :slight_smile:

Iā€™ve done this as well, as well as hiding the ā€˜editā€™ and ā€˜wikiā€™ functionality when a document is realtime-editable (I think of this as a per-post replacement for those functions, which are already mutually exclusive)
image

I disagree with this one; when building features like this I often think of them as ā€˜analogousā€™ with some other feature in the core codebase (retort is analogous to ā€˜likeā€™, for example), and attempt to design the experience to be parallel to that. For this feature, thatā€™s ā€˜wikiā€™ (ie, if youā€™re thinking about doing this to a post, you would probably also consider wiki-fying it), meaning those options want to be together in the interface.

image

Iā€™d like to do this, but the core codebase isnā€™t very extensible to adding category settings at the moment (the one for Retort is a pretty hacky workaround that Iā€™d like to avoid repeating), and so would require changes to core - and Iā€™ve had trouble getting core to accept changes to accommodate plugins. So Iā€™m very hesitant to make plugin features which depend on changes to core, because even very low-risk requests get rejected too often.

4 Likes

I like the fixes - thank you! I can see this being very useful in my community, especially when we are in one of our many webinars or video conference calls and taking notes collaboratively.

Bummer about not being able to replicate the ā€œreplyingā€¦ā€ feature. That way weā€™d know itā€™s worth jumping in to collaborate, and once in collaborating weā€™d know whoā€™s collaborating with us.

Has anyone experienced problems installing this plugin? Is it stable? Iā€™d like to try it in my community as soon as itā€™s reasonably stable. :rocket:

Iā€™m currently installing it. I will report soon :slight_smile: If others already use it though, Iā€™m interested to know as well.

First Report

Installation goes well. I tried to create a new topic and turned it into a collaborative editor using the topic menu option in the wrench. It worked. Then I invited my companion to join the topic and edit. But here thereā€™s a bug: as she started editing, the existing content of the pad disappeared. I tried to edit as well, but our edits would cancel each other.

Iā€™ve seen this with early Etherpad Lite where the pointer would flicker as other people would edit, jumping from your line to their line. I guess something similar is happening here, that the pointers are not handled well. @gdpelican did you encounter this bug?

Note that I have the Embed Etherpad Lite pads into Discourse plugin installed as well, so they might be conflicting.

Interestingly, although the content was removed from the editor, it didnā€™t save.

1 Like

Can you point me to the instance where this is happening?

Itā€™s a private topic but I can invite you there.

Here, I made a public one.

https://how.zoethical.com/t/testing-collude/115

Note that when I click ā€œEditā€, I get an empty post although the sentence ā€œThis is a collaborative editor.ā€ is written in the topicā€™s first post.

Let me run rake multisite:migrate since a log is complaining about itā€¦

OK, I fixed it for the main instance, but another instance on Multisite cannot have it. I wonder whether it comes from a mistake I made with /shared permissionsā€¦ I rebuilt all containers, so it should work now.

On the failing instance I get

ActiveRecord::StatementInvalid (PG::UndefinedColumn: ERROR: column "collusion" does not exist

and

Clearing Active Record cache, this can happen if schema changed while site is running or in a multisite various databases are running different schemas. Consider running rake multisite:migrate. 

Hm, I tried making an account on the linked site, but it didnā€™t work for me; I managed to make one on the site it redirected me to, but clicking ā€˜loginā€™ on how.zoethical.com gives me this:

Iā€™ll shift the migration around to the official method to see if that clears up the migration error; of course I didnā€™t test it on a multisite setup. In the meantime you can run the following to get the migration to go:

require Rails.root.join('plugins', 'collude', 'app', 'migrations', 'add_collusions')
AddCollusions.new.up

This must be because I disallowed Gmail on this instance.

Thanks! Do you happen to remember how to run this in multisite mode? I keep forgetting this.

Nah, it was a straight email signup :slight_smile:

OK, so I ran thisā€¦ Then I tried again and got:

 	
NameError (undefined local variable or method `collusion' for #<Collusion:0x00007fc71730e220>) /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/activemodel-5.2.0/lib/active_model/attribute_methods.rb: 

/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/activemodel-5.2.0/lib/active_model/attribute_methods.rb:430:in `method_missing'
/var/www/discourse/plugins/collude/app/models/collusion.rb:13:in `block (2 levels) in collusion_accessor'
/var/www/discourse/plugins/collude/app/models/collusion.rb:42:in `user='
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/activemodel-5.2.0/lib/active_model/attribute_assignment.rb:51:in `public_send'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/activemodel-5.2.0/lib/active_model/attribute_assignment.rb:51:in `_assign_attribute'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/activemodel-5.2.0/lib/active_model/attribute_assignment.rb:44:in `block in _assign_attributes'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/activemodel-5.2.0/lib/active_model/attribute_assignment.rb:43:in `each'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/activemodel-5.2.0/lib/active_model/attribute_assignment.rb:43:in `_assign_attributes'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/activerecord-5.2.0/lib/active_record/attribute_assignment.rb:23:in `_assign_attributes'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/activemodel-5.2.0/lib/active_model/attribute_assignment.rb:35:in `assign_attributes'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/activerecord-5.2.0/lib/active_record/core.rb:314:in `initialize'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/activerecord-5.2.0/lib/active_record/inheritance.rb:66:in `new'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/activerecord-5.2.0/lib/active_record/inheritance.rb:66:in `new'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/activerecord-5.2.0/lib/active_record/persistence.rb:35:in `create'
/var/www/discourse/plugins/collude/app/models/collusion.rb:19:in `spawn'
/var/www/discourse/plugins/collude/app/controllers/collusions_controller.rb:26:in `create_collusion'
/var/www/discourse/plugins/collude/app/controllers/collusions_controller.rb:7:in `create'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/actionpack-5.2.0/lib/action_controller/metal/basic_implicit_render.rb:6:in `send_action'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/actionpack-5.2.0/lib/abstract_controller/base.rb:194:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/actionpack-5.2.0/lib/action_controller/metal/rendering.rb:30:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/actionpack-5.2.0/lib/abstract_controller/callbacks.rb:42:in `block in process_action'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/activesupport-5.2.0/lib/active_support/callbacks.rb:132:in `run_callbacks'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/actionpack-5.2.0/lib/abstract_controller/callbacks.rb:41:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/actionpack-5.2.0/lib/action_controller/metal/rescue.rb:22:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/actionpack-5.2.0/lib/action_controller/metal/instrumentation.rb:34:in `block in process_action'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/activesupport-5.2.0/lib/active_support/notifications.rb:168:in `block in instrument'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/activesupport-5.2.0/lib/active_support/notifications/instrumenter.rb:23:in `instrument'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/activesupport-5.2.0/lib/active_support/notifications.rb:168:in `instrument'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/actionpack-5.2.0/lib/action_controller/metal/instrumentation.rb:32:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/actionpack-5.2.0/lib/action_controller/metal/params_wrapper.rb:256:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/activerecord-5.2.0/lib/active_record/railties/controller_runtime.rb:24:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/actionpack-5.2.0/lib/abstract_controller/base.rb:134:in `process'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/actionview-5.2.0/lib/action_view/rendering.rb:32:in `process'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/rack-mini-profiler-1.0.0/lib/mini_profiler/profiling_methods.rb:104:in `block in profile_method'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/actionpack-5.2.0/lib/action_controller/metal.rb:191:in `dispatch'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/actionpack-5.2.0/lib/action_controller/metal.rb:252:in `dispatch'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/actionpack-5.2.0/lib/action_dispatch/routing/route_set.rb:52:in `dispatch'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/actionpack-5.2.0/lib/action_dispatch/routing/route_set.rb:34:in `serve'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/actionpack-5.2.0/lib/action_dispatch/journey/router.rb:52:in `block in serve'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/actionpack-5.2.0/lib/action_dispatch/journey/router.rb:35:in `each'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/actionpack-5.2.0/lib/action_dispatch/journey/router.rb:35:in `serve'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/actionpack-5.2.0/lib/action_dispatch/routing/route_set.rb:840:in `call'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/rack-protection-2.0.3/lib/rack/protection/frame_options.rb:31:in `call'
/var/www/discourse/lib/middleware/omniauth_bypass_middleware.rb:24:in `call'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/rack-2.0.5/lib/rack/tempfile_reaper.rb:15:in `call'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/rack-2.0.5/lib/rack/conditional_get.rb:38:in `call'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/rack-2.0.5/lib/rack/head.rb:12:in `call'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/actionpack-5.2.0/lib/action_dispatch/http/content_security_policy.rb:18:in `call'
/var/www/discourse/lib/middleware/anonymous_cache.rb:214:in `call'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/rack-2.0.5/lib/rack/session/abstract/id.rb:232:in `context'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/rack-2.0.5/lib/rack/session/abstract/id.rb:226:in `call'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/actionpack-5.2.0/lib/action_dispatch/middleware/cookies.rb:670:in `call'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/actionpack-5.2.0/lib/action_dispatch/middleware/callbacks.rb:28:in `block in call'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/activesupport-5.2.0/lib/active_support/callbacks.rb:98:in `run_callbacks'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/actionpack-5.2.0/lib/action_dispatch/middleware/callbacks.rb:26:in `call'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/actionpack-5.2.0/lib/action_dispatch/middleware/debug_exceptions.rb:61:in `call'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/actionpack-5.2.0/lib/action_dispatch/middleware/show_exceptions.rb:33:in `call'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/logster-1.2.11/lib/logster/middleware/reporter.rb:31:in `call'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/railties-5.2.0/lib/rails/rack/logger.rb:38:in `call_app'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/railties-5.2.0/lib/rails/rack/logger.rb:28:in `call'
/var/www/discourse/config/initializers/100-quiet_logger.rb:16:in `call'
/var/www/discourse/config/initializers/100-silence_logger.rb:29:in `call'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/actionpack-5.2.0/lib/action_dispatch/middleware/remote_ip.rb:81:in `call'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/actionpack-5.2.0/lib/action_dispatch/middleware/request_id.rb:27:in `call'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/rack-2.0.5/lib/rack/method_override.rb:22:in `call'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/rails_multisite-2.0.4/lib/rails_multisite/middleware.rb:24:in `call'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/rack-2.0.5/lib/rack/sendfile.rb:111:in `call'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/rack-mini-profiler-1.0.0/lib/mini_profiler/profiler.rb:285:in `call'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/message_bus-2.1.5/lib/message_bus/rack/middleware.rb:63:in `call'
/var/www/discourse/lib/middleware/request_tracker.rb:180:in `call'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/railties-5.2.0/lib/rails/engine.rb:524:in `call'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/railties-5.2.0/lib/rails/railtie.rb:190:in `public_send'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/railties-5.2.0/lib/rails/railtie.rb:190:in `method_missing'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/rack-2.0.5/lib/rack/urlmap.rb:68:in `block in call'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/rack-2.0.5/lib/rack/urlmap.rb:53:in `each'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/rack-2.0.5/lib/rack/urlmap.rb:53:in `call'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/unicorn-5.4.0/lib/unicorn/http_server.rb:606:in `process_client'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/unicorn-5.4.0/lib/unicorn/http_server.rb:701:in `worker_loop'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/unicorn-5.4.0/lib/unicorn/http_server.rb:549:in `spawn_missing_workers'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/unicorn-5.4.0/lib/unicorn/http_server.rb:142:in `start'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/unicorn-5.4.0/bin/unicorn:126:in `<top (required)>'
/var/www/discourse/vendor/bundle/ruby/2.5.0/bin/unicorn:23:in `load'
/var/www/discourse/vendor/bundle/ruby/2.5.0/bin/unicorn:23:in `<main>'

Still works fine on the main instance. (@gdpelican I invited you to the test topic.)

Seeing this, I suspect it might be related to SSO. But not: on another secondary instance, it fails but this one has no SSO configured.

Heh. @hellekin you are having the experience I am trying to avoid. :scream:

Sorry you are running into this and hope you get it resolved quickly!

1 Like

Iā€™m just procrastinating really :wink:

2 Likes

Alright, I believe Iā€™ve resolved the multi-site migrate issue. The topic you linked me to seems to be working fine @hellekin, although if you grant me TL2 on that instance (or set the trust level setting for Collude to 0), I can play with it a little.

Let me know if that gets you a bit further?

Done.

Did you commit changes to the plugin? In this case I can rebuild the container and see if that fixed the multisite migration.