WYSIWYG Plugin (again)


#1

Continuing the discussion from Who would prefer a standard wysiwyg to markdown?:


I checked with the founders and they are ok with us restarting this thread with a view to discussing the development of a plugin to deliver some form of WYSIWYG post editing in Discourse.

Bearing in mind @sam’s & @codinghorror’s previous comments that the core team would not consider undertaking such development for the foreseeable future, what remains is the opportunity for the rest of us to get together and produce the plugin ourselves.

That’s assuming that there is enough genuine interest in which to make this project viable.

Assuming there is, how we go about this would be our next consideration. Do we have the required skill sets and sufficient time to make it happen? - or if not, are we prepared to form a small cooperative in which to raise the necessary funds to make it possible?

Sam has already pointed out that this is not an easy task - so let’s not fool ourselves that this a 2 or 3 week project - with obvious ramifications on our time and potentially our pockets.

But if we keep our objectives modest, at least to begin with, then I think we stand a chance of delivering something which would, in my view at least, greatly benefit the wider Discourse community. And of course, given it’s an optional plugin, we’d hopefully avoid discussions relating to bloatware, bandwidth and server requirements that seem to thwart many initiatives of this type.

Before passing this over to the community to discuss, here’s a few of my thoughts that have been developing over the past few weeks.

1 - How much can we achieve by enhancing the existing setup? By that I mean building on top of the existing UI, perhaps with a slightly better editing experience - but importantly, keeping Markdown as the formatting language. I’m deterred from this approach because if any low hanging fruit existed in this area, the founders would probably have added it one quiet weekend. But we should explore this in case we can apply the 80/20 rule and get a reasonably big hit for not a huge amount of effort.

2 - If the above (1) is not viable and, or desirable, the worry is once we step outside of Markdown (which Sam alluded to below), the impact on other areas of the code base would be significant. But that shouldn’t deter us, because if we believe WYSIWIG editing is worth the effort, then we need to find the path of least resistance whilst still achieving our objectives. And besides, there may be an alternative way of thinking (see 3).


3 - I probably should have started with this, but what we mean by WYSIWYG is key. I see it as an enhanced editing experience allowing complex HTML structures to be created. I don’t see it as simply allowing the user to select bold more easily. If it were, we’d take the route described in scenario 1 above.

The Barcelona & Tenerife post below is an example of what I mean.





This is taken from a prototype that I developed a few days back and it’s saved away on the server and renders exactly as you see it above - although I cheated by HTML encoding the HTML and enclosing it in an <a ...></a> wrapper, which was then decoded during the post view render event below - note the following supports multiple encoded wrappers,.

 $(this.element).find('.cooked a[href="#decode"]').each(function (index, element) {
                  
                    var encoded= element.innerHTML;
                    var decoded = $("<div/>").html(encoded).text();
                    $(element).replaceWith(decoded);

               });

It works very well, but it’s a hack and not very Discourse like - although something similar could? be considered.

In case you were wondering, there was no editor involved above - I see that as the simpler bit. For this example I simply copied source HTML from a WYSIWIG editor, HTML encoded it, wrapped with an a tag and pasted it into the post, before saving. Note, that I did manage to leave the existing post contents (shown in red) untouched because only the escaped (a tag wrapped) components were pre and post processed - the rest was left as was. It also quotes very nicely because the HTML in the quoted text is removed during the strip and bake process.

4 - I think the above is overkill for comments, but I would like to see something like the above post made available when creating topics, either limited to specific categories and or against specific trust levels. This also implies that comment creation and editing would not typically be provided with this full fat solution.

Some other issues to consider, but perhaps to be picked up and added to by the community:

5 - What editor?

6 - How is the HTML saved away. Raw? encoded as above …? or?

7 - How are these posts rendered in search, emails and other activity summaries?

8 - What other considerations are there for the rest of the system?

10 - How do we handle images?

And so on.

Over to you. Is there interest and if so, how do we go about it?


Ps. Please bear with me on this, I am fairly new to the platform which I believe to be the best available. Bar none. So please do tread lightly with me if I’ve misunderstood one or more aspects of the system and how they work.


How to integrate a stronger markdown editor other than the basic one
Does civilized discourse entail tone policing?
(Mittineague) #2

Unless I’m very mistaken, it is planned to move from MarkDown to CommonMark sometime in the future.

I don’t know how much difference this will make, but it should be kept in mind before getting too deep into any code that involves working with MarkDown


(Tom Newsom) #3

(Ground rule of this Topic: No Politics!)

1: So, the problem with 1. (and it’s “the problem” that Jeff identifies in his anti-wysiwyg essay) is that in a strict markup+preview model, you can’t accurately translate the cursor position from the preview to the markup. It becomes very easy to make broken/interleaved tag pairs, especially with markdown’s identical opening and closing tags.

The “best option” identified from the previous thread, ProseMirror gets around “the problem” by using an intermediary format that has none of markdown’s confusions. It’s rigidly and flat, and works as you’d expect a rich text editor to work. Try as I could, I couldn’t make it break by bolding, italiccing, listing and de-listing arbitrary chunks of text.

It has nice ToCommonMark and FromCommonMark functions but they don’t support all the features of Discourse’s editor (most importantly, HTML) so as we were warned, it would only be “flawless” on a blank installation.

3: I guess my no.1 concern with such a scheme would be: What happens when/if the plugin is removed? What happens when you try to edit a post that is actually a big blob of html? What happens when you edit a pre-existing markdown post with the new plugin?

I admire this effort, and might even contribute financially if it comes to it (and it appears doable). But I’m not optimisitc. I think Sam’s warnings are real (and he should know).
The problem has so many edge cases, there’s barely anything left in the core :smiley:

I think a “first shot” should be an implementation of ProseMirror, that sits on top of the existing editor, reading and writing to the editing area with FromCommonMark and ToCommonMark - It’ll be clumsy and I’m sure performance will take a hit, but it’s probably a realisitically achievable goal.

I have absolutely zero applicable skills with which to attempt any of this :stuck_out_tongue:


(Joshua Rosenfeld) #4

I also will not be able to help with this plugin’s development from a coding perspective at the moment, but I would like to suggest that instead of developing a plugin that replaces the markdown editor, something similar to the MediaWiki Parsoid + Visual Editor route be taken. Parsoid (in my limited understanding) is a parser that is able to “translate” between wikitext and a format that can be used by the WYSIWYG (Visual) editor. What this does is allow for users to choose between the Visual Editor and the “Source” Editor, although one cannot change back and forth while writing/editing a post. The advantage of this (as I see) is that one can use the Visual Editor when possible, but can always fall back to the source editor when needed. It also allows for one to use “raw” wikitext to do things the visual editor cannot.

Doing something similar here for Discourse would have numerous advantages. First, if developed correctly, the WYSIWYG editor could be deployed even when not working completely, as you could always switch to the built-in markdown editor if needed. This would allow for a much wider range of testing, as more sites might be willing to use it knowing they can instantly fall-back to a working system. Also, it gives users more of a choice, and could theoretically be integrated into an existing site. As it essentially would parse the text/images into markdown, a user could choose to not use it without issue, and if it is able to parse markdown into the visual format, it doesn’t matter what site it is on as the database/back-end wouldn’t know the difference.


(Talin) #5

As one of the more vocal participants in the previous discussion, I have a lot to say on this, so be patient :slightly_smiling:

First of all, on the feature set: I’m actually somewhat opposed to the idea that we allow open-ended creativity in terms of creating complex HTML, in favor of a more restricted set of styles. Here’s my reasoning: I strongly support the mission statement of Discourse, which is “civilized discussion”. I think that to the extent that styled text supports that mission we should allow it. But this is not a word processor or desktop publishing software. In particular, I don’t want to see people go crazy with fonts and colors.

What I would like to see is styles that correspond to useful semantic meanings. So for example:

  • headings are important because they break up text into logical sections.
  • block quotes are important because there’s a lot of quoting in these discussions.
  • bullet lists are important because they help organize thoughts.
  • images are important because a picture is often worth a thousand words.
  • source code formatting / preformatted text is important because some ideas can’t be expressed easily in a proportional font.

But things like pink text on a green background? Or comic sans? I could easily do without either of them. Ideally, one might think of each post as an academic publication, where variations in style from one paper to another are frowned upon.

Now, as to the engineering details:

  1. One of the objections cited was the difficulty of round-tripping between markdown and HTML. However, that problem can be solved, I think, by only supporting a strict subset of HTML. For example, Markdown supports tables very poorly compared to HTML. So the solution is to either disallow tables, or to don’t convert them (i.e. embed the HTML table directly into the Markdown as HTML blocks).

  2. One major area of concern is the plugin API. A WYSIWYG editor might not have the same DOM structure as the existing compose editor, and this might break some plugins. I haven’t really looked at the editor plugin API, so I don’t know how isolated the plugins are from the editor internals. Someone who knows this stuff better might be able to chime in here.

  3. CKEditor has a Markdown plugin which allows the editor to save in Markdown format instead of HTML. However, CKEditor is also huge and has tons of featured we don’t need. I only mention it here as an existence proof of what could be done.

  4. I’ve been doing a lot of my own experiments with TinyMCE. Before I started looking at Discourse, I was working on my own forum server using Meteor and React, so I’ve had a fair bit of experience with it. TinyMCE does a lot of what you would want, with a fairly low footprint, although the API takes some getting used to. One of the nice things is that they host it for you, so all you have to do is include a link element in your HTML. (Although if you are going to be doing any serious customization you have to download and host it yourself.)

TinyMCE also supports several different editing modes which are interesting. There’s an ‘embedded’ mode which takes your textarea and replaced it with an iframe which has a standard-looking toolbar. There’s also an ‘inline’ mode which works on block elements (not textareas) in which the toolbar kind of floats above the editing window. (Medium also has a WYSIWYG editor similar to this, but it only supports like 6 styles I think.)

  1. There’s also Summernote, Froala and others. Summernote is not nearly as polished as the ones above, the docs are written in very poor english, however if you were going to write your own editor it might be a useful starting point since it’s smaller than the others and has fewer features.

The question with all of these choices is whether they support the exact feature set that you want. So the first thing, I think, is to decide exactly what styling features are must-haves, nice-to-haves, and so on. If none of them will work then it’s time to start thinking about writing your own content-editable.

  1. Another issue to consider is sanitization, although I suppose that Discourse already does this. Both CKEditor and TinyMCE will strip harmful attributes and tags from the input, however you still have to sanitize on the server to prevent people from bypassing those safeguards.

  2. TinyMCE and CKEditor also support callbacks for image uploads - that is, pasting an image from the clipboard and having it turn into a URL which is hosted by the site. (I was please to see that the current discourse editor supports this. I was even more pleased that it works with SVG files, which is very important to our users who frequently share graphs and charts related to clinical trials.)


#6

Thanks, much needed correction.


Any more input from anybody else before we start to address comments made so far?


#7

I’m putting my investigation into the above on hold until I get a better understanding of the new Plugin API and topic virtual DOM architecture

In the meantime, please don’t let that stop you continuing with your discussions should you wish to…


(Robin Ward) #8

I should mention that the virtual dom only affects how posts are rendered in a topic. It shouldn’t affect any editor code you create.


(Muíran Shíoraí) #9

I’m 100% on board.
Have not discovered a FOSS plugin for a Discourse or WordPress WYSIWYG Editor
I also hope collaboration and that the spreadsheet idea supports the inclusion of higher math functions for notational logic for scientific ideation and communication. I vote for having the ability to toggle between the standard Discourse and the proposed WYSIWYG Editor functions.


(Rafael dos Santos Silva) #10

This has been discussed before:


(Muíran Shíoraí) #11

'would always cause annoying reformatting of markdown."
Gee, that is a seriously odd point to assert, unless you are sure the people working on the project don’t know what they are doing. I just spent several hours study on various WYSIWYG Editors.
Out of the 15 most popular ones used today only one doesn’t support toggling back forth viewing and editing.

I’m designing a prototypal site for a non-profit, scientific community of practice, a new kind thinktank.

Assuming I get this site up and running, We are going to play around stitching various app feature combinations of WordPress/BuddyPress/Koni.Wiki together with Discourse.

After I’m satisfied the community’s needs are being sustainably addressed, the remaining work will be to streamline the code, stripping out features that don’t serve the mission, now or ever.

Then test it to destruction. Rebuild and do it again and again until we have tuning we can provisionally trust.

Anyway, we need standard communication with a WYSIWYG editor because it’s 2016?

I mean I can’t believe it’s a debate. Anyone, paying attention to tech over the past 5 years knows there isn’t a smartphone built that doesn’t have a WYSIWYG text for sms and email and a dozen other apps.

Call me impractical, but I believe a website featuring protracted text entry, should be at least as advanced, as a 5-year-old smartphone.

Side comment: Is Babble Chat ready for prime-time?

This is the project description in Google Drive
https://goo.gl/yIOUAW

In addition to regular correspondence, we need mathematical notation/markup for scientific ideation.
This is the editor I’ve settled on. I have one other programmer onboard and looking for more.
WYSIWYG = CKEditor


How to?

  1. Swap-out Discourse Native Editor
  2. install and assign CKEditor as default.

(Rob Nicholson) #12

Not got time right now to read the entire thread but big bump up for this. We’ve recently launched Discourse for CAMRA, a consumer organisation of beer & cider drinkers in the UK with over 170,000 members to replace phpBB. It’s not yet been launched to the membership as a whole but I felt compelled to return to Meta to put a plea in for WYSIWYG.

These are not technical users with the average age being over 40. The lack of a WYSIWYG editor is the biggest problem with Discourse. It’s the single consistent thing the early adopters have complained about. It’s seen as a blocker to adoption.

Heck, I’ve worked in IT as a programmer and IT manager for over 30 years and even I don’t like mark-up for this purpose. I hate having to keep switching my focus between left and right panels. When I use italics using ctrl-I, I always forget to move over the trailing underscore - that’s horribly user unfriendly.

So yes PLEASE implement a WYSIWYG editor with some urgency.


(Rob Nicholson) #13

Okay, so I did read a bit more :slight_smile: A limited range of rich formatting is perfectly acceptable - in fact, the ones shown in the toolbar above as I type this is perfectly acceptable. I’ve just tried the numbered list icon by the way - very unfriendly IMO. Do you have to number the list yourself?


(Tom Newsom) #14

Yes, but you don’t have to be sequential

1. One
5. Two
2. Three
4323456834639. Four
  1. One
  2. Two
  3. Three
    4323456834639. Four

(Rafael dos Santos Silva) #15

You may want to reconsider the migration if you need urgency on this.


(Gerhard Schlager) #16

You could even use the same number over and over again. That’s what I’m always doing…

1. One
1. Two
1. Three
1. Four
  1. One
  2. Two
  3. Three
  4. Four

(Tom Newsom) #17

Now explain it to 80,000 middle aged real ale fans :slight_smile:

@Rob_Nicholson - Discourse is not getting WYSIWYG any time soon. I’d dearly love to see it too, but I’ve reached the acceptance stage of grief about it now.


(Rob Nicholson) #18

Sadly then it falls well short of it’s grand aims then to be “Simple. Modern. Fun” doesn’t it. I’m well aware of all of the discussion on this and sadly my take is that Discourse will be part of the reason why this reboot of CAMRA forums may fail. The final decision wasn’t mine and whilst I initially suggested looking at the platform, I’m beginning to wish I hadn’t…


(Rob Nicholson) #19

Sadly, whilst I suggested CAMRA looks at Discourse because I hoped it really would live up to it’s aims, I wasn’t the final decision maker.


(Rafael dos Santos Silva) #20

Having a mostly 40+ old non tech community too, I don’t think this is the big deal people make. 99% of posts are text, and text is the same on markdown or wysiwyg.

But if this is a big roadblock as you say, you can always put some money into a #marketplace topic.