How does the Discourse project work?

[mcwumbly (Dave McClure)]

No, meta itself runs on the tests-passed branch.

tests-passed basically is master. Its just that the unit tests have to all pass before changes committed to master are automatically merged into tests-passed.

[erlend_sh (Erlend Sogge Heggen)]

Naturally I’m in complete agreement with the “we need roadmaps & changelogs” camp.

Adding to that, my main concern for the Discourse project now is that not enough attention is being given to the ecosystem as a whole. The bulk of everyone’s time appears to still be spent on the product. While the core developers might be in their happy place, I’m not seeing the kind of developer adoption you ought to be seeing for v1.0 software.

• Where are the plugin developers?
• Where are the theme designers?

I keep hearing that the best kind of mockup is a functional one, yet the people most able to make that happen are not being facilitated. Discourse’s barrier of entry is still way too high, and not enough is being done to lower it. There’s not even a “Docs” or “API” link on the front page!

[probus]

First of all, I’m sorry that I don’t have time to properly answer everything. I also apologize to @awesomerobot for completely forgetting you, you’ve done some good stuff here. I guess that was because lately more often not the design changes seem to have be coming from other devs. That said, I think other users have said pretty much what I’ve though so there’s not that much to add. I will only comment the (short) example of gutter links.

What would have liked to see is something about the original design. Why the gutter links are there in the first place? Why are they presented like they are? What has changed to commanded the change? If there were complaints, what were they? There must be some new data or at least it’s interpretation must have changed. I don’t have access to any of that. That’s also why the discussions become bikesheddy. You can’t challenge the assumptions behind the changes because they are not made visible.

I’m not a design agency submitting work for you/us to vote on. I’d simply like to contribute in what I can and give my experience and input on things were I can. If I had the data, I’d be able to make my own conclusions, possibly gather some additional data or test the ideas. Only after that am I able to share my findings, ideas and designs back. That’s how I work.

[cpradio]

@probus

If I had the data, I’d be able to make my own conclusions, possibly gather some additional data or test the ideas. Only after that am I able to share my findings, ideas and designs back. That’s how I work.

That’s how a majority of us work. I spend a LOT of time doing analytics and statistics, and writing programs to do some pretty nifty calculations that ultimately lead to key business decisions. Having data is an absolute necessity. Without it, people make their own assumptions as to what the underlying issues/data are and build their arguments for/against based on that.

That definitely leads to confusion as no one truly knows what they are talking about. They have an idea in their head that the data they received was what lead to their stance on the subject, but fail to remember, they never had data to begin with!

[anon21958287]

A. Wanting to test a quick UI change without a lot of overhead

The problem with the way this is done is that it’s implemented in the main branch (i.e., goes through the normal flow so that it’s everywhere) and then even if there are ultimately plans to revert or fix up later, the later is a lot later. Meanwhile, stuff is removed or whatever apparently for the sake of experiment.

Shouldn’t something like try.discourse.org be used to experiment with this sort of thing? With branches other than the main stuff that everyone gets?

[downey (Michael Downey)]

The problem with the way this is done is that it’s implemented in the main branch (i.e., goes through the normal flow so that it’s everywhere) and then even if there are ultimately plans to revert or fix up later, the later is a lot later. Meanwhile, stuff is removed or whatever apparently for the sake of experiment.

Sounds like a good use of feature toggles?

[anon21958287]

Sounds like a good use of feature toggles?

Which ironically, is practically what existed in the first place, in the form of a snippet of custom CSS applied by an admin.

[codinghorror (Jeff Atwood)]

The problem with the way this is done is that it’s implemented in the main branch (i.e., goes through the normal flow so that it’s everywhere)

Incorrect. Change is implemented only in tests-passed… be on Stable or Beta if you don’t want experimental changes. What you’re asking for already exists.

Having data is an absolute necessity. Without it, people make their own assumptions as to what the underlying issues/data are and build their arguments for/against based on that.

I’d say you need to be active on at least 5 different reasonably sized Discourse instances to have enough data. (Not including meta, since meta is kind of a dodo bird in terms of feedback, it’s not really representative.) The team is unique in that we sit at the confluence of probably 100+ Discourse instances, of which I touch at least 10 per day, and probably see up to ~20 on any given day, including weekends.

There’s a serious “sample size of one” problem that you want to avoid. Your particular community may have certain… predilections and quirks… that perhaps may not be shared by many other communities. But from this vantage point, we see everything – we’ve read literally every single meta post since Feb 2013, most meta discussion that occurs on other active Discourse instances, and we sample ~10-20 Discourse instances every day, Monday through Sunday, every week, for the last year and a half.

I’m not seeing the kind of developer adoption you ought to be seeing for v1.0 software.

Inevitable with this highly complex stack – show me another all-JavaScript project running Ember.JS and Rails that has any kind of open source traction.

And to be honest right now I do not care at all about developer adoption, we need customer adoption over the next 12 months if we want the project to survive. Period.

Trends look reasonably good, don’t get me wrong, but the priority is on project survival via customer happiness, not any abstract metrics of developer happiness.

[cpradio]

I’d say you need to be active on at least 5 different reasonably sized Discourse instances to have enough data. (Not including meta, since meta is kind of a dodo bird in terms of feedback, it’s not really representative.) The team is unique in that we sit at the confluence of probably 100+ Discourse instances, of which I touch at least 10 per day, and probably see up to ~20 on any given day, including weekends.

There’s a serious “sample size of one” problem that you want to avoid. Your particular community may have certain… predilections and quirks… that perhaps may not be shared by many other communities. But from this vantage point, we see everything – we’ve read literally every single meta post since Feb 2013, most meta discussion that occurs on other active Discourse instances, and we sample ~10-20 Discourse instances every day, Monday through Sunday, every week, for the last year and a half.

Okay, but that wasn’t exactly my point…

My point was, there was zero sharing of any data that lead to the decision. It simply was, this place doesn’t have it and it works (but oh yeah, they at least have a “Related” title above them so the user knows that they are for…). If you had stated: Out of the 20 most popular Discourse instances, the glyphs are causing a stack load of confusion, we’d be having a different conversation right now, but I don’t believe that is the case (out of the 4 I know are popular and just did a search on the topic for).

I’m simply stating, that if you make a change and don’t provide actual information as to what lead to the decision, we are left to assume the data ourselves (and that isn’t a good thing). I don’t doubt that you have data at your disposal, I worry that the justification you provide doesn’t include the data.

[anon21958287]

Incorrect. Change is implemented only in tests-passed… be on Stable or Beta if you don’t want experimental changes. What you’re asking for already exists.

No, tests-passed is not the place for crazy experiments to do what-if sorts of tests. Not in the way I’m talking about. It’s closer to feature branches, really, to try things out and have a way to test it out more publicly than a dev’s environment.

[sam (Sam Saffron)]

I see room for feature flags, but this is not for “we just removed a glyph” it’s for “let’s see how it looks if we remove the whole aux column”

Requiring feature flags for every minor design change is paralyzing

If we make an incorrect call once in a while we can always correct / adjust before next beta

[dontdieych (YCH)]

I’m perfectly OK with that core dev team can make decision without public discussion. They deserved that power because they made cool thing at first. You and me, didn’t.

You and me also have power because we are user of Discourse and most software are meaningless without user. But these power are not in same level. We cannot taking them down. If you make cool thing, you will have that power. Please respect each other.

If someone have trouble with new change, of course they can speak about it at here. Look around this meta forum. Dev team’s posts and topics are everywhere. I have couple of Discourse powered forum in my bookmarks. They’re also popped there. The point is, they are actually listening from community very hard. If someone ask me “How it looks like become successful open source development?”, I would tell “meta.discourse.org”.

You needs more well-formed changelog, announcement or design document? Feel free ask about it and persuade us why we need it. But please don’t say “Discourse have problem with their process”. You have problem with some specific thing in your case, not Discourse development process.

[downey (Michael Downey)]

You and me also have power because we are user of Discourse and most software are meaningless without user. But these power are not in same level. We cannot taking them down. If you make cool thing, you will have that power.

Actually many people here on meta are also contributors to Discourse, which is a collective work created not only by the small “core team” (for lack of a better term) of volunteers paid to work on Discourse, but also by the ideas, discussion, and code from hundreds (at least 215) of others of volunteers. While clearly some have contributed more than others, they are all co-creators and co-builders of Discourse.

The point is, they are actually listening from community very hard. If someone ask me “How it looks like become successful open source development?”, I would tell “meta.discourse.org”.

Agreed that the listening is good here, but most large open source projects do not use (or at least try to avoid) a two-class system of contributors as it relates to communication and development. There are many more examples you might want to study.

[codinghorror (Jeff Atwood)]

I am sorry we are not living up to your expectations of how we spend our time. If you feel that you should no longer participate in this project based on your disappointment with how the project is being run, I understand.

Life is too short to be unhappy and you should choose to spend your time on things that make you happy. Sorry Discourse did not meet your goals. I hope you find a project that does meet your expectations. Good luck, and thanks for your contributions!

[cpradio]

sigh, I guess it is too difficult to explain what I’m trying to say. I don’t disagree with where you are going or what you are attempting to achieve. I don’t plan to stop trying to help meet those goals, but, I have no idea what the goals are.

Help us, help you by letting us know what you want to achieve and when. I feel that would help us both out. I’m just simply looking for clearer discussions between our differing views and you seem to make it seem I’m asking for something you don’t know (or can’t share, or something).

The glyph change is a prime example of this. When asked for “what problem are we solving”, I was given a very vague problem. How am I supposed to help come up with solutions if I don’t have a specific problem? I don’t feel like I’m asking for the stars here, just clarification.

I’m willing to help figure out solutions but when I’m not given enough information and I need to keep pressing for it (as if squeezing blood from a turnip), it makes the discussion difficult.

I’m sure you can understand that. Look at it from my point of view and it is clear to see that nothing in that discussion was specifically shown as to what problem we are addressing or the new problems that are now in existence because of the solution initially taken.

I’m not disappointed with my efforts or yours, I’m trying to improve that relationship here by offering why it feels difficult from a contributor/participant perspective. If you want us to all keep going in circles, that’s fine, but realize, that just leads to these discussions over and over again. I’m trying to help by bringing this discussion to light and see if we can’t find a way to resolve it so things progress quicker (a goal both of us should be reaching for).

I’m not sure what the issue is with constructive criticism (or if you don’t believe there is a problem with communication here). There isn’t any need to make this personal. I’m not. I’m purely pointing out the problems I see from a contribution perspective. Without knowing where you guys want to go next, it is hard for me (and everyone else) to help you achieve those goals.

[codinghorror (Jeff Atwood)]

If I had that information, I would give it to you. But I don’t. You seem unhappy. Perhaps there are other places you could more constructively spend your time?

[eviltrout (Robin Ward)]

I think the example being argued is exemplar of bikeshedding. It’s literally one character of formatting in quite a large app.

I actually really respect the criticism being offered here that we aren’t as transparent enough about our goals. I think a loose roadmap would be a good thing for us to provide. However, I sincerely doubt “removing character in the right gutter” would have made it on there anyway.

We can’t possibly be expected to open a new topic to debate every little UX change we make. Achieving consensus would be impossible and it would slow development down to a crawl. I am really not sure if people understand how much extra work they are asking us to do here.

I firmly believe for little changes like that, it is far easier to just change it in master and let users start playing with it. Sometimes it’s an improvement, sometimes it sucks. We can always put it back. We do this often.

[cpradio]

I actually really respect the criticism being offered here that we aren’t as transparent enough about our goals. I think a loose roadmap would be a good thing for us to provide.

I put this in a PM, but in short, I feel this is a really good idea. It doesn’t have to be written in stone, things will get pushed all of the time. But a list of short term and long term goals would make a HUGE improvement to allowing us contributors to see if what we are trying to submit, aligns with where the team is heading.

However, I sincerely doubt “removing character in the right gutter” would have made it on there anyway.

We can’t possibly be expected to open a new topic to debate every little UX change we make. Achieving consensus would be impossible and it would slow development down to a crawl. I am really not sure if people understand how much extra work they are asking us to do here.

I firmly believe for little changes like that, it is far easier to just change it in master and let users start playing with it. Sometimes it’s an improvement, sometimes it sucks. We can always put it back. We do this often.

I do to, and that’s fine. But a discussion was opened on it (during the experiment phase), clarification is key. We can’t just assume intentions and I feel like we repeatedly are asking for clarification after an initial response is made (example: the glyph discussion only got clarification after 3 responses were made by the team, if the 2nd response was https://meta.discourse.org/t/right-arrow-missing-from-outgoing-post-links/20117/23 it would have saved a lot of time).

Just my observation here.

[mcwumbly (Dave McClure)]

Allow me to briefly summarize, since I re-ignited this discussion 4 days ago:

• Many decisions are made within the team, based on observations and feedback across 100+ instances and their meta categories. Priorities are set according to the principles outlined in Jeff’s post above about complaint driven development.

• Many of these decisions may be made here on meta out in the open. If Jeff needs to make a decision when consensus cannot be reached in a timely manner, he does so.

• Small changes may be made to master / tests-passed by the core team without advance notice. Everyone has the opportunity to discuss those changes here on meta when they go live. If you don’t want to be subject to erratic changes, choose a more stable deployment branch (beta or stable).

• The team’s current priority is on customer adoption, not developer adoption.

• Community contributions are very welcome. Particularly if they solve a problem that is more important to you or the instance you run, which may not be a priority for others, but are not objectionable either. They are best discussed first here on meta as it’s necessary to get the team’s buy-in before it will be merged into core.

• It has been demonstrated that there is a path toward becoming more involved if you put your focus on solving real problems.

• The community feedback here has been integral to the changes and improvements made and the team pays attention to all of the feedback.

• There are differing views on how transparent the process of making decisions should be and how researching and discussing UX problems and solutions should be done.

• The team does not want to be paralyzed by discussing every change ahead of time on meta, especially for small things. There’s always the opportunity to make adjustments based on complaints and other feedback.

• There is some sympathy for the idea of being more transparent about their goals in the form of a loose roadmap. Perhaps we’ll hear more about this in the future.

[dontdieych (YCH)]

Actually many people here on meta are also contributors to Discourse, which is a collective work created not only by the small “core team” (for lack of a better term) of volunteers paid to work on Discourse, but also by the ideas, discussion, and code from hundreds (at least 215) of others of volunteers. While clearly some have contributed more than others, they are all co-creators and co-builders of Discourse.

I believe those 215 people’s contributions give more power to core team. Those contributions didn’t take power from them.

I’m afraid of you would accept ‘power’ as ‘dirty politics’. Please accept it as ‘trust’ and ‘believe’

All of us, Discourse user, especially 215 contributors are continuously giving them trust. So it would be quite rude if anyone saying “OK, you guys well done. But, from now on, you must show me exactly what is going on here. What is next plan? Give me briefing right now.”

Agreed that the listening is good here, but most large open source projects do not use (or at least try to avoid) a two-class system of contributors as it relates to communication and development. There are many more examples you might want to study.

I don’t believe there is two-class contributors. There is one development unit and others. Think Discourse team as one unit. Why? Because they build this thing as one unit at first. The unit is communicating with us quite well. I don’t think we should look at their every single brainstorming. If we do, we would be looks like …