Use of discourse for "crowd-sourced project evaluation"

I used discourse (via Communiteq (formerly DiscourseHosting)) for a rather unusual purpose - hosting a distributed evaluation of a global UN project. It worked pretty well, both from a technical and professional point of view. I wrote a review: here. Anyone else tried anything similar?

What is crowd sourced evaluation?

An evaluation report in its draft version gets sent around a bunch of stakeholders who make comments perhaps using a tracked changes feature, and the evaluator has to respond to those comments, sometimes in a procedure which gets repeated two more rounds. Over half the inputs and comments might be defensive or off-topic, but there are always at least a few which bring you vital new information and interpretation which I hadn’t picked up during the evaluation process.

I always thought:

How come I have to do all the work of writing an almost complete report & then wait for this highly valuable expert input which comes almost too late? Couldn’t I put these experts to work to write the thing for me in the first place?

So my idea for a crowd sourced evaluation is, essentially, to create an online version of the report, initially consisting of just the evaluation questions as sections and subsections, together with some initial evidence and ideas for answering them; then I send invitations to contribute relevant evidence, opinion and interpretation to help answer any of the questions. The invitation goes out by email to a much broader range of stakeholders than is usually possible, perhaps in waves, for example beginning and ending with invitations to the most important stakeholders.

8 Likes

Nice writeup!

It’s certainly true that discussion is a fundamental building block of all online community, and the “lego brick” that is Discourse can be quite helpful in a variety of “builds”. That’s one reason I was so excited to start on this project – if Stack Overflow was a scalpel, powerful but dangerous, Discourse is a table knife, generally safe and useful almost anywhere people are sitting down to eat together.

That’s why we always called Discourse a :rainbow: system.

I’d be interested to hear any specific pros or cons for this use case and ways we could make Discourse more flexible (or support plugins, if needed) to better adapt to the problem.

Technically Discourse is being used in a similar fashion to comment on the CommonMark spec at http://talk.commonmark.org too.

7 Likes

@Steve_Powell I use discourse for about every thing. From blogging, to backing up internet media, talking with people or making a ticket system for my now dead startup. Only reason I can’t say I’ve tried “anything similar” is because I don’t have a crowd to toy with.

(part edited out)

Anyway, to me the biggest block for using discourse today is not so much the cost for hosting as it is the time required to get a basic setup up and running as you’ve experienced. Although the cost also seem to be much bigger than what could be… At least for pet projects as I’ve being using it.

For any kind of crowd, I’d argue both those “blockages” are quite minor so, yeah, by all means, keep going for it! And share more of what you’ve done! :smiley:

Also…

I think @michaeld (from that Communiteq (formerly DiscourseHosting) service) would like to know about your usage case there! :slight_smile:

2 Likes

Well by the far the most useful thing for this kind of specialised use would be some kind of turnkey script so you a not-too technical user can just answer a couple of questions and get everything set up for them. It took me the best part of a week to get all the permissions etc right and it would be great to be able to do this in one step. Don’t know if it is possible though.

Apart from that, one big stumbling block was the sign-in process (the whole discussion was private) - people didn’t get that they had to use only an approved email address. Would be good to be able to say this even more clearly on the relevant screens.

1 Like

Can you provide more detail on the above, perhaps with annotated screenshots? Not really following.

1 Like

I just mean that when I invited someone to join in a private discussion via their work email address, and then they get to this screen, it isn’t always obvious to them that they need to use just the email address I have registered for them and not some other one.

5 Likes

I see, that helps. I wonder what we could do in that case, when you need a user to sign up with a specific email address. One idea is:

http://meta.discourse.org/login

This works to drop a user on the login page but what if…

http://meta.discourse.org/login?email=name@example.com

Adding the email address as a param, did the exact same thing but pre-populated the user field with the desired email address? @neil what do you think of this?

8 Likes

This is an awesome insight. I wished I could give more hearts to it! :smile:

1 Like

I think that would be perfect.

I tried to make this change today, but the Ember code to fill the field has me stumped. Will have to look into it further.

Perhaps @eviltrout could assist? This would be a nice feature, since if you invite someone via a specific email address, and they enter a completely different email address when signing up… that’s not gonna end well for anyone involved.

I figured it out. :sunglasses:

I added this to other routes where it will be useful to pre-fill the email:

  • /login
  • /signup
  • /password-reset

The password-reset could be very useful after an import so that users can be sent emails asking them to claim their accounts and set their passwords.

8 Likes

Excellent this is a great change thanks for the feedback @Steve_Powell !