Discourse index


(Joffrey Jaffeux) #1


After working on the Discourse Notifier mobile app I realised, one killer feature for the app would be to improve discoverability. A shared index would help with this (and would also help when you know the forum but cant remember the URL).

MVP app would have:

  • a user facing form to register your site. The form would only have a URL field, and we would retrieve data automatically
  • a GET /sites API endpoint with a q params to filter results and probably some pagination logic


  • implement create/update api endpoints to allow some automation, like auto register/update your board to the Discourse Index

What do you think? Am I missing something and it’s already possible to achieve this?

Please visit our Discourse Forum! (Directory)
Discourse mobile app search engine
Global Discourse Site Directory - mockups
(Rafael dos Santos Silva) #2

This sounds like #discourse_hub 3.0 :smile:

(Joffrey Jaffeux) #3

indeed Global Discourse Site Directory - mockups !

(Joffrey Jaffeux) #4

not sure where the discourse_hub is atm, I can read many different ideas and doesnt seem to be on the roadmap, do you have more info ?

(Rafael dos Santos Silva) #5

The conversations about it are from before my time around here mostly, but looks like much of it was postponed.

(Sam Saffron) #6

One big problem in that department is curation of the list.

  • Do we want to display forums with only 3 users and 5 topics a month?

  • What happens with racist,hate speech and other types of forums that we are uncomfortable promoting? Who decides?

  • Do forums we host get a bump in the list?

  • Do larger more active forums get a bump?

Another problem is how we deal with taxonomy here, clearly there are families of forums so browsing by category etc would make sense

(Joffrey Jaffeux) #7

All valid points indeed. Dont’t see any easy solution here.

(Erlend Sogge Heggen) #8

Nope, there should be a cap. Any site still gets to opt in but getting featured is not guaranteed.

Our index, our rules; we decide.

Don’t see why they should. I do however think we should have a concept of “Highlighted Communities” which get prime real estate, e.g. as illustrated in one of @rewphus’ mockups:

We’d highlight sites that are good examples of something, e.g. good design, good community culture, good stewardship of a forum at scale etc.

Nah, not in any way other than potentially getting highlighted.

I’m sure we can come up with 10-20 tags that will be generally applicable to the majority of Discourse forums. Here’s my suggested approach:

  • Every site gets to choose between a maximum of 3 tags
  • One of those tags will be “other”. When that is one of the selected tags we could show (I’m assuming this interface is in /admin somewhere) a small notice like:

Can’t find a fitting tag? Feel free to mail your suggestion to directory@discourse.org

In other words we’d automate only what we have to. Display would work like on the Middleman Directory:

I’m all :heart_eyes: about this. I’ve been wanting a more exhaustive overview of our many “partners” for quite some time. “What are some good example Discourse sites?” is also a very frequent question here in the community.

This is jumping ahead a little bit, but I just wanna clear up real quick:

Do you have some tech in mind already? Some sort of lightweight single-page application?

Just wanted to point out that at least the v1 of this sounded like something like Jekyll + jekyll-get could handle with ease, but

(Joffrey Jaffeux) #9

thx for the awesome answer!

no I dont have any tech in mind, as I said in my first post my basic goal is to improve discourse notifier app by providing better discoverability and autocomplete, so I need one api endpoint to accomplsih this. Im not very familiar with jekyll, but with a server, a db and some programming language we can certainly accomplish this.

(Joffrey Jaffeux) #10

@erlend_sh I’m considering working on this to showcase some frontend work, would you want to mentor me on this? My only requirement would be “no ruby”, I have enough projects to showcase with ruby, and would like more projects in JS (whatever framework) or Elixir/Phoenix/Elm.

(Jeff Atwood) #11

To fit with our stuff it would need to be Ember. We recently upgraded to Ember 2.10 which is smaller and faster.

(Joffrey Jaffeux) #12

Ember is fine for me

(Jeff Atwood) #13

Good news, then, as we are all up to date and recent / modern Ember examples and docs will all work!

(Erlend Sogge Heggen) #14

I’d love to help spec this out, but I’m not a programmer so I don’t think you’d want my guidance on that part :stuck_out_tongue: I’m sure @sam or someone else from the team will be able to lend a hand on this though.

Let’s just make one thing clear right away:

We are now talking strictly about a web app, correct? I.e. a directory app that would live somewhere like directory.discourse.org.

Just need to make that clear, since I realised after my initial reply that your original post was actually about extending the Discourse Mobile App, and not the general purpose directory that I started talking about.

Incorporating a directory into the app would be great, but I think it makes sense to build this as a web app first, because:

  • We can allow all sites, not just those hosted by Discourse.org
  • It’ll be available on both desktop and mobile

(Joffrey Jaffeux) #15

@erlend_sh ok no problem if you are not a programmer, we can find someone else, or I can just start and improve with time :wink: But I do need you for the user facing part.

So my general idea is:

  • an API to improve discoverability in the DiscourseMobile notifier app
  • to have data for this API we need something to fill it, so yes probably some kind of form and as I said earlier, we could maybe have an API endpoint so people can directly opt-in from Discourse admin to appear in this index
  • and then I discovered that you wanted to display this data to users via the “discourse hub”

So as I see it, we need a frontend (sites listing, form, auth) and some backend with few api endpoints and a database to store the…data.

Concerning the backend, I said “no ruby”, but if it’s easier for you to host we can use ruby, I would advocate for elixir or node, as this is where I want to gain xp, but I’m open, and I would understand you would want to keep your stack simple.

Does it make more sense for you?

(Jeff Atwood) #16

Unlikely @sam as he is moving countries for the next 2 to 3 weeks.

(Sam Saffron) #17

I would say if this is something we are maintaining long term we might as well keep with the stack we are used to hosting.

I would vote for Rails 5 API, no sprockets (webpack/ember-cli/rollup) frontend should probably be ember though personally I would be comfortable with vue or something else.

We can easily provide you with a test instance on digital ocean if you would like to show stuff off.

(Joffrey Jaffeux) #18

Ok sam let’s do it, ember or vue.js doesn’t make any difference to me, your call.

(Robin Ward) #20

Personally I care more that it’s in ES6 than whether you pick Ember, Vue or React, if it’s a totally standalone app. I would probably vote for Ember since it’s what we use in several apps already, but it’s the Javascript code format in the front end that really matters to me.

(Joffrey Jaffeux) #21

ok thx :wink:

Then I might try Vue.js, I have already tried Ember and React. Might be a good opportunity.