Questions about the technical architecture of Discourse

Dev
1

Hi

Long ago I was a techie but not these days. I’ve said elsewhere I’m interested in the design of digital community. Sociology built with technology

I don’t know what the architecture of discourse is but I’m aware it has themes and plugins. If there are other bits I’m ignorant of those

Help Pls :slight_smile:

I’ve tried searching on the pair in various ways but I don’t get anything returned that suggests “here is the 101 on what/ why to. EG: there is a presentation layer, a storage layer and a selection process between the two that uses This information to establish context to select from one into the other. Out of the box you get x y and z. a plug-in is a… any in directory will be loaded at initiation of the server process (or whatever) they hook into UI/ back end/ front end etc by…, the ones that exists are…, A theme affects the presentation layer(?) of the out the box plus plugins in the following ways…”

Is there such a description that somebody couldn’t signpost for me to and will take me from 0 to sufficient comprehension to be able to find what’s currently in existence and have enough understanding to do some imaginative thinking

because

then I can use them as conceptual building blocks in the design of community for different purposes .

I’m interested in…:
whether discourse could be ‘hardened’ to meet #medical_grade standards - no I don’t know what they are in any particular jurisdiction yet, but I know there will be necessary for the purposes of interest and different across geographies.

I’m interested in interfacing to instrumented rehab tools and coding for anonymized data extraction (gdpr compliant) of progress and efficacy metrics has improvement data for research.

I’m interested in how the best of the best posts can rise to the top like cream so that new users aren’t faced with a discourse UX that is alien in a subject that they have newly discovered an acute need for at a time of disorientation with a body of information in front of them that runs to thousands of posts and threads with only a hierarchical structure that is academic and not visceral for them…

Hierarchical structures generally are better trawled for topics of potential interest if one can take a breath first search pattern but generally only support depth first search patterns - discord appears to be a case in point - where the presumption is one already knows the subject - that might be slightly ameliorated by hashtags which is another topic on my reading backlog

Maybe some of the AI bot stuff bridges are gap? Another backlog item… :slight_smile:

I’m also going through some videos via the blog with @jonobacon and maybe able to ask more intelligent questions when I finished them

In order to think creatively within the context of discourse I need more understanding of what components and what interfaces exist

Can you help signpost to me :slight_smile:
Thanks in anticipation

2 Likes
2

Having a browse of the existing plugins, themes, and theme components may give you a sense of what’s possible customisation-wise:

6 Likes
3

:wave: Hi @51mon - if you’re interested in Discourse forum development the info here may be of interest to you. :slight_smile:

2 Likes
4

Thanks :slight_smile:

I think you just gave me a dictionary when I hoped for guidance about how to read ! - have you a gentler introduction on the topic of what these things are and how they work rather than what they do :-)?

There is lots of stuff in your links but there’s nothing in overview that says what the architecture of discourse is - I’m less concerned with what capabilities are provided in the plugins etc at the moment - I’m still trying to work out what they actually are

2 Likes
5

Thank you I will have a look :slight_smile:

I am a bit reticent since they are headed “developing…” and I’m not interested in writing code - I want to understand the architecture

I’ll look because it’s not obvious at the moment whether the texts will meet my need (fingers crossed)

I’ll let you know - unless of course you have some other suggestions :). ?

1 Like
6

@51mon I assume you have studied these?

7 Likes
7

I hadn’t :slight_smile:
Thnx

~~ "~~
[[Edits]]

Yikes! Link number one takes me to the header for 15 threads

Link two and three are sales pages but at least there only one page - there feature lists not architectural descriptions though.

And yikes I’m not even sure what last link is except an invitation to login to try.discourse.org with has threads with more than 100 entries against plus one with more than 1,000 on its own -

no map. Must be a bit like Cook felt when he spotted Australia and then spent days sailing along its coast

Edit
The features list

The page names a long list of things. Some of the things are conceptual. When a name refers to a concept that isn’t described all it tells you is that there is a thing here but doesn’t convey any understanding or what the thing is - so I know from the features that there are a lot :slight_smile: but it don’t have a concept of what I can do from many of the named features.
Good for establishing breadth, a good aid memoir for those who already know, but hard for somebody questing for understanding to gain understanding.
Also I have no idea from this features list whether they’re plugins, themes, out of the box, unique to discourse or common in every platform
I’m not saying that I should do for that page really at this stage I’m just making notes because I’ve got a continent to explore. What my OP was asking for was a picture of the earth as taken from the moon so far I haven’t even managed to climb to the top of HMS Endeavor’s mast :grin:

Link number three “what is discourse” is a mix of sales pitch with other things. It doesn’t give architecture either - I’m not saying it should but since it doesn’t it isn’t what I I’m looking for.
It also doesn’t quite ring true for me - for example it claims discourse got rid of complexity however the colour red that’s used in the UI changes based on the number of likes vs posts. That’s a very esoteric element of the UI. Also whether you click in the lists of unread posts on the title, the reply count or a third field I don’t know how to name you get taken to different places in a thread. These are all unintuitive, unannounced unsignaled variations in the UX that until you know that they exist in the UI will be confusing.

2 Likes
8

To be honest @51mon, I don’t think I understand your question. Could you ask it in simple terms, and I’ll try my best to give you an answer.

8 Likes
9

That’s a kind outreach @JammyDodger - and I will when I can but I can’t at the moment .

I think my initial post said it clearly in my ears but clearly it doesn’t convey it clearly to your ears (eyes!)

Let’s try this

If we were both standing and looking at a car then we could have a discussion like “you can sit in it and it will take you to the supermarket or the beach”. Or alternatively we could have a conversation like “it’s got wheels and an engine that’s powered by petrol or a battery and a steering device and a breaking device”.

The conversation about the brakes and indicators and whether the interior light comes on and goes off automatically is a developer conversation. The conversation about going to the beach and taking the kids to school in the morning and whether or not you can get wheelchair in the back or tow a caravan is a user oriented discussion that borders the technical. To tow a caravan you need a hitch installed because that isn’t out of the box. When you buy the car you got a choice of white or grey or black or maybe blue They’re one-time choices -maybe they’re admin topics in the context of discourse.

Now one aspect amongst many I am interested in is could I power a community with discourse where the information that sits somewhere across admin and accumulation of contributors posts and the user IDs etc etc might need to comply with medical grade regulations for the protection of data CIA (confidentiality integrity and availability). But I can’t assess that question when I don’t know what the underlying architecture is .

If this hasn’t made it any clearer you’ll have to wait till I’ve read all the resources that people have signed posted to me so that I can ask a question that is intelligible to those who can answer it because currently I can’t frame the question any better
Thanks - :slight_smile:

It’s a voyage of discovery and I’m content that it will take some time. When I’ve made the discoveries I need I’m quite likely to write them up and donate them back to the community here in the hope that they help others in the future

10

I’m afraid that has not made it a whole lot clearer. :slight_smile: Hopefully someone else can jump in and give you what you need. :crossed_fingers:

Have you created a forum to have a play around in yet? That may help you explore your ideas and get a feel for what’s possible.

5 Likes
11

Hi @51mon.

I suspect part of the problem is metaphors. Anything one types will often be understood to the Author. However others may have difficulty understanding.

So… here goes:

Discourse Meta is designed at the core to be a base community framework.

It provides a basic foundation…That out the box kind of bland.

With your age group and mine you might have been involved or aware of Dos Electronic BBSes.

A fair number were derivatives of Telegard BBS aka Telegard Hacks. (drifting sideways…)

Back on track.

So you want to build a Discourse Meta forum for…

This is you starting point… Sit down an figure out an idea of what your community’s purpose and what you want to have it offer in terms if function & features.

Then take 1 step back.

To get a clearer idea of where you can go. Explore the list of communities already using Discourse. Atm don’t have a link off hand. Checking out existing communities will give you an idea of what can be accomplished…Abd how similiar some communities are to how very different they can be in scope and function.

From here you can jot down features different communities have that you want in your community idea.

You can decide if you want paid hosting that can make quality if life easier or go full control and self host for total control of powerful extensibility of choosing whatever plugins you want. But at a cost of maintaining the vps yourself.

Either way you will need to research what you will need to have the functions and features available to design your community.

So from here let’s perform a thought experiment. Provide a sample community idea. We can even use an existing community and ask us what plugin (if needed), theme and theme-component would we need to accomplish what this community has?

1 Like
12

Hi @Heliosurge
:slight_smile: forget those BBS I remember newsnet run by UKC at Canterbury over dial up lines at 300 baud and I remember NNTP based services & clients like freeagent

I completely agree that analogies and even more broadly just words when arranged in sentences are often only encapsulating understanding between the ears of the writer. Even if they create understanding elsewhere there’s no guarantee that it is the same! :slight_smile:

Let’s try this
I understand discourse has a server end that contains a database of posts and of users and both posts and users have attributes and are linked together. That this is hosted somewhere, the host might be virtual, a content delivery network etc

I understand the client end there is a presentation layer that uses the posts and a users authentication credentials and access rights and links to create a feed. I see that philosophically many of the structures used are hierarchical. The hierarchy is obscured by many mechanisms for cross linkages

I presume - I think safely - that the colour scheme et al used by the client end is bound at execution time to a theme that can be replaced so that subsequent execution binds to a new theme and doesn’t change content but does change presentation look and possibly even some functional navigational stylistic components but I get hazy and speculative for this point

I’m aware that theme components exist and I’m currently holding a model my head that says a combination of theme components makes a theme but again I’m a bit shaky on that. I don’t have a grasp of some examples of what a theme component does

I have less understanding of plugins. I get the architectural bit that they are server side and so sit in the running instance that is serving content. I don’t quite see the interaction between a theme and a plugin. I would guess if you extend the server side capabilities then you have to extend client side presentation but that bit is uncertainty on the uncertainty .

I also don’t know if the full architectural model is made of: server, serving software, client app/ browser, theme components, and plugins or if there are other components

I have questions such as “a standard discourse community doesn’t set behind a pay wall or other monetisation mechanisms could one be added?” The data is stored in a system with some degree of of the triumverat of CIA - confidentiality integrity and availability. Can that be hardened to the standards required for Healthcare interoperability standards on both sides of the Atlantic

This latter requirement moves the conversation towards ethics & how software supports legal moral cultural concerns I think is on topic for this forum. Architectural understanding is a prerequisite too solution design as is determination of philosophical principles that shape the facilities considered desirable essential and undesirable/ unacceptable/ censure-able & censure should be processed.

1 Like
13

Okay you are getting it. The core database of topics grouped.in categories and organized with tags for example here.

There 2 kinds of theme

  • Basic just modifies base Discourse look
  • Full Theme modifies look and is prepackaged with some theme-component to alter what it can with components to change how things work

This pic. Summon Sidebar menu

Screenshots_2023-09-10-05-42-44
Screenshots_2023-09-10-05-42-44806×1651 122 KB

At the bottom where I highlighted with Arrow Expand it. You have a collection of Themes from basic to full. Experiment choose one and see how Meta here is altered in terms of look and feel.

Try a few different ones and explore. I think this will help gain better understanding of what Themes can do with and without components.

Air Theme for example is full theme with pre installed components.

2 Likes
14

Thank you
I think this reply has shown me that themes are loaded into the server side and that the client side on startup queries which themes are available and then with each retrieval of data (or instantiation of a session?) to be presented provides a theme id that the server then uses to encode what HTML elements will be forwarded to the client to display

Also you’ve said the app needs to be rebuilt. I assume that’s a link edit process? Least ways you’re describing a static mechanism Not dynamically linked libraries?

Still don’t have any feel for what services are plugging might provide and how a theme would interact other than by changing my icon from a circle to a square which seems to happen when I selected the one that you named was it air?

16

Unlike older forum platforms (vBulletin, phpBB) Discourse isn’t a collection of flat server-side scripts (php) and a separate database.

Discourse comprises two halves: a backend which sits in Docker, and a single page javascript application that is served to the client device.

Anything which requires a backend change will impact the docker container, which in the most basic of installs necessitates a small amount of downtime. That’s what people are referring to when they say the app needs to be rebuilt. The config file (a yml document) that controls how the container is built needs to be edited and then a rebuild is issued to launcher via SSH. Installing plugins mean a rebuild, whereas simple changes to SMTP are more akin to a restart.

Introducing new themes and theme components are effectively front-end changes done within the running web app. They don’t result in any downtime as the underlying app and database is effectively unchanged.

1 Like
17

Thanks Steven :slight_smile:

I have a few technical deficits. My hands on experience predates docker by a generation! in fact I remember when gosling’s justification of Java as a lightweight language was the hottest publication of the month - at the time I was a K&R C into ingres & oracle & sysadmin & dba

I think I’m detecting use of the words front end and back end as processes of running in the server and not as server-side client side is that right?

Have we got cooperating processes with shared memory or pipes or something between them sitting on the server and then a TCP encapsulated message stream sending stuff to the IP that has the client software on?

Has anybody drawn a block diagram of this architecture stuff ?

18

I think this has definitely shifted away from community :slight_smile: Let’s slide it over to dev as it’s more about the technical elements.

1 Like
20

This topic appears to be a mixture of two ideas:

  1. “A high level overview/diagram would be helpful for me to design my Discourse instance”
  2. “I’m trying to compare the functionaltiy of Discourse to my requirements but can’t find certain information”.

The first point regarding architecture has been discussed a bit, yet unfrotunately there is still an absence of top level diagram. Hopefully someone with a better understanding can draw something for us here with mermaid, however I can at least (hopefully) provide a bit of guidance for your original requirements.

Can Discourse meet Medical/Goverment/Automotive Information Security requirements and Standards?

You would need to be more concrete in what these requirements are exactly. However, considering the medical and automotive world are not too far apart I can share my experience in the hope that it helps. For context, I run an innersource instance for a large automotive supplier in Germany. It’s been a legal headache but it can be achievable with a naive level of persistence, idiotic levels of resilience and an incredibly helpful and patient legal team. Seriously, be extra nice to your legal team :laughing:

The biggest questions you will need to answer are:

  1. Who is accessing the information?
    • The public?
    • Staff?
    • A mixture of staff and public?
  2. What kind of information will be on the platform?
    • Public only?
    • Mixture of Public and Internal?
    • Confidential? – note the second you plan to host this is on the platform things get a lot harder
  3. Where is it being hosted?
    • on-site
    • By Discourse or another host

In our example, we were internal only(Staff), Only internal Information(i.e. share company info that is not confidential) and originally self hosted but moved to Discourse for hosting.

In terms of hosting with Discourse, our Information Security Office saw no significant issues when we chose to migrate.

We are also distributing this information across multiple nations: China, India, Germany, Romania, USA, France etc. China was a bit of a pain but the Discourse guys did fantastic work to get us across the line with the CDN issues we faced.

Note that question number 3: “Where will it be hosted” is the one which answers most of your protection of data and security questions.

Login and Authorisation

For Login you would probably want to lean on SAML. The Discourse team help you set this up if you are an enterprise customer with them. Our IDP is only accessable when you are behind our company’s vpn, so that adds an additional layer of security for us(i.e. you can’t even load the login screen unless you are on our network)

SSH

Further a standard installation will provide ssh encryption. I’m not in the CIA so I don’t know if they need more than that. :male_detective: allegedly

Interfacing Discourse with other tools

Lean on the API

For interfacting, the Discourse API is your friend. You can get and set data using an API key and some python.

There is a great set of examples here: Discourse REST API comprehensive examples

Anonymising user data to be GDPR compliant

In terms of GDPR, you could extract the data from the platform and omit user at the point of source running a query in the data explorer.

This is in contrast to using the Discourse API where the JSON response typically includes the full information about the post such as:

  • The post content (cooked HTML and raw Markdown)
  • The post ID
  • The topic ID it belongs to
  • The username of the poster
  • The post number within the topic
  • The post’s creation and last updated timestamps
  • The number of likes, replies, quotes, and so on.

How to get Rising Posts and familiar UI?

You might not have seen it, but you could combine this theme:

with something like this:

3 Likes
21

For HIPAA law in the U.S., with any kind of medical record data, practitioners can share this with family/friends/caregivers + anyone else the patient chooses if they sign an official release of information form. These usually have an expiration of only so many months or years.

Simple diagram for this would be: all medical data is in a gigantic locked vault that can never be opened for any emergency ever unless emergency services personnel have signed paperwork from the patient.

Unless: patient is unconscious, especially long-term in a coma or otherwise not of sane mind, and/or declared by a court + judge to be unable to take care of their own affairs, in which case someone else may be appointed for that.

Either of those two scenarios could be represented by a key that can open the vault, then move some medical data to a secondary vault with restricted access to only specific people.

Which could be a group of people but they would have to be individually named in most cases I believe to be authorized to have any access to medical data.

1 Like