Includere più plugin popolari con il core di Discourse

Annunci
80

Having been bit by the “plugin failure to rebuild” issue early on in my Discourse self-hosting, I can appreciate the desire to bundle into core known good versions. And potentially offer a richer selection of features.

A customer-focused org could have done a poll on the core plugin direction, or at least the timing. Perhaps I missed that. As a IT tool provider to my customers (i.e. end-users) trying to get real work done with IT, I’ve seen many products rev’d into over complexity and ultimate replacement. Now we will have self-hosters “rm -fr” ing plugins they don’t need. I get this and may join that club. In my experience, additional code increases integration complexity, risk of configuration mistakes, and attack surface. But sooner or later removing something that the app developer assumes will be there, will also break something.

I would have much preferred the Discourse jedi’s efforts gone into working on a robust, maintained, scripted method to enable cloud storage of images including CDN integration. SMTP email integration is trivial by comparison, and that has busted with changes at MailGun and others causing self-hosted sites grief.

81

Indeed, I would strongly recommend against using rm -rf on these plugins. As you say, there are risks around unexpected inter-dependencies in future. But also, you’ll be creating a diff in the core git repository, which means UI updates via docker_manager are almost certain to fail.

Of course, leaving these plugins in their default ‘disabled’ state is totally supported, and will ensure they don’t have any measurable performance impact.

8 Mi Piace
82

What we need is a way to exclude plugins from being found even though they are in the plugins directory. This doesn’t fix really inter-dependencies. But it does reduce DB migration dependencies for unused dependencies (e.g. pgvector for AI).

2 Mi Piace
83

For self-hosters or those like me providing hosting services for customers, here’s a simple grep which will list if any of the new bundled plugins are already in your containers/app.yml

grep -E 'git clone .*(discourse-(adplugin|affiliate|ai|apple-auth|assign|calendar|chat-integration|data-explorer|gamification|github|graphviz|hcaptcha|login-with-amazon|lti|math|microsoft-auth|oauth2-basic|openid-connect|patreon|policy|post-voting|reactions|rss-polling|solved|subscriptions|templates|topic-voting|user-notes|zendesk-plugin|cakeday))' containers/app.yml

Needs to run as root, from /var/discourse, and will output any lines that need to be manually removed from the plugins section of the app.yml before you rebuild.

9 Mi Piace
84

@pacharanero Thanks for putting that together!

Might consider changing it to reference containers.*.yml so that it also helps out anyone who has done a standard two-container install where they would instead by in web-only? You really don’t want them in any of your container definitions, after all. :smiling_face:

@david would you consider adding that to the top post and maintaining it once you integrate cakeday?

2 Mi Piace
85

Thank ChatGPT which crafted it exactly correctly in one go from this prompt:

Please scrape the GitHub urls of each of the plugins mentioned in this post
https://meta.discourse.org/t/bundling-more-popular-plugins-with-discourse-core/373574#p-1810533-affected-plugins-3

Compile them into a list and create a unix command which will tell me if any of these plugins are already mentioned in a ‘git clone’ in that Discourse’s containers/app.yml

3 Mi Piace
86

On the surface, to me, this does sound like a reasonable thing to consider doing at some point – having a more declarative way to define which plugins are loaded or not at boot time, even if the source is still sitting there on disk.

That said, I don’t know what the feasibility or lift of doing that would be.

3 Mi Piace
87

It’s certainly possible. But it adds complexity - yet another thing to support/maintain. And it would only be useful in single-tenant environments (i.e. not in shared hosting environments, where different tenants want different plugins).

If anything, I think it would be more beneficial to spend time improving the UX & performance of ‘disabled’ plugins (which we have already started doing since this big merge into core).

10 Mi Piace
88

also i haven’t tried to remove the plugins, because i feel it would compromise my Bug reports to the meta, alike even the attempt of a Shared Hosting Installation

2 Mi Piace
89

Uff, that was a rough update. Identifying the issue in the mess of rebuild log is not that trivial to start with. Found it, overlooked another plugin to remove from our config two times, hence 3rd rebuild attempt finally passed that part. It would have been really helpful to check for and warn in the first place, that the config needs to be adjusted. discourse-doctor checks (too simple, but can be taken as a start) for plugins in the config, so that can be used as a basis. Probably too late now 3 weeks later, whatever …

But that was not all, we also ran into db:migrate errors. Retried it 2 times, then ran discourse-doctor, which ran the rebuild as well, and weirdly succeeded. I checked its code, and it does absolutely nothing (no change) before the rebuild, and calls the rebuild the exact same way we do. Hence looks like the db:migrate succeeded on 3rd attempt for some reason? I read through the thread that the large number of added plugins add dependencies, which may conflict/be older than what was previously used. Luckily we did not need to add a manual plugin removal step, dependency adjustment or database change, like others seem to have needed. Is it somehow expected that running the db:migrate multiple times will somehow finally succeed? I can just hope nothing is broken …

I fully support this opinion on the matter. Do not suddenly add a whole ton of plugins all at once. If a particular feature is seen as very beneficial for all (starter) instances, discuss it on an individual basis, one by one. It may then be also better to just add it to Discourse natively (with a toggle in settings), instead of keeping it as plugin, bringing their maintainer(s) on board. We are now also considering whether to add a removal step to our config, and what to do with those plugins which are now enabled by default, implying a change to our Discourse instance we did not intend, and which I actually have no time to test and think about now, after midnight local time, also thanks to the upgrade mess and research needed to fix …

Though a question in these regards: Were some features turned into plugins or so? I see narrative bot plugin, which I do not remember from earlier. Its description says “introduces staff”, so at first I thought it would be an addition to discobot, for staff (mods/admins/…) only or so. But from its settings it indeed is discobot, isn’t it? Are all those plugins which appear as enabled now, but were not actively installed via config, features which were present before already?

2 Mi Piace
90

It was installed earlier, we just omitted it from the UI with the rest of the bundled plugins from before. Our UI was improved to properly surface everything that exists. (we had a bunch of omissions including Chat which was hidden via CSS)

I have already observed internal dev team velocity increase in the very short time this has been in place. It is leading to a more stable product, something I am very happy about.

No plans to back track here. You need to adjust to the new world.

4 Mi Piace
91

3 posts were split to a new topic: Help me debug failed migrations

93

IMO if something breaks because a plugin isn’t installed, then that itself is a bug. Core should not depend on plugins. Plugins themselves should clearly list their requirements on their respective pages.

But yes, this is going to make the self-hosted version even less stable going forward, since it’ll be the self-hosters stumbling through these issues. Between this and the forked thread, I really don’t get the impression that stability of self-hosters is a high priority for the team.

2 Mi Piace
94

We hadn’t thought about that in advance, so some approvals might have gone missing. But we managed to transfer a good chunk of them from the plugin projects on Crowdin to core. We’ll do better next time since we now have the tools to transfer approvals between projects.

image
image1161×1236 202 KB

No, we didn’t check beforehand, but I checked afterward. None of the plugins had unanswered comments on Crowdin. We have an internal tool that stores all comments that are posted on our Crowdin plugins. We could even use it to post missing comments on Crowdin again, e.g. when Crowdin deletes comments because the source string was updated. It just hasn’t been a priority to implement this.

6 Mi Piace
96

It would really be nice to have one more option here:

image
image406×223 4.8 KB

“Enabled Plugins”

This would avoid the initial massive list in Installed Plugins.

Also, to facilitate this:

  • Allow custom links in the Plugins section (perhaps)
  • Make this dropdown respond to a route param filter:

image
image396×179 1.17 KB

so:

https://example.com/admin/plugins?filter=enabled

12 Mi Piace
98

Agreed. Maybe an option to list enabled core installed plugins. Vs just having show all plugins. Addituonal filtering options definitely better

1 Mi Piace
99

I agree with the sentiment. Imho the real disconnect is keeping them listed as plugins.

Once merged it should be moved to something branded like “Core Features” as plugins are seen as optional components that can be installed. Vs part of the main program. So it is a misnomer imho to keep them listed as plugins when the intent is not be uninstalled.

Similar to TC that were merged in core like “personal bubbles” is not listed in Theme Components. Granted in that particular case you cannot disable the former TC. If you wanted to go back to what it was before you needed to create a TC to undo the changesm :wink:

100

Definitely agree with additional filtering options. To also have one for disabled plugins as well.

However after more thought and reading more posts. If the plugins are merged with core. They really maybe should no longer be called plugins and not listed in Plugins. But maybe something like Core Features or Optional Features. Since these are no longer really meant to be able to uninstall them.

1 Mi Piace
101

There is no reason properly engineered forum software should require advertising code to run.

If Discourse decides to bake in anti-features, the only thing it’s going to force people to do is either fork Discourse or migrate to something else. Those of us that do not like big tech / ad stuff feel very strongly about this. Discourse will NOT force it on us, no matter how hard it is pushed. (Ubuntu tried this on us and now my highest starred repo is something that removes the ads ;))

102

Not sure I follow. If by advertising code you mean plugins that are merged into the core product vs remaining optional add-ons/installs. If we go back you will find a variety of “Advertising code” was merged in core.

I can see from the DeV team perspective that many of their plugins may have started as plugins to allow more flexible testing before making it integrated in the core program.

I can appreciate like any software there is often features ppl might not use and choose to disable it and up to finding ways to uninstall features.

While I will admit there are a lot of recent merged plugins that I likely would not use. But having a simple disable and filtering them out is something good for all.

I understand that in part as stated by the team the intent is to make things easier with add-ons for self hosted.

Now imho the admin interface should be more customizable like it once was. As this can also help with ppl migrating from another platform by being able to load an admin there that is similar layout as the platform they are coming from. Much like how Linux does this with some mimicking other OSes. But that is another topic. :wink:

I can appreciate the feeling that discourse might be starting to head towards bloatware. Reactors demonstrated how much leaner Windows NT could be.