That’s precisely why this topic is where we need to be directed. The original post includes a list which appears to be updated quite frequently. If you look at the edit history you can tell which plugins were added and when.
3 Likes
We are done for now! last one remaining is cakeday and it will be done in a few more months.
6 Likes
I stumbled and tripped over this a short while ago. I’ve been starting to poke at doing some Discourse development, so I wanted to practice my updating workflow:
# git pull
# bundle install
# pnpm install
# ./bin/rails db:migrate
But my guess is that the Discourse AI plugin requires the Pgvector plugin for PostgreSQL (which I did not previously know existed):
== 20230710171141 EnablePgVectorExtension: migrating ==========================
-- enable_extension(:vector)
bin/rails aborted!
StandardError: An error has occurred, this and all later migrations canceled: (StandardError)
ERROR: current transaction is aborted, commands ignored until end of transaction block
/home/john/development/discourse/lib/mini_sql_multisite_connection.rb:109:in 'MiniSqlMultisiteConnection#run'
/home/john/development/discourse/plugins/discourse-ai/db/migrate/20230710171141_enable_pg_vector_extension.rb:8:in 'EnablePgVectorExtension#change'
/home/john/development/discourse/lib/freedom_patches/schema_migration_details.rb:8:in 'block in FreedomPatches::SchemaMigrationDetails#exec_migration'
/home/john/development/discourse/lib/freedom_patches/schema_migration_details.rb:8:in 'FreedomPatches::SchemaMigrationDetails#exec_migration'
/home/john/development/discourse/lib/migration/safe_migrate.rb:28:in 'Migration::SafeMigrate::SafeMigration#migrate'
/home/john/development/discourse/lib/migration/safe_migrate.rb:53:in 'Migration::SafeMigrate::NiceErrors#migrate'
/home/john/development/discourse/lib/tasks/db.rake:267:in 'block (2 levels) in <main>'
/home/john/development/discourse/lib/distributed_mutex.rb:53:in 'block in DistributedMutex#synchronize'
/home/john/development/discourse/lib/distributed_mutex.rb:49:in 'Thread::Mutex#synchronize'
/home/john/development/discourse/lib/distributed_mutex.rb:49:in 'DistributedMutex#synchronize'
/home/john/development/discourse/lib/distributed_mutex.rb:34:in 'DistributedMutex.synchronize'
/home/john/development/discourse/lib/tasks/db.rake:242:in 'block in <main>'
Caused by:
PG::InFailedSqlTransaction: ERROR: current transaction is aborted, commands ignored until end of transaction block (PG::InFailedSqlTransaction)
/home/john/development/discourse/lib/mini_sql_multisite_connection.rb:109:in 'MiniSqlMultisiteConnection#run'
/home/john/development/discourse/plugins/discourse-ai/db/migrate/20230710171141_enable_pg_vector_extension.rb:8:in 'EnablePgVectorExtension#change'
/home/john/development/discourse/lib/freedom_patches/schema_migration_details.rb:8:in 'block in FreedomPatches::SchemaMigrationDetails#exec_migration'
/home/john/development/discourse/lib/freedom_patches/schema_migration_details.rb:8:in 'FreedomPatches::SchemaMigrationDetails#exec_migration'
/home/john/development/discourse/lib/migration/safe_migrate.rb:28:in 'Migration::SafeMigrate::SafeMigration#migrate'
/home/john/development/discourse/lib/migration/safe_migrate.rb:53:in 'Migration::SafeMigrate::NiceErrors#migrate'
/home/john/development/discourse/lib/tasks/db.rake:267:in 'block (2 levels) in <main>'
/home/john/development/discourse/lib/distributed_mutex.rb:53:in 'block in DistributedMutex#synchronize'
/home/john/development/discourse/lib/distributed_mutex.rb:49:in 'Thread::Mutex#synchronize'
/home/john/development/discourse/lib/distributed_mutex.rb:49:in 'DistributedMutex#synchronize'
/home/john/development/discourse/lib/distributed_mutex.rb:34:in 'DistributedMutex.synchronize'
/home/john/development/discourse/lib/tasks/db.rake:242:in 'block in <main>'
Caused by:
ActiveRecord::StatementInvalid: PG::FeatureNotSupported: ERROR: extension "vector" is not available (ActiveRecord::StatementInvalid)
DETAIL: Could not open extension control file "/usr/share/postgresql/extension/vector.control": No such file or directory.
HINT: The extension must first be installed on the system where PostgreSQL is running.
/home/john/development/discourse/plugins/discourse-ai/db/migrate/20230710171141_enable_pg_vector_extension.rb:6:in 'EnablePgVectorExtension#change'
/home/john/development/discourse/lib/freedom_patches/schema_migration_details.rb:8:in 'block in FreedomPatches::SchemaMigrationDetails#exec_migration'
/home/john/development/discourse/lib/freedom_patches/schema_migration_details.rb:8:in 'FreedomPatches::SchemaMigrationDetails#exec_migration'
/home/john/development/discourse/lib/migration/safe_migrate.rb:28:in 'Migration::SafeMigrate::SafeMigration#migrate'
/home/john/development/discourse/lib/migration/safe_migrate.rb:53:in 'Migration::SafeMigrate::NiceErrors#migrate'
/home/john/development/discourse/lib/tasks/db.rake:267:in 'block (2 levels) in <main>'
/home/john/development/discourse/lib/distributed_mutex.rb:53:in 'block in DistributedMutex#synchronize'
/home/john/development/discourse/lib/distributed_mutex.rb:49:in 'Thread::Mutex#synchronize'
/home/john/development/discourse/lib/distributed_mutex.rb:49:in 'DistributedMutex#synchronize'
/home/john/development/discourse/lib/distributed_mutex.rb:34:in 'DistributedMutex.synchronize'
/home/john/development/discourse/lib/tasks/db.rake:242:in 'block in <main>'
Caused by:
PG::FeatureNotSupported: ERROR: extension "vector" is not available (PG::FeatureNotSupported)
DETAIL: Could not open extension control file "/usr/share/postgresql/extension/vector.control": No such file or directory.
HINT: The extension must first be installed on the system where PostgreSQL is running.
/home/john/development/discourse/plugins/discourse-ai/db/migrate/20230710171141_enable_pg_vector_extension.rb:6:in 'EnablePgVectorExtension#change'
/home/john/development/discourse/lib/freedom_patches/schema_migration_details.rb:8:in 'block in FreedomPatches::SchemaMigrationDetails#exec_migration'
/home/john/development/discourse/lib/freedom_patches/schema_migration_details.rb:8:in 'FreedomPatches::SchemaMigrationDetails#exec_migration'
/home/john/development/discourse/lib/migration/safe_migrate.rb:28:in 'Migration::SafeMigrate::SafeMigration#migrate'
/home/john/development/discourse/lib/migration/safe_migrate.rb:53:in 'Migration::SafeMigrate::NiceErrors#migrate'
/home/john/development/discourse/lib/tasks/db.rake:267:in 'block (2 levels) in <main>'
/home/john/development/discourse/lib/distributed_mutex.rb:53:in 'block in DistributedMutex#synchronize'
/home/john/development/discourse/lib/distributed_mutex.rb:49:in 'Thread::Mutex#synchronize'
/home/john/development/discourse/lib/distributed_mutex.rb:49:in 'DistributedMutex#synchronize'
/home/john/development/discourse/lib/distributed_mutex.rb:34:in 'DistributedMutex.synchronize'
/home/john/development/discourse/lib/tasks/db.rake:242:in 'block in <main>'
Tasks: TOP => db:migrate
I can go install that, but I’m curious if there’s a way to disable plugins at this level so their migrations are just skipped. (I would prefer to not install additional software, particularly for a “plugin” that I’m not interested in exploring for development.)
3 Likes
I also encountered this earlier today. I installed the package via sudo, after some help from ChatGPT. I’m also wondering this; I wonder - perhaps deleting the plugin folder from the /plugins directory will help 
… but git pull later on may re-install it back.
2 Likes
I am opposed to this change. Normally in software dev, having a lean core means the main distribution can be more smaller, faster and less surface area for attacks. My last foray into plugins led me to see that it is technically possible for plugin code to run even while “disabled” as this seemed to be up to the plugin author to check, so it does seem like this increases the risk & bloat significantly.
The most immediate issue is that no instructions seem to be updated on the install guide (perhaps I just missed them?) It’s not clear what we need to install to make things work again. I solved some errors by installing ubuntu package postgresql-16-pgvector but still had some vector errors running the db:migrate. I was able to work around them by locally deleting the ai plugin.
Anyway, this is a whole lot of extra code, many of these plugins are completely irrelevant to the use cases of most discourse communities. (This is not to say these are bad plugins! I am sure they are very useful to the communities that need them. Just I struggle to see that every community forum needs to ship with Zendesk integration, etc). The AI plugin in particular given its additional requirements that are breaking things should definitely get the boot IMO.
On a personal level, when I login to my admin panel and I suddenly see a bunch of ad plugins, even if the code should be inert, it makes me extremely concerned. I, in the strongest terms possible, wish to convey I do NOT want any big tech Ad plugins on my installs by default, even disabled. That is an industry that has historically been incredibly abusive towards user privacy and Discourse does not help itself by shipping such integrations by default. The people who want ads would have no trouble finding the requisite plugin, it’s not necessary to include it on all installs.
TLDR: Please reconsider this change. 
4 Likes
Discourse are integrating plugins into core that were always offered in their official hostings
1 Like
I agree that this brings extra bloat in the admin interface, with a lot of brand names that account for as advertising to my brains. This is not cool. As a free software advocate trying hard to avoid big brands, I consider it takes a step that Ubuntu took some years ago before it became entirely colonized by Armazon, Gaggle and consorts. If you let those come to your eyeballs, they will end up swallowing them.
Bundling the plugins into core also brings an opportunity for developers to mistakenly turn a plugin functionality into a requirement over time, while keeping them as plugins ensures that kind of mistake cannot be made.
Can the @team clarify the rationale behind this change?
If it’s a matter of speeding up installations on Discourse infrastructure, maybe creating a discourse-hosting-bundle package automatically made on CI could achieve the same result?
3 Likes
@david, can you please maintain the alphabetical order in the Affected Plugins list?
The motivation here is to to provide a smoother experience for first-time users of Discourse, so that they get a more “fully featured” Discourse experience right out-the-box.
Another motivation is the developer experience for our team, and for community contributers. By having all of these plugins bundled with core, there’s no longer a need to consider compatibility with different core versions. That’s particularly useful for plugins like discourse-ai which are being very heavily developed right now, in tandem with related core changes.
The branded Auth plugins are the ones which are most likely to be absorbed into core, just like our existing core auth methods like Google/Facebook/etc. So there’s a good chance they’ll be removed from the list in the not-too-distant future.
This change does not relate to performance on our hosting. We already had a pre-built distribution with all of these plugins, and more, as you describe. 
I’ve fixed the ordering 
6 Likes
It’s already been answered in the op:
Few examples of this:
- we don’t have to worry about versioning if we add something in core for a plugin, we know that both of them are at the same version
- easier to test a plugin depending on another plugin if the code is present for both
- when we change something in core we often have to do multiple PRs just to fix specs in plugins, now it means only one standalone PR
End result is more available time for the Discourse team to improve and maintain the product instead of dealing with this kind of issues, so eventually, you win too.
7 Likes
I would very much like those disappearing from my UI, but they’re just occupying space, because they’re not optional nor hidden: they’re part of core. That is exactly why we’re worried.
With regard to first-time user experience, maybe changing the container template to include and document the official bundle would work:
hooks:
after_code:
- exec:
cd: $home/plugins
cmd:
# The docker_manager plugin is mandatory for automated web-based updates
- git clone https://github.com/discourse/docker_manager.git
# The following plugins are installed on all Discourse hosted instances.
# Uncomment the line by removing `#` before the `- git clone` line to enable them.
# See https://meta.discourse.org/t/bundling-more-popular-plugins-with-discourse-core/373574/
# Discourse Advertising plugin (Ads): https://meta.discourse.org/t/discourse-advertising-plugin-ads/33734
#- git clone https://github.com/discourse/discourse-adplugin
# Discourse Affiliate
#- git clone https://github.com/discourse/discourse-affiliate
# ...
# ...
# Please add other plugins below. For a list of official plugins, see:
# https://meta.discourse.org/tag/official
# For all available plugins, see:
# https://meta.discourse.org/c/plugin/22
With regard to the developer experience, surely you can automate the passing of plugin versions in another less intrusive way. For example, using a pups rule to uncomment the plugins using the documentation strategy above.
You could even have a container template that does it for you:
templates:
- "templates/discourse.hosting.yml"
- "templates/discourse.core-bundle.yml"
Note that my first impression was good: I could discover a couple of interesting plugins that were not under my radar yet. But yes, since I’ve seen you optimizing and looking for the least common denominator over the last decade, I’m surprised this would be the way you handle such a proposition.
3 Likes
I believe that you can add an rm - rf discourse-ai where the git clone commands usually are. I’ve not done it myself.
2 Likes
Yes technically you can rm all the folders in the plugin dir and core discourse will still work
4 Likes
It’s a good bet that these plugins developed by the discourse team follow the rules and add minimal overhead. Having them all in for makes it easier to see that they all work together and share the same requirements.
Are you running an external postgres rather than the included one? Are you running a two container setup that hasn’t been updated on a long while?
You can contrive to remove them, but they can’t do anything unless you create an account with the evil companies and get keys to allow the tracking to begin.
4 Likes
Thank you for your response,
Hopefully yes, but it is extra surface area for bugs and attacks, in exchange for no benefit to communities who do not use these plugins (which will be most communities).
I feel like Discourse has identified a real problem here but come up with the wrong solution.
I absolutely agree there is a real root problem of the fragility of the development/plugin ecosystem for Discourse. It is definitely difficult to develop against a moving target, and the API changes in core certainly make that hard. To a certain extent that’s just part of developing against bleeding edge upstream, but that’s exactly why that shouldn’t be the default behavior. Having just a couple simple plugins I can understand and empathize with your problem (which is far larger of scope than mine).
However, what Discourse is doing here is just kicking the can down the road. It maybe solves the problem for Discourse, but it doesn’t solve it for the rest of the plugin developers. Everyone else is going through that same pain, just on a smaller scale.
A better solution would be to have a more robust LTS series that can be developed against. I know this has been discussed elsewhere so I won’t rehash, but one of the greatest challenges for communities, plugin devs, and even Discourse employees seems to be that there is no LTS. The stable branch is actively recommended against using. If a more traditional development pipeline was used, it would be much easier for plugin development. We could have known times when things will break (major version upgrades), and plan around it. Otherwise we can be confident that the smaller upgrades won’t randomly break our forums. (This is probably one of the biggest woes Discourse has IMO, and one I’ve seen brought up elsewhere when people discuss forum options. The volatility is a real drawback.)
Yes – Per the link, I ran into this on a development environment which not would be container based. (I have not tried this in production containers yet)
I think this is one of the things I worry about, is if the ‘standard’ path assumes these plugins exist, then I could run into bugs because everyone else is accepting advertising plugins. I have to either risk unexpected bugs cropping up, or deal with additional plugin load. My goal is to have as much stability as possible for my deployments.
It is also worth pointing out that even Discourse core is pretty heavyweight in terms of resource usage compared to other forums. I think it is worth trying to keep core lightweight so as to not further exacerbate performance issues.
I have not audited these yet to make sure that they’re not pulling in tracking/phone-home JS while disabled, but until then I will assume you are correct. I hope someone double checks this, because it will be a huge debacle if that turns out to be false.
I think Hellekin’s point about clutter is valid, and I also think what they are getting at with big tech ads plugin will not be well-received by some in the open source community – the type who are more likely to use open source forums in the first place.
Anyway, I would also like to say I do appreciate you hearing my feedback, even though it is not easy 
3 Likes
I think it’s difficult to know which ones have been proofread before and which ones were not.
I briefly wondered whether you had checked beforehand to see if there were any open comments that are now missing, since only the texts were added and not all of the data from Crowdin. But I assume that I don’t even want to know the answer to that question. Ultimately, it makes no difference whether no one responds for months or whether questions and comments from translators are simply gone.
1 Like
Are there tests for no side effects when removed plugin directories with the new bundle, to ensure a plugin does not inadvertently become a requirement?
2 Likes
Our core test suite is run without any plugins loaded, so yes any dependency from core → plugins should be detected by CI.
3 Likes
I commented out the lines to clone the plugins but when I run again the “launcher rebuild app” I get the same messages:
FAILED
--------------------
Pups::ExecError: cd /var/www/discourse && su discourse -c 'bundle exec rake db:migrate' failed with return #<Process::Status: pid 546 exit 1>
Location of failure: /usr/local/lib/ruby/gems/3.3.0/gems/pups-1.3.0/lib/pups/exec_command.rb:131:in `spawn'
exec failed with the params {"cd"=>"$home", "tag"=>"migrate", "hook"=>"db_migrate", "cmd"=>["su discourse -c 'bundle exec rake db:migrate'"]}
bootstrap failed with exit code 1
---
HINT: The plugin 'discourse-reactions' is now bundled with Discourse and should not be included in your container configuration.
Remove the line 'git clone https://github.com/discourse/discourse-reactions' from your containers/app.yml file, then try again.
For more information, see https://meta.discourse.org/t/373574
---
---
HINT: The plugin 'discourse-data-explorer' is now bundled with Discourse and should not be included in your container configuration.
Remove the line 'git clone https://github.com/discourse/discourse-data-explorer' from your containers/app.yml file, then try again.
For more information, see https://meta.discourse.org/t/373574
---
---
HINT: The plugin 'discourse-solved' is now bundled with Discourse and should not be included in your container configuration.
Remove the line 'git clone https://github.com/discourse/discourse-solved' from your containers/app.yml file, then try again.
For more information, see https://meta.discourse.org/t/373574
---
** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one.
The reason for the failure is located above the FAILED line.