Static pages sync: Allowing specific private categories to be synced

This is like a Feature - Dev topic, not sure which one it should be in.


Is there a way to modify site settings in a plugin? I’m 99% sure there isn’t, but I’d just like to confirm that.

If there isn’t, can I put forward a suggestion to not make it immutable? Or perhaps, having some API or a way to ‘unlock’ the immutable property of SiteSettings, perhaps?

A possible use case I’m looking at is to the include a list of protected categories to a setting such that it’s easier for the admin to include/exclude them.

Thanks.

1 Like

Do you just want to change the value of site settings? Just do SiteSetting.whatever='new value. Or do you want to change something about them?

Don’t you just want to add a setting for that? Just add to config/settings.yml? something like

If that’s what you want, then using rake plugin:create[plugin-name] will create the settings file for you, complete with example.

Or maybe I don’t understand your question.

1 Like

Let’s start here and work outside in.

Can you first describe the use case from the point of view of the community? What are they trying to accomplish and what makes that hard to do right now? What feature do you imagine solving their need more effectively (regardless of how it’s implemented)?

Then, we can work from there to determine if this is best done in a plugin or as a core feature.

Then, we can work from there to discuss suggestions for how to implement it.

1 Like

@mcwumbly @pfaffman Thanks for telling me about

I wrongly assumed that since settings are ummutable in TCs, they are also as such in plugins in Ruby. I’ll give that a shot.

It would be nice to make them editable in TCs. My use case (which I am using a plugin for now) is that I take all topics and posts in all categories by default and do some stuff with them, but would not like to include protected categories (like a excluded_categories setting).

However, if it were possible to add protected categories to the setting, and then access it thereafter, it would be easier. This way, admins can include some protected categories if they want to by removing them from the setting.

With @pfaffman’s idea, it can probably be done, but not in TCs.

The problem with this is that I don’t know the protected categories beforehand.

If you’re logged in as an admin then a theme component could change siteSettings via the API.

Then just make a theme component setting and add those categories? You’re not referring to some protected_categories site setting that I’m not thinking of, are you? So something like this?

1 Like

I would still love to know more.

Are you willing to share more about your community itself?

Are you the primary user of this feature or are there others on your team who need it? Do you have user facing documentation for the workflow that depends on this feature? If so, what does that look like? If not, can you describe briefly what it might look like?

I think answers to these kinds of questions would help me put this request in better context.

@mcwumbly @pfaffman Okay, let me try to explain this as much as possible.

I am developing a plugin that takes topics and posts and publishes them to GitHub as Markdown files (like an archive).

However, I would not like to include private categories (I think now that’s the correct term?) in this, since they are ‘private’.

Therefore, I am looking for a way to pre-fill a setting with the list of private categories, which can’t be defined within the default parameter, since I wouldn’t know what the private categories are beforehand.

In the event that this can be done by directly changing SiteSetting in Ruby, can the same be done with a Theme Component’s settings? I’m quite certain the latter is immutable. Is there any way to change it in a Theme Component?

Bear with me.

I still want to understand better from the perspective of the community team what problem you’re trying to solve.

What kind of community is this? Who runs it? Why do they want to duplicate their content on GitHub?

What problem are they trying to solve?

1 Like

It’s not really about the community. I’m taking a shot at this (with a backfill job as well) in my own way. This saves each topic and post as a Markdown file in a repo.

1 Like

I still don’t know what private means to you. Does it mean that you want only categories that are visible to everyone, or to anonymous users? Or do you have some other definition.

If you want those, then your plugin can get just those, perhaps by a search that you pass a user to (or maybe by calling a guardian), or just checking the permissions.

Or if private is something that is an opinion, then you can add a setting.

And you probably want to do this in a job that runs daily or whatever?

If you’re pushing stuff to github, I’d think you’d want discourse to just do it rather than muck with having your browser push data to discourse? I don’t see how or why you’d do it with a theme component.

2 Likes

I mean categories like #staff, and others that are limited by groups. I think it’s possible with a plugin, but I guess TC settings can’t be mutable then? I mean, that’s probably irrelevant to my original query now I guess.

Another possible approach would be to externalize this even further, rather than do it as a plugin or a theme component.

Some prior art here: Discourse Public Data Dump

Again, I think approaching this as much as possible from the perspective from the end result you’re working towards, the easier it’ll be to advise.

So thanks for sharing this link:

Perhaps we can use that as a jumping off point to further clarify the functional spec you’ve implicitly defined so far.

The way I’m understanding it now you want to:

  • create a static html archive of a Discourse site
  • keep it up to date as new content is created
  • exclude certain categories

And the design you’re currently exploring is:

  • create a plugin that:
    • allows admins to:
    • explicitly configure which categories to exclude
    • configure a git URL to store static content
    • runs a background job periodically that:
      • creates markdown files for topics and posts
      • store them in some file/directory structure in a git repo
    • pushes that to GitHub
  • end users can see the content on GitHub as html

Is that about right?

Completely spot-on! I’ve made a basic structure of that here.

1 Like

You don’t need a setting for that, that information is already available for a plugin.

1 Like

Am I correct in understanding that you are referring to the method of Category.where(...) to get these “private” categories? But what if the admin wants to include (some of) those - do I need to have a include categories setting which counteracts the “private” categories defined in the code? Would that would be counter-productive?

UPDATE: So SiteSettings can be edited via a plugin. TC settings still cannot, I believe? Marking Modify SiteSettings/make SiteSettings mutable? - #2 by pfaffman as the Solution.

I still don’t understand this. Yes, you can add those settings in a SiteSetting and yes the plugin could read that setting, and even change it. But why would it need to change the setting given the above scenario?

Given how this topic evolved, moving it to Dev

1 Like

Supposing if I have 5 categories: A, B, C, D and E. Now let’s say B and C are only available to some groups. To prevent private topics here from being shared when uploaded to the repository, B and C are added to the excluded_categories setting, along with others like say #staff.

Now supposing the site admin is fine with B’s topics to be shared, he can go ahead and remove B from the setting, while still leaving C there.

So the excluded_categories would have to be changed at the start to add the “private” categories B and C. I’m not sure if that makes sense?

Although that is perfectly possible from a technical point of view, I think the approach would be over complicated, especially since “at the start” is hard to define/detect, and you want to avoid that the plugin keeps adding B back after the site admin removed it. Also, when a new private category is added, the plugin would need to add it to the setting, but it would need to be able to see the difference between a new category (add) and a category that was previously removed by the admin (don’t re-add).

I would opt for a include_private_categories setting which starts out empty, and the plugin would simply process all public categories, AND the categories in include_private_categories. That will give you a lot less headaches.

Also, from a functional point of view, that makes it more easy to spot which private categories are being shared (in your approach one would have to have a list of categories in their mind and then determine which ones are missing by comparing them to the ones in the setting).

3 Likes

I’ve also updated the title to better describe what this topic is really about.

2 Likes

One other alternative design I could imagine here is having two separate repos: one for public content and one for private content.

The private content repo itself could be kept private (you could determine who has access to it independently).