Bulk Export of Raw Post Sources with Markup

Hi,

our company runs Discourse to support our customers and strengthen customer relationships. For staff onboarding, we created documentation using Doc Categories and DiscoTOC, which works great.

However, I’d like to request a feature that would allow exporting the documentation sources – specifically the raw text including markup – to a plain text file.

:information_source: I am aware of the browser’s PDF print option, which doesn’t help when raw sources are needed rather than parsed ones. I also know about the built-in backup function and database queries, but these are not practical solutions for frontend administration.

For now, my workflow is: Edit the postSTRG-A, STRG-Ccreate / open a text fileSTRG-V, STRG-SDiscard Editing… next post.

A built-in bulk export option would provide several benefits:

  • much easier maintenance for knowledge bases
  • better backup and versioning workflows for admins in the frontend
  • flexible reuse of content outside of Discourse

:right_arrow: Such a feature would be a real productivity gain for communities that use Discourse as a documentation hub, not only for support.

I’d be curious to hear your thoughts.

Cheers

1 Like

One of my problems is treating feature requests like they are support requests. Oops, I’ve done it again. Sorry if this is no help.

So you want something like this: https://meta.discourse.org/posts/1849580/raw or this: https://meta.discourse.org/raw/381447 or this https://meta.discourse.org/raw/381447/1

Oh. You want the raw text from all posts in the topic.

This isn’t a solution, but an improved version of that is. But that doesn’t save very many clicks.

That’s pretty easy to automate with some command line tools. A clever one would get the json of the topic to see how many posts there were and then get them all, perhaps pausing to deal with rate limiting.

A theme component could give you links to the raw text for each post. A plugin could do what you want.

2 Likes

Hi Jay,

thanks for your response.

This solution is already discussed in How to iterate over all the topics in order to export them as Markdown?, but it’s not practical for larger documentations (→ efficiency) and frontend administration (→ competence).

So yes… no support, but a real feature request from my side :wink:

Cheers

1 Like

I think 📄 Copy Post Component which copies the post directly could be helpful in this case. It reduces

to one click. But it’s still not bulk copying

4 Likes

It eases the pain… bookmarked.

Thanks

2 Likes

You can create a Data Explorer query that returns posts.raw, and call the query using the API.

5 Likes

Thanks for the tip, Richard :slight_smile: I’ll check out Data Explorer and API calls when I get a chance. Still, that’s just a hook, not a feature.

Take Doc Categories plug-in as an example: to build documentation, all you technically need is an index post containing the links to all topics. But the plug-in adds something extra: automatic page navigation.

That’s exactly the kind of feature request I’m making: an additional functionality that helps regular users – not just another workaround for power users or IT folks.

2 Likes

If I were wishing for such a feature, I would like it to give me a zip file containing all markdown sources referenced by the doc index.

How would one like to have individual files named in that case?

Could you describe your intended workflow in more detail?

If I were to build a documentation workflow like you describe, I would prefer to build an option to mount raw markdown into the filesystem, which would enable further automated tools to work on it.

I do not understand this point.
Why do you not want to present your documentation via discourse?

Hi Thomas, a zip container would of course work just fine :wink: There’s no workflow behind it, just the wish to keep a simple backup.

1 Like

I cannot imagine how a regular user would use raw post sources.

3 Likes

That’s maybe because you aren’t a regular user.

Maybe you can give me a good example instead of responding with an ad hominem.
Even your opening post talks about admins and people managing knowledge bases.

Sorry, you shared your personal views, and I was just responding – no offense intended.

For example, people managing knowledge bases, such as knowledge managers who create content as admins but have limited technical background, want to handle their content themselves and be able to back it up locally.

2 Likes

I wouldn’t consider a knowledge manager a regular user, it is a very specific role.

Let’s leave the whole “regular user” debate for another time, please.

Then the data explorer way would be a perfectly good solution.
You can also make that query available to non-admin users.

1 Like

Hey @soeren-1 and welcome to our community! :hugs:

Your feature request looks interesting! I think it’s great to have this topic available so others who have a similar setup can chime in. Maybe eventually something like this will make it into the UI. But I think you haven’t made a case yet for it being a widely useful feature.

You’re new here, so let me fill you in on some things that may not be obvious to you. Adding features takes time and resources, so typically new features are added to Discourse when one of our enterprise customers ask for it or when there is alot of demand for it, e.g. rule of three. They also often start out as plugins or components before eventually making it into our core product.

For many unique cases the Discourse API and data explorer provide quick and dirty solutions that are usually “good enough”. I recognize that’s not what you’re asking for here! But sometimes trying to solve a problem using the API or a data explorer query can help make the case for adding functionality or finding a better way to solve a particular problem.

Have you seen how data explorer queries can be made available to groups, and can have a UI to specify parameters? This is suitable for “regular users” and might work for you and your team as an interim and to help you make the case here for your new feature idea. I don’t know if there’s a suitable query already out there for you to start from - the place to start looking for that is Data & reporting.

@rgj I think you’re a regular guy, no matter what anybody else says!

d he's just a regular guy like you and me..gif

2 Likes