# Dump all conversations in a file and structured data
**URL:** https://meta.discourse.org/t/dump-all-conversations-in-a-file-and-structured-data/202351
**Category:** Support
**Created:** [September 1, 2021, 2:59pm UTC](https://meta.discourse.org/t/dump-all-conversations-in-a-file-and-structured-data/202351 "2021-09-01T14:59:40Z")
**Posts on this page:** 9
**Page:** 1
### Author:  [@Wall-E](https://meta.discourse.org/u/Wall-E)
#### Post date: [September 1, 2021, 2:59pm UTC](https://meta.discourse.org/t/dump-all-conversations-in-a-file-and-structured-data/202351/1 "2021-09-01T14:59:40Z")
This question is similar to this one: [Does Discourse support export conversations as an organized bulk of data?](https://meta.discourse.org/t/does-discourse-support-export-conversations-as-an-organized-bulk-of-data/180537)
but we are looking for a way to do some NLP on all conversations of our Discourse site. Someone in our team asked if this could be done by acting at some low-level, in the backend, e.g. exporting the database but without the table, with something like `pg_dump --schema-only`. I didn’t fully understand what my colleague meant but I thought maybe you would.
---
### Author:  [@pfaffman](https://meta.discourse.org/u/pfaffman)
#### Post date: [September 1, 2021, 3:02pm UTC](https://meta.discourse.org/t/dump-all-conversations-in-a-file-and-structured-data/202351/2 "2021-09-01T15:02:29Z")
If you’re self-hosted, then they can do the `pg_dump` command that they think will help.
You can also dump data in various formats with the [Data Explorer Plugin](https://meta.discourse.org/t/data-explorer-plugin/32566).
---
### Author:  [@Wall-E](https://meta.discourse.org/u/Wall-E)
#### Post date: [September 1, 2021, 8:44pm UTC](https://meta.discourse.org/t/dump-all-conversations-in-a-file-and-structured-data/202351/3 "2021-09-01T20:44:11Z")
This plugin seems to provide most of what we’re looking for! Thanks!
---
### Author:  [@Wall-E](https://meta.discourse.org/u/Wall-E)
#### Post date: [September 14, 2021, 10:40pm UTC](https://meta.discourse.org/t/dump-all-conversations-in-a-file-and-structured-data/202351/4 "2021-09-14T22:40:02Z")
> [@pfaffman](#):
>
> You can also dump data in various formats with the [Data Explorer Plugin](https://meta.discourse.org/t/data-explorer-plugin/32566)
So I installed the plugin and looked at all the queries made at [(Superseded) What cool data explorer queries have you come up with?](https://meta.discourse.org/t/what-cool-data-explorer-queries-have-you-come-up-with/43516) but there isn’t anything that can export the actual conversations. For example, I have asked for the top 100 active topics. I get database entries with topic IDs (see screenshot), but no conversations. Is this because the plugin is only to extract data from the database only and won’t pull the conversation themselves? If that is correct, is there a way to use the information pulled from the database to pull the conversations in a json files, and whose topic IDs are the ones pulled from the database by the plugin?
---
### Author:  [@pfaffman](https://meta.discourse.org/u/pfaffman)
#### Post date: [September 14, 2021, 11:35pm UTC](https://meta.discourse.org/t/dump-all-conversations-in-a-file-and-structured-data/202351/5 "2021-09-14T23:35:14Z")
```
SELECT * FROM posts where topic_id=425
```
That will give you the posts the first topic in your query (given that I can type on this phone).
But if what you want is JSON, you could do something like
```
https://meta.discourse.org/t/dump-all-conversations-in-a-file-and-structured-data/202351.json
```
---
### Author:  [@Wall-E](https://meta.discourse.org/u/Wall-E)
#### Post date: [September 15, 2021, 12:17am UTC](https://meta.discourse.org/t/dump-all-conversations-in-a-file-and-structured-data/202351/6 "2021-09-15T00:17:19Z")
I didn’t understand your 1st option, maybe a typo in your text? Did you mean I only get the 1st post of the topic?
Regarding the 2nd option with the .json extension, is there an alternative url that uses the topic\_id or any other entry that can be used to have a more programmatic way to get the conversation as a json without having to know the topic title?
---
### Author:  [@pfaffman](https://meta.discourse.org/u/pfaffman)
#### Post date: [September 15, 2021, 12:38am UTC](https://meta.discourse.org/t/dump-all-conversations-in-a-file-and-structured-data/202351/7 "2021-09-15T00:38:57Z")
Did you try the sql query? Was there an error? Edit: I checked. That query will return all posts in a topic.
You can get any topic with only the topic id.
```
https://meta.discourse.org/t/-/202351.json
```
---
### Author:  [@Wall-E](https://meta.discourse.org/u/Wall-E)
#### Post date: [September 15, 2021, 1:03pm UTC](https://meta.discourse.org/t/dump-all-conversations-in-a-file-and-structured-data/202351/8 "2021-09-15T13:03:18Z")
the query was fine, i just misunderstood your explanation of what it actually provides. Thanks for double-checking. These are great solutions.
---
### Author:  [@system](https://meta.discourse.org/u/system)
#### Post date: [October 15, 2021, 1:04pm UTC](https://meta.discourse.org/t/dump-all-conversations-in-a-file-and-structured-data/202351/9 "2021-10-15T13:04:01Z")
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.