# Questions about importing json

**URL:** https://meta.discourse.org/t/questions-about-importing-json/85829
**Category:** Development
**Created:** [April 20, 2018, 6:48pm UTC](https://meta.discourse.org/t/questions-about-importing-json/85829 "2018-04-20T18:48:29Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![tehspaceg](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/tehspaceg/32/119558_2.png) [@tehspaceg](https://meta.discourse.org/u/tehspaceg)
#### Post date: [April 20, 2018, 6:48pm UTC](https://meta.discourse.org/t/questions-about-importing-json/85829/1 "2018-04-20T18:48:29Z")

</div>

I’ll preface that my knowledge of Ruby is essentially zero, so sorry if anything I say sounds dumb.

While I wait for someone in #Marketplace on [my bounty](https://meta.discourse.org/t/paid-migration-script-from-nodebb-redis-to-discourse/85749/3), I figured I could poke around a bit at importing data on my own. I see that there is a [json \_generic.rb](https://github.com/discourse/discourse/blob/master/script/import_scripts/json_generic.rb) file in GitHub, which requires the [base.rb](https://github.com/discourse/discourse/blob/master/script/import_scripts/base.rb) file.

I’ve got a few rookie questions and some questions on what kind of data the base.rb script is looking for.

**create\_users**  
It looks like this function is designed to iterate through a data set and send it over to create\_user. For create user, is there a list of available fields that I can pass into this function?

I have a rough idea from the code, but I want to make sure I know all of them. Here is the list I collected; `id (required, from original source), bio_raw, website, location, avatar_url, name, username, email (required), trust_level, active, import_mode, last_emailed_at, password`

What data format is needed? Is it just straight json that I can pass in to create\_users? Or is there a certain Ruby format I should follow?

Is it possible to import the original creation date of the user for consistency?

**create\_posts**  
This one will be the hardest part. The create\_post function appears to only take the post without looking at topics. The way NodeBB (my data source) stores data is to have a topic and posts that are linked to that topic via topic ID. I’m not seeing anything in this function that creates the topics, then handles importing posts to link to that topic. Am I missing something?

Similar to users, I don’t see a field that would take creation date/time in account for topics or posts. The json\_generic.rb file appears to take that data, so I assume it gets done at some point.

**json\_generic.rb**  
Last set of questions is for this one. I’m trying to understand what kind of format it expects the imported json data to be in. It looks like it is expecting one file with all of the content. Here is an example (missing some fields) that I think it is looking for:

```plaintext
{"topics": [
	{
		"tid": 2,
		"cid": 1,
		"title": "topic_title",
		"user_id": "string_username",
		"raw": "content of first post",
		"blah": "blah"
	},
	{
		"tid": 3,
		"cid": 2,
		"title": "topic_title",
		"user_id": "username for first post",
		"raw": "content of first post",
		"blah": "blah"
	}],
"posts": [
	{
		"pid": 89,
		"user_id": "username",
		"raw": "post content",
		"timestamp": "date"
	}],
"users": [
	{
		"uid": 1,
		"username": "teh g",
		"email": "fake@fake.xyz",
		"joindate": "date"
	}]
}

```

---

<div class="post-metadata">

### Author: ![mbcahyono](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/mbcahyono/32/97721_2.png) [@mbcahyono](https://meta.discourse.org/u/mbcahyono)
#### Post date: [April 20, 2018, 7:45pm UTC](https://meta.discourse.org/t/questions-about-importing-json/85829/2 "2018-04-20T19:45:07Z")

</div>

I didn’t test it myself, but after reading the code, I think the format of the JSON should be similar to this:

```json
  {
    "topics": [
      {
        "id": 1,
        "title": "The title",
        "pinned": false,
        "posts": [
          {
            "id": 2,
            "body": "The body",
            "date": "The date",
            "author": "username"
          }
        ]
      }
    ]
  }

```

---

<div class="post-metadata">

### Author: ![tehspaceg](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/tehspaceg/32/119558_2.png) [@tehspaceg](https://meta.discourse.org/u/tehspaceg)
#### Post date: [April 20, 2018, 8:00pm UTC](https://meta.discourse.org/t/questions-about-importing-json/85829/3 "2018-04-20T20:00:07Z")

</div>

> [@mbcahyono](#):
>
> I didn’t test it myself, but after reading the code, I think the format of the JSON should be similar to this:

Perfect, thanks for taking a look. I’ll have to figure out a good way to merge my topics and posts files into one, but I think I can come up with something for that.
