# Import questions in bulk

**URL:** https://meta.discourse.org/t/import-questions-in-bulk/111587
**Category:** Migration
**Tags:** rest-api
**Created:** [3월 14, 2019, 2:49오후 UTC](https://meta.discourse.org/t/import-questions-in-bulk/111587 "2019-03-14T14:49:26Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![Stephane\_Maarek](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/stephane_maarek/32/133975_2.png) [@Stephane\_Maarek](https://meta.discourse.org/u/Stephane_Maarek)
#### Post date: [3월 14, 2019, 2:49오후 UTC](https://meta.discourse.org/t/import-questions-in-bulk/111587/1 "2019-03-14T14:49:26Z")

</div>

Hello

I have my questions and responses in a JSON format:

> **sample JSON document here**
>
> ```
> {
> "id": "x017c4h221p7T8sboHglB-7kQ==",
> "created": "2018-05-09T20:13:23Z",
> "title": "Docker i/o error",
> "body": "<p>Hey Stephane, I am having to restart docker to run Kafka everytime I exit since it errors out on tcp port binding. My understanding is that when I stop Kafka services and exit, the port is released.</p>",
> "course": {
> "_class": "course",
> "id": "x0190RCkGpZ6FMe4CPAF8aOoQ==",
> "title": "Apache Kafka Series - Learn Apache Kafka for Beginners v2",
> "url": "/apache-kafka/"
> },
> "replies": [{
> "_class": "answer",
> "id": "x01Qx2rNaX48kxP4NFFSSCK7g==",
> "created": "2018-05-10T07:04:10Z",
> "user": {
> "_class": "user",
> "id": "x01N-Fup_OULjTEtHPLwc8JSQ==",
> "name": "Ivan",
> "locale": "en_US"
> },
> "is_top_answer": null,
> "body": "<p>Hi Nandini,\u00a0</p><p>Can you please elaborate in more details what your problem is? If you stop Kafka services, ports used by Kafka\u00a0will be released, of course.</p><p>Regards,</p><p>Ivan</p>",
> },
> {
> "_class": "answer",
> "id": "x01bLG2QPhyLwZ_RJsbMge16A==",
> "created": "2018-05-10T20:45:39Z",
> "user": {
> "_class": "user",
> "id": "x01oX4mwhRQoLXKuhHXDHg3zg==",
> "name": "Nandini",
> "locale": "en_US"
> },
> "is_top_answer": null,
> "body": "<p>I stop kafka services and when i restart, docker ports are not released and I get a TCP binding error on ports 2181 and 3030</p>",
> "is_upvoted": false,
> "num_upvotes": 0
> },
> {
> "_class": "answer",
> "id": "x01yL8D1-inVZE3njAo08-uMw==",
> "created": "2018-05-11T00:32:46Z",
> "user": {
> "_class": "user",
> "id": "x01lNfqEyIqBf47SM76dxq0rw==",
> "name": "Stephane Maarek",
> "locale": "en_US"
> },
> "is_top_answer": true,
> "body": "<p>Restart the docker engine if you can, or your computer. See if that helps !\u00a0</p><p>Otherwise you have something running on port 2181. Please check the FAQ lecture (lecture 22) as a lot of students have been through these issues before </p>",
> "is_upvoted": false,
> "num_upvotes": 0
> }
> ],
> "user": {
> "_class": "user",
> "id": "x01oX4mwhRQoLXKuhHXDHg3zg==",
> "name": "Nandini",
> "locale": "en_US"
> }
> }
> 
> ```

I have developed a python script to hit the API but I’m getting a bunch of errors on the API related to throttling… :

> You’ve performed this action too many times. Please wait X seconds before trying again.

I have to import about 3000 questions total (alongside an average of 2 answers) so I feel the API route might be too long.

**Is there a way to disable this API throttling issue?**

**Is there any other way I can import my data?** They’re only linked to two users (so no need to create users). I’m also using a hosted discourse so I don’t know if I have direct access to the underlying DB

Happy to share the Python code I have, or open a bounty if this requires a lot of effort

---

<div class="post-metadata">

### Author: ![codinghorror](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/codinghorror/32/110067_2.png) [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [3월 14, 2019, 6:50오후 UTC](https://meta.discourse.org/t/import-questions-in-bulk/111587/2 "2019-03-14T18:50:53Z")

</div>

Make sure the user performing the actions via the API key has staff privileges, even if only temporary.

That will help with some of the rate limits.

---

<div class="post-metadata">

### Author: ![Mittineague](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/mittineague/32/114259_2.png) [@Mittineague](https://meta.discourse.org/u/Mittineague)
#### Post date: [3월 14, 2019, 7:04오후 UTC](https://meta.discourse.org/t/import-questions-in-bulk/111587/3 "2019-03-14T19:04:19Z")

</div>

An easy approach I used was to add  
`sleep(0.7)`  
inside the Ruby loop. (you may need to tweak that)

For 3000 requests it would take 35 minutes to complete. A bit painful, but for a once off I don’t think it would be all that bad.

---

<div class="post-metadata">

### Author: ![Stephane\_Maarek](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/stephane_maarek/32/133975_2.png) [@Stephane\_Maarek](https://meta.discourse.org/u/Stephane_Maarek)
#### Post date: [3월 14, 2019, 8:04오후 UTC](https://meta.discourse.org/t/import-questions-in-bulk/111587/4 "2019-03-14T20:04:59Z")

</div>

I have admin permissions (using my own API key) and I still get a throttle it seems every 60 API calls. I tried to change things in settings \> Rate limits but doesn’t seem to help

---

<div class="post-metadata">

### Author: ![paultaylor](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/paultaylor/32/166024_2.png) [@paultaylor](https://meta.discourse.org/u/paultaylor)
#### Post date: [1월 9, 2020, 4:09오후 UTC](https://meta.discourse.org/t/import-questions-in-bulk/111587/6 "2020-01-09T16:09:14Z")

</div>

저도 같은 문제를 겪고 있습니다.

인위적으로 1초 지연을 추가하면 더 이상 발생하지 않습니다.

1분당 60회 호출 제한은 다음과 같이 피할 수 있습니다:

서버에서

```
cd /var/discourse

```

편집기(저는 vi를 사용합니다)로 containers/app.yml 파일을 열고, env 섹션에 다음 줄을 추가하세요.

`DISCOURSE_MAX_ADMIN_API_REQS_PER_KEY_PER_MINUTE : 6000`

그리고 저장하세요.

Rebuild를 실행해야 합니다(단순 재시작만으로는 효과가 없습니다)  
`./launcher rebuild app`

---

<div class="post-metadata">

### Author: ![system](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/system/32/443519_2.png) [@system](https://meta.discourse.org/u/system)
#### Post date: [11월 7, 2023, 1:35오후 UTC](https://meta.discourse.org/t/import-questions-in-bulk/111587/7 "2023-11-07T13:35:08Z")

</div>


