Github issues migrator rate limited by github

I’m trying to migrate all of the issues from Issues · ipfs/faq · GitHub to using the issues-to-discourse tool. There are a total of about 233 issues. After migrating about 30% of them the remaining issues failed to migrate with this message:

{"message":"You have triggered an abuse detection mechanism and have been temporarily blocked from content creation. Please retry your request again later.","documentation_url":""}

@erlend_sh do you have any suggestions how to address this?

Hmm any ideas @cpradio? Can we throttle in the code?

Can’t think of a way off the top of my head (at least not something simple to implement). I’d also need to better understand what sort of limitations we’re looking at from GitHub, as it could theorectically really slow down the entire process. Especially if you also have to worry about the comments being part of what is throttled too (which would be very complicated to code – doing that part was a real P.I.T.A. the first round, not something I’d enjoy reworking).


This made a pretty gross mess.

  • Most of the issues got migrated but about half of the gh issues didn’t get updated with links to discourse and didn’t get closed
  • Some of the issues only got partially migrated. Worse, the order of dropped comments is not predictable. Example: When Is it possible to map the blocks to existing files? · Issue #253 · ipfs/faq · GitHub was migrated it missed lidel’s comment, which happens to be the one with the answer to githubber314159’s question.
  • can’t roll back because many of the issues were migrated

Is this based on size? How many issues are we talking about? Maybe we never did this on a “big” set of GH issues?

We could at least add warnings based on size and rate limiting.

I was attempting to migrate 233 issues. Most of them had 1-3 comments. Some had about 8 comments. Github’s daily rate limit is 5,000 requests.


I tried to import issues from Issues · w3c/activitypub · GitHub but bumped into the same problem.

One thing I noticed is that the importer failed to import comments after a mention of “this issue was mentioned in (other issue)”, dropping a whole lot of comments before resuming.

Another bug I saw is the “From (user) on (date)” opening blurb is in HTML ending with <br/><br/> with no newline, so it breaks the beginning of the Markdown (e.g., <br/><br/>## Some title). I guess the best would be to turn that blurb into Markdown and replace the break rule tags with actual linefeeds.

I will try to import issues one at a time to figure out more problems.

1 Like