Hi all, I am currently writing something to migrate a Woltlab (Wbb) Forum to Discourse and I am willing to share it once it is somewhat done. Right now I am more or less successfully migrating categories, forums, users and their groups without too many issues. However every now and then, during the runtime it gets worse, I get a “Ratelimit Exceeded” error, while processing the posts and threads. My problem is I am still relatively new and I am mostly collecting bits and pieces from other migrators, so I am currently clueless where the message is coming from. Is it coming from the API of Discourse? And if so can I somewhat disable ratelimiting during imports entirely? def migrate_threads_and_posts puts "Migrating threads and posts..." @client.query("SELECT * FROM wbb1_thread ORDER BY time ASC").each do |thread_row| begin next if thread_row[:isDeleted] == 1 || thread_row[:movedThreadID] # Ensure lastPosterID maps to a valid user last_post_user_id = @user_map[thread_row[:lastPosterID]] || @user_map[thread_row[:userID]] # If lastPosterID and userID are null, create a placeholder user if last_post_user_id.nil? && thread_row[:lastPoster] last_post_user_id = find_or_create_placeholder_user(thread_row[:lastPoster]) end # Ensure a valid user ID exists last_post_user_id ||= User.first.id # Get topic creator user ID or create placeholder creator_user_id = @user_map[thread_row[:userID]] || find_or_create_placeholder_user(thread_row[:username]) || User.first.id # Create the topic topic = Topic.new( title: thread_row[:topic], user_id: creator_user_id, category_id: @forum_map[thread_row[:boardID]], created_at: Time.at(thread_row[:time]), updated_at: Time.at(thread_row[:lastPostTime]), last_post_user_id: last_post_user_id ) if topic.save(validate: false) puts "Imported thread: #{topic.title} (ID: #{topic.id})" # Migrate posts migrate_posts(thread_row[:threadID], topic.id) else puts "Error importing thread #{thread_row[:topic]}: #{topic.errors.full_messages.join(', ')}" end sleep(7) # Increased sleep to avoid rate limiting rescue => e puts "Exception importing thread #{thread_row[:topic]}: #{e.message}" end end end I tried to clean up the code and comment as much as possible. Hope that someone can help with the rate limiting. I also tried to search through the forum here but I only saw many had similar issues but never really a solution. So if I missed something, sorry for the duplicate. Thanks all.

Okay. It is literally RateLimiter.disable Nevermind then :slight_smile:

[תמונה] Klaas Tammling: I am currently writing something to migrate a Woltlab (Wbb) Forum to Discourse and I am willing to share it once it is somewhat done. It’ll be much easier if you use one of the import scripts as a starting point. It will solve a bunch of problems that you don’t yet know that you have.

[תמונה] Jay Pfaffman: It will solve a bunch of problems that you don’t yet know that you have. Which problems could that be? I’m currently orienting myself on the phpbb and the xenforo import script.

The phpbb3 script is well-written, but requires a deep understanding of Ruby. xenforo is ok, I think. You should find one that has the features you want (PMs? permalinks? groups? admins? suspended users?) [תמונה] Klaas Tammling: Which problems could that be? I can’t guess what you don’t know, but ways to connect old IDs to the new ones, ways to make sure that the same data doesn’t get imported twice are two that spring to mind.

חריגה ממגבלת קצב בעת ההעברה של נושאים ותגובות

תמיכה הגירה

Klaas_Tammling (Klaas Tammling) 5 בפברואר,‏ 2025,‏ 6:01pm 4

Which problems could that be? I’m currently orienting myself on the phpbb and the xenforo import script.

נושא		תגובות	צפיות
Unable to disable the rate limit Self-hosting	2	715	2 באוגוסט,‏ 2022
[Paid] Migrate proprietary forums to Discourse Marketplace delivered	3	1496	16 באפריל,‏ 2020
Import posts through API (time delay) Product Support	1	1066	27 בינואר,‏ 2020
Any way to turn off RateLimiter temporarily for bulk creation by admin? Development	19	1695	29 במאי,‏ 2024
How do I avoid rate limiting when importing a bunch of data? Development rest-api , rails-console	13	1349	6 ביולי,‏ 2023

חריגה ממגבלת קצב בעת ההעברה של נושאים ותגובות

נושאים קשורים