On a side note, this appears to be the batch size, how many requests in a single call. Perhaps the issue is that number of requests being made per minute (not per batch). Is there a way to throttle how many backfill requests are sent per minute or per hour?
Also found this if it helps other users, the new gemini embedding is having issues with limits set to 0 if exceeded. There’s a temp workaround to using text embedding instead or maybe just wait for a bit and see if resolves. Having said that I still think it’s a good idea for discourse to add an option to limit the number of API calls per minute for backfills to avoid this problem in the first place.
PS: SUPER COOL to see google also using discourse - wonder what AI they use to power their forum search 
