Something bad with embeddings (related topics getting too costly)

Roman · December 4, 2024, 9:09pm

Yeah, I think the bug I fixed revealed another one that the digest check was hiding.

I think the bug is here:

app/jobs/scheduled/embeddings_backfill.rb

main


      
              .where("#{table_name}.updated_at < ?", 6.hours.ago)
              .where("#{table_name}.updated_at < topics.updated_at")
              .limit((limit - rebaked) / 10)
          
          populate_topic_embeddings(vector, relation, force: true)
          
          next unless SiteSetting.ai_embeddings_per_post_enabled
          
          # Now for posts
          table_name = DiscourseAi::Embeddings::Schema::POSTS_TABLE
          posts_batch_size = 1000
          
          posts =
            Post
              .joins(
                "LEFT JOIN #{table_name} ON #{table_name}.post_id = posts.id AND #{table_name}.model_id = #{vector_def.id}",
              )
              .where(deleted_at: nil)
              .where(post_type: Post.types[:regular])
          
          # First, we'll try to backfill embeddings for posts that have none

I changed it from find_each to find_in_batches last week (the former uses batches internally), and since both rely on limit to specify batch size, the original limit of limit - rebaked is ignored. We should use pluck + each_slice instead.

Topic		Replies	Views
"Net::HTTPBadResponse" errors on Gemini Embeddings Bug ai , related-topics	12	772	February 29, 2024
OpenAI embedding related topics stops working Product Support ai , related-topics	3	620	March 10, 2024
Ai:embeddings:backfill - Handling OpenAI's 400 Error for Excessive Tokens in Embeddings Bug ai	9	1043	March 15, 2024
Problem with the new Discourse AI "related / similar topics"-function Product Support ai , related-topics	5	1131	August 21, 2023
Embeddings search still running error Bug ai , ai-search	9	222	November 11, 2026

Something bad with embeddings (related topics getting too costly)

Related topics