Hello :wave: I started use Discourse Ai - OpenAi API. First I seted up it to Related Topics (text-embedding-ada-002). When I try to run the rake task for backfill. I’ve got the following error. ....rake aborted! NoMethodError: undefined method `[]' for nil:NilClass (NoMethodError) response[:data].first[:embedding] ^^^^^^^ /var/www/discourse/plugins/discourse-ai/lib/embeddings/vector_representations/text_embedding_ada_002.rb:37:in `vector_from' /var/www/discourse/plugins/discourse-ai/lib/embeddings/vector_representations/base.rb:145:in `generate_representation_from' /var/www/discourse/plugins/discourse-ai/lib/tasks/modules/embeddings/database.rake:19:in `block (2 levels) in ' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:71:in `each' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:71:in `block in find_each' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:138:in `block in find_in_batches' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:245:in `block in in_batches' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:229:in `loop' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:229:in `in_batches' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:137:in `find_in_batches' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:70:in `find_each' /var/www/discourse/plugins/discourse-ai/lib/tasks/modules/embeddings/database.rake:17:in `block in ' /usr/local/bin/bundle:25:in `load' /usr/local/bin/bundle:25:in ` ' Tasks: TOP => ai:embeddings:backfill And also see the followings in /logs. [Screenshot 2024-01-27 at 7.09.32] Thanks for the help! :slightly_smiling_face:

I think the problem was rate limit, I had to upgrade the tier on OpenAI and I changed the model to text-embedding-3-large. Now, I have lots of dots during rake which is a good sign. :sweat_smile: Will update this post when it’s finished. Edit 1: So after 10 hours of dotting it closed with rake aborted rake aborted! Faraday::TimeoutError: Net::ReadTimeout with # (Faraday::TimeoutError) /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-protocol-0.2.2/lib/net/protocol.rb:229:in `rbuf_fill' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-protocol-0.2.2/lib/net/protocol.rb:199:in `readuntil' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-protocol-0.2.2/lib/net/protocol.rb:209:in `readline' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-http-0.4.1/lib/net/http/response.rb:158:in `read_status_line' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-http-0.4.1/lib/net/http/response.rb:147:in `read_new' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-http-0.4.1/lib/net/http.rb:2342:in `block in transport_request' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-http-0.4.1/lib/net/http.rb:2333:in `catch' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-http-0.4.1/lib/net/http.rb:2333:in `transport_request' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-http-0.4.1/lib/net/http.rb:2306:in `request' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rack-mini-profiler-3.3.0/lib/patches/net_patches.rb:19:in `block in request_with_mini_profiler' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rack-mini-profiler-3.3.0/lib/mini_profiler/profiling_methods.rb:50:in `step' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rack-mini-profiler-3.3.0/lib/patches/net_patches.rb:18:in `request_with_mini_profiler' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-net_http-3.1.0/lib/faraday/adapter/net_http.rb:113:in `block in request_with_wrapped_block' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-http-0.4.1/lib/net/http.rb:1570:in `start' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-net_http-3.1.0/lib/faraday/adapter/net_http.rb:112:in `request_with_wrapped_block' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-net_http-3.1.0/lib/faraday/adapter/net_http.rb:102:in `perform_request' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-net_http-3.1.0/lib/faraday/adapter/net_http.rb:66:in `block in call' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-2.9.0/lib/faraday/adapter.rb:45:in `connection' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-net_http-3.1.0/lib/faraday/adapter/net_http.rb:65:in `call' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-2.9.0/lib/faraday/request/url_encoded.rb:25:in `call' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-2.9.0/lib/faraday/rack_builder.rb:152:in `build_response' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-2.9.0/lib/faraday/connection.rb:444:in `run_request' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-2.9.0/lib/faraday/connection.rb:280:in `post' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-2.9.0/lib/faraday.rb:145:in `method_missing' /var/www/discourse/plugins/discourse-ai/lib/inference/open_ai_embeddings.rb:18:in `perform!' /var/www/discourse/plugins/discourse-ai/lib/embeddings/vector_representations/text_embedding_3_large.rb:39:in `vector_from' /var/www/discourse/plugins/discourse-ai/lib/embeddings/vector_representations/base.rb:147:in `generate_representation_from' /var/www/discourse/plugins/discourse-ai/lib/tasks/modules/embeddings/database.rake:30:in `block (2 levels) in ' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:71:in `each' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:71:in `block in find_each' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:138:in `block in find_in_batches' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:245:in `block in in_batches' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:229:in `loop' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:229:in `in_batches' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:137:in `find_in_batches' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:70:in `find_each' /var/www/discourse/plugins/discourse-ai/lib/tasks/modules/embeddings/database.rake:28:in `block in ' /usr/local/bin/bundle:25:in `load' /usr/local/bin/bundle:25:in ` ' Caused by: Net::ReadTimeout: Net::ReadTimeout with # (Net::ReadTimeout) /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-protocol-0.2.2/lib/net/protocol.rb:229:in `rbuf_fill' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-protocol-0.2.2/lib/net/protocol.rb:199:in `readuntil' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-protocol-0.2.2/lib/net/protocol.rb:209:in `readline' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-http-0.4.1/lib/net/http/response.rb:158:in `read_status_line' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-http-0.4.1/lib/net/http/response.rb:147:in `read_new' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-http-0.4.1/lib/net/http.rb:2342:in `block in transport_request' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-http-0.4.1/lib/net/http.rb:2333:in `catch' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-http-0.4.1/lib/net/http.rb:2333:in `transport_request' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-http-0.4.1/lib/net/http.rb:2306:in `request' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rack-mini-profiler-3.3.0/lib/patches/net_patches.rb:19:in `block in request_with_mini_profiler' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rack-mini-profiler-3.3.0/lib/mini_profiler/profiling_methods.rb:50:in `step' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rack-mini-profiler-3.3.0/lib/patches/net_patches.rb:18:in `request_with_mini_profiler' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-net_http-3.1.0/lib/faraday/adapter/net_http.rb:113:in `block in request_with_wrapped_block' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-http-0.4.1/lib/net/http.rb:1570:in `start' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-net_http-3.1.0/lib/faraday/adapter/net_http.rb:112:in `request_with_wrapped_block' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-net_http-3.1.0/lib/faraday/adapter/net_http.rb:102:in `perform_request' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-net_http-3.1.0/lib/faraday/adapter/net_http.rb:66:in `block in call' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-2.9.0/lib/faraday/adapter.rb:45:in `connection' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-net_http-3.1.0/lib/faraday/adapter/net_http.rb:65:in `call' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-2.9.0/lib/faraday/request/url_encoded.rb:25:in `call' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-2.9.0/lib/faraday/rack_builder.rb:152:in `build_response' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-2.9.0/lib/faraday/connection.rb:444:in `run_request' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-2.9.0/lib/faraday/connection.rb:280:in `post' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-2.9.0/lib/faraday.rb:145:in `method_missing' /var/www/discourse/plugins/discourse-ai/lib/inference/open_ai_embeddings.rb:18:in `perform!' /var/www/discourse/plugins/discourse-ai/lib/embeddings/vector_representations/text_embedding_3_large.rb:39:in `vector_from' /var/www/discourse/plugins/discourse-ai/lib/embeddings/vector_representations/base.rb:147:in `generate_representation_from' /var/www/discourse/plugins/discourse-ai/lib/tasks/modules/embeddings/database.rake:30:in `block (2 levels) in ' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:71:in `each' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:71:in `block in find_each' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:138:in `block in find_in_batches' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:245:in `block in in_batches' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:229:in `loop' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:229:in `in_batches' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:137:in `find_in_batches' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:70:in `find_each' /var/www/discourse/plugins/discourse-ai/lib/tasks/modules/embeddings/database.rake:28:in `block in ' /usr/local/bin/bundle:25:in `load' /usr/local/bin/bundle:25:in ` ' Tasks: TOP => ai:embeddings:backfill (See full trace by running task with --trace) I restarted rake. In this case it will do only the untouched topics or again the all topics? :smile: Edit 2: The rake aborted again with the same error. But I think these processes covered the most of topics and the Jobs::EmbeddingsBackfill in Sidekiq seems run with OK now. So I think that could enough to the remaining topics :slight_smile:

Sounds like you’re getting a nil response. Did you check ai embeddings enabled and add your api key to the settings?

Yeah, This is how settings looks. [Screenshot 2024-01-27 at 9.08.56] [Screenshot 2024-01-27 at 9.07.17] [Screenshot 2024-01-27 at 9.07.34] [Screenshot 2024-01-27 at 9.08.17] The Jobs::EmbeddingsBackfill seems running but always failed with the error I mentioned in OP. [Screenshot 2024-01-27 at 9.11.36] It seems it’s connected with OpenAi. [Screenshot 2024-01-27 at 9.10.11]

AI embeddings backfill rake aborted

Product Support

Don January 27, 2024, 8:12am 3

Yeah, This is how settings looks.

Screenshot 2024-01-27 at 9.08.56
Screenshot 2024-01-27 at 9.07.17

The Jobs::EmbeddingsBackfill seems running but always failed with the error I mentioned in OP.

It seems it’s connected with OpenAi.

Topic		Replies	Views
Ai:embeddings:backfill - Handling OpenAI's 400 Error for Excessive Tokens in Embeddings Bug ai	9	1038	March 15, 2024
"Net::HTTPBadResponse" errors on Gemini Embeddings Bug ai , related-topics	12	768	February 29, 2024
Something bad with embeddings (related topics getting too costly) Bug ai , related-topics	20	680	December 6, 2024
Rake tasks in the AI plugin not working Bug ai	4	256	December 2, 2025
Embedding error with 3.6.0 beta 2 Bug ai	4	483	November 12, 2025

AI embeddings backfill rake aborted

Related topics