# Rake de backfill de embeddings de IA abortado

**URL:** <https://meta.discourse.org/t/ai-embeddings-backfill-rake-aborted/293120>\
**Category:** Product Support\
**Tags:** ai\
**Created:** [Janeiro 27, 2024, 6:10am UTC](https://meta.discourse.org/t/ai-embeddings-backfill-rake-aborted/293120 "2024-01-27T06:10:13Z")\
**Posts on this page:** 1\
**Showing post:** 5

<div class="post-metadata">

**Author:** ![Don](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/don/32/228726_2.png) [@Don](https://meta.discourse.org/u/Don)\
**Post date:** [Janeiro 30, 2024, 6:05am UTC](https://meta.discourse.org/t/ai-embeddings-backfill-rake-aborted/293120/5 "2024-01-30T06:05:09Z")

</div>

Acho que o problema foi o limite de taxa, tive que atualizar o nível na OpenAI e mudei o modelo para `text-embedding-3-large`. Agora, tenho muitos pontos durante o rake, o que é um bom sinal. 😅 Atualizarei este post quando terminar.

> **Edição 1: Então, depois de 10 horas de pontuação, ele fechou com o rake abortado**
>
> ```plaintext
> rake aborted!
> 
> Faraday::TimeoutError: Net::ReadTimeout with #<TCPSocket:(closed)> (Faraday::TimeoutError)
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-protocol-0.2.2/lib/net/protocol.rb:229:in `rbuf_fill'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-protocol-0.2.2/lib/net/protocol.rb:199:in `readuntil'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-protocol-0.2.2/lib/net/protocol.rb:209:in `readline'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-http-0.4.1/lib/net/http/response.rb:158:in `read_status_line'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-http-0.4.1/lib/net/http/response.rb:147:in `read_new'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-http-0.4.1/lib/net/http.rb:2342:in `block in transport_request'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-http-0.4.1/lib/net/http.rb:2333:in `catch'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-http-0.4.1/lib/net/http.rb:2333:in `transport_request'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-http-0.4.1/lib/net/http.rb:2306:in `request'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rack-mini-profiler-3.3.0/lib/patches/net_patches.rb:19:in `block in request_with_mini_profiler'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rack-mini-profiler-3.3.0/lib/mini_profiler/profiling_methods.rb:50:in `step'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rack-mini-profiler-3.3.0/lib/patches/net_patches.rb:18:in `request_with_mini_profiler'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-net_http-3.1.0/lib/faraday/adapter/net_http.rb:113:in `block in request_with_wrapped_block'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-http-0.4.1/lib/net/http.rb:1570:in `start'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-net_http-3.1.0/lib/faraday/adapter/net_http.rb:112:in `request_with_wrapped_block'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-net_http-3.1.0/lib/faraday/adapter/net_http.rb:102:in `perform_request'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-net_http-3.1.0/lib/faraday/adapter/net_http.rb:66:in `block in call'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-2.9.0/lib/faraday/adapter.rb:45:in `connection'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-net_http-3.1.0/lib/faraday/adapter/net_http.rb:65:in `call'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-2.9.0/lib/faraday/request/url_encoded.rb:25:in `call'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-2.9.0/lib/faraday/rack_builder.rb:152:in `build_response'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-2.9.0/lib/faraday/connection.rb:444:in `run_request'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-2.9.0/lib/faraday/connection.rb:280:in `post'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-2.9.0/lib/faraday.rb:145:in `method_missing'
> 
> /var/www/discourse/plugins/discourse-ai/lib/inference/open_ai_embeddings.rb:18:in `perform!'
> 
> /var/www/discourse/plugins/discourse-ai/lib/embeddings/vector_representations/text_embedding_3_large.rb:39:in `vector_from'
> 
> /var/www/discourse/plugins/discourse-ai/lib/embeddings/vector_representations/base.rb:147:in `generate_representation_from'
> 
> /var/www/discourse/plugins/discourse-ai/lib/tasks/modules/embeddings/database.rake:30:in `block (2 levels) in <main>'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:71:in `each'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:71:in `block in find_each'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:138:in `block in find_in_batches'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:245:in `block in in_batches'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:229:in `loop'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:229:in `in_batches'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:137:in `find_in_batches'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:70:in `find_each'
> 
> /var/www/discourse/plugins/discourse-ai/lib/tasks/modules/embeddings/database.rake:28:in `block in <main>'
> 
> /usr/local/bin/bundle:25:in `load'
> 
> /usr/local/bin/bundle:25:in `<main>'
> 
> Caused by:
> 
> Net::ReadTimeout: Net::ReadTimeout with #<TCPSocket:(closed)> (Net::ReadTimeout)
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-protocol-0.2.2/lib/net/protocol.rb:229:in `rbuf_fill'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-protocol-0.2.2/lib/net/protocol.rb:199:in `readuntil'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-protocol-0.2.2/lib/net/protocol.rb:209:in `readline'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-http-0.4.1/lib/net/http/response.rb:158:in `read_status_line'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-http-0.4.1/lib/net/http/response.rb:147:in `read_new'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-http-0.4.1/lib/net/http.rb:2342:in `block in transport_request'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-http-0.4.1/lib/net/http.rb:2333:in `catch'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-http-0.4.1/lib/net/http.rb:2333:in `transport_request'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-http-0.4.1/lib/net/http.rb:2306:in `request'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rack-mini-profiler-3.3.0/lib/patches/net_patches.rb:19:in `block in request_with_mini_profiler'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rack-mini-profiler-3.3.0/lib/mini_profiler/profiling_methods.rb:50:in `step'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rack-mini-profiler-3.3.0/lib/patches/net_patches.rb:18:in `request_with_mini_profiler'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-net_http-3.1.0/lib/faraday/adapter/net_http.rb:113:in `block in request_with_wrapped_block'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-http-0.4.1/lib/net/http.rb:1570:in `start'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-net_http-3.1.0/lib/faraday/adapter/net_http.rb:112:in `request_with_wrapped_block'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-net_http-3.1.0/lib/faraday/adapter/net_http.rb:102:in `perform_request'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-net_http-3.1.0/lib/faraday/adapter/net_http.rb:66:in `block in call'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-2.9.0/lib/faraday/adapter.rb:45:in `connection'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-net_http-3.1.0/lib/faraday/adapter/net_http.rb:65:in `call'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-2.9.0/lib/faraday/request/url_encoded.rb:25:in `call'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-2.9.0/lib/faraday/rack_builder.rb:152:in `build_response'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-2.9.0/lib/faraday/connection.rb:444:in `run_request'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-2.9.0/lib/faraday/connection.rb:280:in `post'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/faraday-2.9.0/lib/faraday.rb:145:in `method_missing'
> 
> /var/www/discourse/plugins/discourse-ai/lib/inference/open_ai_embeddings.rb:18:in `perform!'
> 
> /var/www/discourse/plugins/discourse-ai/lib/embeddings/vector_representations/text_embedding_3_large.rb:39:in `vector_from'
> 
> /var/www/discourse/plugins/discourse-ai/lib/embeddings/vector_representations/base.rb:147:in `generate_representation_from'
> 
> /var/www/discourse/plugins/discourse-ai/lib/tasks/modules/embeddings/database.rake:30:in `block (2 levels) in <main>'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:71:in `each'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:71:in `block in find_each'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:138:in `block in find_in_batches'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:245:in `block in in_batches'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:229:in `loop'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:229:in `in_batches'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:137:in `find_in_batches'
> 
> /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:70:in `find_each'
> 
> /var/www/discourse/plugins/discourse-ai/lib/tasks/modules/embeddings/database.rake:28:in `block in <main>'
> 
> /usr/local/bin/bundle:25:in `load'
> 
> /usr/local/bin/bundle:25:in `<main>'
> 
> Tasks: TOP => ai:embeddings:backfill
> 
> (See full trace by running task with --trace)
> 
> ```

Reiniciei o rake. Neste caso, ele fará apenas os tópicos intocados ou novamente todos os tópicos? 😄

Edição 2: O rake abortou novamente com o mesmo erro. Mas acho que esses processos cobriram a maioria dos tópicos e o `Jobs::EmbeddingsBackfill` no Sidekiq parece estar funcionando bem agora. Então, acho que isso pode ser suficiente para os tópicos restantes 🙂

---

_[View the full topic](https://meta.discourse.org/t/ai-embeddings-backfill-rake-aborted/293120)._