Hugging Faceのbge-large-en embedding設定について？ RAGボットが応答しません！

サポート
, ,
1

bge-large-en 埋め込みモデルをデフォルトの Discourse AI ベクトルサービスとして機能させるための最適な設定について、アドバイスをお願いします。

AWS で bge-large-en インスタンスを実行しており、Discourse AI がそれに接続していることはわかっています（以下のテストを参照）。しかし、埋め込みが一般的に機能していません（OpenAI の埋め込みは正常に機能します）。

問題の概要：RAG ボットが HF bge-large-en に埋め込みを設定すると応答しなくなります。

AWS 埋め込みモデルはこちらです。

image
image1260×535 54 KB

Discourse AI の設定はこちらです。

image
image920×338 35.1 KB

image
image946×185 16.1 KB

接続を確認するための Discourse カスタム LLM の「テスト実行」はこちらです。

image
image1106×151 14.7 KB

AWS 側の bge-large-en ログはこちらです。

image
image1089×132 5.78 KB

よろしくお願いします！！

エラーログはこちらです。

Job exception: can't quote Array

hostname ai-qa-ubuntu-s-1vcpu-2gb-amd-sfo3-01-app
process_id 1165935
application_version f9192835a7e4d2067c3d1844f43f9e7b69de39e7
current_db default
current_hostname ai-qa.net
job Jobs::CreateAiReply
problem_db default
time 7:22 pm
opts post_id 618
--- --- --- ---
--- ---
bot_user_id -1208
persona_id 5
current_site_id default


Backtrace

/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8.1/lib/active_record/connection_adapters/abstract/quoting.rb:25:in `quote'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8.1/lib/active_record/connection_adapters/postgresql/quoting.rb:69:in `quote'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8.1/lib/active_record/connection_adapters/abstract/quoting.rb:51:in `quote_bound_value'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8.1/lib/active_record/sanitization.rb:193:in `block in quote_bound_value'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8.1/lib/active_record/sanitization.rb:193:in `map!'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8.1/lib/active_record/sanitization.rb:193:in `quote_bound_value'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8.1/lib/active_record/sanitization.rb:171:in `replace_bind_variable'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8.1/lib/active_record/sanitization.rb:180:in `block in replace_named_bind_variables'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8.1/lib/active_record/sanitization.rb:176:in `gsub'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8.1/lib/active_record/sanitization.rb:176:in `replace_named_bind_variables'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8.1/lib/active_record/sanitization.rb:128:in `sanitize_sql_array'
/var/www/discourse/lib/mini_sql_multisite_connection.rb:21:in `public_send'
/var/www/discourse/lib/mini_sql_multisite_connection.rb:21:in `encode'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/mini_sql-1.5.0/lib/mini_sql/connection.rb:64:in `to_sql'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/mini_sql-1.5.0/lib/mini_sql/postgres/connection.rb:202:in `run'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/mini_sql-1.5.0/lib/mini_sql/active_record_postgres/connection.rb:38:in `block in run'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/mini_sql-1.5.0/lib/mini_sql/active_record_postgres/connection.rb:34:in `block in with_lock'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.8.1/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.8.1/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.8.1/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.8.1/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/mini_sql-1.5.0/lib/mini_sql/active_record_postgres/connection.rb:34:in `with_lock'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/mini_sql-1.5.0/lib/mini_sql/active_record_postgres/connection.rb:38:in `run'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/mini_sql-1.5.0/lib/mini_sql/postgres/connection.rb:99:in `query'
/var/www/discourse/plugins/discourse-ai/lib/embeddings/vector_representations/base.rb:272:in `asymmetric_rag_fragment_similarity_search'
/var/www/discourse/plugins/discourse-ai/lib/ai_bot/personas/persona.rb:286:in `rag_fragments_prompt'
/var/www/discourse/plugins/discourse-ai/lib/ai_bot/personas/persona.rb:156:in `craft_prompt'
/var/www/discourse/plugins/discourse-ai/lib/ai_bot/bot.rb:54:in `reply'
/var/www/discourse/plugins/discourse-ai/lib/ai_bot/playground.rb:424:in `reply_to'
/var/www/discourse/plugins/discourse-ai/app/jobs/regular/create_ai_reply.rb:18:in `execute'
/var/www/discourse/app/jobs/base.rb:305:in `block (2 levels) in perform'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rails_multisite-6.0.0/lib/rails_multisite/connection_management/null_instance.rb:49:in `with_connection'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rails_multisite-6.0.0/lib/rails_multisite/connection_management.rb:21:in `with_connection'
/var/www/discourse/app/jobs/base.rb:292:in `block in perform'
/var/www/discourse/app/jobs/base.rb:288:in `each'
/var/www/discourse/app/jobs/base.rb:288:in `perform'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:202:in `execute_job'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:170:in `block (2 levels) in process'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/sidekiq-6.5.12/lib/sidekiq/middleware/chain.rb:177:in `block in invoke'
/var/www/discourse/lib/sidekiq/pausable.rb:132:in `call'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/sidekiq-6.5.12/lib/sidekiq/middleware/chain.rb:179:in `block in invoke'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/sidekiq-6.5.12/lib/sidekiq/middleware/chain.rb:182:in `invoke'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:169:in `block in process'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:136:in `block (6 levels) in dispatch'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/sidekiq-6.5.12/lib/sidekiq/job_retry.rb:113:in `local'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:135:in `block (5 levels) in dispatch'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/sidekiq-6.5.12/lib/sidekiq.rb:44:in `block in <module:Sidekiq>'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:131:in `block (4 levels) in dispatch'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:263:in `stats'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:126:in `block (3 levels) in dispatch'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/sidekiq-6.5.12/lib/sidekiq/job_logger.rb:13:in `call'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:125:in `block (2 levels) in dispatch'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/sidekiq-6.5.12/lib/sidekiq/job_retry.rb:80:in `global'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:124:in `block in dispatch'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/sidekiq-6.5.12/lib/sidekiq/job_logger.rb:39:in `prepare'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:123:in `dispatch'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:168:in `process'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:78:in `process_one'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:68:in `run'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/sidekiq-6.5.12/lib/sidekiq/component.rb:8:in `watchdog'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/sidekiq-6.5.12/lib/sidekiq/component.rb:17:in `block in safe_thread'
「いいね！」 2
3

お知らせいただきありがとうございます。確認いたします！

「いいね！」 2
4

Railsコンソールで次のコマンドを実行した場合の出力は何ですか？

strategy = DiscourseAi::Embeddings::Strategies::Truncation.new
vector_rep = DiscourseAi::Embeddings::VectorRepresentations::Base.current_representation(strategy)
vector_rep.vector_from("test")

また、APIは、ドキュメントに従って、誰かが自分で GitHub - huggingface/text-embeddings-inference: A blazing fast inference solution for text embeddings models を実行することに対応するように設計されているため、ホストされているバージョンでは機能しない可能性があります。

バックトレースを提供していただければ、動作するように調査できます。

「いいね！」 2
5

@Falco
AWSの専用エンドポイントインスタンスでbge-large-enを実行し、埋め込みモデルとして設定した状態でテストコードを実行した結果は以下の通りです。

image
image833×121 9.63 KB

root@studyqa-ubuntu-s-1vcpu-2gb-amd-sfo3-01-app:/var/www/discourse# rails c

[1] pry(main)> strategy = DiscourseAi::Embeddings::Strategies::Truncation.new

puts "Strategy initialized"

vector_rep = DiscourseAi::Embeddings::VectorRepresentations::Base.current_representation(strategy)

puts "Vector representation obtained"

vector = vector_rep.vector_from("test")

[1] pry(main)> strategy = DiscourseAi::Embeddings::Strategies::Truncation.new

puts "Strategy initialized"

vector_rep = DiscourseAi::Embeddings::VectorRepresentations::Base.current_representation(strategy)

puts "Vector representation obtained"

vector = vector_rep.vector_from("test")

puts "Vector generated"

puts vector.inspect

Strategy initialized

Vector representation obtained

Vector generated

[:embeddings, [-0.0020444912370294333, 0.008787356317043304, -0.010865539312362671, 0.01865551434457302, -0.02099628746509552, -0.009864491410553455, -0.0011329081607982516, 0.02949545904994011, 0.027839021757245064, 0.043966952711343765, 0.0406080037355423, 0.0016647017328068614, 0.007204003632068634, -0.03770752251148224, -0.025242917239665985, -0.0015279072104021907, -0.02805529721081257, -0.020901955664157867, -0.029206447303295135, -0.006209365092217922, -0.02105099707841873,

etc.

AWSのbge-large-enにヒットしているようです。

- 2024-05-29T13:57:34.609+00:00 Batches: 0%| | 0/1 [00:00<?, ?it/s] Batches: 100%|██████████| 1/1 [00:00<00:00, 4.80it/s] Batches: 100%|██████████| 1/1 [00:00<00:00, 4.79it/s]

• 2024/05/29 09:57:34
INFO | POST / | Duration: 212.84 ms


- 2024-05-29T13:57:53.806+00:00 Batches: 0%| | 0/1 [00:00<?, ?it/s] Batches: 100%|██████████| 1/1 [00:01<00:00, 1.97s/it] Batches: 100%|██████████| 1/1 [00:01<00:00, 1.97s/it]

• 2024/05/29 09:57:53
INFO | POST / | Duration: 1978.36 ms
「いいね！」 1
6

問題なく動作しているようですね。

問題は再ランク付け機能でしょうか？ ai_hugging_face_tei_reranker_endpoint を解除して、RAGが機能するかテストしていただけますか？

7

reranker をオフにしました。まだ embedding されていません。両方のエンドで次のメッセージが表示されています。

Discourse LLM 実行テスト:

モデルに連絡しようとすると、次のエラーが返されました: {“error”:“Body needs to provide a inputs key, recieved: b’{\"model\":\"bge-large-en\",\"temperature\":0.7,\"messages\":[{\"role\":\"system\",\"content\":\"You are a helpful bot\"},{\"role\":\"user\",\"content\":\"How much is 1 + 1?\"}],\"max_tokens\":1009}'”}

bge-large-en ログ

• 2024/05/29 13:40:03

ERROR | Body needs to provide a inputs key, recieved: b’{"model":"bge-large-en","temperature":0.7,"messages":[{"role":"system","content":"You are a helpful bot"},{"role":"user","content":"How much is 1 + 1?"}],"max_tokens":1009}’

discourse b1b218aa99
discourse-ai d812ecf5

8

これは埋め込みをテストする方法ではありません :slight_smile: これはLLMのテストであり、数値が返されることを期待する埋め込みモデルのテストではありません。LLM UIはこれを追加する場所ではなく、まだ持っていない埋め込みUIが必要になります。埋め込みモデルはサイト設定でのみ構成されます。

9

はい、理にかなっています。

（LLM実行テストは「接続性」を確認するためだけに使用したことを注記しようとしました（下記参照）！もっと明確にすべきでした。）

image
image715×305 35.5 KB

「いいね！」 1