Google로부터 embedding-001 모델이 이번 달에 비활성화(deprecated)될 예정이며, 대신 gemini-embedding-001으로 전환하라는 통지를 받았습니다. 저는 이미 전환을 완료했습니다. 하지만 이제 Discourse 로그에 다음과 같은 오류들이 여러 개 발생하고 있습니다:
Error ERROR: expected 768 dimensions, not 3072 querying embeddings for model Gemini embedding
Backtrace
activesupport-8.0.3/lib/active_support/broadcast_logger.rb:218:in `block in dispatch'
activesupport-8.0.3/lib/active_support/broadcast_logger.rb:217:in `map'
activesupport-8.0.3/lib/active_support/broadcast_logger.rb:217:in `dispatch'
activesupport-8.0.3/lib/active_support/broadcast_logger.rb:129:in `error'
/var/www/discourse/plugins/discourse-ai/lib/embeddings/schema.rb:279:in `rescue in symmetric_similarity_search'
/var/www/discourse/plugins/discourse-ai/lib/embeddings/schema.rb:195:in `symmetric_similarity_search'
/var/www/discourse/plugins/discourse-ai/lib/embeddings/semantic_related.rb:24:in `block in related_topic_ids_for'
/var/www/discourse/lib/cache.rb:93:in `fetch'
/var/www/discourse/plugins/discourse-ai/lib/embeddings/semantic_related.rb:21:in `related_topic_ids_for'
/var/www/discourse/plugins/discourse-ai/lib/embeddings/semantic_topic_query.rb:17:in `block in list_semantic_related_topics'
/var/www/discourse/lib/topic_query.rb:532:in `create_list'
/var/www/discourse/plugins/discourse-ai/lib/embeddings/semantic_topic_query.rb:16:in `list_semantic_related_topics'
/var/www/discourse/plugins/discourse-ai/lib/embeddings/entry_point.rb:23:in `block in inject_into'
/var/www/discourse/lib/plugin/instance.rb:435:in `public_send'
/var/www/discourse/lib/plugin/instance.rb:435:in `block (2 levels) in add_to_class'
/var/www/discourse/plugins/discourse-ai/lib/embeddings/entry_point.rb:11:in `block in inject_into'
/var/www/discourse/lib/discourse_plugin_registry.rb:299:in `apply_modifier'
/var/www/discourse/lib/topic_view.rb:726:in `suggested_topics'
/var/www/discourse/app/serializers/suggested_topics_mixin.rb:15:in `include_suggested_topics?'
active_model_serializers-0.8.4/lib/active_model/serializer.rb:375:in `include?'
(eval at active_model_serializers-0.8.4/lib/active_model/serializer.rb:467):7:in `_fast_attributes'
active_model_serializers-0.8.4/lib/active_model/serializer.rb:456:in `attributes'
active_model_serializers-0.8.4/lib/active_model/serializer.rb:480:in `_serializable_hash'
active_model_serializers-0.8.4/lib/active_model/serializer.rb:359:in `serializable_hash'
active_model_serializers-0.8.4/lib/active_model/serializer.rb:347:in `as_json'
activesupport-8.0.3/lib/active_support/json/encoding.rb:41:in `encode'
activesupport-8.0.3/lib/active_support/json/encoding.rb:41:in `encode'
activesupport-8.0.3/lib/active_support/core_ext/object/json.rb:42:in `to_json'
active_model_serializers-0.8.4/lib/active_model/serializer.rb:331:in `to_json'
multi_json-1.17.0/lib/multi_json/adapters/oj.rb:59:in `dump'
multi_json-1.17.0/lib/multi_json/adapters/oj.rb:59:in `dump'
multi_json-1.17.0/lib/multi_json/adapter.rb:26:in `dump'
multi_json-1.17.0/lib/multi_json.rb:128:in `dump'
/var/www/discourse/app/controllers/application_controller.rb:511:in `render_json_dump'
/var/www/discourse/app/controllers/topics_controller.rb:274:in `posts'
actionpack-8.0.3/lib/action_controller/metal/basic_implicit_render.rb:8:in `send_action'
actionpack-8.0.3/lib/abstract_controller/base.rb:215:in `process_action'
actionpack-8.0.3/lib/action_controller/metal/rendering.rb:193:in `process_action'
actionpack-8.0.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'
activesupport-8.0.3/lib/active_support/callbacks.rb:120:in `block in run_callbacks'
/var/www/discourse/app/controllers/application_controller.rb:432:in `block in with_resolved_locale'
i18n-1.14.7/lib/i18n.rb:353:in `with_locale'
/var/www/discourse/app/controllers/application_controller.rb:432:in `with_resolved_locale'
activesupport-8.0.3/lib/active_support/callbacks.rb:129:in `block in run_callbacks'
activesupport-8.0.3/lib/active_support/callbacks.rb:140:in `run_callbacks'
actionpack-8.0.3/lib/abstract_controller/callbacks.rb:260:in `process_action'
actionpack-8.0.3/lib/action_controller/metal/rescue.rb:27:in `process_action'
actionpack-8.0.3/lib/action_controller/metal/instrumentation.rb:76:in `block in process_action'
activesupport-8.0.3/lib/active_support/notifications.rb:210:in `block in instrument'
activesupport-8.0.3/lib/active_support/notifications/instrumenter.rb:58:in `instrument'
activesupport-8.0.3/lib/active_support/notifications.rb:210:in `instrument'
actionpack-8.0.3/lib/action_controller/metal/instrumentation.rb:75:in `process_action'
actionpack-8.0.3/lib/action_controller/metal/params_wrapper.rb:259:in `process_action'
activerecord-8.0.3/lib/active_record/railties/controller_runtime.rb:39:in `process_action'
actionpack-8.0.3/lib/abstract_controller/base.rb:152:in `process'
actionview-8.0.3/lib/action_view/rendering.rb:40:in `process'
rack-mini-profiler-4.0.1/lib/mini_profiler/profiling_methods.rb:90:in `block in profile_method'
actionpack-8.0.3/lib/action_controller/metal.rb:252:in `dispatch'
actionpack-8.0.3/lib/action_controller/metal.rb:335:in `dispatch'
actionpack-8.0.3/lib/action_dispatch/routing/route_set.rb:67:in `dispatch'
actionpack-8.0.3/lib/action_dispatch/routing/route_set.rb:50:in `serve'
actionpack-8.0.3/lib/action_dispatch/journey/router.rb:53:in `block in serve'
actionpack-8.0.3/lib/action_dispatch/journey/router.rb:133:in `block in find_routes'
actionpack-8.0.3/lib/action_dispatch/journey/router.rb:126:in `each'
actionpack-8.0.3/lib/action_dispatch/journey/router.rb:126:in `find_routes'
actionpack-8.0.3/lib/action_dispatch/journey/router.rb:34:in `serve'
actionpack-8.0.3/lib/action_dispatch/routing/route_set.rb:908:in `call'
/var/www/discourse/lib/middleware/omniauth_bypass_middleware.rb:35:in `call'
/var/www/discourse/lib/middleware/crawler_hooks.rb:11:in `call'
rack-2.2.18/lib/rack/tempfile_reaper.rb:15:in `call'
rack-2.2.18/lib/rack/conditional_get.rb:27:in `call'
rack-2.2.18/lib/rack/head.rb:12:in `call'
actionpack-8.0.3/lib/action_dispatch/http/permissions_policy.rb:38:in `call'
/var/www/discourse/lib/content_security_policy/middleware.rb:12:in `call'
/var/www/discourse/lib/middleware/anonymous_cache.rb:415:in `call'
/var/www/discourse/lib/middleware/csp_script_nonce_injector.rb:12:in `call'
/var/www/discourse/config/initializers/008-rack-cors.rb:14:in `call'
rack-2.2.18/lib/rack/session/abstract/id.rb:266:in `context'
rack-2.2.18/lib/rack/session/abstract/id.rb:260:in `call'
actionpack-8.0.3/lib/action_dispatch/middleware/cookies.rb:706:in `call'
actionpack-8.0.3/lib/action_dispatch/middleware/callbacks.rb:31:in `block in call'
activesupport-8.0.3/lib/active_support/callbacks.rb:100:in `run_callbacks'
actionpack-8.0.3/lib/action_dispatch/middleware/callbacks.rb:30:in `call'
actionpack-8.0.3/lib/action_dispatch/middleware/debug_exceptions.rb:31:in `call'
actionpack-8.0.3/lib/action_dispatch/middleware/show_exceptions.rb:32:in `call'
logster-2.20.1/lib/logster/middleware/reporter.rb:40:in `call'
/var/www/discourse/lib/middleware/default_headers.rb:13:in `call'
railties-8.0.3/lib/rails/rack/logger.rb:41:in `call_app'
railties-8.0.3/lib/rails/rack/logger.rb:29:in `call'
/var/www/discourse/config/initializers/100-quiet_logger.rb:20:in `call'
/var/www/discourse/config/initializers/100-silence_logger.rb:29:in `call'
actionpack-8.0.3/lib/action_dispatch/middleware/request_id.rb:34:in `call'
/var/www/discourse/lib/middleware/enforce_hostname.rb:23:in `call'
rack-2.2.18/lib/rack/method_override.rb:24:in `call'
actionpack-8.0.3/lib/action_dispatch/middleware/executor.rb:16:in `call'
rack-2.2.18/lib/rack/sendfile.rb:110:in `call'
rack-mini-profiler-4.0.1/lib/mini_profiler.rb:191:in `call'
/var/www/discourse/lib/middleware/processing_request.rb:12:in `call'
message_bus-4.4.1/lib/message_bus/rack/middleware.rb:60:in `call'
/var/www/discourse/lib/middleware/request_tracker.rb:410:in `call'
actionpack-8.0.3/lib/action_dispatch/middleware/remote_ip.rb:96:in `call'
railties-8.0.3/lib/rails/engine.rb:535:in `call'
railties-8.0.3/lib/rails/railtie.rb:226:in `public_send'
railties-8.0.3/lib/rails/railtie.rb:226:in `method_missing'
rack-2.2.18/lib/rack/urlmap.rb:74:in `block in call'
rack-2.2.18/lib/rack/urlmap.rb:58:in `each'
rack-2.2.18/lib/rack/urlmap.rb:58:in `call'
unicorn-6.1.0/lib/unicorn/http_server.rb:634:in `process_client'
unicorn-6.1.0/lib/unicorn/http_server.rb:739:in `worker_loop'
unicorn-6.1.0/lib/unicorn/http_server.rb:547:in `spawn_missing_workers'
unicorn-6.1.0/lib/unicorn/http_server.rb:143:in `start'
unicorn-6.1.0/bin/unicorn:128:in `<top (required)>'
/var/www/discourse/vendor/bundle/ruby/3.3.0/bin/unicorn:25:in `load'
/var/www/discourse/vendor/bundle/ruby/3.3.0/bin/unicorn:25:in `<main>'
Env
HTTP HOSTS: discourse.spacecityweather.com```
gemini-embedding-001 모델은 요청 JSON 페이로드에 output_dimensionality 파라미터를 받지 않는 한 3072차원 벡터를 반환하는 것 같습니다. 제 Discourse 임베딩 설정에는 "768"이 지정되어 있지만, 해당 파라미터가 전송되지 않거나 인식되지 않는 것으로 보입니다.
제공자를 변경할 수도 있지만, 저는 Google Cloud 크레딧을 사용하고자 하므로 임베딩에는 gemini를 계속 사용하는 것이 좋습니다.
여기서 해결 방법은 무엇인가요? Discourse 패치를 기다려야 하는지, 아니면 3072차원 벡터를 사용하는 gemini 기반 새로운 임베딩 설정을 구성하는 것이 안전한지 궁금합니다.