요약 기능이 작동하지 않아요 :(

Wurzelseppi · 5월 28, 2025, 5:41오전

여러분,

이것은 셀프호스팅 인스턴스에 관한 것입니다

Discourse AI 플러그인을 활성화했는데 몇 가지 문제가 발생했습니다.
Enterprise 환경에서 작업 중이며, AI 접근은 추가 인증이 필요한 내부 엔드포인트를 통해 이루어집니다.
필요한 인증 정보를 헤더에 추가하고 요청을 OPENAI 호환 내부 엔드포인트로 전달하는 역할을 수행하는 Lambda를 갖춘 AWS API Gateway 엔드포인트를 구현했습니다.

LLM 테스트가

를 표시했고, 저는 문제가 없다고 생각했습니다.

또한 API Gateway 호출에 대한 관련 요청을 CloudWatch Logs에서도 확인할 수 있었습니다.

그런 다음 Summarization Personas(Smmmarization 및 … (약식))를 이 LLM과 연결하고 Topic Summarization을 활성화했습니다.

하지만 지금 제가 보고 있는 것은 제가 원했던 것이 아닙니다

그리고 이것은 변하지 않습니다

가장 이상한 점은, 요약 시도 요청이 CloudWatch Logs에 보이지 않는다는 것입니다.

다시 한 번 제가 한 작업을 정리하면 다음과 같습니다.

AI 활성화
제 API Gateway 엔드포인트를 사용하여 vLLM 생성
2개의 Persona(위 참조) 생성 및 제 LLM 설정과 연결
Summarization 활성화 및 해당 Persona 연결(둘 다 시도함)

다음에 무엇을 시도해 볼 수 있을지 모르겠습니다

도와주셔서 감사합니다. 인사를 전합니다.

JP

sam · 5월 28, 2025, 6:36오전

Our test endpoint is a bit misleading and this is something I would like to fix.

We currently only test “non streaming” endpoints. “streaming” is not tested.

Does everything work if you disable streaming on your LLM, maybe it is not implemented by your proxy.

Wurzelseppi · 5월 28, 2025, 6:53오전

Hey @sam,

I disabled the streaming (hard coded) in my LAMBDA while passing the request on.

Thing is, that the LAMBDA doesn´t even get triggered by the summarization API (i see no requests).

The LLM Test issues a call, but the summarization button in a topic doesn´t.

Nothing, no errors (i switched debug level on with DISCOURSE_LOG_LEVEL=debug) in discourse log, nor any logs in my LAMBDA (CloudWatch).

As if I didn´t press anything at all

Thanks and Cheers,

Julian

Falco · 5월 28, 2025, 3:27오후

Why is the provider set to vLLM if it is an OpenAI compatible endpoint?

Wurzelseppi · 6월 2, 2025, 6:06오전

Hi,

I´ve tried many different engines. Sorry, the screenshot was misleading. I´ve tried OpenAI also …

This is the answer from the endpoint which I send 1:1 back to discourse instance.
With this I´m getting an “Internal Server Error” @ discourse (not sure why, because the test worked until this morning )

{
  "statusCode": 200,
  "headers": {
    "date": "Mon, 02 Jun 2025 05:57:44 GMT",
    "content-type": "application/json",
    "content-length": "1315",
    "x-content-type-options": "nosniff",
    "x-frame-options": "DENY",
    "strict-transport-security": "max-age=63072000; includeSubdomains; preload",
    "content-security-policy": "default-src 'none'; img-src 'self'; script-src 'self'; style-src 'self'; object-src 'none'",
    "referrer-policy": "no-referrer",
    "x-xss-protection": "0",
    "x-request-id": "02d33931-1e29-418e-b9a7-4eba07710e8e",
    "x-accel-buffering": "no",
    "via": "1.1 google"
  },
  "body": {
    "id": "24c1474f-bd51-4542-b87a-8608fd90b5fb",
    "choices": [
      {
        "finish_reason": "stop",
        "index": 0,
        "logprobs": null,
        "message": {
          "content": "1 + 1 equals 2.",
          "refusal": null,
          "role": "assistant",
          "annotations": [],
          "audio": null,
          "function_call": null,
          "tool_calls": null,
          "reasoning_content": null
        },
        "content_filter_results": {
          "hate": {
            "filtered": false,
            "severity": "safe"
          },
          "profanity": {
            "filtered": false,
            "detected": false
          },
          "protected_material_code": {
            "filtered": false,
            "detected": false
          },
          "protected_material_text": {
            "filtered": false,
            "detected": false
          },
          "self_harm": {
            "filtered": false,
            "severity": "safe"
          },
          "sexual": {
            "filtered": false,
            "severity": "safe"
          },
          "violence": {
            "filtered": false,
            "severity": "safe"
          }
        }
      }
    ],
    "created": 1748843864,
    "model": "openai/gpt-4o",
    "object": "chat.completion",
    "usage": {
      "prompt_tokens": 25,
      "completion_tokens": 9,
      "total_tokens": 34,
      "prompt_tokens_details": {
        "audio_tokens": 0,
        "cached_tokens": 0
      },
      "completion_tokens_details": {
        "accepted_prediction_tokens": 0,
        "audio_tokens": 0,
        "reasoning_tokens": 0,
        "rejected_prediction_tokens": 0,
        "cached_tokens": 0
      }
    },
    "cost": {
      "interaction": {
        "prompt_tokens": 0.0000688,
        "prompt_tokens_details": {
          "standard": 0.0000688,
          "cached": 0
        },
        "completion_tokens": 0.000099,
        "completion_tokens_details": {
          "standard": 0.000099,
          "reasoning": 0,
          "cached": 0
        },
        "total": 0.0001678,
        "currency": "USD"
      },
      "integrations": {},
      "total": 0.0001678,
      "currency": "USD"
    }
  }
}

@ discourse logs:

 Back to site
DiscourseAi::Completions::Endpoints::Vllm: status: 502 - body: {"message": "Internal server error"}
8:07 am
activesupport-7.2.2.1/lib/active_support/broadcast_logger.rb:134:in `block in error' 
activesupport-7.2.2.1/lib/active_support/broadcast_logger.rb:231:in `block in dispatch' 
activesupport-7.2.2.1/lib/active_support/broadcast_logger.rb:231:in `each' 
activesupport-7.2.2.1/lib/active_support/broadcast_logger.rb:231:in `dispatch' 
activesupport-7.2.2.1/lib/active_support/broadcast_logger.rb:134:in `error' 
/var/www/discourse/plugins/discourse-ai/lib/completions/endpoints/base.rb:160:in `block (2 levels) in perform_completion!' 
net-http-0.6.0/lib/net/http.rb:2433:in `block in transport_request' 
net-http-0.6.0/lib/net/http/response.rb:320:in `reading_body' 
net-http-0.6.0/lib/net/http.rb:2430:in `transport_request' 
net-http-0.6.0/lib/net/http.rb:2384:in `request' 
rack-mini-profiler-3.3.1/lib/patches/net_patches.rb:19:in `block in request_with_mini_profiler' 
rack-mini-profiler-3.3.1/lib/mini_profiler/profiling_methods.rb:50:in `step' 
rack-mini-profiler-3.3.1/lib/patches/net_patches.rb:18:in `request_with_mini_profiler' 
/var/www/discourse/plugins/discourse-ai/lib/completions/endpoints/base.rb:158:in `block in perform_completion!' 
net-http-0.6.0/lib/net/http.rb:1632:in `start' 
net-http-0.6.0/lib/net/http.rb:1070:in `start' 
/var/www/discourse/plugins/discourse-ai/lib/completions/endpoints/base.rb:129:in `perform_completion!' 
/var/www/discourse/plugins/discourse-ai/lib/completions/llm.rb:374:in `generate' 
/var/www/discourse/plugins/discourse-ai/lib/configuration/llm_validator.rb:36:in `run_test' 
/var/www/discourse/plugins/discourse-ai/app/controllers/discourse_ai/admin/ai_llms_controller.rb:128:in `test' 
actionpack-7.2.2.1/lib/action_controller/metal/basic_implicit_render.rb:8:in `send_action' 
actionpack-7.2.2.1/lib/abstract_controller/base.rb:226:in `process_action' 
actionpack-7.2.2.1/lib/action_controller/metal/rendering.rb:193:in `process_action' 
actionpack-7.2.2.1/lib/abstract_controller/callbacks.rb:261:in `block in process_action' 
activesupport-7.2.2.1/lib/active_support/callbacks.rb:121:in `block in run_callbacks' 
/var/www/discourse/app/controllers/application_controller.rb:428:in `block in with_resolved_locale' 
i18n-1.14.7/lib/i18n.rb:353:in `with_locale' 
/var/www/discourse/app/controllers/application_controller.rb:428:in `with_resolved_locale' 
activesupport-7.2.2.1/lib/active_support/callbacks.rb:130:in `block in run_callbacks' 
activesupport-7.2.2.1/lib/active_support/callbacks.rb:141:in `run_callbacks' 
actionpack-7.2.2.1/lib/abstract_controller/callbacks.rb:260:in `process_action' 
actionpack-7.2.2.1/lib/action_controller/metal/rescue.rb:27:in `process_action' 
actionpack-7.2.2.1/lib/action_controller/metal/instrumentation.rb:77:in `block in process_action' 
activesupport-7.2.2.1/lib/active_support/notifications.rb:210:in `block in instrument' 
activesupport-7.2.2.1/lib/active_support/notifications/instrumenter.rb:58:in `instrument' 
activesupport-7.2.2.1/lib/active_support/notifications.rb:210:in `instrument' 
actionpack-7.2.2.1/lib/action_controller/metal/instrumentation.rb:76:in `process_action' 
actionpack-7.2.2.1/lib/action_controller/metal/params_wrapper.rb:259:in `process_action' 
activerecord-7.2.2.1/lib/active_record/railties/controller_runtime.rb:39:in `process_action' 
actionpack-7.2.2.1/lib/abstract_controller/base.rb:163:in `process' 
actionview-7.2.2.1/lib/action_view/rendering.rb:40:in `process' 
rack-mini-profiler-3.3.1/lib/mini_profiler/profiling_methods.rb:89:in `block in profile_method' 
actionpack-7.2.2.1/lib/action_controller/metal.rb:252:in `dispatch' 
actionpack-7.2.2.1/lib/action_controller/metal.rb:335:in `dispatch' 
actionpack-7.2.2.1/lib/action_dispatch/routing/route_set.rb:67:in `dispatch' 
actionpack-7.2.2.1/lib/action_dispatch/routing/route_set.rb:50:in `serve' 
actionpack-7.2.2.1/lib/action_dispatch/routing/mapper.rb:32:in `block in <class:Constraints>' 
actionpack-7.2.2.1/lib/action_dispatch/routing/mapper.rb:62:in `serve' 
actionpack-7.2.2.1/lib/action_dispatch/journey/router.rb:53:in `block in serve' 
actionpack-7.2.2.1/lib/action_dispatch/journey/router.rb:133:in `block in find_routes' 
actionpack-7.2.2.1/lib/action_dispatch/journey/router.rb:126:in `each' 
actionpack-7.2.2.1/lib/action_dispatch/journey/router.rb:126:in `find_routes' 
actionpack-7.2.2.1/lib/action_dispatch/journey/router.rb:34:in `serve' 
actionpack-7.2.2.1/lib/action_dispatch/routing/route_set.rb:896:in `call' 
/var/www/discourse/lib/middleware/omniauth_bypass_middleware.rb:35:in `call' 
rack-2.2.13/lib/rack/tempfile_reaper.rb:15:in `call' 
rack-2.2.13/lib/rack/conditional_get.rb:27:in `call' 
rack-2.2.13/lib/rack/head.rb:12:in `call' 
actionpack-7.2.2.1/lib/action_dispatch/http/permissions_policy.rb:38:in `call' 
/var/www/discourse/lib/content_security_policy/middleware.rb:12:in `call' 
/var/www/discourse/lib/middleware/anonymous_cache.rb:415:in `call' 
/var/www/discourse/lib/middleware/csp_script_nonce_injector.rb:12:in `call' 
/var/www/discourse/config/initializers/008-rack-cors.rb:14:in `call' 
rack-2.2.13/lib/rack/session/abstract/id.rb:266:in `context' 
rack-2.2.13/lib/rack/session/abstract/id.rb:260:in `call' 
actionpack-7.2.2.1/lib/action_dispatch/middleware/cookies.rb:704:in `call' 
actionpack-7.2.2.1/lib/action_dispatch/middleware/callbacks.rb:31:in `block in call' 
activesupport-7.2.2.1/lib/active_support/callbacks.rb:101:in `run_callbacks' 
actionpack-7.2.2.1/lib/action_dispatch/middleware/callbacks.rb:30:in `call' 
actionpack-7.2.2.1/lib/action_dispatch/middleware/debug_exceptions.rb:31:in `call' 
actionpack-7.2.2.1/lib/action_dispatch/middleware/show_exceptions.rb:32:in `call' 
logster-2.20.1/lib/logster/middleware/reporter.rb:40:in `call' 
/var/www/discourse/lib/middleware/default_headers.rb:13:in `call' 
railties-7.2.2.1/lib/rails/rack/logger.rb:41:in `call_app' 
railties-7.2.2.1/lib/rails/rack/logger.rb:29:in `call' 
/var/www/discourse/config/initializers/100-quiet_logger.rb:20:in `call' 
/var/www/discourse/config/initializers/100-silence_logger.rb:29:in `call' 
actionpack-7.2.2.1/lib/action_dispatch/middleware/request_id.rb:33:in `call' 
/var/www/discourse/lib/middleware/enforce_hostname.rb:24:in `call' 
rack-2.2.13/lib/rack/method_override.rb:24:in `call' 
actionpack-7.2.2.1/lib/action_dispatch/middleware/executor.rb:16:in `call' 
rack-2.2.13/lib/rack/sendfile.rb:110:in `call' 
rack-mini-profiler-3.3.1/lib/mini_profiler.rb:191:in `call' 
/var/www/discourse/lib/middleware/processing_request.rb:12:in `call' 
message_bus-4.4.1/lib/message_bus/rack/middleware.rb:60:in `call' 
/var/www/discourse/lib/middleware/request_tracker.rb:385:in `call' 
actionpack-7.2.2.1/lib/action_dispatch/middleware/remote_ip.rb:96:in `call' 
railties-7.2.2.1/lib/rails/engine.rb:535:in `call' 
railties-7.2.2.1/lib/rails/railtie.rb:226:in `public_send' 
railties-7.2.2.1/lib/rails/railtie.rb:226:in `method_missing' 
rack-2.2.13/lib/rack/urlmap.rb:74:in `block in call' 
rack-2.2.13/lib/rack/urlmap.rb:58:in `each' 
rack-2.2.13/lib/rack/urlmap.rb:58:in `call' 
unicorn-6.1.0/lib/unicorn/http_server.rb:634:in `process_client'
unicorn-6.1.0/lib/unicorn/http_server.rb:739:in `worker_loop'
unicorn-6.1.0/lib/unicorn/http_server.rb:547:in `spawn_missing_workers'
unicorn-6.1.0/lib/unicorn/http_server.rb:143:in `start'
unicorn-6.1.0/bin/unicorn:128:in `<top (required)>'
/var/www/discourse/vendor/bundle/ruby/3.3.0/bin/unicorn:25:in `load' 
/var/www/discourse/vendor/bundle/ruby/3.3.0/bin/unicorn:25:in `<main>' 
infobacktraceenv
 Solve  Remove  Protect  Copy  Share
 Debug
 Info
  Warning
  Error
  Fatal
Search

 Clear logs

Wurzelseppi · 6월 3, 2025, 12:22오후

Hey guys,

The translator and Forum Helper are working now, but the Summarize Button doesn´t trigger anything.

I´m not seeing anything in my CloudWatch logs that any request has been triggered from discourse.

The summarizer is configured with the Summarize Persona, and the Perso is using the LLM endpoint which I meantioned earlier in this discussion …

Any hints ?

Many thanks and Greetings,

WS

Wurzelseppi · 6월 3, 2025, 1:59오후

New insight:

Some topics can be summarized, some not…

It seems that the new ones can be summarized.
And another question: When I have a topic with pictures in it, and don’t have any image-recognizing AI in place, is this a problem?

Falco · 6월 3, 2025, 2:55오후

That is a 502 from your own endpoint

Topic length maybe?

It will be for image related tasks, like image caption or post triage automation.

Wurzelseppi · 6월 4, 2025, 4:33오전

Hey @Falco

But discourse even start the request, nor gives any reason (logs, …) why it doesn´t
Hard to narrow down my options

No, it wasn´t´. I found the problem (didn´t JSON.stringify the body from llm endpoint, but discourse could be a bit more talkative about the reason for this 502 )

Thanks and cheers,

WS

글		댓글	조회수
Discourse AI - Summarize Site Management how-to , ai , ai-summarize	32	8557	6월 5, 2025
You need to select a model before the AI Summarizer Persona works Bug ai , ai-summarize	1	175	7월 20, 2025
Setup Inquiry: AI Summarization in Discourse & LLM Integration Support ai	10	402	10월 6, 2025
Customizing AI summarizer to use non-English languages Support ai , ai-summarize	5	283	2월 16, 2026
AI topic summarize hangs (Gemini) Support	2	150	5월 20, 2025

요약 기능이 작동하지 않아요 :(

Related topics