Topic Summarization doesn't work :-(

Wurzelseppi · May 28, 2025, 5:41am

Hey guys,

This is about a selfhosted instance

I´ve activated the Discourse AI Plugin and had a few problems there.
Working in an enterprise environment and AI access is though an internal endpoint with additional authentication.
I implemented an AWS API Gateway endpoint with a Lambda, that takes care of enriching the headers with the required auth info and pass the request on to the OPENAI compatible internal endpoint.

The

LLM Test gives me a

and I thought I´m good to go.

I also see the related calls in my CloudWatch Logs for the API Gateway call.

Then I related the Summarization Personas (Smmmarization and … (short form) with this LLM and activated the Topic Summarization.

What I´m seeing now, though, is not what I wanted to see

And that´s not changing

The strangest thing is, that I don´t see a request in CloudWatch Logs for the attempt to summarize.

Ok, again what I did.

Activated AI
Created vLLM with my API Gateway endpoint
Created 2 Personas (see above) and related them with my LLM config
Activated Summarization and related there the Persona (tried both)

Not sure what I can try next

Thanks guys and Greetings,

JP

sam · May 28, 2025, 6:36am

Our test endpoint is a bit misleading and this is something I would like to fix.

We currently only test “non streaming” endpoints. “streaming” is not tested.

Does everything work if you disable streaming on your LLM, maybe it is not implemented by your proxy.

Wurzelseppi · May 28, 2025, 6:53am

Hey @sam,

I disabled the streaming (hard coded) in my LAMBDA while passing the request on.

Thing is, that the LAMBDA doesn´t even get triggered by the summarization API (i see no requests).

The LLM Test issues a call, but the summarization button in a topic doesn´t.

Nothing, no errors (i switched debug level on with DISCOURSE_LOG_LEVEL=debug) in discourse log, nor any logs in my LAMBDA (CloudWatch).

As if I didn´t press anything at all

Thanks and Cheers,

Julian

Falco · May 28, 2025, 3:27pm

Why is the provider set to vLLM if it is an OpenAI compatible endpoint?

Wurzelseppi · June 2, 2025, 6:06am

Hi,

I´ve tried many different engines. Sorry, the screenshot was misleading. I´ve tried OpenAI also …

This is the answer from the endpoint which I send 1:1 back to discourse instance.
With this I´m getting an “Internal Server Error” @ discourse (not sure why, because the test worked until this morning )

{
  "statusCode": 200,
  "headers": {
    "date": "Mon, 02 Jun 2025 05:57:44 GMT",
    "content-type": "application/json",
    "content-length": "1315",
    "x-content-type-options": "nosniff",
    "x-frame-options": "DENY",
    "strict-transport-security": "max-age=63072000; includeSubdomains; preload",
    "content-security-policy": "default-src 'none'; img-src 'self'; script-src 'self'; style-src 'self'; object-src 'none'",
    "referrer-policy": "no-referrer",
    "x-xss-protection": "0",
    "x-request-id": "02d33931-1e29-418e-b9a7-4eba07710e8e",
    "x-accel-buffering": "no",
    "via": "1.1 google"
  },
  "body": {
    "id": "24c1474f-bd51-4542-b87a-8608fd90b5fb",
    "choices": [
      {
        "finish_reason": "stop",
        "index": 0,
        "logprobs": null,
        "message": {
          "content": "1 + 1 equals 2.",
          "refusal": null,
          "role": "assistant",
          "annotations": [],
          "audio": null,
          "function_call": null,
          "tool_calls": null,
          "reasoning_content": null
        },
        "content_filter_results": {
          "hate": {
            "filtered": false,
            "severity": "safe"
          },
          "profanity": {
            "filtered": false,
            "detected": false
          },
          "protected_material_code": {
            "filtered": false,
            "detected": false
          },
          "protected_material_text": {
            "filtered": false,
            "detected": false
          },
          "self_harm": {
            "filtered": false,
            "severity": "safe"
          },
          "sexual": {
            "filtered": false,
            "severity": "safe"
          },
          "violence": {
            "filtered": false,
            "severity": "safe"
          }
        }
      }
    ],
    "created": 1748843864,
    "model": "openai/gpt-4o",
    "object": "chat.completion",
    "usage": {
      "prompt_tokens": 25,
      "completion_tokens": 9,
      "total_tokens": 34,
      "prompt_tokens_details": {
        "audio_tokens": 0,
        "cached_tokens": 0
      },
      "completion_tokens_details": {
        "accepted_prediction_tokens": 0,
        "audio_tokens": 0,
        "reasoning_tokens": 0,
        "rejected_prediction_tokens": 0,
        "cached_tokens": 0
      }
    },
    "cost": {
      "interaction": {
        "prompt_tokens": 0.0000688,
        "prompt_tokens_details": {
          "standard": 0.0000688,
          "cached": 0
        },
        "completion_tokens": 0.000099,
        "completion_tokens_details": {
          "standard": 0.000099,
          "reasoning": 0,
          "cached": 0
        },
        "total": 0.0001678,
        "currency": "USD"
      },
      "integrations": {},
      "total": 0.0001678,
      "currency": "USD"
    }
  }
}

@ discourse logs:

 Back to site
DiscourseAi::Completions::Endpoints::Vllm: status: 502 - body: {"message": "Internal server error"}
8:07 am
activesupport-7.2.2.1/lib/active_support/broadcast_logger.rb:134:in `block in error' 
activesupport-7.2.2.1/lib/active_support/broadcast_logger.rb:231:in `block in dispatch' 
activesupport-7.2.2.1/lib/active_support/broadcast_logger.rb:231:in `each' 
activesupport-7.2.2.1/lib/active_support/broadcast_logger.rb:231:in `dispatch' 
activesupport-7.2.2.1/lib/active_support/broadcast_logger.rb:134:in `error' 
/var/www/discourse/plugins/discourse-ai/lib/completions/endpoints/base.rb:160:in `block (2 levels) in perform_completion!' 
net-http-0.6.0/lib/net/http.rb:2433:in `block in transport_request' 
net-http-0.6.0/lib/net/http/response.rb:320:in `reading_body' 
net-http-0.6.0/lib/net/http.rb:2430:in `transport_request' 
net-http-0.6.0/lib/net/http.rb:2384:in `request' 
rack-mini-profiler-3.3.1/lib/patches/net_patches.rb:19:in `block in request_with_mini_profiler' 
rack-mini-profiler-3.3.1/lib/mini_profiler/profiling_methods.rb:50:in `step' 
rack-mini-profiler-3.3.1/lib/patches/net_patches.rb:18:in `request_with_mini_profiler' 
/var/www/discourse/plugins/discourse-ai/lib/completions/endpoints/base.rb:158:in `block in perform_completion!' 
net-http-0.6.0/lib/net/http.rb:1632:in `start' 
net-http-0.6.0/lib/net/http.rb:1070:in `start' 
/var/www/discourse/plugins/discourse-ai/lib/completions/endpoints/base.rb:129:in `perform_completion!' 
/var/www/discourse/plugins/discourse-ai/lib/completions/llm.rb:374:in `generate' 
/var/www/discourse/plugins/discourse-ai/lib/configuration/llm_validator.rb:36:in `run_test' 
/var/www/discourse/plugins/discourse-ai/app/controllers/discourse_ai/admin/ai_llms_controller.rb:128:in `test' 
actionpack-7.2.2.1/lib/action_controller/metal/basic_implicit_render.rb:8:in `send_action' 
actionpack-7.2.2.1/lib/abstract_controller/base.rb:226:in `process_action' 
actionpack-7.2.2.1/lib/action_controller/metal/rendering.rb:193:in `process_action' 
actionpack-7.2.2.1/lib/abstract_controller/callbacks.rb:261:in `block in process_action' 
activesupport-7.2.2.1/lib/active_support/callbacks.rb:121:in `block in run_callbacks' 
/var/www/discourse/app/controllers/application_controller.rb:428:in `block in with_resolved_locale' 
i18n-1.14.7/lib/i18n.rb:353:in `with_locale' 
/var/www/discourse/app/controllers/application_controller.rb:428:in `with_resolved_locale' 
activesupport-7.2.2.1/lib/active_support/callbacks.rb:130:in `block in run_callbacks' 
activesupport-7.2.2.1/lib/active_support/callbacks.rb:141:in `run_callbacks' 
actionpack-7.2.2.1/lib/abstract_controller/callbacks.rb:260:in `process_action' 
actionpack-7.2.2.1/lib/action_controller/metal/rescue.rb:27:in `process_action' 
actionpack-7.2.2.1/lib/action_controller/metal/instrumentation.rb:77:in `block in process_action' 
activesupport-7.2.2.1/lib/active_support/notifications.rb:210:in `block in instrument' 
activesupport-7.2.2.1/lib/active_support/notifications/instrumenter.rb:58:in `instrument' 
activesupport-7.2.2.1/lib/active_support/notifications.rb:210:in `instrument' 
actionpack-7.2.2.1/lib/action_controller/metal/instrumentation.rb:76:in `process_action' 
actionpack-7.2.2.1/lib/action_controller/metal/params_wrapper.rb:259:in `process_action' 
activerecord-7.2.2.1/lib/active_record/railties/controller_runtime.rb:39:in `process_action' 
actionpack-7.2.2.1/lib/abstract_controller/base.rb:163:in `process' 
actionview-7.2.2.1/lib/action_view/rendering.rb:40:in `process' 
rack-mini-profiler-3.3.1/lib/mini_profiler/profiling_methods.rb:89:in `block in profile_method' 
actionpack-7.2.2.1/lib/action_controller/metal.rb:252:in `dispatch' 
actionpack-7.2.2.1/lib/action_controller/metal.rb:335:in `dispatch' 
actionpack-7.2.2.1/lib/action_dispatch/routing/route_set.rb:67:in `dispatch' 
actionpack-7.2.2.1/lib/action_dispatch/routing/route_set.rb:50:in `serve' 
actionpack-7.2.2.1/lib/action_dispatch/routing/mapper.rb:32:in `block in <class:Constraints>' 
actionpack-7.2.2.1/lib/action_dispatch/routing/mapper.rb:62:in `serve' 
actionpack-7.2.2.1/lib/action_dispatch/journey/router.rb:53:in `block in serve' 
actionpack-7.2.2.1/lib/action_dispatch/journey/router.rb:133:in `block in find_routes' 
actionpack-7.2.2.1/lib/action_dispatch/journey/router.rb:126:in `each' 
actionpack-7.2.2.1/lib/action_dispatch/journey/router.rb:126:in `find_routes' 
actionpack-7.2.2.1/lib/action_dispatch/journey/router.rb:34:in `serve' 
actionpack-7.2.2.1/lib/action_dispatch/routing/route_set.rb:896:in `call' 
/var/www/discourse/lib/middleware/omniauth_bypass_middleware.rb:35:in `call' 
rack-2.2.13/lib/rack/tempfile_reaper.rb:15:in `call' 
rack-2.2.13/lib/rack/conditional_get.rb:27:in `call' 
rack-2.2.13/lib/rack/head.rb:12:in `call' 
actionpack-7.2.2.1/lib/action_dispatch/http/permissions_policy.rb:38:in `call' 
/var/www/discourse/lib/content_security_policy/middleware.rb:12:in `call' 
/var/www/discourse/lib/middleware/anonymous_cache.rb:415:in `call' 
/var/www/discourse/lib/middleware/csp_script_nonce_injector.rb:12:in `call' 
/var/www/discourse/config/initializers/008-rack-cors.rb:14:in `call' 
rack-2.2.13/lib/rack/session/abstract/id.rb:266:in `context' 
rack-2.2.13/lib/rack/session/abstract/id.rb:260:in `call' 
actionpack-7.2.2.1/lib/action_dispatch/middleware/cookies.rb:704:in `call' 
actionpack-7.2.2.1/lib/action_dispatch/middleware/callbacks.rb:31:in `block in call' 
activesupport-7.2.2.1/lib/active_support/callbacks.rb:101:in `run_callbacks' 
actionpack-7.2.2.1/lib/action_dispatch/middleware/callbacks.rb:30:in `call' 
actionpack-7.2.2.1/lib/action_dispatch/middleware/debug_exceptions.rb:31:in `call' 
actionpack-7.2.2.1/lib/action_dispatch/middleware/show_exceptions.rb:32:in `call' 
logster-2.20.1/lib/logster/middleware/reporter.rb:40:in `call' 
/var/www/discourse/lib/middleware/default_headers.rb:13:in `call' 
railties-7.2.2.1/lib/rails/rack/logger.rb:41:in `call_app' 
railties-7.2.2.1/lib/rails/rack/logger.rb:29:in `call' 
/var/www/discourse/config/initializers/100-quiet_logger.rb:20:in `call' 
/var/www/discourse/config/initializers/100-silence_logger.rb:29:in `call' 
actionpack-7.2.2.1/lib/action_dispatch/middleware/request_id.rb:33:in `call' 
/var/www/discourse/lib/middleware/enforce_hostname.rb:24:in `call' 
rack-2.2.13/lib/rack/method_override.rb:24:in `call' 
actionpack-7.2.2.1/lib/action_dispatch/middleware/executor.rb:16:in `call' 
rack-2.2.13/lib/rack/sendfile.rb:110:in `call' 
rack-mini-profiler-3.3.1/lib/mini_profiler.rb:191:in `call' 
/var/www/discourse/lib/middleware/processing_request.rb:12:in `call' 
message_bus-4.4.1/lib/message_bus/rack/middleware.rb:60:in `call' 
/var/www/discourse/lib/middleware/request_tracker.rb:385:in `call' 
actionpack-7.2.2.1/lib/action_dispatch/middleware/remote_ip.rb:96:in `call' 
railties-7.2.2.1/lib/rails/engine.rb:535:in `call' 
railties-7.2.2.1/lib/rails/railtie.rb:226:in `public_send' 
railties-7.2.2.1/lib/rails/railtie.rb:226:in `method_missing' 
rack-2.2.13/lib/rack/urlmap.rb:74:in `block in call' 
rack-2.2.13/lib/rack/urlmap.rb:58:in `each' 
rack-2.2.13/lib/rack/urlmap.rb:58:in `call' 
unicorn-6.1.0/lib/unicorn/http_server.rb:634:in `process_client'
unicorn-6.1.0/lib/unicorn/http_server.rb:739:in `worker_loop'
unicorn-6.1.0/lib/unicorn/http_server.rb:547:in `spawn_missing_workers'
unicorn-6.1.0/lib/unicorn/http_server.rb:143:in `start'
unicorn-6.1.0/bin/unicorn:128:in `<top (required)>'
/var/www/discourse/vendor/bundle/ruby/3.3.0/bin/unicorn:25:in `load' 
/var/www/discourse/vendor/bundle/ruby/3.3.0/bin/unicorn:25:in `<main>' 
infobacktraceenv
 Solve  Remove  Protect  Copy  Share
 Debug
 Info
  Warning
  Error
  Fatal
Search

 Clear logs

Wurzelseppi · June 3, 2025, 12:22pm

Hey guys,

The translator and Forum Helper are working now, but the Summarize Button doesn´t trigger anything.

I´m not seeing anything in my CloudWatch logs that any request has been triggered from discourse.

The summarizer is configured with the Summarize Persona, and the Perso is using the LLM endpoint which I meantioned earlier in this discussion …

Any hints ?

Many thanks and Greetings,

WS

Wurzelseppi · June 3, 2025, 1:59pm

New insight:

Some topics can be summarized, some not…

It seems that the new ones can be summarized.
And another question: When I have a topic with pictures in it, and don’t have any image-recognizing AI in place, is this a problem?

Falco · June 3, 2025, 2:55pm

That is a 502 from your own endpoint

Topic length maybe?

It will be for image related tasks, like image caption or post triage automation.

Wurzelseppi · June 4, 2025, 4:33am

Hey @Falco

But discourse even start the request, nor gives any reason (logs, …) why it doesn´t
Hard to narrow down my options

No, it wasn´t´. I found the problem (didn´t JSON.stringify the body from llm endpoint, but discourse could be a bit more talkative about the reason for this 502 )

Thanks and cheers,

WS

Topic		Replies	Views
Twitter, Github and normal username and password logins not working after upgrade Installation unsupported-install	35	7239	October 22, 2016
Installation fails on my Debian server Installation	24	2088	March 18, 2019
Error using discourse-oauth2-basic plugin with NeonCRM SSO oauth2	28	4566	February 12, 2020
SSL_connect returned=1 errno=0 peeraddr=162.243.189.2:443 state=error: certificate verify failed (Hostname mismatch) Installation s3	19	3899	January 9, 2024
CSRF problem in development with 'Discourse OpenID Connect' plug-in Dev	8	920	October 28, 2023

Topic Summarization doesn't work :-(

Related topics