How to use internal AI endpoints?

Ondersteuning
1

I wonder what the right approach is for using internal AI API endpoints (like LiteLLM) on self-hosted Discourse instances.

I tried enabling docker bridge, container IPs, and proper SSL-terminated endpoints (to the same public IP where Discourse is hosted), but nothing works. I allowed internal hosts, but I get a 500 error when testing on the LLM-AI UI within Discourse.

It’s probably related to my own configuration, and I know I probably made some mistakes. So, I hope to solve this with the help of the community :slight_smile:

Our main use case is to try Vertex AI (OpenAI) endpoints that did not work out of the box on Discourse because they are not supported. But we also want the layer that LiteLLM adds to our workflow.

Logs:

FinalDestination::SSRFDetector::DisallowedIpError (FinalDestination: all resolved IPs were disallowed)
2

If you are running it on the same server, you will need to allow it via DISCOURSE_ALLOWED_INTERNAL_HOSTS env variable.

3

I did it on UI like I saw on a previous similar post. Tried also with the variable on my app.yml file but still getting the error.

What can be wrong if my credentials are OK and I can use the API endpoint on the host? Can I debug more?

4

How exactly did you add it to the app.yml ? It expects either an IP or a hostname.

5

Well, I tested the endpoint with SSL within Discourse (app container) and it works, but not from the Discourse AI UI test.

I’m using ‘OpenAI’ Provider with ‘https://llm.mydomain.tld/v1/chat/completions’ as URL. The credentials are OK:

/var/www/discourse/plugins/discourse-ai/lib/completions/endpoints/base.rb:182:in 'DiscourseAi::Completions::Endpoints::Base#perform_completion!'
/var/www/discourse/plugins/discourse-ai/lib/completions/endpoints/open_ai_shared.rb:28:in 'DiscourseAi::Completions::Endpoints::OpenAiShared#perform_completion!'
/var/www/discourse/plugins/discourse-ai/lib/completions/llm.rb:214:in 'DiscourseAi::Completions::Llm#generate'

pitchfork-0.18.2/lib/pitchfork/soft_timeout.rb:46:in 'Pitchfork::SoftTimeout::Request#interrupt'
pitchfork-0.18.2/lib/pitchfork/soft_timeout.rb:94:in 'block (2 levels) in Pitchfork::SoftTimeout#create_timeout_thread'
pitchfork-0.18.2/lib/pitchfork/soft_timeout.rb:93:in 'Array#each'
pitchfork-0.18.2/lib/pitchfork/soft_timeout.rb:93:in 'block in Pitchfork::SoftTimeout#create_timeout_thread'

It’s strange, and I’ve been trying to get it done for weeks.

Already tried this:

EDIT 1: DISCOURSE_ALLOWED_INTERNAL_HOSTS: 172.16.X.X (LiteLLM container IP, 127.0.0.1 and the SSL terminated LLM domain host I understand is not requested to be here)

EDIT 2: And I see the same 500 error when trying to enter to configure my MCP server (MCP discourse-ai tab configuration).