That’s awesome! Nicely done!
In the process of diagnosing, I have found another bug - on the LLM config page (/admin/plugins/discourse-ai/ai-llms/%/edit), selecting either option to “Disable native tool support (use XML based tools) (optional)” or “Disable streaming completions (convert streaming to non streaming requests)” and clicking Save shows a “Success!” temporary toast, but then reloading the page has both or either option unchecked.
The connection reset issues still persist and I am still digging into it, however it looks like a combination of the Ruby code (FinalDestination / DNS resolution / Faraday) socket handling, combined with Debian 12 container on a Ubuntu 24.04 VM.
I have spun up a test Ubuntu 22.04 VM and there are no issues, all the embeddings and inference work perfectly. Have not seen a single reset.
I’ll keep chipping away at it, perhaps it’s related to a novel way that Ubuntu 24.04 manages the tcp stack w/ netplan.