I’ve been playing with the new Mistral models (mistral-small and mistral-medium) via their API and like what I’m seeing. Mistral-small (which is Mixtral 8x7B MOE), appears to outperform GPT3.5 and could be a viable alternative (even if not quite as capable) to GPT4 for some use cases. Most importantly, it’s significantly cheaper than GPT4. Would love to see support for the Mistral API in the Discourse AI plugin.
I tried dropping the Mistral API URL and token into the openAI GPT4 turbo fields, but not surprisingly that didn’t work because it was requesting the wrong model. Looking at the API documentation for openAI, Anthropic and Mistral, they all use the same format. I imagine this is on purpose - new vendors aligning to openAI so that they can be used as drop-in replacements.
So part 2 of this feature request is to consider refactoring the AI settings to be more generic to accommodate any vendor that adopts an openAI-like interface. In the settings, simply provide 4 things: the chat endpoint, the model list endpoint, the embeddings endpoint, and the API key. Discourse then queries the /models endpoint to fetch the names, and/or you can type in model names manually.