About a minimum of $5 in additional electricity costs per month for the GPU at idle - although in reality, the incremental cost for discourse is zero since I already run the LLM for other purposes.
But for sure, it would be more economical for small forums and low usage to use an LLM as a service. Though for the scale of Discourse’s hosted offering, I suspect it might make sense to host internally (and also develop knowledge of this area that is likely to be important).