Thank you for an explanation!
Are there git repos for the docker images?
1 Like
Not at the moment, as that would require me to keep two separate repos, one with the app code and another one with the internal tooling to build images and push to our internal repositories and I really couldn’t find time to properly set this up.
The API code is all visible inside the container image tho, even if that is not the best way to peruse it, at least it’s all there.
1 Like
Could anyone shares the exact minimum and recommended server requirements for a forum with standard visitors? Honestly, I want to give it a try, but I don’t know where to start since there is no clear server requirement.
1 Like
You just cree to try with whatever you think is reasonable. Something with 16gb of ram sounds like the minimum.
For your standard users, you’d use a standard server, I guess. 
3 Likes
In my forum, 200-250 online users and an average of 300 posts are created daily. So it can’t be called too much, so I said standard. I understand what you mean, but I plan to rent a new server because the Cloud server I am using now does not allow many upgrades. Thanks for your answer 
2 Likes
That’s incredibly hard to answer.
For example, if you just want to play with embeddings, a $6 droplet doing it on CPU will be enough and that will give you access to the Similar Topics feature.
Now if you want AIHelper and AIBot, you can:
-
pay per call on OpenAI, and the cost will depend on your usage.
-
run an open source LLM in a server you own for privacy. A model like Llama2-70B-Chat will need a server that costs 10k ~ 25k a month tho.
-
run an open source LLM in a pay per hour service, You can run a quantized version of Llama2 in HuggingFace endpoints for $6.50 an hour and it will automatically sleep after 15 minutes without requests.
The ML/Ops area is moving fast, GPUs are super scarce and new models launch every day. Hard to predict, we are all experimenting.
4 Likes
Thank you for your detailed explanation. Then I will try with a single plugin at each step. I think I will learn in detail according to the situation
2 Likes
Is there any particular reason these Docker commands aren’t in detached mode (missing -d)?
If you mean “Shouldn’t I launch these with -d?” The answer is probably yes.
If you really mean “Why didn’t the OP tell me to launch these commands with -d?”, I think they are intended to be just examples of how you might start them up and make them work at all. In practice, you’ll want to do some <other stuff> to launch them in a way that would make them useful in production.
1 Like
That’s exactly my question and you hit the nail right on the head. It’s been a while since I configured my Docker instances, but this is coming back now. When you say “other stuff” that I should do to make them useful for production, is there something else that should be screaming at me “DO THIS!” (besides the obvious of changing the port number from the same 6666 in each of the docker instances)?
OK. So for the pipe-separated API keys, are those completely arbitrary as the service host and do we just specify whatever alphanumeric key(s) we want to accept from the client?
How is changing the port any less “Obvious” than having it run in the background?
That’s the thing. Without having intimate knowledge of what you think is obvious, it’s impossible to answer the question. Mostly, if you are not pretty sure that you know how to make the stuff useful, then you probably need help that you can’t get here. 
Because I have run dozens of docker containers in the past. I just haven’t touched Docker in the last two years and dived in deep. It wasn’t obvious at first since I hadn’t touched it in a while but this basic fundamental knowledge for using Docker came back upon discussion.
That’s the thing. Without having intimate knowledge of what you think is obvious, it’s impossible to answer the question. Mostly, if you are not pretty sure that you know how to make the stuff useful, then you probably need help that you can’t get here.
That’s the rub, though. Sometimes, the obvious isn’t obvious even for those who have experience with different Docker systems like myself. One could interpret what you said as, you should know the answer to the question before you ask it. Understand that some of us run communities as a volunteer service and do not spend 24-7 learning the most intimate details of Discourse down to the Postgres data structures and such. I feel like you were shutting me down and that wasn’t appreciated in what should be a community forum where everyone should be freely and happily helping each other.
To the point here, I’ve done some Googling to try to ascertain how API_KEYS is supposed to be utilized and have come up short. I understand I may be missing the obvious and that might be downright frustrating for a Discourse professional like yourself with extensive knowledge of the platform down to the lowest level, but I am trying to have a community discussion here so others can benefit too who aren’t necessarily at your skill level yet. After all, the point is so people other than the developers of Discourse can use this software as well.
1 Like
I feel your pain. Coming across some service I have running and little idea how I started it happens more than I’d like.
Right. Even instructions that we write for ourselves don’t make sense when we need them.
Sorry. I didn’t mean to be rude or mean, and it looks like I was. My point was just that it’s hard enough supporting people running the Standard Install, so figuring out what your skills are and how you’re planning to launch it and whether it’ll be on the open internet and whether you know how to or want to have it protected with https (you probably do if you think you’re protecting it with API keys, is hard.
Yeah. If you’re putting this somewhere that someone else can contact it, I think you’ll want to define that API_KEYS variable and find some way to generate some random-ish thing to use as a key. And then you’d enter that same key in the settings of the plugin. That’s what I did. I didn’t check that using the wrong key would break it, which TBH, I think I should have. Maybe I’ll do that on the instance I’m about to add the plugin to.
But it might be nicer if the OP included the -d and set the API_KEYS env variable.
1 Like
the API_KEYS env knob is an optional one you can use if you, for any reason, want to restrict the service to only clients who supply one of the configured API_KEYS in their header.
Something you don’t really need if running it internally for a single instance, but that may be useful if running it across the internet or in shared environment.
2 Likes
Thanks @Falco and @pfaffman for your help and sorry if I derailed things here! Both of your help has been greatly appreciated! 
1 Like
Can all of these services be used by multiple Discourse installations, or should these be run on a per-site basis?
1 Like
They are all safe to share between instances.
2 Likes