# למה בחרו ב-AI חיצוני במקום מערכת פנימית? **URL:** https://meta.discourse.org/t/why-was-external-ai-chosen-over-an-internal-system/292904 **Category:** Support **Tags:** ai, related-topics **Created:** [25 בינואר,‏ 2024,‏ 12:34pm UTC](https://meta.discourse.org/t/why-was-external-ai-chosen-over-an-internal-system/292904 "2024-01-25T12:34:47Z") **Posts on this page:** 1 **Showing post:** 3
### Author: ![merefield](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/merefield/32/176214_2.png) [@merefield](https://meta.discourse.org/u/merefield) #### Post date: [25 בינואר,‏ 2024,‏ 1:25pm UTC](https://meta.discourse.org/t/why-was-external-ai-chosen-over-an-internal-system/292904/3 "2024-01-25T13:25:51Z")
> [@Kevin7](#): > > I’ve noticed it relies on external AI. Why was this approach chosen over developing an internal system On use of external resources, you can run your LLM locally if you like?: > [@Self-Hosting an OpenSource LLM for DiscourseAI](https://meta.discourse.org/t/self-hosting-an-opensource-llm-for-discourseai/290923): > > The [Discourse AI](https://meta.discourse.org/t/discourse-ai/259214) plugin has many features that require an LLM to be enabled, like, for example, Summarization, AI Helper, AI Search, AI Bot. While you can use a third party API, like [Configure API Keys for OpenAI](https://meta.discourse.org/t/configure-api-keys-for-openai/280783) or [Configure API Keys for Anthropic](https://meta.discourse.org/t/configure-api-keys-for-anthropic/280936) we built Discourse AI since first day to not be locked into those. Running with HuggingFace TGI HuggingFace provides an awesome container image that can get you running quickly. For example: mkdir -p /opt/tgi-cache docker run --rm --gpus all --shm… But have you done this for a project? It requires you to own or lease particularly impressive hardware! Try the smaller language models (that you might consider hosting) yourself and see how impressed you are: [https://github.com/ollama/ollama](https://github.com/ollama/ollama) Your mileage may vary but imho you’d need to be looking at hosting an at least 70Bn parameter model which is going to be quite costly to self-host. For reference, GPT 3.5 is supposed to be a 175Bn parameter model and GPT 4 has nearly 2 Trillion (they say) 😅 I wrote this plugin: > [@Discourse AI Topic Summary robot](https://meta.discourse.org/t/discourse-ai-topic-summary-automated-summaries-and-smart-tagging/256711): > > information_sourceSummary Uses a remote AI language model to prepare and post a summary of a topic.hammer_and_wrenchRepository Link [GitHub - merefield/discourse-ai-topic-summary: Uses a remote AI language model to prepare and post a summary of a Topic · GitHub](https://github.com/merefield/discourse-ai-topic-summary)open_bookInstall Guide [How to install plugins in Discourse](https://meta.discourse.org/t/install-plugins-in-discourse/19157)heartSponsorship Please consider becoming an ongoing [sponsor of my open source work](https://github.com/sponsors/merefield) at a level that suits your or your organisation’s resources and ne… And it has an AI tagging feature. In my experience you need GPT 4 Turbo to make it work well (and it _really_ works well then!) If you intended to self host something as powerful as those you’d need very deep pockets. This is why the use of an external LLM API is still an attractive, pay-as-you-go, option, especially because you are only paying for calls you make, not an expensive infrastructure that spends any time spinning its wheels unused. Of course if privacy is of major and sufficient concern, this might change the maths. --- _[View the full topic](https://meta.discourse.org/t/why-was-external-ai-chosen-over-an-internal-system/292904)._