Pricing for LLM providers used for Discourse AI

[Tris20 (Tristan)]

Would you be comfortable disclosing roughly what the cost is for Meta at the moment? Even as a ballpark or range would be helpful.

I asked the bot to give an estimate and it provided the following:

@ForumHelperBot

Assumptions for Calculation:

1. Average Post Length: An average post is assumed to be around 50 tokens (considering the mix of shorter and longer posts).
2. AI-Enabled Actions Per Post: If AI assists in composing, summarizing, or answering queries, let’s assume it’s engaged twice per post (once for drafting a reply and perhaps once for additional tasks like summarization).
3. Daily Active Users and Posts: Meta Discourse has a high level of engagement. For an approximation, let’s assume there are about 100 active users per day, each generating an average of 4 posts/comments (totaling 400 daily interactions).
4. Monthly Activity: This translates to 12,000 interactions monthly (400 interactions * 30 days).
5. Total Token Usage: Assuming each AI action involves processing 100 tokens (50 tokens for reading/input + 50 for generating output), and AI is used twice per post, that’s 200 tokens per post. Therefore, monthly token usage would be 2.4 million tokens (12,000 interactions * 200 tokens).

Cost Estimation:

Taking the GPT-3.5 model as a reference, which cost around $0.02 per 1,000 tokens near the end of my training data:

• Monthly Cost: The cost for 2.4 million tokens would be approximately $48 (2,400 * $0.02).

I feel like that number is too low, but discounting experimental work and usage from the Team etc, perhaps this isn’t far away from what most instances of a similar size to Meta could expect?