The discussion revolves around whether having more persona files increases LLM request expenses. wlandgraf initiates the conversation, wondering if uploading many text and PDF files to a persona configuration could significantly increase the cost of LLM requests. Falco explains that the increase in cost is controlled by two settings in Persona: Upload chunk tokens and Search conversation chunks, which can be tweaked to control the cost/quality ratio. wlandgraf seeks guidance on optimal values for these settings, while Falco advises leaving the defaults unless there’s a good reason to change, as Discourse defaults work for most use cases.
Probably this is a newbie question, but I wonder if the amount of information in the persona configuration could have an impact on the costs of LLM usage like Claude or OpenAI?
I mean, if I upload many text and PDF files, will that cause a significant increase in the request tokens so that each request using that persona is more expensive than a request using a persona which doesn’t have many files (or which searches fewer topics in the forum)?
The increase in cost is controlled via the following two settings in Persona:
When using a persona with files, it will use
$
Upload\ chunk\ tokens * Search\ conversation\ chunks
$
extra tokens. You can tweak those to control the cost / quality ratio.
Thanks a lot. Does anyone have a good experience on which values to use on those? I have no idea.
A PDF with 30 pages would have around 7500 words, maybe. That might be extrapolated to 20000 tokens.
Assuming that, would would it mean if I set Upload chunk tokens with value 2000, for example? Will it “read and take into account only 10% of the PDF” when building a response?
Leave the defaults unless you have a good reason to change. Discourse defaults work for most use cases.