Probably this is a newbie question, but I wonder if the amount of information in the persona configuration could have an impact on the costs of LLM usage like Claude or OpenAI?
I mean, if I upload many text and PDF files, will that cause a significant increase in the request tokens so that each request using that persona is more expensive than a request using a persona which doesn’t have many files (or which searches fewer topics in the forum)?
The increase in cost is controlled via the following two settings in Persona:
When using a persona with files, it will use
Upload\ chunk\ tokens * Search\ conversation\ chunks
extra tokens. You can tweak those to control the cost / quality ratio.
Thanks a lot. Does anyone have a good experience on which values to use on those? I have no idea.
A PDF with 30 pages would have around 7500 words, maybe. That might be extrapolated to 20000 tokens.
Assuming that, would would it mean if I set Upload chunk tokens with value 2000, for example? Will it “read and take into account only 10% of the PDF” when building a response?
Leave the defaults unless you have a good reason to change. Discourse defaults work for most use cases.