Frustrations on AI spam detector

[lilydjwg]

Thanks, but I know what a tokenizer is. What I don’t know is what Discourse uses it for, and what would happen if I picked the wrong tokenizer? I don’t need a tokenizer to call an LLM anyway.