How to use AI models with custom tokenizers

Ondersteuning
2

TL;DR pick the closest tokenizer and set the maximum context to some thousands less to allow for the difference to not affect you.