This is a very important insight: you are never really done with these types of systems.
We end up repeating the process of
- User asks AI and gets a bad answer
- We review
- Fix documentation, accept a correct answer, and delete search landmines
- Ask the same question and get a correct answer
These are not the type of systems you can deploy and forget about; they need constant tuning.
Note that it really helps us a ton if you thumbs down any bad answers