How can I restrict an AI Agent to answer only questions related to my forum?

Have you taken a look at Agents? Perhaps the Forum Researcher or Forum Helper Agents would be a better fit, as they are tuned to work with the forum specifically. Or is that what you are already using?