What I do to ground 3.5 is adding a second, shorter system prompt lower in the final prompt to “remind” the model of some of the rules in the main system prompt.
So it would look something like (typing from phone, trying..)
system role
user
assistant
..
..
system role “reminder”
new user prompt
Just by repeating the most important system role contents, the model adds more weight to it. I’ve been using this workaround for a few months now without too much strange responses.
Especially if prompts are becoming longer, the model tends to “forget” things that are higher in the final prompt. Things in AI are very hacky, it’s something I experience this in GPT models and langchain as well. Just today I got such a strong personality in langchain that the actions when asking the time in a random city, were “checking my watch”, “changing the timezone of my watch” and “ask a stranger”