Thank you for the ideas! I think we’re in a much better spot now. I tweaked some settings and ran the same set of questions. Here are some results of my testing today:
A little additional info
| Change | Avg time | Verdict |
|---|---|---|
| Original: Sol model, high thinking | 56.3 s | Good answers, too slow |
| Model reasoning set to low | 47.6 s | Small gain |
| Agent thinking effort low | 18.9 s | Fast, but it stopped answering collection-frequency questions |
| Agent thinking effort medium | 29.8 s | Same quality problem, no good middle |
| Fewer RAG chunks (7 to 3) | 54.8 s | No gain, and one hard_overflow error |
| Smaller model (Luna) at high effort | 35.2 s | Faster, but weaker use of the support knowledge |
| Luna + Quick Facts block in the prompt | 29.6 s and 24.8 s | Collection and root-DC answers correct again |
| + report-impact line | 25.6 s | Report impact now correct |
| + troubleshooting checklist | 24.0 s | Group-change turn 49.7 s to 27.2 s |
Quick Facts was offloading some answers to the prompt out of the RAG.
