Just learned about these tools for use creating better prompts
Maybe someday in the future they could be created and incorporated for those creating prompts with Discourse.
Funnel: decomposes each eval from a binary outcome of pass/fail into a series of cascading steps, each with its own pass/fail criteria.
Flux: Flux is our quantitative measure of movement through the funnel. We look at flux both in aggregate, to quantify the net outcome of a treatment on our funnel, and broken out by stage, to see how evals are transitioning from stage to stage.
Orbit: The orbit chart visualizes individual evals as they move through “orbits” representing the funnel, with earlier stages closer to the center. It’s an extremely information-dense view of an experimental result.
Images
Funnel
Flux
Orbit