You can't govern what you can't verify
I set out to answer a simple question: does wrapping a model in an orchestration loop improve its answers? The idea was to put the scientific method around the model: generate an answer, test it, inspect the failure, and revise. I never got a clean answer. I learned something more useful instead: I couldn’t govern even this small system until I could independently verify its behavior. Several apparent model failures were something else: a token ceiling, a serving default, an uneven comparison, or a bug in my control flow. The score told me that a model had failed. It rarely told me which part of the system had failed or why. ...