
09: Evaluator-optimizer
The evaluator-optimizer workflow ensures task requirements are fully met through iterative refinement. An LLM performs a task, followed by a second LLM evaluating whether the result satisfies all specified criteria. If not, the process repeats with adjustments, continuing until the evaluator confirms all requirements are met.
Use Cases:
- Generating code that meets specific requirements, such as ensuring runtime complexity.
- Searching for information and using an evaluator to verify that the results include all the required details.
- Writing a story or article with specific tone or style requirements and using an evaluator to ensure the output matches the desired criteria, such as adhering to a particular voice or narrative structure.
- Generating structured data from unstructured input and using an evaluator to verify that the data is properly formatted, complete, and consistent.
- Creating user interface text, like tooltips or error messages, and using an evaluator to confirm the text is concise, clear, and contextually appropriate.