Skip to main content

Credits of the image to Anthropic

Overview

In the evaluator-optimizer workflow, one LLM call generates a response while another provides evaluation and feedback in a loop. This workflow is particularly effective when we have clear evaluation criteria, and when iterative refinement provides measurable value. The two signs of good fit are, first, that LLM responses can be demonstrably improved when a human articulates their feedback; and second, that the LLM can provide such feedback. This is analogous to the iterative writing process a human writer might go through when producing a polished document.

When to use

This workflow is particularly effective when we have clear evaluation criteria, and when iterative refinement provides measurable value. It works best when LLM responses can be demonstrably improved through feedback, and when an LLM can provide meaningful evaluation of that feedback.

Using evaluator-optimizer in Latitude

Literary Translation with Iterative Refinement
This pattern demonstrates the power of iterative refinement through structured evaluation and optimization cycles, particularly valuable for complex tasks requiring high-quality outputs where initial attempts can be systematically improved through expert feedback.