Debate and peer review

Both take LLMClient instances, not agents, and call chat(prompt) with fixed prompts. They are not related to the @Debate messaging protocol in Protocols or the DEBATE group strategy in Group topologies.

Debate

execute(topic) runs the rounds and then the judge:

1. In each round the proponent argues in favor and the opponent argues against. In the first round the proponent opens and the opponent answers it. In later rounds the proponent sees the opponent's last argument, and the opponent sees the proponent's newest argument and its own previous one.
2. If a side's call throws, its argument for that round is the text Error: Unable to generate argument. (or counter-argument. for the opponent), and the debate goes on.
3. After the last round the judge gets the whole transcript and is asked for five numbered sections: VERDICT, REASONING, PRO SUMMARY, CON SUMMARY and CONFIDENCE.

The timeout only stops new rounds from starting. A round that has started finishes, and the judge always runs.

package com.example.tnsai.docs;

import com.tnsai.coordination.debate.DebateExecutor;
import com.tnsai.coordination.debate.DebateResult;
import com.tnsai.coordination.debate.DebateRound;
import com.tnsai.coordination.debate.PeerReviewExecutor;
import com.tnsai.coordination.debate.PeerReviewExecutor.PeerReviewIteration;
import com.tnsai.coordination.debate.PeerReviewExecutor.PeerReviewResult;
import com.tnsai.llm.LLMClient;

import java.time.Duration;

/** Compilation contract for multi-agent/debate.md. */
public final class DebateExample {

    private DebateExample() {
    }

    static DebateResult debate(LLMClient pro, LLMClient con, LLMClient judge) {
        DebateExecutor debate = DebateExecutor.builder()
            .proponent(pro)
            .opponent(con)
            .judge(judge)
            .maxRounds(3)
            .timeout(Duration.ofMinutes(5))   // checked before each round
            .build();

        DebateResult result = debate.execute("Should this service move to microservices?");

        for (DebateRound round : result.rounds()) {
            System.out.println("Round " + round.roundNumber());
            System.out.println("  Pro: " + round.proponentArgument());
            System.out.println("  Con: " + round.opponentArgument());
        }
        System.out.println("Verdict: " + result.verdict());
        System.out.println("Confidence: " + result.confidence()
            + (result.isHighConfidence() ? " (high)" : ""));
        return result;
    }

    static String review(LLMClient writer, LLMClient editor) {
        PeerReviewResult result = PeerReviewExecutor.builder()
            .producer(writer)
            .reviewer(editor)
            .maxIterations(3)
            .build()
            .execute("Write release notes for version 2.0");

        for (PeerReviewIteration iteration : result.iterations()) {
            System.out.println(iteration.iterationNumber() + ": approved=" + iteration.approved());
        }
        // Without approval, finalOutput is the last draft, which was not approved.
        return result.approved() ? result.finalOutput() : null;
    }
}

The result

DebateResult is a record:

isHighConfidence() is true when confidence is at least 0.7. If the judge's call throws, the verdict starts with Unable to reach verdict due to error:, the confidence is 0 and the summaries are empty.

Peer review

execute(task) loops for up to maxIterations iterations:

1. The producer writes a draft. In the first iteration it gets the task. In later iterations it gets its previous draft and the reviewer's feedback, but not the task text.
2. The reviewer gets the task and the draft and is asked for APPROVED (yes or no), FEEDBACK and STRENGTHS.
3. The draft counts as approved when the reviewer's answer contains approved: yes or approved:yes, ignoring case. Any other wording, such as Approved: **yes**, counts as a rejection.
4. An approved draft ends the loop. Otherwise the feedback goes into the next iteration.

If the reviewer's call throws, the draft is not approved and the next revision gets Review unavailable: … as its feedback. If the producer's call throws, the loop stops.

PeerReviewResult has finalOutput, approved, iterations, iterationsUsed and duration. Each PeerReviewIteration has iterationNumber, output, feedback, approved and duration. When approved is false, finalOutput is the last draft, which was not approved, or an empty string if the producer never answered. Check approved before you use it.

Related

• Council and voting
• Judge
• Recursive task solver
• Protocols