Package dev.agenor.tools.eval

Class ScenarioRunner

java.lang.Object

dev.agenor.tools.eval.ScenarioRunner

public class ScenarioRunner
extends Object

Executes evaluation scenarios against a Agenor runtime.

The runner orchestrates the complete evaluation lifecycle:

1. Setup - prepare the runtime environment
2. Execute - trigger agent behavior
3. Collect - gather metrics, health status, messages
4. Verify - run assertions against collected data
5. Teardown - cleanup resources

Reuses existing Agenor tools services:

• MetricsService - for metrics collection
• HealthCheckService - for health verification
• MessageHistoryService - for message tracking

Example usage:

 AgenorRuntime runtime = AgenorRuntime.create();
 ScenarioRunner runner = new ScenarioRunner(runtime);

 EvaluationResult result = runner.run(myScenario);

 if (result.passed()) {
     System.out.println("All assertions passed!");
 } else {
     result.failedAssertions().forEach(System.out::println);
 }
 

Since:

0.5.0

Constructor Summary

Constructors

Constructor	Description
ScenarioRunner(AgenorRuntime runtime)	Creates a runner with a new set of services.
ScenarioRunner(AgenorRuntime runtime, int historySize)	Creates a runner with custom message history size.
ScenarioRunner(AgenorRuntime runtime, MetricsService metrics, HealthCheckService health, MessageHistoryService history)	Creates a runner with externally provided services.

Method Summary

Modifier and Type	Method	Description
EvaluationReport	createReport(List<EvaluationResult> results)	Creates an evaluation report from results.
EvaluationResult	run(Scenario scenario)	Runs a single scenario.
List<EvaluationResult>	runAll(List<Scenario> scenarios)	Runs multiple scenarios and returns all results.
List<EvaluationResult>	runAllParallel(List<Scenario> scenarios)	Runs multiple scenarios in parallel.
void	shutdown()	Shuts down the runner and releases resources.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

ScenarioRunner

public ScenarioRunner(AgenorRuntime runtime)

Creates a runner with a new set of services.

Parameters:

runtime - the Agenor runtime to evaluate

ScenarioRunner

public ScenarioRunner(AgenorRuntime runtime,
 int historySize)

Creates a runner with custom message history size.

Parameters:

runtime - the Agenor runtime to evaluate

historySize - maximum messages to retain

ScenarioRunner

public ScenarioRunner(AgenorRuntime runtime,
 MetricsService metrics,
 HealthCheckService health,
 MessageHistoryService history)

Creates a runner with externally provided services.

Parameters:

runtime - the Agenor runtime

metrics - metrics service

health - health check service

history - message history service

Method Details

run

public EvaluationResult run(Scenario scenario)

Runs a single scenario.

Parameters:

scenario - the scenario to run

Returns:

evaluation result

runAll

public List<EvaluationResult> runAll(List<Scenario> scenarios)

Runs multiple scenarios and returns all results.

Parameters:

scenarios - scenarios to run

Returns:

list of results in order

runAllParallel

public List<EvaluationResult> runAllParallel(List<Scenario> scenarios)

Runs multiple scenarios in parallel.

Parameters:

scenarios - scenarios to run

Returns:

list of results (order may vary)

createReport

public EvaluationReport createReport(List<EvaluationResult> results)

Creates an evaluation report from results.

Parameters:

results - evaluation results

Returns:

formatted report

shutdown

public void shutdown()

Shuts down the runner and releases resources.