Skip to content

Evals API

Generated from the typed public surface.

tesserix_adk.evals.Aggregate

Stability: alpha

One metric across a set of cases, with the sample it rests on shown.

class Aggregate(object) {line(self) -> 'str'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.Answer

Stability: alpha

What a solver produced, and what it had to fetch to produce it.

class Answer(AdkModel)

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.BASELINE_FORMAT

Stability: alpha

int([x]) -> integer int(x, base=10) -> integer

BASELINE_FORMAT: int

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.Baseline

Stability: alpha

A suite's numbers and where they came from, as the file CI reads.

class Baseline(AdkModel) {snapshot(self, metric: 'str') -> 'MetricSnapshot | None'; write(self, path: 'Path') -> 'None'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.BaselinePolicy

Stability: alpha

What this project blocks a merge on, declared once and read by CI.

class BaselinePolicy(AdkModel) {blocks(self, metric: 'str') -> 'bool'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.BaselineReport

Stability: alpha

The verdict, what moved, which cases moved it, and any override taken.

class BaselineReport(AdkModel) {as_dict(self) -> 'dict[str, object]'; comment(self, *, artefacts: 'str' = '') -> 'str'; failing(self) -> 'tuple[CaseRegression, ...]'; summary(self) -> 'str'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.Bypass

Stability: alpha

An exception taken deliberately, during an incident, with a name on it.

class Bypass(AdkModel)

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.CacheHitRate

Stability: alpha

The share of prompt tokens the provider served from its own cache.

class CacheHitRate(object) {compute(self, case: 'EvalCase', run: 'Run[Any]') -> 'MetricValue'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.Calibration

Stability: alpha

How well one judge agreed with people, and whether that is enough to gate on.

class Calibration(AdkModel) {require(self, *, stamp: 'str' = '') -> 'None'; summary(self) -> 'str'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.CaseExecutor

Stability: alpha · Kind: Protocol

How one case is turned into a run.

class CaseExecutor(Protocol)

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.CaseMeasurement

Stability: alpha

One fixture, measured on both axes at once.

class CaseMeasurement(AdkModel)

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.CaseRegression

Stability: alpha

One case that got worse, which is what a reviewer actually opens.

class CaseRegression(AdkModel) {artefact(self, base: 'str', suite: 'str') -> 'str'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.CaseResult

Stability: alpha

One case's outcome, and the run behind it where there is one.

class CaseResult(object) {answer(self) -> 'dict[str, Any]'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.CaseStatus

Stability: alpha

What happened to one case.

class CaseStatus(StrEnum)

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.Comparison

Stability: alpha

Which of two candidates a judge preferred, and which order it saw them in.

class Comparison(AdkModel)

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.CompressionCase

Stability: alpha

One fixture: content, a question about it, and what a right answer contains.

class CompressionCase(AdkModel)

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.CompressionFixtures

Stability: alpha

The fixture set, versioned so a result names the corpus it was measured on.

class CompressionFixtures(AdkModel)

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.CompressionReport

Stability: alpha

Savings and accuracy per content type, and the verdict that gates the merge.

class CompressionReport(AdkModel) {as_dict(self) -> 'dict[str, object]'; case(self, case_id: 'str') -> 'CaseMeasurement'; failing(self) -> 'tuple[KindReport, ...]'; kind(self, kind: 'ContentKind') -> 'KindReport'; summary(self) -> 'str'; table(self) -> 'str'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.CostPerCase

Stability: alpha

What one case cost, in the currency the provider priced it in.

class CostPerCase(object) {compute(self, case: 'EvalCase', run: 'Run[Any]') -> 'MetricValue'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.DATASET_FORMAT

Stability: alpha

int([x]) -> integer int(x, base=10) -> integer

DATASET_FORMAT: int

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.DEFAULT_FIXTURES

Stability: alpha

The fixture set, versioned so a result names the corpus it was measured on.

DEFAULT_FIXTURES: CompressionFixtures

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.DEFAULT_FLOOR

Stability: alpha

Convert a string or number to a floating-point number, if possible.

DEFAULT_FLOOR: float

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.DEFAULT_FLOORS

Stability: alpha

The floors this project holds itself to, versioned so lowering one is reviewable.

DEFAULT_FLOORS: FloorPolicy

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.DEFAULT_POLICY

Stability: alpha

What this project will accept, declared once and read by CI.

DEFAULT_POLICY: GatePolicy

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.Delta

Stability: alpha

What one metric did between the baseline and the candidate.

class Delta(AdkModel) {line(self) -> 'str'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.EvalCase

Stability: alpha

One example: what the agent is asked, as whom, and what a good answer looks like.

class EvalCase(AdkModel) {redacted(self) -> 'EvalCase'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.EvalSuite

Stability: alpha

A named, versioned set of cases, read and written as one file.

class EvalSuite(AdkModel) {tagged(self, tag: 'str') -> 'tuple[EvalCase, ...]'; to_jsonl(self, path: 'Path') -> 'None'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.ExactMatch

Stability: alpha

Whether the answer is the one the case declared, ignoring case and outer spacing.

class ExactMatch(object) {compute(self, case: 'EvalCase', run: 'Run[Any]') -> 'MetricValue'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.Floor

Stability: alpha

The least a content type may do before the build fails.

class Floor(AdkModel)

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.FloorPolicy

Stability: alpha

The floors this project holds itself to, versioned so lowering one is reviewable.

class FloorPolicy(AdkModel) {floor(self, kind: 'ContentKind') -> 'Floor | None'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.GatePolicy

Stability: alpha

What this project will accept, declared once and read by CI.

class GatePolicy(AdkModel)

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.GateReport

Stability: alpha

The verdict, what moved, and the record a rollback can read later.

class GateReport(AdkModel) {attributes(self) -> 'dict[str, str]'; permits(self, digest: 'str') -> 'bool'; summary(self) -> 'str'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.Groundedness

Stability: alpha

The share of the answer's citations that name a source the case declared.

class Groundedness(object) {compute(self, case: 'EvalCase', run: 'Run[Any]') -> 'MetricValue'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.HumanLabel

Stability: alpha

What a person scored one case at, which is the only ground truth here.

class HumanLabel(AdkModel)

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.JUDGE_PROMPT_VERSION

Stability: alpha

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

JUDGE_PROMPT_VERSION: str

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.Judge

Stability: alpha · Kind: Protocol

Something that scores one candidate answer against a rubric.

class Judge(Protocol) {score(self, case: 'EvalCase', candidate: 'str', *, candidate_model: 'str' = '') -> 'JudgeScore'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.JudgeMetric

Stability: alpha

Judged scores as a metric, refusing to exist unless the judge is calibrated.

class JudgeMetric(object) {compute(self, case: 'EvalCase', run: 'Run[Any]') -> 'MetricValue'}

Typed errors: JudgeNotCalibratedError

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.JudgeScore

Stability: alpha

One judged case, with everything needed to decide whether to believe it.

class JudgeScore(AdkModel)

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.KindReport

Stability: alpha

One content type: what it saved, what it cost, and whether that clears its floor.

class KindReport(AdkModel) {line(self) -> 'str'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.Labelled

Stability: alpha

A calibration example: a case, the answer under review, and the human score.

class Labelled(AdkModel)

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.LatencyMs

Stability: alpha

Wall-clock milliseconds from the run starting to the run ending.

class LatencyMs(object) {compute(self, case: 'EvalCase', run: 'Run[Any]') -> 'MetricValue'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.LlmJudge

Stability: alpha

A judge backed by a model, scoring against a rubric and quoting its evidence.

class LlmJudge(object) {calibrate(self, examples: 'Sequence[Labelled]') -> 'Calibration'; compare(self, case: 'EvalCase', a: 'str', b: 'str') -> 'Comparison'; score(self, case: 'EvalCase', candidate: 'str', *, candidate_model: 'str' = '') -> 'JudgeScore'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.Measured

Stability: alpha

One prompt version's numbers on one dataset.

class Measured(AdkModel)

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.Metric

Stability: alpha · Kind: Protocol

One thing a suite measures, built in or consumer-written.

class Metric(Protocol) {compute(self, case: 'EvalCase', run: 'Run[Any]') -> 'MetricValue'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.MetricFailure

Stability: alpha

A metric that raised on a case, kept whole rather than scored as zero.

class MetricFailure(object)

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.MetricMove

Stability: alpha

What one metric did between the baseline and the candidate.

class MetricMove(AdkModel)

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.MetricReport

Stability: alpha

Every metric over a suite: aggregates, per-tag breakdowns, verdicts and breakages.

class MetricReport(object) {aggregate(self, metric: 'str', *, tag: 'str | None' = None) -> 'Aggregate'; as_dict(self) -> 'dict[str, Any]'; summary(self) -> 'str'; table(self) -> 'str'; verdict(self, metric: 'str') -> 'ThresholdResult'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.MetricSnapshot

Stability: alpha

One metric's aggregate, frozen into the artefact.

class MetricSnapshot(AdkModel)

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.MetricValue

Stability: alpha

One metric's answer for one case, or its reason for not having one.

class MetricValue(object)

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.Provenance

Stability: alpha

What produced a set of numbers, which is what makes them comparable to another set.

class Provenance(AdkModel) {differences(self, other: 'Provenance') -> 'tuple[str, ...]'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.RefusalRate

Stability: alpha

Whether the agent declined. Lower is better, and a refused case is still measured.

class RefusalRate(object) {compute(self, case: 'EvalCase', run: 'Run[Any]') -> 'MetricValue'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.Rubric

Stability: alpha

A named, versioned scale, and the agreement it demands before it may gate.

class Rubric(AdkModel) {contains(self, score: 'int') -> 'bool'; instructions(self) -> 'str'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.RubricLevel

Stability: alpha

One point on a rubric's scale.

class RubricLevel(AdkModel)

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.SchemaValidity

Stability: alpha

Whether the run produced the structured output its agent declared.

class SchemaValidity(object) {compute(self, case: 'EvalCase', run: 'Run[Any]') -> 'MetricValue'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.Solver

Stability: alpha · Kind: Protocol

Whatever answers a fixture's question from the content in front of it.

class Solver(Protocol) {answer(self, case: 'CompressionCase', content: 'str', *, handle: 'str') -> 'Answer'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.SuiteResult

Stability: alpha

Every case's outcome, in dataset order.

class SuiteResult(object) {digest(self) -> 'str'; errored(self) -> 'tuple[CaseResult, ...]'; incomplete(self) -> 'tuple[CaseResult, ...]'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.SuiteRunner

Stability: alpha

Runs a dataset through an executor and reports what happened to each case.

class SuiteRunner(object) {run(self, suite: 'EvalSuite') -> 'SuiteResult'; run_id_for(self, suite: 'EvalSuite', case: 'EvalCase') -> 'str'}

Typed errors: ConfigurationError

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.TIE_CEILING

Stability: alpha

Convert a string or number to a floating-point number, if possible.

TIE_CEILING: float

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.Threshold

Stability: alpha

A limit one metric has to stay inside for the report to pass.

class Threshold(object) {judge(self, value: 'float | None') -> 'ThresholdResult'}

Typed errors: ValueError

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.ThresholdResult

Stability: alpha

What one threshold decided, and about which number.

class ThresholdResult(object)

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.TokensIn

Stability: alpha

Prompt tokens, cached ones included, as the provider counted them.

class TokensIn(object) {compute(self, case: 'EvalCase', run: 'Run[Any]') -> 'MetricValue'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.TokensOut

Stability: alpha

Generated tokens, as the provider counted them.

class TokensOut(object) {compute(self, case: 'EvalCase', run: 'Run[Any]') -> 'MetricValue'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.Tolerance

Stability: alpha

How far one metric may move before the change is a regression.

class Tolerance(AdkModel) {regression(self, before: 'float', after: 'float') -> 'float'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.ToolSequenceMatch

Stability: alpha

Whether the tools ran in the order the case declared in expected_tools.

class ToolSequenceMatch(object) {compute(self, case: 'EvalCase', run: 'Run[Any]') -> 'MetricValue'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.Verdict

Stability: alpha

Exported value.

Verdict = typing.Literal['pass', 'fail', 'repeat']

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.agreement

Stability: alpha

Measure a judge against human labels on the cases both scored.

def agreement(scores: 'Sequence[JudgeScore]', labels: 'Sequence[HumanLabel]', *, floor: 'float' = 0.6) -> 'Calibration'

Typed errors: IncomparableEvalError

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.baseline.BASELINE_FORMAT

Stability: alpha

int([x]) -> integer int(x, base=10) -> integer

BASELINE_FORMAT: int

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.baseline.Baseline

Stability: alpha

A suite's numbers and where they came from, as the file CI reads.

class Baseline(AdkModel) {snapshot(self, metric: 'str') -> 'MetricSnapshot | None'; write(self, path: 'Path') -> 'None'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.baseline.BaselinePolicy

Stability: alpha

What this project blocks a merge on, declared once and read by CI.

class BaselinePolicy(AdkModel) {blocks(self, metric: 'str') -> 'bool'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.baseline.BaselineReport

Stability: alpha

The verdict, what moved, which cases moved it, and any override taken.

class BaselineReport(AdkModel) {as_dict(self) -> 'dict[str, object]'; comment(self, *, artefacts: 'str' = '') -> 'str'; failing(self) -> 'tuple[CaseRegression, ...]'; summary(self) -> 'str'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.baseline.CaseRegression

Stability: alpha

One case that got worse, which is what a reviewer actually opens.

class CaseRegression(AdkModel) {artefact(self, base: 'str', suite: 'str') -> 'str'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.baseline.Delta

Stability: alpha

What one metric did between the baseline and the candidate.

class Delta(AdkModel) {line(self) -> 'str'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.baseline.MetricSnapshot

Stability: alpha

One metric's aggregate, frozen into the artefact.

class MetricSnapshot(AdkModel)

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.baseline.Provenance

Stability: alpha

What produced a set of numbers, which is what makes them comparable to another set.

class Provenance(AdkModel) {differences(self, other: 'Provenance') -> 'tuple[str, ...]'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.baseline.compare

Stability: alpha

Compare a candidate run against the stored baseline and say whether it may merge.

def compare(baseline: 'Baseline', candidate: 'Baseline', *, policy: 'BaselinePolicy', override: 'Bypass | None' = None) -> 'BaselineReport'

Typed errors: BaselineUnusableError

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.baseline.promote

Stability: alpha

Store candidate as the new baseline, keeping the one it replaces.

def promote(candidate: 'Baseline', path: 'Path') -> 'Path | None'

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.compare

Stability: alpha

Compare a candidate run against the stored baseline and say whether it may merge.

def compare(baseline: 'Baseline', candidate: 'Baseline', *, policy: 'BaselinePolicy', override: 'Bypass | None' = None) -> 'BaselineReport'

Typed errors: BaselineUnusableError

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.compression.Answer

Stability: alpha

What a solver produced, and what it had to fetch to produce it.

class Answer(AdkModel)

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.compression.CaseMeasurement

Stability: alpha

One fixture, measured on both axes at once.

class CaseMeasurement(AdkModel)

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.compression.CompressionCase

Stability: alpha

One fixture: content, a question about it, and what a right answer contains.

class CompressionCase(AdkModel)

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.compression.CompressionFixtures

Stability: alpha

The fixture set, versioned so a result names the corpus it was measured on.

class CompressionFixtures(AdkModel)

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.compression.CompressionReport

Stability: alpha

Savings and accuracy per content type, and the verdict that gates the merge.

class CompressionReport(AdkModel) {as_dict(self) -> 'dict[str, object]'; case(self, case_id: 'str') -> 'CaseMeasurement'; failing(self) -> 'tuple[KindReport, ...]'; kind(self, kind: 'ContentKind') -> 'KindReport'; summary(self) -> 'str'; table(self) -> 'str'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.compression.DEFAULT_FIXTURES

Stability: alpha

The fixture set, versioned so a result names the corpus it was measured on.

DEFAULT_FIXTURES: CompressionFixtures

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.compression.DEFAULT_FLOORS

Stability: alpha

The floors this project holds itself to, versioned so lowering one is reviewable.

DEFAULT_FLOORS: FloorPolicy

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.compression.Floor

Stability: alpha

The least a content type may do before the build fails.

class Floor(AdkModel)

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.compression.FloorPolicy

Stability: alpha

The floors this project holds itself to, versioned so lowering one is reviewable.

class FloorPolicy(AdkModel) {floor(self, kind: 'ContentKind') -> 'Floor | None'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.compression.KindReport

Stability: alpha

One content type: what it saved, what it cost, and whether that clears its floor.

class KindReport(AdkModel) {line(self) -> 'str'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.compression.Solver

Stability: alpha · Kind: Protocol

Whatever answers a fixture's question from the content in front of it.

class Solver(Protocol) {answer(self, case: 'CompressionCase', content: 'str', *, handle: 'str') -> 'Answer'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.compression.measure_compression

Stability: alpha

Measure savings and accuracy over the fixture set and judge every content type.

def measure_compression(fixtures: 'CompressionFixtures', router: 'ReversibleRouter', solver: 'Solver', *, floors: 'FloorPolicy', tenant: 'str' = 'evals', run_id: 'str' = 'compression-gate') -> 'CompressionReport'

Typed errors: EvalIncompleteError

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.dataset.DATASET_FORMAT

Stability: alpha

int([x]) -> integer int(x, base=10) -> integer

DATASET_FORMAT: int

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.dataset.EvalCase

Stability: alpha

One example: what the agent is asked, as whom, and what a good answer looks like.

class EvalCase(AdkModel) {redacted(self) -> 'EvalCase'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.dataset.EvalSuite

Stability: alpha

A named, versioned set of cases, read and written as one file.

class EvalSuite(AdkModel) {tagged(self, tag: 'str') -> 'tuple[EvalCase, ...]'; to_jsonl(self, path: 'Path') -> 'None'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.gate

Stability: alpha

Compare two measured versions and say whether the candidate may ship.

def gate(baseline: 'Measured', candidate: 'Measured', *, policy: 'GatePolicy' = GatePolicy(tolerances=(Tolerance(metric='task_success', direction='higher_is_better', tolerance=0.01, noise=0.0), Tolerance(metric='schema_validity', direction='higher_is_better', tolerance=0.0, noise=0.0), Tolerance(metric='judge_score', direction='higher_is_better', tolerance=0.05, noise=0.02), Tolerance(metric='p95_latency_ms', direction='lower_is_better', tolerance=250.0, noise=0.0), Tolerance(metric='cost_per_run', direction='lower_is_better', tolerance=0.0005, noise=0.0)), repeats=3), bypass: 'Bypass | None' = None) -> 'GateReport'

Typed errors: EvalIncompleteError, IncomparableEvalError

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.gate.Bypass

Stability: alpha

An exception taken deliberately, during an incident, with a name on it.

class Bypass(AdkModel)

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.gate.DEFAULT_POLICY

Stability: alpha

What this project will accept, declared once and read by CI.

DEFAULT_POLICY: GatePolicy

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.gate.GatePolicy

Stability: alpha

What this project will accept, declared once and read by CI.

class GatePolicy(AdkModel)

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.gate.GateReport

Stability: alpha

The verdict, what moved, and the record a rollback can read later.

class GateReport(AdkModel) {attributes(self) -> 'dict[str, str]'; permits(self, digest: 'str') -> 'bool'; summary(self) -> 'str'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.gate.Measured

Stability: alpha

One prompt version's numbers on one dataset.

class Measured(AdkModel)

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.gate.MetricMove

Stability: alpha

What one metric did between the baseline and the candidate.

class MetricMove(AdkModel)

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.gate.Tolerance

Stability: alpha

How far one metric may move before the change is a regression.

class Tolerance(AdkModel) {regression(self, before: 'float', after: 'float') -> 'float'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.gate.Verdict

Stability: alpha

Exported value.

Verdict = typing.Literal['pass', 'fail', 'repeat']

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.gate.gate

Stability: alpha

Compare two measured versions and say whether the candidate may ship.

def gate(baseline: 'Measured', candidate: 'Measured', *, policy: 'GatePolicy' = GatePolicy(tolerances=(Tolerance(metric='task_success', direction='higher_is_better', tolerance=0.01, noise=0.0), Tolerance(metric='schema_validity', direction='higher_is_better', tolerance=0.0, noise=0.0), Tolerance(metric='judge_score', direction='higher_is_better', tolerance=0.05, noise=0.02), Tolerance(metric='p95_latency_ms', direction='lower_is_better', tolerance=250.0, noise=0.0), Tolerance(metric='cost_per_run', direction='lower_is_better', tolerance=0.0005, noise=0.0)), repeats=3), bypass: 'Bypass | None' = None) -> 'GateReport'

Typed errors: EvalIncompleteError, IncomparableEvalError

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.judge.Calibration

Stability: alpha

How well one judge agreed with people, and whether that is enough to gate on.

class Calibration(AdkModel) {require(self, *, stamp: 'str' = '') -> 'None'; summary(self) -> 'str'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.judge.Comparison

Stability: alpha

Which of two candidates a judge preferred, and which order it saw them in.

class Comparison(AdkModel)

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.judge.DEFAULT_FLOOR

Stability: alpha

Convert a string or number to a floating-point number, if possible.

DEFAULT_FLOOR: float

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.judge.HumanLabel

Stability: alpha

What a person scored one case at, which is the only ground truth here.

class HumanLabel(AdkModel)

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.judge.JUDGE_PROMPT_VERSION

Stability: alpha

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

JUDGE_PROMPT_VERSION: str

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.judge.Judge

Stability: alpha · Kind: Protocol

Something that scores one candidate answer against a rubric.

class Judge(Protocol) {score(self, case: 'EvalCase', candidate: 'str', *, candidate_model: 'str' = '') -> 'JudgeScore'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.judge.JudgeMetric

Stability: alpha

Judged scores as a metric, refusing to exist unless the judge is calibrated.

class JudgeMetric(object) {compute(self, case: 'EvalCase', run: 'Run[Any]') -> 'MetricValue'}

Typed errors: JudgeNotCalibratedError

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.judge.JudgeScore

Stability: alpha

One judged case, with everything needed to decide whether to believe it.

class JudgeScore(AdkModel)

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.judge.Labelled

Stability: alpha

A calibration example: a case, the answer under review, and the human score.

class Labelled(AdkModel)

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.judge.LlmJudge

Stability: alpha

A judge backed by a model, scoring against a rubric and quoting its evidence.

class LlmJudge(object) {calibrate(self, examples: 'Sequence[Labelled]') -> 'Calibration'; compare(self, case: 'EvalCase', a: 'str', b: 'str') -> 'Comparison'; score(self, case: 'EvalCase', candidate: 'str', *, candidate_model: 'str' = '') -> 'JudgeScore'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.judge.Rubric

Stability: alpha

A named, versioned scale, and the agreement it demands before it may gate.

class Rubric(AdkModel) {contains(self, score: 'int') -> 'bool'; instructions(self) -> 'str'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.judge.RubricLevel

Stability: alpha

One point on a rubric's scale.

class RubricLevel(AdkModel)

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.judge.TIE_CEILING

Stability: alpha

Convert a string or number to a floating-point number, if possible.

TIE_CEILING: float

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.judge.agreement

Stability: alpha

Measure a judge against human labels on the cases both scored.

def agreement(scores: 'Sequence[JudgeScore]', labels: 'Sequence[HumanLabel]', *, floor: 'float' = 0.6) -> 'Calibration'

Typed errors: IncomparableEvalError

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.judge.shares_family

Stability: alpha

Whether two model names come from one family, so self-preference is in play.

def shares_family(judge_model: 'str', candidate_model: 'str') -> 'bool'

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.measure

Stability: alpha

Compute every metric over every case that ran, and judge the declared thresholds.

def measure(suite: 'EvalSuite', result: 'SuiteResult', metrics: 'Sequence[Metric]', *, thresholds: 'Sequence[Threshold]' = ()) -> 'MetricReport'

Typed errors: KeyError, ValueError

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.measure_compression

Stability: alpha

Measure savings and accuracy over the fixture set and judge every content type.

def measure_compression(fixtures: 'CompressionFixtures', router: 'ReversibleRouter', solver: 'Solver', *, floors: 'FloorPolicy', tenant: 'str' = 'evals', run_id: 'str' = 'compression-gate') -> 'CompressionReport'

Typed errors: EvalIncompleteError

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.metrics.Aggregate

Stability: alpha

One metric across a set of cases, with the sample it rests on shown.

class Aggregate(object) {line(self) -> 'str'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.metrics.CacheHitRate

Stability: alpha

The share of prompt tokens the provider served from its own cache.

class CacheHitRate(object) {compute(self, case: 'EvalCase', run: 'Run[Any]') -> 'MetricValue'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.metrics.CostPerCase

Stability: alpha

What one case cost, in the currency the provider priced it in.

class CostPerCase(object) {compute(self, case: 'EvalCase', run: 'Run[Any]') -> 'MetricValue'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.metrics.ExactMatch

Stability: alpha

Whether the answer is the one the case declared, ignoring case and outer spacing.

class ExactMatch(object) {compute(self, case: 'EvalCase', run: 'Run[Any]') -> 'MetricValue'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.metrics.Groundedness

Stability: alpha

The share of the answer's citations that name a source the case declared.

class Groundedness(object) {compute(self, case: 'EvalCase', run: 'Run[Any]') -> 'MetricValue'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.metrics.LatencyMs

Stability: alpha

Wall-clock milliseconds from the run starting to the run ending.

class LatencyMs(object) {compute(self, case: 'EvalCase', run: 'Run[Any]') -> 'MetricValue'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.metrics.Metric

Stability: alpha · Kind: Protocol

One thing a suite measures, built in or consumer-written.

class Metric(Protocol) {compute(self, case: 'EvalCase', run: 'Run[Any]') -> 'MetricValue'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.metrics.MetricFailure

Stability: alpha

A metric that raised on a case, kept whole rather than scored as zero.

class MetricFailure(object)

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.metrics.MetricReport

Stability: alpha

Every metric over a suite: aggregates, per-tag breakdowns, verdicts and breakages.

class MetricReport(object) {aggregate(self, metric: 'str', *, tag: 'str | None' = None) -> 'Aggregate'; as_dict(self) -> 'dict[str, Any]'; summary(self) -> 'str'; table(self) -> 'str'; verdict(self, metric: 'str') -> 'ThresholdResult'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.metrics.MetricValue

Stability: alpha

One metric's answer for one case, or its reason for not having one.

class MetricValue(object)

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.metrics.RefusalRate

Stability: alpha

Whether the agent declined. Lower is better, and a refused case is still measured.

class RefusalRate(object) {compute(self, case: 'EvalCase', run: 'Run[Any]') -> 'MetricValue'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.metrics.SchemaValidity

Stability: alpha

Whether the run produced the structured output its agent declared.

class SchemaValidity(object) {compute(self, case: 'EvalCase', run: 'Run[Any]') -> 'MetricValue'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.metrics.Threshold

Stability: alpha

A limit one metric has to stay inside for the report to pass.

class Threshold(object) {judge(self, value: 'float | None') -> 'ThresholdResult'}

Typed errors: ValueError

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.metrics.ThresholdResult

Stability: alpha

What one threshold decided, and about which number.

class ThresholdResult(object)

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.metrics.TokensIn

Stability: alpha

Prompt tokens, cached ones included, as the provider counted them.

class TokensIn(object) {compute(self, case: 'EvalCase', run: 'Run[Any]') -> 'MetricValue'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.metrics.TokensOut

Stability: alpha

Generated tokens, as the provider counted them.

class TokensOut(object) {compute(self, case: 'EvalCase', run: 'Run[Any]') -> 'MetricValue'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.metrics.ToolSequenceMatch

Stability: alpha

Whether the tools ran in the order the case declared in expected_tools.

class ToolSequenceMatch(object) {compute(self, case: 'EvalCase', run: 'Run[Any]') -> 'MetricValue'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.metrics.measure

Stability: alpha

Compute every metric over every case that ran, and judge the declared thresholds.

def measure(suite: 'EvalSuite', result: 'SuiteResult', metrics: 'Sequence[Metric]', *, thresholds: 'Sequence[Threshold]' = ()) -> 'MetricReport'

Typed errors: KeyError, ValueError

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.promote

Stability: alpha

Store candidate as the new baseline, keeping the one it replaces.

def promote(candidate: 'Baseline', path: 'Path') -> 'Path | None'

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.shares_family

Stability: alpha

Whether two model names come from one family, so self-preference is in play.

def shares_family(judge_model: 'str', candidate_model: 'str') -> 'bool'

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.suite.CaseExecutor

Stability: alpha · Kind: Protocol

How one case is turned into a run.

class CaseExecutor(Protocol)

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.suite.CaseResult

Stability: alpha

One case's outcome, and the run behind it where there is one.

class CaseResult(object) {answer(self) -> 'dict[str, Any]'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.suite.CaseStatus

Stability: alpha

What happened to one case.

class CaseStatus(StrEnum)

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.suite.SuiteResult

Stability: alpha

Every case's outcome, in dataset order.

class SuiteResult(object) {digest(self) -> 'str'; errored(self) -> 'tuple[CaseResult, ...]'; incomplete(self) -> 'tuple[CaseResult, ...]'}

Typed errors: None declared.

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.

tesserix_adk.evals.suite.SuiteRunner

Stability: alpha

Runs a dataset through an executor and reports what happened to each case.

class SuiteRunner(object) {run(self, suite: 'EvalSuite') -> 'SuiteResult'; run_id_for(self, suite: 'EvalSuite', case: 'EvalCase') -> 'str'}

Typed errors: ConfigurationError

Runnable recipe: examples/eval_suite.py — quality gates use typed outcomes rather than plausible prose.