k4bench.regression.catalog¶
k4bench.regression.catalog ¶
What each suite's nightly regression report judges, and where it reads it.
A run group's history reaches the engine as trend frames, one per kind of
record its runs hold (:mod:k4bench.analysis.trend): their results, their
per-event records and, for a reco job, the costs of its components. A
:class:MetricSource names one such frame, how it is built from the run
directories, which of its columns are judged — and in which family, which sets
their floors (:data:~k4bench.regression.engine.EFFECT_FLOOR) — and which are
recorded only. A suite's report reads its sources (:func:sources) and nothing
else, so what a suite is judged on is said here once, for the report and for
every view that draws a verdict's history.
MetricSource
dataclass
¶
MetricSource(name: str, build: Callable[[tuple[str, ...]], DataFrame | None], suffix: str, judged: Mapping[str, str], reported_only: Mapping[str, str] = dict(), component_judged: Mapping[str, str] = dict(), configs: frozenset[str] | None = None)
sources ¶
sources(suite: str) -> tuple[MetricSource, ...]
source_of ¶
source_of(suite: str, metric: str, component: str | None) -> MetricSource | None
The source of suite metric is read from — of the whole job, or of
one component — or None for a metric no source records.