Skip to content

k4bench.regression.catalog

k4bench.regression.catalog

What each suite's nightly regression report judges, and where it reads it.

A run group's history reaches the engine as trend frames, one per kind of record its runs hold (:mod:k4bench.analysis.trend): their results, their per-event records and, for a reco job, the costs of its components. A :class:MetricSource names one such frame, how it is built from the run directories, which of its columns are judged — and in which family, which sets their floors (:data:~k4bench.regression.engine.EFFECT_FLOOR) — and which are recorded only. A suite's report reads its sources (:func:sources) and nothing else, so what a suite is judged on is said here once, for the report and for every view that draws a verdict's history.

MetricSource dataclass

MetricSource(name: str, build: Callable[[tuple[str, ...]], DataFrame | None], suffix: str, judged: Mapping[str, str], reported_only: Mapping[str, str] = dict(), component_judged: Mapping[str, str] = dict(), configs: frozenset[str] | None = None)

One trend frame of a run group's history, and what is judged off it.

job_rows

job_rows(frame: DataFrame) -> DataFrame

frame's rows for a whole job, of the configs read.

Source code in k4bench/regression/catalog.py
def job_rows(self, frame: pd.DataFrame) -> pd.DataFrame:
    """*frame*'s rows for a whole job, of the configs read."""
    frame = self._configs_rows(frame)
    return frame[frame[COMPONENT].isna()] if COMPONENT in frame.columns else frame

component_rows

component_rows(frame: DataFrame) -> DataFrame

frame's rows naming a component, of the configs read.

Source code in k4bench/regression/catalog.py
def component_rows(self, frame: pd.DataFrame) -> pd.DataFrame:
    """*frame*'s rows naming a component, of the configs read."""
    if COMPONENT not in frame.columns:
        return frame.iloc[0:0]
    frame = self._configs_rows(frame)
    return frame[frame[COMPONENT].notna()]

sources

sources(suite: str) -> tuple[MetricSource, ...]

The sources suite's report judges.

Source code in k4bench/regression/catalog.py
def sources(suite: str) -> tuple[MetricSource, ...]:
    """The sources *suite*'s report judges."""
    try:
        return CATALOGS[suite]
    except KeyError:
        raise ValueError(f"unknown suite {suite!r}") from None

source_of

source_of(suite: str, metric: str, component: str | None) -> MetricSource | None

The source of suite metric is read from — of the whole job, or of one component — or None for a metric no source records.

Source code in k4bench/regression/catalog.py
def source_of(suite: str, metric: str, component: str | None) -> MetricSource | None:
    """The source of *suite* *metric* is read from — of the whole job, or of
    one *component* — or ``None`` for a metric no source records."""
    for source in sources(suite):
        recorded = (
            source.component_judged if component is not None
            else {**source.judged, **source.reported_only}
        )
        if metric in recorded:
            return source
    return None