Skip to content

k4bench.regression.components

k4bench.regression.components

Which components of a reco job a confirmed step moved.

The reco counterpart of :mod:k4bench.regression.regions. A confirmed step is one number — the job's mean event time, its initialize, the anonymous memory it held — and one number names no mechanism. The k4Bench auditor records what each algorithm and service spent of its own (_components.json), and the memory pass the heap each one holds (_heap.json), so the step can be taken apart into the components whose own cost moved: "the IDEA job got 15% faster" becomes "SimulateSiPMwithEdep went from 3.9 to 1.5 s per event and retired 66% fewer instructions", which can be matched against a diff.

This module judges nothing (the engine has decided that the metric stepped) and adds no thresholds; it reports the decomposition, largest movement first. It reads the files of one configuration at the two ends of one window, and only for a verdict that confirmed. A component one end did not record is absent, never zero: an algorithm added to or removed from the job is a real change and must stay distinguishable from one that stood still.

decomposed_as

decomposed_as(verdict: MetricVerdict) -> str | None

The component cost verdict's step is taken apart into, or None for a metric that has none.

Source code in k4bench/regression/components.py
def decomposed_as(verdict: MetricVerdict) -> str | None:
    """The component cost *verdict*'s step is taken apart into, or ``None``
    for a metric that has none."""
    return DECOMPOSED_AS.get(verdict.metric)

component_deltas

component_deltas(run_dirs: Sequence[str], *, label: str, quantity: str, base_release: str, onset_release: str, base_run_id: str | None = None, onset_run_id: str | None = None, judgeable_configs: set[tuple[str, str]] | None = None) -> tuple[ComponentDelta, ...]

How each component's quantity (:data:DECOMPOSED_AS) moved across (base, onset] for configuration label, largest movement first.

Empty when either end recorded no component costs at all: with one side measured there is no comparison, and treating the missing side as zero would report every component as newly added. The ends are those of :func:~k4bench.regression.regions.window_end_dirs, and when judgeable_configs is supplied a failed or orphaned config-night absent from it enters neither.

Source code in k4bench/regression/components.py
def component_deltas(
    run_dirs: Sequence[str],
    *,
    label: str,
    quantity: str,
    base_release: str,
    onset_release: str,
    base_run_id: str | None = None,
    onset_run_id: str | None = None,
    judgeable_configs: set[tuple[str, str]] | None = None,
) -> tuple[ComponentDelta, ...]:
    """How each component's *quantity* (:data:`DECOMPOSED_AS`) moved across
    ``(base, onset]`` for configuration *label*, largest movement first.

    Empty when either end recorded no component costs at all: with one side
    measured there is no comparison, and treating the missing side as zero
    would report every component as newly added. The ends are those of
    :func:`~k4bench.regression.regions.window_end_dirs`, and when
    *judgeable_configs* is supplied a failed or orphaned config-night absent
    from it enters neither.
    """
    ends = window_end_dirs(
        run_dirs, base_release=base_release, onset_release=onset_release,
        base_run_id=base_run_id, onset_run_id=onset_run_id,
    )
    if ends is None:
        return ()
    base_nights, onset_nights = (
        _read_nights(dirs, label, quantity, judgeable_configs) for dirs in ends
    )
    base = _median_of(base_nights, lambda night: night.cost)
    onset = _median_of(onset_nights, lambda night: night.cost)
    if not base or not onset:
        return ()
    base_work = _median_of(base_nights, lambda night: night.instructions)
    onset_work = _median_of(onset_nights, lambda night: night.instructions)
    packages = {
        name: package
        for night in (*base_nights, *onset_nights)
        for name, package in night.packages.items()
    }

    deltas = []
    for component in sorted(set(base) | set(onset)):
        before, after = base.get(component), onset.get(component)
        # A component one end did not run was added or removed; its whole cost
        # is the movement, and saying so is the point.
        delta = (after or 0.0) - (before or 0.0)
        if not math.isfinite(delta):
            continue
        deltas.append(ComponentDelta(
            component=component, base=before, onset=after, delta=delta,
            package=packages.get(component, ""),
            instructions_change=_change(base_work.get(component), onset_work.get(component)),
        ))
    deltas.sort(key=lambda d: (-abs(d.delta), d.component))
    return tuple(deltas)