k4bench.analysis.event_source

k4bench.analysis.event_source

What went into a run's events: event_source.json.gz.

One file per run, shared by all its configurations, since every configuration of a run processes the same events. It holds, entry by entry, the generator particles of each event and, for a job that reads a file (reconstruction on a frozen simulated input), what that file's collections hold. The per-event costs (:mod:k4bench.analysis.components, the timing plugins' files) can then be set against what each event is.

Where it comes from

:func:record_event_source reads a podio file with uproot: a ddsim benchmark's own output file after its first run, a k4run benchmark's input file before its first run. Its origin says how it was made:

• recorded: by the benchmark, from the run's own file;
• backfill-verified: later, from the very file the run read, matched by its checksum;
• backfill-inferred: later, from a file that should hold the same generator particles, e.g. another detector's simulation of the same sample.

Its source names the file, its checksum when one was taken, the configuration whose output it is (simulation only) and the entries read: events of them from first_entry.

Which event is which

The entries are those the job processed, in order: entry i is the i-th event the job's profiling counted (G4Event ID in simulation, EventContext::evt() in reconstruction), as a job reads its input from its first entry on. The EventHeader's event and run numbers of each entry are kept beside them (None for an entry without one) but are not the key.

The generator particles

The particles of the generator record (generatorStatus not 0) (:data:GENERATOR_COLUMNS), floats to six significant digits; edm4hep's units: GeV, mm, and the elementary charge. What transport made of them (simulatorStatus, end points) is not kept: it depends on the detector, so it would differ between the configurations sharing the file. Neither is the record wholly the generator's: simulation boosts it by the crossing angle its steering sets, and the products of a decay the generator assigned but Geant4 performs (a b hadron's, say) start where and as transport left the parent, so in Z→bb̄ about 1% of the particles, all born off the interaction point, differ slightly from detector to detector.

EventSource dataclass

EventSource(origin: str, file: str, sha256: str | None, label: str | None, first_entry: int, events: int, event_numbers: tuple[int | None, ...] | None, run_numbers: tuple[int | None, ...] | None, offsets: ndarray, columns: Mapping[str, ndarray], input_counts: DataFrame, input_energies: DataFrame)

What went into a run's events, read back from event_source.json.gz.

event_index property

event_index: ndarray

The event of each generator particle.

particles

particles() -> dict[str, ndarray]

Every field of :data:PARTICLE_FIELDS per generator particle: its columns and what follows from them. A quantity undefined for a particle (eta along the beam axis, abs_cos_theta at rest) is NaN.

Source code in k4bench/analysis/event_source.py

def particles(self) -> dict[str, np.ndarray]:
    """Every field of :data:`PARTICLE_FIELDS` per generator particle: its
    columns and what follows from them. A quantity undefined for a
    particle (``eta`` along the beam axis, ``abs_cos_theta`` at rest) is
    NaN."""
    c = {name: np.asarray(values, dtype=float) for name, values in self.columns.items()}
    with np.errstate(divide="ignore", invalid="ignore"):
        pt = np.hypot(c["px"], c["py"])
        p = np.sqrt(pt**2 + c["pz"] ** 2)
        eta = np.arcsinh(c["pz"] / pt)
        abs_cos_theta = np.abs(c["pz"]) / p
    return {
        **c,
        "p": p,
        "pt": pt,
        "E": np.sqrt(p**2 + c["mass"] ** 2),
        "theta": np.arctan2(pt, c["pz"]),
        "phi": np.arctan2(c["py"], c["px"]),
        "eta": np.where(np.isfinite(eta), eta, np.nan),
        "abs_cos_theta": abs_cos_theta,
        "vr": np.hypot(c["vx"], c["vy"]),
        "final": c["gen_status"] == 1,
        "charged": c["charge"] != 0,
    }

load_event_source

load_event_source(run_dir: str | Path) -> EventSource | None

The event_source.json.gz of run_dir, or None when it has none.

Raises ValueError for an unsupported or malformed file.

Source code in k4bench/analysis/event_source.py

def load_event_source(run_dir: str | Path) -> EventSource | None:
    """The ``event_source.json.gz`` of *run_dir*, or ``None`` when it has none.

    Raises ``ValueError`` for an unsupported or malformed file.
    """
    path = Path(run_dir) / EVENT_SOURCE
    if not path.is_file():
        return None
    return read_event_source(path)

read_event_source

read_event_source(path: Path) -> EventSource

The event source in path; ValueError for an unsupported schema_version or a malformed file.

Source code in k4bench/analysis/event_source.py

def read_event_source(path: Path) -> EventSource:
    """The event source in *path*; ``ValueError`` for an unsupported
    ``schema_version`` or a malformed file."""
    try:
        raw = json.loads(gzip.decompress(path.read_bytes()))
    except (OSError, EOFError, json.JSONDecodeError) as exc:
        raise ValueError(f"{path}: not a gzipped JSON event source ({exc})") from exc
    validate_event_source_schema(raw, source=path)
    return EventSource.from_json(raw, source=path)

record_event_source

record_event_source(out_path: Path, source_file: Path, *, n_events: int, origin: str = 'recorded', label: str | None = None, input_content: bool = False, checksum: bool = False) -> bool

Write what went into the first n_events events of the podio source_file to out_path: their generator particles and, with input_content, what the file's collections hold in each; with checksum, the file's sha256. label names the configuration whose output the file is. Whether it was written: a failure is warned about and costs only this file, never the run.

Source code in k4bench/analysis/event_source.py

def record_event_source(
    out_path: Path,
    source_file: Path,
    *,
    n_events: int,
    origin: str = "recorded",
    label: str | None = None,
    input_content: bool = False,
    checksum: bool = False,
) -> bool:
    """Write what went into the first *n_events* events of the podio
    *source_file* to *out_path*: their generator particles and, with
    *input_content*, what the file's collections hold in each; with
    *checksum*, the file's sha256. *label* names the configuration whose
    output the file is. Whether it was written: a failure is warned about and
    costs only this file, never the run."""
    try:
        record = describe_source(
            source_file, n_events=n_events, origin=origin, label=label,
            input_content=input_content, checksum=checksum,
        )
        out_path.write_bytes(gzip.compress(
            json.dumps(record, separators=(",", ":")).encode(), mtime=0,
        ))
    except Exception:  # noqa: BLE001 - the event source is an extra, never the run's failure
        out_path.unlink(missing_ok=True)
        print(f"  WARNING: no event source recorded:\n{traceback.format_exc()}")
        return False
    return True

describe_source

describe_source(source_file: Path, *, n_events: int, origin: str = 'recorded', label: str | None = None, input_content: bool = False, checksum: bool = False) -> dict

The event_source.json.gz record of source_file's first n_events events; see :func:record_event_source.

Source code in k4bench/analysis/event_source.py

def describe_source(
    source_file: Path,
    *,
    n_events: int,
    origin: str = "recorded",
    label: str | None = None,
    input_content: bool = False,
    checksum: bool = False,
) -> dict:
    """The ``event_source.json.gz`` record of *source_file*'s first
    *n_events* events; see :func:`record_event_source`."""
    if origin not in ORIGINS:
        raise ValueError(f"unknown origin {origin!r}, not one of {ORIGINS}")
    if n_events < 1:
        raise ValueError(f"n_events must be positive, got {n_events}")
    open_root = _event_size._root_opener()
    if open_root is None:
        raise RuntimeError("uproot is not available")
    with open_root(source_file) as root_file:
        collections = podio_collections(root_file)
        if collections is None:
            raise ValueError(f"{source_file.name} is not a podio file")
        tree = root_file["events"]
        available = int(tree.num_entries)
        if available < n_events:
            raise ValueError(
                f"{source_file.name} holds {available} events, the job processed {n_events}"
            )
        generator = _generator_particles(tree, _mc_particles(collections), n_events)
        headers = read_event_headers(tree, collections, n_events)
        content = (
            event_content(root_file, entry_stop=n_events)["collections"] if input_content else None
        )
    source = {"file": source_file.name, "first_entry": 0, "events": n_events}
    if checksum:
        source["sha256"] = _sha256(source_file)
    if label is not None:
        source["label"] = label
    record = {
        "schema_version": EVENT_SOURCE_SCHEMA_VERSION,
        "origin": origin,
        "source": source,
        "generator": generator,
    }
    if headers:
        record["event_headers"] = headers
    if content is not None:
        record["input_content"] = {
            "collections": [
                {"name": name, "type": collections[name][0], **measured}
                for name, measured in content.items()
            ],
        }
    return record