Skip to main content

metrics.eval_interval_score()

Mean Winkler interval score for the central alpha prediction interval.

Usage

Source

metrics.eval_interval_score(
    pred,
    truth,
    *,
    alpha=0.9,
)

For the central alpha interval [l, u] (the (1-\alpha)/2 and 1-(1-\alpha)/2 quantiles), the interval score is (u - l) + \tfrac{2}{1-\alpha}\big[(l - y)\mathbf 1_{y<l} + (y - u)\mathbf 1_{y>u}\big], averaged over all data elements. It rewards narrow intervals and penalizes ground truth falling outside them; lower is better. A pure JAX scalar kernel (see Metric). pred and truth are moved to device memory first (float_view()), so either (or both) may be host-committed, e.g. draws sampled with device="host". An integer operand (the draws of a count model) is cast to the default float dtype in the same step.

Parameters

pred: Real[ArrayLike, " sample *batch"]

Forecast samples with the sample axis first.

truth: Real[ArrayLike, " *batch"]

Ground-truth values (matching pred without the sample axis).

alpha: float = 0.9
Nominal interval level in (0, 1).

Returns

Array
The mean interval score as a scalar array. NaN when a truth value or a forecast draw is NaN (see DEFAULT_METRICS).

Raises

ValueError
If alpha is not strictly inside (0, 1), or truth.shape is not pred.shape[1:] (see Metric).