Skip to main content

metrics.crps_empirical()

Compute the empirical Continuous Ranked Probability Score (CRPS).

Usage

Source

metrics.crps_empirical(
    pred,
    truth,
)

The CRPS generalises the mean absolute error to probabilistic forecasts and is computed elementwise as

\mathrm{CRPS}(F, y) = \mathbb{E}|X - y| - \tfrac{1}{2}\,\mathbb{E}|X - X'|,

where X, X' are independent draws from the forecast distribution F. The expectations are estimated from the forecast sample axis using the sorted-sample O(n \log n) identity.

Parameters

pred: Real[ArrayLike, " sample *batch"]

Forecast samples with the sample axis first, shape (sample, *batch). May be host-committed (e.g. draws sampled with device="host"), regardless of whether truth is: either operand is moved to device memory first (float_view()). Integer draws (a count model) are cast to the default float dtype there; a floating pred keeps its dtype.

truth: Real[ArrayLike, " *batch"]
Ground-truth values with shape (*batch) (matching pred without the sample axis). Integer values are cast like pred.

Returns

Float[Array, "*batch"]
Elementwise CRPS, one value per batch location.

Raises

ValueError
If truth.shape is not pred.shape[1:] (see Metric), or pred has fewer than 2 samples.

References

Tilmann Gneiting, Adrian E. Raftery (2007). “Strictly Proper Scoring Rules, Prediction, and Estimation”. Journal of the American Statistical Association.