arviz.loo#

arviz.loo(data, pointwise=None, var_name=None, reff=None, scale=None)[source]#

Compute Pareto-smoothed importance sampling leave-one-out cross-validation (PSIS-LOO-CV).

Estimates the expected log pointwise predictive density (elpd) using Pareto-smoothed importance sampling leave-one-out cross-validation (PSIS-LOO-CV). Also calculates LOO’s standard error and the effective number of parameters. Read more theory here https://arxiv.org/abs/1507.04544 and here https://arxiv.org/abs/1507.02646

Parameters
data: obj

Any object that can be converted to an arviz.InferenceData object. Refer to documentation of arviz.convert_to_dataset() for details.

pointwise: bool, optional

If True the pointwise predictive accuracy will be returned. Defaults to stats.ic_pointwise rcParam.

var_namestr, optional

The name of the variable in log_likelihood groups storing the pointwise log likelihood data to use for loo computation.

reff: float, optional

Relative MCMC efficiency, ess / n i.e. number of effective samples divided by the number of actual samples. Computed from trace by default.

scale: str

Output scale for loo. Available options are:

  • log : (default) log-score

  • negative_log : -1 * log-score

  • deviance : -2 * log-score

A higher log-score (or a lower deviance or negative log_score) indicates a model with better predictive accuracy.

Returns
ELPDData object (inherits fromclass:pandas.Series) with the following row/attributes:
elpd: approximated expected log pointwise predictive density (elpd)
se: standard error of the elpd
p_loo: effective number of parameters
shape_warn: bool

True if the estimated shape parameter of Pareto distribution is greater than 0.7 for one or more samples

loo_i: array of pointwise predictive accuracy, only if pointwise True
pareto_k: array of Pareto shape values, only if pointwise True
scale: scale of the elpd

The returned object has a custom print method that overrides pd.Series method.

See also

compare

Compare models based on PSIS-LOO loo or WAIC waic cross-validation.

waic

Compute the widely applicable information criterion.

plot_compare

Summary plot for model comparison.

plot_elpd

Plot pointwise elpd differences between two or more models.

plot_khat

Plot Pareto tail indices for diagnosing convergence.

Examples

Calculate LOO of a model:

In [1]: import arviz as az
   ...: data = az.load_arviz_data("centered_eight")
   ...: az.loo(data)
   ...: 
Out[1]: 
Computed from 2000 posterior samples and 8 observations log-likelihood matrix.

         Estimate       SE
elpd_loo   -30.81     1.43
p_loo        0.95        -
------

Pareto k diagnostic values:
                         Count   Pct.
(-Inf, 0.5]   (good)        6   75.0%
 (0.5, 0.7]   (ok)          2   25.0%
   (0.7, 1]   (bad)         0    0.0%
   (1, Inf)   (very bad)    0    0.0%

Calculate LOO of a model and return the pointwise values:

In [2]: data_loo = az.loo(data, pointwise=True)
   ...: data_loo.loo_i
   ...: 
Out[2]: 
<xarray.DataArray 'loo_i' (school: 8)>
array([-4.91696767, -3.41751961, -3.85571898, -3.46734224, -3.40488294,
       -3.47796516, -4.31633415, -3.95382766])
Coordinates:
  * school   (school) object 'Choate' 'Deerfield' ... "St. Paul's" 'Mt. Hermon'