arviz.loo

arviz.loo(data, pointwise=None, var_name=None, reff=None, scale=None)[source]

Compute Pareto-smoothed importance sampling leave-one-out cross-validation (PSIS-LOO-CV).

Estimates the expected log pointwise predictive density (elpd) using Pareto-smoothed importance sampling leave-one-out cross-validation (PSIS-LOO-CV). Also calculates LOO’s standard error and the effective number of parameters. Read more theory here https://arxiv.org/abs/1507.04544 and here https://arxiv.org/abs/1507.02646

Parameters
data: obj

Any object that can be converted to an arviz.InferenceData object. Refer to documentation of arviz.convert_to_dataset() for details.

pointwise: bool, optional

If True the pointwise predictive accuracy will be returned. Defaults to stats.ic_pointwise rcParam.

var_namestr, optional

The name of the variable in log_likelihood groups storing the pointwise log likelihood data to use for loo computation.

reff: float, optional

Relative MCMC efficiency, ess / n i.e. number of effective samples divided by the number of actual samples. Computed from trace by default.

scale: str

Output scale for loo. Available options are:

  • log : (default) log-score

  • negative_log : -1 * log-score

  • deviance : -2 * log-score

A higher log-score (or a lower deviance or negative log_score) indicates a model with better predictive accuracy.

Returns
ELPDData object (inherits from pandas.Series) with the following row/attributes:
loo: approximated expected log pointwise predictive density (elpd)
loo_se: standard error of loo
p_loo: effective number of parameters
shape_warn: bool

True if the estimated shape parameter of Pareto distribution is greater than 0.7 for one or more samples

loo_i: array of pointwise predictive accuracy, only if pointwise True
pareto_k: array of Pareto shape values, only if pointwise True
loo_scale: scale of the loo results

The returned object has a custom print method that overrides pd.Series method.

See also

compare

Compare models based on PSIS-LOO loo or WAIC waic cross-validation.

waic

Compute the widely applicable information criterion.

plot_compare

Summary plot for model comparison.

plot_elpd

Plot pointwise elpd differences between two or more models.

plot_khat

Plot Pareto tail indices for diagnosing convergence.

Examples

Calculate LOO of a model:

In [1]: import arviz as az
   ...: data = az.load_arviz_data("centered_eight")
   ...: az.loo(data)
   ...: 
Out[1]: 
Computed from 2000 by 8 log-likelihood matrix

         Estimate       SE
elpd_loo   -30.81     1.43
p_loo        0.95        -

Calculate LOO of a model and return the pointwise values:

In [2]: data_loo = az.loo(data, pointwise=True)
   ...: data_loo.loo_i
   ...: 
Out[2]: 
<xarray.DataArray 'loo_i' (school: 8)>
array([-4.91734122, -3.41799371, -3.85570741, -3.46748115, -3.4048676 ,
       -3.47797698, -4.31512626, -3.95387985])
Coordinates:
  * school   (school) object 'Choate' 'Deerfield' ... "St. Paul's" 'Mt. Hermon'