PosteriorModeApprox¶
An approximation of the posterior mode location (MAP), optionally along with a rough estimate of the variance (Laplace).
While these modes are not usually considered a variational approximation, both variants are very efficient and robust (more so when using regularization), and can be used during model bring-up. The maximum a posteriori (MAP) estimate only returns the location of the posterior maximum (mode) without any uncertainty estimates. This has some connections with classic regularized maximum likelihood estimators (when the regularizer can be interpreted as the negative log density of a specific prior distribution) and this can be used to check equivalence when porting models between Bayesian and non-Bayesian (e.g., convex or deep learning) forms. The Laplace approximation has a long history in Bayesian analysis and can be quite useful in practice, especially for nearly Gaussian posteriors, cases where the tail of the posterior distribution is less important, or cases where the posterior mode is of interest. This variant returns the same mode location as MAP, but also estimates the curvature of the (log-)posterior distribution at the mode (precision matrix), which approximates the variance of the posterior distribution if it is (near-)Gaussian. However, since this quantity is derived from only the mode, if the true distribution is non-Gaussian, then this method can easily under- or over-estimate the true variance of the posterior distribution. In such cases, a mean-field or multi-variate normal approximation will be more faithful. More Info... Version 1.0.0
Ports/Properties¶
variant¶
The variant to use. The MAP variant yields the basic maximum a posteriori estimate, i.e., only the location of the posterior mode without any uncertainty estimates ("error bars"). This type of approximation is in many cases equivalent to what is oftentimes obtained with regularized maximum-likelihood estimation (if the regularizer corresponds to a specific prior distribution). The Laplace variant also finds the same posterior mode, and in addition estimates the curvature of the posterior distribution at the mode (precision matrix), which quantifies to some extent the uncertainty in the estimate, although potentially less accurately than a mean-field or multivariate normal approximation.
- verbose name: Variant
- default value: Laplace
- port type: EnumPort
- value type: str (can be None)
hessian_approx¶
Type of Hessian matrix approximation to use, if computing the Laplace approximation. Both are mathematically equivalent, but the 'hessian' mode uses forward- and reverse-mode differentiation, while the 'reverse-mode' mode uses only reverse-mode differentiation. Depending on the ratio of data points to parameters, these two can have different computational costs; also forward-mode is not universally supported for all models (namely those that may contain control flow such as If/Else or loops).
- verbose name: Estimator (Laplace Only)
- default value: hessian
- port type: EnumPort
- value type: str (can be None)
regularization¶
Optional reularization parameter for the hessian matrix. Only used with the Laplace variant. This can be set to a small number such as 1e-3 to robustify the computation.
- verbose name: Regularization
- default value: None
- port type: FloatPort
- value type: float (can be None)
set_breakpoint¶
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.
- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
metadata¶
User-definable meta-data associated with the node. Usually reserved for technical purposes.
- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)