NUTSSampler¶
The No U-Turn Sampler (NUTS) for use with MCMC inference.
This is the main workhorse for MCMC inference in NeuroPype, especially for models that do not contain discrete variables, and is suitable for high-dimensional models. The main strength of the NUTS sampler compared to its predecessor HMC is that the latter requires manual tuning of the "trajectory length" parameter, which can require some experimentation. Currently, for models that involve discrete latent variables, the user may have to use the DiscreteHMC sampler, which is a hybrid sampler that includes NUTS as a building block, or alternatively the MixedHMC sampler (which does not enjoy the the relatively tuning-free nature of NUTS). Note that, if the model contains subsampled plates (via the With Stacked Variables node), this sampler automatically uses an energy-conserving subsampling scheme (HMCECS); see the subsampling parameter of WithStackedVariables for more details. Tip: in low-dimensional models, it can be useful to use a full-rank mass matrix, which can help with sampling efficiency. More Info... Version 1.0.0
Ports/Properties¶
mass_matrix_shape¶
The shape of the inverse mass matrix. This determines the efficiency with which the sampler can explore the posterior parameter space in case one or more variables may have highly correlated posterior distributions. The 'uncorrelated' form uses a simplifying diagonal approximation that is computationally cheap per update but will not handle strong correlations very well so may require more updates, while 'all-to-all' uses a full-rank ("dense") mass matrix. The latter can in principle handle correlated posterior variables more efficiently and require fewer updates, but at increased computational cost per update, especially if the parameter space is high dimensional.
An in-between is the blockwise form: this allows one to express that one or more individual variables in the model that are multivariate may each have a correlated posterior distribution, without also assuming that the variables are necessarily correlated with each other. This is written as in blockwise(myvar1,myvar2) where myvar1 and myvar2 are the names of the random variables in question; all other variables will have diagonal entries in the (overall block-diagonal) mass matrix. The final form is one in which one may denote sets of variables that could have strong mutual correlations (in addition to their own internal correlations), which is written as in blockwise(myvar1/myvar2,myvar3) where myvar1 and myvar2 are assumed to be correlated with each other but not with myvar3, but myvar3 may have its own correlation structure. Any unlisted variables again receive diagonal entries in the mass matrix.
- verbose name: Mass Matrix Shape
- default value: uncorrelated
- port type: ComboPort
- value type: str (can be None)
max_tree_depth¶
The maximum depth of the doubling scheme used in this sampler. Models with especially complex parameter spaces or posterior geometry can benefit from a larger value here.
- verbose name: Max Tree Depth
- default value: 10
- port type: IntPort
- value type: int (can be None)
max_tree_depth_postwarmup¶
The maximum depth during the post-warmup phase. If not provided, this is the same as max_tree_depth.
- verbose name: Max Tree Depth (Post-Warmup)
- default value: None
- port type: IntPort
- value type: int (can be None)
desired_accept_prob¶
The desired acceptance probability for step-size adaptation. This will adjust the step size such that on average a step is accepted (within the posterior distribution) with this probability. Increasing this value will lead to a smaller step size, thus the sampling will be slower but more robust.
- verbose name: Desired Accept Prob
- default value: 0.8
- port type: FloatPort
- value type: float (can be None)
step_size¶
The initial step size used by the underlying Verlet integrator. This normally does not need to be touched unless step size adaptation is disabled, or the adaptation diverges immediately even when using conservative choices for the init strategy in the inference node, and having tried the step size heuristic option in the sampler. The step size is often tuned based on the acceptance rate of the sampler, where 0.8 is usually a good target (if greater than 0.8, the step size could be decreased and if less than 0.8, the step size could be increased).
- verbose name: Step Size
- default value: 1.0
- port type: FloatPort
- value type: float (can be None)
adapt_step_size¶
Whether to adapt the step size during warmup. This uses the dual-averaging scheme.
- verbose name: Adapt Step Size
- default value: True
- port type: BoolPort
- value type: bool (can be None)
adapt_mass_matrix¶
Whether to adapt the mass matrix during warmup. This uses the Welford scheme.
- verbose name: Adapt Mass Matrix
- default value: True
- port type: BoolPort
- value type: bool (can be None)
step_size_heuristic¶
Whether to use a heuristic to adjust the step size at the beginning of each adaptation. This uses the doubling/halving strategy proposed in "The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo" (2014). This heuristic can speed up initial convergence but introduces its own computational cost. Another use case is that it can help with instant divergence due to a bad choice of initial step size, if the model has steep gradients or other pathological features.
- verbose name: Pretune Step Size
- default value: False
- port type: BoolPort
- value type: bool (can be None)
plate_subsampling¶
Use energy-conserving subsampling of plates (HMCECS). It is recommended to leave this set to auto, in which case it will automatically apply if the model contains subsampled plates, and otherwise not. One may set it to off to be sure that one uses a plain HMC sampler, where that is of interest. When used, this is according to "Hamiltonian Monte Carlo with energy conserving subsampling" (2019). (Algorithm 1).
- verbose name: Plate Subsampling
- default value: auto
- port type: EnumPort
- value type: str (can be None)
plate_subblocks¶
Optionally the number of sub-blocks into which to partition any subsampled plates. This is analogous to the Block Pseudo-Marginal Sampler. This can help with increasing the acceptance rate if some of the sub-blocks are more difficult to sample from than others. If enabled, this follows "The Block Pseudo-Margional Sampler" (2017).
- verbose name: Plate Subblocks
- default value: 1
- port type: IntPort
- value type: int (can be None)
subsample_control_variate¶
The control variate to use when performing plate subsampling. Bardanet is according to "Towards scaling up Markov chain Monte Carlo: an adaptive subsampling approach" (2014) and Betancourt is per "The Fundamental Incompatibility of Scalable Hamiltonian Monte Carlo and Naive Data Subsampling" (2015). The latter is the suggested default.
- verbose name: Subsample Control Variate
- default value: betancourt
- port type: EnumPort
- value type: str (can be None)
regularize_mass_matrix¶
Wether to apply regularization to the mass matrix. This is generally recommended, particularly for higher-dimensional posterior distribution, since otherwise sampling may become unstable.
- verbose name: Regularize Mass Matrix
- default value: True
- port type: BoolPort
- value type: bool (can be None)
set_breakpoint¶
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.
- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
metadata¶
User-definable meta-data associated with the node. Usually reserved for technical purposes.
- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)