← machine_learning package

SkinnyGibbsClassification

Binary sparse Bayesian logistic regression using the Skinny Gibbs Markov chain Monte Carlo sampler.

The sampler is based on Narisetty, Shen and He (2019) with Pólya-Gamma data augmentation from Polson, Scott and Windle (2013). Augmenting the logistic likelihood with Pólya-Gamma latent variables converts each MCMC sweep into a Gaussian linear model, enabling conjugate updates for the regression coefficients under a spike-and-slab prior. The "skinny" approximation draws inactive coefficients from the spike prior rather than integrating them out, giving an O(np) cost per sweep and allowing the sampler to scale to p >> n problems.

After the MCMC run, features with posterior inclusion probability (PIP) above ``support_threshold`` form the median probability model (MPM). An unpenalised logistic model is then refit on those features and used for prediction. When no features exceed the threshold, the method falls back to features above the prior inclusion probability q_n. Two Z update strategies are available: ``sequential`` (the paper's algorithm, one feature at a time with a running residual, matching the R reference) and ``block`` (all features updated simultaneously from a frozen residual, vectorised but tends to over-select). ``auto`` (the default) picks ``sequential`` when p ≤ 1000 and ``block`` for larger p. The sampler core can run either on the NumPy reference implementation or on a JAX implementation that supports JIT compilation and accelerator execution.

More Info... Version 0.7.1

Ports/Properties

data

Data to process.

• verbose name: Data
• default value: None
• port type: DataPort
• value type: Packet (can be None)
• data direction: INOUT

cond_field

Instance data field that contains the class labels to discriminate. The field must be binary.

• verbose name: Cond Field
• default value: TargetValue
• port type: StringPort
• value type: str (can be None)

feature_scaling

Feature scaling applied before fitting. Standardisation is strongly recommended because the spike-and-slab prior is scale-sensitive.

• verbose name: Feature Scaling
• default value: std
• port type: EnumPort
• value type: str (can be None)

sample_size_factor

Sample size correction factor or effective sample size. Controls the effective sample size used when computing the paper's data-adaptive hyperparameters (spike variance, slab variance, model-size cap, and prior inclusion probability). When observations are not independent (e.g., multiple trials per participant), the effective information content may be considerably lower than the nominal sample count. Values between 0 and 1 (inclusive) are interpreted as a multiplicative factor applied to the actual number of observations; for instance, if you have 2000 trials from 30 participants, the effective information content may be closer to 30 independent samples, so set this to 30/2000 = 0.015. Values > 1 must be integers and are interpreted as an absolute effective sample size; for instance, 30 means that the effective sample size is 30 regardless of the actual number of observations. This setting only affects the data-adaptive defaults; manually specified values for spike variance, slab variance, prior inclusion probability, and maximum active features are used as-is.

• verbose name: Sample Size Correction
• default value: 1.0
• port type: FloatPort
• value type: float (can be None)

tau0_sq

Variance of the spike (irrelevant-feature) component. Features assigned to the spike have their coefficients drawn from a narrow distribution centered at zero, effectively suppressing them. Smaller values enforce tighter suppression of inactive features. The default of 0 uses the paper's recommended data-adaptive value of 1/n_eff, where n_eff is the effective sample size (adjusted by the sample size correction factor). There is rarely a need to change this.

• verbose name: Spike Variance
• default value: 0.0
• port type: FloatPort
• value type: float (can be None)

tau1_sq

Variance of the slab (relevant-feature) component. Features assigned to the slab have their coefficients drawn from a broad distribution that allows large values, representing truly relevant features. Larger values give relevant features more room to take on large coefficients. The default of 0 uses the paper's recommended data-adaptive value, which scales with the number of features and the effective sample size (adjusted by the sample size correction factor). There is rarely a need to change this.

• verbose name: Slab Variance
• default value: 0.0
• port type: FloatPort
• value type: float (can be None)

q_n

Prior probability that any given feature is included in the model. This encodes the prior belief about sparsity: smaller values express a stronger belief that most features are irrelevant. The default of 0 uses the paper's recommended data-adaptive value, which is calibrated so that the prior probability of seeing more than a moderate number of active features is low (10%%). The effective sample size (adjusted by the sample size correction factor) is used in the calibration. There is rarely a need to override this unless you have strong prior knowledge about the expected number of relevant features.

• verbose name: Prior Inclusion Probability
• default value: 0.0
• port type: FloatPort
• value type: float (can be None)

max_active

Maximum number of features that may be simultaneously active in any single MCMC iteration. This cap prevents the sampler from exploring models that are too large, which improves both computational efficiency and statistical reliability. The default of 0 uses the paper's recommended data-adaptive value of max(30, sqrt(n_eff)), where n_eff is the effective sample size (adjusted by the sample size correction factor). Increase this if you expect the true model to contain more active features than the default allows.

• verbose name: Maximum Active Features
• default value: 0
• port type: IntPort
• value type: int (can be None)

support_threshold

Threshold on the posterior inclusion probability (PIP) for defining the final set of active features. After the MCMC run, each feature receives a PIP estimate based on how often it was included across the collected samples. Features with PIP at or above this threshold are considered relevant and included in the final predictive model. The default of 0.5 corresponds to the median probability model, which selects features that are more likely relevant than not.

• verbose name: Support Threshold
• default value: 0.5
• port type: FloatPort
• value type: float (can be None)

include_bias

Include an intercept term. When enabled, the intercept is always active (not subject to spike-and-slab selection) and uses the slab prior.

• verbose name: Include Bias Term
• default value: True
• port type: BoolPort
• value type: bool (can be None)

prediction_method

Prediction strategy used after the MCMC run. 'bma' (Bayesian Model Averaging) averages the predicted probabilities over all collected MCMC samples, naturally accounting for model uncertainty and generally giving the best predictions when the sampler explores models of moderate size. 'mpm' (median probability model) selects features whose posterior inclusion probability exceeds the support threshold, refits an unpenalized logistic regression on those features, and uses that single model for prediction -- this is preferred when the sampler tends to activate many correlated features simultaneously (as can happen with the block Z update mode). 'auto' (the default) selects 'bma' when the sequential Z update mode is in effect and 'mpm' otherwise.

• verbose name: Prediction Method
• default value: auto
• port type: EnumPort
• value type: str (can be None)

num_burnin

Number of burn-in MCMC iterations (discarded before collecting samples). 1000 is adequate for most problems. Can be reduced to 300-500 for large datasets (n > 1000) to save time, though this may increase the variability of PIP estimates.

• verbose name: Burn-In Iterations
• default value: 1000
• port type: IntPort
• value type: int (can be None)

num_samples

Number of post-burn-in MCMC samples used to estimate posterior inclusion probabilities. More samples yield more stable PIPs and BMA predictions. Can be reduced to 500-1000 for large datasets where per-iteration cost is high.

• verbose name: Posterior Samples
• default value: 2000
• port type: IntPort
• value type: int (can be None)

pg_n_terms

Number of terms in the series approximation used to sample the Polya-Gamma auxiliary variables that underlie the Gibbs sampler. More terms give more accurate samples at higher computational cost per MCMC iteration. The default of 200 gives accurate results across the full range of conditions encountered in practice. Reducing to 50 can speed up fitting, but may cause the sampler to over-select features on high-signal datasets due to approximation bias in the auxiliary variable distribution.

• verbose name: Pg Series Terms
• default value: 200
• port type: IntPort
• value type: int (can be None)

z_update_mode

Strategy for updating the feature inclusion indicators at each MCMC iteration. 'sequential' updates features one at a time, where each feature's update sees the effect of the preceding features' updates within the same sweep. This closely matches the paper's algorithm and generally produces better-calibrated results. 'block' updates all features simultaneously based on a snapshot of the current state, which is faster for very large feature counts but tends to over-select correlated features. 'auto' (the default) uses 'sequential' when the number of features is at most 1000, and 'block' for larger problems where the sequential Python loop would be too slow.

• verbose name: Z Update Mode
• default value: auto
• port type: EnumPort
• value type: str (can be None)

backend

Numeric backend used for the sampler core. 'jnp' uses the JAX implementation, enabling JIT compilation and accelerator execution when available. 'numpy' uses the reference NumPy implementation. The default is 'jnp'; however, the sequential Z-update mode still uses the NumPy runner because its JAX implementation is not yet selected by default here.

• verbose name: Sampler Backend
• default value: jnp
• port type: EnumPort
• value type: str (can be None)

random_seed

Random seed for the MCMC chain.

• verbose name: Random Seed
• default value: 12345
• port type: IntPort
• value type: int (can be None)

initialize_once

Calibrate the model only once on the first labelled data that arrives.

• verbose name: Calibrate Only Once
• default value: True
• port type: BoolPort
• value type: bool (can be None)

dont_reset_model

Do not reset the model when the preceding graph changes.

• verbose name: Do Not Reset Model
• default value: False
• port type: BoolPort
• value type: bool (can be None)

probabilistic

Output class probabilities rather than hard class labels.

• verbose name: Output Probabilities
• default value: True
• port type: BoolPort
• value type: bool (can be None)

verbosity

Verbosity level. Set to 1 to log sampler hyperparameters and final active-feature count; set to 2 for per-iteration diagnostics.

• verbose name: Verbosity Level
• default value: 0
• port type: IntPort
• value type: int (can be None)

set_breakpoint

Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

• verbose name: Set Breakpoint (Debug Only)
• default value: False
• port type: BoolPort
• value type: bool (can be None)

metadata

User-definable meta-data associated with the node. Usually reserved for technical purposes.

• verbose name: Metadata
• default value: {}
• port type: DictPort
• value type: dict (can be None)