← machine_learning package

TangMartinClassification

Binary sparse logistic regression using the model- space variational empirical-Bayes approximation of Tang and Martin (2025).

The variational distribution is an independent Bernoulli approximation directly on the model configuration space, not on the regression coefficients themselves.

The implementation follows the paper's coordinate-ascent variational inference algorithm based on the Jaakkola-Jordan lower bound and a fixed sparse pilot estimator beta_tilde. By default the pilot is a lasso model followed by an unpenalized refit on its selected support to reduce shrinkage bias. An SCAD pilot is also available as an expert option when skglm is installed, but it can be substantially slower and is not guaranteed to improve the resulting model. The pilot estimator's internal cross-validation can optionally be grouped or stratified to better match the structure of the training data. Posterior inclusion probabilities are approximated by the variational parameters phi, and the final predictive model is obtained by refitting logistic regression on the coordinates with phi >= 0.5.

More Info... Version 0.7.1

Ports/Properties

data

Data to process.

• verbose name: Data
• default value: None
• port type: DataPort
• value type: Packet (can be None)
• data direction: INOUT

cond_field

Instance data field that contains the class labels to discriminate. The field must be binary.

• verbose name: Cond Field
• default value: TargetValue
• port type: StringPort
• value type: str (can be None)

feature_scaling

Feature scaling applied before fitting. Standardization is recommended because the pilot sparse estimator and the variational updates are both scale-sensitive.

• verbose name: Feature Scaling
• default value: std
• port type: EnumPort
• value type: str (can be None)

complexity_penalty

Complexity penalty applied to each additional feature included in the model. Larger values impose a stronger cost for model complexity, preferring simpler models with fewer active features; smaller values are more permissive and allow more features to be retained. The penalty scales with the total number of features, so in very high-dimensional settings (many more features than observations) the penalty can become so dominant that no feature is selected. The implementation detects this situation and automatically falls back to evaluating only the features identified by the pilot estimator. The default of 0.01 is conservative and works well across a wide range of problem sizes.

• verbose name: Complexity Penalty
• default value: 0.01
• port type: FloatPort
• value type: float (can be None)

likelihood_power

Controls how strongly the model follows the training data versus relying on prior assumptions. A value of 1.0 gives the data full weight; values below 1.0 discount the data contribution, which acts as a form of regularization that can help prevent overfitting, especially when the number of features is much larger than the number of observations. The default of 0.99 applies a minimal discount that is sufficient for theoretical benefits while leaving the data signal nearly intact. Values around 0.9-0.95 can be tried if the model appears to overfit.

• verbose name: Likelihood Power
• default value: 0.99
• port type: FloatPort
• value type: float (can be None)

posterior_spread

Controls how tightly the final model coefficients are constrained around the initial pilot estimate. Smaller values force the model to stay close to the pilot, which is appropriate when the pilot is well calibrated. Larger values give the model more freedom to deviate from the pilot, which can help on datasets where the pilot estimate is unreliable or poorly calibrated (e.g., due to high noise or very few observations). The default of 0.1 is a moderately tight setting that works well in most cases.

• verbose name: Posterior Spread
• default value: 0.1
• port type: FloatPort
• value type: float (can be None)

include_bias

Include a bias (intercept) term. The intercept captures the overall baseline log-odds of the positive class and is not subject to feature selection. If disabled, the features should be centered or include a constant feature set to 1.

• verbose name: Include Bias Term
• default value: True
• port type: BoolPort
• value type: bool (can be None)

support_threshold

Threshold on the estimated probability that a feature is relevant. Features whose inclusion probability meets or exceeds this threshold are considered active and included in the final predictive model; all other features are discarded. A value of 0.5 (the default) corresponds to the median probability model, which selects features that are more likely relevant than not. Lower values (e.g., 0.3) will retain more features at the risk of including some noise; higher values (e.g., 0.7) are more conservative.

• verbose name: Support Threshold
• default value: 0.5
• port type: FloatPort
• value type: float (can be None)

sample_size_factor

Sample size correction factor or effective sample size. When observations are not independent (e.g., multiple trials per participant), the effective information content may be considerably lower than the nominal sample count, causing the variational inference to be overconfident in feature selection. This parameter tempers the likelihood by the factor n_eff/n, reducing the data terms in the CAVI updates relative to the complexity penalty. The correction is applied by internally multiplying the likelihood power parameter by n_eff/n. Values between 0 and 1 (inclusive) are interpreted as a multiplicative factor applied to the actual number of observations; for instance, if you have 2000 trials from 30 participants, set this to 30/2000 = 0.015. Values > 1 must be integers and are interpreted as an absolute effective sample size; for instance, 30 means that the effective sample size is 30 regardless of the actual number of observations.

• verbose name: Sample Size Correction
• default value: 1.0
• port type: FloatPort
• value type: float (can be None)

pilot_method

Method used to obtain an initial sparse estimate of the coefficients before the main variational inference step. The lasso (default) uses L1-regularized logistic regression with cross-validated regularization strength, and is fast and reliable for most problems. SCAD (smoothly clipped absolute deviation) is an alternative non-convex penalty that can reduce the shrinkage bias of the lasso on large coefficients, but requires the optional 'skglm' package and can be substantially slower. In practice, lasso is recommended unless there is a specific reason to prefer SCAD.

• verbose name: Pilot Method
• default value: lasso
• port type: EnumPort
• value type: str (can be None)

pilot_refit

Refit an unpenalized logistic regression on only the features selected by the pilot before running the main variational inference step. This is enabled by default because the pilot's regularization tends to shrink the selected coefficients toward zero, which can cause the subsequent variational step to underestimate how many features are truly relevant. The refit removes this shrinkage bias and generally improves results, especially on high-dimensional datasets. Disabling this is not recommended unless the pilot already produces well-calibrated coefficient magnitudes.

• verbose name: Refit Pilot Support
• default value: True
• port type: BoolPort
• value type: bool (can be None)

scad_gamma

Shape parameter for the SCAD penalty when the SCAD pilot method is selected. Larger values make the penalty behave more like the lasso; the default of 3.7 is the value recommended in the SCAD literature and works well in most settings. Only used when the pilot method is set to SCAD.

• verbose name: Scad Gamma
• default value: 3.7
• port type: FloatPort
• value type: float (can be None)

pilot_noise_scale

Scale of the small random noise injected into pilot coefficients that are exactly zero. This ensures that features not selected by the pilot still have a chance of being included by the variational inference step if the data supports it. The default of 1e-6 is negligibly small but sufficient to keep all features updatable. There is rarely a reason to change this value.

• verbose name: Pilot Noise Scale
• default value: 1e-06
• port type: FloatPort
• value type: float (can be None)

max_iterations

Maximum number of iterations for the variational inference optimization. Convergence typically occurs within 50-150 iterations for most problems. Higher-dimensional datasets may need up to 200 iterations. The default of 300 provides a generous safety margin. A warning is logged if the method does not converge within the given number of iterations.

• verbose name: Maximum Iterations
• default value: 300
• port type: IntPort
• value type: int (can be None)

convergence_tolerance

Convergence tolerance for the variational optimization. The method stops when the largest change in any feature's inclusion probability between consecutive iterations falls below this threshold. The default of 1e-4 is well calibrated for this method; tighter values rarely improve results but increase the number of iterations needed.

• verbose name: Convergence Tolerance
• default value: 0.0001
• port type: FloatPort
• value type: float (can be None)

num_folds

Number of cross-validation folds used by the pilot estimator to select its regularization strength. More folds give a more reliable selection at the cost of longer fitting time. The default of 5 is a good balance for most problems.

• verbose name: Number Of Pilot Cv Folds
• default value: 5
• port type: IntPort
• value type: int (can be None)

cv_group_field

Optionally a field indicating the group from which each trial is sourced. If given, then pilot-model cross-validation is split such that validation folds contain unseen groups. Examples groups are SubjectID, SessionID, etc.

• verbose name: Grouping Field (Pilot Cross-Validation)
• default value:
• port type: StringPort
• value type: str (can be None)

cv_stratified

Optionally perform stratified cross-validation for the pilot estimator. This means that all the folds have the same relative percentage of trials in each class.

• verbose name: Stratified Pilot Cross-Validation
• default value: True
• port type: BoolPort
• value type: bool (can be None)

pilot_max_iterations

Maximum number of iterations for the pilot estimator and the final logistic refit. The default of 3000 is generous and ensures convergence even on difficult datasets. Reducing this value can speed up fitting at the risk of the pilot not fully converging.

• verbose name: Pilot Max Iterations
• default value: 3000
• port type: IntPort
• value type: int (can be None)

random_seed

Random seed used for pilot-noise injection and deterministic pilot fitting.

• verbose name: Random Seed
• default value: 12345
• port type: IntPort
• value type: int (can be None)

initialize_once

Calibrate the model only once on the first labeled data that arrives.

• verbose name: Calibrate Only Once
• default value: True
• port type: BoolPort
• value type: bool (can be None)

dont_reset_model

Do not reset the model when the preceding graph changes.

• verbose name: Do Not Reset Model
• default value: False
• port type: BoolPort
• value type: bool (can be None)

probabilistic

Output class probabilities rather than hard class labels.

• verbose name: Output Probabilities
• default value: True
• port type: BoolPort
• value type: bool (can be None)

verbosity

Verbosity level. Higher values emit more diagnostics during fitting.

• verbose name: Verbosity Level
• default value: 0
• port type: IntPort
• value type: int (can be None)

set_breakpoint

Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

• verbose name: Set Breakpoint (Debug Only)
• default value: False
• port type: BoolPort
• value type: bool (can be None)

metadata

User-definable meta-data associated with the node. Usually reserved for technical purposes.

• verbose name: Metadata
• default value: {}
• port type: DictPort
• value type: dict (can be None)