← machine_learning package

RegularizedHorseshoeClassification

Classify data instances using sparse Bayesian logistic regression with a regularized horseshoe prior (Piironen and Vehtari, 2017).

This method learns which features are relevant and which are irrelevant, automatically pruning irrelevant features by shrinking their coefficients toward zero. The horseshoe prior is a continuous shrinkage prior that has excellent theoretical properties for sparse signal recovery: it simultaneously applies strong shrinkage to irrelevant features and minimal shrinkage to truly relevant ones. Unlike L1-regularized logistic regression (LASSO), which requires cross-validation to tune a regularization parameter, this Bayesian method estimates the optimal degree of sparsity directly from the data. The regularized variant adds a slab component that prevents the non-zero coefficients from growing excessively large, improving stability. Two inference methods are available. SVI (stochastic variational inference) with a mean-field normal guide is fast and scales efficiently to thousands of features. NUTS (No-U-Turn Sampler) gives higher-quality posterior estimates and is recommended when the feature count is moderate and posterior accuracy matters (e.g., when effective sample size is very low relative to the number of features). In both cases, the posterior mean of the coefficients is used for predictions, providing the near-minimax shrinkage behavior that makes the horseshoe estimator attractive. When observations are not independent (e.g., multiple trials per participant), the sample size correction factor can be used to scale down the effective sample size, which adjusts the global shrinkage to account for the reduced information content. Like all machine learning methods, this method needs to be calibrated ("trained") before it can make any predictions on data. For this, the method requires training instances and associated training labels. The typical way to get such labels associated with time-series data is to make sure that a marker stream is included in the data, which is usually imported together with the data using one of the Import nodes, or received over the network alongside with the data, e.g., using the LSL Input node (with a non-empty marker query). These markers are then annotated with target labels using the Assign Targets node. To generate instances of training data for each of the training markers, one usually uses the Segmentation node to extract segments from the continuous time series around each marker. Since this machine learning method is not capable of being trained incrementally on streaming data, the method requires a data packet that contains the entire training data; this training data packet can either be accumulated online and then released in one shot using the Accumulate Calibration Data node, or it can be imported from a separate calibration recording and then spliced into the processing pipeline using the Inject Calibration Data, where it passes through the same nodes as the regular data until it reaches the machine learning node, where it is used for calibration. Once this node is calibrated, the trainable state of this node can be saved to a model file and later loaded for continued use. Version 0.7.0

Ports/Properties

data

Data to process.

• verbose name: Data
• default value: None
• port type: DataPort
• value type: Packet (can be None)
• data direction: INOUT

cond_field

The name of the instance data field that contains the conditions to be discriminated. This parameter will be ignored if the packet has previously been processed by a DescribeStatisticalDesign node.

• verbose name: Cond Field
• default value: TargetValue
• port type: StringPort
• value type: str (can be None)

feature_scaling

Feature scaling to use. The horseshoe prior is scale-sensitive, so scaling the features is recommended for good shrinkage behavior. The default of std (standardization to zero mean and unit variance) is a good choice for most data.

• verbose name: Feature Scaling
• default value: std
• port type: EnumPort
• value type: str (can be None)

expected_relevant_features

Expected number or fraction of relevant features. Controls the global shrinkage parameter tau0 of the regularized horseshoe prior, which determines how aggressively coefficients are shrunk toward zero. Smaller values imply stronger sparsity (more features pruned). Values between 0 and 1 (exclusive) are interpreted as a fraction of the total feature count; for instance, 0.1 means that roughly 10%% of features are expected to be relevant. Values >= 1 must be integers and are interpreted as an absolute count; for instance, 5 means that roughly 5 features are expected to be relevant. When set to 0, tau0 is set to 1/p (maximally sparse). This is the most impactful hyperparameter of the regularized horseshoe prior (Piironen and Vehtari, 2017). The default of 0.1 works well for moderate-dimensional problems (p up to ~2000). For very high-dimensional datasets (p > 5000), values of 0.02-0.05 are recommended. Values above 0.2 risk selecting a large number of noise features.

• verbose name: Expected Relevant Features
• default value: 0.1
• port type: FloatPort
• value type: float (can be None)

sample_size_factor

Sample size correction factor or effective sample size. Controls the effective sample size used in the tau0 formula. Values between 0 and 1 (inclusive) are interpreted as a multiplicative factor applied to the actual number of observations; for instance, if you have 2000 trials from 30 participants, the effective information content may be closer to 30 independent samples, so set this to 30/2000 = 0.015. Values > 1 must be integers and are interpreted as an absolute effective sample size; for instance, 30 means that the effective sample size is 30 regardless of the actual number of observations.

• verbose name: Sample Size Correction
• default value: 1.0
• port type: FloatPort
• value type: float (can be None)

slab_scale

Slab scale for the regularized horseshoe. Controls the prior scale of the non-zero (large) coefficients after standardization. Larger values allow bigger coefficients in the slab component. A value of 2.5 is the recommended default for standardized features (Piironen and Vehtari, 2017).

• verbose name: Slab Scale
• default value: 2.5
• port type: FloatPort
• value type: float (can be None)

slab_df

Slab degrees of freedom. Controls how heavy-tailed the slab component is. Lower values give a heavier tail. A value of 4 is a moderately informative choice, which is conservative enough to prevent extreme coefficient values while permitting a range of scales.

• verbose name: Slab Degrees Of Freedom
• default value: 4.0
• port type: FloatPort
• value type: float (can be None)

include_bias

Include a bias (intercept) term. If false, features need to be centered or include a constant feature set to 1.

• verbose name: Include Bias Term
• default value: True
• port type: BoolPort
• value type: bool (can be None)

inference_method

Inference method. SVI (stochastic variational inference) is fast and scales well to large feature counts, using a centered parameterization with fixed global shrinkage (tau) and a mean-field normal guide. NUTS (No-U-Turn Sampler) is a Hamiltonian Monte Carlo method that gives higher-quality posterior estimates and learns the global shrinkage from data, but is significantly slower, especially for many features. NUTS is recommended when the feature count is moderate (up to ~1000) and posterior quality matters.

• verbose name: Inference Method
• default value: svi
• port type: EnumPort
• value type: str (can be None)

num_svi_steps

Number of SVI optimization steps. More steps can yield a better posterior approximation at the cost of longer training time. For most problems 2000-5000 steps are sufficient. Only used when inference method is SVI.

• verbose name: Number Of Svi Steps
• default value: 3000
• port type: IntPort
• value type: int (can be None)

learning_rate

Initial learning rate for the Adam optimizer. The rate decays with a cosine schedule over the course of training. The default of 0.01 works well for most problems. For high-dimensional datasets (p >> n), reducing this to 0.005 can improve convergence stability, but the number of SVI steps should be increased proportionally (e.g., to 4000-5000). Values below 0.005 may require substantially more steps to converge. Only used when inference method is SVI.

• verbose name: Learning Rate
• default value: 0.01
• port type: FloatPort
• value type: float (can be None)

svi_warm_start

Enable warm-start initialization for SVI when the number of features exceeds the number of observations. When enabled, the method identifies candidate features by their correlation with the labels and gives them favorable starting conditions for the optimization, allowing the optimizer to discover relevant features efficiently. Without this warm start, the SVI optimizer can get stuck in a state where no features are selected, especially when the feature count greatly exceeds the observation count. This setting has no effect when the feature count is less than or equal to the observation count, or when the inference method is NUTS. Only used when inference method is SVI.

• verbose name: Svi Warm Start (P>n)
• default value: True
• port type: BoolPort
• value type: bool (can be None)

num_warmup

Number of NUTS warmup (burn-in) steps. During warmup the sampler adapts its step size and mass matrix. 500 is generally sufficient; for low-dimensional problems (p < 50) this can be reduced to 200-300 to save time. Only used when inference method is NUTS.

• verbose name: Number Of Warmup Steps
• default value: 500
• port type: IntPort
• value type: int (can be None)

num_mcmc_samples

Number of NUTS posterior samples to draw after warmup. More samples yield more precise posterior estimates but increase runtime. For low-dimensional problems (p < 50), 200-300 samples are often adequate. Only used when inference method is NUTS.

• verbose name: Number Of Mcmc Samples
• default value: 500
• port type: IntPort
• value type: int (can be None)

num_posterior_samples

Number of posterior samples used to compute the posterior mean of the coefficients. More samples give a more precise estimate of the mean but increase memory and compute at the end of fitting. Only used when inference method is SVI.

• verbose name: Number Of Posterior Samples
• default value: 1000
• port type: IntPort
• value type: int (can be None)

random_seed

Random seed. Different values may give slightly different outcomes.

• verbose name: Random Seed
• default value: 12345
• port type: IntPort
• value type: int (can be None)

initialize_once

Calibrate the model only once. If set to False, then this node will recalibrate itself whenever a non-streaming data chunk is received that has both training labels and associated training instances.

• verbose name: Calibrate Only Once
• default value: True
• port type: BoolPort
• value type: bool (can be None)

dont_reset_model

Do not reset the model when the preceding graph is changed. Normally, when certain parameters of preceding nodes are being changed, the model will be reset. If this is enabled, the model will persist, but there is a chance that the model is incompatible when input data format to this node has changed.

• verbose name: Do Not Reset Model
• default value: False
• port type: BoolPort
• value type: bool (can be None)

probabilistic

Use probabilistic outputs. If enabled, the node will output for each class the probability that a given trial is of that class; otherwise it will output the most likely class label.

• verbose name: Output Probabilities
• default value: True
• port type: BoolPort
• value type: bool (can be None)

verbosity

Verbosity level. Higher numbers will produce more extensive diagnostic output.

• verbose name: Verbosity Level
• default value: 0
• port type: IntPort
• value type: int (can be None)

set_breakpoint

Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

• verbose name: Set Breakpoint (Debug Only)
• default value: False
• port type: BoolPort
• value type: bool (can be None)

metadata

User-definable meta-data associated with the node. Usually reserved for technical purposes.

• verbose name: Metadata
• default value: {}
• port type: DictPort
• value type: dict (can be None)