Package: machine_learning

Machine learning (ML) algorithms and related nodes.

The majority of these nodes are one-stop shops that can be inserted after data has been appropriately preprocessed and labeled, and output predictions, but others fulfill other special management roles. The most important specialty nodes are Assign Targets, Accumulate Calibration Data, Measure Loss, and Crossvalidation. The most important classic ML techniques are Logistic Regression, Linear Discriminant Analysis and Convex Model (for advanced uses), and some of the Classification and Regression nodes for categorical or continuous-valued outputs, respectively. Most machine learning methods expect their incoming data packets to have an instance axis that indexes the observations and optionally one or more other axes (e.g,, feature, space, time) to make up the input data space. Most ML methods will output categorical or continuous-valued predictions that take on a specific format in NeuroPype. The most common case are classifier predictions that will have an instance axis and a feature axis with the number of features equal to the number of classes, where each feature has the probability of the corresponding class being the correct label for the instance. Importantly, nodes implementing learning algorithms are generally stateful, meaning that they can adapt themselves to the data that they receive and then use their internal state (model) to process subsequent data. Typically the adaptation happens the first time the node is invoked on a data packet, which is then expected to not only have an instance axis, but also that the instance axis payload (its .data array) has ground-truth training labels in it, which are usually expected in a field named 'TargetValue'. Then the node will train itself on that packet and remain fixed for the remainder of the node's life cycle, (although it is possible to reconfigure nodes to retrain themselves on each subsequent packet). Note that when ML nodes are used in conjunction with graph-accepting nodes such as in function definitions (FunctionDef/FunctionCall) or in loop bodies (e.g., with ForEach), some of the latter nodes will clear the state on each subsequent update unless configured to behave in a stateful or "materialized" way). A special case is the Crossvalidation node, which will, for each CV fold, reset the state of the ML node, then pass in the training data (and get training predictions out), and then pass in the test data (and get test predictions out for the current fold), then reset the state again for the next fold. The category also includes several key ML workflow ndoes such as for assigning target labels from event markers or other per-instance conditions (AssignTargets), quantifying a loss (or negative score) via MeasureLoss.

Nodes in this package:

• Accumulate Calibration Data
  Accumulate calibration data and emit it in one large chunk.

• Assign Target Values
  Select which markers contain event-related signal activity, and optionally assign numeric target values to these markers for use in machine learning.

• Balance Classes
  Balance the per-class trial counts in the data.

• Bayesian Ridge Regression
  Estimate a continuous output value from features using Bayesian Ridge Regression.

• Classifier Threshold Tuning
  Tune the classification threshold of a predictive model so that the resulting class labels maximise a user-defined performance metric.

• Clear Target Information
  Clear the target information from the markers in the given data.

• Convex Model
  A flexible convex optimization based machine learning model.

• Covariance Minimum Distance to Mean
  Classify covariance matrices based on a Riemannian minimum-distance-to-mean criterion.

• Cross-validation
  Perform a cross-validation and return a loss measure that quantifies how well the pipeline performed when generalizing to previously unseen data.

• Elastic Net Regression
  Estimate a continuous output value from features using linear regression with Elastic Net regularization.

• Empirical Bayes Classification
  Sparse Bayesian logistic regression using the empirical Bayes (type-II maximum likelihood) framework.

• Ensemble Predictor
  A model that combines outputs from multiple models trained on subsets of the data.

• Hierarchical Discriminant Component Analysis
  Use Hierarchical Discriminant Component Analysis (HDCA) to classify data instances.

• Inject Calibration Data
  Insert calibration data into the stream.

• k-Nearest Neighbor Imputation
  Impute missing data with an (optionally weighted) average of the k nearest neighbors (KNN).

• LASSO Regression
  Estimate a continuous output value from features using LASSO Regression.

• LARS Regression
  Estimate a continuous output value from features using Least-Angle (LARS) Regression.

• Linear Discriminant Analysis
  Classify data instances using Linear Discriminant Analysis (LDA).

• Linear Support Vector Classification
  Use linear support vector machines to classify data instances.

• Linear To (Pseudo-)Probabilties
  Convert linear predictions to (two-class) pseudo-probabilities.

• Logistic Regression
  Classify data instances using regularized Logistic Regression.

• Measure Loss
  Measure discrepancy between some predicted labels and true labels.

• Method Ensemble Predictor
  Combine predictions from multiple distinct predictive pipelines.

• Gaussian Naive Bayes
  Use naive Bayes to classify data instances.

• Probability Calibration
  Calibrate continuous outputs of a method to be valid probabilities.

• Quadratic Discriminant Analysis
  Classify data instances using Quadratic Discriminant Analysis (QDA).

• R2-D2 Bayesian Classification
  Classify data instances using sparse Bayesian logistic regression with the R2-D2 prior.

• Regularized Horseshoe Classification
  Classify data instances using sparse Bayesian logistic regression with a regularized horseshoe prior (Piironen and Vehtari, 2017).

• Regularized Logistic Regression
  Classify data instances using regularized Logistic Regression with complex regularization terms.

• Spike-and-Slab Bayesian Classification
  Classify data instances using binary sparse Bayesian logistic regression with a relaxed (Gaussian) spike-and-slab prior.

• Ridge Regression
  Estimate a continuous output value from features using Ridge Regression.

• Skinny Gibbs Classification
  Binary sparse Bayesian logistic regression using the Skinny Gibbs Markov chain Monte Carlo sampler.

• Sparse Bayesian Regression
  Estimate a continuous output value from features using Sparse Bayesian Regression.

• Spike-Slab Heredity Classification
  Classify data instances using sparse Bayesian logistic regression with explicit main effects and interaction terms under a heredity-structured spike-and-slab prior.

• Spike-Slab VB Classification
  Classify data instances using binary sparse Bayesian logistic regression with a point-mass spike-and-slab variational posterior.

• Stacked Predictor
  Learn a binary meta-predictor from out-of-fold predictions of several distinct base methods.

• Stochastic Gradient Descent Classification
  Classify data instances using models trained via stochastic gradient descent.

• Stochastic Gradient Descent Regression
  Estimate a continuous output value from features using linear regression with using stochastic gradient descent.

• Support Vector Classification
  Classification using support vector machines.

• Support Vector Regression
  Regression using support vector machines.

• Tang-Martin Classification
  Binary sparse logistic regression using the model- space variational empirical-Bayes approximation of Tang and Martin (2025).

• Trial Aggregate Predictor
  A model that makes aggregate predictions based on multiple trials.