← machine_learning package

SparseBayesianRegression

Estimate a continuous output value from features using Sparse Bayesian Regression.

Sparse Bayesian regression is a an elegant Bayesian method to learn a linear mapping between input data and desired output values from training data, which is closely related to LASSO regression. The main difference to LASSO regression is that in LASSO regression, there is a tunable parameter that controls how strongly the method should be regularized. This parameter controls how "sparse" the solution is, that is, how many features shall be relied on by the model versus being pruned as irrelevant. This controls the complexity of the solution to prevent over-fitting to random details of the data and therefore to improve the generalization to new data. In LASSO, this parameter is tuned using cross-validation, that is, by empirical testing on held-out data. In the Bayesian variant, there is such a parameter as well, however, the optimal degree of regularization is estimated from the data, as well, in a theoretically clean and principled fashion. This method assumes that both inputs and outputs are Gaussian distributed, that is, have no or very few major statistical outliers. If the output follows a radically different distribution, for instance between 0 and 1, or nonnegative, or discrete values, then different methods may be more appropriate (for instance, classification methods for disrete values). To ameliorate the issue of outliers in the data, the raw data can be cleaned of artifacts with various artifact removal methods. To the extent that the assumptions hold true, this method is highly competitive with other (sparse) linear methods. Like all machine learning methods, this method needs to be calibrated ("trained") before it can make any predictions on data. For this, the method requires training instances and associated training labels. The typical way to get such labels associated with time-series data is to make sure that a marker stream is included in the data, which is usually imported together with the data using one of the Import nodes, or received over the network alongside with the data, e.g., using the LSL Input node (with a non-empty marker query). These markers are then annotated with target labels using the Assign Targets node. To generate instances of training data for each of the training markers, one usually uses the Segmentation node to extract segments from the continuous time series around each marker. Since this machine learning method is not capable of being trained incrementally on streaming data, the method requires a data packet that contains the entire training data; this training data packet can either be accumulated online and then released in one shot using the Accumulate Calibration Data node, or it can be imported from a separate calibration recording and then spliced into the processing pipeline using the Inject Calibration Data, where it passes through the same nodes as the regular data until it reaches the machine learning node, where it is used for calibration. Once this node is calibrated, the trainable state of this node can be saved to a model file and later loaded for continued use. Version 1.1.1

Ports/Properties

data

Data to process.

• verbose name: Data
• default value: None
• port type: DataPort
• value type: Packet (can be None)
• data direction: INOUT

method

Method to use. ARD is Automatic Relevance Determination (aka sparse Bayesian learning), which is also analogous to the relevance vector machine (RVM) for a linear kernel. Note the legacy RVM option, while Bayesian, does not actually realize an RVM-type sparse model but rather a Bayesian ridge regression.

• verbose name: Method
• default value: ARD
• port type: EnumPort
• value type: str (can be None)

max_iter

Maximum number of iterations. This is one of the stopping criteria to limit the compute time. The default is usually fine, and gains from increasing the number of iterations will be minimal (it can be worth experimenting with lower iteration numbers if the algorithm must finish in a fixed time budget, at a cost of potentially less accurate solutions).

• verbose name: Maximum Number Of Iterations
• default value: 300
• port type: IntPort
• value type: int (can be None)

tolerance

Convergence tolerance. This is the desired errors tolerance or acceptable inaccuracy in the solution. Using larger values gives less accurate results, but will lead to faster compute times. Note that, for biosignal-driven machine learning systems, one often does not need very small tolerances.

• verbose name: Tolerance
• default value: 0.001
• port type: FloatPort
• value type: float (can be None)

verbosity

Verbosity level. Higher numbers will produce more extensive diagnostic output.

• verbose name: Verbosity Level
• default value: 0
• port type: IntPort
• value type: int (can be None)

include_bias

Include a bias term. If false, your features need to be centered, or include a dummy feature set to 1.

• verbose name: Include Bias Term
• default value: True
• port type: BoolPort
• value type: bool (can be None)

normalize_features

Normalize features. Should only be disabled if the data comes with a predictable scale (e.g., normalized in some other way).

• verbose name: Normalize Features
• default value: True
• port type: BoolPort
• value type: bool (can be None)

alpha_shape

Alpha shape parameter. This is only included for completeness and usually does not have to be adjusted. This is the shape parameter for the Gamma distribution prior over the alpha parameter (inverse noise variance). By default, this is an uninformative prior.

• verbose name: Alpha Shape Parameter
• default value: 1e-06
• port type: FloatPort
• value type: float (can be None)

alpha_rate

Alpha rate parameter. This is only included for completeness and usually does not have to be adjusted. This is the rate parameter for the Gamma distribution prior over the alpha parameter (inverse noise variance). By default, this is an uninformative prior.

• verbose name: Alpha Rate Parameter
• default value: 1e-06
• port type: FloatPort
• value type: float (can be None)

lambda_shape

Lambda shape parameter. This is only included for completeness and usually does not have to be adjusted. This is the shape parameter for the Gamma distribution prior over the lambda parameter (inverse variance of the weights). By default, this is an uninformative prior.

• verbose name: Lambda Shape Parameter
• default value: 1e-06
• port type: FloatPort
• value type: float (can be None)

lambda_rate

Lambda rate parameter. This is only included for completeness and usually does not have to be adjusted. This is the rate parameter for the Gamma distribution prior over the lambda parameter (inverse variance of the weights). By default, this is an uninformative prior.

• verbose name: Lambda Rate Parameter
• default value: 1e-06
• port type: FloatPort
• value type: float (can be None)

pruning_threshold

Pruning threshold for small weights. Only used by the ARD flavor (note this is the inverse of threshold_lambda in the implementation).

• verbose name: Pruning Threshold
• default value: 0.0001
• port type: FloatPort
• value type: float (can be None)

initialize_once

Calibrate the model only once. If set to False, then this node will recalibrate itself whenever a non-streaming data chunk is received that has both training labels and associated training instances.

• verbose name: Calibrate Only Once
• default value: True
• port type: BoolPort
• value type: bool (can be None)

dont_reset_model

Do not reset the model when the preceding graph is changed. Normally, when certain parameters of preceding nodes are being changed, the model will be reset. If this is enabled, the model will persist, but there is a chance that the model is incompatible when input data format to this node has changed.

• verbose name: Do Not Reset Model
• default value: False
• port type: BoolPort
• value type: bool (can be None)

set_breakpoint

Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

• verbose name: Set Breakpoint (Debug Only)
• default value: False
• port type: BoolPort
• value type: bool (can be None)

metadata

User-definable meta-data associated with the node. Usually reserved for technical purposes.

• verbose name: Metadata
• default value: {}
• port type: DictPort
• value type: dict (can be None)