← machine_learning package

LinearDiscriminantAnalysis

Classify data instances using Linear Discriminant Analysis (LDA).

The LDA method is a fast statistical method that learns a linear mapping from input data to discrete category labels. LDA assumes that the data are Gaussian-distributed, that is, have no or very few major statistical outliers. To the extent that these assumptions hold true, this method is highly competitive with other linear (or generalized linear) methods. To ameliorate the outlier issue, the raw data can be cleaned of artifacts with various artifact removal methods. This implementation uses shrinkage regularization by default, which allows it to handle large numbers of features quite gracefully, compared to the unregularized variant, which will overfit or fail on more than a few dozen features. This method can be implemented using a number of different numerical approaches which have different tradeoffs -- it is worth experimenting with different choices of the solver parameter if you cannot get the result that you seek. By default, this method will return not the most likely class label for each trial it is given, but instead the probabilies for each class (=category of labels), that the trial is actually of that class. This method can also optionally use a regularization parameter that is tuned using an internal cross-validation on the data. If there are very few trials, or some extensive stretches of the data exhibit only one class, this cross-validation can fail with an error that there were too few or no trials of a given class present. Like all machine learning methods, this method needs to be calibrated ("trained") before it can make any predictions on data. For this, the method requires training instances and associated training labels. The typical way to get such labels associated with time-series data is to make sure that a marker stream is included in the data, which is usually imported together with the data using one of the Import nodes, or received over the network alongside with the data, e.g., using the LSL Input node (with a non-empty marker query). These markers are then annotated with target labels using the Assign Targets node. To generate instances of training data for each of the training markers, one usually uses the Segmentation node to extract segments from the continuous time series around each marker. Since this machine learning method is not capable of being trained incrementally on streaming data, the method requires a data packet that contains the entire training data; this training data packet can either be accumulated online and then released in one shot using the Accumulate Calibration Data node, or it can be imported from a separate calibration recording and then spliced into the processing pipeline using the Inject Calibration Data, where it passes through the same nodes as the regular data until it reaches the machine learning node, where it is used for calibration. Once this node is calibrated, the trainable state of this node can be saved to a model file and later loaded for continued use. More Info... Version 1.1.1

Ports/Properties

data

Data to process.

• verbose name: Data
• default value: None
• port type: DataPort
• value type: Packet (can be None)
• data direction: INOUT

class_weights

Per-class weights. Optionally this is a mapping from class label to weight. The weights represent the a priori ("prior") probability of encountering a specific class that the model shall assume. The weights will be renormalized so that they add up to one. Example syntax: {'0': 0.5, '1': 0.5} (note the quotes before the colons).

• verbose name: Per-Class Weight
• default value: None
• port type: Port
• value type: object (can be None)

cond_field

The name of the instance data field that contains the conditions to be discriminated. This parameter will be ignored if the packet has previously been processed by a DescribeStatisticalDesign node.

• verbose name: Cond Field
• default value: TargetValue
• port type: StringPort
• value type: str (can be None)

shrinkage

Shrinkage regularization strength. If using 'auto', then a fast automatic method is used to determine the regularization parameter (using the Ledoit-Wolf method). However, if given as a list of numbers between 0 and 1 (formatted as in, e.g., [0.1, 0.2, 0.3]) where 0 is no regularization and 1 is maximal regularization, then the best parameter is searched, which can be slow. The details of the parameter search can be controlled via the search metric and number of folds parameters.

• verbose name: Shrinkage Level
• default value: auto
• port type: Port
• value type: object (can be None)

feature_selection

Feature selection criterion to use if sparse solutions are desired. If set to None, no feature selection is enabled. If set to rfe, recursive feature elimination is used. If set to anova, features with the highest ANOVA f-score are selected. In either case, the number of features is automatically tuned using a cross-validation, and can be capped using max_feature_select.

• verbose name: Feature Selection
• default value: none
• port type: EnumPort
• value type: str (can be None)

max_feature_select

Maximum number of features to retain, if feature selection is used. Using a value that is less than the number of features can be much faster than searching over all possible counts.

• verbose name: Max Number Of Features
• default value: None
• port type: IntPort
• value type: int (can be None)

feature_sel_group_size

Number of successive features that form a group, for grouped feature selection.

• verbose name: Group Size (Feature Sel.)
• default value: 1
• port type: IntPort
• value type: int (can be None)

feature_sel_group_op

Operation to use to calculate groupwise feature importance.

• verbose name: Group Measure (Feature Sel.)
• default value: max
• port type: EnumPort
• value type: str (can be None)

search_metric

Parameter search metric to optimize certain hyper-parameters of the node. When certain parameters (e.g., shrinkage or robust_gamma) are used and given as a list of values, then the method will run a cross-validation for each possible parameter value and use this metric to score how well the method performs in each case, in order to select the best parameter. Therefore, when a search is done the running time of the method is multiplied by the number of parameter values and the number of folds in the cross-validation, which can be slow. While 'accuracy' is usually a good default, some other metrics can be useful under special circumstances, e.g., roc_auc for highly imbalanced ratios of trials from different classes.

• verbose name: Scoring Metric
• default value: accuracy
• port type: EnumPort
• value type: str (can be None)

num_folds

Number of cross-validation folds for parameter search. Cross-validation proceeds by splitting up the data into this many blocks of trials, and then tests the method on each block. For each fold, the method is re-trained on all the other blocks, excluding the test block. This is not a randomized cross-validation, but a blockwise cross-validation, which is usually the correct choice if the data stem from a time series. If there are few trials in the data, one can use a higher number here (e.g., 10) to ensure that more data is available for training

• verbose name: Number Of Cross-Validation Folds
• default value: 5
• port type: IntPort
• value type: int (can be None)

cv_group_field

Optionally a field indicating the group from which each trial is sourced. If given, then data will be split such that test sets contain unseen groups. Examples groups are Subject_id, Session_id, etc.

• verbose name: Grouping Field (Cross-Validation)
• default value:
• port type: StringPort
• value type: str (can be None)

cv_stratified

Optionally perform stratified cross-validation. This means that all the folds have the same relative percentage of trials with each label. Note that this requires labels to be quantized or binned to be meaningful.

• verbose name: Stratified Cross-Validation
• default value: False
• port type: BoolPort
• value type: bool (can be None)

num_jobs

Number of parallel compute jobs. This is only in effect when a parameter search is applied (e.g., in robust mode, or when passing shrinkage as a list). This value only affects the running time and not the results. Values between 1 and twice the number of CPU cores make sense to expedite computation, but may temporarily reduce the responsiveness of the machine. The value of -1 stands for all available CPU cores.

• verbose name: Number Of Parallel Jobs
• default value: 1
• port type: IntPort
• value type: int (can be None)

solver

Solver to use. This node supports formulations based on a least-squares solution (lsqr), eigenvalue decomposition (eigen), and singular-value decomposition (svd). Some of these methods are known to have numerical issues under various circumstances -- consider trying different settings if you cannot achieve the desired results. Note: the svd method can handle many features, but does not support shrinkage-type regularization.

• verbose name: Solver To Use
• default value: eigen
• port type: EnumPort
• value type: str (can be None)

tolerance

Threshold for rank estimation in SVD. Using a larger value will more aggressively prune features, but can make the difference between it working or not.

• verbose name: Rank Estimation Threshold (Sgd Solver)
• default value: 0.0001
• port type: FloatPort
• value type: float (can be None)

robust_method

Type of robust estimator to use, if any. This applies to the covariance matrix only at this point. The MCD method is a slow but accurate estimator using the minimum covariance determinant. The SGD method is a fast estimator using stochastic gradient descent, but requires some parameter tuning (automatic tuning is slow, but a single value can usually be determined for a given study and then used going forward). The SAG method uses stochastic average gradient and tends to be more resilient to bad parameter settings, but is otherwise similar to SGD. Both SGD and SAG are configured via the huber_threshold and robust_gamma parameters. Thus one may use MCD first to determine if robustness is helpful on the given data, and if fast runtime is needed, one can switch to SAG or SGD and tune the parameters until a similar performance is achieved. Note that, if shrinkage is set to 'auto', the shrinkage amount will generally be estimated in a non-robust fashion, which may cause some amount of over- or under-shrinkage.

• verbose name: Robust Method
• default value: none
• port type: EnumPort
• value type: str (can be None)

huber_threshold

Huber threshold, applies only when using the SGD robust estimator. If set to 0, the geometric (or l1) median is used (which is maximally robust but has low statistical efficiency). If set to 'auto' or the empty list, a value will be searched. An if given as a list of values, the list will be used as the search range.

• verbose name: Huber Threshold (Sag/sgd Method)
• default value: 0
• port type: Port
• value type: object (can be None)

robust_gamma

Search range for gamma parameter when using the SGD robust estimation. Note that typically a much smaller range, or even a single value (which requires no built-in grid search), will do, but the value needs to be empirically determined.

• verbose name: Learning Rate (Sag/sgd Method)
• default value: [1, 1.25, 1.5, 2, 2.5, 5, 10, 20, 40, 80, 160]
• port type: ListPort
• value type: list (can be None)

robust_max_contamination

Maximum contaminated data fraction for MCD robust estimator. This is a tradeoff between resistance to larger proportions of bad data (the maximum is around 0.5, and can be used by setting the parameter to its default None value), and higher statistical efficiency since more data is used for the estimate.

• verbose name: Max Contamination (Mcd Method)
• default value: None
• port type: FloatPort
• value type: float (can be None)

initialize_once

Calibrate the model only once. If set to False, then this node will recalibrate itself whenever a non-streaming data chunk is received that has both training labels and associated training instances.

• verbose name: Calibrate Only Once
• default value: True
• port type: BoolPort
• value type: bool (can be None)

dont_reset_model

Do not reset the model when the preceding graph is changed. Normally, when certain parameters of preceding nodes are being changed, the model will be reset. If this is enabled, the model will persist, but there is a chance that the model is incompatible when input data format to this node has changed.

• verbose name: Do Not Reset Model
• default value: False
• port type: BoolPort
• value type: bool (can be None)

dimensionality_reduction

Reduce dimensionality of data to the given number of dimensions. If given, this overrides the regular outputs of LDA (governed by the probabilistic flag), and causes it to produce data of the desired number of dimensions.

• verbose name: Dimensionality Reduction
• default value: None
• port type: IntPort
• value type: int (can be None)

probabilistic

Use probabilistic outputs. If enabled, the node will output for each class the probability that a given trial is of that class; otherwise it will output the most likely class label.

• verbose name: Output Probabilities
• default value: True
• port type: BoolPort
• value type: bool (can be None)

verbosity

Verbosity level. Higher numbers will produce more extensive diagnostic output.

• verbose name: Verbosity Level
• default value: 0
• port type: IntPort
• value type: int (can be None)

set_breakpoint

Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

• verbose name: Set Breakpoint (Debug Only)
• default value: False
• port type: BoolPort
• value type: bool (can be None)

metadata

User-definable meta-data associated with the node. Usually reserved for technical purposes.

• verbose name: Metadata
• default value: {}
• port type: DictPort
• value type: dict (can be None)