← machine_learning package

SupportVectorClassification

Classification using support vector machines.

This is the traditional kernel support vector machine, which is very efficient at finding non-linear structure in the data. This method has two important regularization parameters (cost and gamma), which are by default searched over a relatively coarse interval. For best results you may have to use a finer range (at some performance cost). Note that it is typically not very reasonable to use a non-linear classifier like this on the channel signal without any sort of spatial filtering (such as CSP or ICA). Like all machine learning methods, this method needs to be calibrated ("trained") before it can make any predictions on data. For this, the method requires training instances and associated training labels. The typical way to get such labels associated with time-series data is to make sure that a marker stream is included in the data, which is usually imported together with the data using one of the Import nodes, or received over the network alongside with the data, e.g., using the LSL Input node (with a non-empty marker query). These markers are then annotated with target labels using the Assign Targets node. To generate instances of training data for each of the training markers, one usually uses the Segmentation node to extract segments from the continuous time series around each marker. Since this machine learning method is not capable of being trained incrementally on streaming data, the method requires a data packet that contains the entire training data; this training data packet can either be accumulated online and then released in one shot using the Accumulate Calibration Data node, or it can be imported from a separate calibration recording and then spliced into the processing pipeline using the Inject Calibration Data, where it passes through the same nodes as the regular data until it reaches the machine learning node, where it is used for calibration. Once this node is calibrated, the trainable state of this node can be saved to a model file and later loaded for continued use. More Info... Version 1.1.1

Ports/Properties

data

Data to process.

• verbose name: Data
• default value: None
• port type: DataPort
• value type: Packet (can be None)
• data direction: INOUT

kernel

Kernel type to use. This is a non-linear transform of the feature space, allowing for non-linear decision boundaries between classes. The different kernels are Linear kernel (a trivial do-nothing kernel), Polynomial, which yields components that are all possible polynomial combinations of input features up to the desired degree, Radial-Basis Functions, which is one of the most commonly-used non-linear kernels and the Sigmoid kernel.

• verbose name: Kernel Type
• default value: rbf
• port type: EnumPort
• value type: str (can be None)

feature_selection

Feature selection criterion to use. If set to None, no feature selection is enabled. If set to anova, features with the highest ANOVA f-score are selected; this is a good option for simple kernels like linear SVM. If set to mi, features with highest mutual information are selected, which is better for highly non-linear kernels like rbf. In either case, the number of features is automatically selected using a cross-validation, and can be constrained using num_features.

• verbose name: Feature Selection
• default value: none
• port type: EnumPort
• value type: str (can be None)

probabilistic

Use probabilistic outputs. If enabled, the node will output for each class the probability that a given trial is of that class; otherwise it will output the most likely class label.

• verbose name: Output Probabilities
• default value: True
• port type: BoolPort
• value type: bool (can be None)

tolerance

Convergence tolerance. This is the desired errors tolerance or acceptable inaccuracy in the solution. Using larger values gives less accurate results, but will lead to faster compute times. Note that, for biosignal-driven machine learning systems, one often does not need very small tolerances.

• verbose name: Tolerance
• default value: 0.001
• port type: FloatPort
• value type: float (can be None)

max_iter

Maximum number of iterations. This is one of the stopping criteria to limit the compute time. The default is usually fine, and gains from increasing the number of iterations will be minimal (it can be worth experimenting with lower iteration numbers if the algorithm must finish in a fixed time budget, at a cost of potentially less accurate solutions). If set to -1, no limit is in effect.

• verbose name: Maximum Number Of Iterations
• default value: -1
• port type: IntPort
• value type: int (can be None)

cost

SVM cost parameter. This value determines the degree to which solutions are penalized that would mis-classify data points. Higher values result in models that are less likely to mis-classify the training data, but at the expense of potentially worse generalization to new data (less margin for error when slightly different trials are encountered in future test data). This is a list of candidate values, the best of which is found via an exhaustive search (i.e., each value is tested one by one, and therefore the total running time is proportional to the number of values listed here). The details of the parameter search can be controlled via the search metric and number of folds parameters. Larger values cause stronger regularization, that is, less risk of the method over-fitting to random details of the data, and thus better generalization to new data. A very small value means effectively no penalty, and there is no upper limit to how large the values that can be given here may be -- however, depending on the scale of the data and the number of trials. Often one covers a range between 0.1 and 10, and at times 0.01 to 100. Typically the values here are not linearly spaced, but follow an exponential progression (e.g., 0.25, 0.5, 1, 2, 4, 8, ... etc). The default search range is intentionally coarse for quick running times; refine it to smaller steps to obtain potentially better solutions, but do not expect massive gains from refining.

• verbose name: Cost
• default value: [0.01, 0.1, 1.0, 10.0, 100]
• port type: Port
• value type: list (can be None)

poly_degree

Degree of the polynomial kernel. Ignored by other kernel types. This is the maximum degree of polynomial combinations of feature that are generated. This is also a list of possible values, and is searched in a grid search just like the cost parameter (see cost parameter for details on this procedure).

• verbose name: Degree (Polynomial Kernel Only)
• default value: [1, 2, 3]
• port type: Port
• value type: list (can be None)

gamma

Gamma parameter of the RBF kernel. This parameter controls the scale of the kernel mapping, where lower scales can capture smaller-scale structure in the data. When left at the default, it resolves to 1 divided by the number of features. This is a list of possible values, and is searched in a grid search just like the cost parameter (see cost parameter for details on this procedure).

• verbose name: Scale (Rbf Kernel Only)
• default value: [0.0001, 0.001, 0.01, 0.1, 1.0, 10.0]
• port type: Port
• value type: list (can be None)

coef0

Constant term in kernel function. Only used in polynomial and sigmoid kernels.

• verbose name: Constant (Poly Or Sigmoid Kernels Only)
• default value: [0.0, 1.0]
• port type: Port
• value type: list (can be None)

num_features

How many features to select. Given as a list of possible values to search over alongside all other parameters. Can also be a single value, which is equivalent to the full range up to this number. If not given, defaults to max_feature_select or the number of features in the data otherwise.

• verbose name: Num Features
• default value: None
• port type: ListPort
• value type: list (can be None)

max_feature_select

Maximum number of features to consider. This is applied using ANOVA f-scores, and can be used to speed up the mutual information based feature-selection mode.

• verbose name: Max Feature Select (Anova)
• default value: None
• port type: IntPort
• value type: int (can be None)

search_metric

Parameter search metric. This method will run a cross-validation for each possible shrinkage parameter value and use this metric to score how well the method performs in each case, in order to select the best parameter. Therefore, the running time of the method is proportional to the number of parameter values times the number of folds in the cross-validation, which can be slow. While 'accuracy' is usually a good default, some other metrics can be useful under special circumstances, e.g., roc_auc for highly imbalanced ratios of trials from different classes.

• verbose name: Scoring Metric For Parameter Search
• default value: accuracy
• port type: EnumPort
• value type: str (can be None)

num_folds

Number of cross-validation folds for parameter search. Cross-validation proceeds by splitting up the data into this many blocks of trials, and then tests the method on each block. For each fold, the method is re-trained on all the other blocks, excluding the test block. This is not a randomized cross-validation, but a blockwise cross-validation, which is usually the correct choice if the data stem from a time series. If there are few trials in the data, one can use a higher number here (e.g., 10) to ensure that more data is available for training

• verbose name: Number Of Cross-Validation Folds
• default value: 5
• port type: IntPort
• value type: int (can be None)

cv_group_field

Optionally a field indicating the group from which each trial is sourced. If given, then data will be split such that test sets contain unseen groups. Examples groups are SubjectID, SessionID, etc.

• verbose name: Grouping Field (Cross-Validation)
• default value:
• port type: StringPort
• value type: str (can be None)

cv_stratified

Optionally perform stratified cross-validation. This means that all the folds have the same relative percentage of trials in each class.

• verbose name: Stratified Cross-Validation
• default value: False
• port type: BoolPort
• value type: bool (can be None)

verbosity

Verbosity level. Higher numbers will produce more extensive diagnostic output.

• verbose name: Verbosity Level
• default value: 0
• port type: IntPort
• value type: int (can be None)

class_weights

Per-class weights. Optionally this is a mapping from class label to weight. The weights represent the a priori ("prior") probability of encountering a specific class that the model shall assume. The weights will be renormalized so that they add up to one. Example syntax: {'0': 0.5, '1': 0.5} (note the quotes before the colons).

• verbose name: Per-Class Weight
• default value: None
• port type: Port
• value type: object (can be None)

shrinking

Use shrinking heuristic.

• verbose name: Shrinking
• default value: True
• port type: BoolPort
• value type: bool (can be None)

cache_size

"Cache size in MB.

• verbose name: Cache Size
• default value: 200
• port type: IntPort
• value type: int (can be None)

random_seed

Random seed. Different values may give slightly different outcomes.

• verbose name: Random Seed
• default value: 12345
• port type: IntPort
• value type: int (can be None)

initialize_once

Calibrate the model only once. If set to False, then this node will recalibrate itself whenever a non-streaming data chunk is received that has both training labels and associated training instances.

• verbose name: Calibrate Only Once
• default value: True
• port type: BoolPort
• value type: bool (can be None)

dont_reset_model

Do not reset the model when the preceding graph is changed. Normally, when certain parameters of preceding nodes are being changed, the model will be reset. If this is enabled, the model will persist, but there is a chance that the model is incompatible when input data format to this node has changed.

• verbose name: Do Not Reset Model
• default value: False
• port type: BoolPort
• value type: bool (can be None)

cond_field

The name of the instance data field that contains the conditions to be discriminated. This parameter will be ignored if the packet has previously been processed by a BakeDesignMatrix node.

• verbose name: Cond Field
• default value: TargetValue
• port type: StringPort
• value type: str (can be None)

set_breakpoint

Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

• verbose name: Set Breakpoint (Debug Only)
• default value: False
• port type: BoolPort
• value type: bool (can be None)

metadata

User-definable meta-data associated with the node. Usually reserved for technical purposes.

• verbose name: Metadata
• default value: {}
• port type: DictPort
• value type: dict (can be None)