← machine_learning package

HierarchicalDiscriminantComponentAnalysis

Use Hierarchical Discriminant Component Analysis (HDCA) to classify data instances.

This node learns a hierarchical linear model that is usually applied to band-pass filtered and segmented neural data, for example EEG. When applied to such data, the method learns a model of the time-domain (event related potential like) signal to explain the dependent variable (e.g., to classify the EEG). HDCA can also be thought of as a hierarchical version of linear discriminant analysis (LDA) where first LDA is applied to each time slice individually, reducing the data to a single dimension per time slice, and then a second-level classifier (usually again an LDA) is applied to the resulting set of 1d features. HDCA is known to perform well on EEG segments at full resolution (i.e., little to no reduction in channel desity and little to no reduction in sampling rate) compared to many other methods since it is agnostic to (often spurious) correlations between channels at different time points. In NeuroPype, HDCA is usually used in conjunction with shrinkage regularization (using shrinkage LDA or sLDA) as the building block and has good defaults. It can also be generalized to use other types of linear models, such as logistic regression or ridge regression, which may have a different robustness profile or can be used as a stepping stone to simulate the behavior of more complex models (e.g., using the ConvexModel node). The regression mode is useful for use with continuous labels. Another generalization is to multi-stream data (e.g., from multiple modalities such as EEG and eye tracking), where the data is fused either at the level of temporal features or at the level of streams (the latter yields a 3-level hierarchy). Version 1.1.1

Ports/Properties

data

Data to process.

• verbose name: Data
• default value: None
• port type: DataPort
• value type: Packet (can be None)
• data direction: INOUT

shrinkage_within

Regularization strength within each time slice. If using 'auto', the parameter is estimated automatically (using the Ledoit-Wolf method for discriminant-type models, GCV for regression, and k-fold cross-validation for logistic models). Otherwise, this is a number (0-1 for discriminant or typically 1e-3 to 1e3 for the other models).

• verbose name: Shrinkage Within
• default value: 0.01
• port type: Port
• value type: object

shrinkage_across

Regularization strength across time slices and/or modalities. Other details same as documented in the shrinkage within parameter.

• verbose name: Shrinkage Across
• default value: 0.01
• port type: Port
• value type: object

model_within

Type of linear model used within each time slice. See also model_across for model types that operate across time slices. The discriminant option is a shrinkage linear discriminant analysis (sLDA), while the regression option is a ridge regression formulation, and logistic is l2-regulared logistic regression. Note that, even though discriminant and logistic typically output discrete values, in this setting only the learned linear mapping is applied, yielding continuous-valued features. Note that for models other than discriminant, it can be useful to scale features prior to this node, especially if the feature scale is very far from unity or very uneven. Huber, Theil-Sen, RANSAC, and RLDA are robusts variant of regression with different performance characteristics, of which Huber should be the first choice. The parameter for Huber is the cutoff in standard deviations beyond which residuals are treated as outliers, and defaults to a value that yields 95% statistical efficiency. The parameters for Theil-Sen are the maximum number of subpopulations to consider and the number of subsamples to draw; this estimator can be quite slow (e.g., 10x slower than Huber). The parameters for RANSAC are the name of the underlying base estimator, which can be 'ols', 'discriminant', 'regression', or 'logistic', optionally the minimum fraction or count of samples to use (unless this is 'ols', this argument must be given), and optionally the threshold in terms of target value error at or below which a data point is flagged as an inlier (defaulting to MAD(TargetValue)). The minimum fraction is a fairly important parameter since it cannot be too small (yielding an ill-posed estimator) or too large (being unlikely to avoid outliers). RLDA is an experimental robust LDA method whose parameters are the covariance estimator ('mcd', 'sgd', 'batch'), the huber threshold (ignored by the mcd method), and optionally an algorithm-specific parameter, which is the base learning rate for the sgd method, the block size for the batch method, and the maximum contamination fraction for the mcd method. One may also specify a separate value per stream by giving this parameter as a dictionary with modality names as keys and model types as values. One may also use the '~' key to specify a catch-all model type for stream types not otherwise listed, as in {'eeg': 'discriminant', '~': 'huber(1.35)'}.

• verbose name: Model Within
• default value: discriminant
• port type: ComboPort
• value type: str (can be None)

model_across

Type of (generalized) linear model to learn across time slices and/or modalities; see also model_within for model types that apply within each time slice. The discriminant analyis is sLDA (see model_within), regression is a ridge regression, and logistic is a logistic regression. Of these, only the ridge regression is suitable for continuous labels. Huber is a robust variant of regression. This can also be a per-stream setting; in this case, and assuming that there are multiple streams, there will be a hierarchy layer that goes across streams (whether temporal when doing temporal multi-stream fusion, or late when doing late fusion); in either case, the model form for this multi-stream "pseudo-modality" can be specified using a dedicated key named '*' (otherwise this defaults to logistic).

• verbose name: Model Across
• default value: discriminant
• port type: ComboPort
• value type: str (can be None)

fusion

Multi-stream fusion strategy. The temporal strategy will fuse multiple streams (if any) at the level of temporal features, and allow the second-level classifier to learn interactions between streams in terms of their detailed time course. The late strategy will fuse streams at the end ('late fusion'), in which case each stream is reduced to a single score, and a third-level classifier is applied to those scores. If each stream stems from a different modality, then this is the multi-modal fusion strategy. The skip option causes the node to skip the fusion step and output the features generated by the temporal level.

• verbose name: Multi-Stream Fusion
• default value: temporal
• port type: EnumPort
• value type: str (can be None)

probabilistic

Output probabilities instead of class labels.

• verbose name: Probabilistic
• default value: True
• port type: BoolPort
• value type: bool (can be None)

class_weights

Per-class weight (dictionary). This is formatted as a Python-style dictionary that assigns to each numeric class (e.g., 0, 1) a weighting (e.g., 1.0). The weights represent the a priori ("prior") probability of encountering a specific class that the model shall assume. The weights will be renormalized so that they add up to one. Example syntax: {'0': 0.5, '1': 0.5} (note the quotes before the colons).

• verbose name: Class Weights
• default value: None
• port type: Port
• value type: object (can be None)

cond_field

The name of the instance data field that contains the conditions to be discriminated. This parameter will be ignored if the packet has previously been processed by a DescribeStatisticalDesign node.

• verbose name: Cond Field
• default value: TargetValue
• port type: StringPort
• value type: str (can be None)

tolerance

Convergence tolerance. Smaller values result in more accurately estimated models at the cost of performance. The exact meaning depends on the model type used (for discriminant analysis, this is the accuracy of rank estimation in the SVD and is an absolute threshold, meaning that it depends on the scale of the data).

• verbose name: Tolerance
• default value: None
• port type: FloatPort
• value type: float (can be None)

solver_type

Type of solver to use. Auto is usually a good choice for EEG data. For logistic regression, other solvers, incl. newton-cg, newton-cholesky, and liblinear are available and could help performance.

• verbose name: Solver Type
• default value: auto
• port type: ComboPort
• value type: str (can be None)

solver_maxiter

Max number of iterations for use with an iterative solver. This currently applies only to the logistic formulation, in which case the default is 100.

• verbose name: Max Iterations
• default value: None
• port type: IntPort
• value type: int (can be None)

solver_extra

Extra solver options. This is a dictionary that can be used to pass additional options to the solver. The exact options depend on the solver used.

• verbose name: Solver Extra
• default value: None
• port type: DictPort
• value type: dict (can be None)

initialize_once

If set to True, the model is trained once.

• verbose name: Initialize Once
• default value: True
• port type: BoolPort
• value type: bool (can be None)

dont_reset_model

Do not reset the model when the preceding graph is changed. Normally, when certain parameters of preceding nodes are being changed, the model will be reset. If this is enabled, the model will persist, but there is a chance that the model is incompatible when input data format to this node has changed.

• verbose name: Do Not Reset Model
• default value: False
• port type: BoolPort
• value type: bool (can be None)

backend

Compute backend to use. This is only used by a few solvers (specifically the rlda method); selecting torch will use a multi-core implementation that can be faster than numpy, but is not necessarily more efficient in terms of compute per core.

• verbose name: Backend
• default value: numpy
• port type: EnumPort
• value type: str (can be None)

verbose

Enable verbose output.

• verbose name: Verbose
• default value: False
• port type: BoolPort
• value type: bool (can be None)

set_breakpoint

Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

• verbose name: Set Breakpoint (Debug Only)
• default value: False
• port type: BoolPort
• value type: bool (can be None)

metadata

User-definable meta-data associated with the node. Usually reserved for technical purposes.

• verbose name: Metadata
• default value: {}
• port type: DictPort
• value type: dict (can be None)