Neural oscillatory state classification (motor imagery)

Neural oscillations indicate various neural activities or states, such as relaxation, emotions, cognitive processing and motor control. This example pipeline, MachineLearningMotorImageryClassification.pyp, shows how to harness them to classify between two states, in this case a participant imagining a movement of their left hand or of their right hand (motor imagery). The imagined movements change the power of the oscillations over the left and right motor cortex, and this is what the classifier picks up.

The main node of the pipeline is the Common Spatial Pattern (CSP) filter, in its filter bank variant, which extracts the components of the signal that best discriminate between the two classes. The pipeline is calibrated on a training recording, MotorImageryTrain.set, and then classifies a stream of data from a second recording, MotorImageryTest.set, as it comes in. Both are example data sets that ship with NeuroPype.

The pipeline can be divided into four parts, which we discuss in the following: data acquisition and calibration, preprocessing, classification, and the output of the results.

Data Acquisition and Calibration

Includes: Import SET, LSL Input, Dejitter Timestamps and Inject Calibration Data

The data acquisition is the same as in the oscillatory regression and calibrate on prerecorded data examples. The Inject Calibration Data node first lets the calibration recording through (from the Import SET node titled Import Calibration Data), so that the adaptive and machine learning nodes can train on it, and then lets through the streaming data, which comes from the LSL Input node and Dejitter Timestamps. Its delay_streaming_packets property is enabled, so that no streaming data is lost on the tick on which the calibration data passes. See the calibrate on prerecorded data page for more on this node.

The three nodes in the bottom left of the pipeline (Import SET titled Import Test Data, Stream Data and LSL Output) play back the test recording as an LSL stream called OutStreamData (with its markers as a second stream, OutStreamData-markers), so that the example can be run without any hardware. See Data input in the example pipelines for more on this setup and for how to use your own live data. With live data, the calibration recording should come from the same device (with the same channels) as the live stream.

Preprocessing

Includes: Assign Targets, Select Range, FIR Filter and Segmentation

• Assign Targets assigns a target value to the markers that the classifier is trained on. Its mapping assigns 0 to the marker left and 1 to the marker right (from the Marker column, its iv_column), which are the markers in the data that indicate whether the participant was to imagine a movement with the left or the right hand. The markers must exist in the data.

• Select Range removes the last 6 channels (selection of :-6 on the space axis, using indices), which are not EEG channels in this data set.

• FIR Filter is a band-pass filter that keeps the frequencies from 4 to 42 Hz (its frequencies are 2, 4, 42 and 45 Hz).

• Segmentation cuts the data into one segment (a trial, along the instance axis) for each target marker, from 0.5 to 3 seconds after the marker (time_bounds). Its online_epoching property is marker-locked, which extracts the segments around the markers in both the calibration and the streaming data. (The alternative, sliding, extracts a single segment at the end of the data on every tick, and is used when there are no markers.)

At this point, you could add nodes to clean the data, as in the artifact removal example.

Classification

Includes: Filter Bank Common Spatial Patterns, Variance, Logarithm, Logistic Regression and Discard Long Chunks

• Filter Bank Common Spatial Patterns (FBCSP) extracts components whose variance optimally discriminates between the two conditions (found in the TargetValue field, cond_field), in four frequency bands (bands: 4-7, 8-12, 13-30 and 31-42 Hz), with 3 pattern pairs per band (nof). It trains itself on the calibration data and applies the same filters to the streaming data. Other variants of this node are found in the Neural category (CSP, for instance), and can be used instead of it.
• Variance computes the variance of each component over time, which is its power in the trial, and Logarithm takes its logarithm. These are the features for the classifier.
• Logistic Regression classifies the features. It is trained on the calibration data, with an l2 regularization whose strength is chosen from the values in alphas by 5-fold cross-validation (num_folds), using accuracy as the criterion (search_metric). Its probabilistic property is enabled, so for each trial, it outputs the probability of each of the two classes, instead of just the class label.
• Discard Long Chunks drops the chunks longer than 10 seconds (threshold). The calibration data goes through the pipeline first, and the long chunk that results from it should not be plotted or sent out. The results for the streaming data are short, and pass through.

Output

Includes: LSL Output, Inspect Packet, Set Axis Values, Mean, Bar Plot, Add Constant Value and Time Series Plot

The results of the classification are used in several ways (see the screenshot of the whole pipeline):

• An LSL Output node sends them out over LSL, as a stream called OutStream (with a stream_type of Control), to be picked up by another pipeline or application in real time.
• An Inspect Packet node shows the numerical values in a window, with the markers.

• A Set Axis Values node names the two features left and right, a Mean node averages them over the trials (the instance axis), since a Bar Plot requires a 1D vector, and the Bar Plot shows the average probability of each class (with the y_limits set to 0 to 1).

• A second branch shows the probability for each trial, over time. An Add Constant Value node subtracts 0.5 (offset of -0.5) from the probabilities, so that the zero line of each class represents a probability of 0.5, and points above the line are classified as that class. A Set Axis Values node names the classes, and a Time Series Plot shows them (30 seconds, time_range) for each trial. Its plot_markers property is enabled, which also plots the markers, so that the predictions can be compared with the ground truth for each trial.