Deep learning: multi-class classification

This advanced example pipeline, DeepLearningMulticlass.pyp, uses a deep learning model, EEGNet, for multi-class classification of EEG, with leave-one-subject-out cross-validation. EEGNet is a compact convolutional neural network for EEG-based brain-computer interfaces (see arxiv.org/abs/1611.08024). The pipeline uses dataset 2a of the BCI Competition IV, a motor imagery data set with nine subjects, to classify four imagined movements: left hand, right hand, foot and tongue.

The pipeline has the same structure as the Cross-validation example, which uses logistic regression on FBCSP features instead of a neural network. We therefore refer to that page for the parts that they have in common (finding the data files, the preprocessing chain, the cross-validation loop and the check of the data set), and discuss here what is different: the loop with caching, the Deep Model node, and the network graph.

The pipeline can be divided into the following parts, which we discuss in the following: data files and preprocessing, caching, cross-validation and results, the deep model, and the network graph.

Data Files and Preprocessing

Includes: Parameter Port, Format (String), Path Iterator, For Each (Parallel), Placeholder, Import File and the preprocessing nodes

As in the Cross-validation example, the Parameter Port node (data_root) holds the folder in which you put the .gdf files of the data set, which you need to download first (from https://www.bbci.de/competition/download/competition_iv/BCICIV_2a_gdf.zip), and the path needs to be adapted accordingly. The Format (String) and Path Iterator nodes find the training file of each subject (A*T.gdf). The function that is called by the Function Call node at the top of the pipeline checks that all the files of the data set were found, as discussed in the Cross-validation example.

Jumping to a function

In Pipeline Designer, you can select a Function Call node and press Ctrl+B to go to the function node that it calls.

The files are processed in a loop by a For Each (Parallel) node (see the screenshot in the next section), which, unlike the sequential For Each node of the Cross-validation example, can run the loop body for several files in parallel (including on GPUs, see compute_backends in the node docs). The nodes in the loop body do not carry over any state from one iteration to the next. The num_procs property sets the number of processes to use, and multiprocess_backend the way in which they are run. Here, num_procs is 1 and the backend is serial, so the files are processed one after the other, but you can increase num_procs to speed this up. The loop body, which is wired into the body port of the loop node, is built in two steps (see the next section): a Placeholder node named path receives the current file path from the loop node, and passes it to a With Cache node, which runs the preprocessing chain on that file. The chain starts with the Placeholder node on the left, with the slotname filepath, which is defined by the Graph [signature] property of the With Cache node, and receives the file path from it. The chain contains the EEG preprocessing nodes:

The preprocessing is very similar to that of the Cross-validation example: Import File imports the file, Rewrite Markers rewrites the event codes to descriptive marker names, Select Range keeps the first 22 (EEG) channels, Resample resamples to 125 Hz, Fix Signal Unit fixes the unit of the signal, IIR Filter band-pass filters it (3 to 40 Hz), and Segmentation cuts segments from 0.5 to 2.505 seconds after the markers. The differences are the following:

• The Standardization node, which scales the data along the time axis so that its standard deviation (measure of std) is 1, is used instead of an exponential standardization.
• Assign Target Values maps four markers to targets (left to 0, right to 1, foot to 2 and tongue to 3), since there are four classes here.

Caching

Includes: With Cache

The output of the preprocessing chain is not wired directly into the loop node, but goes through a With Cache node, which caches the result of the preprocessing for each file. The loop body is thus split into two parts: the path comes from the Placeholder node (named path, defined by the Graph [signature] of the loop node) into the data port of With Cache, and the preprocessing chain is wired into its graph port (a dotted wire), with the signature (filepath). When With Cache is given a new path, it returns the cached data for that file if it is present, and otherwise passes the path on to the filepath placeholder of the chain, runs the chain and stores the result.

As a result, preprocessing the data is only done once per file, which speeds up repeated runs of the pipeline, for instance if you are testing different changes to the model. The cache is invalidated automatically if the preprocessing nodes or their properties have changed, or if the ver property of the node is incremented. The cache is stored in a directory named after the topic property of the node.

Cross-Validation and Results

The Cross-validation node, and the calculation and the printing of the accuracy, work as in the Cross-validation example, which describes them in more detail. The only difference is the method that the Cross-validation node calls: it is the deep learning model, a Deep Model node (see below), instead of the logistic regression model of the other pipeline. As there, the node receives the list of preprocessed packets from the loop node, and calls the model repeatedly, holding out one subject at a time (folds of 1): for each fold, it first calls the Deep Model to train it on the data of the other subjects, and then to test it on the data of the held-out subject. It sends out the misclassification rate through its loss port and its standard deviation through its loss_std port. The accuracy is 1 - MCR, which is calculated by the Subtract node and printed by the Print node.

The Deep Model

Includes: Placeholder, Concatenate Successive Packets, Get Current Task Number and Deep Model

The method that is wired into the method port of the cross-validation node is a Deep Model node, which can be used as a drop-in replacement for a conventional machine learning node. The Cross-validation node passes the incoming data, the list of preprocessed packets, to the placeholder named data (defined in the Method [signature] property of the cross-validation node). A Concatenate Successive Packets node concatenates them into a single packet along the instance axis, which goes into the data port of the Deep Model node, to train the model. The network itself is a graph that is wired into the net port of the node (see the next section).

The Get Current Task Number node returns the number of the current task, which is the number of the fold of the cross-validation, and it is wired into the random_seed port of the Deep Model node, so that the fold number is used as the random seed. The random seed determines the pseudo-random choices that are made during the training of the network, such as the initial values of its weights and the shuffling of the trials. Using the fold number as the seed means that every fold is trained with its own seed, so that the folds do not all start out from the same random initialization, while the results of the whole cross-validation remain reproducible from one run to the next. (This is also what the node does by default inside a cross-validation, if its random_seed is left unspecified; the wiring here makes it explicit.)

The main settings of the Deep Model node are:

• the loss_function used here is softmax_categorical_crossentropy, and the model is trained with the adam(0.001) optimizer (the optimizer property).
• The data is processed in batches of 18 trials (batch_size) and trained for up to 50 epochs (maxsteps and stepunit), with the trials shuffled in every epoch (shuffle).
• A quarter of the training data (validation_split of 0.25) is held out of the training, to evaluate the model during training, using the balanced accuracy (eval_metric), and to stop the training early when the validation performance no longer improves. The model with the lowest loss is retained at the end (retain_model).
• train_on is set to initial offline, so that the model is trained once, on the first non-streaming packet that the node receives, which is the training set. The model then makes predictions on any data that it receives afterwards, such as the held-out test data, without being trained further.

The neural network runs on a compute GPU if you have one, and otherwise on the CPU. See the documentation of the Cross-validation node for options for parallelizing the cross-validation.

The Network Graph

Includes: Placeholder, Convolution Layer (Net), Batch Normalization (Net), Depthwise Convolution Layer (Net), Activation Function (Net), Pooling (Net), Dropout (Net), Depthwise Separable Convolution Layer (Net), Batch Flatten (Net), Dense Layer (Net) and Override Axis

The model/DNN is defined by the graph in the bottom of the pipeline, which is a representation of EEGNet. It starts with a Placeholder node which receives the packets to train or test on from the Deep Model node. The output of its last node, Override Axis, is wired into the net port of the Deep Model node (with a dotted wire).

The slotname of this first Placeholder is input, because that is the name that is listed first in the Net [signature] property of the Deep Model node, (input,is_training). This property defines the arguments that the network graph receives, and the network graph must have a Placeholder node for each of them, whose slotname matches the name in the signature. Whenever the Deep Model node sends data through the network, it passes the data from its data input to the placeholder input, and then takes the result of the network back into the node. The second name in the signature, is_training, is the flag that is discussed at the end of this page.

Net nodes

The nodes whose names end with (Net) are nodes from the deep learning category, which can only be used in a network graph.

• The first Convolution Layer (Net) is the temporal convolution: it sweeps over the time axis (sweep_axes), with the number of filters (output_features) given by the F1 parameter and the length of the kernel (kernel_shape) given by the kernLength parameter.
• Batch Normalization (Net) normalizes the output of the layer.
• The Depthwise Convolution Layer (Net) learns spatial filters, by sweeping over the space axis. The number of spatial filters to learn for each temporal filter, output_features, is given by the D parameter, and its kernel_shape is the number of channels, which is taken from the data by the Get Nested (numChans) node (see below).
• After a second batch normalization, the Activation Function (Net) applies the elu activation function.

• Pooling (Net) takes the average (pool_type) over the time axis in windows of 4 samples (window), and Dropout (Net) randomly drops units during training (rate of 0.25).
• The Depthwise Separable Convolution Layer (Net) and the following Convolution Layer (Net) (with a kernel_shape of 1) are a separable convolution, a spatial convolution of each feature map (kernel_shape of 16) followed by a pointwise convolution. The number of pointwise filters is given by the F2 parameter.
• These are again followed by batch normalization, the elu activation, and by a second pooling (with a window of 8 samples) and dropout.

• Batch Flatten (Net) flattens the features of each trial, and Dense Layer (Net) is the classifier, with one unit per class (units of 4).
• Override Axis renames the feature axis of the output to statistic, so that the result is in the format that the Cross-validation node expects.

Parameters of the Network

The network is configured by four Parameter Port nodes: F1 (the number of temporal filters), F2 (the number of pointwise filters), kernLength (the length of the temporal convolution in the first layer) and D (the number of spatial filters to learn for each temporal filter). They are wired into the properties of the layers described above. The Multiply, Divide and Round nodes calculate a value from the three parameters F1, D and F2, which is used for the output_features of the depthwise separable convolution layer.

The Get Nested (numChans) node extracts the number of channels of the input data (the length of its space axis, with a path of :chunks | eeg | :block | :axes | space | :length) to set the kernel size of the spatial convolution.

The Placeholder (is_training) is a flag that the Deep Model node sets automatically, to indicate whether the model is being trained or tested, and which is wired into the batch normalization and dropout layers, since these behave differently in training and in testing.