Offline Gaze event detection and saccade-locked EEG ERPs¶

gaze offline

Before getting started, make sure that you are using Pipeline Designer with "Show expert parameters" checked (under File->Settings).

Data Import¶

Here we are importing data from an XDF file comprising several streams, including an 'eeg'-type stream and a 'gaze'-type stream.

Gaze Data Processing¶

We first extract the gaze data with the ExtractStreams node with the stream_names parameter set to ['gaze'].

Most eye trackers provide enough data so one could reconstruct the eye position and orientation as well as its gaze direction, and they usually also provide one or more channels describing the eye-tracker's confidence in the reported data. Here we are only interested in the position of the gaze on the screen and the confidence associated with those positions, so we use the SelectRange node to select only the channels of interest. We are selecting the X and Y positions for both the Left and Right eyes, as well as their Validity (Confidence), for a total of 6 channels. Parameters: axis='space', selection=['LeftEye_GazePoint_PositionOnDisplayArea_X', 'LeftEye_GazePoint_PositionOnDisplayArea_Y', 'RightEye_GazePoint_PositionOnDisplayArea_X', 'RightEye_GazePoint_PositionOnDisplayArea_Y', 'LeftEye_GazePoint_Validity', 'RightEye_GazePoint_Validity'], unit='names'.

We next use the SanitizeGazeData node which uses the Validity measures and the binocular data to reconstruct invalid samples using cross-interpolation. Parameters: confidence_channel='Validity', handle_bad='cross-interpolate', interp_identifiers=['Left', 'Right'].

After the binocular data has been used to reconstruct invalid samples, we next use the SelectRange node once again to keep only one eye (typically the dominant eye). One could also average the two eyes to get a uni-eye position. It is also OK to keep X and Y channels for both eyes in the data, but then the gaze event detection node will treat the gaze position as 4-dimensional which is unintuitive and is more computationally intensive without much benefit. Parameters: axis='space', selection=['RightEye_GazePoint_PositionOnDisplayArea_X', 'RightEye_GazePoint_PositionOnDisplayArea_Y'], unit='names'.

Some gaze-event detection algorithms may expect gaze data to be in specific units. Gaze data can be converted between units using the GazeScreenCoordinateConversion node, but only if the required information is in the packet, and that information can be first set with the SetGazeMetaData node. One can supply the SetGazeMetaData node with the distance between the user and the screen, and with other physical characteristics of the screen. If the latter information isn't provided then the dimensions may be queried from the currently connected monitor. It turns out that the gaze-event detection method we're using in this example is not dependent on the units because it normalizes the position data, so we do not need to supply meta data or convert coordinates here.

The gaze-event detection method we're using does, however, require velocity and acceleration. We next calculate gaze velocity and acceleration from its position using two parallel instances of the FiniteDifferences node, the first with order=1 (default) and the second with order=2. Make sure to check the keep shape box for both nodes. The velocity and acceleration channels must be renamed using the RenameChannels node and parameters search_expr='(Position)' for both and either replace_expr='Velocity' or replace_expr='Acceleration'.

The position, velocity, and acceleration are then concatenated together into a single data packet with the Concatenate Tensors node, along the space axis, and with Append new axis unchecked.

We are now ready to detect gaze events with the ExtractGazeEvents node. As we are not interested in fixations, we set fixation_algorithm='none' and merge_fixation_distance_threshold=0.0 (it will not be necessary to change the latter parameter when fixation_algorithm='none' after the next Neuropype release). We use the velocity and acceleration thresholding algorithm to detect saccades, by setting saccade_algorithm='V+A threshold'. The velocity_threshold argument is very important and depends a lot on the quality of your gaze data and whether or not you want to detect tiny (micro?) saccades. In this example we set velocity_threshold=5.0 but I often satisfied with the default value of 8.0. The ExtractGazeEvents returns a packet containing a table of gaze events, in this case all with the 'Marker' field set to 'Saccade'.

EEG Processing¶

We begin by using ExtractStreams to isolate the EEG packet. We then use AssignChannelLocations to set the physical locations of known EEG channels using a montage file or, if no montage file provided, by assuming default locations for channels named in the 10-20 system. As these data were collected with a BioSemi A-B cap, we can use the built-in montage file that comes with Neuropype named 'biosemi-mne-64-ab.locs'.

The remaining EEG processing steps are standard procedure, and we will only briefly mention a few parameters we changed for this pipeline. We downsample to 256 Hz and use a high-pass at 2 Hz (typically highpass is 0.5 Hz, but we aren't interested in slow components).

ERP Extraction¶

We use the MergeStreams node to merge the EEG and gaze events packets together. This allows us to then use the Segmentation node to extract individual segments with limits [-0.3, 0.4] around each gaze event. We then subtract the per-ERP baseline with the Re-referencing node, setting the baseline to -0.3...-0.2 seconds on the time axis. Finally, we take the Mean along the instance axis to get the multichannel ERPs averaged across all detected saccades.

We additionally use two parallel branches of SelectRange and Mean nodes to calculate the average ERP amplitude in specific time windows. 0.01...0.055 will yield an eye-movement artifact, and 0.21...0.28 will yield an occipital ~P200.