← neural package

ChannelRepair

Repair channels during periods where they record garbage data.

This is decided using a correlation threshold, which is the main tunable parameter of the filter that governs the aggressiveness. Important: This filter assumes the the input signal has already been highpass filtered to remove drifts (e.g., using an IIR filter). Also, this filter will not repair channels that are already bad throughout the calibration data -- it is meant to remove temporary glitches (e.g., sensor disconnected). You can precede this filter with a Bad Channel Removal filter to ensure that such channels have already been removed. If this filter is used on streaming data and has not yet been calibrated, then it will first buffer n seconds of calibration data to determine some statistics, before any output is produced. This filter operates on a sliding window, and by default outputs processed data only after a certain amount of future data has been seen, in order to make a good decision on what is an artifact and what isn't, and this timing behavior is tunable through two parameters. Also, the correlation statistic is robustly estimated, and there is a tradeoff between quality of this estimate and CPU load (which can be quite heavy) and which can be adjusted through two related parameters. The default setting is tuned for fast processing rather than quality. The statistics gathered on the calibration data are considered "trainable state", and can be saved in model files for later reuse. Note that for the impedance metric, no calibration is necessary if the 'prefer location-based statistics' box is checked, in which case the 'calibration data' length can be set to 0. Impedance is only available for EEG data with an impedance carrier wave present and supported on select devices only. Version 1.3.2

Ports/Properties

data

Data to process.

• verbose name: Data
• default value: None
• port type: DataPort
• value type: Packet (can be None)
• data direction: INOUT

diagnostic

Diagnostic information indicating the channels that were flagged as bad based on the metric and threshold.

• verbose name: Diagnostic
• default value: None
• port type: DataPort
• value type: Packet (can be None)
• data direction: OUT

diagnostic_avg_corr

Diagnostic information (the computed per channel values of the selected quality metric, i.e ., correlation, CoV, impedance, etc.). (Note: despite the original port name which was left unchanged for backward compatibility reasons, this port will output the values corresponding to selected metric even if it is not correlation.)

• verbose name: Diagnostic Avg Corr
• default value: None
• port type: DataPort
• value type: Packet (can be None)
• data direction: OUT

prior_cov

Prior covariance matrix.

• verbose name: Prior Cov
• default value: None
• port type: DataPort
• value type: Packet (can be None)
• data direction: IN

metric

Quality metric. This is a measure of the quality of fit between a robust surrogate of a channel (computed from other channels), and the observed channel itself. Corr is the correlation, and expvar is the explained variance score. Blanket marks everything for repair (re-estimation) and is at best experimental. CoV is specific to NIRS data, and uses the coefficient of variation as the metric, while CoV@ramp is the coefficient of variation normalized by channel length (the cutoff is the equiv to 1cm, e.g. 7.5). CoV@quad is the coeffient of variation normalized to a quadratic curve (example thresholds are 1-1.5); Impedance will compute the impedance and threshold the data based on the computed impedance; it assumes that an impedance carrier wave is present in the data (only supported for certain devices).

• verbose name: Quality Metric
• default value: corr
• port type: EnumPort
• value type: str (can be None)

min_corr

Quality threshold, for the metric of interest. The most aggressive (impossible to attain) quality level would be 1.0, and 0.0 corresponds to a roughly random (uncorrelated) signal. High values (above 0.7) are quite aggressive and will cause channels to be repaired even when they have only moderate artifacts. Values below 0.5 would be considered rather lax (i.e., only the worst channel artifacts get repaired). This threshold is based on the correlation between a channel and what one would expect the channel to be based on the other channels. For the impedance metric, this threshold is expressed in kOhms and is device dependant (i.e., 2000 kOhms).

• verbose name: Quality Threshold
• default value: 0.7
• port type: FloatPort
• value type: float (can be None)

soft_zone

Soft-threshold zone around min_corr threshold. If the proxy-soft reconstruction method is used, then a sigmoid-type soft threshold is applied at min_corr +/- soft_zone, yielding a soft blend between the original channel content and reconstructed channel content.

• verbose name: Soft Zone
• default value: None
• port type: FloatPort
• value type: float (can be None)

window_len

Window length to compute correlations. The length of the time windows, in seconds, for which channel quality is computed, i.e. time granularity of the measure. Ideally this is short enough to reasonably capture periods where artifacts happen, but no shorter (otherwise the statistic becomes too noisy and thus unreliable).

• verbose name: Sliding Window Length
• default value: 0.5
• port type: FloatPort
• value type: float (can be None)

processing_delay

Acceptable processing delay. This controls the amount by which the filter delays the signal, in seconds. This can be anywhere between 0 and half of the window length. For low-latency real-time operation, this should obviously be as low as possible, but if too low, some sharp-onset artifacts can leave brief knacks in the data.

• verbose name: Max Processing Delay
• default value: 0.05
• port type: FloatPort
• value type: float (can be None)

subset_size

Size of random channel subsets to use. This is for use in a RANSAC estimator, and can be given as number of channels or, if below 1.0, as a fraction of the total number of channels. Smaller subsets (e.g., 0.1-0.15) together with higher number of Ransac samples (e.g., 50-100) gives a more robust estimate, at increased computational cost, which can be quite substantial.

• verbose name: Ransac Subset Size
• default value: 0.25
• port type: FloatPort
• value type: float (can be None)

num_samples

Number of random channel subsets to use. This for use in a RANSAC estimator. Higher numbers (e.g., 50-100) together with smaller subset sizes (e.g., 0.1-0.15) give a more robust estimate, at higher computational cost.

• verbose name: Ransac Subset Count
• default value: 50
• port type: IntPort
• value type: int (can be None)

location_wise_corr

Calculate correlation based on channel locations, instead of calibration statistics (experimental). If the amount of collected calibration data was very small, then using channel locations, if known a priori, can result in better correlation statisics, and thus better identification of artifacts. This is primarily for the case where this method is used without a preceding removal of bad channels, and with little initial calibration data.

• verbose name: Prefer Location-Based Statistics
• default value: False
• port type: BoolPort
• value type: bool (can be None)

linenoise_aware

Use line-noise aware processing. Whether the operation should be performed in a line-noise aware manner. If enabled, the correlation measure will not be affected by the presence or absence of line noise. May be turned off if line noise has already been removed.

• verbose name: Line Noise Aware Processing
• default value: True
• port type: BoolPort
• value type: bool (can be None)

filter_type

Type of filter to apply to data. Notch filter is an autoregressive moving average (ARMA) model that is best used when wanting to repair bad channels. The lowpass filter method provides better estimates when observing channel correlations for use in diagnostic analyses. These filters only apply if line noise aware is selected.

• verbose name: Filter Type (If Linenoise Aware)
• default value: Notch
• port type: EnumPort
• value type: str (can be None)

init_on

Time range to calibrate (initialize) the filter on. This parameter can take a single number or two numbers. In case of streaming data, this should always be a single number, representing the number of seconds to buffer from the start of the data, for calibration. In case of offline (recorded) data, this can either be a single number, in which case it represents the window of time in seconds from the beginning of the recording to be used; or, it can be a list of two numbers, in which case this refers to a range of data in seconds, relative to the start of the data, to be used for calibration. The latter allows you to calibrate on data other than the first segment of the data (i.e., if known to be bad), or to avoid running the (fairly expensive) filter on a very long file or on each fold of a cross-validation. Note that a value of 0 here will in the case of offline data be interpreted as the entire file, and in the case of streaming data will raise an error.

• verbose name: Calibration Range
• default value: []
• port type: ListPort
• value type: list (can be None)

calib_seconds

Amount of data, in seconds, to buffer for calibration on streaming data. For offline (non-streaming) data use the "calibration range" (init_on) parameter instead.

• verbose name: Calib Seconds
• default value: 20
• port type: IntPort
• value type: int (can be None)

emit_calib_data

Emit the data buffer that was used for calibration, after calibration is complete, in a single chunk. If False, this filter will discard the calibration data. Since this chunk can be quite long, it is often preferable to discard it in a real-time pipeline, but if subsequent nodes need to see the processed calibration data to calibrate themselves (quite likely the case), it needs to be emitted.

• verbose name: Emit Calibration Data
• default value: True
• port type: BoolPort
• value type: bool (can be None)

chunk_length

Maximum chunk length for offline processing. Process the data in chunks of no larger than this (to avoid running out of memory).

• verbose name: Max Offline Processing Chunk Length
• default value: 50000
• port type: IntPort
• value type: int (can be None)

calib_precision

Block size for robust estimator. Data statistics will be estimated in blocks of this many samples and then robustly averaged. If this is too small, especially for a large number of channels, you may end up with lower-quality estimates.

• verbose name: Estimator Block Size
• default value: 10
• port type: IntPort
• value type: int (can be None)

coords_override

Override sensor coordinates. Allows overriding the coordinates of the channels.

• verbose name: Override Sensor Coordinates
• default value: None
• port type: FloatPort
• value type: float (can be None)

use_clean_window

Use clean time windows for calibration. This is available only during offline processing.

• verbose name: Use Clean Time Windows For Calibration
• default value: False
• port type: BoolPort
• value type: bool (can be None)

zscore_thresholds

Minimum and maximum of clean signal range (for use clean window option). The minimum and maximum standard deviations within which the power of a channel must lie (relative to a robust estimate of the clean EEG power distribution in the channel) for it to be considered not bad.

• verbose name: Clean Signal Range
• default value: [-3.5, 5]
• port type: ListPort
• value type: list (can be None)

max_bad_channels

Maximum fraction of bad channels allowed (for use clean window option). The maximum fraction of bad channels that a retained window may still contain (more than this and it is removed). Reasonable range is 0.05 (very clean output) to 0.3 (very lax cleaning of only coarse artifacts).

• verbose name: Max Fraction Of Bad Channels
• default value: 0.15
• port type: FloatPort
• value type: float (can be None)

window_len_cleanwindow

Length of sliding window (for use clean window option). This is the window length used to check the data for artifact content, in seconds. This is ideally as long as the expected time scale of the artifacts but short enough to allow for several 1000 windows to compute statistics over.

• verbose name: Window Length
• default value: 0.5
• port type: FloatPort
• value type: float (can be None)

window_overlap

Window overlap fraction (for use clean window option). The fraction of two successive windows that overlaps. Higher overlap ensures that fewer artifact portions are going to be missed, in expense of being slower.

• verbose name: Window Overlap Fraction
• default value: 0.66
• port type: FloatPort
• value type: float (can be None)

gpu_mode

GPU mode. The -transform modes allow the cricial operation in the optimal and optimal-cached reconstruction modes to be performed efficiently on the GPU using the respective backend. The -core modes perform a broader set of core operations on the GPU. The most efficient of these for transform is currently torch.

• verbose name: Gpu Mode
• default value:
• port type: EnumPort
• value type: str (can be None)

calibration_backend

Backend for initializing the RANSAC projection. This is a one-time computation, which can however be fairly time consuming for many channels. The multiprocessing, loky, and nestable backends support both the case where location_wise_corr is enabled (use locations) and disabled (use data statistics). Multiprocessing is the simplest, but loky and nestable can be used when the node is itself invoked inside a parallelized computation. The other backends currently only support the case where no locations are provided, but are potentially much faster. Of those, torch-cpu works on the CPU, and the others (tf, jax, cupy, torch) work on the GPU.

• verbose name: Calibration Backend
• default value: multiprocessing
• port type: EnumPort
• value type: str (can be None)

calibration_cores

Number of cores to use for calibration. The special value -1 stands for all cores.

• verbose name: Calibration Cores
• default value: 4
• port type: IntPort
• value type: int (can be None)

reconstruction

Reconstruction mode. Optimal is the optimal least-squares reconstruction given the estimated pattern of good/bad channels (can be very slow for >500 channels). Optimal-cached additionally caches the most recent k patterns, and is therefore much faster if the set of bad channels does not change very rapidly (but otherwise it'll just use more memory use without much benefit). Proxy reconstructs a channel with its robust RANSAC proxy; this is less accurate, but is very fast and has no CPU load spikes regardless of how fast bad channels are changing. Nan sets stretches of bad channels to NaN. Proxy-soft blends between the original and proxy data depending on the degree of threshold violation. In blanket mode, simply everything will be replaced by reconstructed data.

• verbose name: Reconstruction
• default value: optimal
• port type: EnumPort
• value type: str (can be None)

use_caching

Use caching. Only applicable on offline data.

• verbose name: Use Caching
• default value: False
• port type: BoolPort
• value type: bool (can be None)

verbose

Show verbose output.

• verbose name: Verbose
• default value: False
• port type: BoolPort
• value type: bool (can be None)

set_breakpoint

Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

• verbose name: Set Breakpoint (Debug Only)
• default value: False
• port type: BoolPort
• value type: bool (can be None)

metadata

User-definable meta-data associated with the node. Usually reserved for technical purposes.

• verbose name: Metadata
• default value: {}
• port type: DictPort
• value type: dict (can be None)