← file_system package

ImportXDF

Import data from an xdf file.

The XDF file format (see "more.." link below) can store one or more streams of multi-channel time series data, such as EEG, EXG, eye tracking, motion capture, audio, and video, as well as marker data, and can be recorded to using, e.g., the Lab Streaming Layer. This node can import any subset of streams from an XDF file, and supports the minimum necessary per-stream meta-data. (You can specify the streams to retain using the 'retain_streams' property, or alternatively, use the ExtractStreams node after import.) The node outputs the entire data in a single large packet on the first update, so any processing applied to the result will be "offline" or "batch" style processing on data that isn't streaming ( consequently, the output packet is flagged as non-streaming). However, if you intend to simulate online processing, it is possible to chain a StreamData node after this one, which will take the imported recording and play it back in a streaming fashion (that is, in small chunks, one at a time). The contents of an XDF file are time-synced, and this node supports some options for processing the respective time stamps to ensure good data alignment in the presence of clock drift and jitter. This node creates a Packet with each imported stream. Each stream holds a block with a 2D tensor array holding the data, and a space axis ( channels) and a time axis (samples), which hold the channel names and timestamps, respectively. If the data has markers (stream type 'markers'), the stream is formatted as a vector of numeric data (one per marker, initially all NaN) and a single instance axis that indexes the marker instances and whose entries are associated each with a timestamp and a string payload that is the respective event/marker type from the .xdf file. The numeric data can be overridden based on the event type string using the Assign Targets node, which is required for segmentation and supervised machine learning. Imported streams are automatically named based on their modality (e.g., 'eeg', 'nirs'), and if multiple streams of the same modality are present, a number is appended: 'eeg-1', 'eeg-2', and so on. In the latter case, if you want to be able to identify the original stream name recorded in the XDF file (i.e., the LSL stream name it originated with), select the use_streamnames option which will use the original stream names and not rename them by modality. (You should still have access to the modality in the Origin.modality stream props which you can access with the ExtractStreamProperties node.) Technical details: When imported, the XDF file data is stored in an object of class Packet which holds a dictionary of data streams which are objects of class Chunk (The dictionary keys are the stream names. Note that we generally refer chunk objects as "streams" in the documentation for simplicity.) Each chunk holds a single object of class Block, which in turn holds a 2D numpy array of the signal (i.e., EEG) data, accessible at .data, and at least 2 axes, accessible at .axes, of class SpaceAxis and TimeAxis, which themselves hold properties of the channels and timestamps. For example, the EEG channel names will be at .chunks['eeg'].block.axes[space].names, and the EEG data at .chunks['eeg'].block.data. See the Neuropype Developer docs for more information on the Neuropype data model. If the XDF file was generated by NeuroPype (using RecordToXDF), the stream properties for that data will be available in a neuropype dictionary in the stream (aka Chunk) props (available with GetStreamProperty or GetNested). More Info... Version 1.6.0

Ports/Properties

data

Output signal.

• verbose name: Data
• default value: None
• port type: DataPort
• value type: Packet (can be None)
• data direction: OUT

filename

Name of the recording dataset to import. If a relative path is given, a file of this name will be looked up in the standard data directories (/resources and Examples/).

• verbose name: Filename
• default value:
• port type: StringPort
• value type: str (can be None)

retain_streams

List of streams to retain. The order doesn't actually matter (it's always data streams first, marker streams second).

• verbose name: Retain Streams
• default value: None
• port type: Port
• value type: object (can be None)

handle_clock_sync

Enable clock synchronization. Needed if data were recorded across multiple computers.

• verbose name: Handle Clock Sync
• default value: True
• port type: BoolPort
• value type: bool (can be None)

handle_jitter_removal

Enable jitter removal for regularly sampled streams. This removed jitter under the assumption that the sampling rate of the data was constant (unless the sampling rate of a stream is explicitly marked as irregular).

• verbose name: Handle Jitter Removal
• default value: True
• port type: BoolPort
• value type: bool (can be None)

handle_clock_resets

Handle clock resets. Whether the importer should check for potential resets of the clock of a stream (e.g. computer restart during recording, or hot-swap).

• verbose name: Handle Clock Resets
• default value: True
• port type: BoolPort
• value type: bool (can be None)

reorder_timestamps

If the file contains a stream with irregular sampling rate and timestamps that are out of order, reorder the samples so that the timestamps are monotonically increasing.

• verbose name: Reorder Timestamps
• default value: False
• port type: BoolPort
• value type: bool (can be None)

use_streamnames

Use the stream names in the file to name streams. If enabled, the streams loaded will be named as in the file. Otherwise, the streams use canonical names based on the content type, such as eeg or markers.

• verbose name: Use Streamnames
• default value: False
• port type: BoolPort
• value type: bool (can be None)

max_marker_len

Optionally a max length on the event markers. Markers longer than this will be substituted by the string where XXXX is a string key into the chunk's .props['long_markers'] field, which is basically a string table. This is only useful if long markers slow down or otherwise throw off downstream processing.

• verbose name: Max Marker Len
• default value: None
• port type: IntPort
• value type: int (can be None)

file_missing

Whether to raise an exception (error) if the file is missing or not found (which may stop the pipeline), or print a log message and continue.

• verbose name: File Missing
• default value: raise
• port type: EnumPort
• value type: str (can be None)

use_caching

Enable caching. Will use cached imported data, if available, instead of reimporting.

• verbose name: Use Caching
• default value: False
• port type: BoolPort
• value type: bool (can be None)

verbose

Print verbose diagnostics.

• verbose name: Verbose
• default value: False
• port type: BoolPort
• value type: bool (can be None)

set_breakpoint

Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

• verbose name: Set Breakpoint (Debug Only)
• default value: False
• port type: BoolPort
• value type: bool (can be None)

metadata

User-definable meta-data associated with the node. Usually reserved for technical purposes.

• verbose name: Metadata
• default value: {}
• port type: DictPort
• value type: dict (can be None)

cloud_host

Cloud storage host to use (if any). You can override this option to select from what kind of cloud storage service data should be downloaded. On some environments (e.g., on NeuroScale), the value Default will be map to the default storage provider on that environment.

• verbose name: Cloud Host
• default value: Default
• port type: EnumPort
• value type: str (can be None)

cloud_account

Cloud account name on storage provider (use default if omitted). You can override this to choose a non-default account name for some storage provider (e.g., Azure or S3.). On some environments (e.g., on NeuroScale), this value will be default-initialized to your account.

• verbose name: Cloud Account
• default value:
• port type: StringPort
• value type: str (can be None)

cloud_bucket

Cloud bucket to read from (use default if omitted). This is the bucket or container on the cloud storage provider that the file would be written to. On some environments (e.g., on NeuroScale), this value will be default-initialized to a bucket that has been created for you.

• verbose name: Cloud Bucket
• default value:
• port type: StringPort
• value type: str (can be None)

cloud_credentials

Secure credential to access cloud data (use default if omitted). These are the security credentials (e.g., password or access token) for the the cloud storage provider. On some environments (e.g., on NeuroScale), this value will be default-initialized to the right credentials for you.

• verbose name: Cloud Credentials
• default value:
• port type: StringPort
• value type: str (can be None)