← file_system package

ImportCSV

Import a CSV file containing EEG, or other non-signal data.

This node can import four types of CSV files: 1) EEG data with timestamps and markers (events). The assumed format is the first record in the CSV file contains the column header names, including the EEG channel labels. It is assumed that all columns except the timestamp and marker names are EEG channel data, but if some are not, these can be excluded with the Exclude Columns parameter (alternatively, use the SelectRange node after this one to filter out channels (columns) that do not contain EEG data). You can specify the column holding the timestamps either by name (if you have headers, or by index (0-based, meaning the first column is 0). The same applies to the marker column. If the marker column is not specified, it is assumed Note that timestamps are expected to be in seconds. The node outputs the entire data in a single Packet. The packet generated by this node is formatted as follows: the first stream is called eeg (this can be changed using the data stream name parameter), and holds the EEG/signal data as a 2d array with a space axis (channels) and a time axis (samples). If the data has markers and marker column was specified, a second stream named markers is included, which has a single instance axis with a .data['Marker'] array containing the markers, and .times array with the corresponding event timestamps (marker names that are numeric values are converted to strings). 2. CSV file event event markers only. It is also possible to import a CSV file containing markers only, by importing a 2 column CSV with the event name and timestamp, and filling in the timestamp column and marker column respectively. In that case, the packet will have a markers stream only. This could be merged into a signal stream from another source using the MergeStreams node. If the CSV file contains some rows (lines) that are comments or metadata, these can be ignored by setting the non_data_rows parameter to the number of rows at the top of the file that should be ignored on input. A file without a header row can be imported by setting the no_header_row parameter to True. 3. Non-signal data without a time axis. This node can also import non-signal data that does not have a time axis. In such a case, the resulting Packet will have a Feature axis with the columns, and a Instance axis with the rows (much like the space x time matrix for signal data). If there is a column header row (used to identify the columns), this will be used to name the feature elements in the Feature axis (in the .names field). If there is a rows header column (to identify the rows, i.e., SubjectID), this will be used to populate a field in the Instance axis by that same name. 4. Statistical data. Alternatively, it is also possible to import a CSV file containing data for use in statistical analyses (e.g. a spreadsheet with headers that are independent and dependent variables). If the dependent variable columns parameter is specified the packet will have a feature axis containing each dependent variable (e.g. computed metrics), plus an instance axis of all the rows with the other columns set as individual .data fields (e.g. .data['subject_id'], .data['age'], .data['date'], .data['condition'], etc.), that can be used as factors for statistics. Typically each row would either be an individual trial, in case of a single subject, or session means for each subject if importing a group analysis data. Version 1.7.0

Ports/Properties

data

Output signal.

• verbose name: Data
• default value: None
• port type: DataPort
• value type: Packet (can be None)
• data direction: OUT

eeg_channel_names

EEG channel labels

• verbose name: Eeg Channel Names
• default value: None
• port type: DataPort
• value type: list (can be None)
• data direction: OUT

filename

Name of the recording file to import. If a relative path is given, a file of this name will be looked up in the standard data directories (/resources and Examples/).

• verbose name: Filename
• default value:
• port type: StringPort
• value type: str (can be None)

modality

Modality of the incoming data. This will be stored in the stream props as props[Origin.modality].

• verbose name: Modality
• default value: EEG
• port type: ComboPort
• value type: str (can be None)

data_stream_name

Name to be used for the emitted data stream. If left empty, this will be based on the modality ('markers' will be used automatically for the event marker stream if importing event markers together with signal data).

• verbose name: Data Stream Name
• default value:
• port type: StringPort
• value type: str (can be None)

quantity

The name describing the quantity of the data, i.e ., voltage for EEG, intensity for NIRS, etc. This will be stored in the stream props as props[Metadata.quantity].

• verbose name: Quantity
• default value: voltage
• port type: ComboPort
• value type: str (can be None)

unit

Unit for the incoming data, if known. This will be stored in the stream props as props[Metadata.unit]. If unknown, leave blank and place the FixSignalUnit node after this one to detect and store the unit.

• verbose name: Unit
• default value:
• port type: ComboPort
• value type: str (can be None)

timestamp_column

The name or number (counting from 0) of the column which holds the timestamps.

• verbose name: Timestamp Column
• default value: None
• port type: Port
• value type: object (can be None)

timestamp_units

Unit of the timestamps. Values in the specified unit will be converted to seconds (used internally in Neuropype).

• verbose name: Timestamp Units
• default value: seconds
• port type: EnumPort
• value type: str (can be None)

marker_column

The name or number (counting from 0) of the column which holds the event markers.

• verbose name: Marker Column
• default value: None
• port type: Port
• value type: object (can be None)

instance_column_name

Name of a column which holds the instance payload data, if any.

• verbose name: Instance Column Name
• default value:
• port type: StringPort
• value type: str (can be None)

exclude_columns

List of columns which should be excluded. Can be names or numbers (counting from 0). Example: [2,7,8,'Subject ID', 'Gender'].

• verbose name: Exclude Columns
• default value: []
• port type: ListPort
• value type: list (can be None)

include_columns

List of columns which should be included. Can be names or numbers (counting from 0). If not given, all columns will be included.

• verbose name: Include Columns
• default value: []
• port type: ListPort
• value type: list (can be None)

no_header_row

If True, the first row of the CSV file will be treated as data and not as a header row.

• verbose name: No Header Row
• default value: False
• port type: BoolPort
• value type: bool (can be None)

ignore_empty_rows

Ignore empty rows on import.

• verbose name: Ignore Empty Rows
• default value: True
• port type: BoolPort
• value type: bool (can be None)

delimiter

A one-character string used to separate columns in the CSV file. Some files might use for example semicolons.

• verbose name: Delimiter
• default value: ,
• port type: StringPort
• value type: str (can be None)

dependent_variable_columns

Special case to import statistics related data instead of signal data. List of column variable names or numbers (counting from 0) that are dependent variables (DVs) and will become separate elements along a feature axis with their data values stored in the block array. This format is recommended for spreadsheet like data that would be used in statistical analyses in which there are a mix of independent variables (IVs) (e.g. subid, age, condition, etc.) and DVs (e.g. alpha power, heart rate, workload metric, etc.). Row data will become instance axis elements with the remaining columns (IVs) each set as fields (attributes) within the instance axis. This is similar to a dataframe with the ability to set instance axis fields (e.g. condition) as test factors in statistics nodes (e.g. T-Test, Z-Test, etc.). By default (an empty list) all columns will be treated as DVs as separate elements along a feature axis and their data values will be stored as string types in the data array. However, if this list is non-empty, but none of the variable names or numbers are found in the CSV headers, all columns will be treated as IVs on an instance axis and no feature axis will exist.

• verbose name: Dependent Variable Columns
• default value: []
• port type: ListPort
• value type: list (can be None)

nondata_rows

Number of rows at the top of the CSV file which do not contain data and should be ignored on import (i.e ., comments or metadata related to the signal data). The contents of those rows will be preserved in the props of the signal stream, under the key csv_metadata. (These can be extracted using the GetStreamProperties or GetNested nodes.) These will be removed before a header row is detected. (So if this is set to 6, it is assumed that the 7th row will be the header row unless no_header_row is set to True.)

• verbose name: Nondata Rows
• default value: 0
• port type: IntPort
• value type: int (can be None)

rows_header_column

Name of a column which holds the row headers, for non-signal files without a time axis. This column will be removed from the Feature axis (containing the other columns) and added to the Instance axis with an axis array having the same name as this column.

• verbose name: Rows Header Column
• default value:
• port type: StringPort
• value type: str (can be None)

timestamp_column_name

Name of a column which holds the timestamps. (Deprecated. Use timestamp_column instead.)

• verbose name: Timestamp Column Name
• default value:
• port type: StringPort
• value type: str (can be None)

marker_column_name

Name of a column which holds the event marker names.

• verbose name: Marker Column Name
• default value:
• port type: StringPort
• value type: str (can be None)

emit_each_tick

Emit data on every tick.

• verbose name: Emit Each Tick
• default value: False
• port type: BoolPort
• value type: bool (can be None)

file_missing

Whether to raise an exception (error) if the file is missing or not found (which may stop the pipeline), or print a log message and continue.

• verbose name: File Missing
• default value: raise
• port type: EnumPort
• value type: str (can be None)

set_breakpoint

Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

• verbose name: Set Breakpoint (Debug Only)
• default value: False
• port type: BoolPort
• value type: bool (can be None)

metadata

User-definable meta-data associated with the node. Usually reserved for technical purposes.

• verbose name: Metadata
• default value: {}
• port type: DictPort
• value type: dict (can be None)

cloud_host

Cloud storage host to use (if any). You can override this option to select from what kind of cloud storage service data should be downloaded. On some environments (e.g., on NeuroScale), the value Default will be map to the default storage provider on that environment.

• verbose name: Cloud Host
• default value: Default
• port type: EnumPort
• value type: str (can be None)

cloud_account

Cloud account name on storage provider (use default if omitted). You can override this to choose a non-default account name for some storage provider (e.g., Azure or S3.). On some environments (e.g., on NeuroScale), this value will be default-initialized to your account.

• verbose name: Cloud Account
• default value:
• port type: StringPort
• value type: str (can be None)

cloud_bucket

Cloud bucket to read from (use default if omitted). This is the bucket or container on the cloud storage provider that the file would be written to. On some environments (e.g., on NeuroScale), this value will be default-initialized to a bucket that has been created for you.

• verbose name: Cloud Bucket
• default value:
• port type: StringPort
• value type: str (can be None)

cloud_credentials

Secure credential to access cloud data (use default if omitted). These are the security credentials (e.g., password or access token) for the the cloud storage provider. On some environments (e.g., on NeuroScale), this value will be default-initialized to the right credentials for you.

• verbose name: Cloud Credentials
• default value:
• port type: StringPort
• value type: str (can be None)