← file_system package

ExportStructure

Save a dictionary to disk.

This node is typically used after CreateStructure which creates a dictionary that contains data of different types (i.e., Packets, numpy arrays, etc.). It is intended primarily for saving data that will be reloaded back into NeuroPype, such as a machine learning model or calibration data. (Example, you can wire the model port of a machine learning node, such as LinearDiscriminantAnalysis, to CreateStructure, and then wire the output of CreateStructure to this node to save the model to disk.) Use the ImportStructure node to load the data back into NeuroPype. If you export a dictionary containing NeuroPype data types (i.e., Packets), or numpy arrays and want to access it outside of NeuroPype (i.e., using python directly), you will need to import NeuroPype as a library to access any Packets in the data structure. This node can also be used to save a dictionary to JSON, YAML or Msgpack instead of Pickle, though not all data types are supported. (Packets are supported by JSON, but pickle is a much faster format for saving and loading complex data structures.) For YAML, only standard data types (lists, dictionaries, etc.) are supported (i.e., no numpy arrays or Packets). Note that pickle is very flexible format used by Python which can also be unsafe (e.g., one should not open untrusted pickle files from the internet as these can contain viruses). Therefore if sharing data publicly, you should use JSON, not pickle. Another good lossless option for signal data (i.e., EEG, fNIRS) is to export to HDF using the ExportH5 node. Version 1.2.2

Ports/Properties

data

Data to save.

• verbose name: Data
• default value: None
• port type: DataPort
• value type: dict (can be None)
• data direction: IN

done

Flag to indicate that the node is finished processing if wired into the PipelineDone node.

• verbose name: Done
• default value: False
• port type: DataPort
• value type: bool (can be None)
• data direction: OUT

filename

File name to export data to. Can be the full path or a relative path or filename if `output_root is specified.

• verbose name: Filename
• default value: untitled.pkl
• port type: StringPort
• value type: str (can be None)

output_root

Path to the output folder; if specified, the filename will be relative to this folder. Alternatively, this can be left empty and the full path may be specified in filename).

• verbose name: Output Root
• default value:
• port type: StringPort
• value type: str (can be None)

file_exists

What to do if a file with the name specified in filename (and located in output_root) already exists. If skip, this node will perform no export. If error, the pipeline execution will stop. If rename, the original existing file will be renamed with "_n" added to the filename where _n is an incremental number (i.e. _1), and a new file will be written with the specified filename. Note that this option only currently works on a local filesystem.

• verbose name: File Exists
• default value: overwrite
• port type: EnumPort
• value type: str (can be None)

encoding

Format to use for saving the file. Pickle is the default format and recommended for saving complex data structures such as Packets.

• verbose name: Encoding
• default value: pickle
• port type: EnumPort
• value type: str (can be None)

allow_pickle_fallback

If encoding is set to json or msgpack, fall back to pickle for objects that are not JSON or msgpack-serializable.

• verbose name: Allow Pickle Fallback
• default value: True
• port type: BoolPort
• value type: bool (can be None)

save_if_empty

Save empty dictionaries. If false, incoming empty dictionaries will be ignored.

• verbose name: Save If Empty
• default value: False
• port type: BoolPort
• value type: bool (can be None)

protocol_ver

Pickle protocol version. This is the internal protocol version to use. This should not be changed unless you will need to open the exported files with an older Neuropype release (pre 2023.0.0) or an application that uses an older version of python (pre python 3.8).

• verbose name: Protocol Ver
• default value: 5
• port type: IntPort
• value type: int (can be None)

set_breakpoint

Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

• verbose name: Set Breakpoint (Debug Only)
• default value: False
• port type: BoolPort
• value type: bool (can be None)

metadata

User-definable meta-data associated with the node. Usually reserved for technical purposes.

• verbose name: Metadata
• default value: {}
• port type: DictPort
• value type: dict (can be None)

cloud_host

Cloud storage host to use (if any). You can override this option to select from what kind of cloud storage service data should be downloaded. On some environments (e.g., on NeuroScale), the value Default will be map to the default storage provider on that environment.

• verbose name: Cloud Host
• default value: Default
• port type: EnumPort
• value type: str (can be None)

cloud_account

Cloud account name on storage provider (use default if omitted). You can override this to choose a non-default account name for some storage provider (e.g., Azure or S3.). On some environments (e.g., on NeuroScale), this value will be default-initialized to your account.

• verbose name: Cloud Account
• default value:
• port type: StringPort
• value type: str (can be None)

cloud_bucket

Cloud bucket to read from (use default if omitted). This is the bucket or container on the cloud storage provider that the file would be written to. On some environments (e.g., on NeuroScale), this value will be default-initialized to a bucket that has been created for you.

• verbose name: Cloud Bucket
• default value:
• port type: StringPort
• value type: str (can be None)

cloud_credentials

Secure credential to access cloud data (use default if omitted). These are the security credentials (e.g., password or access token) for the the cloud storage provider. On some environments (e.g., on NeuroScale), this value will be default-initialized to the right credentials for you.

• verbose name: Cloud Credentials
• default value:
• port type: StringPort
• value type: str (can be None)