Skip to content

← file_system package

PathIterator

Iterate over a list of path names.

This node accepts a list of path names, possibly given as a wildcard expression like /parent/child_*, or as a comma-separated list of paths, or as the name of a study manifest file. On each successive update this node will then output one pathname at a time, until the list is exhausted. Using a comma-separated list of wildcard expressions is not supported. Terminate the search string with a / to indicate that the returned results should be paths only; Terminate with *.* (or *.ext) to indicate the results should be files only. Instead of a *, a "capture name" can be assigned in curly braces, e.g., {Subject}. You can also use ** to include matching paths from all subfolders (i.e. myfolder/**/*.xdf will include xdf files in all subfolders whereas myfolder/*.xdf will not). The current pathname emitted by this node can then be wired into a subsequent node as the current path/file, and thereby multiple files can be imported, processed, and then exported in succession. Using capture names allows extracting meta-data from the rwa file path, which is returned by the path iterator in its curmeta output. Such meta-data can, for example, be attached to segments extracted from the data in later processing, using the Set Instance Details node. This node also has an allpaths port which outputs a list of all matching paths at once. This can be wired into a node to process all filenames at once (i.e., create a table of paths), or into a ForEach node to loop over part of a pipeline for each file. Likewise, there is also an allmeta port that has a list of all metadata dictionaries, including a convenience 'FilePath' entry. (Note that you can also wire the this port of PathIterator into a ForEach node, which will, with each loop, emit the next item from PathIterator and pass it to the ForEach node which will then execute the loop for that item.) Version 1.2.5

Ports/Properties

curpath

Pathname for the current iteration. With each tick of the pipeline, this will be set to the next path that was found. When all paths are processed this will be set to None.

  • verbose name: Curpath
  • default value: None
  • port type: DataPort
  • value type: str (can be None)
  • data direction: OUT

curmeta

Metadata for the file of the current iteration, in dictionary format. This can include data from capture clauses in the path specification, BIDS metadata, and optionally the file path itself (controlled by the include_filepath parameter). This can be wired into the SetInstanceDetails node to set the instance fields of segmented trials with metadata from the current file. Will be None if no metadata is available and include_filepath is False.

  • verbose name: Curmeta
  • default value: None
  • port type: DataPort
  • value type: dict (can be None)
  • data direction: OUT

allpaths

A list of all paths that were found and will be iterated over.

  • verbose name: Allpaths
  • default value: None
  • port type: DataPort
  • value type: list (can be None)
  • data direction: OUT

allmeta

A list of all metadata dictionaries for the paths that were found. Each entry corresponds to one path in allpaths and follows the same structure as curmeta. Entries may be None if no metadata is available and include_filepath is False.

  • verbose name: Allmeta
  • default value: None
  • port type: DataPort
  • value type: list (can be None)
  • data direction: OUT

paths

Paths to iterate over. This can be a wildcard ("glob") expression, such as /myfolder/, or point to a study manifest file (e.g., for an ESS study, or top-level BIDS json or tsv file), or be comma-separated list of paths, or can be a Neuroinformatics query. Also, instead of a , a "capture name" in curly braces can be given, e.g., {Session}. This will match the same as a *, but the resulting content will be returned in the secondary output of the path iterator, under curmeta, in the form of a dictionary holding the matched values for all used capture names.

  • verbose name: Paths To Iterate Over
  • default value:
  • port type: StringPort
  • value type: str (can be None)

filter

Filter conditions. This is an optional expression of filter conditions that can be used to narrow down the files emitted by this node. The conditions are given in Python syntax, can use any meta-data properties as if they were Python variables, and should evaluate to True. Example: age>42 and gender=='male'. For the list of Python constructs allowed, see NeuroPype documentation of its sandboxed Python expression grammar.

  • verbose name: Filter
  • default value:
  • port type: StringPort
  • value type: str (can be None)

sort_by

Sort the located paths (based on the paths and filter parameters), in alphabetical (default), chronological, or given order. Alphabetical sorting is case insensitive. If chronological is selected, the timestamp in each filename, not the date created or modified on the computer system, will be used. Only filenames containing a timestamp in the following format: YYYY-MM-DD_HH-MM-SS will be sorted (others will still be processed but not sorted). If order is selected and a comma separate list of files is given this method will keep the same order of the list.

  • verbose name: Sort By
  • default value: alphabetical
  • port type: EnumPort
  • value type: str (can be None)

iter_reorder

Iterate over items in a re-ordered fashion. Using 0 is forward, 1 is reverse, 2 is from-the-middle-out, 3 is out from the center left, and so forth. This will be applied whether "sort by" is alphabetical or chronological.

  • verbose name: Iteration Reordering
  • default value: 0
  • port type: IntPort
  • value type: int (can be None)

include_filepath

Include file path in metadata. When enabled, the curmeta and allmeta outputs will include a 'FilePath' entry containing the full path to the file. When disabled, only metadata from capture clauses and other sources will be included.

  • verbose name: Include Filepath In Metadata
  • default value: True
  • port type: BoolPort
  • value type: bool (can be None)

cloud_host

Cloud storage host to use (if any). You can override this option to select from what kind of cloud storage service data should be downloaded. On some environments (e.g., on NeuroScale), the value Default will be map to the default storage provider on that environment.

  • verbose name: Cloud Host
  • default value: Default
  • port type: EnumPort
  • value type: str (can be None)

cloud_account

Cloud account name on storage provider (use default if omitted). You can override this to choose a non-default account name for some storage provider (e.g., Azure or S3.). On some environments (e.g., on NeuroScale), this value will be default-initialized to your account.

  • verbose name: Cloud Account
  • default value:
  • port type: StringPort
  • value type: str (can be None)

cloud_bucket

Cloud bucket to read from (use default if omitted). This is the bucket or container on the cloud storage provider that the file would be written to. On some environments (e.g., on NeuroScale), this value will be default-initialized to a bucket that has been created for you.

  • verbose name: Cloud Bucket
  • default value:
  • port type: StringPort
  • value type: str (can be None)

cloud_credentials

Secure credential to access cloud data (use default if omitted). These are the security credentials (e.g., password or access token) for the the cloud storage provider. On some environments (e.g., on NeuroScale), this value will be default-initialized to the right credentials for you.

  • verbose name: Cloud Credentials
  • default value:
  • port type: StringPort
  • value type: str (can be None)

verbose

Verbose output.

  • verbose name: Verbose
  • default value: False
  • port type: BoolPort
  • value type: bool (can be None)

set_breakpoint

Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

  • verbose name: Set Breakpoint (Debug Only)
  • default value: False
  • port type: BoolPort
  • value type: bool (can be None)

metadata

User-definable meta-data associated with the node. Usually reserved for technical purposes.

  • verbose name: Metadata
  • default value: {}
  • port type: DictPort
  • value type: dict (can be None)