Skip to content

← deep_learning package

StepApply

Apply an optimizer step to given gradients, prior state, and optionally prior weights.

This node, along with "Init Step" (StepInit) provides a purely functional programming interface to the optimization steps. This is an alternative way of using steps: the simplest way of using a step is to wire data into a step node, and it will behave statefully like any other filter node (e.g., FIRFilter). However, when exactly replicating Python code that uses optax, you may need to follow the functional programming pattern where the state is explicitly passed around. One concrete difference is that providing data to a step node for the first time will first initialize it, and then also update it on the given data before returning it and the processed outputs, whereas "Init Step" will return just the initial state before the first update was applied. Note that the Call function, when given a graph that implements an optimizer step, can stand in for StepApply. The input state then is the graph itself, which is wired into the "function" input of Call, and the output state is the "snapshot" output of Call, which is the graph after it has processed the inputs. The initial state can likewise be obtained by using Call on some initial data (i.e., gradients), and keeping its "snapshot" output as the initial state. Both the StepApply and Call method can also be used with the "Gradient" node to differentiate the optimizer with respect to one or more of its hyper-parameters (e.g., learning rate) at some data point; for this, one would create a graph that accepts and initial state via a GraphPort and which returns a measure of the optimizer's performance along with the updated state (Call "snapshot" output). Version 0.2.0

Ports/Properties

step

Step to apply.

  • verbose name: Step
  • default value: None
  • port type: DataPort
  • value type: BaseNode (can be None)
  • data direction: IN

gradients

Gradients to be transformed.

  • verbose name: Gradients
  • default value: None
  • port type: DataPort
  • value type: object (can be None)
  • data direction: INOUT

weights

Optional current weights.

  • verbose name: Weights
  • default value: None
  • port type: DataPort
  • value type: object (can be None)
  • data direction: IN

state

Explicit state of the step.

  • verbose name: State
  • default value: None
  • port type: DataPort
  • value type: object (can be None)
  • data direction: INOUT

set_breakpoint

Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

  • verbose name: Set Breakpoint (Debug Only)
  • default value: False
  • port type: BoolPort
  • value type: bool (can be None)

metadata

User-definable meta-data associated with the node. Usually reserved for technical purposes.

  • verbose name: Metadata
  • default value: {}
  • port type: DictPort
  • value type: dict (can be None)