CustomStep¶
A custom optimizer step that is defined by a graph applied to gradients and, optionally, current weights.
The graph must have either one or two placeholders (one for the gradients and one for the current weights), which must be listed in the graph's positional signature in that order. You can then implement any computation that processes and returns the updated gradients and, if you are building an end-to-end optimizer step, also converts the gradients into additive updates (by negating them), since the result of end-to-end steps is expected to be applied additively. Your graph may use any of the Step nodes (e.g., AdamStep, GradientClippinStep, etc.) or stateless operations (e.g., Add etc). If you are optimizing parameters that are in the form of a Packet (by setting prefer_packets to True in NetTransform), you can also use stateless nodes that operate on Packets such as ExtractStreams, MergeStreams, and so forth, which is useful when processing subsets of the parameters (e.g., corresponding to different layers) differently. There is a small set of stateful nodes that can also be used safely here; these are the nodes that have a "state" input and output port. Your graph may also expose additional hyper- parameters (e.g., the weight decay parameters), which can be defined using ParameterPort nodes and which need to have default values. These parameters can be overridden by passing name/value pairs into Custom Step. Version 0.2.0
Ports/Properties¶
gradients¶
Gradients to be transformed.
- verbose name: Gradients
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: INOUT
weights¶
Optional current weights.
- verbose name: Weights
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: IN
state¶
Explicit state of the node.
- verbose name: State
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: INOUT
graph¶
Optimizer step returning additive updates.
- verbose name: Graph
- default value: None
- port type: GraphPort
- value type: Graph
graph__signature¶
Signature for the "graph" input. This represents the signature for the subgraph that is wired into the "graph" port. This is formatted as in (a,b,c) where a,b,c are names of placeholders that are expected in the subgraph that goes into the "graph" port. Alternatively, it can also be provided in data structure form as a list of lists, as in: [['a','b','c']].
- verbose name: Graph [Signature]
- default value: (gradients)
- port type: Port
- value type: object (can be None)
name1¶
Override 1.
- verbose name: Name1
- default value: None
- port type: StringPort
- value type: str (can be None)
val1¶
Value 1.
- verbose name: Val1
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: IN
name2¶
Override 2.
- verbose name: Name2
- default value: None
- port type: StringPort
- value type: str (can be None)
val2¶
Value 2.
- verbose name: Val2
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: IN
name3¶
Override 3.
- verbose name: Name3
- default value: None
- port type: StringPort
- value type: str (can be None)
val3¶
Value 3.
- verbose name: Val3
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: IN
name4¶
Override 4.
- verbose name: Name4
- default value: None
- port type: StringPort
- value type: str (can be None)
val4¶
Value 4.
- verbose name: Val4
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: IN
name5¶
Override 5.
- verbose name: Name5
- default value: None
- port type: StringPort
- value type: str (can be None)
val5¶
Value 5.
- verbose name: Val5
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: IN
name6¶
Override 6.
- verbose name: Name6
- default value: None
- port type: StringPort
- value type: str (can be None)
val6¶
Value 6.
- verbose name: Val6
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: IN
name7¶
Override 7.
- verbose name: Name7
- default value: None
- port type: StringPort
- value type: str (can be None)
val7¶
Value 7.
- verbose name: Val7
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: IN
name8¶
Override 8.
- verbose name: Name8
- default value: None
- port type: StringPort
- value type: str (can be None)
val8¶
Value 8.
- verbose name: Val8
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: IN
name9¶
Override 9.
- verbose name: Name9
- default value: None
- port type: StringPort
- value type: str (can be None)
val9¶
Value 9.
- verbose name: Val9
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: IN
nameN¶
Additional overridden names.. .
- verbose name: Namen
- default value: None
- port type: ListPort
- value type: list (can be None)
valN¶
Additional overridden values.. .
- verbose name: Valn
- default value: None
- port type: DataPort
- value type: list (can be None)
- data direction: IN
name0¶
Override 0.
- verbose name: Name0
- default value: None
- port type: StringPort
- value type: str (can be None)
val0¶
Value 0.
- verbose name: Val0
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: IN
set_breakpoint¶
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.
- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
metadata¶
User-definable meta-data associated with the node. Usually reserved for technical purposes.
- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)