DenseLayer¶
Dense neural network layer.
This is a fully-connected layer, where each input feature is connected to each output feature. Optionally includes a bias term. As with all built-in layers, you can override the initializer for the weights and/or the bias, which default to Lecun Normal and zero, respectively. More Info... Version 0.2.0
Ports/Properties¶
data¶
Data to process.
- verbose name: Data
- default value: None
- port type: DataPort
- value type: AnyNumeric (can be None)
- data direction: INOUT
w_init¶
Initializer for the weights.
- verbose name: W Init
- default value: None
- port type: DataPort
- value type: BaseNode (can be None)
- data direction: IN
b_init¶
Initializer for the bias.
- verbose name: B Init
- default value: None
- port type: DataPort
- value type: BaseNode (can be None)
- data direction: IN
w_prior¶
Optional prior distribution for the weights.
- verbose name: W Prior
- default value: None
- port type: DataPort
- value type: Distribution (can be None)
- data direction: IN
b_prior¶
Optional prior distribution for the bias.
- verbose name: B Prior
- default value: None
- port type: DataPort
- value type: Distribution (can be None)
- data direction: IN
units¶
Number of units (i.e . output features).
- verbose name: Units
- default value: 1
- port type: IntPort
- value type: int (can be None)
with_bias¶
Whether to include a bias term.
- verbose name: With Bias
- default value: True
- port type: BoolPort
- value type: bool (can be None)
w_initializer¶
Choice of weight initializer. This can either be one of the provided initializers, or the value "custom", in which case one of the Initializer nodes must be wired into the respective input port. For beginners it is recommended to stick to the defaults, since initialization of deep net layers is nuanced and can be tricky, otherwise be prepared to experiment with different choices. In general, the variance-scaling (lecun, glorot/xavier, he/kaiming) initializers are recommended, except for very simple/small layers where you may have a good default assumption as to the distribution of the weights (e.g., truncated_normal or uniform). Bias layers are typically zero-initialized. For initializers that take arguments, you can also type out the arguments positionally as in "truncated_normal(1.0,0.0)" (note reversed order of stddev, mean). The following initializers have arguments (here listed with their defaults): constant(value), those ending in normal(stddev=1, mean=0), those ending in uniform(min=0,max=1), orthogonal(scale=1,axis=-1), identity(gain=1), and variance_scaling(scale=1, "fan_in" (default)/"fan_avg"/"fan_out", "truncated_normal"(default)/"normal"/"uniform",optional-axis-indices=auto). Note that glorot and xavier are aliases for each other, and likewise he and kaiming are aliases for each other.
- verbose name: Weight Initializer
- default value: lecun_normal
- port type: ComboPort
- value type: str (can be None)
b_initializer¶
Choice of bias initializer. This can either be one of the provided initializers, or the value "custom", in which case one of the Initializer nodes must be wired into the respective input port. For beginners it is recommended to stick to the defaults, since initialization of deep net layers is nuanced and can be tricky, otherwise be prepared to experiment with different choices. In general, the variance-scaling (lecun, glorot/xavier, he/kaiming) initializers are recommended, except for very simple/small layers where you may have a good default assumption as to the distribution of the weights (e.g., truncated_normal or uniform). Bias layers are typically zero-initialized. For initializers that take arguments, you can also type out the arguments positionally as in "truncated_normal(1.0,0.0)" (note reversed order of stddev, mean). The following initializers have arguments (here listed with their defaults): constant(value), those ending in normal(stddev=1, mean=0), those ending in uniform(min=0,max=1), orthogonal(scale=1,axis=-1), identity(gain=1), and variance_scaling(scale=1, "fan_in" (default)/"fan_avg"/"fan_out", "truncated_normal"(default)/"normal"/"uniform",optional-axis-indices=auto). Note that glorot and xavier are aliases for each other, and likewise he and kaiming are aliases for each other.
- verbose name: Bias Initializer
- default value: zeros
- port type: ComboPort
- value type: str (can be None)
op_precision¶
Operation precision. This is a compute performance optimization. See jax documentation for details on these options. Note that this only applies to the operation, while the storage precision may be separately configurable depending on the node in question.
- verbose name: Operation Precision
- default value: default
- port type: EnumPort
- value type: str (can be None)
layername¶
Name of the layer. Used for naming of weights.
- verbose name: Layer Name
- default value: dense
- port type: StringPort
- value type: str (can be None)
set_breakpoint¶
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.
- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
metadata¶
User-definable meta-data associated with the node. Usually reserved for technical purposes.
- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)