Skip to content

← deep_learning package

NetWeightArray

Define network weight array.

This node can only occur inside the network graph wired into a Define Net (NetDefine) node. The node is used to request a weight array whose value can be used as part of a neural network computation. The weight array is implicit in the containing subnet, and is optimized during training, as in the built-in NN nodes (e.g., convolution, etc). As with all implicit parameters, if you want to optimize them manually (i.e., using the Calculate Gradient node) rather than using one of the higher-level nodes such as "Deep Model" (DeepModel), you first need to transform your subnet (or rather a computational graph in which it is materialized) into functional style, which is done using the NetTransform node. Version 0.2.0

Ports/Properties

w_init

Initializer for the weights.

  • verbose name: W Init
  • default value: None
  • port type: DataPort
  • value type: BaseNode (can be None)
  • data direction: IN

w_prior

Optional prior distribution for the weights.

  • verbose name: W Prior
  • default value: None
  • port type: DataPort
  • value type: Distribution (can be None)
  • data direction: IN

value

Value of the weights.

  • verbose name: Value
  • default value: None
  • port type: DataPort
  • value type: object (can be None)
  • data direction: OUT

name

Name of the weight array. Must be unique within the subnet where it is used.

  • verbose name: Name
  • default value: weights
  • port type: StringPort
  • value type: str (can be None)

shape

Shape (dimensions) of the weight array.

  • verbose name: Shape
  • default value: [1]
  • port type: ListPort
  • value type: list (can be None)

dtype

Data type of the weight array. Float32 is the easiest to use, but the 16-bit data types can be more efficient -- however, float16 has limited range, and can cause numeric problems, which typically manifest as the network failing to train. Some hardware supports bfloat16, which is as efficient as float16, but has a wider range and is therefore an easier drop-in replacement for float32 when it is supported.

  • verbose name: Data Type
  • default value: float32
  • port type: EnumPort
  • value type: str (can be None)

initializer

Choice of initializer. This can either be one of the named initializers, or the value "custom", in which case the initializer must be wired into the the initializer port. For some initializers that take arguments, you can also specify these positionally as in "truncated_normal(1.0,0.0)" (note order of stddev, mean).

  • verbose name: Initializer
  • default value: lecun_normal
  • port type: ComboPort
  • value type: str (can be None)

set_breakpoint

Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

  • verbose name: Set Breakpoint (Debug Only)
  • default value: False
  • port type: BoolPort
  • value type: bool (can be None)

metadata

User-definable meta-data associated with the node. Usually reserved for technical purposes.

  • verbose name: Metadata
  • default value: {}
  • port type: DictPort
  • value type: dict (can be None)