Skip to content

← deep_learning package

LongShortTermMemoryLayer

A long short-term memory (LSTM) recurrent layer, based on Hochreiter and Schmidhuber (1997).

This is a recurrent layer that conceptually retains internal network activations across successive time steps when the node is used to process time series data (see below for more details). In contrast to earlier recurrent cores, LSTMs have an improved ability to learn long-term dependencies using a gating mechanism that can selectively retain or forget past information. This implementation follows Zaremba et al. 2015, which is a minor modification that reduces forgetting during initial training. Like all recurrent layers, the node does not usually store and carry over these activations, but depends on either a special loop node (the RecurrentLoop node, see docs for more details) to step aross the time axis of some given data array, or requires that the user manually passes in and retrieves that carry state via the carry in/out port. Also like all recurrent nodes, this node will move the instance axis of the input first, optionally retain any axes listed in the parallel_axes port, and and flatten any other axes into a single feature axis at the end. When managing the carry state manually, the state for this node can be obtained via get_initial() or be constructed as either an array or packet (depending on the data that it is used with), where the first axis is of size 2, the last axis is of size units, and the middle axis is sized to be the product of the instance axes and other parallel axes of the data. More Info... Version 0.2.0

Ports/Properties

data

Data to process at current time step.

  • verbose name: Data
  • default value: None
  • port type: DataPort
  • value type: AnyNumeric (can be None)
  • data direction: INOUT

carry

Carried-over activations from previous time step.

  • verbose name: Carry
  • default value: None
  • port type: DataPort
  • value type: AnyNumeric (can be None)
  • data direction: INOUT

units

Number of hidden units.

  • verbose name: Hidden Units
  • default value: 256
  • port type: IntPort
  • value type: int (can be None)

layername

Name of the layer. Used for naming of weights.

  • verbose name: Layer Name
  • default value: lstm
  • port type: StringPort
  • value type: str (can be None)

set_breakpoint

Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

  • verbose name: Set Breakpoint (Debug Only)
  • default value: False
  • port type: BoolPort
  • value type: bool (can be None)

metadata

User-definable meta-data associated with the node. Usually reserved for technical purposes.

  • verbose name: Metadata
  • default value: {}
  • port type: DictPort
  • value type: dict (can be None)

parallel_axes

Optionally an axis or comma-separated list of axes that shall be processed in parallel by the layer, using the same learned weights. This is useful if you have multiple parallel processes that you wish to process separately but you assume that they are all ultimately governed by the same rules, you can learn a single kernel for all of them. Note that, for packet data, the instance axis (if present) and for plain-array data the first axis is always treated as a parallel axis, so you don't need to list it here.

  • verbose name: Parallel Axes
  • default value:
  • port type: ComboPort
  • value type: str (can be None)