Skip to content

← deep_learning package

Activation

Apply an elementwise non-linear neural network activation function to the given data.

This node supports a number of commonly-used activation functions; which one performs best on the given data may have to be determined by experimentation. The most traditional functions are sigmoid and tanh, and the former is commonly used in the output layer of a network to produce a probability distribution over the possible classes. The relu (rectified linear unit) function is extremely compute efficient (it is max(0,x)) and can work quite well in hidden layers, but it has a number of potential shortcomings that limit its use as a general-purpose activation function (some of these are partially addressed in functions like leaky_relu and relu6). Some of the more recently developed activation functions are designed to work well with deep networks and are everywhere differentiable unlike relu; these include silu (aka swish), celu, selu, gelu, and softplus, and one can generally not go wrong with any of these. The softsign and elu functions are negative for negative inputs, which can have interesting effects on the network's learning behavior. The hard_* functions are simple (3-part) piecewise linear approximations of the activation functions they model. The 'linear' option is a pass-through to bypass the activation function. A few functions have a tunable parameter, which can be specified via the alpha parameter. More Info... Version 0.2.0

Ports/Properties

data

Data to process.

  • verbose name: Data
  • default value: None
  • port type: DataPort
  • value type: AnyNumeric (can be None)
  • data direction: INOUT

activation

Activation function to apply. See https://jax.readthedocs.io/en/latest/jax.nn.html#activation-functions for details.

  • verbose name: Activation Function
  • default value: relu
  • port type: EnumPort
  • value type: str (can be None)

alpha

Alpha value for elu, leaky_relu, and celu.

  • verbose name: Alpha (Elu, Leaky_relu, Celu)
  • default value: None
  • port type: FloatPort
  • value type: float (can be None)

set_breakpoint

Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

  • verbose name: Set Breakpoint (Debug Only)
  • default value: False
  • port type: BoolPort
  • value type: bool (can be None)

metadata

User-definable meta-data associated with the node. Usually reserved for technical purposes.

  • verbose name: Metadata
  • default value: {}
  • port type: DictPort
  • value type: dict (can be None)