Skip to content

← optimization package

LinearSolve

Solve a linear system of equations of the form Ax=b.

This node currently always uses the jax backend, but will convert the solution back to the original backend of b if necessary. The matrix A can either be wired in directly (as an array) or alternatively as a small graph that applies A to a Placeholder named x, typically as a matrix-vector product, or using any of the specialized nodes that apply a linear operator (e.g. FFT, finite differences, etc.). The node implements the standard methods for this problem, which are largely equivalent to the SciPy or MATLAB(tm) implementations; note that the best choice depends on the desired accuracy, speed, and properties of the operator A (size, symmetry/positive definiteness, and conditioning); see the docstring of the method property for details. Like with all solve nodes, the solution is differentiable, meaning that the node can be used in a place where gradients are taken, for example in the network graph a DeepModel node. More Info... Version 0.5.0

Ports/Properties

Ax

Product of a matrix (or linear operator) A and vector x.

  • verbose name: Ax
  • default value: None
  • port type: GraphPort
  • value type: Graph

Ax__signature

Signature for the "Ax" input. This represents the signature for the subgraph that is wired into the "Ax" port. This is formatted as in (a,b,c) where a,b,c are names of placeholders that are expected in the subgraph that goes into the "Ax" port. Alternatively, it can also be provided in data structure form as a list of lists, as in: [['a','b','c']].

  • verbose name: Ax [Signature]
  • default value: (x)
  • port type: Port
  • value type: object (can be None)

A

Alternative direct specification of A.

  • verbose name: A
  • default value: None
  • port type: DataPort
  • value type: AnyNumeric (can be None)
  • data direction: IN

b

Coefficient vector b.

  • verbose name: B
  • default value: None
  • port type: DataPort
  • value type: AnyNumeric (can be None)
  • data direction: IN

x

Initial and final solution. If not provided, x will be initialized to a all-zeros. Ignored by the direct methods.

  • verbose name: X
  • default value: None
  • port type: DataPort
  • value type: AnyNumeric (can be None)
  • data direction: INOUT

P

Optional preconditioning matrix P. Ignored by the direct methods.

  • verbose name: P
  • default value: None
  • port type: DataPort
  • value type: AnyNumeric (can be None)
  • data direction: IN

method

Method to use for solving the system. The lu and cholesky methods are "direct" solvers, which materialize the matrix A in memory, and these are the fastest methods when A is small or when you need close to machine precision. The other methods are "indirect" solvers, which only need access to the matrix-vector product operation (the graph wired into the Ax port); this is more efficient if A is a large linear operator or has an more efficient implementation than dense matrix multiplication (examples are A being a Fourier transform, finite differences, or a large sparse matrix). These solvers may also not solve to the maximum precision. The solvers ending in (posdef) require A furthermore to be Hermitian and positive definite, for example this applies to covariance matrices. Of the available methods, cg (posdef) is the standard conjugate gradient method; note that for badly conditioned A, it may be necessary to precondition the linear system by pre-multiplying A and b by a skillfully chosen preconditioning matrix for best performance. The cg (normal) method instead solves the normal equation A'Ax = A'b, but note that this is not the most stable solver for badly conditioned matrices, and you may need to use gmres or bicgstab (also, you must specify an initial x if A is non-square since the shape is otherwise unknown). gmres is the generalized minimal residual method, which is robust but tends to be slow for near-symmetric or symmetric A. bicgstab is a hybrid method that alternates between an update equivalent to cg (normal) and an update equivalent to gmres, and is a good middle ground in terms of speed and robustness. lu is the LU decomposition method, which, is the most efficient choice for small A. Note that LU will ignore both the initial value provided for x, and the regularization parameter. cholesky (posdef) is the Cholesky decomposition method, which is the fastest method if A is small and positive definite.

  • verbose name: Method
  • default value: lu
  • port type: EnumPort
  • value type: str (can be None)

regu

Ridge regularization parameter. This is a scalar value that will be added to the diagonal of the matrix A before solving the system, which can help to stabilize the solution if A is ill-conditioned. This is ignored by the LU method.

  • verbose name: Regularization (Ridge)
  • default value: 0.0
  • port type: FloatPort
  • value type: float (can be None)

reltol

Relative convergence tolerance for the indirect solvers. The direct solvers lu and cholesky will ignore this parameter and always solve to approx. machine precision.

  • verbose name: Convergence Tolerance (Rel.)
  • default value: 1e-05
  • port type: FloatPort
  • value type: float (can be None)

max_iter

Maximum number of iterations. This need not be specified in most cases, since all solvers have a well-defined termination criterion. However, if reltol is too stringent, or if a bounded runtime is needed (e.g. for real-time use), this can be used to limit the number of iterations.

  • verbose name: Max. Iterations
  • default value: None
  • port type: IntPort
  • value type: int (can be None)

set_breakpoint

Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

  • verbose name: Set Breakpoint (Debug Only)
  • default value: False
  • port type: BoolPort
  • value type: bool (can be None)

metadata

User-definable meta-data associated with the node. Usually reserved for technical purposes.

  • verbose name: Metadata
  • default value: {}
  • port type: DictPort
  • value type: dict (can be None)