ClusteredPenalty¶
A penalty encouraging the patterns along individual rows of the solution matrix to condense into few similar stereotypical patterns; implemented as a proximal operator.
The assumption realized by this node is that a model may have sets of weights which are similar to each other, in the sense that only a few stereotypical weight patterns (cluster centroids) occur and each individual weight pattern deviates by a small amount from one of the centroids. This is most commonly used in a machine-learning context where a model is learned that has weights for multiple related tasks (e.g., for different users), and those weights are assumed to be similar to each other, also known as multi-task learning (MTL). The first axis parameter indicates the axis indexing the observations (here tasks) and the second axis specifies the features or variables across which the weight patterns are expressed. However, this node can also be used with tensor-shaped data, in which case each 2d slice through the data will independently satisfy the clustering assumption. The norm is convex and can be viewed as a convex relaxation of the k-means clustering problem; as such, the solution admits global optimality, but the solution is also not exactly classical clustering, but rather a convex surrogate. Note that there is a correspondence between this penalty and the low-rank penalty (LowRankProx), in that the low-rank penalty can be viewed as likewise concentrating the rows of the solution matrix towards latent profiles (cluster centroids), although with different characteristics. In practice, the low-rank penalty is the more commonly used one, while the cluster penalty could yield better performance when there are many observations (tasks) and relatively few features per cluster, and when the number of clusters is approximately known. The node may be used in conjunction with a technique called centered multi-task learning (CMTL), where a model consists of a shared component and a task-specific component that are summed together in the model formulation. In this case, the cluster penalty is applied to the task-specific component. Like all proximal operators, this is normally used as part of the optimization problem formulation given to a (typically convex) solver node. Version 1.0.0
Ports/Properties¶
data¶
Data to process.
- verbose name: Data
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: INOUT
step_size¶
Step size.
- verbose name: Step Size
- default value: None
- port type: DataPort
- value type: float (can be None)
- data direction: IN
step_count¶
Current step count for outer solver.
- verbose name: Step Count
- default value: None
- port type: DataPort
- value type: int (can be None)
- data direction: IN
axis1¶
Axis that represents the individual observations that are being clustered. Together with axis2, this specifies the 2d slices of the solution matrix along which the clustering penalty is applied. Additional dimensions in the data are treated independently. If the input is given as a packet, this can be an axis name,a comma-separated list of axis names (e.g., time,frequency), or an axis index (0, 1, etc.) or comma-separated list thereof. If the input is given as an array, only axis indices or lists thereof can be given. Lists mean that the given axes are first flattened, and then a profile is learned along that flattened axis, which is then combined with the profile along axis2. The effect on the original data is in all cases back-transformed to the original data shape.
- verbose name: Observations Axis
- default value: instance
- port type: ComboPort
- value type: str (can be None)
axis2¶
Axis that represents the feature-space dimensions in which the clusters exist. See the description of axis1 for more details.
- verbose name: Features Axis
- default value: feature
- port type: ComboPort
- value type: str (can be None)
clustering_strength¶
Degree of clustering that is encouraged in the solution. Larger values will cause solution rows to be more similar to some cluster centroids.
- verbose name: Clustering Strength
- default value: 1.0
- port type: FloatPort
- value type: float (can be None)
num_clusters¶
Assumed number of clusters to learn. The norm performs best if this is reasonably close to the true number of clusters, but the behavior degrades gracefully if this is inaccurate.
- verbose name: Num Clusters
- default value: 3
- port type: IntPort
- value type: int (can be None)
within_cluster_variance¶
Degree of variance that is allowed within each cluster. Larger values will allow more variance within each cluster, while smaller values will cause tighther clusters.
- verbose name: Within Cluster Variance
- default value: 0.02
- port type: FloatPort
- value type: float (can be None)
between_cluster_variance¶
Degree of variance that is allowed between clusters. Larger values will allow more variance (separation) between clusters, while smaller values will cause the centroids for individual clusters (and thus all weights) to move closter towards a common centroid. Along with the former parameter, this may be set empirically, and choosing this as 10x as large as the former parameter is a good starting point.
- verbose name: Between Cluster Variance
- default value: 0.2
- port type: FloatPort
- value type: float (can be None)
sanity_checks¶
Enable sanity checks. Not supported when using the jax backend.
- verbose name: Sanity Checks
- default value: False
- port type: BoolPort
- value type: bool (can be None)
set_breakpoint¶
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.
- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
metadata¶
User-definable meta-data associated with the node. Usually reserved for technical purposes.
- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)