peal.training.criterions.orthogonality_criterion

peal.training.criterions.orthogonality_criterion(model, pred, y, latent_code=None)

Penalise non-orthogonal weight matrices of model.

Every parameter with two or more dimensions is reshaped to [out, in] (conv kernels are flattened over in * kh * kw) and the Frobenius norm of W W^T - I is summed. Vectors (biases, norm weights) are skipped. pred, y and latent_code are ignored.

Returns:

Scalar penalty on the model’s device.

Return type:

torch.Tensor