peal.global_utils.replace_relu_with_leakysoftplus

peal.global_utils.replace_relu_with_leakysoftplus(model)

Replace every ReLU in the module tree by a LeakySoftplus.

Parameters:

model (torch.nn.Module) – Model modified in place.

Returns:

The same model, for convenience.

Return type:

torch.nn.Module

Notes

Functional torch.relu calls inside a forward are not touched, only ReLU submodules.