tensorplay.nn.init.calculate_gain

tensorplay.nn.init.calculate_gain(nonlinearity, param=None)[source]

Return the recommended gain value for the given nonlinearity.

Supported names: linear/conv{1,2,3}d/conv_transpose{1,2,3}d and sigmoid map to 1, tanh to 5/3, relu to sqrt(2), selu to 3/4, and leaky_relu to sqrt(2 / (1 + negative_slope**2)) where the slope is taken from param (default 0.01).