Question

What are the hyperparameters that can be optimized for the batch normalization layer?

Answer 1

The

 and

 hyperparameters for the batch normalization layer are learned end to end by the network. In batch-normalization, the outputs of the intermediate layers are normalized to have a mean of 0 and standard deviation of 1. Rescaling by

 and shifting by

 helps us change the mean and standard deviation to other values.