Clip layer - resurrection

[Noiredd]

#3791 introduced a clip layer to constrain activations to some predefined range of values. It had some problems with unit tests however:

sometimes an input value will land near the function's discontinuity (...) [which] causes the gradient estimate to be different than the computed value

This is a resubmission of that PR, with modified gradient tests: values that fall within the discontinuity regions are excluded from the blob before the GradientChecker runs.

Closes #3791

[shelhamer]

@Noiredd for testing you could consider the kink, kink_range arguments of GradientChecker as used in the absolute value gradient test for instance https://github.com/BVLC/caffe/blob/master/src/caffe/test/test_neuron_layer.cpp#L205. Note however that this can only handle a single non-smooth region, so you would have to test the lower and upper extremes of the clipping separately, which is less than ideal.

Of course your test handles everything just fine by handling the clipping boundaries itself. I thought I would mention the kink setting of the gradient checker in case it is useful elsewhere since it is not often invoked.

[Noiredd]

Oooh, so this is what kink and kink_range do. Thanks for the hint, I'll pay attention to them next time.