Sigmoid neurons: http://neuralnetworksanddeeplearning.com/chap1.html#sigmoid_neurons
Fast sigmoid algorithm: http://stackoverflow.com/questions/10732027/fast-sigmoid-algorithm
(*fun)(xs[i]); atan(pi*x/2)*2/pi 24.1 ns atan(x) 23.0 ns 1/(1+exp(-x)) 20.4 ns 1/sqrt(1+x^2) 13.4 ns erf(sqrt(pi)*x/2) 6.7 ns tanh(x) 5.5 ns x/(1+|x|) 5.5 ns
xs[i] = (*fun)(xs[i]); atan(pi*x/2)*2/pi 9.2 ns atan(x) 7.9 ns 1/(1+exp(-x)) 19.3 ns 1/sqrt(1+x^2) 10.8 ns erf(sqrt(pi)*x/2) 6.5 ns tanh(x) 15.4 ns x/(1+|x|) 4.4 ns
xs[i] = (*fun)(xs[i]) + 0.5; atan(pi*x/2)*2/pi 20.5 ns atan(x) 19.4 ns 1/(1+exp(-x)) 19.3 ns 1/sqrt(1+x^2) 10.8 ns erf(sqrt(pi)*x/2) 49.9 ns tanh(x) 28.1 ns x/(1+|x|) 2.5 ns
Source: https://gist.github.com/astanin/5270668 (‘Benchmark various sigmoid functions’ by https://github.com/astanin)