How to determine the number of hidden layers, the filter size, stride size, and other convolutional neural network parameters?
Parameters of a Convolutional Neural Network (CNN parameters) for image classification:
The Number layers – depends on complexity of your dataset, in general the deeper networks is better, but this will cost you more in computations and amount of data required.
Reading on number of nodes per hidden layer, and max pooling window size: http://stackoverflow.com/questions/24509921/how-do-you-decide-the-parameters-of-a-convolutional-neural-network-for-image-cla
Number of neurons – depends on the complexity of the objects you are trying to recognize.
Stride size in CNN – depends both on properties of input data and your network and filter sizes: https://www.quora.com/How-does-one-determine-stride-size-in-CNN-filters (Quora: How does one determine stride size in CNN filters?)
Learning rate – in general, small step sizes are safe. https://www.quora.com/In-deep-learning-why-dont-we-use-the-whole-training-set-to-compute-the-gradient (Quora: In deep learning, why don’t we use the whole training set to compute the gradient?)
Reading materials on Convolutional neural network parameters:
- Systematic evaluation of CNN advances on the ImageNet [PDF] [GitHub]
- Efficient backpropagation: [PDF]
- Very Deep Convolution Networks for large-scale image recognition: https://arxiv.org/pdf/1409.1556.pdf (PDF)
This paper includes the Comparison of VGG with the state of the art in ILSVRC classification. - Related chapter in Deep Learning book: http://www.deeplearningbook.org/contents/numerical.html [PDF]