What’s next to improve the model accuracy of Convolutional Neural Networks (CNNs)

Since the success of AlexNet in 2012, deep convolution neural networks have become the dominating approach for some areas of computer vision. Many recent studies are focusing on proposing new architecture to improve model accuracy. Most of the CNN architecture introduces new techniques from the model architecture level. He et al. [1] suggested that only changing model architecture is not responsible for improving model performance. Training procedures and Network tweaks also add value to gaining more model accuracy.