Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. ICLR; 2015, p. 1-14.

Zhou Q, Huang Z, Ding M, et al. Medical image classification using light-weight cnn with spiking cortical model based attention module. IEEE J Biomed Health Inform. 2023;27(4):1991–2002.

Article  Google Scholar 

Hafiz A, Bhat R, Hassaballah M. Image classification using convolutional neural network tree ensembles. Multimed Tools Appl. 2023;82(5):6867–84.

Article  Google Scholar 

Redmon J, Farhadi A. Yolo9000: better, faster, stronger. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2017. p. 7263–7271.

Farhadi A, Redmon J. Yolov3: An incremental improvement. In: Computer vision and pattern recognition. Berlin/Heidelberg, Germany: Springer; 2018. p. 1–6.

Bochkovskiy A, Wang CY, Liao HYM. Yolov4: Optimal speed and accuracy of object detection. arXiv preprint; 2020. arXiv:2004.10934.

Collobert R, Weston J, Bottou L, et al. Natural language processing (almost) from scratch. J Mach Learn Res. 2011;12(ARTICLE):2493–2537.

Jiao X, Yin Y, Shang L, et al. Tinybert: Distilling bert for natural language understanding; 2019. arXiv preprint arXiv:1909.10351

Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. In: International Conference on Neural Information Processing Systems; 2012, p. 1106–1114.

Szegedy C, Liu W, Jia Y, et al. Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2015, p. 1–9.

Liu W, Anguelov D, Erhan D, et al. Ssd: Single shot multibox detector. In: Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part I 14, Springer; 2016, p. 21–37.

Ren S, He K, Girshick R, et al. Faster r-cnn: Towards real-time object detection with region proposal networks. Adv Neural Inform Process Syst. 2015;28.

Allen-Zhu Z, Li Y, Liang Y. Learning and generalization in overparameterized neural networks, going beyond two layers. Adv Neural Inform Process Syst. 2019;32.

Liu S, Lin Y, Zhou Z, et al. On-demand deep model compression for mobile devices: A usage-driven model selection framework. In: Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services; 2018a, p. 389–400.

Denton EL, Zaremba W, Bruna J, et al. Exploiting linear structure within convolutional networks for efficient evaluation. Adv Neural Inform Process Syst. 2014;27.

Kim YD, Park E, Yoo S, et al. Compression of deep convolutional neural networks for fast and low power mobile applications. arXiv preprint; 2015. arXiv:1511.06530.

Astrid M, Lee SI. Deep compression of convolutional neural networks with low-rank approximation. ETRI J. 2018;40(4):421–34.

Article  Google Scholar 

Kholiavchenko M. Iterative low-rank approximation for cnn compression. arXiv preprint; 2018. arXiv:1803.08995.

Lee D, Kwon SJ, Kim B, et al. Learning low-rank approximation for cnns. arXiv preprint; 2019. arXiv:1905.10145.

Ahn S, Hu SX, Damianou A, et al. Variational information distillation for knowledge transfer. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; 2019, p. 9163–9171.

Yin H, Molchanov P, Alvarez JM, et al. Dreaming to distill: Data-free knowledge transfer via deepinversion. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; 2020, p. 8715–8724.

Yim J, Joo D, Bae J, et al. A gift from knowledge distillation: Fast optimization, network minimization and transfer learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2017, p. 4133–4141.

Yim J, Joo D, Bae J, et al. A gift from knowledge distillation: Fast optimization, network minimization and transfer learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2017, p. 4133–4141.

Zhao H, Sun X, Dong J, et al. Highlight every step: Knowledge distillation via collaborative teaching. IEEE Trans Cybern. 2020;52(4):2070–81.

Article  Google Scholar 

Ma X, Li G, Liu L, et al. Accelerating deep neural network filter pruning with mask-aware convolutional computations on modern cpus. Neurocomputing. 2022;505:375–87.

Article  Google Scholar 

Lin M, Cao L, Zhang Y, et al. Pruning networks with cross-layer ranking & k-reciprocal nearest filters. IEEE Trans Neural Netw Learn Syst. 2022.

LeCun Y, Denker J, Solla S. Optimal brain damage. Adv Neural Inform Process Syst. 1990;2.

Dong X, Yang Y. Network pruning via transformable architecture search. Adv Neural Inform Process Syst. 2019;32.

Liu X, Wu L, Dai C, et al. Compressing cnns using multilevel filter pruning for the edge nodes of multimedia internet of things. IEEE Internet Things J. 2021;8(14):11041–51.

Article  Google Scholar 

Liu Y, Guo Y, Guo J, et al. Conditional automated channel pruning for deep neural networks. IEEE Signal Process Lett. 2021;28:1275–9.

Article  Google Scholar 

Chang J, Lu Y, Xue P, et al. Iterative clustering pruning for convolutional neural networks. Knowl-Based Syst. 2023;265(110):386.

Google Scholar 

Shi C, Hao Y, Li G, et al. Vngep: Filter pruning based on von neumann graph entropy. Neurocomputing. 2023.

Hu W, Che Z, Liu N, et al. Channel pruning via class-aware trace ratio optimization. IEEE Trans Neural Netw Learn Syst. 2023.

Banner R, Hubara I, Hoffer E, et al. Scalable methods for 8-bit training of neural networks. Adv Neural Inform Process Syst. 2018;31.

Micikevicius P, Narang S, Alben J, et al. Mixed precision training. arXiv preprint; 2017. arXiv:1710.03740.

Chmiel B, Ben-Uri L, Shkolnik M, et al. Neural gradients are near-lognormal: improved quantized and sparse training. arXiv preprint; 2020. arXiv:2006.08173.

Cai Y, Yao Z, Dong Z, et al. Zeroq: A novel zero shot quantization framework. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; 2020, p. 13169–13178.

Lee J, Yu M, Kwon Y, et al. Quantune: Post-training quantization of convolutional neural networks using extreme gradient boosting for fast deployment. Future Gener Comput Syst. 2022;132:124–35.

Liu J, Tripathi S, Kurup U, et al. Pruning algorithms to accelerate convolutional neural networks for edge applications: A survey. arXiv preprint; 2020. arXiv:2005.04275.

Vadera S, Ameen S. Methods for pruning deep neural networks. IEEE Access. 2022;10:63280–300.

Article  Google Scholar 

Wang H, Qin C, Bai Y, et al. Recent advances on neural network pruning at initialization. In: Proceedings of the International Joint Conference on Artificial Intelligence, IJCAI, Vienna, Austria; 2022, p. 23–29.

Wimmer P, Mehnert J, Condurache AP. Dimensionality reduced training by pruning and freezing parts of a deep neural network: a survey. Artif Intell Rev; 2023, p. 1–39.

Cong S, Zhou Y. A review of convolutional neural network architectures and their optimizations. Artif Intell Rev. 2023;56(3):1905–69.

Article  Google Scholar 

Schwartz R, Dodge J, Smith NA, et al. Green ai. Communications of the ACM. 2020;63(12):54–63.

Article  Google Scholar 

Strubell E, Ganesh A, McCallum A. Energy and policy considerations for deep learning in nlp. arXiv preprint; 2019. arXiv:1906.02243.

Zoph B, Le QV. Neural architecture search with reinforcement learning. arXiv preprint; 2016. arXiv:1611.01578.

Zoph B, Vasudevan V, Shlens J, et al. Learning transferable architectures for scalable image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2018, p. 8697–8710.

Real E, Moore S, Selle A, et al. Large-scale evolution of image classifiers. In: International Conference on Machine Learning, PMLR; 2017, p. 2902–2911.

Real E, Aggarwal A, Huang Y, et al. Regularized evolution for image classifier architecture search. In: Proceedings of the aaai conference on artificial intelligence; 2019, p. 4780–4789.

Liu H, Simonyan K, Vinyals O, et al. Hierarchical representations for efficient architecture search. arXiv preprint; 2017a, arXiv:1711.00436.

Li H, Liu N, Ma X, et al. Admm-based weight pruning for real-time deep learning acceleration on mobile devices. In: Proceedings of the 2019 on Great Lakes Symposium on VLSI; 2019a, p. 501–506.

Han S, Pool J, Tran J, et al. Learning both weights and connections for efficient neural network. Adv Neural Inform Process Syst. 2015b;28.

Jin S, Di S, Liang X, et al. Deepsz: A novel framework to compress deep neural networks by using error-bounded lossy compression. In: Proceedings of the 28th international symposium on high-performance parallel and distributed computing; 2019, p. 159–170.

Han S, Mao H, Dally WJ. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint; 2015. arXiv:1510.00149

Xie X, Zhang H, Wang J, et al. Learning optimized structure of neural networks by hidden node pruning with \(l_\\) regularization. IEEE Trans Cybern. 2019;50(3):1333–46.

Article  Google Scholar 

Mantena G, Sim KC. Entropy-based pruning of hidden units to reduce dnn parameters. In: 2016 IEEE Spoken Language Technology Workshop (SLT). IEEE. 2016:672–9.

Cheng Y, Yu FX, Feris RS, et al. An exploration of parameter redundancy in deep networks with circulant projections. In: Proceedings of the IEEE international conference on computer vision; 2015, p. 2857–2865.

Li H, Kadav A, Durdanovic I, et al. Pruning filters for efficient convnets. arXiv preprint; 2016. arXiv:1608.08710.

He Y, Liu P, Wang Z, et al. Filter pruning via geometric median for deep convolutional neural networks acceleration. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition; 2019, p. 4340–4349.

He Y, Kang G, Dong X, et al. Soft filter pruning for accelerating deep convolutional neural networks. arXiv preprint; 2018. arXiv:1808.06866.

You Z, Yan K, Ye J, et al. Gate decorator: Global filter pruning method for accelerating deep convolutional neural networks. Adv Neural Inform Process Syst. 2019:32.

Jordao A, Lie M, Schwartz WR. Discriminative layer pruning for convolutional neural networks. IEEE Journal of Selected Topics in Signal Processing. 2020;14(4):828–37.

Article  Google Scholar 

Elkerdawy S, Elhoushi M, Singh A, et al. One-shot layer-wise accuracy approximation for layer pruning. In: 2020 IEEE International Conference on Image Processing (ICIP), IEEE; 2020, p. 2940–2944.

Wang W, Zhao S, Chen M, et al. Dbp: Discrimination based block-level pruning for deep model acceleration. arXiv preprint; 2019. arXiv:1912.10178.

Yang W, Jin L, Wang S, et al. Thinning of convolutional neural network with mixed pruning. IET Image Process. 2019;13(5):779–84.

Article  Google Scholar 

Chang X, Pan H, Lin W, et al. A mixed-pruning based framework for embedded convolutional neural network acceleration. IEEE Transactions on Circuits and Systems I: Regular Papers. 2021;68(4):1706–15.

Article  Google Scholar 

Lee N, Ajanthan T, Torr PH. Snip: Single-shot network pruning based on connection sensitivity. arXiv preprint; 2018. arXiv:1810.02340.

Lee N, Ajanthan T, Gould S, et al. A signal propagation perspective for pruning neural networks at initialization. arXiv preprint; 2019. arXiv:1906.06307.

Wang C, Zhang G, Grosse R. Picking winning tickets before training by preserving gradient flow. arXiv preprint; 2020. arXiv:2002.07376.

Hayou S, Ton JF, Doucet A, et al. Pruning untrained neural networks: Principles and analysis. arXiv preprint; 2020. arXiv:2002.08797.

Malach E, Yehudai G, Shalev-Schwartz S, et al. Proving the lottery ticket hypothesis: Pruning is all you need. In: International Conference on Machine Learning, PMLR; 2020, p. 6682–6691.

Liu T, Zenke F. Finding trainable sparse networks through neural tangent transfer. In: International Conference on Machine Learning, PMLR; 2020, p. 6336–6347.

Roy S, Panda P, Srinivasan G, et al. Pruning filters while training for efficiently optimizing deep learning networks. In: 2020 International Joint Conference on Neural Networks (IJCNN), IEEE; 2020, p. 1–7.

Aketi SA, Roy S, Raghunathan A, et al. Gradual channel pruning while training using feature relevance scores for convolutional neural networks. IEEE Access. 2020;8:171924–32.

Article  Google Scholar 

Yue L, Weibin Z, Lin S. Really should we pruning after model be totally trained? pruning based on a small amount of training. arXiv preprint; 2019. arXiv:1901.08455

Lym S, Choukse E, Zangeneh S, et al. Prunetrain: fast neural network training by dynamic sparse model reconfiguration. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis; 2019, p. 1–13

Sun X, Ren X, Ma S, et al. meprop: Sparsified back propagation for accelerated deep learning with reduced overfitting. In: International Conference on Machine Learning, PMLR; 2017, p. 3299–3308.

Luo JH, Wu J, Lin W. Thinet: A filter level pruning method for deep neural network compression. In: Proceedings of the IEEE international conference on computer vision; 2017, p. 5058–5066.

Shao M, Dai J, Wang R, et al. Cshe: network pruning by using cluster similarity and matrix eigenvalues. Int J Mach Learn Cybern; 2022, p. 1–12.

Lin M, Cao L, Li S, et al. Filter sketch for network pruning. IEEE Trans Neural Netw Learn Syst. 2021;33(12):7091–100.

Article  Google Scholar 

Yeom SK, Seegerer P, Lapuschkin S, et al. Pruning by explaining: A novel criterion for deep neural network pruning. Pattern Recognition. 2021;115(107):899.

Google Scholar 

Chen Y, Wen X, Zhang Y, et al. Ccprune: Collaborative channel pruning for learning compact convolutional networks. Neurocomputing. 2021;451:35–45.

Article  Google Scholar 

Cai L, An Z, Yang C, et al. Softer pruning, incremental regularization. In: 2020 25th International Conference on Pattern Recognition (ICPR), IEEE; 2021, p. 224–230.

Mitsuno K, Kurita T. Filter pruning using hierarchical group sparse regularization for deep convolutional neural networks. In: 2020 25th international conference on pattern recognition (ICPR). IEEE. 2021:1089–95.

He Y, Han S. Adc: Automated deep compression and acceleration with reinforcement learning. arXiv preprint; 2018. arXiv:1802.03494

He Y, Lin J, Liu Z, et al. Amc: Automl for model compression and acceleration on mobile devices. In: Proceedings of the European conference on computer vision (ECCV); 2018, p. 784–800.

Cai H, Lin J, Lin Y, et al. Automl for architecting efficient and specialized neural networks. IEEE Micro. 2019;40(1):75–82.

Article  Google Scholar 

Lin M, Ji R, Zhang Y, et al. Channel pruning via automatic structure search. arXiv preprint; 2020. arXiv:2001.08565.

Manessi F, Rozza A, Bianco S, et al. Automated pruning for deep neural network compression. In: 2018 24th International conference on pattern recognition (ICPR). IEEE. 2018:657–64.

Ayinde BO, Inanc T, Zurada JM. Redundant feature pruning for accelerated inference in deep neural networks. Neural Netw. 2019;118:148–58.

Article  Google Scholar 

Zhang W, Wang Z. Fpfs: Filter-level pruning via distance weight measuring filter similarity. Neurocomputing. 2022;512:40–51.

Article  Google Scholar 

Singh P, Verma VK, Rai P, et al. Leveraging filter correlations for deep model compression. In: Proceedings of the IEEE/CVF Winter Conference on applications of computer vision; 2020, p. 835–844.

Yang C, Liu H. Channel pruning based on convolutional neural network sensitivity. Neurocomputing. 2022;507:97–106.

Article  Google Scholar 

Chen Z, Xu TB, Du C, et al. Dynamical channel pruning by conditional accuracy change for deep neural networks. IEEE Trans Neural Netw Learn Syst. 2020;32(2):799–813.

Article  Google Scholar 

Hu H, Peng R, Tai YW, et al. Network trimming: A data-driven neuron pruning approach towards efficient deep architectures. arXiv preprint; 2016. arXiv:1607.03250.

Luo JH, Wu J. An entropy-based pruning method for cnn compression. arXiv preprint; 2017. arXiv:1706.05791.

Liu C, Wu H. Channel pruning based on mean gradient for accelerating convolutional neural networks. Signal Process. 2019;156:84–91.

Article  Google Scholar 

Lin M, Ji R, Wang Y, et al. Hrank: Filter pruning using high-rank feature map. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition; 2020, p. 1529–1538.

Li H, Ma C, Xu W, et al. Feature statistics guided efficient filter pruning. arXiv preprint; 2020. arXiv:2005.12193.

Wang Z, Liu X, Huang L, et al. Model pruning based on quantified similarity of feature maps. arXiv preprint; 2021.