Review Research of Medical Image Analysis Using Deep Learning

  • Bakhtyar Ahmed Mohammed University of Human Development, College of Science and Technology, Department of Computer Science, Sulaymaniyah, KRG, Iraq. University of Sulaimani, College of Science, Department of Computer, Sulaymaniyah, KRG, Iraq
  • Muzhir Shaban Al-Ani University of Human Development, College of Science and Technology, Department of Computer Science, Sulaymaniyah, KRG, Iraq


In modern globe, medical image analysis significantly participates in diagnosis process. In general, it involves five processes, such as medical image classification, medical image detection, medical image segmentation, medical image registration, and medical image localization. Medical imaging uses in diagnosis process for most of the human body organs, such as brain tumor, chest, breast, colonoscopy, retinal, and many other cases relate to medical image analysis using various modalities. Multi-modality images include magnetic resonance imaging, single photon emission computed tomography (CT), positron emission tomography, optical coherence tomography, confocal laser endoscopy, magnetic resonance spectroscopy, CT, X-ray, wireless capsule endoscopy, breast cancer, papanicolaou smear, hyper spectral image, and ultrasound use to diagnose different body organs and cases. Medical image analysis is appropriate environment to interact with automate intelligent system technologies. Among the intelligent systems deep learning (DL) is the modern one to manipulate medical image analysis processes and processing an image into fundamental components to extract meaningful information. The best model to establish its systems is deep convolutional neural network. This study relied on reviewing of some of these studies because of these reasons; improvements of medical imaging increase demand on automate systems of medical image analysis using DL, in most tested cases, accuracy of intelligent methods especially DL methods higher than accuracy of hand-crafted works. Furthermore, manually works need a lot of time compare to systematic diagnosis.


[1] K. Suzuki. “Overview of deep learning in medical imaging”. Radiological Physics and Technology, vol. 10, no. 3, pp. 257-273, 2017.
[2] D. Ravi, C. Wong, F. Deligianni, M. Berthelot and J. Andreau-Perez. “Deep learning for health informatics”. IEEE Journal of Biomedical and Health Informatics, vol. 21, no. 1, pp. 4-21, 2017.
[3] A. Maier, C. Syben, T. Lasser and C. Riess. “A gentle introduction to deep learning in medical image processing”. Zeitschrift für Medizinische Physik, vol. 29, no. 2, pp. 86-101, 2019.
[4] J. K. Han. Terahertz medical imaging. In: “Convergence of Terahertz Sciences in Biomedical Systems”. Springer, Netherlands, pp. 351- 371, 2012.
[5] J. O’Doherty, B. Rojas-Fisher and S. O’Doherty. “Real-life radioactive men: The advantages and disadvantages of radiation exposure”. Superhero Science and Technology, vol. 1, no. 1, p. 2928, 2018.
[6] A. S. Lundervold and A. Lundervold. “An overview of deep learning in medical imaging focusing on MRI”. Zeitschrift für Medizinische Physik, vol. 29, no. 2, pp. 102-127, 2019.
[7] A. Eklund, P. Dufort, D. Forsberg and S. M. LaConte. “Medical image processing on the GPU-past, present and future”. Medical Image Analysis, vol. 17, no. 8, pp. 1073-1094, 2013.
[8] D. Shen, G. Wu and H. Suk. “Deep learning in medical image analysis”. Review in Advance, vol. 19, pp. 221-248, 2017.
[9] A. Mitra, P. S. Banerjee, S, Roy, S. Roy and S. K. Setua. “The region of interest localization for glaucoma analysis from retinal fundus image using deep learning”. Computer Methods and Programs in Biomedicine, vol. 165, pp. 25-35, 2018.
[10] G. Urban, P. Tripathi, T. Alkayali, M. Mittal, F. Jalali, W. Karnes and P. Baldi. “Deep learning localizes and identifies polyps in real time with 96% accuracy in screening colonoscopy”. Gastroenterology, vol. 155, no. 4, pp. 1069-1078.e8, 2018.
[11] N. Banerjee, R. Sathish and D. Sheet. “Deep neural architecture for localization and tracking of surgical tools in cataract surgery”. Computer Aided Intervention and Diagnostics in Clinical and Medical Images, vol. 31, pp. 31-38, 2019.
[12] Y. Zheng, D. Liu, B. Georgescu, D. Xu and D. Comaniciu. “Deep Learning Based Automatic Segmentation of Pathological Kidney in CT: Local Versus Global Image Context”. Springer, Cham, Switzerland, pp. 241-255, 2017.
[13] M. Berahim, N. A. Samsudin and S. S. Nathan. “A review: Image analysis techniques to improve labeling accuracy of medical image classification”. Advances in Intelligent Systems and Computing, vol. 700, pp. 1-11, 2018.
[14] M. A. El-Sayed, Y. A. Estaitia and M. A. Khafagy. “Automated edge detection using convolutional neural network”. International Journal of Advanced Computer Science and Applications, vol. 4, no. 10, p. 11, 2013.
[15] M. Havaei, A. Davy, D. Warde-Farley, A. Biard, A. Courville, Y. Bengio, C. Pal, P. M. Jodoin and H. Larochelle. “Brain tumor segmentation with deep neural networks”. Medical Image Analysis, vol. 35, pp. 18-31, 2017.
[16] S. Kumar, A. Negi, J. N. Singh, H. Verman. “A Deep Learning for Brain Tumor MRI Images Semantic Segmentation using FCN”. In: 2018 4th International Conference on Computing Communication and Automation, Greater Noida, India, India, 14-15 Dec 2018, 2018.
[17] M. K. Abd-Ellah, A. I. Awad, A. A. M. Khalafd and H. F. A. Hamed. “A review on brain tumor diagnosis from MRI images: Practical implications, key achievements, and lessons learned”. Magnetic Resonance Imaging, vol. 61, pp. 300-318, 2019.
[18] R. P. M. Krishnammal and S. Selvakumar. “Convolutional Neural Network based Image Classification and Detection of Abnormalities in MRI Brain Images”. In: 2019 International Conference on Communication and Signal Processing, Chennai, India, India, 4-6 April, 2019.
[19] H. H. Sultan, N. M. Salem and W. Al-Atabany. “Multi-Classification of Brain Tumor Images Using Deep Neural Network”. IEEE Access, vol. 1, pp. 1-11, 2019.
[20] K. Kushibar, S. Valverde, S. Gonzalez-Villa, J. Bernal, M. Cabezas, A. Oliver and X. Liado. “Automated sub-cortical brain structure segmentation combining spatial and deep convolutional features”. Medical Image Analysis, vol. 48, pp. 177-186, 2018.
[21] F. Guo, M. Ng, M. Goubran, S. E. Petersen, S. K. Piechnik, S. N. Bauerd and G. Wright. “Improving cardiac MRI convolutional neural network segmentation on small training datasets and dataset shift: A continuous kernel cut approach”. Medical Image Analysis, vol. 61, p. 101636, 2020.
[22] A. Bidani, M. S. Gouider and C. M. Traviesco-Gonzalez. “Dementia Detection and Classification from MRI Images Using Deep Neural Networks and Transfer Learning”. In: International Work- Conference on Artificial Neural Networks IWANN 2019, vol. 11506, pp. 925-933, 2019.
[23] H. N. G. Geok, M. Kerzel, J. Mehnert, A. May and S. Wermter. “Classification of MRI Migraine Medical Data Using 3D Convolutional Neural Network”. ICANN 2018, vol. 11141, pp. 300- 309, 2018.
[24] Z. J. Islam and Y. Yanqing. “A novel deep learning based multi-class classification method for Alzheimer’s disease detection using brain MRI data. In: International Conference, BI 2017, Beijing, China, November 16-18, 2017, Beijing, China, 2017.
[25] S. Goswami and L. K. P. Bhaiya. “Brain Tumor Detection Using Unsupervised Learning Based Neural Network”. In: 2013 International Conference on Communication Systems and Network Technologies, Gwalior, India, 6-8 April 2013.
[26] H. S. A. Pardakhti. “Age prediction based on brain MRI image: A survey”. Journal of Medical Systems, vol. 43(8), p. 279, 2019.
[27] H. Mohsen, A. E. S. A. El-Dahshan, E. S. M. El-Horbaty, A. B. M. Salem. “Classification using deep learning neural networks for brain tumors”. Future Computing and Informatics Journal, vol. 3, no. 1, pp. 68-71, 2018.
[28] R. Alkadi, F. Taher, A. El-Baz and N. Werghi. “A deep learning-based approach for the detection and localization of prostate cancer in T2 magnetic resonance images”. Journal of Digital Imaging, vol. 32, no. 12, pp. 793-807, 2018.
[29] M. Talo, U. B. Baloglu, O. Yildirim and U. R. Acharya. “Application of deep transfer learning for automated brain abnormality classification using MR images”. Cognitive Systems Research, vol. 54, pp. 176-188, 2018.
[30] L. Aghaghazvini, P. Pirouzi, H. Sharifian, N. Yazdani, S. Kooraki, A. Ghadiri and M. Assadi. “3T magnetic resonance spectroscopy as a powerful diagnostic modality for assessment of thyroid nodules”. SciELO Analytics, vol. 62, no. 5, pp. 2359-4292, 2018.
[31] A. Elangovan and T. Jeyaseelan. “Medical Imaging Modalities: A Survey. In: “2016 International Conference on Emerging Trends in Engineering, Technology and Science”. Pudukkottai, India, 24-26 Feb, 2016.
[32] Y. Song, S. Zheng, L. Li, X. Zhang, X. Zhang, Z. Huang, J. Chen, H. Zhao, Y. Jie, R. Wang, Y. Chong, J. Shen and Y. Yang. “Deep learning Enables Accurate Diagnosis of Novel Coronavirus (COVID-19) with CT images”. medRxiv, 2020.
[33] J. Men, Y. Huang, J. Solanki, X. Zeng, A. Alex, J. Jerwick, Z. Zhang, R. E. Tanzi, A. Li and C. Zhou. “Optical coherence tomography for brain imaging and developmental biology”. IEEE Journal of Selected Topics in Quantum Electronics, vol. 22, no. 4, p. 6803213, 2016.
[34] L. Ngo, G. Yih, S. Ji and J. H. Han. “A Study on Automated Segmentation of Retinal Layers in Optical Coherence Tomography Images. In: 2016 4th International Winter Conference on Brain- Computer Interface (BCI). Yongpyong, South Korea, 22-24 Feb, 2016.
[35] L. Moraru, C. D. Obreja, N. Dey and A. S. Ashour. Dempster-Shafer Fusion for Effective Retinal Vessels Diameter Measurement. Elsevier, Amsterdam, Netherlands, pp. 149-160, 2018.
[36] J. Sun, C. Wan, J. Cheng, F. Yu and J. Liu. “Retinal Image Quality Classification using Fine-Tuned CNN. In: OMIA 2017, FIFI 2017: Fetal, Infant and Ophthalmic Medical Image Analysis”. vol. 10554. Springer, Berlin, Germany, pp. 126-133, 2017.
[37] Y. Song, J. Z. Cheng, D. Ni, S. Chen, B. Lei and T. Wang. “Segmenting Overlapping Cervical Cell in Pap Smear Images. In: 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI), Prague, Czech Republic, 13-16 April, 2016.
[38] L. D. Nguyen, D. Lin, Z. Lin and J. Cao. “Deep CNNs for Microscopic Image Classification by Exploiting Transfer Learning and Feature Concatenation. In: 2018 IEEE International Symposium on Circuits and Systems (ISCAS), Florence, Italy, 27-30 May 2018.
[39] S. Dey, D.N. Tibarewala, S. P. Maity and A. Barui. “Automated Detection of Early Oral Cancer Trends in Habitual Smokers. Elsevier, Amsterdam, Netherlands, pp. 83-107, 2018.
[40] M. Izadyyazdanabadi, E. Belykh, M. Mooney, N. Martirosyan, J. Eschbacher, P. Nakaji, M. C. Preul and Y. Yang. “Convolutional neural networks: Ensemble modeling, fine-tuning and unsupervised semantic localization for neurosurgical CLE images”. The Journal of Visual Communication and Image Representation, vol. 1, pp. 10-20, 2018.
[41] L. Lan, C. Ye, C. Wang and S. Zhou. “Deep convolutional neural networks for WCE abnormality detection: CNN architecture, region proposal and transfer learning”. IEEE Access, vol. 7, pp. 30017- 30032, 2019.
[42] A. H. Shahin, A. Kamal and M. A. Elattar. “Deep Ensemble Learning for Skin Lesion Classification from Dermoscopic Images”. In: 2018 9th Cairo International Biomedical Engineering Conference, Cairo, Egypt, Egypt, 20-22 Dec. 2018.
[43] S. V. Georgakopoulos, K. Kottari, K. Delibasis, V. P. Plagianakos and I. Maglogiannis. “Improving the performance of convolutional neural network for skin image classification using the response of image analysis filters”. Neural Computing and Applications, vol. 31, no. 6, pp. 1805-1822, 2019.
[44] G. Murtaza, L. Shuib, A. W. A. Wahab, G. Mujtaba, H. F. Nweke, M. A. Al-Garadi, F. Zulfiqar, G. Raza and N. A. Azmi. “Deep learning-based breast cancer classification through medical imaging modalities: State of the art and research challenges”. Artificial Intelligence Review, 53, pp. 1-66, 2019.
[45] Y. Li, J. Wu and Q. S. Wu. “Classification of breast cancer histology images using multi-size and discriminative patches based on deep learning. IEEE Access, vol. 7, pp. 21400-21408, 2019.
[46] A. M. Ahmad, G. Muhammad and J. F. Miller. “Breast Cancer Detection Using Cartesian Genetic Programming evolved Artificial Neural Networks. In: GECCO ‘12 Proceedings of the 14th Annual Conference on Genetic and Evolutionary Computation, Philadelphia, Pennsylvania, USA, July 07-11, 2012.
[47] P. R. Jeyaraj E. R. S. Nadar. “Computer-assisted medical image classification for early diagnosis of oral cancer employing deep learning algorithm”. Journal of Cancer Research and Clinical Oncology, vol. 145, no. 4, pp. 829-837, 2019.
[48] R. Wei, F. Zhou, B. Liu, X. Bai, D. Fu, Y. Li and B. Liang. “Convolutional neural network (CNN) based three dimensional tumor localization using single X-ray projection”. IEEE Access, vol. 7, pp. 37026-37038, 2019.
[49] Z. Lai and H. F. Deng. “Medical image classification based on deep features extracted by deep model and statistic feature fusion with multilayer perceptron”. Computational Intelligence and Neuroscience, vol. 2018, pp. 1-13, 2018.
[50] G. Litjens, T. Kooi, B. E. Bejnordi, A. A. A. Setio, F. Ciompi, M. Ghafoorian, J. A. W. V. Laak, B. Van Ginneken and C. S. Diagnostic. “A survey on deep learning in medical image analysis”. Medical Image Analysis, vol. 42, pp. 60-88, 2017.
[51] K. C. L. Wong, T. Syeda-Mahmood and M. Moradi. “Building medical image classifiers with very limited data using segmentation networks”. Medical Image Analysis, vol. 49, pp. 105-116, 2018
[52] N. T. Member, J. Y. Shin, S. R. Gurudu, R. T. Hurst, C. Kendall, M. Gotway and J. Liang. “Convolutional neural networks for medical image analysis: Full training or fine tuning”. IEEE Transactions on Medical Imaging, vol. 35, no. 5, pp. 1299-1312, 2016.
[53] J. Ker, L. Wang, J. Rao and T. Lim. “Deep learning applications in medical image analysis”. IEEE Access, vol. 6, pp. 9375-9389, 2017.
[54] K. U. Rani. “Analysis of heart diseases dataset using neural network approach”. International Journal of Data Mining and Knowledge Management Process, vol. 1, no. 5, pp. 1-8, 2011.
[55] R. Yamashita, M. Nishio, R. K. G. Do and K. Togashi. “Convolutional neural networks: an overview and application in radiology”. Insights into Imaging, vol. 9, no. 4, pp. 611-629, 2018.
[56] A. D. Ruvalcaba-Cardenas, T. Scolery and G. Day. “Object classification using deep learning on extremely low-resolution time-of-flight data”. In: 2018 Digital Image Computing: Techniques and Applications (DICTA), Canberra, Australia, Australia, 10-13 Dec 2018.
[57] E. Ahn, A. Kumar, M. Fulham, D. Feng and J. Kim. “Convolutional sparse kernel network for unsupervised medical image analysis”. Medical Image Analysis, vol. 56, pp. 140-151, 2019.
[58] Z. Wu, S. Zhao, Y. Peng, X. He, X. Zhao, K. Huang, X. Wu, W. Fan, F. Li, M. Chen, J. Li, W. Huang, X. Chen and Y. Li. “Studies on different CNN algorithms for face skin disease classification based on clinical images”. IEEE Access, vol. 7, pp. 66505-66511, 2019.
[59] F. Altaf, S. M. S. Islam, N. Akhtar and N. K. Janjua. “Going deep in medical image analysis: Concepts, methods, challenges and future directions”. IEEE Access, vol. 7, pp. 99540-99572, 2019.
[60] K. M. Hosny, M. A. Kassem and M. M. Foaud. “Classification of skin lesions using transfer learning and augmentation with Alex-net”. PLoS One, vol. 14, no. 5, p. e0217293, 2019.
[61] J. Arevalo, F. A. Gonzalez, R. R. Pollan, J. L. Oliveira and M. A. G. Lopez. “Convolutional neural networks for mammography mass lesion classification”. In: 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Milan, Italy, 25-29 Aug 2015.
[62] M. F. B. Othman, N. B. Abdullah and N. F. Kamal. “MRI Brain Classification Using Support Vector Machine. In: 2011 4th International Conference on Modeling, Simulation and Applied Optimization, Kuala Lumpur, Malaysia, 19-21 April 2011.
[63] S. H. Shirazi, S. Naz, M. I. Razzak, A. I. Umar and A. Zaib. “Automated Pathology Image Analysis”. Elsevier, Pakistan, pp. 13- 29, 2018.
[64] N. C. Ouseph and K. Shruti. “A reliable method for brain tumor detection using cnn technique”. IOSR Journal of Electrical and Electronics Engineering, vol. 1. pp. 64-68, 2017.
[65] A. Srivastava, S. Sengupta, S. J. Kang, K. Kant, M. Khan, S. A. Ali, S. R. Moore, B. C. Amadi, P. Kelly, S. Syed and D. E. Brown. “Deep Learning for Detecting Diseases in Gastrointestinal Biopsy Images. In: 2019 Systems and Information Engineering Design Symposium (SIEDS), Charlottesville, VA, USA, USA, 26-26 April 2019.
[66] R. M. Summers. “Deep Learning and Computer-Aided Diagnosis for Medical Image Processing: A Personal Perspective”. Springer International Publishing, Switzerland, pp. 3-10, 2017.
[67] G. Carnerio, Y. Zheng, F. Xing and L. Yang. Review of Deep Learning Methods in Mammography, Cardiovascular,and Microscopy Image Analysis. Springer, Switzerland, 2017, pp. 11-35.
[68] V. V. Kumar, K. S. Krishna and S. Kusumavathi. “Genetic algorithm based feature selection brain tumour segmentation and classification”. International Journal of Intelligent Engineering and Systems, vol. 12, no. 5, pp. 214-223, 2019.
[69] D. Zikic, Y. Ioannou, M. Brown and A. Criminisi. “Segmentation of brain tumor tissues with convolutional neural networks”. MICCAI Workshop on Multimodal Brain Tumor Segmentation Challenge (BRATS) At, Boston, Massachusetts. pp. 36-39, 2014.
[70] A. Norouzi, M. S. M. Rahim, A. Altameem, T. Saba, A. E. Rad, A. Rehman and M. Uddin. “Medical image segmentation methods, algorithms, and applications”. IETE Technical Review, vol. 31, no. 3, pp. 199-213, 2014.
[71] N. Dey and A. S. Ashour. “Computing in Medical Image Analysis”. Elsevier, Amsterdam, Netherlands, pp. 3-11, 2018.
[72] N. Padmasini, R. Umamaheswari and M. Y. Sikkandar. “State-of-the-Art of Level-Set Methods in Segmentation and Registration of Spectral Domain Optical Coherence Tomographic Retinal Images. Elsevier, United Kingdom, 2018, pp. 163-181.
[73] R. R. Agravat and M. S. Raval. “Deep Learning for Automated Brain Tumor Segmentation in MRI Images”. Elsevier, United Kingdom, pp. 183-201, 2018.
[74] T. A. Ngo and G. Carneiro. “Fully automated segmentation using distance regularised level set and deep-structured learning and inference. In: L. Lu, Y. Zheng, G. Carneiro, L. Yang, (eds) “Deep Learning and Convolutional Neural Networks for Medical Image Computing. Advances in Computer Vision and Pattern Recognition”. Springer, Cham, 2017, pp. 197-224.
[75] J. Bernal, K. Kushibar, M. Cabezas, S. Valverde, A. Oliver and X. Llado. “Quantitative analysis of patch-based fully convolutional neural networks for tissue segmentation on brain magnetic resonance imaging”. IEEE Access, vol. 7, pp. 89986-90002, 2019.
[76] R. Ceylan and H. Koyuncu. “ScPSO-Based Multithresholding Modalities for Susoicious Region Detection on Mammograms”. Elsevier, Amsterdam, Netherlands, pp. 109-135, 2018.
[77] N. Dhungel, G. Carneiro, A. P. Bradley. Combining deep learning and structured prediction for segmenting masses in mammograms. In: L. Lu, Y. Zheng, G. Carneiro, L. Yang, (eds). “Deep Learning and Convolutional Neural Networks for Medical Image Computing. Advances in Computer Vision and Pattern Recognition”. Springer, Cham, pp. 225-240, 2017.
[78] A. O. Mader, C. Lorenz, M. Bergtholdt, J. von Berg, H. Schramm, J. Modersitzki and C. Meyer. “Detection and localization of spatially correlated point landmarks in medical images using an automatically learned conditional random field”. Computer Vision and Image Understanding, vol. 176-177, pp. 45-53, 2018.
[79] K. Marstal, F. Berendsen, N. Dekker, M. Staring and S. Klein. “The Continuous Registration Challenge: Evaluation-as-a-Service for Medical Image Registration Algorithms”. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), Venice, Italy, Italy, 8-11 April 2019.
[80] S. Ramamoorthy, R. Vinodhini and R. Sivasubramaniam. “Monitoring the growth of Polycystic Ovary Syndrome using Mono-modal Image Registration Technique”. In: International Conference on Data Science and Management of Data (CoDS-COMAD’19), Kolkata, India, January 03-05, 2019.
[81] B. D. de Vos, J. M. Wolterink, P. A. de Jong, T. Leiner, M. A. Viergever and I. Isgum. “Conv net-based localization of anatomical”. IEEE Transactions on Medical Imaging, vol. 36, no. 7, pp. 1470-1481, 2017.
[82] U. Bagci. “Medical image computing CAVA: Computer Aided Visualization”. University of Central Florida, Florida, 2017.
[83] M. M. Murray, M. L. Rosenberg, A. J. Allen, M. Baranoski, R. Bernstein, J. Blair, C. H. Brown, E. Caine, S. Greenberg and V. M. Mays. “Violence and Mental Health: Opportunities for Prevention and Early Detection: Proceedings of a Workshop”. The National Academies Press, Washington, DC, 2018.
[84] S. S. Yadav and S. M. Jadhav. “Deep convolutional neural network based medical image classification for disease diagnosis”. Journal of Big Data, vol. 6, p. 113, 2019.
How to Cite
MOHAMMED, Bakhtyar Ahmed; AL-ANI, Muzhir Shaban. Review Research of Medical Image Analysis Using Deep Learning. UHD Journal of Science and Technology, [S.l.], v. 4, n. 2, p. 75-90, aug. 2020. ISSN 2521-4217. Available at: <>. Date accessed: 20 sep. 2020. doi: