Contributions: (I) Conception and design: J Olveres, N Méndez-Sánchez, B Escalante-Ramírez; (II) Administrative support: J Olveres, B Escalante-Ramírez; (III) Provision of study materials or patients: All authors; (IV) Collection and assembly of data: All authors; (V) Data analysis and interpretation: All authors; (VI) Manuscript writing: All authors; (VII) Final approval of manuscript: All authors.
Correspondence to: Boris Escalante-Ramírez. Centro de Estudios en Computación Avanzada CECAV. Universidad Nacional Autónoma de México. 3er piso, Torre de Ingeniería, Cd. Universitaria, Mexico City, C.P. 04510, Mexico. Email: xm.manu@sirob.
^ ORCID: Jimena Olveres, 0000-0002-1514-4520; Boris Escalante-Ramírez, 0000-0003-4936-8714. Received 2020 Oct 13; Accepted 2021 Apr 20. Copyright 2021 Quantitative Imaging in Medicine and Surgery. All rights reserved.Computer vision and artificial intelligence applications in medicine are becoming increasingly important day by day, especially in the field of image technology. In this paper we cover different artificial intelligence advances that tackle some of the most important worldwide medical problems such as cardiology, cancer, dermatology, neurodegenerative disorders, respiratory problems, and gastroenterology. We show how both areas have resulted in a large variety of methods that range from enhancement, detection, segmentation and characterizations of anatomical structures and lesions to complete systems that automatically identify and classify several diseases in order to aid clinical diagnosis and treatment. Different imaging modalities such as computer tomography, magnetic resonance, radiography, ultrasound, dermoscopy and microscopy offer multiple opportunities to build automatic systems that help medical diagnosis, taking advantage of their own physical nature. However, these imaging modalities also impose important limitations to the design of automatic image analysis systems for diagnosis aid due to their inherent characteristics such as signal to noise ratio, contrast and resolutions in time, space and wavelength. Finally, we discuss future trends and challenges that computer vision and artificial intelligence must face in the coming years in order to build systems that are able to solve more complex problems that assist medical diagnosis.
Keywords: Artificial intelligence (AI), computer vision (CV), medical image analysis, cardiology, oncology, microscopy, neurodegenerative disorders, respiratory diseases, gastroenterology
Undoubtedly, it is intangible to think about all the technological advances that have been developed in the field of medicine in recent decades. Not only have they allowed us to understand more precisely the anatomy and physiology of the different organs that structure the human body, but they have also let us advance in the identification and, therefore, the treatment of several diseases from very early stages in different areas of medicine. This has been largely accomplished by the development of computer vision (CV) and artificial intelligence (AI). Briefly, these tools provide us the ability to acquire, process, analyze, and understand an infinite number of static and dynamic images in real time, which will represent a better characterization of each disease, and a better patient selection for early interventions.
Since many diagnostic methods available to date present the disadvantage of being invasive, expensive and/or very complex for their standardization in most parts of the world, assisted diagnosis through CV and AI represent a feasible solution that allows to identify a broad number of different diseases from initial stages, define better the treatment and follow-up, and decrease the health care costs associated with each patient.
The union of high-performance computing with machine learning (ML) offers the capacity to deal with big medical image data for accurate and efficient diagnosis. Moreover, AI and CV may reduce the significant intra- and inter-observer variability, which undermines the significance of the clinical findings. AI allows to automatically make quantitative assessments of complex medical image with increased diagnosis accuracy.
The objective of this review is to assess in an understandable and well-structured way recent advances in automatic medical image analysis in some of the diseases with higher incidence and prevalence rates worldwide.
Cardiovascular diseases (CVD), oncological disorders and pulmonary diseases such as chronic obstructive pulmonary disease (COPD) and coronavirus disease 2019 (COVID-19) have positioned themselves as the main causes of mortality in individuals 50 years of age and older (1-5). Moreover, CVD are also the main cause of premature death (4). Gastrointestinal and liver diseases account for some of the highest burden and cost in public health care worldwide (4,6). Moreover, the lack of timely attention at early stages of these diseases lead to high morbidity and mortality rates (3). Neurodegenerative diseases such as Alzheimer’s and Parkinson’s have increased their incidence in modern times. As life expectancy increases, so does the occurrence of these diseases (7). Although AI has contributed to develop a large number of methods that assist the diagnosis of these diseases (8,9), we limit this review to CV and ML methods applied on medical imaging, namely ultrasound, computer tomography, magnetic resonance, microscopy and dermoscopy. We show how these methods are used to detect, classify, characterize and enhance relevant information in order to aid clinical diagnosis and treatment.
In the next section, the relation of medical imaging with CV and AI is discussed from a general point of view. Subsequent sections show recent advances and applications of these two areas of computer science that have improved diagnosis performance of the above-mentioned diseases.
According to Patel et al. (10) some of the earliest works on AI applied to medicine date from the 1970s when AI was an already known discipline and the term was created at the very famous 1956 Dartmouth College conference (11). While many researchers created model-based image processing based on AI, they did not call them that, even when their algorithms were from this area (10). Since then, new applications started to emerge that led to the first conference of the new organization Artificial Intelligence in Medicine Europe (AIME), at Maastricht, The Netherlands on June 1991, that established the term AI in Medicine. At first, most methods were based on expert systems. More recently, Kononenko (12) punctuates that the goal of AI in medicine is to try to make computers more intelligent and one of the basic requirements of intelligence is the ability to learn. ML appeals to that concept and some of the first successful algorithms in medicine were naive Bayesian approaches (12).
CV deals with a large range of problems such as image segmentation, object recognition, detection, reconstruction, etc. It aims at modeling and understanding the visual world by extracting useful information from digital images, often inspired by complex tasks of human vision. Although it exists since the 1960s, it remains an unsolved and challenging task to the extent that only recently computers have been able to provide useful solutions in different application fields. It is a multidisciplinary subject closely related to AI. AI is a broad area of computer science that aims at building automatic methods to solve problems that typically require human intelligence. ML, in turn, is a part of AI that builds systems that are able to automatically learn from data and observations. Most successful methods of CV have been developed with ML techniques. Among the most widely used methods of ML are Support Vector Machines (SVM), Random Forests, Regression (linear and logistic), K-Means, k-nearest neighbors (k-NN), Linear Discriminant Analysis, Naive Bayes (NB), etc.
Deep learning (DL) is a subfield of ML that has become an attractive and popular tool in CV because of its amazing results in complex problems of visual information analysis and interpretation. The term deep refers to multiple-layer neural networks models. In recent years, there has been a rising interest in applying DL models to medical problems (12-14). For example, deep neural networks have shown amazing results in skin lesion classification tasks. Some outstanding examples can be found in annual challenges (15).
Examples of DL models are convolutional neural networks (CNNs), recurrent neural networks, long short-term memory, generative adversarial networks, etc. Recent advances in this field have shown impressive accuracies and measured results. A schematic relation between CV and AI in medical imaging is presented in Figure 1 .
Relation between computer vision and artificial intelligence.
The interaction between CV and AI in medical imaging is shown in Figure 2 . Relevant feature extraction in medical image databases is a first critical step to train a new ML model. The training process aims at obtaining a model that has learned a specific task such as segmentation, classification, detection, recognition, etc. on the specific training data. Next, the model is tested with new input data that undergoes the same feature extraction process. The results of the task achieved with this test data are evaluated with performance metrics. If results do not meet the user requirements, the process is repeated until a new combination of feature extraction and ML methods satisfies the required performance level.
The flow of computer vision and machine learning interaction in medical imaging
Heart analysis is relevant for understanding cardiac diseases which are a major concern in the health area. They constitute a multifactorial and multidisciplinary clinical disease, leading to a great detriment to the quality of patients’ life. The World Heart Federation reports CVD as the main cause of morbidity and mortality in almost two thirds of the world population. Some of the most important diseases are related to the heart and blood vessels. They include coronary heart diseases, cerebrovascular disease, peripheral arterial disease, rheumatic heart disease, congenital heart disease, deep vein thrombosis and pulmonary embolism. Behavioral risk factors such as unhealthy diet, physical inactivity, alcohol consumption, and obesity represent major risk factors for heart failure (16).
Different medical imaging modalities are used for the assessment of heart failure such as computed tomography (CT), magnetic resonance imaging (MRI) and echocardiography (EG). Clinical analysis of these images comprises qualitative as well as quantitative examinations.
Applications of AI aiming at assessing heart function have ranged from localizing and segmenting anatomical structures to recognizing structural as well as dynamic patterns, and more recently to automatically identifying and classifying several diseases. Whenever heart failure occurs, the heart shows a reduced function. This may cause the left ventricle (LV) to lose its ability to contract or relax normally. In response, LV compensates for this stress by modifying its behavior, which creates hypertrophy and progresses to congestive heart failure (17). These patterns can be recognized and evaluated by AI.
Anatomical and dynamical parameter estimations are necessary to assess heart failure. These tasks related with imaging technology are so relevant that in the last decades their research and development allowed not only to search for better resolution and contrast in their studies, but also the possibility to merge anatomical and functional information. Moreover, new imaging protocols have been developed to comply with analysis and segmentation tasks. Chen et al. presented a complete study of different DL techniques in various image modalities to segment one, two or all cardiac structures (18), as well as coronary arteries, all of them supporting clinical measurements such as volume and ejection fraction.
Some of the more difficult problems of automatic segmentation include poor image quality, low contrast and poor structure border detection. Moreover, cardiac anatomy and gray value distribution of the images varies from person to person, especially when a cardiopathy exists. If we add the intra vendor and scanners differences, the challenge for the development of more sophisticated tools is considerable.
Many different methods for ventricle segmentation have been reported in the literature, where classical active contours are some of the more popular (19). CNNs often excel because of its improved segmentation performance despite their dependence on large amounts of training data (18). This is a serious issue in the case of the medical area, because of data availability restrictions in clinical environments and because of the lack of medical experts willing to annotate large amounts of data. This is a tedious task, causing fatigue as well as intra and inter variability between experts (20). In some areas such as CT and MRI challenges have emerged with annotated databases (18,21,22).
The use of DL has become common not only for cardiac segmentation but also to support classic algorithms, for example when searching for an initial localization of the cardiac structures to segment. Recently, this initialization has been achieved with a CNN that finds a coarse shape while a level set refines the contour, as shown on Figure 3 . This hybrid method has the advantage of using only a small database to train a neural network, e.g., a UNet, since only a rough initial segmentation is needed; detail refining is a task left to the active contour method
