Object detection is an important processing task for many medical applications,
especially for lesion detection. Deep learning allows for an automatic localization
of suspicious masses in several imaging modalities such as CT, MRI or US, and
sometimes even for the classification of the lesions as benign or malignant,
helping radiologists and providing valuable input to computer aided detection
systems. Another relevant application of convolutional neural network detection
systems is the automatic localization of the plane of interest, which can save
practitioners a lot of time when trying to find significant structures within the
whole volumes. Furthermore, localization and detection of anatomical landmarks
can assist the initialization of other image processing algorithms, such as
registration or segmentation. There are different approaches to object detection
that can be applied to medical image processing. Scanning-based systems rely on
a region proposal step to generate patches that are later classified according to the
object they contain, whereas more recent systems directly generate bounding
boxes from the whole input image, improving the accuracy and allowing realtime detection. However, training convolutional neural networks for medical
image detection still faces important challenges. The main limitation is the lack
of large public databases that can be used to train or to do transfer learning.
Furthermore, when the objective is to detect pathological structures, there is a
class imbalance problem, as there is usually more data from healthy patients than
from a specific pathology. Data augmentation strategies are commonly used to
alleviate this problem, as well as hard example mining strategies. Some authors
have tried weakly supervised approaches, reducing the cost of generating
annotated databases. More effort needs to be done in the creation of accessible
databases and in developing training strategies that allow for the use of weakly
annotated data, noisy annotations and unsupervised learning.
(2) Description of the Invention-
3 of 9
Accurate and fast detection of anatomical or pathological structures or landmarks
is essential in the medical field for a wide variety of tasks. For instance, the
localization of anatomical landmarks is necessary to guide image registration or
to initialize the volumetric segmentation of organs. Lesion detection is a crucial
step towards the development of Computer Aided Detection and Diagnosis
(CAD) systems, which have become increasingly popular in the last decades.
Besides, detection algorithms are also valuable to facilitate structure tracking
during interventions or to localize relevant image planes within the whole
medical image volumes. Object detection algorithms differ from classification
algorithms in that they not only recognize which objects or structures are present
in the image, but also their exact location within the image by outputting their
bounding boxes. Localization and detection are similar tasks: localization
algorithms usually identify one single object in the image, while object detection
algorithms are able to find the presence and location of several objects that are
present in the image (as shown in Fig. 1).
Therefore, a detection algorithm will output a bounding box for each object
present in the image and, associated to each bounding box, the type of object that
it contains (often a value indicating the probability that the object belongs to that
class). This chapter will focus mainly on the object detection task, though some
examples of landmark localization in the medical domain are also provided.
Generally, algorithms for object detection in computer vision follow two steps:
(1) region proposal, which refers to the extraction of several patches from the
image to find potential patches containing the object of interest. The whole image
can be scanned and divided into patches with a sliding window approach or with
specialized region proposal algorithms to find the regions that are most likely to
contain certain objects.
(2) classification of the extracted patches to output bounding boxes with a certain
probability of containing an object.
4 of 9
(3) BACKGROUND OF THE INVENTION
Traditionally, feature extraction algorithms (for instance, SIFT, HOG, LBP, Haar
wavelets, or hough transform) are used to characterize image patches and these
features are fed to classifiers such as support vector machines (SVMs) or random
forests. This approach has been used in several object detection problems in the
medical domain.
The success of convolutional neural networks (CNNs) for image classification
tasks motivated the use of deep learning for image detection, exploiting features
extracted by CNNs instead of using sets of hand-crafted features. In 2014,
Girshick et al. proposed the R-CNN (Regions with CNN features). For region
proposal they used a popular algorithm known as selective search. The extracted
image patches are then fed to a CNN, AlexNet, to extract features, and finally a
SVM is used for classification. Since then, many other CNN-based models for
image detection have been designed and evaluated. State of the art deep learningbased methods for object detection eliminate the region proposal step or extract
region proposals directly from the feature map instead of the image, improving
the speed and outperforming the results of traditional object detection algorithms.
However, as compared to the computer vision domain, the detection task in
medical imaging needs to deal with some domain-specific challenges, such as the
lack of large databases with annotated data. This requires that researchers
working in the medical field need to modify or develop detection algorithms
particularly adapted to this field.
Deep learning-based object detectors have been used in a wide range of
pathologies, for instance, breast cancer, prostate cancer and retinopathy, as well
as for the localization of landmarks and anatomical structures, which can be used
as a guide for image registration or segmentation.
(4) SUMMARY OF THE INVENTION:
5 of 9
Deep learning allows for an automatic localization of suspicious masses in
several imaging modalities such as CT, MRI or US, and sometimes even for the
classification of the lesions as benign or malignant, helping radiologists and
providing valuable input to computer aided detection systems. Another relevant
application of convolutional neural network detection systems is the automatic
localization of the plane of interest, which can save practitioners a lot of time
when trying to find significant structures within the whole volumes.
(5) BRIEF DESCRIPTION OF THE DRAWINGS
Localization and detection are similar tasks: localization algorithms usually
identify one single object in the image, while object detection algorithms are able
to find the presence and location of several objects that are present in the image
(Figure 1). The first CNN-based approaches for object detection were based on
this schema, introducing the CNN either for feature extraction or for patch
classification (Figure 2). At each sliding-window location, k region proposals are
predicted simultaneously, so the regression layer encodes the coordinates of k
boxes and the classification layer out puts 2k scores, the probability of object or
not object for each proposal (Figure 3).
(6) DETAILED DESCRIPTION OF THE INVENTION
The choice of a proper loss function to be optimized and the selection of the best
training strategy are very important factors for the network to converge and can
have a great impact on performance times. Some of the previously described
approaches require a complex training strategy, for example Faster R-CNN has to
alternate fine-tuning for the RPN and fine-tuning for object detection, while other
networks can be trained directly end-to-end. The error metrics or loss functions to
be optimized are different for each approach as well, but in general, they have to
be designed to quantify both the classification error and the localization error.
Intersection over Union, IoU is the metric typically used to assess the results of
6 of 9
object detection. It is a value between 0 and 1 that represents the overlapping area
between the predicted box and the ground truth box. Detection networks have not
only to predict the bounding box but also the classify the object contained in that
box. Therefore, multi-task losses that combine metrics for classification with
metrics for localization, taking into account the IoU, are proposed. For RPN the
classification loss is logarithmic loss over two classes (object or not) and the
regression loss is smooth L1 over the parametrized coordinates of the bounding
box. It is only activated for positive region proposals, that is, regions with the
highest IoU with a ground-truth box or with a IoU higher than 0.7. In YOLO they
optimized for the sum-squared error, weighting localization error and
classification error. Each grid cell predicts several bounding boxes, a bounding
box is considered responsible for detecting a ground truth object object if it has
the highest IoU of any predictor in that grid cell.

CLAIMS
We / I Claim:
1. The proposed system is used to detection and enhance the image using
deep learning technique.
2. Deep learning allows for an automatic localization of suspicious masses
in several imaging modalities such as CT, MRI or US, and sometimes
even for the classification of the lesions as benign or malignant, helping
radiologists and providing valuable input to computer aided detection
systems.
3. Another relevant application of convolutional neural network detection
systems is the automatic localization of the plane of interest, which can
save practitioners a lot of time when trying to find significant structures
within the whole volumes.
4. Furthermore, localization and detection of anatomical landmarks can
assist the initialization of other image processing algorithms, such as
registration or segmentation.