Description:TECHNICAL FIELD

[0001] The present invention relates to the field of computer science, and more particularly, the present invention relates to the graphical user interface for detecting malignancy in thyroid nodule using hybrid convolutional neural network.

BACKGROUND ART
[0002] The following discussion of the background of the invention is intended to facilitate an understanding of the present invention. However, it should be appreciated that the discussion is not an acknowledgment or admission that any of the material referred to was published, known, or part of the common general knowledge in any jurisdiction as of the application’s priority date. The details provided herein the background if belongs to any publication is taken only as a reference for describing the problems, in general terminologies or principles or both of science and technology in the associated prior art.
[0003] Health is a wide-ranging concept referring to a state of complete physical, mental and social well-being; however, the main focal point remains absence of disease or infirmity. Diseases can be majorly classified into four categories namely communicable diseases, physiological diseases, genetic diseases, and deficiency diseases.
[0004] Cancer is an example of a physiological disease which originates by mutations of cells allowing cells to grow and multiply rapidly. Tumor formation due to accumulation of the abnormal thyroid cells is a reason behind thyroid cancer. Radiation, smoking, alcohol, nutritional deficiencies and history of benign tumor thyroid or goiter cancer are the main risk factors for thyroid malignancy.
[0005] More than one-half of individuals have remarkably common outgrowths in or on the thyroid endocrine gland called as thyroid nodules; these can be malignant (cancerous) or benign (non-cancerous). The majority i.e. more than 90% nodules are benign (non-cancerous), but still there is a risk of malignancy ranges from 7% to 15% in adults. Worldwide thyroid is one of the most common malignant endocrine gland accountings for 2% of all cancers with standardized mortality rate of 0.5 per 100,000.
[0006] The perfect imaging technique of the neck is Ultrasound, which performs a significant function as both a therapeutic and diagnostic instrument, in the assessment of patients with neck lumps. Ultrasound (US) captures all fine but crucial variations in morphology and provides better image of anatomic relationships in contrast to other imaging modalities. This research employs Ultrasound images for detecting malignancy.
[0007] Conventional methods for thyroid tumor diagnosis are constrained due to dependence on doctor’s proficiency which causes variations in diagnostic performance. With the application of traditional machine learning algorithms reproducibility was achieved but still the process of feature extraction called for intervention by human experts. To automate feature learning an Artificial Neural Network (ANN) was employed but ANN is inhibited by problem of over fitting. Convolutional Neural Network (CNN) is a sophisticated image classification technique which offers a reliable way out to the weaknesses of earlier applied diagnosing methods. This work aims to provide not only optimal solution for thyroid malignancy detection using CNN but also intends to give assistance to radiologists in the form of graphical user interface (GUI). This work seeks to contribute in achievement of the 3A’s goals of healthcare sector which involves - Affordability, Accessibility and Availability.
[0008] The available systems are not economical, accurate and time efficient. Further, the available systems are not user-friendly as these systems take time to respond. Some of the systems are not effectively used for remote locations. Also, the available systems are provided with the wrong information, which may mislead the user.
[0009] Earlier only conventional methods have been used for detection of thyroid nodule malignancy. Accomplishments of these methods are majorly subject to the skills of doctor and radiologist. Conventional methods are still holding their ground but with the involvement of techniques like histogam analysis, traditional ultrasound computer-aided diagnosis system (CAD), and artificial neural network, the burden on doctors and radiologists has been reduced.
Conventional Methods for Thyroid Detection-
[0010] Smaller thyroid nodules, typically less than 1 cm and those positioned posteriorly or subternally escape detection in physical examination.
[0011] Palpation is merely an initial test and does not give subtle details about the state of thyroid nodules.
[0012] Thyroid Function Tests- These are sequences of blood tests including the T3, T3RU, T4, and thyroid stimulating hormone (TSH) to measure any abnormality in thyroid functionality.
[0013] Invasive, time-consuming and labor-intensive nature.
[0014] Thyroid Fine-Needle Aspiration Biopsy (FNA)- It is a procedure in which sample of tissues are removed through a small, hollow needle.
[0015] The range of false negative rate in FNA is up to 21% which delays thyroid cancer treatment.
[0016] Invasive, time-consuming and labor-intensive character.
[0017] Histogram Analysis- It is an unsupervised discretization technique. Image statistics can be further used for classification of thyroid nodules.
[0018] Limitations- Its major drawback is relatively inferior diagnostic performance than subjective analysis performance by radiologists.
[0019] Traditional Ultrasound Computer-Aided Diagnosis (CAD) System- It comprises four basic steps to be precise preprocessing of image, segmentation of image, feature extraction, and nodule classification.
[0020] Image preprocessing ensures an improved input for the next CAD system steps, by suppressing redundant distortions and enhancement of some important image features. The region of interest (ROI) is location of thyroid nodule; image segmentation step involves extraction of ROI, generally with the help of radiologist. Feature extraction is a crucial step founded on statistical, textural, shape and/or gradient based methods. At last, in classification of nodule step classifier compares the image patterns with the target patterns and categorizes the nodule. The most widely used classifiers for thyroid nodule classification are Decision Tree (DT), Gaussian Mixture Model, K-Nearest Neighbor (KNN), Adaboost, Fuzzy Sugeno, Navies Bayes Classifier (NBC), and Support Vector Machine (SVM).
[0021] Complicated task of feature selection, which needs several passes of trial-and-error design or identification explicitly by human experts.
[0022] Use of small database in evaluation of algorithms but for being employed as diagnostic apparatus in daily clinical practice, examination on a large database is a vital requirement.
[0023] Artificial Neural Network (ANN) – It is a statistical learning algorithm built by taking inspiration from the properties of biological neural networks.
[0024] Nodes are interconnected processing elements, which are functionally similar to biological neurons. Weights are numerical values of connections between different nodes, which can be adjusted to get the preferred function. It is a feed forward neural network encompassing three layers. The first layer (input layer) gathers inputs, the middle layer (hidden layer) works as individual feature detector to recognize complex patterns in the images, and the last layer (output layer) produces the result.
[0025] It responds differently to an input (image) and its shifted version.
[0026] The quantity of weights swiftly becomes uncontrollable for large images because it applies one perceptron for each input (e.g., pixel in a picture).
[0027] It is fully connected forming a very dense web and consisting huge number of parameters—resulting in redundancy, inefficiency, difficulties during training and over fitting.
[0028] In light of the foregoing, there is a need for Graphical user interface for detecting malignancy in thyroid nodule using hybrid convolutional neural network that overcomes problems prevalent in the prior art associated with the traditionally available method or system, of the above-mentioned inventions that can be used with the presented disclosed technique with or without modification.
[0029] All publications herein are incorporated by reference to the same extent as if each individual publication or patent application were specifically and individually indicated to be incorporated by reference. Where a definition or use of a term in an incorporated reference is inconsistent or contrary to the definition of that term provided herein, the definition of that term provided herein applies, and the definition of that term in the reference does not apply.

OBJECTS OF THE INVENTION

[0030] The principal object of the present invention is to overcome the disadvantages of the prior art by providing Graphical user interface for detecting malignancy in thyroid nodule using hybrid convolutional neural network.
[0031] An object of the present invention is to provide Graphical user interface for detecting malignancy in thyroid nodule using hybrid convolutional neural network that diagnoses the thyroid malignancy in ultrasound images with high specificity.
[0032] Another object of the present invention is to provide Graphical user interface for detecting malignancy in thyroid nodule using hybrid convolutional neural network that diagnoses the thyroid malignancy in ultrasound images with negative predictive value (NPV).
[0033] Another object of the present invention is to provide a graphical user interface for detecting malignancy in thyroid nodule using hybrid convolutional neural network that diagnoses the thyroid malignancy in ultrasound images with high accuracy.
[0034] Another object of the present invention is to provide a graphical user interface for detecting malignancy in thyroid nodule using hybrid convolutional neural network that diagnoses the thyroid malignancy in ultrasound images with sensitivity, and positive predictive value (PPV).
[0035] Another object of the present invention is to provide a graphical user interface for detecting malignancy in thyroid nodule using hybrid convolutional neural network that diagnoses the thyroid malignancy in ultrasound images with positive predictive value (PPV).
[0036] Another object of the present invention is to provide a graphical user interface for detecting malignancy in thyroid nodule using hybrid convolutional neural network that constructs a hybrid model using different CNN models for thyroid malignancy detection using thyroid nodule ultrasound images.
[0037] Further object of the present invention is to provide a graphical user interface for detecting malignancy in thyroid nodule using hybrid convolutional neural network that designs a Graphical User Interface for the aforementioned optimum hybrid model.
[0038] The foregoing and other objects of the present invention will become readily apparent upon further review of the following detailed description of the embodiments as illustrated in the accompanying drawings.

SUMMARY OF THE INVENTION
[0039] The present invention relates to a graphical user interface for detecting malignancy in thyroid nodule using hybrid convolutional neural network.
[0040] The proposed hybrid model will diagnose the thyroid malignancy in ultrasound images with high specificity, negative predictive value (NPV), accuracy, sensitivity, and positive predictive value (PPV).
[0041] Detection performance of the proposed classification system for predicting thyroid malignancy will be calculated in terms of following parameters-
[0042] Specificity- The measure of classification system’s ability in correct detection of benign cases.
- SPECIFICITY=TNC/ (FPC+ TNC)
- Where, TNC=True negative cases tally; FPC=False positive cases tally
[0043] Negative Predictive Value (NPV) - It is the probability of identification of true negatives concomitantly avoidance of false negatives, while giving result as benign.
- NPV = TNC/ (FNC+TNC)
- Where, TNC=True negative cases tally; FNC=False negative cases tally
[0044] Accuracy- The ratio between the tally of correctly classified cases and the total tally of cases.
- ACCURACY= (TNC+ TPC)/ (TNC+ TPC+ FNC +FPC);
- Where, TNC=True negative cases tally; TPC= True positive cases tally; FNC=False negative cases tally; FPC=False positive cases tally
[0045] Receiver operating characteristic (ROC) graph- The X-axis represents degree of false positives and the Y-axis denotes degree of true positives. This graph illustrates quid pro quo between gains (true positives) and losses (false positives).
[0046] Sensitivity- The measure of classification system’s ability in correct detection of malignant cases.
- SENSITIVITY=TPC/ (TPC+FNC)
- Where, TPC= True positive cases tally; FNC=False negative cases tally
[0047] Positive Predictive Value (PPV)- It is the probability of identification of true positives concomitantly avoidance of false positives, while giving result as malignant.
- PPV = TPC/ (FPC+ TPC)
- Where, TPC= True positive cases tally; FPC=False positive cases tally
[0048] While the invention has been described and shown with reference to the preferred embodiment, it will be apparent that variations might be possible that would fall within the scope of the present invention.

BRIEF DESCRIPTION OF DRAWINGS
[0049] So that the manner in which the above-recited features of the present invention can be understood in detail, a more particular description of the invention, briefly summarized above, may have been referred by embodiments, some of which are illustrated in the appended drawings. It is to be noted, however, that the appended drawings illustrate only typical embodiments of this invention and are therefore not to be considered limiting of its scope, for the invention may admit to other equally effective embodiments.
[0050] These and other features, benefits, and advantages of the present invention will become apparent by reference to the following text figure, with like reference numbers referring to like structures across the views, wherein:
[0051] Figure 1: The Traditional ultrasound Computer-Aided Diagnosis (CAD) system workflow, in accordance with an exemplary embodiment of the present invention;
[0052] Figure 2: Biological neuron;
[0053] Figure 3: An overview of CNN architecture and the training process;
[0054] Figure 4: Overview of proposed work.

DETAILED DESCRIPTION OF THE INVENTION
[0055] While the present invention is described herein by way of example using embodiments and illustrative drawings, those skilled in the art will recognize that the invention is not limited to the embodiments of drawing or drawings described and are not intended to represent the scale of the various components. Further, some components that may form a part of the invention may not be illustrated in certain figures, for ease of illustration, and such omissions do not limit the embodiments outlined in any way. It should be understood that the drawings and the detailed description thereto are not intended to limit the invention to the particular form disclosed, but on the contrary, the invention is to cover all modifications, equivalents, and alternatives falling within the scope of the present invention as defined by the appended claim.
[0056] As used throughout this description, the word "may" is used in a permissive sense (i.e. meaning having the potential to), rather than the mandatory sense, (i.e. meaning must). Further, the words "a" or "an" mean "at least one” and the word “plurality” means “one or more” unless otherwise mentioned. Furthermore, the terminology and phraseology used herein are solely used for descriptive purposes and should not be construed as limiting in scope. Language such as "including," "comprising," "having," "containing," or "involving," and variations thereof, is intended to be broad and encompass the subject matter listed thereafter, equivalents, and additional subject matter not recited, and is not intended to exclude other additives, components, integers, or steps. Likewise, the term "comprising" is considered synonymous with the terms "including" or "containing" for applicable legal purposes. Any discussion of documents, acts, materials, devices, articles, and the like are included in the specification solely for the purpose of providing a context for the present invention. It is not suggested or represented that any or all these matters form part of the prior art base or were common general knowledge in the field relevant to the present invention.
[0057] In this disclosure, whenever a composition or an element or a group of elements is preceded with the transitional phrase “comprising”, it is understood that we also contemplate the same composition, element, or group of elements with transitional phrases “consisting of”, “consisting”, “selected from the group of consisting of, “including”, or “is” preceding the recitation of the composition, element or group of elements and vice versa.
[0058] The present invention is described hereinafter by various embodiments with reference to the accompanying drawing, wherein reference numerals used in the accompanying drawing correspond to the like elements throughout the description. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiment set forth herein. Rather, the embodiment is provided so that this disclosure will be thorough and complete and will fully convey the scope of the invention to those skilled in the art. In the following detailed description, numeric values and ranges are provided for various aspects of the implementations described. These values and ranges are to be treated as examples only and are not intended to limit the scope of the claims. In addition, several materials are identified as suitable for various facets of the implementations. These materials are to be treated as exemplary and are not intended to limit the scope of the invention.
[0059] The present invention relates to a graphical user interface for detecting malignancy in thyroid nodule using hybrid convolutional neural network.
[0060] Discovery of CNN by Yann Lecunn revolutionarized the sphere of deep learning, which mimics human brain working in data processing and pattern recognition for decision making. CNN has fully trainable models and it is a distinctive cutting-edge deep learning technique. It demonstrates outstanding feat in resolving computer vision problems because unlike other neural network models in this the multiple arrays are acquired as input and then with the application of convolution operator within a narrow field input are processed.
Table 1: CNN model summary

[0061] Table 1 shows different models of CNN. Despite variations, all share four fundamental features weight sharing, local connection, employment of many layers, and pooling.
[0062] Most commonly used layers are-
[0063] Input layer- It receives multiple arrays as input.
[0064] Convolutional layer- A filter slides on top of the image, sighting small number of pixels at a time and computing dot product of initial pixel values with weight characterized in a filter. Afterward, application of activation function generates Feature map that predicts the class to which each feature fit in. Activation function is a mathematical equation, which helps in learning complex features in an image.
[0065] Subsampling Layer (pooling layer)- It works on each feature map independently and decreases the spatial size of the depiction while maintaining the most significant information. As a result, the quantity of parameters and computation are also reduced. Max pooling and average pooling are the most commonly used methods for subsampling.
[0066] Fully-connected Layer- It connects neurons in one layer to another layer. Input fully connected layer flattens the output of the preceding layer and change them into a distinct vector. The next fully connected layer applies weights to classify images between different categories.
[0067] Output Layer- It is fully connected to previous layer and provides the final probabilities for each label.[25-27]
[0068] Advantages:
[0069] In comparison to traditional machine learning algorithms, it does not require hand-crafted feature extraction.
[0070] At the input layer CNN input does not require centralization and size-normalization because it is translation invariant.
[0071] Unlike ANN, it generates just adequate weights to scan a small area of the image at any instance. Parameter sharing makes process more efficient in terms of memory, training speed and complexity.
[0072] The review of literature has been divided into two parts. The first part has the purpose of pointing out drawbacks of existing technologies in thyroid malignancy detection that creates the space for use of convolutional neural network (CNN) technique. The second part provides the overview of work done on the thyroid malignancy detection using CNN technique to expose the unexplored area or research gap.
[0073] Existing technologies for thyroid malignancy detection-
[0074] Grani G et al. have attempted numerical estimates through histogram analysis of thyroid echogenicity for stratification of ultrasound features to distinguish benign nodules from malignant nodules.
[0075] Substandard diagnostic performance of histogram analysis compared to subjective analysis by radiologists makes it less appealing.
[0076] Chang et al. have selected 78 texture features and used six Support Vector Machines (SVMs) to classify the nodule lesions of a thyroid. The execution time has been found 3-37 times faster than sequential-floating-forward-selection (SFSS) method and accuracy has also been observed higher than Bhattacharyya distance, T-test, entropy, and F-score methods.
[0077] Gopinath et al. have extracted ROI by mathematical segmentation method. Then Gabor filter-based method has been applied for taking out statistical textural features of segmented images. The diagnostic sensitivity of 95%, accuracy of 96.7%, and specificity of 100% have been achieved from SVM classifier.
[0078] Sudarshan et al. have employed Discrete Wavelet Transform, a scale space technique for decomposing images into images at different scales. Intensity changes detected by high pass and low pass filters have been reflected in wavelet coefficients, which have been further used for pulling out statistical, image based and textural features. Supervised classifiers have been applied to discriminate malignant and thyroid lumps on the basis of significant features identified by statistical analysis.
[0079] Raghavendra et al. have used graph supported Marginal Fisher Analysis (MFA) on fractal textures and Spatial Gray Level Dependence Features (SGLDF) for reducing number of feature and then have applied classifiers to achieve the average accuracy of 97.52% in differentiating benign and malignant lesions.
[0080] Ouyang et al. have discovered similar diagnostic capabilities of linear and non-linear machine learning algorithms in assessment of malignancy risk.
[0081] The features selection is subjected to the knowledge of author due to design of handcrafted image feature extractors in traditional machine learning based Computer-Aided-Diagnosis systems (CAD). The performance is limited by the partial selection of features covering small facet of problem.
[0082] Zhu et al. have developed a three-layer 6-8-1 feed-forward ANN model by using six ultrasound-based features incorporating margin, shape, echogenicity, presence of calcification, internal composition, and peripheral halo as input neurons. The sensitivity of 83.8%, accuracy of 83.1%, and specificity of 81.8% have been achieved in validation cohort.
[0083] Unmanageable quantity of weights in ANN results in inefficiency, redundancy and overfitting.
[0084] CNN for thyroid malignancy detection-
[0085] Ma et al. have developed a model centered on cascade deep convolutional neual networks (CNNs) comprising two dissimilar CNNs and a novel splitting method. First CNN has used the ground correct data and gained the knowledge of the segmentation probability maps. Then splitting has been employed to divide the segmentation probability maps into diverse related sections. Lastly, for the automatic detection of thyroid tumors from the sonographic thyroid images the second CNN has been employed. The model has achieved AUC value of 98.51%.
[0086] Ko SY et al. have designed deep convolutional neual network that has demonstrated similar diagnostic performance compared to experienced radiologists with specificity of 82.0%-90.0%, accuracy of 86.0%-88.0%, and PPVs of 90.9%-94.4% in differentiating thyroid malignancy on US.
[0087] Similar diagnostic performance of CNNs and radiologist attracts further research work in this field.
[0088] Liu et al. have transferred a CNN model learned dataset to the novel ultrasound image dataset from substantial ordinary dataset to manage the small sample problem and extract high-level deep features and. Semantic deep features have been united with traditional low-level features obtained to develop a mixed feature plot. This proposed feature-fusion method has achieved 93.10% accuracy.
[0089] Transfer learning involving the pretrained model reuse on separate dataset has demonstrated promising result.
[0090] Chi et al. have extracted features through fine-tuning the pre-trained GoogleLeNet model, and then the features have been supplied to a Random Forest classifier to categorize the images into cancerous and non-cancerous cases. The proposed model used the images of an open access database and has achieved a sensitivity of 99.10%, accuracy of 98.29%, and specificity of 93.90%.
[0091] Moussa et al. have fine-tuned the resNet-50 model which outperformed the VGG-19 model with accuracy of 97.33% and sensitivity of 80.69%.
[0092] Fine-tunining of pretrained convolutional neural network’s weights provides better results than other baseline CNNs.
[0093] Li et al. have developed the hybrid model by using ResNet-50 and Darknet-19 that showed improved performance in identifying thyroid cancer with sensitivity of 84.3% to 93.4% and specificity of 86.1% to 87.8% compared to skilled radiologists.
[0094] Nguyen et al. have used weighted binary cross-entropy loss function for the purpose of training the deep convolutional neural networks and combined multiple CNN-based models for superior information extraction. The proposed hybrid model has achieved 92.05% accuracy.
[0095] Different combinations of CNN models have been tried for further improvement in diagnostic performance. But, still there is a huge space left for diverse combinations.
[0096] Comparison of different CNN models for thyroid malignancy detection is missing in existing work. The optimization after the comparison is yet to be explored. Deep fine-tuning has already given promising results, which asks for further study. Despite the encouraging results of mixed models, there is a wide uncharted region of hybrid models for thyroid malignancy detection.
[0097] It includes the procedure for achieving the aforementioned objectives as well as takes account of performance indices to assess the performance of proposed model for thyroid nodule malignancy detection.
[0098] Various modifications to these embodiments are apparent to those skilled in the art from the description and the accompanying drawings. The principles associated with the various embodiments described herein may be applied to other embodiments. Therefore, the description is not intended to be limited to the 5 embodiments shown along with the accompanying drawings but is to be providing the broadest scope consistent with the principles and the novel and inventive features disclosed or suggested herein. Accordingly, the invention is anticipated to hold on to all other such alternatives, modifications, and variations that fall within the scope of the present invention and appended claims.
, Claims:We Claim:

1) A graphical user interface for detecting malignancy in thyroid nodule using hybrid convolutional neural network, the system diagnoses the thyroid malignancy in ultrasound images with high specificity, negative predictive value (NPV), accuracy, sensitivity, and positive predictive value (PPV).

2) The system as claimed in claim 1, wherein the detection performance of the proposed classification system for predicting thyroid malignancy is calculated in terms of Specificity, wherein the measure of classification system’s ability in correct detection of benign cases.; wherein:
- SPECIFICITY=TNC/ (FPC+ TNC)
- Where, TNC=True negative cases tally; FPC=False positive cases tally.

3) The system (100) as claimed in claim 1, wherein the detection performance of the proposed classification system for predicting thyroid malignancy is calculated in terms of Negative Predictive Value (NPV), wherein is the probability of identification of true negatives concomitantly avoidance of false negatives, while giving result as benign, wherein:
- NPV = TNC/ (FNC+TNC) ;
- TNC=True negative cases tally; FNC=False negative cases tally.

4) The system (100) as claimed in claim 1, wherein the detection performance of the proposed classification system for predicting thyroid malignancy is calculated in terms of Accuracy, wherein the ratio between the tally of correctly classified cases and the total tally of cases, wherein:
- ACCURACY= (TNC+ TPC)/ (TNC+ TPC+ FNC +FPC);
- Where, TNC=True negative cases tally; TPC= True positive cases tally; FNC=False negative cases tally; FPC=False positive cases tally.

5) The system (100) as claimed in claim 1, wherein the detection performance of the proposed classification system for predicting thyroid malignancy is calculated in terms of Receiver operating characteristic (ROC) graph, wherein the X-axis represents degree of false positives and the Y-axis denotes degree of true positives.

6) The system (100) as claimed in claim 1, wherein the detection performance of the proposed classification system for predicting thyroid malignancy is calculated in terms of Sensitivity- wherein the measure of classification system’s ability in correct detection of malignant cases, wherein:
- SENSITIVITY=TPC/ (TPC+FNC);
- Where, TPC= True positive cases tally; FNC=False negative cases tally.

7) The system (100) as claimed in claim 1, wherein the detection performance of the proposed classification system for predicting thyroid malignancy is calculated in terms of Positive Predictive Value (PPV)- It is the probability of identification of true positives concomitantly avoidance of false positives, while giving result as malignant, wherein:
- PPV = TPC/ (FPC+ TPC);
- Where, TPC= True positive cases tally; FPC=False positive cases tally.