Description:FORM 2
THE PATENTS ACT, 1970
(39 of 1970)
&
THE PATENT RULES, 2003

COMPLETE SPECIFICATION
(See Section 10 and Rule 13)

Title of invention:
A SYSTEM AND METHOD FOR DETECTING LUNG ABNORMALITIES IN A MEDICAL IMAGE

Applicant:
Qure.ai Technologies Private Limited
An Indian Entity having address as:
Level 7, Commerz II, International Business Park, Oberoi Garden
City, Off Western Express Highway, Goregaon (East), Mumbai 400063,
Maharashtra, India

The following specification particularly describes the invention and the manner in which it is to be performed.
CROSS-REFERENCE TO RELATED APPLICATIONS AND PRIORITY
The present application does not claim priority from any other patent application.
FIELD OF INVENTION
The present subject matter described herein, in general, relates to management of medical abnormalities. More specifically, the present invention relates to detection of lung abnormalities. More particularly, the present invention relates to artificial intelligence (AI) enabled clinical decision support system for detection and management of lung abnormalities on a medical image.
BACKGROUND OF THE INVENTION
This section is intended to introduce the reader to various aspects of art, which may be related to various aspects of the present disclosure that are described or claimed below. This discussion is believed to be helpful in providing the reader with background information to facilitate a better understanding of the various aspects of the present disclosure. Accordingly, it should be understood that these statements in this background section are to be read in this light, and not as admissions of prior art. Similarly, a problem mentioned in the background section or associated with the subject matter of the background section should not be assumed to have been previously recognized in the prior art. The subject matter in the background section merely represents different approaches, which in and of themselves may also correspond to implementations of the claimed technology.
Artificial intelligence has revolutionized the healthcare industry by enabling the analysis of patient data, whether in the form of text or medical images. AI-driven systems can swiftly and accurately interpret complex medical records and diagnostic images, aiding healthcare professionals in making more informed decisions. Specifically, the usage of imaging AI for scanning medical images can detect lung abnormalities which may either overlooked or difficult for human experts to identify. However, challenges persist in conventional systems. In text analysis, the sheer volume of unstructured patient data poses difficulties for manual processing, potentially leading to overlooked insights. In image analysis, AI models may require extensive and high-quality datasets for training, which can be scarce. Moreover, concerns about patient data privacy and model interpretability must be addressed to ensure ethical and secure AI integration into healthcare.
Further, Personalization is a cornerstone of effective disease management. Each patient's medical history, genetics, lifestyle, and preferences play a pivotal role in determining the most suitable interventions. Empowering patients with personalized care plans encourage greater adherence to treatment regimens, leading to improved health care and reduced complications.
Traditional healthcare decision support system lacks personalized disease management due to leveraging a single AI models (specifically Imaging AI) for analysing user’s medical image. The usage of image-based AI model to identify medical abnormalities are solely based on real-time inputs from the medical image scanned by the imaging AI model. However medical history and current medical condition of the user may have an impact on the medical abnormalities which may or may not be captured in the observations made through scanning the medical image solely. Therefore, a need for analysing the medical history or reports along with user’s medical image is expected to perform comprehensive analysis on the user data.
Conventionally healthcare system utilizing the NLP (natural language processing) model has several drawbacks. It often struggles with understanding and interpreting nuanced medical language, leading to potential misinterpretations and errors in clinical documentation. NLP systems can be rigid, requiring constant updates to adapt to evolving medical terminology and language patterns. They may lack the contextual awareness and comprehension capabilities that large language models possess, limiting their ability to extract meaningful insights from complex medical texts. Additionally, building and maintaining NLP-based systems can be resource-intensive and time-consuming, making them less efficient and costly approach.
Further, the absence of large language models (LLMs) such as GPT (generative pre-trained transformer) models in the analysis of patient medical health presents significant shortcomings in healthcare. GPT models, with their natural language understanding capabilities, could greatly enhance the interpretation of unstructured patient data, such as clinical notes and medical histories. Without their use, healthcare providers may struggle to extract valuable insights from these textual sources, potentially missing critical information that could influence patient diagnoses and treatment decisions. GPT models can also aid in automating tasks like medical coding and summarizing complex patient records, saving time and reducing human errors. Nevertheless, the promise of artificial intelligence in healthcare remains significant, offering the potential to enhance diagnosis, treatment, and patient outcomes.
Thus, there exists a need of an innovative approach to detect lung abnormalities in a medical image, that integrates hybrid Artificial Intelligence (AI) technologies with personalized patient diagnosis and care. Furthermore, this holistic framework combines data-driven insights with empathetic human interactions to create a comprehensive solution that addresses the complexities of health and helps to enhance patient outcomes and quality of life.
SUMMARY OF THE INVENTION
Before the present system and device and its components are described, it is to be understood that this disclosure is not limited to the system and its arrangement as described, as there can be multiple possible embodiments which are not expressly illustrated in the present disclosure. The present disclosure overcomes one or more shortcomings of the prior art and provides additional advantages discussed throughout the present disclosure. Additional features and advantages are realized through the techniques of the present disclosure. It is also to be understood that the terminology used in the description is for the purpose of describing the versions or embodiments only and is not intended to limit the scope of the present application. This summary is not intended to identify essential features of the claimed subject matter nor is it intended for use in detecting or limiting the scope of the claimed subject matter.
In one embodiment of the present disclosure, a system for the detection of lung abnormalities in a medical image is disclosed. The system includes a memory and a processor coupled with the memory. The processor may be configured to execute programmed instructions stored in the memory. Further, the system may comprise a data collection module. In one embodiment, data collected by the data collection module may correspond to a medical image and a patient clinical data. Further, the system may comprise an abnormality detection module for identifying lung abnormalities based on the collected data. The abnormality detection module may be configured to use a hybrid machine learning system for identifying lung abnormalities based on the collected data. In one exemplary embodiment, the hybrid machine learning system may correspond to combination of an imaging model and a large language model (LLM). Further, the hybrid machine learning system may comprise a step of detecting lung abnormality by analysing the medical image using the imaging model. Further, the hybrid machine learning system may comprise a step of calculating an image confidence score corresponding to the detected lung abnormality. Further, the hybrid machine learning system may comprise a step of extracting patient related information by using a language model. Further, the hybrid machine learning system may comprise a step of calculating a patient confidence score corresponding to the extracted patient related information. Further, the hybrid machine learning system may comprise a step of generating an abnormality confidence score by combining the image confidence score and the patient confidence score, enhancing the reliability of the diagnosis. The abnormality confidence score may correspond to overall confidence in the abnormality being detected and the diagnosis/treatment pathway being provided. Furthermore, the system may comprise an output generation module for generating an output. The generated output may combine the output of both the imaging model and the large language model (LLM). The generated output may corresponds to the lung abnormality with a comprehensive diagnosis and the abnormality confidence score. Further, the output generated by the output generation module may corresponds to personalized treatment and diagnosis recommendation based on patient medical history and clinical data analyzed by the LLM. In an exemplary embodiment, the output may correspond to an abnormality diagnostic report. The abnormality diagnostic report may comprise the medical image with one or more visual bounding boxes to annotate the one or more lung abnormalities detected by the system. In an embodiment, the abnormality detection report is configured to be extracted in multiple readable formats, which includes the detected abnormalities in greater details including but not limited to the medical image with one or more visual bounding boxes, corresponding labels to annotate the one or more detected lung abnormalities, a diagnosis report, a treatment recommendation report, the abnormality confidence score and a brief explanation on relevance of the abnormality confidence score. This system holds significant potential for enhancing the accuracy and efficiency of lung abnormality detection in medical practice.
In one embodiment, the system may utilize the medical image data in various forms namely x-ray scans, CT scans, MRI scans, ultrasound images, and more. Additionally, the patient clinical data used by the system may encompass a wide range of sources such as patient history records, electronic health records (EHR), clinical research data, and updated medical device or healthcare guidelines available on public platforms, allowing for a comprehensive assessment of the patient's health. The clinical research data incorporated into the system may be specifically tailored to the domain of lung abnormalities, enhancing its relevance and precision. In an embodiment, the abnormality detection module may be configured to train the imaging model and the language model using the data collected by the data collection module.
In another embodiment, the imaging model may correspond to a deep learning algorithm for analysing the medical images for detecting lung abnormalities. The lung abnormalities detected using the imaging model may corresponds to one of Asthma, Chronic obstructive pulmonary disease (COPD), Bronchiectasis, Bronchitis, pneumothorax, atelectasis, lung inflammation, Pulmonary fibrosis, Sarcoidosis, Lung cancer, Lung Infection (Pneumonia), Hyperinflation/ Emphysema, Consolidation, Opacity, Scoliosis, Fibrosis, Tuberculosis screening, Atelectasis, Reticulo-nodular pattern, Nodules, Cavity, Calcification, Linear Opacities, Lung Nodule Malignancy, Covid-19 risk and a combination thereof. Further, the imaging model, along with the lung abnormalities, may detect other abnormalities, including but not limited to, Cardiomegaly, Degenerative spine conditions, Prominence in Hilar region, Blunted Costophrenic Angle, Pleural effusion, Tracheal shift, Pneumothorax, Elevated Hemidiaphragm, Pneumoperitoneum, Rib Fractures, Mediastinal Widening, Presence of Tracheal Tube, Presence of Gastric Tube, Placement of Tracheal Tube, Placement of Gastric Tube. Along with detecting the lung abnormalities, the imaging model may be configured to calculate the image confidence score based on the analysis of the medical image.
In yet another embodiment, the language model may correspond to one of Generative Pre-trained Transformer (GPT) model, a Large Language Model (LLM), and a combination thereof, for extracting the patient related information. The patient related information extracted by the language model may correspond to one of identified risk factors, underlying cause of lung disease and a combination thereof. Along with extraction of patient related information, the language model may be configured to calculate the patient confidence score corresponding to the extracted patient related information. Further, the language model may be configured to suggest treatment recommendation to the user based on extracted patient related information from the patient history records, electronic health records (EHR), the clinical research data, and the updated medical device or healthcare guidelines available on the public platforms.
In yet another embodiment, the system may enables monitoring the lung abnormalities over time. Monitoring the lung abnormalities may help in tracking disease progression and adjustment of treatment plan. Further, the system may be configured to provide accurate and up-to-date lung abnormality diagnosis based on continuously learning from clinical evidence over time and adapting the system accordingly. The system is configured to be self-evolving the analysis, to be performed by the system, based on the output analysis previously resulted by the system and any subsequent feedback provided, to the system, by the user.
In another embodiment of the present disclosure, a method for detecting lung abnormalities in a medical image is disclosed. The method is a comprehensive approach that involves several key stages. The method may comprise a step for collecting data using a data collection module. The collected data may correspond to the medical image and the patient clinical data. Further, the method may comprise a step of analysing the collected data by an abnormality detection module using a hybrid machine learning system. The hybrid machine learning system may correspond to combination of an imaging model and a large language model (LLM). Further, the hybrid machine learning system may comprise a step of detecting lung abnormality by analysing the medical image using the imaging model. Further, the hybrid machine learning system may comprise a step of calculating an image confidence score corresponding to the detected lung abnormality. Further, the hybrid machine learning system may comprise a step of extracting patient related information by using a language model. Further, the hybrid machine learning system may comprise a step of calculating a patient confidence score corresponding to the extracted patient related information. Further, the hybrid machine learning system may comprise a step of generating an abnormality confidence score by combining the image confidence score and the patient confidence score. The abnormality confidence score may correspond to overall confidence in the abnormality being detected and the diagnosis/treatment pathway being provided. Furthermore, the method may comprise a step of synthesizes this information into a comprehensive output, providing healthcare professionals with a holistic evaluation that incorporates both the image analysis and clinical data, thus improving the accuracy and confidence in detecting lung abnormalities. The generated output may combine the output of both the imaging model and the large language model (LLM). The generated output may corresponds to the lung abnormality with a comprehensive diagnosis and the abnormality confidence score. Further, the output generated by the output generation module may corresponds to personalized treatment and diagnosis recommendation based on patient medical history and clinical data analyzed by the LLM. In an exemplary embodiment, the output may correspond to an abnormality diagnostic report. The abnormality diagnostic report may comprise the medical image with one or more visual bounding boxes to annotate the one or more lung abnormalities detected by the system. In an embodiment, the abnormality detection report is configured to be extracted in multiple readable formats, which includes the detected abnormalities in greater details including but not limited to the medical image with one or more visual bounding boxes, corresponding labels to annotate the one or more detected lung abnormalities, a diagnosis report, a treatment recommendation report, the abnormality confidence score and a brief explanation on relevance of the abnormality confidence score.
In one embodiment, the method may include various aspects of data collection and analysis. A medical image data collection encompasses a wide range of modalities such as x-ray scans, CT scans, MRI scans, and ultrasound images, and more, while a patient clinical data collection includes patient history records, electronic health records (EHR), clinical research data, particularly focused on lung abnormalities, and updated medical device or healthcare guidelines available on public platforms. The analysis of medical images is carried out using deep learning algorithms within the imaging model, and the image confidence score is calculated based on this analysis. Further, the patient clinical data analysis may include GPT or large language models (LLM), extracting patient relevant information from unstructured clinical text and recommending treatment options, with a patient confidence score calculated accordingly. The overall abnormality confidence score is determined through a combination formula involving both the imaging and language models. Ultimately, the method may generate a comprehensive output including a diagnosis, confidence score, and tailored treatment recommendations based on the collected data, enhancing clinical decision-making in pulmonary health.
The foregoing summary is illustrative only and is not intended to be in any way limiting. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features will become apparent by reference to the drawings and the following detailed description.
BRIEF DESCRIPTION OF DRAWINGS
The detailed description is described with reference to the accompanying figures. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. The same numbers are used throughout the drawings to refer like features and components.
Figure 1 illustrates a block diagram describing a system (100) for detecting lung abnormalities in a medical image (101), in accordance with an embodiment of a present subject matter.
Figure 2 illustrates a block diagram (200) showing an overview of a server (104) for detecting lung abnormalities in a medical image (101), in accordance with an embodiment of a present subject matter; and
Figure 3 illustrates a flowchart describing a method (300) for detecting lung abnormalities in a medical image (101) using a hybrid machine learning system, in accordance with an embodiment of the present subject matter.
Figure 4 illustrates a lung abnormality report (400) generated by an output generation module (209) of the system (100) for detecting lung abnormalities in the medical image (101), in accordance with an embodiment of the present subject matter.
DETAILED DESCRIPTION OF THE INVENTION
Reference throughout the specification to “various embodiments,” “some embodiments,” “one embodiment,” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. Thus, appearances of the phrases “in various embodiments,” “in some embodiments,” “in one embodiment,” or “in an embodiment” in places throughout the specification are not necessarily all referring to the same embodiment. Furthermore, the features, structures or characteristics may be combined in any suitable manner in one or more embodiments.
The words "comprising," "having," "containing," and "including," and other forms thereof, are intended to be equivalent in meaning and be open ended in that an item or items following any one of these words is not meant to be an exhaustive listing of such item or items or meant to be limited to only the listed item or items. It must also be noted that, the singular forms "a" "an" and "the" include plural references unless the context clearly dictates otherwise. Although any methods similar or equivalent to those described herein can be used in the practice or testing of embodiments of the present disclosure, the exemplary methods are described. The disclosed embodiments are merely exemplary of the disclosure, which may be embodied in various forms.
The integration of artificial intelligence (AI) techniques in medical healthcare has revolutionized the industry by leveraging advanced algorithms and machine learning to analyse vast datasets, enhance diagnostic accuracy, personalize treatment plans, and streamline administrative processes.
Further, the transformer based deep learning models such as large language models (LLMs), GPT (generative pre-trained transformer) model has advanced medical healthcare systems by leveraging its natural language understanding capabilities. It can efficiently analyse vast volumes of medical literature, patient records, and clinical notes to extract valuable insights and assist in diagnosis, treatment recommendations, and medical research. GPT's ability to understand and generate human-like text and adapt to evolving medical terminology makes it a powerful tool for enhancing communication among healthcare professionals and patients.
However, a conventional healthcare system based on the NLP (natural language processing) model has several drawbacks. It often struggles with understanding and interpreting nuanced medical language, leading to potential misinterpretations and errors in clinical documentation. NLP systems can be rigid, requiring constant updates to adapt to evolving medical terminology and language patterns. They may lack the contextual awareness and comprehension capabilities that large language models possess, limiting their ability to extract meaningful insights from complex medical texts. Additionally, building and maintaining NLP-based systems can be resource-intensive and time-consuming, making them less efficient and cost-effective compared to newer LLM approaches in the healthcare.
In the light of the above-mentioned limitations, the large language models (LLMs) like GPT offer significant advantages over traditional natural language processing (NLP) methods in the context of detecting lung abnormalities in healthcare. LLMs excel at understanding and contextualizing complex medical language, allowing them to analyse unstructured clinical text with greater accuracy. They adapt to evolving medical terminology and language patterns, reducing the need for constant rule-based updates. LLMs can also integrate multimodal data, such as medical images and clinical notes, for a more comprehensive analysis, improving diagnostic precision. Furthermore, these models can be fine-tuned for specific healthcare tasks, enhancing their performance. Overall, LLMs offer a more versatile, adaptable, and powerful approach to lung abnormality detection in healthcare.
In one non-limiting embodiment, a system for the detection of lung abnormalities in the medical image, by using a hybrid machine learning system is disclosed. The system may include variety of data collected in form of a medical image data and a patient clinical data. The hybrid machine learning system corresponds to combination of deep learning-based imaging model and LLM based language model, which helps in identifying the lung abnormalities from the medical image in an intelligent manner. The system may be capable of learning from its operation, improving its abnormality detection capabilities over time. The system can learn from each interaction, continuously refining its understanding of individual medical parameters and corresponding effects on the lungs, and ultimately, the efficiency of the lung abnormality it generates.
Now referring to figure 1, a block diagram describing a system (100) for detecting lung abnormalities in a medical image (101), is illustrated in accordance with an embodiment of a present subject matter. The system (100) may include variety of data collected in form of a medical image (101) data and a patient clinical data (102). The collected data is coupled with the server (104) via a network (103) connection. Further, considering that the system (100) is implemented on a server (104), it may be understood that the system (100) may be accessed via a variety of user devices (105). The user device (105) may correspond to an interface which enables the user to interact with the system (100). The user device (105) may comprise one selected from a group consisting of a cell phone, personal digital assistant (PDA), laptop computer, stationary personal computer, IPTV remote control, web tablet, laptop computer, pocket PC, a television set capable of receiving IP based video services and mobile IP device. In an embodiment, the system (100) may be configured to receive user data from one or more users, via the user device (105). The collected data may be in form of the medical image (101) data and the patient clinical data (102). In an exemplary embodiment, the medical image (101) corresponds to a chest image data in form of one of x-ray scans, CT scans, MRI scans, ultrasound images, and a combination thereof. In another exemplary embodiment, the patient clinical data (102) corresponds to one of a patient history records, an electronic health records (EHR), clinical research data, updated medical device or healthcare guidelines available on public platforms, and a combination thereof. The clinical research data corresponds to research information published in domain of lung abnormalities. Both medical image (101) and the patient clinical data (102) may be used by the system (100) to detect ling abnormalities with an accurate and comprehensive diagnosis of the patient. In an embodiment, medical image (101) and the patient clinical data (102) are collected by the server (104) via the network (103), is transmitted to a memory (203) (as illustrated in Figure 2) for storage. Further, the functionality and other characteristics of the server (104) would be provided in description of Figure 2.
In yet another embodiment, the collected data (101, 102), the server (104) and the user device (105) may communicate with each other via the network (103). In one implementation, the network (103) may be a wireless network, a wired network, or a combination thereof. The network (103) can be implemented as one of the different types of networks, such as intranet, local area network (LAN), wide area network (WAN), the internet, and the like. The network (103) may either be a dedicated network or a shared network. The shared network represents an association of the different types of networks that use a variety of protocols, for example, Hypertext Transfer Protocol (HTTP), Transmission Control Protocol/Internet Protocol (TCP/IP), Wireless Application Protocol (WAP), and the like, to communicate with one another. Further, the network (103) may include a variety of network devices, including routers, bridges, servers, computing devices, storage devices, and the like.
In another embodiment, the network (103) may include any one of the following: a cable network, the wireless network, a telephone network (e.g., Analog, Digital, POTS, PSTN, ISDN, xDSL), a cellular communication network, a mobile telephone network (e.g., CDMA, GSM, NDAC, TDMA, E-TDMA, NAMPS, WCDMA, CDMA-2000, UMTS, 3G, 4G, 5G, 6G), a radio network, a television network, the Internet, the intranet, the local area network (LAN), the wide area network (WAN), an electronic positioning network, an X.25 network, an optical network (e.g., PON), a satellite network (e.g., VSAT), a packet-switched network, a circuit-switched network, a public network, a private network, and/or other wired or wireless communications network configured to carry data.
The system (100) can be implemented using hardware, software, or a combination of both, which includes using where suitable, one or more computer programs, mobile applications, or “apps” by deploying either on-premises over the corresponding computing terminals or virtually over cloud infrastructure. The system (100) may include various micro-services or groups of independent computer programs which can act independently in collaboration with other micro-services. The system (100) may also interact with a third-party or external computer system. Internally, the system (100) may be the central processor of all requests for transactions by the various actors or users of the system. Aa critical attribute of the system (100) is that it can concurrently and instantly complete an online transaction by a system user in collaboration with other systems.
Now, referring to figure 2, a block diagram (200) showing an overview of the server (104) for detecting lung abnormalities in the medical image (101), is illustrated in accordance with an embodiment of a present subject matter. The server (104) includes a processor (201), an input/output (I/O) interface (202), and the memory (203). The processor (201) is coupled with the memory (203). The processor (201) is configured to execute programmed instructions stored in the memory (203). The processor, in one embodiment, may comprise a standard microprocessor, microcontroller, central processing unit (CPU), distributed or cloud processing unit, state machines, logic circuitries, and/or any devices that manipulate signals based on operational instructions and/or other processing logic that accommodates the requirements of the present invention.
Further, the I/O interface (202) is an interface to other components of the server (104) and the system (100). The I/O interface (202) may include a variety of software and hardware interfaces, for example, a web interface, a graphical user interface, and the like. The I/O interface (202) may allow the system (100) to interact with the user directly or through the user devices (105). Further, the I/O interface (202) may enable the system (100) to communicate with other computing devices, such as web servers and external data servers (not shown). The I/O interface (202) can facilitate multiple communications within a wide variety of networks and protocol types, including wired networks, for example, LAN, cable, etc., and wireless networks, such as WLAN, cellular, or satellite. The I/O interface (202) may include one or more ports for connecting a number of devices to one another or to another server. In one embodiment, the I/O interface (202) allow the server (104) to be logically coupled to other user device (105), some of which may be built in. Illustrative components include tablets, mobile phones, scanner, printer, wireless device, etc. Further, the processors (201) can read data from various entities such as memory (203) or I/O interface (202). The processor’s (201) primary functions encompass data acquisition, wherein it gathers medical image data (101) and patient clinical data (102). Following this, the collected data undergoes analysis through specialized modules, and the system (100) computes a overall confidence score. Ultimately, it generates an output, which typically includes a diagnosis or evaluation on the lung abnormalities, informed by the comprehensive analysis of both the medical image and the patient clinical data. In an exemplary embodiment, the output may correspond to an abnormality diagnostic report. The abnormality diagnostic report may comprise the medical image with one or more visual bounding boxes to annotate the one or more lung abnormalities detected by the system (100). In an embodiment, the abnormality detection report is configured to be extracted in multiple readable formats, which includes the detected abnormalities in greater details including but not limited to the medical image with one or more visual bounding boxes, corresponding labels to annotate the one or more detected lung abnormalities, a diagnosis report, a treatment recommendation report, the abnormality confidence score and a brief explanation on relevance of the abnormality confidence score.
The memory (203) may include any computer-readable medium or computer program product known in the art including, for example, volatile memory, such as static random-access memory (SRAM) and dynamic random-access memory (DRAM), and/or non-volatile memory, such as read only memory (ROM), erasable programmable ROM, flash memories, hard disks, Solid State Disks (SSD), optical disks, magnetic tapes, memory cards, virtual memory and distributed cloud storage. The memory (203) may be removable, non-removable, or a combination thereof. The memory (203) may include routines, programs, objects, components, data structures, etc., which perform particular tasks or implement particular abstract data types. The memory (203) may include programs or coded instructions that supplement applications and functions of the system (100). In one embodiment, the memory (203), amongst other things, serves as a repository for storing data processed, received, and generated by one or more of the programs or the coded instructions. In yet another embodiment, the memory (203) may be managed under a federated structure that enables adaptability and responsiveness of the server (104). The memory further may include various modules (204) namely a data collection module (205), an abnormality detection module (206) and an output generation module (209). The abnormality detection module (206) may further include an imaging model (207) and a language model (208). In one embodiment, the server (104) utilizes the processor (201) for executing the various modules (204) stored in the memory (203).
The data collection module (205) is configured to collect data corresponding to the medical image (101) and the patient clinical data (102). In an exemplary embodiment, the medical image (101) corresponds to a chest image data in form of one of x-ray scans, CT scans, MRI scans, ultrasound images, and a combination thereof. In another exemplary embodiment, the patient clinical data (102) corresponds to one of a patient history records, an electronic health records (EHR), clinical research data, updated medical device or healthcare guidelines available on public platforms and a combination thereof. The clinical research data corresponds to research information published in domain of lung abnormalities. The data collected by the data collection module (205) is utilized by the abnormality detection module (206) for detecting the lung abnormalities. In one exemplary embodiment, the abnormality detection module (206) is implemented by using a hybrid machine learning system. In one implementation, the abnormality detection module (206) analyses the imaging data (101) and the patient clinical data (102) by using the hybrid machine learning system and generates an output regarding detection of lung abnormalities in the medical images with comprehensive diagnosis. In an embodiment, the abnormality detection module (206) may utilize the processor (201) to implement the hybrid machine learning system. The hybrid machine learning system corresponds to combination of the imaging model (207) and the language model (208). In a related implementation, the abnormality detection module (206) is configured to train the hybrid machine learning system, including the imaging model (207) and the language model (208), using the imaging data (101) and the patient clinical data (102).
In one embodiment, the imaging model (207), with the help of processor (201), is configured to analyse the medical images (101) to detect lung abnormality based on the medical image (101). In one embodiment, the imaging model (207) corresponds to a deep learning algorithm. The imaging model (207) may further be configured to calculate an image confidence score corresponding to the detected lung abnormality. The image confidence score gives the probability of detected lung abnormality in the medical image (101) being detected correctly by the algorithm of the imaging model (207) and may be given as a percentage. Further, the lung abnormalities detected using the imaging model (207) corresponds to one of Asthma, Chronic obstructive pulmonary disease (COPD), Bronchiectasis, Bronchitis, pneumothorax, atelectasis, lung inflammation, Pulmonary fibrosis, Sarcoidosis, Lung cancer, Lung Infection (Pneumonia), Hyperinflation/ Emphysema, Consolidation, Opacity, Scoliosis, Fibrosis, Tuberculosis screening, Atelectasis, Reticulo-nodular pattern, Nodules, Cavity, Calcification, Linear Opacities, Lung Nodule Malignancy, Covid-19 risk and a combination thereof. Further, the imaging model (207), along with the lung abnormalities, may detect other abnormalities, including but not limited to, Cardiomegaly, Degenerative spine conditions, Prominence in Hilar region, Blunted Costophrenic Angle, Pleural effusion, Tracheal shift, Pneumothorax, Elevated Hemidiaphragm, Pneumoperitoneum, Rib Fractures, Mediastinal Widening, Presence of Tracheal Tube, Presence of Gastric Tube, Placement of Tracheal Tube, Placement of Gastric Tube.
In one embodiment, the language model (208), with the help of processor (201), is configured to analyse the patient clinical data (102) to extract patient related information. In a related embodiment, the language model (208) is configured to extract patient relevant information from an unstructured clinical text and to provide treatment options. In one embodiment, the language model (208) may correspond to one of Generative Pre-trained Transformer (GPT) model, a Large Language Model (LLM), and a combination thereof. Further, the language model (208) is configured to calculate a patient confidence score corresponding to the extracted patient related information. The patient confidence score gives the probability of extracted patient related information from the patient clinical data (102) being detected correctly by the algorithm of the language model (207) and may be given as a percentage. The patient confidence score may also showcase confidence in the diagnosis and treatment pathway recommended by the system (100). In one embodiment, the patient related information extracted by the language model (208) corresponds to one of identified risk factors, underlying cause of lung disease and a combination thereof.
In an embodiment, the abnormality detection module (206) is configured to generate an abnormality confidence score by combining the image confidence score and the patient confidence score. The abnormality confidence score may correspond to overall confidence in the abnormality being detected and the diagnosis/treatment pathway being provided. The abnormality confidence score may cover a comprehensive aspect of the lung abnormality detected based on the patient’s medical image, patient history records, the EHR, the clinical research data, and updated medical device or healthcare guidelines available on public platforms.
Further, the output generation module (209) is configured to producing a comprehensive output that encompasses the comprehensive diagnosis on lung abnormality, the abnormality confidence score, and the customized and personalized treatment recommendations based on the collected data. The generated output may combine the output of both the imaging model (207) and the language model (208). In an exemplary embodiment, the output may correspond to an abnormality diagnostic report. The abnormality diagnostic report (400) (as shown in Figure 4) may comprise the medical image with one or more visual bounding boxes (401) to annotate the one or more lung abnormalities (402) detected by the system (100). In an embodiment, the abnormality detection report (400) is configured to be extracted in multiple readable formats, which includes the detected abnormalities in greater details including but not limited to the medical image with one or more visual bounding boxes (401), corresponding labels (402) to annotate the one or more detected lung abnormalities, a diagnosis report (not illustrated), a treatment recommendation report (not illustrated), the abnormality confidence score (not illustrated) and a brief explanation on relevance of the abnormality confidence score (not illustrated).
In another embodiment, the server (104) may present the comprehensive output regarding the lung abnormality on the user device (105) of the system (100). In yet another embodiment, the system (100) may enables monitoring the lung abnormalities over time, which may help in tracking disease progression over time and adjustment of treatment plan. In yet another embodiment, the system (100) may enables providing accurate and up-to-date lung abnormality diagnosis based on continuously learning from clinical evidence over time and adapting the system (100) accordingly. The disclosed system (100) is configured to be self-evolving the analysis, to be performed by the system (100), based on the output analysis previously resulted by the system (100) and any subsequent feedback provided, to the system (100), by the user.
Although the present disclosure is explained considering that the system (100) is implemented on a server, it may be understood that the system (100) may be implemented in a variety of computing systems, such as a laptop computer, a desktop computer, a notebook, a workstation, a virtual environment, a mainframe computer, a server, a network server, a cloud-based computing environment. It will be understood that the system (100) may be accessed by multiple users through one or more user devices. In one implementation, the system (100) may comprise the cloud-based computing environment in which the user may operate individual computing systems configured to execute remotely located applications. Examples of the user devices may include, but are not limited to, a portable computer, a personal digital assistant, a handheld device, and a workstation.
Now referring to figure 3, a flowchart describing a method (300) for detecting lung abnormalities in a medical image (101), is illustrated in accordance with an embodiment of the present subject matter. The method (300) is structured as a step-by-step process. The method (300) comprises a step for collecting (301) relevant data, including the medical image (101) and a patient's clinical data (102). Subsequently, the method (300) comprises a step of analysing (302) the collected data by an abnormality detection module (206), by using a hybrid machine learning system. Further, the method (300) comprises a step of detecting (303) lung abnormality by analysing the medical image (101) using an imaging model (207). Further, the method (300) comprises a step of calculating (304) an image confidence score corresponding to the detected lung abnormality by the imaging model (207). Further, the method (300) comprises a step of extracting (305) patient related information by using a language model (208). The method (300) may further comprise a step of calculating (306) a patient confidence score corresponding to the extracted patient related information by the language model (208). Further, the method (300) comprises a step of generating (307) an abnormality confidence score by combining the image confidence score and the patient confidence score. The abnormality confidence score may correspond to overall confidence in the abnormality being detected and the diagnosis/treatment pathway being provided. Further, the method (300) comprises a step of generating (308) an output using an output generation module (209). The output corresponds to the lung abnormality with a comprehensive diagnosis and the abnormality confidence score. providing healthcare professionals with a holistic evaluation that incorporates both the image analysis and clinical data, thus improving the accuracy and confidence in detecting lung abnormalities. The generated output may combine the output of both the imaging model (207) and the language model (208). In an exemplary embodiment, the output may correspond to an abnormality diagnostic report. The abnormality diagnostic report (400) (as shown in Figure 4) may comprise the medical image with one or more visual bounding boxes (401) to annotate the one or more lung abnormalities (402) detected by the system (100). In an embodiment, the abnormality detection report (400) is configured to be extracted in multiple readable formats, which includes the detected abnormalities in greater details including but not limited to the medical image with one or more visual bounding boxes (401), corresponding labels (402) to annotate the one or more detected lung abnormalities, a diagnosis report (not illustrated), a treatment recommendation report (not illustrated), the abnormality confidence score (not illustrated) and a brief explanation on relevance of the abnormality confidence score (not illustrated).
In one embodiment, the method (300) may include various aspects of data collection module (205) and abnormality detection module (206). Further, the medical image (101) data collected encompasses a wide range of modalities such as x-ray scans, CT scans, MRI scans, and ultrasound images, and more, while the patient clinical data (102) collection includes patient history records, electronic health records (EHR), clinical research data, particularly focused on lung abnormalities, and updated medical device or healthcare guidelines available on the public platforms. The analysis of medical images is carried out using deep learning algorithms within the image model, and the confidence score is calculated based on this analysis. Further, the patient clinical data (102) analysis may include GPT and Large Language Models (LLM), extracting relevant information from unstructured clinical text and recommending treatment options, with a confidence score calculated accordingly. The overall confidence score is determined through a combination formula involving both the image and language models. Ultimately, the method may generate a comprehensive output including a diagnosis, confidence score, and tailored treatment recommendations based on the collected data, enhancing clinical decision-making in pulmonary health. The generated output may combine the output of both the imaging model (207) and the language model (208). In an exemplary embodiment, the output may correspond to an abnormality diagnostic report.
Now referring to Figure 4, the lung abnormality report (400) generated by an output generation module (209) of the system (100) for detecting lung abnormalities in the medical image (101), is illustrated, in accordance with an embodiment of the present subject matter. The lung abnormality report (400) may provide the medical image (101) with one or more visual bounding boxes (401). The visual bounding box (401) corresponds to localize one or more regions in the medical image (101) which may be affected by the lung abnormalities as detected by the system (100). In an instance, there may be more than one boxes (401) may be visualized in the medical image (101) which represents the more than one lung abnormalities as detected by the system (100). In another instance the one or more visual bounding boxes (401) may represent different other abnormalities along with the lung abnormality detected by the system (100). In one exemplary embodiment, the one or more visual bounding box (401) corresponds to a rectangular shape box representing the affected area in the medical image (101). The visual bounding box (401) may correspond to any other shape boundaries representing the affected area in the medical image (101). Further, the lung abnormality report (400) may provide the medical image (101) with labels (402) to annotate the one or more lung abnormalities corresponding to each visual bounding box (401). The label (402) may correspond to name of the lung abnormality detected by the system (100).
In one embodiment, deep learning (may also be called as deep learning techniques) may be a part of a machine learning family. The deep learning techniques may comprise multiple layers in a network. The deep learning techniques may comprise a Convolutional Neural Network (CNN) technique. The deep learning techniques may use a trained data model for an operation. The trained data model comprises historical information, chest X-rays of a set of patients, medical history of the set of patients, the information associated to the nodules for the set of patients. It may be noted that the trained data model may information associated with other abnormalities, to be identified from the chest X-rays. It may be noted that the trained data model may be trained using a continuous learning approach like Reinforcement Learning techniques. The trained data model may correspond to data learned by the system to operate efficiently. The trained data model may enable an accurate operation of the deep learning techniques. In one aspect, the historical data may comprise historical clinical reports and historical chest X-rays associated with the set of patients. In an example, the set of patients may comprise patients suffering from lung abnormalities. In another aspect, the historical data may comprise a previous clinical report and a previous chest X-ray of the patient.
In another embodiment, he utilization of Large Language Models (LLMs) in detecting lung abnormalities represents a transformative advancement in healthcare. LLMs excel at processing and understanding vast amounts of unstructured clinical text, enabling them to extract crucial insights from medical reports, patient histories, research data related to pulmonary health, and updated medical device or healthcare guidelines available on the public platforms. By harnessing their natural language processing capabilities, LLMs can assist medical professionals in rapidly identifying subtle patterns and anomalies in clinical narratives. This, in turn, enhances diagnostic accuracy and provides valuable decision support, ultimately contributing to earlier detection and more effective management of lung abnormalities, thereby improving patient outcomes and streamlining healthcare workflows.
In one aspect, LLM refers to the broader category of large-scale language models that includes GPT as well as other models like BERT (Bidirectional Encoder Representations from Transformers), XLNet, RoBERTa, and more. These models share the common characteristic of being pre-trained on massive text corpora and can be fine-tuned for specific natural language processing tasks.
Overall, this system leverages LLM (large language model) and deep learning techniques to enhance lung abnormalities detection with advanced data analysis.
Example
Normal scenario (Unaided): If a patient comes to the hospital with flu-like symptoms and wheezing, the healthcare professional would first take the patient's medical history, including any existing medical conditions, allergies, and medications. They would then perform a physical examination, which may include listening to the patient's chest with a stethoscope and ordering diagnostic tests such as chest x-rays.
Using Disclosed System (Aided): The mentioned technology could be used to assist in the diagnosis process. The healthcare professional would input the patient's medical history, test results, and any other relevant clinical data into the device. The AI imaging model would analyse the chest x-ray scan to detect any abnormalities, such as pneumonia or lung inflammation, while the language model would analyse the patient's medical history to identify any risk factors or potential underlying causes of the wheezing.
The system would then generate a confidence score that combines the scores generated by the AI imaging model and the language model, providing additional diagnostic support to the healthcare professional. Based on the results, the healthcare professional could then make a more informed diagnosis and recommend appropriate treatment options.
Overall, the system could help healthcare professionals in making a more accurate and timely diagnosis of lung disease, leading to better patient outcomes and potentially reducing the overall cost of care.
The system (100) as disclosed in the disclosure may help in clinical decision support system for detection and management of Lung Abnormalities on a Chest X-ray in the following advantages:
• Accurate diagnosis of Lung Abnormalities: The imaging AI can detect abnormalities in the lung that may be difficult for human experts to identify, while the language model can analyse patient history and related clinical data together to provide a more accurate diagnosis.
• Personalized treatment recommendations: The language model can analyse patient history and other clinical data to provide personalized treatment recommendations that are tailored to the patient's specific needs and medical history. By taking into account the unique characteristics of each patient, the system can help to improve treatment outcomes and reduce the risk of complications.
• Efficient use of healthcare resources: The system can help to streamline the diagnosis and treatment process, reducing the time and resources required to diagnose and treat lung disease. By providing accurate and timely diagnoses, the system can help to reduce the need for additional testing and treatments, which can be costly and time-consuming.
• Monitoring disease progression: The system can be used to track changes in lung abnormalities over time, helping healthcare professionals to monitor disease progression and adjust treatment plans as needed. By providing regular updates on the patient's condition, the system can help to improve patient outcomes and reduce the risk of complications.
• Overall, the system can help to solve a range of problems related to diagnosing and treating lung disease, improving the accuracy and efficiency of the clinical decision-making process and enhancing patient outcomes.
Various modifications to the embodiment will be readily apparent to those skilled in the art and the generic principles herein may be applied to other embodiments. However, one of ordinary skill in the art will readily recognize that the present disclosure is not intended to be limited to the embodiments illustrated but is to be accorded the widest scope consistent with the principles and features described herein.
The foregoing description shall be interpreted as illustrative and not in any limiting sense. A person of ordinary skill in the art would understand that certain modifications could come within the scope of this disclosure.
The embodiments, examples and alternatives of the preceding paragraphs or the description and drawings, including any of their various aspects or respective individual features, may be taken independently or in any combination. Features described in connection with one embodiment are applicable to all embodiments unless such features are incompatible.
, Claims:WE CLAIM:
1. A system (100) for detecting lung abnormalities in a medical image (101), characterized in that, the system (100) comprises:
a memory (203);
a processor (201) coupled with the memory (203), wherein the processor (201) is configured to execute programmed instructions stored in the memory (203);
a data collection module (205), wherein data collected by the data collection module (205) corresponds to the medical image (101) and a patient clinical data (102);
an abnormality detection module (206) for identifying lung abnormalities based on the collected data, using a hybrid machine learning system carried out by the processor (201), wherein the hybrid machine learning system is configured to perform steps of:
detecting lung abnormality by analysing the medical image (101) using an imaging model (207);
calculating an image confidence score corresponding to the detected lung abnormality;
extracting patient related information by analysing patient clinical data (102) using a language model (208);
calculating a patient confidence score corresponding to the extracted patient related information;
generating an abnormality confidence score by combining the image confidence score and the patient confidence score; and
an output generation module (209) for generating an output, wherein the output corresponds to the lung abnormality with a comprehensive diagnosis and the abnormality confidence score.
2. The system (100) as claimed in claim 1, wherein the output generated by the output generation module (209) corresponds to personalized treatment recommendation on patient medical history and clinical data.
3. The system (100) as claimed in claim 1, wherein the system (100) enables monitoring the lung abnormalities over time; wherein monitoring the lung abnormalities helps in tracking disease progression and adjustment of treatment plan.
4. The system (100) as claimed in claim 1, wherein the system (100) enables providing accurate and up-to-date lung abnormality diagnosis based on continuously learning from clinical evidence over time and adapting the system (100) accordingly.
5. The system (100) as claimed in claim 1, wherein the medical image (101) corresponds to chest image data in form of one of x-ray scans, CT scans, MRI scans, ultrasound images, and a combination thereof.
6. The system (100) as claimed in claim 1, wherein the patient clinical data (102) corresponds to one of a patient history records, an electronic health records (EHR), clinical research data, updated medical device or healthcare guidelines available on public platforms and a combination thereof.
7. The system (100) as claimed in claim 6, wherein the clinical research data corresponds to research information published in domain of lung abnormalities.
8. The system (100) as claimed in claim 1, wherein the abnormality detection module (206) is configured to train the imaging model (207) and the language model (208) using the data collected by the data collection module (205).
9. The system (100) as claimed in claim 1, wherein the imaging model (207) corresponds to a deep learning algorithm.
10. The system (100) as claimed in claim 1, wherein lung abnormalities detected using the imaging model (207) corresponds to one of Asthma, Chronic obstructive pulmonary disease (COPD), Bronchiectasis, Bronchitis, pneumothorax, atelectasis, lung inflammation, Pulmonary fibrosis, Sarcoidosis, Lung cancer, Lung Infection (Pneumonia), Hyperinflation/ Emphysema, Consolidation, Opacity, Scoliosis, Fibrosis, Tuberculosis screening, Atelectasis, Reticulo-nodular pattern, Nodules, Cavity, Calcification, Linear Opacities, Lung Nodule Malignancy, Covid-19 risk and a combination thereof.
11. The system (100) as claimed in claim 1, wherein the imaging model (207) is configured to calculate the image confidence score based on the analysis of the medical image (101).
12. The system (100) as claimed in claim 1, wherein the language model (208) corresponds to one of Generative Pre-trained Transformer (GPT) model, a Large Language Model (LLM), and a combination thereof.
13. The system (100) as claimed in claim 1, wherein patient related information extracted by the language model (208) corresponds to one of identified risk factors, underlying cause of lung disease and a combination thereof.
14. The system (100) as claimed in claim 1, wherein the language model (208) is configured to extract patient relevant information from an unstructured clinical text and to provide treatment options.
15. A method (300) for detecting lung abnormalities in a medical image (101), characterized in that, the method (300) comprises:
collecting (301) data using a data collection module (205), wherein the data corresponds to the medical image (101) and a patient clinical data (102);
analysing (302) the collected data by an abnormality detection module (206) by using a hybrid machine learning system, wherein the hybrid machine learning system perform steps of:
detecting (303) lung abnormality by analysing the medical image (101) using an imaging model (207);
calculating (304) an image confidence score corresponding to the detected lung abnormality;
extracting (305) patient related information by using a language model (208);
calculating (306) a patient confidence score corresponding to the extracted patient related information;
generating (307) an abnormality confidence score by combining the image confidence score and the patient confidence score; and
generating (308) an output using an output generation module (209), wherein the output corresponds to the lung abnormality with a comprehensive diagnosis and the abnormality confidence score.
16. The method (300) as claimed in claim 15, wherein the medical image (101) data corresponds to chest image data in form of one of x-ray scans, CT scans, MRI scans, ultrasound images, and a combination thereof.
17. The method (300) as claimed in claim 15, wherein the patient clinical data (102) corresponds to one of a patient history records, an electronic health records (EHR), clinical research data, updated medical device or healthcare guidelines available on public platforms and a combination thereof.
18. The method (300) as claimed in claim 17, wherein the clinical research data corresponds to research information published in domain of lung abnormalities.
19. The method (300) as claimed in claim 15, wherein the method (300) comprises training the imaging model (207) and the language model (208) using the data collected by the data collection module (205).
20. The method (300) as claimed in claim 15, wherein the imaging model (207) corresponds to a deep learning algorithm.
21. The method (300) as claimed in claim 15, wherein lung abnormalities detecting using the imaging model (207) corresponds to one of Asthma, Chronic obstructive pulmonary disease (COPD), Bronchiectasis, Bronchitis, pneumothorax, atelectasis, lung inflammation, Pulmonary fibrosis, Sarcoidosis, Lung cancer, Lung Infection (Pneumonia), Hyperinflation/ Emphysema, Consolidation, Opacity, Scoliosis, Fibrosis, Tuberculosis screening, Atelectasis, Reticulo-nodular pattern, Nodules, Cavity, Calcification, Linear Opacities, Lung Nodule Malignancy, Covid-19 risk and a combination thereof.
22. The method (300) as claimed in claim 15, wherein the language model (208) corresponds to one of Generative Pre-trained Transformer (GPT) model, a Large Language Model (LLM), and a combination thereof.
23. The method (300) as claimed in claim 15, wherein patient related information extracted by the language model (208) corresponds to one of identified risk factors, underlying cause of lung disease and a combination thereof.
24. The method (300) as claimed in claim 15, wherein the patient related information is extracted from an unstructured clinical text and a treatment option recommended by the language model (208).
25. The method (300) as claimed in claim 15, wherein the output generated by the output generation module (209) corresponds to personalized treatment recommendation on patient medical history and clinical data.
26. The method (300) as claimed in claim 15, wherein the method (300) enables monitoring the lung abnormalities over time; wherein monitoring the lung abnormalities helps in tracking disease progression and adjustment of treatment plan.
27. The method (300) as claimed in claim 15, wherein the method (300) enables providing accurate and up-to-date lung abnormality diagnosis based on continuously learning from clinical evidence over time and adapting the system (100) accordingly.

Dated this 20th day of October 2023

Deepak Pawar
Agent for the Applicant
IN/PA-2052