DESC:BACKGROUND
Technical Field
[0001] The embodiments herein generally relate to an image processing method, and more specifically to a system and method for generating a segmented image corresponding to an object from a computed tomography (CT) image using a deep learning model, an edge detection technique, and a morphological operation with enhanced segmentation accuracy.
Description of the Related Art
[0002] Medical imaging plays an important role in diagnosing and treating various diseases. With the advent of artificial intelligence (AI), several deep-learning approaches have been developed to segment anatomical structures in medical images. Segmentation of medical images is the process of identifying and separating different anatomical structures present in the image. It is an essential step in medical image analysis, as it helps in the accurate diagnosis and treatment of various diseases. Accurate segmentation provides a better understanding of the anatomy of organs and tissues and aids in developing better treatment plans. Accurate segmentation is crucial for making critical surgical-related decisions based on specific metrics computed according to segmented anatomical structures. For example, computing certain values using the segmented output may determine whether a patient is sarcopenic or not.
[0003] Recent advances in deep learning methods have improved the accuracy of segmenting anatomical structures. Despite the success of these deep learning approaches, accurate segmentation of anatomical structures in medical images remains a challenging task. One of the main challenges is identifying the edges more clearly to distinguish anatomies in a two-dimensional plane. In medical images, anatomical structures can overlap, and their boundaries may not be well-defined. This can lead to incorrect segmentation and affect the accuracy of diagnosis and treatment. To address this challenge, several approaches have been proposed to improve the edge detection and boundary refinement of segmented anatomical structures. While deep learning approaches for medical image segmentation have shown significant potential in accurately identifying anatomical structures, several limitations need to be addressed. The availability of annotated training data and over-fitting are major challenges that can limit the performance of deep learning models. In addition, the computational requirements, lack of interpretability, sensitivity to the imaging modality and quality, and limited performance on small or rare structures can also impact their applicability in clinical settings.
[0001] Further, for many AI-based readings of medical images, various deep-learning approaches have been developed to segment coarse to fine anatomical structures. Accurate segmentation is essential for making critical surgical-related decisions based on specific metrics that are computed according to segmented anatomical structures. An example of such a metric is determining whether a patient has optimal muscle mass by calculating a score using the segmented output. With existing deep learning methods, it is possible to segment out the anatomical structures with greater accuracy, however, there is a need to identify the edges more clearly to distinguish anatomies in a two-dimensional plane. Further, existing edge-based semantic image segmentation using deep learning has been extensively used for various segmentation tasks. However, these approaches require an image containing only the edges or borders marked for the landmark or object to be segmented. When the edges of the landmark or object are not clearly demarcated, it is not possible to determine the boundaries of the objects accurately.
[0002] Further, the segmentation algorithm provides accuracy levels based on the number of annotated images that are used for creating the model and other criteria. Due to the challenges of acquiring large data sets for accurate segmentation, it has to be augmented with additional techniques to increase the accuracy of determining the anatomies or objects.
[0004] Therefore, there arises a need to address the aforementioned technical drawbacks in existing technologies in segmenting the medical image to determine the one or more objects with increased segmentation accuracy.
SUMMARY
[0005] In view of a foregoing, an embodiment herein provides an image segmentation system for generating a segmented image corresponding to an object from an input computed tomography (CT) image with enhanced segmentation accuracy. The system includes a memory that includes a database; and a processor. The processor is configured to (i) receive the input CT image from an input data source; (ii) detect, using a deep learning model, the object in the input CT image by performing a segmentation process on the input CT image. The object includes one or more anatomical structures of a subject; (iii) generate, using an edge detection technique, one or more boundaries of the object that is detected based on at least one of pixel intensity or color change; and (iv) perform at least one morphological operation on at least one of a shape, a structure or the one or more boundaries of the object, resulting in generation of the segmented image corresponding to the object with enhanced segmentation accuracy.
[0006] In some embodiments, the processor is configured to perform at least one morphological operation by (a) extracting, using a skeletonization technique, a skeleton of the one or more boundaries of the object by reducing the thickness of the one or more boundaries to a one-pixel-wide representation; (b) filling, using a binary filling technique, a content within the skeleton by gradually expanding the one-pixel-wide representation from one or more seed points within the skeleton with each iteration; (c) color-coding, using a color-labelling technique, the content within the skeleton by assigning distinct labels or colors to the object to obtain a color-coded image; and (d) mapping the color-coded image with the input CT image by superimposing the color-coded image with the input CT image to obtain the segmented image that visually differentiates the object from other regions.
[0007] In some embodiments, the edge detection technique is performed by (i) computing, using a gradient operator, the gradient of the image that is detected with the object to highlight areas of at least one rapid pixel intensity or color change; (ii) applying a threshold to a pixel intensity or color change in the image that is computed with the gradient to distinguish edges from the surrounding areas; and (iii) connecting, using an edge linking technique, the edges in the image that are thresholded to generate the one or more boundaries of the object.
[0008] In some embodiments, the segmentation process is performed by (a) inputting, by the processor, the input CT image into the deep learning model that includes one or more layers; (b) extracting, by the deep learning model, one or more features associated with the object using the one or more layers; and (c) generating, by the deep learning model, a segmentation mask associated with the object by up-sampling the one or more features using skip connections.
[0009] In some embodiments, the deep learning model is trained by correlating one or more labeled historical CT images and corresponding segmentation masks associated with one or more historical objects.
[0010] In some embodiments, the processor is configured to pre-processes the input CT image by (i) resizing, using a downsampling technique, the input CT image to obtain a downsampled input image, if the memory is not sufficient to receive the input CT image; and (ii) normalizing, using a minimum-maximum normalization technique, the downsampled input image to obtain a normalized input image to input into the deep learning model.
[0011] In one aspect, a method for generating a segmented image corresponding to an object from an input computed tomography (CT) image with enhanced segmentation accuracy is provided. The method includes (i) receiving, by a processor of an image segmentation system, the input CT image from an input data source; (ii) detecting, by the processor, the object in the input CT image by performing a segmentation process on the input CT image using a deep learning model. The object includes one or more anatomical structures of a subject; (iii) generating, by the processor, one or more boundaries of the object that is detected, using an edge detection technique based on at least one of pixel intensity or color change; and (iv) performing, by the processor, at least one morphological operation on at least one of a shape, a structure or the one or more boundaries of the object, resulting in generation of the segmented image corresponding to the object with enhanced segmentation accuracy.
[0012] In some embodiments, the method includes performing, by the processor, at least one morphological operation by (a) extracting, using a skeletonization technique, a skeleton of the one or more boundaries of the object by reducing the thickness of the one or more boundaries to a one-pixel-wide representation; (b) filling, using a binary filling technique, a content within the skeleton by gradually expanding the one-pixel-wide representation from one or more seed points within the skeleton with each iteration; (c) color-coding, using a color-labelling technique, the content within the skeleton by assigning distinct labels or colors to the object to obtain a color-coded image; and (d) mapping the color-coded image with the input CT image by superimposing the color-coded image with the input CT image to obtain the segmented image that visually differentiates the object from other regions.
[0013] In some embodiments, the method includes performing, by the processor, the edge detection technique by (i) computing, using a gradient operator, the gradient of the image that is detected with the object to highlight areas of at least one rapid pixel intensity or color change; (ii) applying a threshold to a pixel intensity or color change in the image that is computed with the gradient to distinguish edges from the surrounding areas; and (iii) connecting, using an edge linking technique, the edges in the image that are thresholded to generate the one or more boundaries of the object.
[0014] In some embodiments, the segmentation process is performed by (a) inputting, by the processor, the input CT image into the deep learning model that includes one or more layers; (b) extracting, by the deep learning model, one or more features associated with the object using the one or more layers; and (c) generating, by the deep learning model, a segmentation mask associated with the object by up-sampling the one or more features using skip connections.
[0015] The image segmentation system of the present disclosure reinforces the segmentation process along with the edge detection technique to improve the prediction of the boundaries or edges of the object in the input CT image. That is, once the object is segmented, the image segmentation system employs the edge detection technique in combination with some of the morphological operations to clearly define the boundaries of the object, thereby improving the segmentation accuracy. More clearly, the image segmentation system employs the edge detection technique followed by some computer vision algorithms on the output of the deep learning model to improve accuracy of segmentation of anatomies or objects further and produce more precise metrics (for example sarcopenic score) to aid surgical decisions.
[0016] These and other aspects of the embodiments herein will be better appreciated and understood when considered in conjunction with the following description and the accompanying drawings. It should be understood, however, that the following descriptions, while indicating preferred embodiments and numerous specific details thereof, are given by way of illustration and not of limitation. Many changes and modifications may be made within the scope of the embodiments herein without departing from the spirit thereof, and the embodiments herein include all such modifications
BRIEF DESCRIPTION OF THE DRAWINGS
[0017] The embodiments herein will be better understood from the following detailed description with reference to the drawings, in which:
[0018] FIG. 1 illustrates a system for generating a segmented image corresponding to an object from an input computed tomography (CT) image with enhanced segmentation accuracy according to some embodiments herein;
[0019] FIG. 2 illustrates a block diagram of an image segmentation system of FIG. 1 according to some embodiments herein;
[0020] FIG. 3 is a block diagram that illustrates an exemplary process of generating a segmented image corresponding to a left PSOAS muscle and a right PSOAS muscle from an input computed tomography (CT) image with an enhanced segmentation accuracy using an image segmentation system of FIG. 1 according to some embodiments herein;
[0021] FIG. 4 is a flow diagram that illustrates a method for generating a segmented image corresponding to an object from an input computed tomography (CT) image with enhanced segmentation accuracy using an image segmentation system of FIG. 1 according to some embodiments herein;
[0022] FIG. 5 shows graphical representations that illustrate training or validation loss and intersection over union (IoU) curves for deep learning models, UNet and TransUNet according to some embodiments herein;
[0023] FIG. 6 illustrates exemplary segmented images that are generated from computed tomography (CT) images using different methods involving deep learning models and an image segmentation system of FIG. 1 according to some embodiments herein;
[0024] FIG. 7 is an exemplary diagram that illustrates a performance evaluation of different methods of FIG. 6 by comparing an output with a ground truth according to some embodiments herein; and
[0025] FIG. 8 is a schematic diagram of a computer architecture in accordance with the embodiments herein.
DETAILED DESCRIPTION OF THE DRAWINGS
[0026] The embodiments herein and the various features and advantageous details thereof are explained more fully with reference to the non-limiting embodiments that are illustrated in the accompanying drawings and detailed in the following description. Descriptions of well-known components and processing techniques are omitted so as to not unnecessarily obscure the embodiments herein. The examples used herein are intended merely to facilitate an understanding of ways in which the embodiments herein may be practiced and to further enable those of skill in the art to practice the embodiments herein. Accordingly, the examples should not be construed as limiting the scope of the embodiments herein.
[0027] As mentioned, there remains a need for a segmentation system to determine one or more objects in a medical image with increased segmentation accuracy. The embodiments herein achieve this by proposing a system and method for generating a segmented image corresponding to an object from an input computed tomography (CT) image with enhanced segmentation accuracy. Referring now to the drawings, and more particularly to FIGS. 1 through 8, where similar reference characters denote corresponding features consistently throughout the figures, there are shown preferred embodiments.
[0028] FIG. 1 illustrates a system 100 for generating a segmented image corresponding to an object from an input computed tomography (CT) image with enhanced segmentation accuracy according to some embodiments herein. The system 100 includes an input data source 102, an image segmentation system 104, and a network 106. The image segmentation system 104 includes a processor and a non-transitory computer-readable storage medium (or memory) storing a database and one or more sequences of instructions, which when executed by the processor causes the segmentation of input CT image with enhanced segmentation accuracy. The image segmentation system 104 may be a server, a handheld device, a mobile phone, a Personal Digital Assistant (PDA), a tablet, a music player, a computer, a laptop, an electronic notebook, or a Smartphone.
[0029] The image segmentation system 104 is communicatively connected with the input data source 102 through the network 106. The network 106 may be one or more of a wired network, a wireless network based on at least one of a 2G protocol, a 3G protocol, a 4G protocol, or a 5G protocol, Bluetooth Low Energy (BLE), Near Field Communication (NFC), Bluetooth, WiFi, and a narrowband internet of things protocol (NBIoT), a combination of the wired network and the wireless network or the Internet.
[0030] The image segmentation system 104 is configured to receive the input CT image associated with a subject from the input data source 102. The input data source 102 may be an imaging modality, an image-capturing device, any personal device, or a digital source that provides the input data associated with the subject. The input image may also be of a magnetic resonance imaging (MRI) image, an X-ray image, an ultrasound image, a positron emission tomography (PET) image, a nuclear medicine image, a fluoroscopy image, or a mammography image. The subject may be a human body or body parts of a human. The input CT image includes the object. The object may include one or more anatomical structures of the subject. The one or more anatomical structures may be organs, tissues, bones, or other relevant features of the subject.
[0031] The image segmentation system 104 is configured to detect the object in the input CT image by performing a segmentation process on the input CT image. The image segmentation system 104 may use a deep learning model for the segmentation process. The deep learning model may include one or more layers to detect the object. The deep learning model may be a neural network. The deep learning model may be trained by correlating labelled historical input CT images and corresponding segmentation masks associated with historical objects.
[0032] In some embodiments, the image segmentation system 104 is configured to pre-process the input CT image upon receiving the input CT image from the input data source 102 to obtain a pre-processed input CT image. The image segmentation system 104 may use one or more pre-processing technique to obtain the pre-processed input CT image to input into the deep learning model. The one or more pre-processing techniques may include a downsampling technique, a minimum-maximum normalization technique, or a denoising technique.
[0033] In an exemplary embodiment, the deep learning model is a UNet or a TransUNet. The UNet architecture may include an encoder layer, a bottleneck layer, a decoder layer, and an output layer. The encoder layer may include one or more convolutional layers with a small receptive field followed by rectified linear unit (ReLU) activation and one or more pooling layers. The bottleneck layer may include one or more convolutional layers with ReLU activation. The decoder layer may include one or more convolutional layers with a larger receptive field followed by ReLU activation and one or more up-sampling layers (or transposed convolution). The output layer may include a convolutional layer with a kernel followed by a suitable activation function. The UNet architecture further includes skip connections that skip one or more layers and directly connect the layers in the encoder layer to the corresponding layers in the decoder layer. The skip connections allow the deep learning model to retain high-resolution information and details during an up-sampling process. The TransUNet may be formed by replacing a bottleneck layer of the UNet with one or more transformers to segment the one or more objects from the pre-processed input image.
[0034] The image segmentation system 104 is configured to generate one or more boundaries of the object that is detected using the deep learning model. The image segmentation system 104 may use an edge detection technique to generate the one or more boundaries based on at least one of pixel intensity or color change. In some embodiments, a canny edge detection technique is used to generate one or more boundaries of the object.
[0035] The image segmentation system 104 is further configured to perform at least one morphological operation on at least one of a shape, a structure, or the one or more boundaries of the object to generate the segmented image. The at least one morphological operation includes a skeletonization, a binary filling, and a color-labeling. The resulting segmented image includes one or more segmented objects with enhanced segmentation accuracy.
[0036] In some embodiments, the segmented image with enhanced segmentation accuracy provides more precise computation of metrics to determine one or more medical actions or make surgical decisions. For example, the system 100 aids in computing a sarcopenia score using the segmented image, and enables users (for example, medical professionals) to determine one or more medical actions (e.g. surgery) based on the computed sarcopenia score.
[0037] FIG. 2 illustrates a block diagram of an image segmentation system 104 of FIG. 1 according to some embodiments herein. The image segmentation system 104 includes a database 200, a processor 202, a receiving module 204, a pre-processing module 206, an object detection module 208, a deep learning model 210, an edge detection module 212, and a morphological processing module 214. The database 200 stores a set of modules of the image segmentation system 104. The processor 202 executes the set of modules in the database 200 for generating a segmented image corresponding to an object from an input computed tomography (CT) image with enhanced segmentation accuracy. The receiving module 204 receives the input CT image of a subject from an input data source 102 through network 106 and stores it in the database 200.
[0038] The object detection module 208 detects the object in the input CT image by performing a segmentation process on the input CT image. The object detection module 208 may use the deep learning model 210 for the segmentation process. The deep learning model 210 may include one or more layers to detect the object. In some embodiments, the deep learning model 210 is at least one of a UNet or a TransUNet. In some embodiments, the object detection module 208 performs the segmentation process by (i) inputting the input CT image into the deep learning model 210; (ii) extracting one or more features associated with the object using the one or more layers of the deep learning model 210; and (iii) generating a segmentation mask associated with the object by up-sampling the one or more features using skip connections of the deep learning model 210.
[0039] In some embodiments, the pre-processing module 206 pre-processes the input CT image that is received at the receiving module 204 to obtain a pre-processed input CT image. The pre-processing module 206 resizes the input CT image using a downsampling technique to obtain a downsampled input image if the memory or database 200 is not sufficient to receive the input CT image. The downsampling technique may be at least one of an averaging, a max pooling, a Gaussian blurring and subsampling, decimation, or wavelet transform. The pre-processing module 206 may resize the input CT image by downsampling to a 128*128 pixel image or any other suitable resolution. The pre-processing module 206 further normalizes the downsampled input image to obtain a normalized input image as the pre-processed input image. The pre-processing module 206 may use a minimum-maximum normalization technique to scale the pixels of the downsampled input image between 0 and 1. The pre-processed input image may be inputted into the deep learning model 210 to detect the object in the image.
[0040] The edge detection module 212 generates one or more edges or boundaries of the object that is detected at the object detection module 208. The edge detection module 212 may use an edge detection technique to generate the one or more boundaries based on at least one of pixel intensity or color change. In some embodiments, the edge detection module 212 performs the edge detection technique by (i) computing the gradient of the image that is detected with the object to highlight areas of at least one rapid pixel intensity or color change; (ii) applying a threshold to the pixel intensity or color change in the image that is computed with the gradient to distinguish edges from the surrounding areas; and (iii) connecting the edges in the image that is thresholded to generate the one or more boundaries of the object. The edge detection module 212 may use a gradient operator to compute the gradient of the image. The edge detection module 212 may use an edge linking technique to connect the edges in the image.
[0041] In some embodiments, the edge detection module 212 generates the one or more boundaries of the object using a canny edge detection technique. The canny edge detection technique may be performed by (i) smoothing the image that is detected with the object using a Gaussian filter to reduce noise; (ii) calculating the gradient of the image using techniques such as the Sobel operator, which identifies intensity gradients in both horizontal and vertical directions at each pixel; (iii) applying non-maximum suppression to thin the edges by suppressing all gradient values except local maxima, indicative of potential edges; and (iv) performing edge tracking by hysteresis, where two thresholds (lower and upper) are set. Pixels with gradient values above the upper threshold are marked as strong edges, those below the lower threshold are discarded, and those between are considered weak edges. Weak edges are then connected to strong edges if they are adjacent, resulting in generation of the one or more boundaries of the object. Other suitable edge detection techniques may also be applied to detect the edges or boundaries of the object.
[0042] The morphological processing module 214 performs at least one morphological operation on at least one of a shape, a structure, or the one or more boundaries of the object to generate a segmented image. The morphological processing module 214 extracts a skeleton of the one or more boundaries of the object using a skeletonization technique. The skeletonization technique may reduce the thickness of the one or more boundaries to a one-pixel-wide representation to extract the skeleton. The one-pixel-wide representation refers to a representation of the object, where the one or more boundaries are reduced to a single pixel in width.
[0043] The morphological processing module 214 further fills a content within the skeleton using a binary filling technique. The binary filling technique is performed by identifying one or more seed points within the skeleton and gradually or iteratively expanding the one-pixel-wide representation from one or more seed points within the skeleton to fill the content within the skeleton. During each iteration, pixels adjacent to the existing filled region are examined, and added to the filled region, if the pixels meet certain criteria (e.g., transitioning from background to foreground). The iterative expansion continues until no further additions can be made, ensuring the complete filling of the content within the skeleton.
[0044] The morphological processing module 214 further color-codes the content within the skeletonusing a color-labelling technique to obtain a color-coded image. The color-labelling technique may assign distinct labels or colors to the object to obtain the color-coded image. Each pixel associated with a particular object is replaced with the corresponding color. The morphological processing module 214 further maps the color-coded image with the input CT image (initial image) by superimposing the color-coded image with the input CT image to obtain a segmented image. The morphological processing module 214 may overlay the color-coded pixels onto the corresponding pixels of the input CT image to superimpose the images for obtaining the segmented image. The segmented image visually differentiates the object from other regions with enhanced segmentation accuracy.
[0045] FIG. 3 is a block diagram that illustrates an exemplary process of generating a segmented image corresponding to a left PSOAS muscle and a right PSOAS muscle from an input computed tomography (CT) image with an enhanced segmentation accuracy using an image segmentation system 104 of FIG. 1 according to some embodiments herein. At step 302, the image segmentation system 104 receives the input CT image from an input data source 102. The input CT image may be a CT image of PSOAS muscle that is taken from the input data source 102 (for example an imaging modality: CT). The input CT image includes one or more objects such as a left PSOAS muscle and a right PSOAS muscle. At step 304, the image segmentation system 104 detects the one or more objects including the left PSOAS muscle and the right PSOAS muscle in the input CT image by performing a segmentation process on the input CT image using a deep learning model 210. At step 306, the image segmentation system 104 generates one or more edges or boundaries of the left PSOAS muscle and the right PSOAS muscle that are detected with the deep learning model 210 at the step 304. The image segmentation system 104 detects the one or more boundaries of the left PSOAS muscle and the right PSOAS muscle by applying an edge detection technique.
[0046] At step 308, the image segmentation system 104 extracts a skeleton of the one or more boundaries of the left PSOAS muscle and the right PSOAS muscle using a skeletonization technique. At step 310, the image segmentation system 104 fills a content within the skeleton associated with the left PSOAS muscle and the right PSOAS muscle using a binary filling technique. At step 312, the image segmentation system 104 further color-codes the content within the skeleton associated with the left PSOAS muscle and the right PSOAS muscle using a color-labeling technique to obtain a color-coded image. At step 314, the image segmentation system 104 maps the color-coded image with the input CT image (initial image) by superimposing the color-coded image with the input CT image to obtain a segmented image as output with increased segmentation accuracy.
[0047] The output image or the segmented image associated with the left PSOAS muscle and the right PSOAS muscle may be used to determine a sarcopenia score and enables the users to determine one or more medical actions (e.g. a surgery) based on the computed sarcopenia score. For example, in clinical practice, a surgeon may be required to make critical decisions based on the sarcopenia score. This score necessitates the computation of the left and right PSOAS regions, along with determining the mean Hounsfield Unit (HU) value within the delineated or segmented areas. When the anatomical or object boundaries are not clearly defined by the sub-optimal algorithms, the system 100 may retrieve HU values from areas outside the intended anatomical structure delineated by the segmented boundaries. This could lead to erroneous surgical decisions if the sarcopenia score is inaccurately determined.
[0048] FIG. 4 is a flow diagram that illustrates a method for generating a segmented image corresponding to an object from an input computed tomography (CT) image with enhanced segmentation accuracy using an image segmentation system 104 of FIG. 1 according to some embodiments herein. At step 402, the input CT image is received from an input data source 102. At step 404, the object in the input CT image is detected by performing a segmentation process on the input CT image using a deep learning model 210. The object includes one or more anatomical structures of a subject. At step 406, one or more boundaries of the object that is detected are generated using an edge detection technique based on at least one of pixel intensity or color change. At step 408, at least one morphological operation is performed on at least one of a shape, a structure, or the one or more boundaries of the object, resulting in generation of the segmented image corresponding to the object with enhanced segmentation accuracy.
[0049] In some embodiments, at least one morphological operation is performed by (i) extracting, using a skeletonization technique, a skeleton of the one or more boundaries of the object by reducing the thickness of the one or more boundaries to a one-pixel-wide representation; (ii) filling, using a binary filling technique, a content within the skeleton by gradually expanding the one-pixel-wide representation from one or more seed points within the skeleton with each iteration; (iii) color-coding, using a color-labelling technique, the content within the skeleton by assigning distinct labels or colors to the object to obtain a color-coded image; and (iv) mapping the color-coded image with the input CT image by superimposing the color-coded image with the input CT image to obtain the segmented image that visually differentiates the object from other regions.
[0050] FIG. 5 shows graphical representations that illustrate training or validation loss and intersection over union (IoU) curves for deep learning models, UNet and TransUNet according to some embodiments herein. An abdominal Computed Tomography (CT) image is captured using an image capturing unit and the abdominal CT images are processed by reducing them to 128*128 pixels or any other suitable resolution. A min-max normalized method is performed and then annotated to determine right PSOAS muscles. In some embodiments, the contrast-adjusted CT images and their corresponding masks are then used for training the deep learning models. The annotated masks are generated in such a way that the right PSOAS is assigned with value 1, and the background with 0. The deep learning models UNet and TransUNet are trained using 8000 CT slices or images with around 500 epochs with a learning rate of 1 x10-5, and batch size 10. In TransUNet, patches of size 4x4 are generated to highlight the area that needs segmentation. An Adam optimizer is used to optimize the loss function consisting of binary cross entropy and dice loss at the time of training. In FIG. 5, 502 represents IoU curves of UNet, 504 represents loss curves of UNet, 506 represents IoU curves of TransUNet, and 508 represents loss curves of TransUNet. The mean intersection over union (IoU) and loss are tracked to ensure that the optimal learning has been performed by the deep learning models such as UNet and TransUNet.
[0051] FIG. 6 illustrates exemplary segmented images that are generated from computed tomography (CT) images using different methods involving deep learning models and an image segmentation system 104 of FIG. 1 according to some embodiments herein. In FIG. 6, 602 denotes segmented images that are generated using a deep learning model, UNet 608A from an input CT image; 604 denotes segmented images that are generated using a deep learning model 210, TransUNet 608B from the input CT image; and 606 denotes segmented images that are generated using a combination of the deep learning model 210, TransUNet + edge detection technique + morphological operation 610 (that is, using a method of FIG. 4 using the image segmentation system 104 of FIG. 1) from an input CT image. The input CT image may be an image of PSOAS muscle. Both UNet and TransUNet may have a kernel size of 3x3 and a stride rate of 2 across the convolution and transpose convolution layers. From FIG. 6, it is inferred that the combination of the deep learning model 210 (that is, TransUNet), the edge detection technique and the morphological operation 610 has improved accuracy when compared to the segmentation performed using the individual deep learning models 608A-B (that is, UNet or TransUNet).
[0052] FIG. 7 is an exemplary diagram that illustrates a performance evaluation of different methods of FIG. 6 by comparing an output with a ground truth according to some embodiments herein. Around 400 CT images are used to test the deep learning models. The mean IoU and dice coefficient for slices with the different methods are shown in Table 1.
[0053] Table 1: Mean IoU and dice coefficient values
MODELS / DIFFERENT METHODS MEAN INTERSECTION OVER UNION (IoU) MEAN DICE COEFFICIENT
UNet 0.899607641046697 0.877924277836626
TransUNet 0.903849032792178 0.878982530398802
TransUNet + Edge Detection technique + Morphological Operation 0.91238082052144 0.891782319410281

[0054] For a better performance evaluation of all three methods, the output of these methods is compared with the ground truth as shown in Fig. 7. The values shown in Table 1, and the results shown in FIG. 6 and FIG. 7 state that adding edge detection technique and morphological operations to the output of deep learning models (UNet and TransUNet) further enhances the segmentation accuracy.
[0055] In some exemplary embodiments, the system 100 determines PSOAS muscle dimensions, determines the accuracy of the measurements, and compares them with manual calculations by trained radiographers and/or surgeons. The manual calculation of the dimensions of the PSOAS muscle over 100 CT images is performed and compared with the PSOAS muscle dimensions determined using the system 100. It is observed that the variance in dimensions predicted by the system 100 is less than 10%. The values, as shown in Table 1 and manual validations, depict that reinforcing the segmentation along with edge detection and morphological operations improves or enhances the segmentation accuracy.
[0056] A representative hardware environment for practicing the embodiments herein is depicted in FIG. 8, with reference to FIGS. 1 through 7. This schematic drawing illustrates a hardware configuration of an image segmentation system 104/computer system/ computing device in accordance with the embodiments herein. The system includes at least one processing device CPU 10 that may be interconnected via system bus 14 to various devices such as a random-access memory (RAM) 12, read-only memory (ROM) 16, and an input/output (I/O) adapter 18. The I/O adapter 18 can connect to peripheral devices, such as disk units 38 and program storage devices 40 that are readable by the system. The system can read the inventive instructions on the program storage devices 40 and follow these instructions to execute the methodology of the embodiments herein. The system further includes a user interface adapter 22 that connects a keyboard 28, mouse 30, speaker 32, microphone 34, and/or other user interface devices such as a touch screen device (not shown) to the bus 14 to gather user input. Additionally, a communication adapter 20 connects the bus 14 to a network 42, and a display adapter 24 connects the bus 14 to a display device 26, which provides a graphical user interface (GUI) 36 of the output data in accordance with the embodiments herein, or which may be embodied as an output device such as a monitor, printer, or transmitter, for example.
[0024] The foregoing description of the specific embodiments will so fully reveal the general nature of the embodiments herein that others can, by applying current knowledge, readily modify and/or adapt for various applications without departing from the generic concept, and, therefore, such adaptations and modifications should be comprehended within the meaning and range of equivalents of the disclosed embodiments. It is to be understood that the phraseology or terminology employed herein is for the purpose of description and not of limitation. Therefore, while the embodiments herein have been described in terms of preferred embodiments, those skilled in the art will recognize that the embodiments herein can be practiced with modification within the spirit and scope of the appended claims.
,CLAIMS:I/We Claim:

1. An image segmentation system (104) for generating a segmented image corresponding to an object from an input computed tomography (CT) image with enhanced segmentation accuracy, wherein the image segmentation system (104) comprising:
a memory including a database (200); and
a processor (202), wherein the processor (202) is configured to
receive the input CT image from an input data source (102);
characterized in that,
detect, using a deep learning model (210), the object in the input CT image by performing a segmentation process on the input CT image, wherein the object comprises one or more anatomical structures of a subject;
generate, using an edge detection technique, one or more boundaries of the object that is detected based on at least one of pixel intensity or color change; and
perform at least one morphological operation on at least one of a shape, a structure or the one or more boundaries of the object, resulting in generation of the segmented image corresponding to the object with enhanced segmentation accuracy.

2. The image segmentation system (104) as claimed in claim 1, wherein the processor (202) is configured to perform at least one morphological operation by
extracting, using a skeletonization technique, a skeleton of the one or more boundaries of the object by reducing the thickness of the one or more boundaries to a one-pixel-wide representation;
filling, using a binary filling technique, a content within the skeleton by gradually expanding the one-pixel-wide representation from one or more seed points within the skeleton with each iteration;
color-coding, using a color-labelling technique, the content within the skeleton by assigning distinct labels or colors to the object to obtain a color-coded image; and
mapping the color-coded image with the input CT image by superimposing the color-coded image with the input CT image to obtain the segmented image that visually differentiates the object from other regions.

3. The image segmentation system (104) as claimed in claim 1, wherein the edge detection technique is performed by
computing, using a gradient operator, the gradient of the image that is detected with the object to highlight areas of at least one rapid pixel intensity or color change;
applying a threshold to a pixel intensity or color change in the image that is computed with the gradient to distinguish edges from the surrounding areas; and
connecting, using an edge linking technique, the edges in the image that are thresholded to generate the one or more boundaries of the object.

4. The image segmentation system (104) as claimed in claim 1, wherein the segmentation process is performed by
inputting, by the processor (202), the input CT image into the deep learning model (210), wherein the deep learning model (210) comprises a plurality of layers;
extracting, by the deep learning model (210), one or more features associated with the object using the plurality of layers; and
generating, by the deep learning model (210), a segmentation mask associated with the object by up-sampling the one or more features using skip connections.

5. The image segmentation system (104) as claimed in claim 4, wherein the deep learning model (210) is trained by correlating a plurality of labelled historical CT images and corresponding segmentation masks associated with a plurality of historical objects.

6. The image segmentation system (104) as claimed in claim 1, wherein the processor (202)is configured topre-processes the input CT image by
resizing, using a downsampling technique, the input CT image to obtain a downsampled input image, if the memory is not sufficient to receive the input CT image; and
normalizing, using a minimum-maximum normalization technique, the downsampled input image to obtain a normalized input image to input into the deep learning model (210).

7. A method for generating a segmented image corresponding to an object from an input computed tomography (CT) image with enhanced segmentation accuracy, wherein the method comprising:
receiving, by a processor (202) of an image segmentation system (104), the input CT image from an input data source (102);
characterized in that,
detecting, by the processor (202), the object in the input CT image by performing a segmentation process on the input CT image using a deep learning model (210), wherein the object comprises one or more anatomical structures of a subject;
generating, by the processor (202), one or more boundaries of the object that is detected using an edge detection technique based on at least one of pixel intensity or color change; and
performing, by the processor (202), at least one morphological operation on at least one of a shape, a structure or the one or more boundaries of the object, resulting in generation of the segmented image corresponding to the object with enhanced segmentation accuracy.

8. The method as claimed in claim 7, wherein the method comprises performing, by the processor (202), at least one morphological operation by
extracting, using a skeletonization technique, a skeleton of the one or more boundaries of the object by reducing the thickness of the one or more boundaries to a one-pixel-wide representation;
filling, using a binary filling technique, a content within the skeleton by gradually expanding the one-pixel-wide representation from one or more seed points within the skeleton with each iteration;
color-coding, using a color-labelling technique, the content within the skeleton by assigning distinct labels or colors to the object to obtain a color-coded image; and
mapping the color-coded image with the input CT image by superimposing the color-coded image with the input CT image to obtain the segmented image that visually differentiates the object from other regions.

9. The method as claimed in claim 7, wherein the method comprises performing, by the processor (202), the edge detection technique by
computing, using a gradient operator, the gradient of the image that is detected with the object to highlight areas of at least one rapid pixel intensity or color change;
applying a threshold to a pixel intensity or color change in the image that is computed with the gradient to distinguish edges from the surrounding areas; and
connecting, using an edge linking technique, the edges in the image that are thresholded to generate the one or more boundaries of the object.

10. The method as claimed in claim 7, wherein the segmentation process is performed by
inputting, by the processor (202), the input CT image into the deep learning model (210), wherein the deep learning model (210) comprises a plurality of layers;
extracting, by the deep learning model (210), one or more features associated with the object using the plurality of layers; and
generating, by the deep learning model (210), a segmentation mask associated with the object by up-sampling the one or more features using skip connections.

Dated this March22nd, 2024

Arjun Karthik Bala
(IN/PA 1021)
Agent for Applicant