Claims:CLAIMS
We claim:
1. A computer implemented system 100 to detect an eye type of a retinal fundus image of a patient, comprising:

at least one processor;

a non-transitory computer readable storage medium communicatively coupled to the at least one processor, the non-transitory computer readable storage medium configured to store an eye type determination application 103, the at least one processor configured to execute the eye type determination application 103; and

the eye type determination application 103 comprising:

a graphical user interface 103i comprising a plurality of interactive elements 103h configured to enable capture and process of the retinal fundus image of the patient via a user device 101a, 101b or 101c;

a reception means 103a adapted to receive an input from an image capturing device, wherein the input is the retinal fundus image of the patient;

an interactive retinal fundus image rendering means 103b adapted to dynamically render the input, wherein the dynamically rendered input is configurably accessible on the graphical user interface 103i via the user device 101a, 101b or 101c using the interactive elements 103i;

a retinal fundus image capture means 103c adapted to capture the retinal fundus image of the patient based on the dynamically rendered input;

a processing means 103g adapted to detect the eye type of the retinal fundus image of the patient, comprising:

train each of a plurality of neural networks over a training dataset with an associated label for each of a training retinal fundus image in the training dataset, wherein the associated label is based on the location of an optic disc and a macula in the training retinal fundus image; and

process the retinal fundus image to determine the eye type by the trained neural networks.

2. The system 100 as claimed in claim 1, wherein the associated label for each of the training retinal fundus image comprises a predefined mask for the training retinal fundus image based on a presence of the optic disc and the macula and an eye type of the training retinal fundus image.

3. The system 100 as claimed in claim 1, wherein the processing means 103g adapted to determining the eye type of the training retinal fundus image comprises the following steps:
dividing the retinal fundus image into a predefined number of rows and columns to have a predefined number of regions;
computing a region pixel intensity score for each region based on a pixel intensity distribution;
comparing an optic disc pixel intensity score with the region pixel intensity score of each of the regions to determine a first region with the presence of the optic disc;
comparing a macula pixel intensity score with the region pixel intensity score of each of the regions to determine a second region with the presence of the macula;
determining a presence of the first region and the second region in a central row of the retinal fundus image;
voting for a left eye when the first region is located to the left of the second region in the central row of the retinal fundus image; and
voting for the right eye when the first region is located to the right of the second region in the central row of the retinal fundus image.
4. The system 100 as claimed in claim 3, wherein the predefined number of rows are three, the predefined number of columns are three and the predefined number of regions are nine.

5. The system 100 as claimed in claim 3, wherein the processing means 103g discards the retinal fundus image on an absence of either the optic disc or the macula in the central row of the retinal fundus image.

6. The system 100 as claimed in claim 2, wherein the predefined mask of the training retinal fundus image indicates a location and a structure of the optic disc and the macula.

7. A method to detect an eye type of a retinal fundus image of a patient, said method comprising:
receiving the retinal fundus image of the patient;
training each of a plurality of neural networks over a training dataset with an associated label for each of a training retinal fundus image in the training dataset, wherein the associated label is based on the location of an optic disc and a macula in the training retinal fundus image; and
processing the retinal fundus image to determine the eye type by the trained neural networks.

8. The method as claimed in claim 7, wherein the associated label for each of the training retinal fundus image comprises a predefined mask for the training retinal fundus image based on a presence of the optic disc and the macula and an eye type of the training retinal fundus image.

9. The method as claimed in claim 7, wherein determining the eye type of the training retinal fundus image comprises:

dividing the retinal fundus image into a predefined number of rows and columns to have a predefined number of regions;
computing a region pixel intensity score for each region based on a pixel intensity distribution;
comparing an optic disc pixel intensity score with the region pixel intensity score of each of the regions to determine a first region with the presence of the optic disc;
comparing a macula pixel intensity score with the region pixel intensity score of each of the regions to determine a second region with the presence of the macula;
determining a presence of the first region and the second region in a central row of the retinal fundus image;
voting for a left eye when the first region is located to the left of the second region in the central row of the retinal fundus image; and
voting for the right eye when the first region is located to the right of the second region in the central row of the retinal fundus image.

10. The method as claimed in claim 9, wherein discarding the retinal fundus image on an absence of either the optic disc or the macula in the central row of the retinal fundus image.

, Description:F O R M 2

THE PATENTS ACT, 1970

(39 of 1970)

COMPLETE SPECIFICATION

(See section 10; rule 13)

1. TITLE OF THE INVENTION

EYE TYPE DETECTOR

2. APPLICANT:

a. Name: ARTIFICIAL LEARNING SYSTEMS INDIA PVT LTD

b. Nationality: INDIA

c. Address: Hansa Complex, 1665/A, second floor, 14th Main, 7th, sector, HSR Layout, HSR Layout, Bengaluru, Karnataka 560102, India
.

Complete specification:

The following specification particularly describes the invention and the manner in which it is to be performed.


Technical field of the invention

[0001] The invention relates to the field of medical decision support. More particularly, the invention relates to identification of an eye type using a retinal fundus image of a patient.

Background of the invention

[0002] Vision is an important survival attribute for a human, thus making eyes as one of the most vital sensory body part. The eye is an organ of sight and has a complex structure with an outer fibrous layer, a middle vascular layer, and a nervous tissue layer. Though most of the eye diseases may not be fatal, failure of proper diagnosis and treatment of an eye disease may lead to vision loss. Analysis of retinal fundus images of a patient is a very convenient way of screening and monitoring eye diseases. The retinal fundus image of the patient illustrates several elements such as optic disc, blood vessels, macula, etc. The fundus of the eye provides indications of several diseases, in particular, eye diseases like diabetic retinopathy.

[0003] In recent times, computer-aided screening systems assists doctors to improve the quality of examination of retinal fundus images for screening of eye diseases. Noninvasive methods are a significantly rapid emerging division of medical science used for medical research and early diagnostics of diseases, specifically retinal diseases. Errors may occur, for example, misinterpretation of retina retinal fundus images by an operator of the system while transferring information to the screening systems. The operator may wrongly determine whether the retinal fundus image is of the left eye or the right eye and mislabel the retinal fundus image. This may lead to the wrong treatment of the patient at a later stage. Therefore, it would be desirable to provide methods and systems for differentiating between left and right eye images and automatically identify an eye type of the retinal fundus image reducing the likelihood of human error during labelling of the retinal fundus images.

Summary of invention

[0004] This summary is provided to introduce a selection of concepts in a simplified form that are further disclosed in the detailed description of the invention. This summary is not intended to identify key or essential inventive concepts of the claimed subject matter, nor is it intended for determining the scope of the claimed subject matter.

[0005] The present invention discloses a computer implemented system for processing a retinal fundus image of a patient. The system comprises at least one processor; a non-transitory computer readable storage medium communicatively coupled to the at least one processor, the non-transitory computer readable storage medium configured to store an eye type determination application, the at least one processor configured to execute the eye type determination application; and the eye type determination application comprising: a graphical user interface comprising a plurality of interactive elements configured to enable capture and process of the retinal fundus image of the patient via a user device; a reception means adapted to receive an input from an image capturing device, wherein the input is the retinal fundus image of the patient; an interactive retinal fundus image rendering means adapted to dynamically render the input, wherein the dynamically rendered input is configurably accessible on the graphical user interface via the user device using the interactive elements; a retinal fundus image capture means adapted to capture the retinal fundus image of the patient based on the dynamically rendered input; a processing means adapted to detect the eye type of the retinal fundus image of the patient, comprising: train each of a plurality of neural networks over a training dataset with an associated label for each of a training retinal fundus image in the training dataset, wherein the associated label is based on the location of an optic disc and a macula in the training retinal fundus image; and process the retinal fundus image to determine the eye type by the trained neural networks.

[0006] The user device is, for example, a personal computer, a laptop, a tablet computing device, a personal digital assistant, a client device, a web browser, etc. Here, the image capturing device refers to a camera for photographing the retinal fundus of the patient. The eye type is either a left eye or a right eye of the patient.

Brief description of the drawings

[0007] The present invention is described with reference to the accompanying figures. The accompanying figures, which are incorporated herein, are given by way of illustration only and form part of the specification together with the description to explain the make and use the invention, in which,

[0008] Figure 1 illustrates a block diagram of a computer implemented system to detect an eye type of a retinal fundus image of a patient in accordance with the invention;

[0009] Figure 2 exemplarily illustrates the architecture of a computer system employed by an eye type determination application; and

[0010] Figure 3 illustrates a flowchart for detect an eye type of a retinal fundus image of a patient in accordance with the invention.

Detailed description of the invention

[0011] Figure 1 illustrates a block diagram of a computer implemented system 100 to detect an eye type of a retinal fundus image of a patient in accordance with the invention. The system 100 comprises at least one processor; a non-transitory computer readable storage medium communicatively coupled to the at least one processor, the non-transitory computer readable storage medium configured to store an eye type determination application 103, the at least one processor configured to execute the eye type determination application 103; and the eye type determination application 103 comprising: a graphical user interface 103i comprising a plurality of interactive elements 103h configured to enable capture and process of the retinal fundus image of the patient via a user device, for example, 101a, 101b or 101c; a reception means 103a adapted to receive an input from an image capturing device, wherein the input is the retinal fundus image of the patient; an interactive retinal fundus image rendering means 103b adapted to dynamically render the input, wherein the dynamically rendered input is configurably accessible on the graphical user interface 103i via the user device 101a, 101b or 101c using the interactive elements 103i; a retinal fundus image capture means 103c adapted to capture the retinal fundus image of the patient based on the dynamically rendered input; a processing means 103g adapted to detect the eye type of the retinal fundus image of the patient, comprising: train each of a plurality of neural networks over a training dataset with an associated label for each of a training retinal fundus image in the training dataset, wherein the associated label is based on the location of an optic disc and a macula in the training retinal fundus image; and process the retinal fundus image to determine the eye type by the trained neural networks.

[0012] As used herein, the term “patient” refers to an individual receiving or registered to receive medical treatment. The patient is, for example, an individual undergoing a regular health checkup, an individual with a condition of diabetes mellitus, etc. As used herein, the term “retinal fundus image” refers to a two-dimensional array of digital image data, however, this is merely illustrative and not limiting of the scope of the invention.

[0013] The user device 101a, 101b or 101c is, for example, a personal computer, a laptop, a tablet computing device, a personal digital assistant, a client device, a web browser, etc. Here, the image capturing device refers to a camera for photographing the fundus of the patient.

[0014] In an embodiment, the eye type determination application 103 is a web application implemented on a web based platform, for example, a website hosted on a server or a setup of servers. For example, the eye type determination application 103 is implemented on a web based platform, for example, a retinal fundus image processing platform 104 as illustrated in Figure 1.

[0015] The retinal fundus image processing platform 104 hosts the eye type determination application 103. The eye type determination application 103 is accessible to one or more user devices 101a, 101b or 101c. In an example, the user device 101a, 101b or 101c is accessible over a network such as the internet, a mobile telecommunication network, a Wi-Fi® network of the Wireless Ethernet Compatibility Alliance, Inc., etc. The eye type determination application 103 is accessible through browsers such as Internet Explorer® (IE) 8, IE 9, IE 10, IE 11 and IE 12 of Microsoft Corporation, Safari® of Apple Inc., Mozilla® Firefox® of Mozilla Foundation, Chrome of Google, Inc., etc., and is compatible with technologies such as hypertext markup language 5 (HTML5), etc.

[0016] In another embodiment, the eye type determination application 103 is configured as a software application, for example, a mobile application downloadable by a user on the user device 101a, 101b or 101c, for example, a tablet computing device, a mobile phone, etc. As used herein, the term “user” is an individual who operates the eye type determination application 103 to capture the retinal fundus images of the patient and generate a report resulting from the processing of the captured retinal fundus images.

[0017] The eye type determination application 103 is accessible by the user device 101a, 101b or 101c via the GUI 103i provided by the eye type determination application 103. In an example, the eye type determination application 103 is accessible over the network 102. The network 102 is, for example, the internet, an intranet, a wireless network, a wired network, a Wi-Fi® network of the Wireless Ethernet Compatibility Alliance, Inc., a universal serial bus (USB) communication network, a ZigBee® network of ZigBee Alliance Corporation, a general packet radio service (GPRS) network, a global system for mobile (GSM) communications network, a code division multiple access (CDMA) network, a third generation (3G) mobile communication network, a fourth generation (4G) mobile communication network, a wide area network, a local area network, an internet connection network, an infrared communication network, etc., or any combination of these networks.

[0018] The eye type determination application 103 comprises the GUI 103i comprising a plurality of interactive elements 103h configured to enable capture and processing of the retinal fundus image via the user device 101a, 101b or 101c. As used herein, the term “interactive elements 103h” refers to interface components on the GUI 103i configured to perform a combination of processes, for example, a retrieval process from the input received from the user, for example, the retinal fundus images of the patient, processes that enable real time user interactions, etc. The interactive elements 103h comprise, for example, clickable buttons.

[0019] The eye type determination application 103 comprises the reception means 103a adapted to receive the input from the image capturing device. The input is the retinal fundus image of the patient. The input may also be a plurality of retinal fundus images of the patient. As used herein, the term “image capturing device” refers to a camera for photographing the fundus of the patient. In an example, the image capturing device is a Zeiss FF 450+ fundus camera comprising a Charged Coupled Device (CCD) photographic unit. In another example, the image capturing device is a smart phone with a camera capable of capturing the retinal fundus images of the patient.

[0020] The reception means 103a receives information associated with the patient from the user device, for example, 101a, 101b or 101c via the GUI 103i. The information associated with the patient is, for example, personal details about the patient, medical condition of the patient, etc.

[0021] The image capturing device is in communication with the eye type determination application 103 via the network 102, for example, the internet, an intranet, a wireless network, a wired network, a Wi-Fi® network of the Wireless Ethernet Compatibility Alliance, Inc., a universal serial bus (USB) communication network, a ZigBee® network of ZigBee Alliance Corporation, a general packet radio service (GPRS) network, a global system for mobile (GSM) communications network, a code division multiple access (CDMA) network, a third generation (3G) mobile communication network, a fourth generation (4G) mobile communication network, a wide area network, a local area network, an internet connection network, an infrared communication network, etc., or any combination of these networks.

[0022] The eye type determination application 103 accesses the image capturing device to receive the input. The eye type determination application 103 comprises a transmission means to request the image capturing device for a permission to control the activities of the image capturing device to capture the input associated with the patient. The image capturing device responds to the request received from the transmission means. The reception means 103a receives the response of the image capturing device.

[0023] In other words, the image capturing device permits the user of the eye type determination application 103 to control the activities of the image capturing device via the interactive elements 103h of the GUI 103i. As used herein, the term “activities” refer to a viewing of a live mode of the fundus of the patient on a screen of the GUI 103i, focusing a field of view by zooming in or zooming out the field of view to observe the fundus of the patient and capturing the retinal fundus image of the patient from the displayed live mode of the fundus of the patient.

[0024] In an embodiment, the eye type determination application 103 adaptably controls the activities specific to the image capturing device based on a plurality of parameters of the image capturing device. The eye type determination application 103 is customizable to suit the parameters of the image capturing device such as the version, the manufacturer, the model details, etc. In other terms, the eye type determination application 103 is customizable and can be suitable adapted to capture the retinal fundus images of the patient for different manufacturers of the image capturing device.

[0025] Once the eye type determination application 103 has the permission to control the activities of the image capturing device, the user of the eye type determination application 103 can view the input of the image capturing device on the screen of the GUI 103i. The interactive retinal fundus image rendering means 103b dynamically renders the input on the GUI 103i. The dynamically rendered input is configurably accessible on the GUI 103i via the user device 101a, 101b or 101c using the interactive elements 103h. The field of view of the image capturing device is displayed on a screen of the GUI 103i via the user device 101a, 101b or 101c. The user can focus the field of view by zooming in or zooming out the field of view to observe the fundus of the patient by using with the interactive elements 103h via a user input device such as a mouse, a trackball, a joystick, etc. The user captures the retinal fundus image of the patient from the displayed live mode of the fundus of the patient using the interactive elements 103h of the GUI 103i via the user device 101a, 101b or 101c. As used herein, the term “live mode” refers to the seamless display of the fundus of the patient in real time via the GUI 103i. In an embodiment, the input is an already existing retinal fundus image of the patient stored in the database 104a.

[0026] The processing means 103g trains the neural networks over the training dataset with the associated label for each of the training retinal fundus image in the training dataset. As used herein, the term “training retinal fundus image” is a two-dimensional array of digital image data used for the purpose of training the neural networks. In this invention, the term ‘training’ refers to a process of developing the neural networks for the processing the retinal fundus image based the training dataset and the ground-truth file. Each of the training retinal fundus image in the training dataset is associated with a label.

[0027] The associated label for each of the training retinal fundus image comprises a predefined mask for the training retinal fundus image based on a presence of the optic disc and the macula and an eye type of the training retinal fundus image. The predefined mask of the training retinal fundus image indicates a location and a structure of the optic disc and the macula. The predefined mask of the training retinal fundus image with a map with the location and structure of the optic disc and the macula in the training retinal fundus image. The eye type of the training retinal fundus image is either a left eye or a right eye. The associated label comprises an indication about the eye type of the training retinal fundus image.

[0028] The ground-truth file comprises the associated label and a retinal fundus image identifier for each of the training retinal fundus image. The retinal fundus image identifier of the training retinal fundus image is, for example, a name or an identity assigned to the training retinal fundus image.

[0029] In an embodiment, the predefined mask is a mask for the training retinal fundus image as annotated by an annotator. The annotator annotates the training fundus image by analyzing the training retinal fundus image and create the predefined mask which is an indication of the location and structure of the optic disc and the macula in the training retinal fundus image. For each training fundus image in the training dataset, the annotator defines a mask for the optic disc and the macula. Here, ‘mask’ is a highlighted colored region indicating a structure of the optic disc and the macula. The optic disc and the macula are identified by a pre-defined color associated with it. The annotator completes the predefined mask by marking the mask for optic disc and the macula on the training fundus image. The predefined mask provides an insight into the location and structure of the optic disc and the macula in the training retinal fundus image.

[0030] In another embodiment, the processing means 103g employs image processing methods to define the predefined mask for each of the training retinal fundus image in the training dataset.

[0031] The associated label for each of the training retinal fundus image also comprises details about the eye type of the training retinal fundus image. Here, the “eye type” refers to whether the training retinal fundus image belongs to either the left eye or the right eye of the patient. The processing means 103g adapted to determining the eye type of the training retinal fundus image comprises the following steps: dividing the retinal fundus image into a predefined number of rows and columns to have a predefined number of regions; computing a region pixel intensity score for each region based on a pixel intensity distribution; comparing an optic disc pixel intensity score with the region pixel intensity score of each of the regions to determine a first region with the presence of the optic disc; comparing a macula pixel intensity score with the region pixel intensity score of each of the regions to determine a second region with the presence of the macula; determining a presence of the first region and the second region in a central row of the retinal fundus image; voting for a left eye when the first region is located to the left of the second region in the central row of the retinal fundus image; and voting for the right eye when the first region is located to the right of the second region in the central row of the retinal fundus image.

[0032] The predefined number of rows are three, the predefined number of columns are three and the predefined number of regions are nine. The processing means 103g discards the retinal fundus image on an absence of either the optic disc or the macula in the central row of the retinal fundus image. The processing means 103g divides each of the training retinal fundus image into nine equal regions – with three rows and three columns. The three row are a top row, the central row and a bottom row.

[0033] As used herein, the term “region pixel intensity score” refers to an average value of the pixel intensities of the pixels in the region and is based on the pixel intensity distribution for the region. The optic disc pixel intensity score is a predetermined value indicating the optic disc based on a level of brightness. The macula pixel intensity score is a predetermined value indicating the macula based on a level of brightness. In an example, the optic disc pixel intensity score and the macula pixel intensity score are predetermined by the user of the system 100.

[0034] The processing means 103g compares the region pixel intensity score of each of the regions with the optic disc pixel intensity score to determine the location of the optic disc in the retinal fundus image. When a difference between the region pixel intensity score of a region and the optic disc pixel intensity score is below a first threshold, then the processing means 103g determines that the optic disc is present in the region. The processing means 103g determines the region as the first region with the presence of the optic disc. The first threshold is predefined by the user of the system 100.

[0035] Similarly, the processing means 103g compares the region pixel intensity score of each of the regions with the macula pixel intensity score to determine the location of the macula in the retinal fundus image. When a difference between the region pixel intensity score of a region and the macula pixel intensity score is below a second threshold, then the processing means 103g determines that the macula is present in the region. The processing means 103g determines the region as the second region with the presence of the macula. The second threshold is predefined by the user of the system 100.

[0036] In an example, the processing means 103g initiates the comparison from a top left corner region and continues the comparison row-wise to determine the first region and the second region. In an embodiment, the processing means 103g discards the retinal fundus image in an absence of either the first region or the second region in the central row of the divided retinal fundus image. That is, the processing means 103g discards the retinal fundus image in the absence of either the optic disc or the macula in the central row of the divided retinal fundus image. Since the relative locations of both the optic disc and the macula are required for the determination of the eye type, the processing means 103g discards the retinal fundus image in the absence of either the first region or the second region is the central row of the divided retinal fundus image.

[0037] When either the optic disc or the macula is detected in a top row or a bottom row, the determination of the eye type may yield wrong predictions and hence the processing means 103g discards such retinal fundus images. For example, if the optic disc of the macula is present in the top row and the macula is missing from the captured retinal fundus image, then the eye type of the retinal fundus image cannot be determined. To rule out such possibilities, the processing means 103g considers the presence of the optic disc or the macula in the central row to determine the eye type of the retinal fundus image.

[0038] The processing means 103g discards the retinal fundus image in the absence of either the optic disc or the macula in the retinal fundus image. The processing means 103g votes for either the left eye or the right eye when the processing means 103g determines the presence of both the optic disc and the macula in the central row of the divided retinal fundus image.

[0039] As used herein, the term “neural networks” refers to a class of deep artificial neural networks that can be applied to analyzing visual imagery. The neural networks corresponds to a specific model of an artificial neural network. In an embodiment, a first neural network generates a mask for the retinal fundus image of the patient. The mask refers is a map of the retinal fundus image indicating the locations and structures of the optic disc and the macula in the retinal fundus image of the patient. A second neural network further determines the eye type based on the presence and locations of the optic disc and the macula. In another embodiment, a single neural network generates the mask for the retinal fundus image of the patient and determines the eye type based on the presence and locations of the optic disc and the macula in the retinal fundus image. In another embodiment, multiple neural networks together process the retinal fundus image of the patient.

[0040] The neural networks are trained over a training dataset of retinal fundus images to accomplish a function associated with the neural networks. Here, the term “function” of the neural networks refers to the processing of the retinal fundus image to identify the occurrence of the optic disc and the macula in the retinal fundus image; generate the mask displaying the location and structure of the optic disc and the macula in the received retinal fundus image of the patient; and determine the eye type of the retinal fundus image of the patient.

[0041] In an embodiment, the trained first neural network determines a category for each pixel in the retinal fundus image based on a pixel intensity; and identifies the location and structure of the optic disc and the macula in the retinal fundus image based on the determined category of the pixels. The category defines a division of retinal features. The retinal features are the optic disc and the macula.

[0042] The trained first neural network thus generates the “mask” denoting the structure and location of the optic disc and the macula in the retinal fundus image by highlighting the optic disc and the macula with a predetermined pixel intensity. The predetermined pixel intensity indicates a distinct color to distinguish the optic disc from the macula.

[0043] In an embodiment, the trained second neural network determines the eye type of the retinal fundus image of the patient. The trained second neural network determines whether the retinal fundus image of the patient belongs to either the left eye or the right eye of the patient.

[0044] The eye type determination application 103 receives the training dataset from one or more devices. The training dataset comprises a plurality of retinal fundus images. The retinal fundus images in the training dataset are referred to as training retinal fundus images. The device is, for example, the image capturing device such as a camera incorporated into a mobile device, a server, a network of personal computers, or simply a personal computer, a mainframe, a tablet computer, etc. The eye type determination application 103 stores the training dataset in a database 104a of the system 100. The system 100 comprises the database 104a in communication with the eye type determination application 103. The database 104a is also configured to store patient profile information, patient medical history, the training retinal fundus images of patients, reports of the patients, etc.

[0045] The eye type determination application 103 comprises a pre-processing means 103d to pre-process each of the training retinal fundus images. The pre-processing means 103d communicates with the database 104a to access the training dataset. For each of the training retinal fundus image, the pre-processing means 103d executes the following steps as part of the pre-processing. The pre-processing means 103d calculates a bounding box for the retina to identify a retina from the training retinal fundus image using image processing methods. The pre-processing means 103d further blurs the training retinal fundus image using a Gaussian filter. The pre-processing means 103d extracts foreground pixel values from the blurred training retinal fundus image. The pre-processing means 103d processes the bounding box to obtain a resized image using cubic interpolation of shape. The training retinal fundus image at this stage is referred to as the pre-processed training retinal fundus image. The pre-processing means 103d stores the pre-processed training retinal fundus images in a pre-processed training dataset. The ground-truth file associated with the training dataset holds good even from the pre-processed training dataset. The pre-processing means 103d stores the pre-processed training dataset in the database 104a.

[0046] The eye type determination application 103 further comprises a segregation means 103e. The segregation means 103e splits the pre-processed training dataset into two sets – a learning set and a validation set. The learning set is used to train the neural networks to assess the training retinal fundus images based on the associated label with each of the training retinal fundus image. In an embodiment, the first neural network learns to generate the predefined mask with the location and structure of the optic disc and the macula in the training retinal fundus images. The second neural network learns to determine the eye type of the training retinal fundus images. The validation set is typically used to test the accuracy of both the neural networks.

[0047] The eye type determination application 103 further comprises an augmentation means 103f. The augmentation means 103f augments the learning set of each of the neural networks. For each of the neural networks, the augmentation means 103f preforms the following steps for the augmentation of the learning set. The augmentation means 103f randomly shuffles the training retinal fundus images to divide the learning set into a plurality of batches. Each batch is a collection of a predefined number of training retinal fundus images. The augmentation means 103f randomly samples each batch of training retinal fundus images and corresponding labels. The augmentation means 103f processes each batch of the training retinal fundus images and corresponding labels using affine transformations.

[0048] The processing means 103g trains each of the neural networks using the batches of augmented training retinal fundus images of the learning set. For each of the neural networks, the segregation means 103e groups the training retinal fundus images of the validation set into a plurality of batches. The processing means 103g validates each of the training retinal fundus images in each batch of the validation set using the neural networks. The processing means 103g compares a result of the validation against a corresponding label of the training retinal fundus image by referring to the ground-truth file. The processing means 103g evaluates a network performance for the batch of validation set of each of the neural networks.

[0049] The processing means 103g optimizes the neural networks parameters using an optimizer, for example, a Nadam optimizer which is an Adam optimizer with Nesterov Momentum. The optimizer iteratively optimizes the parameters of each of the neural networks during multiple iterations using the corresponding learning set. Here, each iteration refers to a batch of the learning set of the neural network. The processing means 103g evaluates the network performance of each of the neural networks after a predefined number of iterations on the validation set. Here, each iteration refers to a batch of the validation set.

[0050] The processing means 103g trains the neural networks based on the augmented learning set and tests the neural networks based on the validation set. Upon completion of training and validation of the neural networks based on the convolutional network performance, the processing means 103g is ready to generate the candidate segment mask and determine the eye type for the retinal fundus image of the patient. Hereafter, the retinal fundus image of the patient received by the eye type determination application 103 for processing is referred to as input retinal fundus image.

[0051] The pre-processing means 103d of the eye type determination application 103 receives the input retinal fundus image from, for example, the image capturing device. The pre-processing means 103d processes the input retinal fundus image in the same method as the training retinal fundus image. The pre-processing means 103d transmits the preprocessed the input retinal fundus image to the processing means 103g. The processing means 103g processes the preprocessed input retinal fundus image using the neural networks. The processing means 103g determines the category of each pixel based on the pixel intensity value of the pixel using the first neural network. The processing means 103g identifies the location and structure of the optic disc and the macula in the retinal fundus image based on the determined category of the pixels using the first neural network. Here, the “category” defines a division of retinal features. The retinal features are the optic disc and the macula. The final prediction result of the first neural network provides a probability value for each category for each pixel of the input retinal fundus image. The probability value is an indication of a confidence denoting the category to which each pixel of the input retinal fundus image belongs to. The output in turn indicates the mask associated with the input retinal fundus image. The second neural network further identifies the eye type of the input retinal fundus image.

[0052] The eye type determination application 103 further superimposes the mask and the input retinal fundus image and displays on the GUI 103i. The eye type determination application 103 also displays the eye type, for example, left eye or right eye, for the input retinal fundus image. In an example, suitable messages to the user are and provided via a pop-up box displayed on a screen. The eye type determination application 103 may also generate a report comprising the superimposed mask and the input retinal fundus image and an indication of the eye type of the input retinal fundus image. The eye type determination application 103 communicates the report to the patient via an electronic mail. The report could also be stored in the database 104a of the system 100.

[0053] In an embodiment, other retinal features are also identified and located on the mask of the retinal fundus image. This makes the further analysis of the input retinal fundus image easier for the medical practitioner in identifying retinal diseases and the severity of the retinal diseases.

[0054] Figure 2 exemplarily illustrates the architecture of a computer system 200 employed by the eye type determination application 103. The eye type determination application 103 of the computer implemented system 100 exemplarily illustrated in Figure 1 employs the architecture of the computer system 200 exemplarily illustrated in Figure 3. The computer system 200 is programmable using a high level computer programming language. The computer system 200 may be implemented using programmed and purposeful hardware.

[0055] The retinal fundus image processing platform 104 hosting the eye type determination application 103 communicates with user devices, for example, 101a, 101b, 101c, etc., of a user registered with the eye type determination application 103 via the network 102. The network 102 is, for example, the internet, a local area network, a wide area network, a wired network, a wireless network, a mobile communication network, etc. The computer system 200 comprises, for example, a processor 201, a memory unit 202 for storing programs and data, an input/output (I/O) controller 203, a network interface 204, a data bus 205, a display unit 206, input devices 207, fixed disks 208, removable disks 209, output devices 210, etc.

[0056] As used herein, the term “processor” refers to any one or more central processing unit (CPU) devices, microprocessors, an application specific integrated circuit (ASIC), computers, microcontrollers, digital signal processors, logic, an electronic circuit, a field-programmable gate array (FPGA), etc., or any combination thereof, capable of executing computer programs or a series of commands, instructions, or state transitions. The processor 201 may also be realized as a processor set comprising, for example, a math or graphics co-processor and a general purpose microprocessor. The processor 201 is selected, for example, from the Intel® processors such as the Itanium® microprocessor or the Pentium® processors, Advanced Micro Devices (AMD®) processors such as the Athlon® processor, MicroSPARC® processors, UltraSPARC® processors, hp® processors, International Business Machines (IBM®) processors, the MIPS® reduced instruction set computer (RISC) processor, Inc., RISC based computer processors of ARM Holdings, etc. The computer implemented system 100 disclosed herein is not limited to a computer system 200 employing a processor 201 but may also employ a controller or a microcontroller.

[0057] The memory unit 202 is used for storing data, programs, and applications. The memory unit 202 is, for example, a random access memory (RAM) or any type of dynamic storage device that stores information for execution by the processor 201. The memory unit 202 also stores temporary variables and other intermediate information used during execution of the instructions by the processor 201. The computer system 200 further comprises a read only memory (ROM) or another type of static storage device that stores static information and instructions for the processor 201.

[0058] The I/O controller 203 controls input actions and output actions performed by the eye type determination application 103. The network interface 204 enables connection of the computer system 200 to the network 102. For example, the retinal fundus image processing platform 104 hosting the eye type determination application 103 connects to the network 102 via the network interface 204. The network interface 204 comprises, for example, one or more of a universal serial bus (USB) interface, a cable interface, an interface implementing Wi-Fi® of the Wireless Ethernet Compatibility Alliance, Inc., a FireWire® interface of Apple, Inc., an Ethernet interface, a digital subscriber line (DSL) interface, a token ring interface, a peripheral controller interconnect (PCI) interface, a local area network (LAN) interface, a wide area network (WAN) interface, interfaces using serial protocols, interfaces using parallel protocols, and Ethernet communication interfaces, asynchronous transfer mode (ATM) interfaces, interfaces based on transmission control protocol (TCP)/internet protocol (IP), radio frequency (RF) technology, etc. The data bus 205 permits communications between the means/modules (103a, 103b, 103c, 103d, 103e, 103f and 103g) of the eye type determination application 103.

[0059] The display unit 206, via the GUI 103i, displays information, display interfaces, interactive elements 103h j such as drop down menus, text fields, checkboxes, text boxes, floating windows, hyperlinks, etc., for example, for allowing the user to enter inputs associated with the patient. In an example, the display unit 206 comprises a liquid crystal display, a plasma display, etc. The input devices 207 are used for inputting data into the computer system 200. A user, for example, an operator, registered with the eye type determination application 103 uses one or more of the input devices 207 of the user devices, for example, 101a, 101b, 101c, etc., to provide inputs to the eye type determination application 103. For example, a user may enter a patient’s profile information, the patient’s medical history, etc., using the input devices 207. The input devices 207 are, for example, a keyboard such as an alphanumeric keyboard, a touch pad, a joystick, a computer mouse, a light pen, a physical button, a touch sensitive display device, a track ball, etc.

[0060] Computer applications and programs are used for operating the computer system 200. The programs are loaded onto the fixed disks 208 and into the memory unit 202 of the computer system 200 via the removable disks 209. In an embodiment, the computer applications and programs may be loaded directly via the network 102. The output devices 210 output the results of operations performed by the eye type determination application 103.

[0061] The processor 201 executes an operating system, for example, the Linux® operating system, the Unix® operating system, any version of the Microsoft® Windows® operating system, the Mac OS of Apple Inc., the IBM® OS/2, VxWorks® of Wind River Systems, Palm OS®, the Solaris operating system, the Android operating system, Windows Phone™ operating system developed by Microsoft Corporation, the iOS operating system of Apple Inc., etc.

[0062] The computer system 200 employs the operating system for performing multiple tasks. The operating system is responsible for management and coordination of activities and sharing of resources of the computer system 200. The operating system employed on the computer system 200 recognizes, for example, inputs provided by the user using one of the input devices 207, the output display, files, and directories stored locally on the fixed disks 208. The operating system on the computer system 200 executes different programs using the processor 201. The processor 201 and the operating system together define a computer platform for which application programs in high level programming languages are written.

[0063] The processor 201 retrieves instructions for executing the modules (103a, 103b, 103c, 103d, 103e, 103f and 103g) of the eye type determination application 103 from the memory unit 202. A program counter determines the location of the instructions in the memory unit 202. The program counter stores a number that identifies the current position in the program of each of the modules (103a, 103b, 103c, 103d, 103e, 103f and 103g) of the eye type determination application 103. The instructions fetched by the processor 201 from the memory unit 202 after being processed are decoded. The instructions are stored in an instruction register in the processor 201. After processing and decoding, the processor 201 executes the instructions.

[0064] Figure 3 illustrates a flowchart for detect an eye type of a retinal fundus image of a patient in accordance with the invention. At step S1, the eye type determination application 103 receives the retinal fundus image of the patient. The non-transitory computer readable storage medium is configured to store the eye type determination application 103 and at least one processor is configured to execute the eye type determination application 103. The eye type determination application 103 is thus a part of the system 100 comprising the non-transitory computer readable storage medium communicatively coupled to the at least one processor. The eye type determination application 103 comprises the GUI 103i comprising multiple interactive elements 103h configured to enable capture and processing of the retinal fundus image via the user device 101a, 101b or 101c. The reception means 103a is adapted to receive the input from the image capturing device. The input is the retinal fundus image of the patient displayed in a live mode. In an embodiment, the eye type determination application 103 is a web application implemented on a web based platform, for example, a website hosted on a server or a setup of servers.

[0065] At step S2, the interactive retinal fundus image rendering means 103b is adapted to dynamically render the input. The dynamically rendered input is configurably accessible on the GUI 103i via the user device 101a, 101b or 101c using the interactive elements 103h. At step S3, the retinal fundus image capture means 103c is adapted to capture the retinal fundus image based on the dynamically rendered input.

[0066] At step S4, the processing means 103g is adapted train each of the neural networks over the training dataset with the associated label for each of the training retinal fundus image in the training dataset. The associated label is based on the location of the optic disc and the macula in the training retinal fundus image.

[0067] At step S5, the processing means 103g is adapted to process the retinal fundus image to process the retinal fundus image to determine the eye type by the trained neural networks. The trained first neural network denotes the locations of the optic disc and the macula which is a pictorial indication of the eye type of the retinal fundus image. The trained second neural network provides a message of the eye type of the retinal fundus image of the patient.

[0068] The system 100 reduces errors in resulting from manual identification of the eye type during screening of the patient for various medical conditions. The system 100 acts as an important supporting tool in detecting/monitoring one or more retinal diseases and/or or a response to a therapy. The system 100 reduces the time-consumption involved in a manual recording of the eye type of each of the retinal fundus image by the user while capturing digital photographs of the retinal fundus.

[0069] The present invention described above, although described functionally or sensibly, may be configured to work in a network environment comprising a computer in communication with one or more devices. It will be readily apparent that the various methods, algorithms, and computer programs disclosed herein may be implemented on computer readable media appropriately programmed for general purpose computers and computing devices. As used herein, the term “computer readable media” refers to non-transitory computer readable media that participate in providing data, for example, instructions that may be read by a computer, a processor or a similar device. Non-transitory computer readable media comprise all computer readable media. Non-volatile media comprise, for example, optical discs or magnetic disks and other persistent memory volatile media including a dynamic random access memory (DRAM), which typically constitutes a main memory. Volatile media comprise, for example, a processor cache, a register memory, a random access memory (RAM), etc. Transmission media comprise, for example, coaxial cables, copper wire, fiber optic cables, modems, etc., including wires that constitute a system bus coupled to a processor, etc. Common forms of computer readable media comprise, for example, a floppy disk, a flexible disk, a hard disk, magnetic tape, a Blu-ray Disc®, a magnetic medium, a compact disc-read only memory (CD-ROM), a digital versatile disc (DVD), any optical medium, a flash memory card, a laser disc, RAM, a programmable read only memory (PROM), an erasable programmable read only memory (EPROM), an electrically erasable programmable read only memory (EEPROM), a flash memory, any other cartridge, etc.

[0070] The database 104a is, for example, a structured query language (SQL) data base or a not only SQL (NoSQL) data base such as the Microsoft® SQL Server®, the Oracle® servers, the MySQL® database of MySQL AB Company, the MongoDB® of 10gen, Inc., the Neo4j graph database, the Cassandra database of the Apache Software Foundation, the HBase™ database of the Apache Software Foundation, etc. In an embodiment, the database 104a can also be a location on a file system. The database 104a is any storage area or medium that can be used for storing data and files. In another embodiment, the database 104a can be remotely accessed by the eye type determination application 103 via the network 102. In another embodiment, the database 104a a is configured as a cloud based database 104a implemented in a cloud computing environment, where computing resources are delivered as a service over the network 102, for example, the internet.

The foregoing examples have been provided merely for the purpose of explanation and does not limit the present invention disclosed herein. While the invention has been described with reference to various embodiments, it is understood that the words are used for illustration and are not limiting. Those skilled in the art, may effect numerous modifications thereto and changes may be made without departing from the scope and spirit of the invention in its aspects.