Description:METHOD, SYSTEM AND APPARATUS FOR IMAGE-BASED ANALYSIS OF GRAIN KERNELS

FIELD OF THE PRESENT DISCLOSURE
[0001] The present disclosure relates generally to image analysis and processing. More particularly, the present disclosure relates to a computer-implemented method, a system, and an integrated apparatus for analysing images of grain kernels, specifically by classifying grain kernels based on their features and defects, ascertaining their relative weights, and determining their quality.

BACKGROUND
[0002] Grain production forms the backbone of global agriculture, supplying essential nutrients and serving as a primary food source for a substantial portion of the world's population. The grain industry, comprising processes such as cultivation, storage, marketing, and distribution of grains, plays a crucial role in global food security and economic stability. Given the magnitude of this industry, the quality and safety of the grains are paramount. Grain quality can significantly affect the price of the grain, its suitability for specific uses, and its acceptability to consumers.
[0003] Traditionally, the evaluation of grain quality and safety was accomplished manually, involving physical and sensory examination of grain samples. This labour-intensive approach often led to inconsistencies and inaccuracies, largely due to its subjective nature and the human error factor. Furthermore, the manual process is time-consuming and not feasible for large-scale operations. With the increasing demand for high-quality grains and the need for efficient, accurate, and objective grain assessment techniques, modern technologies have been integrated into the grain industry.
[0004] A significant step forward has been the incorporation of imaging technologies for grain analysis, leveraging computer vision and machine learning techniques. These systems capture images of grain samples and use algorithms to analyse various grain characteristics, such as size, shape, colour, and surface texture. This digital approach enhances the accuracy and objectivity of grain assessment, but also introduces new challenges.
[0005] One such challenge lies in the segmentation of grains in the captured images, especially when the grains are touching or overlapping. Existing image segmentation techniques, such as edge detection or thresholding, may fail to accurately separate touching grains, leading to inaccurate analysis. Moreover, these techniques often struggle to handle the inherent variability in grain characteristics, such as different shapes and sizes, leading to inconsistent results. Advanced image segmentation methods, such as watershed transformation, have been introduced to tackle these issues, but they are not always sufficient, especially when grains have an elliptic shape and when clusters of grains contain many objects.
[0006] Apart from segmentation, another challenge is the detection and classification of defects in grains. Grains may exhibit a variety of defects, including cracks, breaks, discolorations, and contaminations. Detecting and classifying these defects accurately is essential for determining grain quality and safety. Traditional machine vision techniques, such as feature extraction and pattern recognition, have been used for defect detection and classification. However, these techniques often require intricate design and tuning, and may not perform well with complex and subtle defects.
[0007] Deep learning techniques, such as convolutional neural networks (CNNs), have shown promising results in image analysis tasks, including segmentation and defect detection. They can learn complex patterns and hierarchies of features directly from image data, offering potential improvements in performance. However, training deep learning models requires large amounts of labelled data, which can be difficult and expensive to obtain in the context of grain analysis. Additionally, deep learning models can be computationally intensive and require substantial resources, which may not always be available in grain processing facilities.
[0008] Moreover, the assessment of grain quality is not limited to the analysis of individual grain characteristics. Other aspects, such as the estimation of grain weight and the evaluation of grain group characteristics, are also important. Conventional methods for estimating grain weight often involve direct measurement or volumetric approximation, both of which may not be accurate or feasible for large quantities of grains. Similarly, the evaluation of grain group characteristics usually requires manual counting and categorization of grains, which is time-consuming and prone to error.
[0009] In light of these challenges, there is a need for an advanced, reliable, and efficient method for image-based analysis of grain kernels. The method should be able to accurately segment touching or overlapping grains, detect and classify grain defects, estimate grain weights, and evaluate grain group characteristics. Furthermore, it should be capable of leveraging advanced machine learning techniques, while being practical for implementation in real-world grain processing facilities.

SUMMARY
[0010] The present disclosure addresses these needs by providing a computer-implemented method, a system and an apparatus employing a combination of image processing techniques, deep learning, and machine vision technologies. The present disclosure allows for accurate segmentation of touching grains, precise detection and classification of defects, and the computation of grain weights and group characteristics. The result is a comprehensive, objective, and efficient grain assessment method that can enhance grain marketing, storage, and processing, ultimately improving food quality, safety, and sustainability.
[0011] In an aspect, the present disclosure provides a computer-implemented method for the image-based analysis of grain kernels. The technique encompasses obtaining a sample image of the grain kernels, and implementing one or more image pre-processing operations on this image to generate a corresponding binary image. Subsequently, a distance function is computed on the binary image, with each pixel assigned a numeric value representing the shortest distance to a nearest zero-pixel, leading to the creation of a distance map. From this map, a morphological skeleton of the binary image is derived using the maxima of the distance function, where each point in the skeleton signifies the centre of a maximum inscribed circle within the associated grain kernel. A U-Net based convolutional neural network model is applied on the distance map and the morphological skeleton to generate a segmentation map, which demarcates the boundaries of each grain kernel in the sample image. Finally, the grain kernels are classified based on the segmentation map, assigning each kernel to a corresponding group representing varying types of defects, including intact, broken, slightly cracked, and strongly cracked grains.
[0012] In one or more embodiments, the method includes estimating relative weights of each grain kernel within their respective groups by applying a weight-per-pixel metric to the area of each kernel. The estimated relative weights of each kernel within the same group are cumulatively summed to calculate an overall estimated weight for the respective group. Furthermore, a weight ratio for each group is calculated using the overall area and the weight-per-pixel metric.
[0013] In one or more embodiments, the U-Net based convolutional neural network model is composed of an encoding module, which performs a series of convolution operations to capture contextual features from the distance map and the morphological skeleton. There is also a decoding module, symmetric to the encoding module, which performs a series of convolution and transpose convolution operations for generating the segmentation map.
[0014] In one or more embodiments, the method includes refining the segmentation map by utilizing the morphological skeleton to draw clipping lines between touching grain kernels in the binary image. Post refinement, the grain kernels are classified based on the segmentation map, thereby distinctly demarcating boundaries of the grain kernels in the sample image.
[0015] In one or more embodiments, the method includes utilizing an open-line detection technique to define open-lines for identifying concavities in the grain kernels within the morphological skeleton. Additionally, these open-lines are paired to generate possible split lines between touching or overlapping grain kernels in the morphological skeleton.
[0016] In one or more embodiments, the method includes utilizing one or more machine-vision techniques on the segmentation map to classify the grain kernels.
[0017] In one or more embodiments, the image pre-processing operations include grayscale conversion, noise reduction, and thresholding.
[0018] In another aspect, the present disclosure provides a system for image-based analysis of grain kernels. The system includes an image acquisition unit designed to obtain a sample image of the grain kernels, and a processing unit configured to communicate with the image acquisition unit, receiving the sample image. The processing unit is designed to implement the steps as outlined in the method, from implementing image pre-processing operations to classify the grain kernels.
[0019] In yet another aspect, the present disclosure provides an apparatus for the image-based analysis of grain kernels. The apparatus comprises a housing, an image acquisition unit, a processing unit, and an output device all integrated within the housing. The image acquisition unit captures a sample image of the grain kernels, and the processing unit carries out a series of operations from image pre-processing to grain kernel classification, as outlined in the method. The output device generates a report based on the classification of the grain kernels.
[0020] In still another aspect, the present disclosure provides a computer program product, which contains computer program code. Upon execution by a processing unit, the code prompts the processing module to carry out the steps as outlined in the method.

BRIEF DESCRIPTION OF THE FIGURES
[0021] For a more complete understanding of example embodiments of the present disclosure, reference is now made to the following descriptions taken in connection with the accompanying drawings in which:
[0022] FIG. 1 illustrates a flowchart of a method for image-based analysis of grain kernels, in accordance with one or more embodiments of the present disclosure;
[0023] FIG. 2 illustrates a block diagram of a system for image-based analysis of grain kernels, in accordance with one or more embodiments of the present disclosure;
[0024] FIG. 3A illustrates a perspective view of an apparatus for image-based analysis of grain kernels, in accordance with one or more embodiments of the present disclosure;
[0025] FIG. 3B depicts implementation of the apparatus of FIG. 3A for placement of grain kernels therein, in accordance with one or more embodiments of the present disclosure;
[0026] FIG. 4A illustrates a sample image, obtained using the apparatus, of the grain kernels placed thereon, in accordance with one or more embodiments of the present disclosure;
[0027] FIG. 4B illustrates a binary image generated using the sample image, in accordance with one or more embodiments of the present disclosure;
[0028] FIG. 5A illustrates a portion of the binary image for further analysis, in accordance with one or more embodiments of the present disclosure;
[0029] FIG. 5B illustrates a morphological skeleton derived for the portion of the binary image of FIG. 5A, in accordance with one or more embodiments of the present disclosure;
[0030] FIG. 5C illustrates a depiction of the binary image over the morphological skeleton, in accordance with one or more embodiments of the present disclosure;
[0031] FIG. 6A illustrates a stage of end-points localisation in a gap-filling process, in accordance with one or more embodiments of the present disclosure;
[0032] FIG. 6B illustrates a stage of open-lines direction evaluation in a gap-filling process, in accordance with one or more embodiments of the present disclosure;
[0033] FIG. 6C illustrates a stage of open-lines prolongation in a gap-filling process, in accordance with one or more embodiments of the present disclosure;
[0034] FIG. 6D illustrates a stage of end-points junction in a gap-filling process, in accordance with one or more embodiments of the present disclosure;
[0035] FIG. 7A illustrates a stage of detection of two lines proximity in a pairing procedure, in accordance with one or more embodiments of the present disclosure;
[0036] FIG. 7B illustrates a stage of direction calculation in a pairing procedure, in accordance with one or more embodiments of the present disclosure;
[0037] FIG. 7C illustrates a stage of determining prolongation length in a pairing procedure, in accordance with one or more embodiments of the present disclosure;
[0038] FIG. 8A illustrates a first stage in a sequence of pairing process involving the open-lines with low values of parameters Lp, Rn and Lc, in accordance with one or more embodiments of the present disclosure;
[0039] FIG. 8B illustrates a second stage in a sequence of pairing process involving the open-lines with medium values of parameters Lp, Rn and Lc, in accordance with one or more embodiments of the present disclosure;
[0040] FIG. 8C illustrates a third stage in a sequence of pairing process involving the open-lines with high values of parameters Lp, Rn and Lc, in accordance with one or more embodiments of the present disclosure;
[0041] FIG. 9A illustrates result of gap-filling on morphological skeleton, in accordance with one or more embodiments of the present disclosure;
[0042] FIG. 9B illustrates result of gap-filling on binary image, in accordance with one or more embodiments of the present disclosure; and
[0043] FIG. 10 illustrates a segmentation map obtained by implementation of the steps of the method, in accordance with one or more embodiments of the present disclosure.

DETAILED DESCRIPTION
[0044] In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. It will be apparent, however, to one skilled in the art that the present disclosure is not limited to these specific details.
[0045] Reference in this specification to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. The appearance of the phrase “in one embodiment” in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Further, the terms “a” and “an” herein do not denote a limitation of quantity, but rather denote the presence of at least one of the referenced items. Moreover, various features are described which may be exhibited by some embodiments and not by others. Similarly, various requirements are described which may be requirements for some embodiments but not for other embodiments.
[0046] Furthermore, in the following detailed description of the present disclosure, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. However, it will be understood that the present disclosure may be practiced without these specific details. In other instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to unnecessarily obscure aspects of the present disclosure.
[0047] Embodiments described herein may be discussed in the general context of computer-executable instructions residing on some form of computer-readable storage medium, such as program modules, executed by one or more computers or other devices. By way of example, and not limitation, computer-readable storage media may comprise non-transitory computer-readable storage media and communication media; non-transitory computer-readable media include all computer-readable media except for a transitory, propagating signal. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform particular tasks or implement particular abstract data types. The functionality of the program modules may be combined or distributed as desired in various embodiments.
[0048] Some portions of the detailed description that follows are presented and discussed in terms of a process or method. Although steps and sequencing thereof are disclosed in figures herein describing the operations of this method, such steps and sequencing are exemplary. Embodiments are well suited to performing various other steps or variations of the steps recited in the flowchart of the figure herein, and in a sequence other than that depicted and described herein. Some portions of the detailed descriptions that follow are presented in terms of procedures, logic blocks, processing, and other symbolic representations of operations on data bits within a computer memory. These descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. In the present application, a procedure, logic block, process, or the like, is conceived to be a self-consistent sequence of steps or instructions leading to a desired result. The steps are those utilizing physical manipulations of physical quantities. Usually, although not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated in a computer system. It has proven convenient at times, principally for reasons of common usage, to refer to these signals as transactions, bits, values, elements, symbols, characters, samples, pixels, or the like.
[0049] In some implementations, any suitable computer usable or computer readable medium (or media) may be utilized. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer-usable, or computer-readable, storage medium (including a storage device associated with a computing device) may be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable medium may include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fibre, a portable compact disc read-only memory (CD-ROM), an optical storage device, a digital versatile disk (DVD), a static random access memory (SRAM), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, a media such as those supporting the internet or an intranet, or a magnetic storage device. Note that the computer-usable or computer-readable medium could even be a suitable medium upon which the program is stored, scanned, compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory. In the context of the present disclosure, a computer-usable or computer-readable storage medium may be any tangible medium that can contain or store a program for use by or in connection with the instruction execution system, apparatus, or device.
[0050] In some implementations, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. In some implementations, such a propagated signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. In some implementations, the computer readable program code may be transmitted using any appropriate medium, including but not limited to the internet, wireline, optical fibre cable, RF, etc. In some implementations, a computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
[0051] In some implementations, computer program code for carrying out operations of the present disclosure may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like. Java and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle and/or its affiliates. However, the computer program code for carrying out operations of the present disclosure may also be written in conventional procedural programming languages, such as the "C" programming language, PASCAL, or similar programming languages, as well as in scripting languages such as JavaScript, PERL, or Python. In present implementations, the used language for training may be one of Python, Tensorflow, Bazel, C, C++. Further, decoder in user device (as will be discussed) may use C, C++ or any processor specific ISA. Furthermore, assembly code inside C/C++ may be utilized for specific operation. Also, ASR (automatic speech recognition) and G2P decoder along with entire user system can be run in embedded Linux (any distribution), Android, iOS, Windows, or the like, without any limitations. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user’s computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user’s computer through a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the internet using an Internet Service Provider). In some implementations, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGAs) or other hardware accelerators, micro-controller units (MCUs), or programmable logic arrays (PLAs) may execute the computer readable program instructions/code by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present disclosure.
[0052] In some implementations, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus (systems), methods and computer program products according to various implementations of the present disclosure. Each block in the flowchart and/or block diagrams, and combinations of blocks in the flowchart and/or block diagrams, may represent a module, segment, or portion of code, which comprises one or more executable computer program instructions for implementing the specified logical function(s)/act(s). These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program instructions, which may execute via the processor of the computer or other programmable data processing apparatus, create the ability to implement one or more of the functions/acts specified in the flowchart and/or block diagram block or blocks or combinations thereof. It should be noted that, in some implementations, the functions noted in the block(s) may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.
[0053] In some implementations, these computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function/act specified in the flowchart and/or block diagram block or blocks or combinations thereof.
[0054] In some implementations, the computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed (not necessarily in a particular order) on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions/acts (not necessarily in a particular order) specified in the flowchart and/or block diagram block or blocks or combinations thereof.
[0055] The present disclosure provides a computer-implemented method for image-based analysis of grain kernels. The method involves obtaining a sample image of the grain kernels and implementing image pre-processing operations to generate a binary image corresponding to the sample image. A distance function is computed on the binary image, assigning a numeric value to each pixel representing the shortest distance from that pixel to a nearest zero-pixel, resulting in a distance map. From the distance map, a morphological skeleton of the binary image is derived using a maxima of the distance function, where each point in the morphological skeleton represents a peak in the distance map indicating a centre of a maximum inscribed circle within the corresponding grain kernel. A U-Net based convolutional neural network model is then applied to the distance map and the morphological skeleton. The model is configured to generate a segmentation map for demarcating boundaries of each one of the grain kernels with a bounding box in the sample image. The grain kernels are then classified based on the segmentation map utilizing machine-vision techniques to assign each grain to one of the corresponding groups representing various types of defects, including intact, broken, slightly cracked, and strongly cracked grains. In addition to the above, the method also comprises estimating the relative weights of the grain kernels within each group by measuring their density via a weight-per-pixel metric and multiplying it with the segmented areas. The areas are cumulatively added for each grain kernel corresponding to each group to an overall area corresponding to the respective group. A weight ratio for each group is calculated using the overall area and the weight-per-pixel metric.
[0056] Referring now to FIG. 1, illustrated is a flowchart of a method (as represented by reference numeral 100) for image-based analysis of grain kernels. The method 100 includes multiple steps that help to transform a raw image of grain kernels into a segmented and classified image, offering detailed information about the physical characteristics and quality of the grains. The method 100 particularly aims to solve the problem of accurately classifying complex overlapping and touching grain kernels and estimating their relative weights. The method employs advanced image processing techniques and machine learning algorithms to address the challenges associated with traditional grain quality assessment methods.
[0057] Referring to FIG. 2, illustrated is a block diagram of a system (as represented by reference numeral 200) for image-based analysis of grain kernels, in accordance with one or more embodiments of the present disclosure. It may be appreciated that the system 200 described herein may be implemented in various forms of hardware, software, firmware, special purpose processors, or a combination thereof. One or more of the present embodiments may take a form of a computer program product comprising program modules accessible from computer-usable or computer-readable medium storing program code for use by or in connection with one or more computers, processors, or instruction execution systems. For the purpose of this description, a computer-usable or computer-readable medium may be any apparatus that may contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The medium may be electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system (or apparatus or device) or a propagation mediums in and of themselves as signal carriers are not included in the definition of physical computer-readable medium include a semiconductor or solid state memory, magnetic tape, a removable computer diskette, random access memory (RAM), a read only memory (ROM), a rigid magnetic disk and optical disk such as compact disk read-only memory (CD-ROM), compact disk read/write, and digital versatile disc (DVD). Both processors and program code for implementing each aspect of the technology may be centralized or distributed (or a combination thereof) as known to those skilled in the art.
[0058] In an example, the system 200 may be embodied as a computer-program product programmed for performing the said purpose. The system 200 may be incorporated in one or more physical packages (e.g., chips). By way of example, a physical package includes an arrangement of one or more materials, components, and/or wires on a structural assembly (e.g., a baseboard) to provide one or more characteristics such as physical strength, conservation of size, and/or limitation of electrical interaction. It is contemplated that in certain embodiments the computing device may be implemented in a single chip. As illustrated, the system 200 includes a communication mechanism such as a bus 202 for passing information among the components of the system 200. The system 200 includes a processing unit 204 and a memory 206. Herein, the memory 206 is communicatively coupled to the processing unit 204. In an example, the memory 206 may be embodied as a computer readable medium on which program code sections of a computer program are saved, the program code sections being loadable into and/or executable in a system to make the system 200 execute the steps for performing the said purpose.
[0059] Generally, as used herein, the term “processor” refers to a computational element that is operable to respond to and processes instructions that drive the system 200. Optionally, the processor includes, but is not limited to, a microprocessor, a microcontroller, a complex instruction set computing (CISC) microprocessor, a reduced instruction set (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, or any other type of processing circuit. Furthermore, the term “processor” may refer to one or more individual processors, processing devices and various elements associated with a processing device that may be shared by other processing devices. Additionally, the one or more individual processors, processing devices and elements are arranged in various architectures for responding to and processing the instructions that drive the system 200.
[0060] Herein, the memory 206 may be volatile memory and/or non-volatile memory. The memory 206 may be coupled for communication with the processing unit 204. The processing unit 204 may execute instructions and/or code stored in the memory 206. A variety of computer-readable storage media may be stored in and accessed from the memory 206. The memory 206 may include any suitable elements for storing data and machine-readable instructions, such as read only memory, random access memory, erasable programmable read only memory, electrically erasable programmable read only memory, a hard drive, a removable media drive for handling compact disks, digital video disks, diskettes, magnetic tape cartridges, memory cards, and the like.
[0061] In particular, the processing unit 204 has connectivity to the bus 202 to execute instructions and process information stored in the memory 206. The processing unit 204 may include one or more processing cores with each core configured to perform independently. A multi-core processor enables multiprocessing within a single physical package. Examples of a multi-core processor include two, four, eight, or greater numbers of processing cores. Alternatively, or in addition, the processing unit 204 may include one or more microprocessors configured in tandem via the bus 202 to enable independent execution of instructions, pipelining, and multithreading. The processing unit 204 may also be accompanied with one or more specialized components to perform certain processing functions and tasks such as one or more digital signal processors (DSP), and/or one or more application-specific integrated circuits (ASIC). Other specialized components to aid in performing the inventive functions described herein include one or more field programmable gate arrays (FPGA) (not shown), one or more controllers (not shown), or one or more other special-purpose computer chips.
[0062] The system 200 may further include an interface 208, such as a communication interface (with the said terms being interchangeably used) which may enable the system 200 to communicate with other systems for receiving and transmitting information. The communication interface 208 may include a medium (e.g., a communication channel) through which the system 200 communicates with other systems. Examples of the communication interface 208 may include, but are not limited to, a communication channel in a computer cluster, a Local Area Communication channel (LAN), a cellular communication channel, a wireless sensor communication channel (WSN), a cloud communication channel, a Metropolitan Area Communication channel (MAN), and/or the Internet. Optionally, the communication interface 208 may include one or more of a wired connection, a wireless network, cellular networks such as 2G, 3G, 4G, 5G mobile networks, and a Zigbee connection.
[0063] The system 200 also includes a database 210. As used herein, the database 210 is an organized collection of structured data, typically stored in a computer system and designed to be easily accessed, managed, and updated. The database 210 may be in the form of a central repository of information that can be queried, analysed, and processed to support various applications and business processes. In the system 200, the database 210 provides mechanisms for storing, retrieving, updating, and deleting data, and typically includes features such as data validation, security, backup and recovery, and data modelling.
[0064] The system 200 further includes an input device 212 and an output device 214. The input device 212 may take various forms depending on the specific application of the system 200. In an example, the input device 212 may include one or more of a keyboard, a mouse, a touchscreen display, a microphone, a camera, or any other hardware component that enables the user to interact with the system 200. Further, the output device 214 may be in the form of a display, a printer, or the like. It is to be understood that, when reference is made in the present disclosure to the term “display” this refers generically either to a display screen on its own or to the screen and an associated housing, drive circuitry and possibly a physical supporting structure, of which all, or part of is provided for displaying information.
[0065] In the present system 200, the processing unit 204 and accompanying components have connectivity to the memory 206 via the bus 202. The memory 206 includes both dynamic memory (e.g., RAM, magnetic disk, writable optical disk, etc.) and static memory (e.g., ROM, CD-ROM, etc.) for storing executable instructions that when executed perform the method steps described herein for image-based analysis of grain kernels. In particular, the memory 206 includes a module arrangement 216 to perform steps for block scheduling for the EV. Also, in the system 200, the memory 206 may be configured to store the data associated with or generated by the execution of the inventive steps.
[0066] FIG. 3A illustrates a perspective view of the apparatus 300 for image-based analysis of grain kernels. The apparatus 300 includes a housing 302, an image acquisition unit 304, and an output device 306. In the present examples, the output device 306 is implemented in the form of a display (with the two terms being interchangeably used); however, it may be appreciated that the output device 306 may be implemented as a printer or the like without departing from the spirit and the scope of the present disclosure. The image acquisition unit 304 is configured to obtain a sample image of the grain kernels. In the present embodiment, the image acquisition unit 304 is implemented as a flat-bed scanner to obtain high-quality images of the grain kernels with little lighting interference and background noise. However, in other embodiments, the image acquisition unit 304 may be a digital camera, such as an overhead camera, capable of capturing high-resolution images of the grain kernels. The image acquisition unit 204 is designed to be capable of capturing images with a resolution that is high enough to accurately capture the details of the grain kernels. Further, in accordance with some embodiments of the present disclosure, the processing unit 204 is integrated into the housing 302 of the apparatus 200. However, in alternative embodiments, the processing unit 204 may be remote from the housing 302 of the apparatus 200 without departing from the spirit and the scope of the present disclosure. Herein, the processing unit 204 communicates with the image acquisition unit 304 to receive the sample image. The display 306 is also integrated into the housing 302 and configured to output a generated report based, at least in part, on the classification of the grain kernels (as discussed later in the description). In an example, the display 306 may also provide a user interface for allowing a user to interact with the apparatus 300, as may be required for providing inputs, issuing commands, and the like.
[0067] FIG. 3B depicts implementation of the apparatus 300 using the image acquisition unit 304 for the placement of grain kernels therein. For this purpose, the grain kernels may be segregated manually over the scanner bed and then scanning is performed (as shown). In the illustrated examples, the grain kernels that have been employed are rice kernels, but it may be appreciated that the teachings of the present disclosure may be implemented for other varieties of grain kernels such as, but not limited to, wheat, barley, maize, corn, rye, pulses, etc. without any limitations. It may be appreciated by a person skilled in the art that samples weight to scan may differ commodity wise; for example, for rice sample weight, the recommended weight is about 20 grams. Such sample weight threshold has been set to avoid overlapping of grains.
[0068] Referring to FIGS. 1, 2, 3A and 3B in combination, the various steps of the method 100 as described hereinafter may be executed in the system 200 and/or the apparatus 300, or specifically in the processing unit 204 thereof, for image-based analysis of grain kernels. For purposes of the present disclosure, the system 200 is embodied as the apparatus 300 for image-based analysis of grain kernels, with the two terms being generally interchangeably used hereinafter. It may be appreciated that although the method 100 is illustrated and described as a sequence of steps, it may be contemplated that various embodiments of the method 100 may be performed in any order or a combination and need not include all of the illustrated steps.
[0069] It may be appreciated by a person skilled in the art that in order to detect and classify each grain kernel with various types of defect, it is important to separate the touching grain kernels in the acquired image. Separation of touching grain kernels is a recurring problem in image analysis. Morphological methods to separate merged objects in binary images are generally based on the watershed transform applied to the inverse of the distance function. This method is efficient with roughly circular objects, but cannot separate objects beyond a certain elliptic shape nor when the contact zones are too numerous or too large.
[0070] The present disclosure introduces a fragmentation method applied to the skeleton of the image background as an alternative technique to go further in the fused objects separation process. Open lines resulting from skeletonization are prolonged according to their direction from corresponding end points. If the distance between two lines is smaller than a certain value, their respective end points are connected. Results of combined use of watershed and gap-filling based methods are presented on sample binary images. An example of its use on an particularly complex image containing rice grains shows that it allows to segment up to 90% of the grains when classical watershed methods allow only to segment 25% of the grains. Fused grain kernels separation is a classical problem in binary images segmentation. For the general case, solutions are proposed with different approaches contour or active contour based techniques, graph or level set approaches, parametric fitting or morphological methods. Morphological methods used to separate fused objects in binary images are generally based on the watershed transform on the complementary of the distance function.
[0071] However, when fused particles have elliptic shape or when they are fused beyond a certain point, watershed algorithms may be insufficient to produce an appropriate segmentation. For instance, classical watershed based algorithms are effective even for an important degree of overlap (d = 1.2 R). Indeed, we know that as long as the centres of two disks lie on each side of their dissecting line, watershed algorithm will succeed in separating them. On the contrary, the result of watershed based segmentation on elliptical fused objects with a constant distance between their centres and a growing Rmax/Rmin ratio appear to show that above a certain shape factor, watershed algorithm fails to separate two overlapping objects. These functions operate on markers (intrinsic markers such as the maxima of the Euclidean distance function, or markers defined arbitrarily by means of more elaborate techniques) of fused objects and their success depends on the ability to obtain as many markers as the number of fused kernels. It may be noted that at the ultimate points (the maxima of the distance function) of the elliptical fused kernels, it appears that the more the objects have an important Rmax/Rmin ratio, the more their markers position converge until they merge. From that point, it is almost impossible for watershed based algorithm to detect more than a single kernel.
[0072] For purposes of the present disclosure, a light and fast solution has been developed to extract individual properties such as length, width or state (broken, damaged, etc.) from a grain sample. Some studies give an important initial effort to position manually grains to make the stage of image analysis easier. It may be understood that samples sometimes contain hundreds of grains and manual grain placement would lead to a non-negligible waste of time, and thus a need of quick means for image acquisition. In the present disclosure, the process is designed to be as simple and as fast as extracting a grain sample from a batch, place it on the image acquisition unit 304 of the apparatus 300 and launch the acquisition and analysis process.
[0073] In the present embodiments, at step 110, the method 100 begins with obtaining a sample image (as represented by reference numeral 400A in FIG. 4A) of the grain kernels. In general, the sample image 400A may be obtained by any imaging device capable of capturing high-resolution images, such as a digital camera, a smartphone camera, or a specialized imaging device designed for grain analysis. The sample image 400A may be a digital photograph, a scanned image, or any other suitable form of digital image. In the present configuration the sample image 400A is obtained by implementing the image acquisition unit 304 of the apparatus 300, as depicted in FIG. 3B. The image acquisition unit 304 is a standard flatbed scanner. Grains are tabled directly on the glass of the image acquisition unit 304 and are quickly apportioned to avoid excessive overlapping. Then a black box is used to recover the grains and isolate them from ambient light. As may be seen in FIG. 4A, the sample image 400A may include multiple grain kernels that could be of different sizes, shapes, and orientations. Some of the grain kernels may be touching or overlapping each other.
[0074] Once the sample image is obtained, at step 120, the method 100 includes implementing one or more image pre-processing operations on the sample image 400A to generate a binary image (as represented by reference numeral 400B in FIG. 4B) corresponding thereto. This step 120 ensures that the image of the grain kernels is in a format that can be further processed and analysed with a high degree of accuracy and efficiency. The image pre-processing operations may involve as a series of transformations that are applied to the sample image 400A in order to prepare it for the subsequent steps in the image-based analysis. These operations are typically designed to enhance the image data, making the features of interest more noticeable and easier to detect, and/or to reduce the complexity of the image data, making it easier to process. It may be understood that the sample image 400A has two distinct classes of pixels (grains and background). In an example, the Otsu algorithm, which allows to find automatically an optimum threshold, is used for generating the binary image 400B. The Otsu algorithm is configured for separating the two classes so that their combined spread (within-class variance) is maximal. The binary image 400B, as shown in FIG. 4B, is a simplified version of the sample image 400A where each pixel of the image is assigned one of two colour values, typically black or white. Each grain kernel in the binary image 400B is represented by a group of white pixels (1-pixels), and the background of the image is represented by black pixels (0-pixels). The binary image 400B simplifies the subsequent processing steps by reducing the complexity of the image data.
[0075] In an embodiment, the one or more image pre-processing operations comprise at least one of: grayscale conversion, noise reduction, and thresholding. That is, the first operation in the image pre-processing phase may be the grayscale conversion of the sample image 400A. As the name suggests, this operation converts a coloured image into a grayscale image. Each pixel in the grayscale image has a value representing the intensity of the original colour at that location in the image, with higher values indicating brighter colours and lower values indicating darker colours. This grayscale conversion simplifies the image, reducing the amount of information that needs to be processed in the subsequent steps and making it easier to identify the features of interest (in this case, the grain kernels). The next operation may be noise reduction. In the context of image processing, noise refers to random variations in the pixel values. These variations may be introduced during the image acquisition process, for example, due to limitations in the image sensor or environmental interference. Noise reduction algorithms work by smoothing the image, reducing the amount of high-frequency detail and thereby reducing the impact of the noise. This results in a cleaner image that is easier to analyse. The last operation in the image pre-processing phase may be thresholding. The process of thresholding involves choosing a threshold value and then assigning each pixel a 'black' or 'white' value based on whether its intensity is above or below this threshold. In the context of grain analysis, the thresholding operation helps to clearly distinguish the grain kernels (which are represented as 'white' pixels) from the background (which is represented as 'black' pixels).
[0076] Now in order to locate concavities between the grains kernels in the binary image 400B, it is possible to use the cluster convex hull. Another possibility is to analyse the concavity of the contours of the grain clusters, but because of noise in the image grid along contours, it is usual to detect false concave regions, and to obtain an overestimation of the number of concave points. In the present embodiments, a morphological skeleton is used to detect the corners of the complementary image and then extract the concave points from the skeleton open lines. The use of the morphological skeleton to detect object corners in the binary image 400B can be found with different implementations. The main advantages are that it is a robust and non-parametric method. The skeleton is calculated on the background of the binary image 400B. It is obtained by means of a morphological thinning. In particular, the algorithm used here is the single pass thinning algorithm.
[0077] For this purpose, at step 130, the method 100 includes computing a distance function on the binary image 400B. The distance function assigns a numeric value to each pixel representing the shortest distance from that pixel to the nearest zero-pixel. This results in a distance map where the pixel values indicate the distances to the nearest boundary (0-pixel) of the corresponding grain kernel. FIG. 5A depicts a portion (as represented by reference numeral 500A) of the binary image 400B, with the element “portion of the binary image” being interchangeably/simply referred to as binary image for purposes of the present disclosure. Based on the binary image 500A, the processing unit 204 computes the distance function. The computation of the distance function involves examining each pixel in the binary image and determining the shortest distance to the nearest zero-pixel. The distance may be measured in various ways such as Euclidean distance, Manhattan distance, or Chessboard distance, among others. The result of this computation is the distance map where the pixel values represent the distances to the nearest grain boundary. The distance map serves as an intermediary step, aiding in the extraction of important features from the image.
[0078] At step 140, the method 100 includes deriving a morphological skeleton of the binary image 500A from the distance map by using a maxima of the distance function, wherein each point in the morphological skeleton represents a peak in the distance map indicating a centre of a maximum inscribed circle within the corresponding grain kernel. That is, to derive the morphological skeleton, a maxima of the distance function is used, in which maxima represents the points in the distance map where the value of the function reaches a local maximum. In the present embodiments, the maxima represent the points that are furthest away from the boundary of the grain kernels in the binary image 500A. The morphological skeleton is then formed by connecting these maxima, creating a structure that lies within the grain kernels and represents their fundamental shape and structure.
[0079] For example, consider a grain kernel as represented in the binary image 500A. For this grain kernel, the distance map provides a numeric value to each pixel indicating its shortest distance to a nearest zero-pixel (i.e., a pixel representing the background). When the grain kernel is inscribed with the maximum possible circle that fits entirely within its boundary, the centre of this circle corresponds to a pixel with the maximum distance value in the distance map. This pixel is considered a peak, or a maximum of the distance function. By detecting these maxima across the distance map, a collection of pixels may be obtained that represent the centres of the largest possible circles that may be inscribed within each grain kernel. Joining these points creates the morphological skeleton of the grain kernels in the binary image 500A, which provides a simplified representation that still captures the essential structural and orientational features of the grain kernels.
[0080] FIG. 5B depicts the morphological skeleton (as represented by reference numeral 500B) for the binary image 500A. The morphological skeleton 500B reduces the grain kernels to a one-pixel wide representation, thereby enabling efficient and effective extraction of important features while minimizing the amount of redundant data. The morphological skeleton 500B serves several purposes in the subsequent steps of the grain analysis process, as discussed later in the description. Further, FIG. 5C depicts the binary image 500A overlaid on a background image of the morphological skeleton 500B. This overlay provides a comprehensive visual representation, showing the correlation between the original binary image 500A and the derived morphological skeleton 500B. Although the morphological skeleton 500B serves to capture the central structure of individual grain kernels, it might not adequately demarcate the boundaries of touching or overlapping kernels. This is due to the fact that the morphological skeleton 500B represents the peak points of the distance map, corresponding to the centre of the maximum inscribed circle within a grain kernel. While this provides useful structural information about individual kernels, it may not accurately discern the individual kernels in regions where they touch or overlap with each other.
[0081] It may be understood that automatic reconnection of linear segments is also a classical problem in image analysis. When portions of missing lines are sufficiently small, morphological operations such as closing (dilation followed by erosion) or localisation of nearest neighbours are commonly used in order to connect them. In the present disclosure, the distance between the endpoints of the lines to be connected is often larger than the width of the grains, and the size of closing necessary to link them would lead to completely fill skeleton cells. The pairing procedure used here to join endpoints is quite similar to the “linking edge-chains” method, except that herein 2D sphero-cylinders are used in place of isosceles trapezia to extend and join open-lines. Specifically, first, segment endpoints are localized. Then, an “average” direction of the segment is calculated on a given length from its endpoint. Open lines are then linearly extended from their endpoint on a specified distance until they join (or until another extending line reaches their neighbourhood in a given radius).
[0082] Further, endpoints are detected from the skeleton thanks to the morphological hit-or-miss operation. Considering a couple of two disjoint structuring elements T = {T ′ ,T ′′}, the hit-or-miss operation is defined upon image X as :
ηT (X) = {z : T ′′(z) ⊆ X c ;T ′ (z) ⊆ X} = εT ′(X)∩εT ′′(X c ) ,
wherein, εT ′(X) and εT ′′(X c ) are respectively the erosion of X with T ′ and the erosion of the complementary image X c with the structuring element T ′′ . In order to extract the extremity of open lines, the present disclosure uses the hit-or-miss operation with a “U” structuring element and its seven successive rotations.
[0083] In particular, in an embodiment, the method 100 comprises utilizing an open-line detection technique to define open-lines for identifying concavities in the grain kernels in the morphological skeleton 500B. Further, the method 100 comprises pairing the open-lines to generate possible split lines between touching or overlapping grain kernels in the morphological skeleton 500B. For this purpose, starting from the endpoints, successive morphological thinning is applied with the same “U” structuring element (and rotations) to thin the open lines on a certain length, or until idempotence (in this case, the junction of several lines). The thinning operation corresponds to the residue between an image and its hit-or-miss transformation :
ΘT (X) = X \ηT (X) = X \ [εT ′(X)∩εT ′′(X c )]
[0084] Further, FIG. 6A to FIG. 6D illustrate stages involved in the gap-filling process, in accordance with embodiments of the present disclosure. As previously discussed, this is an important process in the present disclosure that helps in identifying the concavities in the grain kernels in the morphological skeleton 500B and generating possible split lines between touching or overlapping grain kernels. FIG. 6A illustrates a stage (as represented by reference numeral 600A) demonstrating the process for end-points localisation. In this process, the end-points of the open-lines in the morphological skeleton 500B are identified. These endpoints often correspond to the points on the boundary of the grain kernels where the curvature changes abruptly, indicating potential regions of concavity or points of contact between different grain kernels. Further, FIG. 6B illustrates a stage (as represented by reference numeral 600B) demonstrating the process for open-lines direction evaluation. The “average” direction of a line is calculated between an end point and the last point obtained by thinning the corresponding line. This line of average direction is then extended from terminal points on a fixed length. Further, FIG. 6C illustrates a stage (as represented by reference numeral 600C) demonstrating the process for open-lines prolongation. Herein, at every stage of reconstruction, the line of a point (pixel) is extended and it is determined if it intersects with another one. In order to detect the intersection between different lines, each line is represented on a black image with a different value. Each time a line ‘i’ is extended from one pixel, it is checked if the corresponding pixel on the black image has the value ‘’0 or the value of another line ‘j’. In the first case, the extending process is continued, and in the second one, a new contact pair is defined between lines ‘i’ and ‘j’. When a contact pair has been established, a straight line is drawn between the corresponding endpoints. This line is then subtracted to the binary image to split the convex grains. Note that line crossings are banned, i.e., when prolongating a line, if the next pixel corresponds to the skeleton or to a previous link, the prolongation process of the line is stopped. Finally, FIG. 6D illustrates a stage (as represented by reference numeral 600D) demonstrating the process for end-points junction. As used herein, "end-point junction" would typically refer to a stage where the endpoints of detected lines (often referred to as "open-lines" in the context of binary images) are connected or joined. In this stage 600D, the extended open-lines are joined at their end-points to form continuous lines, which represent potential splitting lines between touching or overlapping grain kernels.
[0085] It may be noted that during the pairing procedure, some problems can occur. For example, in one instance, two corresponding lines may never connect. This happens if two lines are parallel or if they don’t intersect between the two end-points. To ensure that two corresponding contact zone lines join, at each step of prolongation, a nearby line is detected within a radius of fixed size ‘Rn’. As previously, each time a line ‘I’ is extended from one pixel, it is checked on the black image (on which extending lines are represented) for the presence of a pixel with a different value in a radius of ‘Rn’. If a pixel with a value ‘j’ is found, a new contact pair between lines ‘i’ and ‘j’ is set, as illustrated in a stage 700A of FIG. 7A. In another instance, an average direction calculated from the endpoint to the open line root may not fit the global curvature of the line. This will be prolonged in an unexpected direction. For this reason, it is often better to calculate the average direction on a fixed length Lc rather than on the whole length of the line, as illustrated in a stage 700B of FIG. 7B. This direction information helps in understanding the orientation of the grain kernels and in identifying the precise boundary between touching or overlapping grains. In yet another instance, if a line is prolonged on a too long distance, unexpected contacts can occur between lines corresponding to different contact zones. To avoid this situation, lines are prolonged only on a fixed distance ‘Lp’, generally smaller than the specific length of a grain, as illustrated in a stage 700C of FIG. 7C. This length provides important information regarding the size of the grain kernels and the extent of their overlapping or touching areas.
[0086] Now in order to obtain good results, it is necessary to execute the procedure sequentially with growing values for the parameters Lc, Lp and Rn. First, small values of Lc, Lp and Rn are used in order to fill small gaps. Then the operation is repeated with higher values, to join progressively more distant lines. The maximum values for the parameters are respectively the specific length and width of a grain for Lp and Rn (Lc is limited by the length of the open-line).
[0087] Referring to FIGS. 8A-8C, illustrated are stages in a sequence of pairing processes involving the open-lines. The pairing process is carried out to identify pairs of open-lines that form valid split lines between touching or overlapping grain kernels. FIG. 8A illustrates a first stage (as represented by reference numeral 800A) for the sequence of pairing procedures with low values of parameters Lp, Rn, and Lc. Lp denotes the maximum permissible distance between the end-points of the two lines to be considered a pair, Rn represents the maximum allowed difference in direction between the two lines, and Lc is the minimum length required for a line to be considered in the pairing process. Lines satisfying these criteria are paired together. As shown, only two lines (on the bottom left) are linked. FIG. 8B illustrates a second stage (as represented by reference numeral 800B) for the sequence of pairing procedures with medium values of parameters Lp, Rn, and Lc. As the values of the parameters are increased, fewer lines satisfy the pairing criteria. The process is repeated with these new parameters, pairing the remaining lines that meet the criteria. As shown, the values chosen for the parameters allow to build the two upper links. FIG. 8C illustrates a third (last) stage (as represented by reference numeral 800C) for the sequence of pairing procedures with high values of parameters Lp, Rn, and Lc. At this point, only the longest and most similarly directed lines are paired together, refining the selection of potential split lines. At each stage 800A-800C, the particles revealed by the new links are measured. If their width and length are lower or equal to those of a typical grain, they are considered as correctly segmented and are removed from the image. The procedure is then continued with higher parameters, until the image is empty (or until the maximum values of parameters are reached).
[0088] The separation of touching grains presents unique challenges, particularly when the grains are elliptically shaped or when clusters contain numerous grains. Traditional watershed-based algorithms tend to fall short in these scenarios. To address this, the present disclosure introduces a gap-filling method that uses the skeleton of the background image along with a U-Net model to draw clipping lines between touching grains. A practical implementation of this algorithm was then demonstrated to facilitate the separation process. Consequently, the proposed "gap-filling" based algorithm can significantly enhance the segmentation process for images composed of multiple touching grain kernels with elliptic shapes. This advanced algorithm has a myriad of applications, especially within the food industry, which can greatly benefit from its precise grain segmentation capabilities.
[0089] FIG. 9A and FIG. 9B depict the results after implementing the gap-filling process and the grain splitting operation on the morphological skeleton 500B (as represented by reference numeral 900A) and the binary image 400B (as represented by reference numeral 900B) respectively. The results largely align with the anticipated outcomes, barring the case of the two grain kernels situated at the image's top, which remain inseparable. In this specific scenario, the morphological skeleton 500B does not identify two separate concavities but perceives them as one (as illustrated in FIG. 9A), resulting in the detection of only one open-line. Without a pair, the single open-line remains unlinked and thus no split line can materialize to separate the two touching grain kernels. However, utilising the paired open-lines as split lines, the touching or overlapping grain kernels in the binary image 400B are effectively separated. Each grain kernel is distinctly delineated, making it an independent entity in the image that can be independently analysed in the subsequent steps.
[0090] Now for big clusters of grain kernels which may not have been fragmented using explained methodology above, the present disclosure employs a U-Net based convolutional neural network. The encoder-decoder structure utilized in the U-Net based convolutional neural network is very effective for image segmentation. The U-Net based convolutional neural network has also been found the best among the three considered models (pyramid scene parsing network (PSPNet), DeepLabv3, and U-Net). In the present disclosure, the U-Net based convolutional neural network has been used to effectively solve the semantic segmentation problem in grain scanned images on clustered kernel parts.
[0091] For this purpose, at step 150, the method 100 includes applying a U-Net based convolutional neural network model (hereinafter, referred to as “U-Net model”) on the distance map and the morphological skeleton, the U-Net model configured to generate a segmentation map (referred by reference numeral 1000 in FIG. 10) for demarcating boundaries each one of the grain kernels with a bounding box in the sample image. The U-Net model consists of two parts - the contracting path (also known as the encoder) and the expansive path (also known as the decoder). The encoder captures the context in the image, i.e., it reduces the spatial dimension of the input image and extracts high-level semantic information. This is typically achieved by applying multiple convolutional layers followed by a max-pooling layer. The output of each of these steps is a downscaled version of the input, where the semantic information is gradually enhanced while the spatial information is reduced. The decoder, on the other hand, allows localization, i.e., it increases the spatial dimension of the input from the encoder to generate a segmentation map 1000 that is of the same size as the original input image. This is typically achieved by a series of up-convolutions (also known as deconvolutions or transposed convolutions) and concatenation with the feature maps from the encoder.
[0092] In the present method 100, the U-Net model is applied on the distance map and the morphological skeleton 500B derived from the sample image 400A (or specifically, the binary image 400B) of the grain kernels. The distance map and the morphological skeleton 500B provide valuable contextual and spatial information to the network, enabling it to understand the spatial relations and boundaries of the grain kernels in the sample image 400A. In particular, herein, the U-Net model comprises an encoding module configured to perform a series of convolution operations to capture contextual features from the distance map and the morphological skeleton, and a decoding module, symmetric to the encoding module, configured to perform a series of convolution and transpose convolution operations, for generating the segmentation map 1000. The U-Net model is trained to learn the complex mapping from the input (the distance map and the morphological skeleton 500B) to the output, and is capable of generating the segmentation map 1000 which demarcates the boundaries of each grain kernel with a bounding box. This process allows for the identification and separation of individual grain kernels, even if they are touching or overlapping in the original sample image 400A. This is particularly useful for further analysis such as defect detection or classification, which can be performed on individual grain kernels. In present embodiments, to combine features between different levels, the U-Net model adopts a layer-hopping connection mechanism, which combines the features not compressed by the encoder module with features corresponding to the same layer in the decoder module to better fuse different semantic information. The training strategy of the U-Net model is to use image enhancement technology to effectively learn from a small number of annotated sample images.
[0093] Referring to FIG. 10, illustrated is the segmentation map 1000 which is a result of segmentation obtained as per embodiments of the present disclosure. The segmentation map 1000, presented with the contours superimposed on the original grey-level image, showcases the final outcome of the segmentation process. Mostly, the contours successfully encapsulate individual grains, demonstrating a high degree of precision in the segmentation. However, a handful of grains may not be accurately segmented. This typically happens when the skeleton does not sufficiently reveal grain concavities. The issue arises when there are either no discernible open-lines or when these open-lines are too short to distinctly demarcate the grain's boundary. This is still in contrast to (and better than) the segmentation obtained using the classical watershed transformation on the distance function on a grain sample image with a particularly difficult configuration: lots of conglomerates, important contact zones (only 15% of the grains are single). In order to control the level of segmentation obtained by the watershed, the markers used to perform the flooding are determined using the h-minima of the distance function. As may be seen from the binary image 400B of FIG. 4B, even with a relatively high value of h (h = 4), there are still unwanted grain clippings. In order to completely avoid any inappropriate cuttings, it may be required to process the watershed with h ≥ 5. This handles only very simple contact cases (clusters containing two or three grains with small contact zones), and the amount of merged grains remains very large.
[0094] Further, at step 160, the method 100 includes classifying the grain kernels based on the segmentation map to assign each one of the grain kernels to one of corresponding groups representing one or more types of defects, including intact, broken, slightly cracked, and strongly cracked grains. The segmentation map 1000, prepared by previous steps, carries essential information about the boundaries, shape, and size of each grain. By evaluating these properties, along with other potential features extracted from the image, the grain kernels can be categorized into different groups. The groups in this context represent the different types of physical conditions or defects that the grains may exhibit. Four primary categories or groups are considered for the classification process in this method: (i) Intact grains, which includes the grain kernels that are complete and show no signs of physical damage; (ii) Broken grains, that are visibly fragmented or fractured are classified under this group; (iii) Slightly cracked grains, which pertains to grains that have minor, less obvious cracks; and (iv) Strongly cracked grains, which are the grains with significant cracks or fissures. The classification of the grain kernels into these categories helps in evaluating the overall quality of the grain sample. The categorization provides insights into the proportion of grains in each category, enabling appropriate decisions regarding the usage, storage, or sale of these grains.
[0095] In the illustration of FIG. 10, the grain kernels that have been successfully segmented are highlighted by contours of different colours, with each colour denoting the grain's condition. In this case, blue signifies intact grains, green corresponds to broken grains, while yellow and red represent slightly and strongly cracked grains respectively. By assigning colour codes to each grain based on their condition, it enables us to visualize the state of each grain kernel after the image processing and classification steps.
[0096] In an embodiment, the method 100 further comprises utilizing one or more machine-vision techniques on the segmentation map to classify the grain kernels. That is, the method 100 relies on the application of various image analysis and pattern recognition techniques to identify and categorize the grain kernels into distinct groups based on their physical condition. Machine vision is a branch of computer science that emulates human vision using digital image processing. Machine vision involves techniques to acquire, process, analyse, and understand images to make decisions or predictions. In the present disclosure, the machine vision techniques are applied on the segmentation map 1000 generated from previous steps. The segmentation map 1000 provides an outline or contour for each grain kernel, which makes it easier to analyse each kernel individually. In present examples, the machine-vision techniques for grain classification may involve feature extraction like shape, size, texture, and other morphological characteristics from each grain kernel in the segmentation map 1000; pattern recognition to classify the grain kernels, such as a rule-based system where certain rules are defined based on the features; and the like, without any limitations.
[0097] In certain embodiments, the method 100 further includes refining the segmentation map 1000 by utilizing the morphological skeleton 500B to draw clipping lines between touching grain kernels in the binary image 400A, and classifying the grain kernels based on the refined segmentation map, thereby distinctly demarcating boundaries of the grain kernels in the sample image 400A. This is particularly useful in instances where the grain kernels in the binary image are touching or overlapping. The refinement process leverages the morphological skeleton 500B to draw clipping lines between touching grain kernels. Clipping lines are essentially lines that cut through the regions where the grains touch or overlap, thereby distinctly demarcating boundaries of each grain kernel. This application of the morphological skeleton 500B helps in resolving ambiguities caused due to close proximity or overlap of grain kernels in the binary image 400A. The drawn clipping lines function as boundary markers, providing clearer segmentation of each individual grain kernel from the group. Consequently, this increases the precision of the segmentation map 1000, making the subsequent steps like feature extraction or classification more accurate and reliable.
[0098] The described algorithm has been rigorously tested across hundreds of images, handling a wide variety of cases. These range from images containing a small handful of grains, as few as ten, to significantly more complex scenarios involving up to one thousand grains. The results consistently demonstrate a high rate of success, regardless of the complexity of the grain arrangement. The time required for processing each image varies depending on the number of grains and the complexity of their arrangement, averaging at several seconds. For example, the segmentation map 1000 (as shown in FIG. 10) of an 850 x 800 pixels image takes about 8 seconds on a standard laptop computer.
[0099] In further embodiments of the present disclosure, the method 100 includes additional steps focused on the estimation of relative weights of each grain kernel. In these steps, a weight-per-pixel metric is applied to the area of each individual grain kernel within the groups, as defined by the segmentation map. This weight-per-pixel metric may be based on several factors such as density, moisture content, or other relevant factors that contribute to the weight of the individual grain kernel. These estimated weights are then cumulatively summed within each group to calculate an overall estimated weight for each corresponding group of grain kernels. This overall estimated weight can provide valuable information on the total mass or potential yield of each group of grain kernels. Furthermore, a weight ratio for each group is calculated using the overall area occupied by the group in the binary image and the applied weight-per-pixel metric. The weight ratio provides a standardized measurement that can be used for comparisons across different groups of grain kernels and different sample images, making it a valuable metric for understanding the distribution of grain types within a given sample.
[00100] In particular, herein, the area of each grain kernel is calculated based on the number of pixels within its bounding box in the segmentation map 1000. The weight-per-pixel metric is a predefined value that represents the average weight of a grain kernel per pixel. The area of each grain kernel can be determined by counting the number of pixels that belong to each grain kernel in the segmentation map. The weight-per-pixel metric is a value that is obtained empirically by comparing the pixel area of a known number of grain kernels to their total weight. This metric allows the method to estimate the weight of each grain kernel based on its pixel area in the image. Following the weight estimation of each grain kernel, the method 100 then proceeds to the step of cumulatively adding the area of each grain kernel corresponding to each group to an overall area corresponding to the respective group. This step involves calculating the cumulative area for each group of grain kernels by summing the areas of all grain kernels within that group. It may be understood that the overall area of each group provides an indication of the total amount of grain kernels that belong to that group. By considering the area of each grain kernel, rather than simply the number of grain kernels, the method 100 provides a more accurate representation of the actual volume or weight of grain kernels in each category. This is important because grain kernels can significantly vary in size and weight, and therefore the number of grain kernels alone may not accurately represent the total amount of grain. Finally, the method 100 includes the step of calculating the weight ratio for each group using the overall area and the weight-per-pixel metric. The weight ratio for each group is calculated by multiplying the overall area of the group by the weight-per-pixel metric. The weight ratio provides an estimate of the relative weight of grain kernels in each group, which can be used to quantify the distribution of grain quality in the sample. The weight ratio is a key output of the method 100 because it provides a quantitative measure of the distribution of grain quality in the sample. By calculating the weight ratio for each group of grain kernels, the method 100 can provide detailed information about the quality of the grain sample. This information can be used for various purposes such as quality control, grading, pricing, and marketing of the grain.
[00101] In conclusion, the present disclosure provides the computer-implemented method 100, the system 200 and the apparatus 300 for image-based analysis of grain kernels that is capable of accurately and efficiently assessing the quality of grain kernels. The method 100, the system 200 and the apparatus 300 of the present disclosure are implemented via a combination of software and hardware components, all of which work together to perform the analysis and categorization of grain kernels. The software component is executed on a computing device that is equipped with the necessary processing power, memory, and storage capacity to run the algorithms and processes involved in the analysis. This software can be deployed on a variety of devices, including but not limited to desktop computers, laptop computers, servers, or embedded systems in the apparatus 300. The present disclosure uses advanced image processing and machine learning techniques to analyse digital images of grain kernels, providing detailed information about the size, shape, and quality of each grain kernel. The present disclosure also provides estimates of the relative weights of different quality categories of grain kernels, providing valuable information for the management and marketing of grain. The present disclosure is automated and non-destructive, making it a valuable tool for the grain industry.
[00102] The present disclosure provides a powerful tool for the analysis of grain kernels, offering a level of accuracy and reliability that surpasses conventional manual methods. By leveraging advanced image processing and machine learning techniques, the present disclosure delivers fast, objective, and non-destructive analysis of grain kernels, thereby making it a valuable asset for the grain industry. The ability of the present disclosure to accurately categorize grain kernels based on their morphological characteristics, and to estimate their relative weights, allows for a more comprehensive evaluation of grain quality. This, in turn, can lead to more consistent quality control, and ultimately, to the enhancement of the value of the grain. In addition, the system 200 and the apparatus 300 of the present disclosure may be provided with cloud-enabled technology and with blockchain technology to enable secure, end-to-end traceability and transaction, as well as convenient access to the analysis data from anywhere. These features make the present disclosure not only a powerful tool for quality assessment but also a versatile platform for data analytics and forecasting.
[00103] Specifically, the apparatus 300 of the present disclosure brings a multitude of advantages to the grain industry. The apparatus 300 aids in better grain marketing by enabling farmers and traders to make informed decisions about the quality and value of their produce. Early detection of quality issues helps avoid unnecessary costs due to spoilage or rejections. By promoting consistency in grain assessment, the apparatus 300 enhances industry standards. The apparatus 300 is designed to be user-friendly, reducing the need for specialized training. The apparatus 300 serves as a crucial tool for food quality assessment, detecting contaminants and mycotoxins in grains. For stored grains, the apparatus 300 monitors quality and optimizes storage conditions. The features of the apparatus 300 extend to sorting grains based on size, shape, and color, and optimizing milling processes. In controlling grain industry processes, the apparatus 300 helps monitor moisture content and control drying processes. The apparatus 300 also aids in early detection of crop diseases, reducing crop losses. The apparatus 300 also enables real-time quality data tracking for supply chain optimization and real-time grain quality monitoring during transportation. The apparatus 300 further automates grading of grains, improving efficiency, and ensures processed grain products meet industry standards, thus aiding in quality control. The apparatus 300 also promotes sustainability by identifying areas of environmental impact reduction and improves traceability. Also, the apparatus 300 helps ensure export compliance in the grain industry. The apparatus 300 can be customized for specific applications and collect large volumes of data for analytics, providing valuable insights. Furthermore, the apparatus 300 can be employed in research and development of new grain varieties and ensures consumer safety by detecting allergens or pathogens.
[00104] The foregoing description of the specific embodiments will so fully reveal the general nature of the embodiments herein that others can, by applying current knowledge, readily modify and/or adapt for various applications such specific embodiments without departing from the generic concept, and, therefore, such adaptations and modifications should and are intended to be comprehended within the meaning and range of equivalents of the disclosed embodiments. It is to be understood that the phraseology or terminology employed herein is for the purpose of description and not of limitation. Therefore, while the embodiments herein have been described in terms of preferred embodiments, those skilled in the art will recognize that the embodiments herein can be practiced with modification within the spirit and scope of the present disclosure. , Claims:WE CLAIM:
1. A computer-implemented method for image-based analysis of grain kernels, the method comprising:
obtaining a sample image of the grain kernels;
implementing one or more image pre-processing operations on the sample image to generate a binary image corresponding thereto;
computing a distance function on the binary image, wherein the distance function assigns a numeric value to each pixel representing a shortest distance from that pixel to a nearest zero-pixel, resulting in a distance map;
deriving a morphological skeleton of the binary image from the distance map by using a maxima of the distance function, wherein each point in the morphological skeleton represents a peak in the distance map indicating a centre of a maximum inscribed circle within the corresponding grain kernel;
applying a U-Net based convolutional neural network model on the distance map and the morphological skeleton, the U-Net based convolutional neural network model configured to generate a segmentation map for demarcating boundaries each one of the grain kernels with a bounding box in the sample image; and
classifying the grain kernels based on the segmentation map to assign each one of the grain kernels to one of corresponding groups representing one or more types of defects, including intact, broken, slightly cracked, and strongly cracked grains.

2. The method as claimed in claim 1 further comprising:
estimating relative weights of each grain kernel within the groups by applying a weight-per-pixel metric to an area of each grain kernel;
cumulatively summing the estimated relative weights of each grain kernel within the same group to calculate an overall estimated weight for the corresponding group; and
calculating a weight ratio for each group using an overall area and the weight-per-pixel metric.

3. The method as claimed in claim 1, wherein the U-Net based convolutional neural network model comprises an encoding module configured to perform a series of convolution operations to capture contextual features from the distance map and the morphological skeleton, and a decoding module, symmetric to the encoding module, configured to perform a series of convolution and transpose convolution operations, for generating the segmentation map.

4. The method as claimed in claim 1 further comprising refining the segmentation map by utilizing the morphological skeleton to draw clipping lines between touching grain kernels in the binary image, and classifying the grain kernels based on the refined segmentation map, thereby distinctly demarcating boundaries of the grain kernels in the sample image.

5. The method as claimed in claim 1 further comprising:
utilizing an open-line detection technique to define open-lines for identifying concavities in the grain kernels in the morphological skeleton; and
pairing the open-lines to generate possible split lines between touching or overlapping grain kernels in the morphological skeleton.

6. The method as claimed in claim 1 further comprising utilizing one or more machine-vision techniques on the segmentation map to classify the grain kernels.

7. The method as claimed in claim 1, wherein the one or more image pre-processing operations comprise at least one of: grayscale conversion, noise reduction, and thresholding.

8. A system for image-based analysis of grain kernels, the system comprising:
an image acquisition unit configured to obtain a sample image of the grain kernels; and
a processing unit disposed in communication with the image acquisition unit to receive the sample image, the processing unit configured to:
implement one or more image pre-processing operations on the sample image to generate a binary image corresponding thereto;
compute a distance function on the binary image, wherein the distance function assigns a numeric value to each pixel representing a shortest distance from that pixel to a nearest zero-pixel, resulting in a distance map;
derive a morphological skeleton of the binary image from the distance map by using a maxima of the distance function, wherein each point in the morphological skeleton represents a peak in the distance map indicating a centre of a maximum inscribed circle within the corresponding grain kernel;
apply a U-Net based convolutional neural network model on the distance map and the morphological skeleton, the U-Net based convolutional neural network model configured to generate a segmentation map for demarcating boundaries each one of the grain kernels with a bounding box in the sample image; and
classify the grain kernels based on the segmentation map by implementing a fusion algorithm utilizing one or more machine-vision techniques to assign each one of the grain kernels to one of corresponding groups representing one or more types of defects, including intact, broken, slightly cracked, and strongly cracked grains.

9. An apparatus for image-based analysis of grain kernels, the apparatus comprising:
a housing;
an image acquisition unit integrated in the housing, the image acquisition unit configured to obtain a sample image of the grain kernels placed thereon;
a processing unit integrated in the housing and disposed in communication with the image acquisition unit to receive the sample image, the processing unit configured to:
implement one or more image pre-processing operations on the sample image to generate a binary image corresponding thereto;
compute a distance function on the binary image, wherein the distance function assigns a numeric value to each pixel representing a shortest distance from that pixel to a nearest zero-pixel, resulting in a distance map;
derive a morphological skeleton of the binary image from the distance map by using a maxima of the distance function, wherein each point in the morphological skeleton represents a peak in the distance map indicating a centre of a maximum inscribed circle within the corresponding grain kernel;
apply a U-Net based convolutional neural network model on the distance map and the morphological skeleton, the U-Net based convolutional neural network model configured to generate a segmentation map for demarcating boundaries each one of the grain kernels with a bounding box in the sample image; and
classify the grain kernels based on the segmentation map by implementing a fusion algorithm utilizing one or more machine-vision techniques to assign each one of the grain kernels to one of corresponding groups representing one or more types of defects, including intact, broken, slightly cracked, and strongly cracked grains; and
an output device integrated in the housing, the output device configured to generate a report based, at least in part, on the classification of the grain kernels.

10. A computer program product, comprising computer program code which, when executed by a processing unit, causes the processing module to carry out the method of any of the claims 1 to 7.