Claims:WE CLAIM:
1. A method for automated classification of coal maceral, the method comprising:
obtaining, by a classification system, a plurality of image patches from a petrographic image of a coal sample;
analysing, by the classification system, each of the plurality of image patches using one of a plurality of U-net Convolutional Neural Network (CNN) classifiers, wherein each of the plurality of U-net CNN classifiers are trained for identifying one of a plurality of maceral classes; and
classifying, by the classification system, the coal maceral into the plurality of maceral classes by amalgamating analysis results from each of the plurality of U-net CNN classifiers.

2. The method as claimed in claim 1, wherein the plurality of image patches are obtained by segmenting the petrographic image into a plurality of image patches of predetermined pixel dimension.

3. The method as claimed in claim 2, wherein the predetermined pixel dimension is a value relative to the pixel dimension of the petrographic image.

4. The method as claimed in claim 1, wherein plurality of maceral classes comprises at least one of vitrinite, inertinite, liptinite and mineral.

5. The method as claimed in claim 1, wherein each of the plurality of U-net CNN classifiers are trained using predetermined petrographic images comprising ground truth information related to each of the plurality of maceral classes.

6. The method as claimed in claim 5, wherein training the plurality of U-net CNN classifiers comprises:
training the plurality of U-net CNN classifiers for a majority maceral class, comprising at least one of vitrinite and inertinite, by minimizing a binary cross-entropy loss; and
training the plurality of U-net CNN classifiers for a minority maceral class, comprising at least one of liptinite and mineral, by performing area based regularization and intensity based regularization.

7. The method as claimed in claim 1, wherein the plurality of U-net CNN classifiers identify the plurality of maceral classes by:
sequentially analysing each of a plurality of pixels in the plurality of image patches; and
identifying the plurality of maceral classes based on the sequential analysis.

8. The method as claimed in claim 1, wherein the analysis results comprise image patches having distinctly labelled regions of the plurality of maceral classes.

9. The method as claimed in claim 1, wherein amalgamating the analysis results of each of the plurality of U-net CNN classifiers comprises eliminating insignificant, smaller and irregular regions from the analysis results.

10. A classification system for automated classification of coal maceral, the analysis system comprising:
a processor; and
a memory, communicatively coupled to the processor, wherein the memory stores processor-executable instructions, which on execution, cause the processor to:
obtain a plurality of image patches from a petrographic image of a coal sample;
analyse each of the plurality of image patches using one of a plurality of U-net Convolutional Neural Network (CNN) classifiers, wherein each of the plurality of U-net CNN classifiers are trained for identifying one of a plurality of maceral classes; and
classify the coal maceral into the plurality of maceral classes by amalgamating analysis results from each of the plurality of U-net CNN classifiers.

11. The classification system as claimed in claim 10, wherein the processor obtains the plurality of image patches by segmenting the petrographic image into a plurality of image patches of predetermined pixel dimension.

12. The classification system as claimed in claim 11, wherein the predetermined pixel dimension is a value relative to the pixel dimension of the petrographic image.

13. The classification system as claimed in claim 10, wherein plurality of maceral classes comprises at least one of vitrinite, inertinite, liptinite and mineral.

14. The classification system as claimed in claim 10, wherein each of the plurality of U-net CNN classifiers are trained using predetermined petrographic images comprising ground truth information related to each of the plurality of maceral classes.

15. The classification system as claimed in claim 14, wherein training the plurality of U-net CNN classifiers comprises:
training the plurality of U-net CNN classifiers for a majority maceral class, comprising at least one of vitrinite and inertinite, by minimizing a binary cross-entropy loss; and
training the plurality of U-net CNN classifiers for a minority maceral class, comprising at least one of liptinite and mineral, by performing area based regularization and intensity based regularization.

16. The classification system as claimed in claim 10, wherein the plurality of U-net CNN classifiers identify the plurality of maceral classes by:
sequentially analysing each of a plurality of pixels in the plurality of image patches; and
identifying the plurality of maceral classes based on the sequential analysis.

17. The classification system as claimed in claim 10, wherein the analysis results comprise image patches having distinctly labelled regions of the plurality of maceral classes.
18. The classification system as claimed in claim 10, wherein the processor amalgamates the analysis results of each of the plurality of U-net CNN classifiers by eliminating insignificant, smaller and irregular regions from the analysis results.

Dated this 31st day of March 2021

NIKHIL S R
OF K & S PARTNERS
AGENT FOR THE APPLICANT(S)
IN/PA-2127

, Description:FORM 2
THE PATENTS ACT 1970
[39 OF 1970]
&
THE PATENTS RULES, 2003

COMPLETE SPECIFICATION
[See section 10; Rule 13]

TITLE: “METHOD AND SYSTEM FOR AUTOMATED CLASSIFICATION OF COAL MACERAL”

Name and Address of the Applicants:
(1) TATA STEEL LIMITED, Jamshedpur, Jharkhand, India 831001
(2) Indian Statistical Institute, 203, B T Road, Kolkata-700108

Nationality: INDIAN

The following specification particularly describes the invention and the manner in which it is to be performed.

TECHNICAL FIELD

The present subject matter is, in general, related to characterization of coal and more particularly, but not exclusively, to method and system for automated classification of coal maceral.

BACKGROUND

Generally, the quality of coal is determined by analyzing various characteristics such as composition of the coal, type of the coal and degree of coalification. A clear understanding of the characteristics of the coal blend is important to decide how coking operations will impact a final product. One of the techniques used for studying the coal characteristics is ‘coal petrography’. Coal petrography is a microscopic technique that uses optical microscopic images of the coal sample to determine coal characteristics. Particularly, coal petrography involves use of reflected microscopy to determine a maceral composition and a rank of the coal.
Presently, manual coal petrographic process takes about 5 hours to give a complete analysis of a single coal pellet sample. More importantly, the outcome of manual coal petrography is subjective and depends on the expertise and efficiency of the human expert performing the petrography. On the other hand, there is a continued industry-wide need for an accurate and a reliable petrographic technique, which can be used for studying and selecting coal blends without compromising on coke quality. Also, to optimize a parent coal blend formulation, the objective analysis of the coal samples must be performed quickly. Therefore, there is a need for a faster and an accurate petrographic technique.

One of the existing techniques adapts a combination of image processing and machine learning techniques to obtain quick and accurate petrographic results. These existing methods use machine learning approaches such as minimum distance classifier and random forest and have shown promising results in automating the estimation of phase fraction. However, these methods suffer from class imbalance problem when the classes are not equally represented during training. Therefore, there is need for a more consistent and accurate approach for classification of the coal sample.

The information disclosed in this background of the disclosure section is only for enhancement of understanding of the general background of the invention and should not be taken as an acknowledgement or any form of suggestion that this information forms the prior art already known to a person skilled in the art.

SUMMARY

Disclosed herein is a method for automated classification of coal maceral. The method comprises obtaining, by a classification system, a plurality of image patches from a petrographic image of a coal sample. Further, the method comprises analyzing each of the plurality of image patches using one of a plurality of U-net Convolutional Neural Network (CNN) classifiers. Each of the plurality of U-net CNN classifiers are trained for identifying one of a plurality of maceral classes. Thereafter, the method comprises classifying the coal maceral into the plurality of maceral classes by amalgamating analysis results from each of the plurality of U-net CNN classifiers.

Further, the present disclosure relates to a classification system for automated classification of coal maceral. The classification system comprises a processor and a memory. The memory is communicatively coupled to the processor and stores processor-executable instructions, which on execution, cause the processor to obtain a plurality of image patches from a petrographic image of a coal sample. Further, the instructions cause the processor to analyze each of the plurality of image patches using one of a plurality of U-net Convolutional Neural Network (CNN) classifiers. Each of the plurality of U-net CNN classifiers are trained for identifying one of a plurality of maceral classes. Thereafter, the instructions cause the processor to classify the coal maceral into the plurality of maceral classes by amalgamating analysis results from each of the plurality of U-net CNN classifiers.

In an embodiment of the present disclosure, the plurality of image patches is obtained by segmenting the petrographic image into a plurality of image patches of predetermined pixel dimension. Further, the predetermined pixel dimension is a value relative to the pixel dimension of the petrographic image.

In a further embodiment of the present disclosure, each the plurality of maceral classes comprises at least one of vitrinite, inertinite, liptinite and mineral.

In a further embodiment of the present disclosure, the plurality of U-net CNN classifiers is trained using predetermined petrographic images comprising ground truth information related to each of the plurality of maceral classes.
In a further embodiment of the present disclosure, the analysis results comprise image patches having distinctly labelled regions of the plurality of maceral classes. Further, amalgamating the analysis results of each of the plurality of U-net CNN classifiers comprises eliminating insignificant, smaller, and irregular regions from the analysis results.

The foregoing summary is illustrative only and is not intended to be in any way limiting. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features will become apparent by reference to the drawings and the following detailed description.

BRIEF DESCRIPTION OF THE ACCOMPANYING DRAWINGS

The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate exemplary embodiments and, together with the description, explain the disclosed principles. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. The same numbers are used throughout the figures to reference like features and components. Some embodiments of system and/or methods in accordance with embodiments of the present subject matter are now described, by way of example only, and regarding the accompanying figures, in which:

FIG. 1 provides an overview of a method and a system for automated classification of coal maceral in accordance with some embodiments of the present disclosure.

FIG. 2A shows a detailed block diagram of a classification system in accordance with some embodiments of the present disclosure.

FIGS. 2B and 2C show exemplary representations of the U-Net classifier in accordance with some embodiments of the present disclosure.

FIGS. 3A and 3B show a comparison between the classification results obtained from the existing techniques and the proposed method in accordance with some exemplary embodiments.

FIGS. 4A-4C show various comparisons between results obtained by the existing techniques and the proposed method in accordance with some exemplary embodiments.

FIG. 5 shows a flowchart illustrating a method for automated classification of coal maceral in accordance with some embodiments of the present disclosure.

FIG. 6 illustrates a block diagram of an exemplary computer system for implementing embodiments consistent with the present disclosure.

It should be appreciated by those skilled in the art that any block diagrams herein represent conceptual views of illustrative systems embodying the principles of the present subject matter. Similarly, it will be appreciated that any flow charts, flow diagrams, state transition diagrams, pseudo code, and the like represent various processes which may be substantially represented in computer readable medium and executed by a computer or processor, whether such computer or processor is explicitly shown.

DETAILED DESCRIPTION

In the present document, the word “exemplary” is used herein to mean “serving as an example, instance, or illustration”. Any embodiment or implementation of the present subject matter described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.
While the disclosure is susceptible to various modifications and alternative forms, specific embodiment thereof has been shown by way of example in the drawings and will be described in detail below. It should be understood, however that it is not intended to limit the disclosure to the specific forms disclosed, but on the contrary, the disclosure is to cover all modifications, equivalents, and alternative falling within the scope of the disclosure.
The terms “comprises”, “comprising”, “includes”, or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a setup, device, or method that comprises a list of components or steps does not include only those components or steps but may include other components or steps not expressly listed or inherent to such setup or device or method. In other words, one or more elements in a system or apparatus proceeded by “comprises… a” does not, without more constraints, preclude the existence of other elements or additional elements in the system or method.
The present disclosure relates to method and classification system for automated classification of coal maceral. In an embodiment, the present disclosure adapts an ‘one-vs-all approach’ to multiclass classification using deep learning techniques. The ‘one-vs-all’ approach uses multiple binary classifiers instead of a single multiclass classifier. Each binary classifier performs the task of identification of whether a pixel belongs to a class, that is vitrinite, inertinite, liptinite or minerals class, or not. Also, the present disclosure uses a U-Net Convolutional Neural Network (CNN) classifier as a pixel-level classifier. Specific regularization terms have been used to train each individual binary U-Net classifiers and the outcome of each U-Net classifier is amalgamated to generate a final segmentation result. The regularization terms help in solving the class imbalance problem. Therefore, the method of present disclosure results in a more intricate segmentation and superior minority class detection compared to existing machine learning based classification approaches.

Machine learning approaches used for coal quality estimation generally use minimum distance classifier and random forest techniques. The approaches have shown promising results in automating the estimation of phase fraction, but suffer from class imbalance problem when the classes are not equally represented during the training. Therefore, the present disclosure proposes a deep learning based solution, wherein the U-net classifier is used as a pixel-level classifier with tailor-made loss functions to obtain superior classification. The proposed method is both reliable and faster compared to the existing approaches that use random forest technique.

In the following detailed description of the embodiments of the disclosure, reference is made to the accompanying drawings that form a part hereof, and in which are shown by way of illustration specific embodiments in which the disclosure may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the disclosure, and it is to be understood that other embodiments may be utilized and that changes may be made without departing from the scope of the present disclosure. The following description is, therefore, not to be taken in a limiting sense.

FIG. 1 provides an overview of method and system for automated classification of coal maceral in accordance with some embodiments of the present disclosure.

As an example, environment 100 may represent a petrographic laboratory or a coal testing laboratory, which may be used for carrying out coal petrography. Coal petrography is a microscopic analysis technique used to determine a coal’s rank (i.e., degree of coalification) and a type of coal (i.e., amount and type of macerals) on various specimens of coal. In an embodiment, the environment 100 may include, without limiting to, a classification system 101, an image repository 103 and a classification model 107.

In an implementation, the image repository 103 may be a storage unit and/or a database that stores a plurality of petrographic images 105 of a coal sample. As an example, the petrographic image 105 may be microscopic image of the coal sample. In an embodiment, the image repository 103 may be communicatively attached to an image capturing device, such as a microscope, which may be used for capturing a plurality of petrographic images 105 of the coal sample. In an exemplary implementation, the image repository 103 may be configured within the classification system 101.

In an embodiment, the classification model 107 may be a deep learning model trained for analysing the petrographic image 105 of the coal sample and provide analysis results 111 related to classification of the coal sample. In an embodiment, the classification model 107 may comprise, without limiting to, a plurality of classifiers 109 (alternatively referred as plurality of U-net CNN classifiers 109). The plurality of classifiers 109 may include, without limiting to, a plurality of U-net Convolution Neural Network (CNN) classifiers that are trained to perform segmentation of the petrographic image 105. Particularly, each of the plurality of U-net CNN classifiers 109 in the classification model 107 may be trained for segmenting/classifying at least one class of one or more maceral classes. In other words, each of the plurality of U-net CNN classifiers 109 are trained to individually identify and segment one of the maceral classes among vitrinite, inertinite, liptinite and minerals.

In an embodiment, the classification system 101 may be a computing device such as, without limiting to, a desktop computer, a server, a laptop and the like, which may be configured for automated classification of coal maceral in accordance with various embodiments of the present disclosure. In an exemplary implementation, both the image repository 103 and the classification model 107 may be installed/configured within the classification system 101.

In an embodiment, upon receiving the petrographic image 105 from the image repository 103, the classification system 101 may process the petrographic image 105 and obtain a plurality of overlapping image patches from the petrographic image 105. In an embodiment, the number of image patches to be obtained/extracted from a single petrographic image 105 may be determined based on the size and/or pixel dimension of the petrographic image 105. As an example, if the petrographic image 105 has a pixel dimension of 960*1280 pixels, the classification system 101 may retrieve 6 overlapping image patches, each having a pixel dimension of 512*512 pixels, such that a combined dimension of each of the overlapping image patches is equal to or more than the dimension of the petrographic image 105. In an embodiment, the classification system 101 may divide the petrographic image 105 into the plurality of overlapping image patches for reducing training time of the classification model 107. In other words, training the classification model 107 on the whole petrographic image 105 may require more number of layers, and hence more number training parameters, when compared to training the classification model 107 on smaller image patches. Therefore, obtaining the plurality of overlapping image patches from the petrographic image 105 is an important step in the analysis.

In an embodiment, once the plurality of image patches is obtained, the classification system 101 may feed each of the plurality of image patches as inputs to one of the plurality of U-net CNN classifiers 109 in the classification model 107. Subsequently, each of the plurality of U-net CNN classifiers 109 may identify and segment one of the one or more maceral classes present in each of the plurality of image patches. That is, each U-net CNN classifier performs a binary classification on the image patch assigned to it, to identify whether a particular pixel in the image patch belongs to one of the maceral classes, for which the U-net CNN classifier is trained. Thus, each of the plurality of U-net CNN classifiers 109 function as a pixel-level classifier to accurately determine whether a selected pixel of the image patch belongs to a selected maceral class.

In an embodiment, after each of the plurality of image patches are analysed by the classification model 107, the classification system 101 may amalgamate the analysis results 111 obtained from each of the plurality of U-net CNN classifiers 109 to generate a final classification result 113. As an example, the classification result 113 may indicate, without limitation, a proportion of the coal maceral belonging to each maceral class including, without limiting to, vitrinite, inertinite, liptinite and minerals. In an embodiment, the classification result 113 generated by the classification system 101 may be used for deriving insights into various factors such as quality of coal sample, rank of coal and the like.

FIG. 2A shows a detailed block diagram of a classification system 101 in accordance with some embodiments of the present disclosure.

In some implementations, the classification system 101 may include an I/O interface 201, a processor 203, a User Interface 205 and a memory 207. The I/O interface 201 may be communicatively interfaced with an image repository 103 for receiving the petrographic image 105 of the coal maceral to be analyzed and classified. Further, the I/O interface 201 may be communicatively interfaced with the classification model 107 for receiving the analysis results 111 related to analysis of the petrographic image 105. The User Interface (UI) 205 may be used for, without limitation, displaying the classification result 113 generated by the classification system 101 to a user of the classification system 101. The memory 207 may be communicatively coupled to the processor 203 and may store data 209 and one or more modules 211. The processor 203 may be configured to perform one or more functions of the classification system 101 for automated classification of the coal maceral, using the data 209 and the one or more modules 211.

In an embodiment, the data 209 stored in the memory 207 may include, without limitation, the petrographic image 105, image patches 213 obtained from the petrographic image 105, the analysis results 111, the classification results 113 and other data 215. In some implementations, the data 209 may be stored within the memory 207 in the form of various data structures. Additionally, the data 209 may be organized using data models, such as relational or hierarchical data models. The other data 215 may include various temporary data and files generated by the one or more modules 211 while performing various functions of the classification system 101. As an example, the other data 215 may include, without limitation, one or more reference and/or historical petrographic images used for training the classification model 107, training and inputs parameters of the classification model 107 and the like.

In an embodiment, the petrographic image 105 may be the microscopic image of the coal sample and/or the coal maceral which needs to be classified by the classification system 101. That is, the petrographic image 105 may be obtained during petrographic analysis of the coal sample. An exemplary petrographic image 105 has been shown in FIG. 3A (a).

In an embodiment, the image patches 213 (also referred as plurality of image patches 213 or plurality of overlapping image patches 213) may be obtained by dividing the petrographic image 105 into a predetermined number of image fragments. In an embodiment, the image patches 213 are obtained with an objective of reducing the training time of the classification model 107. Therefore, the number of image patches 213 to be obtained from the petrographic image 105 may be determined based on the size and/or dimension of the petrographic image 105. Accordingly, the number of image patches 213 obtained from the petrographic image 105 may increase with an increase in the size/dimension of the petrographic image 105. In an embodiment, the image patches 213 may be generated with overlapping boundaries. This is to ensure that no information from the original petrographic image 105 is lost while obtaining the image patches 213 from the petrographic image 105.

In an embodiment, the analysis results 111 may be the outcome and/or results generated by each of the plurality of U-net CNN classifiers 109, after analysis of the plurality of image patches 213. In an embodiment, the analysis results 111 may indicate, without limitations, distinctly labelled regions of the plurality of maceral classes on each of the plurality of image patches 213. In an embodiment, the classification result 113 may be generated by amalgamating the analysis results 111 obtained from each of the plurality of U-net CNN classifiers 109. In other words, the classification result 113 may be the final outcome of the classification system 101, and it may indicate distinctly labelled regions of each maceral classes of the sample. Using the classification result 113, a petrographer or an analyst may easily derive information such as, without limiting to, composition of the coal maceral, proportion of different maceral classes present in the coal maceral and the like.

In an embodiment, the data 209 may be processed by the one or more modules 211 of the classification system 101. In some implementations, the one or more modules 211 may be communicatively coupled to the processor 203 for performing one or more functions of the classification system 101. In an implementation, the one or more modules 211 may include, without limiting to, a receiving module 217, the classification model 107 and other modules 219.

As used herein, the term module may refer to an Application Specific Integrated Circuit (ASIC), an electronic circuit, a hardware processor (shared, dedicated, or group) and memory that execute one or more software or firmware programs, a combinational logic circuit, and/or other suitable components that provide the described functionality. In an implementation, each of the one or more modules 211 may be configured as stand-alone hardware computing units. In an embodiment, the other modules 219 may be used to perform various miscellaneous functionalities of the classification system 101. It will be appreciated that such one or more modules 211 may be represented as a single module or a combination of different modules.

In an embodiment, the receiving module 217 may be used for receiving the petrographic image 105 of the coal maceral from the image repository 103 associated with the classification system 101. In an embodiment, the receiving module 217 may also directly interface with a microscopic and/or similar image capturing device that was used to capture the petrographic image 105 of the coal maceral, to directly receive the petrographic image 105 of the coal maceral.

In an embodiment, the classification model 107 may be configured for performing various actions including, without limitation, obtaining plurality of image patches 213 from the petrographic image 105, analyzing the each of the plurality of image patches 213 and classifying the coal maceral based on analysis of each of the plurality of image patches 213.

In an embodiment, upon receiving the petrographic image 105, the classification model 107 may obtain the predetermined number of image patches 213 from the petrographic image 105 by dividing the petrographic image 105 into overlapping patches. As an example, the classification model 107 may generate six image patches 213 of size 512x512 from a petrographic image 105 of size 960x1280. In an embodiment, in the training phase, each of the plurality of image patches 213 may be used for training individual U-net CNN classifiers 109 and each image patch may contain a ground truth information related to the different maceral classes. In an exemplary embodiment, two datasets, one having 204 image patches 213 and the other having 371 image patches 213, may be used in the training of the U-net CNN classifiers 109.

In an embodiment, for obtained better and accurate classification result 113, the U-net CNN classifiers 109 may be trained extensively for classification of majority classes and minority classes. In an embodiment, training for the majority class (that is, for example vitrinite or inertinite classes) is done by minimizing the binary cross-entropy loss. The intersection-over-union between a predicted image segment and a ground truth segment may be used the evaluation metric for evaluating cross-entropy loss. Further, an area and intensity-based regularization may be combined with the binary cross-entropy loss for minority classes (for example, liptinite or mineral classes). Here, while the area regularization differentiates smooth embedded mineral matter region with respect to larger resin-based background region, the intensity based regularization penalizes the image intensity band that is not expected for a given class. Thereafter, the classification model 107 may separately model each class using individual dedicated U-net CNN classifiers 109.
Finally, the classification model 107 may combine the analysis results 111 from each of the plurality of U-net CNN classifiers 109 to generate a final classification result 113, after post-processing each of the plurality of image patches 213 and removing insignificant smaller and irregular regions from each of the plurality of image patches 213.

FIGS. 2B and 2C show exemplary representations of the U-Net classifier in accordance with some embodiments of the present disclosure.

FIG. 2B shows architecture of an exemplary U-net CNN classifier, which may be used for identifying and segmenting one of the maceral classes from the plurality of image patches 213. In an embodiment, the U-net CNN classifier may consist of a contracting path and an expansive path. The contracting path follows the typical architecture of a convolutional network. It consists of a repeated application of two 3x3 convolutions (unpadded convolutions), each followed by a Rectified Linear Unit (ReLU) and a 2x2 Maxpool operation with stride 2 for downsampling. Further, every step in the expansive path may consist of an upsampling of the feature map followed by a 2x2 convolution that halves the number of feature channels, a concatenation with the correspondingly cropped feature map from the contracting path, and two 3x3 convolutions, each followed by a ReLU. The cropping may be necessary due to loss of border pixels in every convolution. At the final layer, a 1x1 convolution may be used to map each 64-component feature vector to the desired number of classes.

FIG. 2C shows an architecture of the proposed multiple U-net CNN classifiers 109. In an embodiment, each of the binary U-net CNN classifiers 109 may perform the task of identifying whether a pixel belongs to a particular maceral class or not. For example, to classify the coal maceral into five maceral classes, namely liptinite, inertinite, vitrinite, minerals and resin, five distinct U-net CNN classifiers 109 may be used. Here, each U-net CNN classifier may be trained to classify a pixel of one particular color, which uniquely represents a single maceral class. That is, the five U-net CNN classifiers 109 may be trained for segmenting the pixels of colors – magenta, cyan, red, green, and blue which represent the maceral classes – minerals, resin, vitrinite, inertinite and liptinite respectively.

In an embodiment, after each of the plurality of U-net CNN classifiers 109 successfully segments the plurality of image patches 213 assigned to them, the analysis results 111 from each of the plurality U-net CNN classifiers 109 may be further processed to perform various corrective actions including, without limitation, border correction, uniformity-based correction, region-based correction and shape-based corrections. Finally, the fine segmentation result obtained after performing the above corrective action may be considered as the final classification result 113. The classification result 113 may be an image, in which each maceral class is distinctly labelled and/or color coded, as shown in FIG. 3A (c).

FIGS. 3A and 3B show a comparison between the classification results obtained from the existing techniques and the proposed method in accordance with some exemplary embodiments.

FIG. 3A draws a comparison between the classification results obtained from traditional random forest approach and the proposed deep learning method. FIG. 3A (a) shows a sample petrographic image 105 before analysis. FIG. 3A (b) and 3A (c) represent the classification results obtained by the random forest approach and the proposed deep learning method, respectively. By visual comparison of the images (b) and (c), it may be observed that, in the result of random forest, the green labelled ‘inertinite’ is taking more larger area as compared to actual ones present in the input image (i.e., bright white inertinite). That is, the random forest approach has overpredicted the inertinite as compared to the actual ones. Whereas, in the deep learning analysis result in FIG. 3A (c), the ‘inertinite’ (green labelled) has distinct and clear band, which is very similar to the input image. Also, the mineral class (labelled as magenta) has distinct appearance in the analysis results 111 obtained using the proposed deep learning method.

Similarly, FIG. 3B draws a comparison between intensity-based regularization for liptinite (blue labelled) detection, performed using the random forest approach and the proposed deep learning method. FIG. 3B (a) shows an input image with liptinite region. FIG. 3B (b) and (c) represent the analysis results 111 obtained from the random forest approach and the proposed deep learning method, respectively. Based on the visual comparison of FIG. 3B (b) and (c), it may be observed that the liptinite region (blue labelled) is more distinctly identified after using the intensity-based regularization of the proposed deep learning method.

FIGS. 4A-4C show various comparisons between results obtained by the existing techniques and the proposed method in accordance with some exemplary embodiments.

FIG. 4A (a) and (b) represent the normalized confusion matrix obtained for the predictions from random forest approach and the proposed deep learning method. As seen in FIG. 4A (a) and (b), the proposed deep learning method makes more consistent and accurate predictions than the random forest approach.

FIG. 4B (a) and (b) show an Area Under the Curve (AUC) and Receiver Operating Characteristics (ROC) or AUC-ROC evaluation metrics for comparing the classification performance of the random forest approach and the proposed deep learning method, respectively. Here, the vitrinite, inertinite, liptinite, mineral and background classes are labelled as red, green, blue, magenta, and cyan respectively. From the inspection of the AUC-ROC curves in FIG. 4B (a) and (b), it may be observed that, in the proposed deep learning method, the overall accuracy of the vitrinite, inertinite, liptinite and mineral classification is around 90%. Whereas, in the case of random forest approach, the mineral class identification is around 65% only. This indicates that the proposed deep learning method does better minority class detection than the random forest approach.

FIG. 4C (a) and (b) show a comparison of gridding method accuracy for the random forest approach and the proposed deep learning method. From FIG. 4C (a) and (b), it may be observed that the inertinite is overpredicted in the case of random forest approach, with an accuracy of around 100 %. However, the vitrinite and mineral class accuracy is found to be around 44 and 0 percentage, respectively. But for same image, the proposed deep learning method results in more than 70 % accuracy for vitrinite, inertinite and mineral.

In an embodiment, proper and accurate minority class identification may be an important prospect for automated maceral analysis. In case of random forest approach, due to improper identification of minority class, inertinite is overpredicted as compared to actual petrography. The inertinite is a maceral having heterogenous texture of mineral within grain. Therefore, proper identification of the mineral associated with inertinite can improve the accuracy of classification of both inertinite and mineral. The proposed deep learning method solves this challenge and ensures the inertinite is not overpredicted.

FIG. 5 shows a flowchart illustrating a method for automated classification of coal maceral in accordance with some embodiments of the present disclosure.

As illustrated in FIG. 5, the method 500 may include one or more blocks illustrating a method for automated classification of coal maceral using a classification system 101 illustrated in FIG. 1. The method 500 may be described in the general context of computer executable instructions. Generally, computer executable instructions can include routines, programs, objects, components, data structures, procedures, modules, and functions, which perform specific functions or implement specific abstract data types.

The order in which the method 500 is described is not intended to be construed as a limitation, and any number of the described method blocks can be combined in any order to implement the method. Additionally, individual blocks may be deleted from the methods without departing from the scope of the subject matter described herein. Furthermore, the method can be implemented in any suitable hardware, software, firmware, or combination thereof.

At block 501, the method 500 includes obtaining, by the classification system 101, a plurality of image patches 213 from a petrographic image 105 of a coal sample. In an embodiment, the plurality of image patches 213 may be obtained by segmenting the petrographic image 105 into a plurality of image patches 213 of predetermined pixel dimension. Further, the predetermined pixel dimension may be a value relative to the pixel dimension of the petrographic image 105.

At block 503, the method 500 includes analyzing, by the classification system 101, each of the plurality of image patches 213 using one of a plurality of U-net Convolutional Neural Network (CNN) classifiers. In an embodiment, each of the plurality of U-net CNN classifiers 109 may be trained for identifying one of a plurality of maceral classes. As an example, the plurality of maceral classes may include, without limiting to, at least one of vitrinite, inertinite, liptinite and mineral.

In an embodiment, each of the plurality of U-net CNN classifiers 109 may be trained using predetermined petrographic images comprising ground truth information related to each of the plurality of maceral classes. Further, training the plurality of U-net CNN classifiers 109 may comprise training the plurality of U-net CNN classifiers 109 for a majority maceral class, which comprises at least one of vitrinite and inertinite, by minimizing a binary cross-entropy loss. Furthermore, training the plurality of U-net CNN classifiers 109 may include training the plurality of U-net CNN classifiers 109 for a minority maceral class, which comprises at least one of liptinite and mineral, by performing area based regularization and intensity based regularization.

In an embodiment, the plurality of U-net CNN classifiers 109 may identify the plurality of maceral classes by sequentially analysing each of a plurality of pixels in the plurality of image patches 213 and then identifying the plurality of maceral classes based on the sequential analysis.

At block 505, the method 500 includes classifying, by the classification system 101, the coal maceral into the plurality of maceral classes by amalgamating analysis results 111 from each of the plurality of U-net CNN classifiers 109. As an example, the analysis results 111 may comprise image patches 213 having distinctly labelled regions of the plurality of maceral classes. In an embodiment, amalgamating the analysis results 111 of each of the plurality of U-net CNN classifiers 109 may comprise eliminating insignificant, smaller, and irregular regions from the analysis results 111.

Computer System
FIG. 6 illustrates a block diagram of an exemplary computer system 600 for implementing embodiments consistent with the present disclosure. In an embodiment, the computer system 600 may be the classification system 101 illustrated in FIG. 1, which may be used for automated classification of coal maceral. The computer system 600 may include a Central Processing Unit (“CPU” or “processor”) 602. The processor 602 may comprise at least one data processor for executing program components for executing user- or system-generated business processes. A user may include a technician, a petrographic analyst, or an operator of the classification system 101 or any system/sub-system being operated parallelly to the computer system 600. The processor 602 may include specialized processing units such as integrated system (bus) controllers, memory management control units, floating point units, graphics processing units, digital signal processing units, etc.
The processor 602 may be disposed in communication with one or more Input/Output (I/O) devices (611 and 612) via I/O interface 601. The I/O interface 601 may employ communication protocols/methods such as, without limitation, audio, analog, digital, stereo, IEEE®-1394, serial bus, Universal Serial Bus (USB), infrared, PS/2, BNC, coaxial, component, composite, Digital Visual Interface (DVI), high-definition multimedia interface (HDMI), Radio Frequency (RF) antennas, S-Video, Video Graphics Array (VGA), IEEE® 802.n /b/g/n/x, Bluetooth, cellular (e.g., Code-Division Multiple Access (CDMA), High-Speed Packet Access (HSPA+), Global System For Mobile Communications (GSM), Long-Term Evolution (LTE) or the like), etc. Using the I/O interface 601, the computer system 600 may communicate with one or more I/O devices 611 and 612.
In some embodiments, the processor 602 may be disposed in communication with a communication network 609 via a network interface 603. The network interface 603 may communicate with the communication network 609. The network interface 603 may employ connection protocols including, without limitation, direct connect, Ethernet (e.g., twisted pair 10/100/1000 Base T), Transmission Control Protocol/Internet Protocol (TCP/IP), token ring, IEEE® 802.11a/b/g/n/x, etc. Using the network interface 603 and the communication network 609, the computer system 600 may connect with an image repository 103 for receiving a petrographic image 105 of a coal sample. Further, the communication network 609 may be used for interfacing the computer system 600 with a classification model 107 that analyses the petrographic image 105 from the image repository 103 and provides analysis results 111 required for classification of the coal maceral.
In an implementation, the communication network 609 may be implemented as one of the several types of networks, such as intranet or Local Area Network (LAN) and such within the organization. The communication network 609 may either be a dedicated network or a shared network, which represents an association of several types of networks that use a variety of protocols, for example, Hypertext Transfer Protocol (HTTP), Transmission Control Protocol/Internet Protocol (TCP/IP), Wireless Application Protocol (WAP), etc., to communicate with each other. Further, the communication network 609 may include a variety of network devices, including routers, bridges, servers, computing devices, storage devices, etc.
In some embodiments, the processor 602 may be disposed in communication with a memory 605 (e.g., RAM 613, ROM 614, etc. as shown in FIG. 6) via a storage interface 604. The storage interface 604 may connect to memory 605 including, without limitation, memory drives, removable disc drives, etc., employing connection protocols such as Serial Advanced Technology Attachment (SATA), Integrated Drive Electronics (IDE), IEEE-1394, Universal Serial Bus (USB), fiber channel, Small Computer Systems Interface (SCSI), etc. The memory drives may further include a drum, magnetic disc drive, magneto-optical drive, optical drive, Redundant Array of Independent Discs (RAID), solid-state memory devices, solid-state drives, etc.
The memory 605 may store a collection of program or database components, including, without limitation, user/application interface 606, an operating system 607, a web browser 608, and the like. In some embodiments, computer system 600 may store user/application data 606, such as the data, variables, records, etc. as described in this invention. Such databases may be implemented as fault-tolerant, relational, scalable, secure databases such as Oracle® or Sybase®.
The operating system 607 may facilitate resource management and operation of the computer system 600. Examples of operating systems include, without limitation, APPLE® MACINTOSH® OS X®, UNIX®, UNIX-like system distributions (E.G., BERKELEY SOFTWARE DISTRIBUTION® (BSD), FREEBSD®, NETBSD®, OPENBSD, etc.), LINUX® DISTRIBUTIONS (E.G., RED HAT®, UBUNTU®, KUBUNTU®, etc.), IBM® OS/2®, MICROSOFT® WINDOWS® (XP®, VISTA®/7/8, 10 etc.), APPLE® IOS®, GOOGLE TM ANDROID TM, BLACKBERRY® OS, or the like.
The user interface 606 may facilitate display, execution, interaction, manipulation, or operation of program components through textual or graphical facilities. For example, the user interface 606 may provide computer interaction interface elements on a display system operatively connected to the computer system 600, such as cursors, icons, check boxes, menus, scrollers, windows, widgets, and the like. Further, Graphical User Interfaces (GUIs) may be employed, including, without limitation, APPLE® MACINTOSH® operating systems’ Aqua®, IBM® OS/2®, MICROSOFT® WINDOWS® (e.g., Aero, Metro, etc.), web interface libraries (e.g., ActiveX®, JAVA®, JAVASCRIPT®, AJAX, HTML, ADOBE® FLASH®, etc.), or the like.

The web browser 608 may be a hypertext viewing application. Secure web browsing may be provided using Secure Hypertext Transport Protocol (HTTPS), Secure Sockets Layer (SSL), Transport Layer Security (TLS), and the like. The web browsers 608 may utilize facilities such as AJAX, DHTML, ADOBE® FLASH®, JAVASCRIPT®, JAVA®, Application Programming Interfaces (APIs), and the like. Further, the computer system 600 may implement a mail server stored program component. The mail server may utilize facilities such as ASP, ACTIVEX®, ANSI® C++/C#, MICROSOFT®, .NET, CGI SCRIPTS, JAVA®, JAVASCRIPT®, PERL®, PHP, PYTHON®, WEBOBJECTS®, etc. The mail server may utilize communication protocols such as Internet Message Access Protocol (IMAP), Messaging Application Programming Interface (MAPI), MICROSOFT® exchange, Post Office Protocol (POP), Simple Mail Transfer Protocol (SMTP), or the like. In some embodiments, the computer system 600 may implement a mail client stored program component. The mail client may be a mail viewing application, such as APPLE® MAIL, MICROSOFT® ENTOURAGE®, MICROSOFT® OUTLOOK®, MOZILLA® THUNDERBIRD®, and the like.
Furthermore, one or more computer-readable storage media may be utilized in implementing embodiments consistent with the present invention. A computer-readable storage medium refers to any type of physical memory on which information or data readable by a processor may be stored. Thus, a computer-readable storage medium may store instructions for execution by one or more processors, including instructions for causing the processor(s) to perform steps or stages consistent with the embodiments described herein. The term “computer-readable medium” should be understood to include tangible items and exclude carrier waves and transient signals, i.e., non-transitory. Examples include Random Access Memory (RAM), Read-Only Memory (ROM), volatile memory, nonvolatile memory, hard drives, Compact Disc (CD) ROMs, Digital Video Disc (DVDs), flash drives, disks, and any other known physical storage media.

Advantages of the embodiments of the present disclosure are illustrated herein.
In an embodiment, the present disclosure helps in automatically classifying coal maceral into various classes such as vitrinite, inertinite, liptinite and mineral and thereby helps in studying and selecting coal blends without compromising on coke quality.

In an embodiment, the method of present disclosure replaces existing manual petrographic approaches with automated petrographic techniques that are accurate, reliable and faster compared to the existing manual petrographic approaches.

In an embodiment, with the use of deep learning techniques, the method of present disclosure results in more intricate segmentation and superior minority class detection compared to existing machine learning based classification approaches.

In an embodiment, the method of present disclosure eliminates requirement of manual feature engineering as the deep learning models are data driven and capable of replacing hand-crafted feature extraction.

In light of the technical advancements provided by the proposed method and the tracking system, the claimed steps, as discussed above, are not routine, conventional, or well-known aspects in the art, as the claimed steps provide the aforesaid solutions to the technical problems existing in the conventional technologies. Further, the claimed steps clearly bring an improvement in the functioning of the system itself, as the claimed steps provide a technical solution to a technical problem.
The terms "an embodiment", "embodiment", "embodiments", "the embodiment", "the embodiments", "one or more embodiments", "some embodiments", and "one embodiment" mean "one or more (but not all) embodiments of the invention(s)" unless expressly specified otherwise.

The terms "including", "comprising", “having” and variations thereof mean "including but not limited to", unless expressly specified otherwise.

The enumerated listing of items does not imply that any or all the items are mutually exclusive, unless expressly specified otherwise. The terms "a", "an" and "the" mean "one or more", unless expressly specified otherwise.

A description of an embodiment with several components in communication with each other does not imply that all such components are required. On the contrary, a variety of optional components are described to illustrate the wide variety of possible embodiments of the invention.

When a single device or article is described herein, it will be clear that more than one device/article (whether they cooperate) may be used in place of a single device/article. Similarly, where more than one device/article is described herein (whether they cooperate), it will be clear that a single device/article may be used in place of the more than one device/article or a different number of devices/articles may be used instead of the shown number of devices or programs. The functionality and/or features of a device may be alternatively embodied by one or more other devices which are not explicitly described as having such functionality/features. Thus, other embodiments of invention need not include the device itself.

Finally, the language used in the specification has been principally selected for readability and instructional purposes, and it may not have been selected to delineate or circumscribe the inventive subject matter. It is therefore intended that the scope of the invention be limited not by this detailed description, but rather by any claims that issue on an application based here on. Accordingly, the embodiments of the present invention are intended to be illustrative, but not limiting, of the scope of the invention, which is set forth in the following claims.

While various aspects and embodiments have been disclosed herein, other aspects and embodiments will be apparent to those skilled in the art. The various aspects and embodiments disclosed herein are for purposes of illustration and are not intended to be limiting, with the true scope and spirit being indicated by the following claims.

Referral Numerals:
Reference Number Description
100 Exemplary environment
101 Classification system
103 Image repository
105 Petrographic image
107 Classification model
109 Plurality of classifiers
111 Analysis results
113 Classification result
201 I/O Interface
203 Processor
205 User interface
207 Memory
209 Data
211 Modules
213 Image patches
215 Other data
217 Receiving module
219 Other modules
600 Exemplary computer system
601 I/O Interface of the exemplary computer system
602 Processor of the exemplary computer system
603 Network interface
604 Storage interface
605 Memory of the exemplary computer system
606 User/Application
607 Operating system
608 Web browser
609 Communication network
611 Input devices
612 Output devices
613 RAM
614 ROM