Abstract: Present disclosure provides systems and methods for remotely detecting defects in an industrial asset by using image processing techniques. An aspect of the present disclosure pertains to a system including an image receive module to receive an assessment image pertaining to the industrial asset from a computing device, a reference image retrieve module to retrieve a reference image based on an industry associated with the industrial asset, an image based feature extraction module to extracts a feature value of each pixel of the assessment image and the reference image by processing the said assessment image and the said reference image, a defect detection module to detect the defect in the industrial asset by determining a difference image from the assessment image and the reference image, and a defect based notification module to notify the computing device on detection of the defect in the industrial asset.
[0001] The present disclosure relates to the field of defect detection in industrial assets.
In particular, the present disclosure provides systems and methods for remotely detecting defects in an industrial equipment based asset by using image processing techniques.
BACKGROUND
[0002] The background description includes information that may be useful in
understanding the present invention. It is not an admission that any of the information provided herein is prior art or relevant to the presently claimed invention, or that any publication specifically or implicitly referenced is prior art
[0003] Monitoring and inspection in an industry aids in increasing efficiency to control
quality of output that the industry is capable to provide. Defects in an asset pertaining to the industry can be in the form of corrosion, cracks, misalignment of components, missing or deformed material, dents, tolerance defects, etc that can occur in various industrial equipments. In case said defects are not rectified timely, safety can be compromised that can give rise to undesirable situations such as rise in maintenance costs.
[0004] Defect detection in industrial equipments has been a topic of considerable
research using different approaches. Currently, many defect detection techniques are available, each tailored for inspecting certain industry with the objective of locating defects accurately. The output of these techniques however reveals little information about the defects themselves and therefore human intervention is required, thereby, making defect detection usually more time consuming and rendering low accuracy rates. Thus, automation for defect detection is required to reduce human intervention and improve the accuracy of defect detection.
[0005] There is therefore a need in the art to provide a system and method that can
remotely detect defects in industrial equipment without human intervention in order to enhance performance and reliability of an industry, thereby increasing its efficiency.
OBJECTS OF THE PRESENT DISCLOSURE
[0006] Some of the objects of the present disclosure, which at least one embodiment
herein satisfies are as listed herein below.
[0007] It is a general object of the present disclosure to provide system and method for
detecting a defect in an industrial equipment based asset.
[0008] It is another object of the present disclosure to provide system and method for
detecting defects in an industrial equipment based asset by incorporating image processing
techniques.
[0009] It is another object of the present disclosure to provide system and method for
detecting defects in an industrial equipment based asset that can detect industrial defects in real
time.
[0010] It is another object of the present disclosure to provide system and method for
detecting defects in an industrial equipment based asset that is cost and time effective.
[0011] These and other objects of the present invention will become readily apparent
from the following detailed description taken in conjunction with the accompanying drawings.
SUMMARY
[0012] The present disclosure relates to the field of defect detection in industrial assets.
In particular, the present disclosure provides systems and methods for remotely detecting defects
in an industrial equipment based asset by using image processing techniques.
[0013] An aspect of the present disclosure pertains to a system including a non-transitory
storage device having embodied therein one or more routines operable to detect a defect in an industrial equipment based asset; and one or more processors coupled to the non-transitory storage device and operable to execute the one or more routines, wherein the one or more routines include: an image receive module, which when executed by the one or more processors, receives an assessment image pertaining to the industrial equipment based asset from a computing device; a reference image retrieve module, which when executed by the one or more processors, retrieves at least one reference image based on an industry associated with the industrial equipment based asset; an image based feature extraction module, which when executed by the one or more processors, extracts a feature value of each pixel of the assessment image and the at least one reference image by processing the said assessment image and the said
at least one reference image; a defect detection module, which when executed by the one or more
processors, detects the defect in the industrial equipment based asset by determining a difference
image from the assessment image and the at least one reference image, wherein the difference
image is obtained by comparing the feature value of each pixel of the assessment image with the
feature value of corresponding pixel of the at least one reference image; and a defect based
notification module, which when executed by the one or more processors, notifies the computing
device on detection of the defect in the industrial equipment based asset.
[0014] In an embodiment, the defect based notification module can notify geographical
location of the industrial equipment based asset associated with the defect to the computing
device.
[0015] In an embodiment, the feature value can be calculated based on image parameters
such as color, brightness, contrast, saturation, and sharpness.
[0016] In an embodiment, the reference image retrieve module can retrieve the at least
one reference image from a storage device configured with a server.
[0017] In an embodiment, the industrial equipment based asset can be an asset pertaining
to industry such as oil and gas, steel, power plant, wind energy, power utility, roadways,
railways, shipping, water services, mining and infrastructure.
[0018] In an embodiment, the defect detection module can detect the defect by providing
any or a combination of the assessment image, the at least one reference image and the
difference image to an artificial neural network trained to detect the defect.
[0019] In an embodiment, the defect detection module can associate a defect category
based on the detected defect with the industrial equipment based asset.
[0020] Another aspect of the present disclosure pertains to a method comprising steps of:
receiving, by one or more processors, an assessment image pertaining to the industrial equipment
based asset from a computing device; retrieving, by the one or more processors, at least one
reference image based on an industry associated with the industrial equipment based asset;
extracting, by the one or more processors, a feature value of each pixel of the assessment image
and the at least one reference image by processing the said assessment image and the said at least
one reference image; detecting, by the one or more processors, a defect in the industrial
equipment based asset by determining a difference image from the assessment image and the at
least one reference image, wherein the difference image is obtained by comparing the feature
value of each pixel of the assessment image with the feature value of corresponding pixel of the at least one reference image; and notifying, by the one or more processors, the computing device on detection of the defect in the industrial equipment based asset.
[0021] Various objects, features, aspects and advantages of the inventive subject matter
will become more apparent from the following detailed description of preferred embodiments, along with the accompanying drawing figures in which like numerals represent like components.
BRIEF DESCRIPTION OF THE DRAWINGS
[0022] In the figures, similar components and/or features may have the same reference
label. Further, various components of the same type may be distinguished by following the
reference label with a second label that distinguishes among the similar components. If only the
first reference label is used in the specification, the description is applicable to any one of the
similar components having the same first reference label irrespective of the second reference
label.
[0023] FIG. 1 illustrates exemplary network architecture in which or with which
proposed system can be implemented in accordance with an embodiment of the present
disclosure.
[0024] FIG. 2 illustrates exemplary functional modules of the proposed system in
accordance with an exemplary embodiment of the present disclosure.
[0025] FIG. 3 is a flow diagram illustrating a process for detecting a defect in an
industrial equipment based asset in accordance with an embodiment of the present disclosure.
[0026] FIG. 4 illustrates exemplary representation of a user interface for detecting a
defect by the proposed system in accordance with an embodiment of the present disclosure.
[0027] FIG. 5A-D illustrates exemplary representations of assessment images and
reference images that the proposed system can use to perform defect detection in accordance
with an embodiment of the present disclosure.
[0028] FIG. 6 illustrates an exemplary computer system in which or with which
embodiments of the present invention can be utilized in accordance with embodiments of the
present disclosure.
DETAILED DESCRIPTION
[0029] In the following description, numerous specific details are set forth in order to
provide a thorough understanding of embodiments of the present invention. It will be apparent to one skilled in the art that embodiments of the present invention may be practiced without some of these specific details.
[0030] Embodiments of the present invention may be provided as a computer program
product, which may include a machine-readable storage medium tangibly embodying thereon instructions, which may be used to program a computer (or other electronic devices) to perform a process. The machine-readable medium may include, but is not limited to, fixed (hard) drives, magnetic tape, floppy diskettes, optical disks, compact disc read-only memories (CD-ROMs), and magneto-optical disks, semiconductor memories, such as ROMs, PROMs, random access memories (RAMs), programmable read-only memories (PROMs), erasable PROMs (EPROMs), electrically erasable PROMs (EEPROMs), flash memory, magnetic or optical cards, or other type of media/machine-readable medium suitable for storing electronic instructions (e.g., computer programming code, such as software or firmware).
[0031] Various methods described herein may be practiced by combining one or more
machine-readable storage media containing the code according to the present invention with appropriate standard computer hardware to execute the code contained therein. An apparatus for practicing various embodiments of the present invention may involve one or more computers (or one or more processors within a single computer) and storage systems containing or having network access to computer program(s) coded in accordance with various methods described herein, and the method steps of the invention could be accomplished by modules, routines, subroutines, or subparts of a computer program product.
[0032] If the specification states a component or feature “may”, “can”, “could”, or
“might” be included or have a characteristic, that particular component or feature is not required to be included or have the characteristic.
[0033] As used in the description herein and throughout the claims that follow, the
meaning of “a,” “an,” and “the” includes plural reference unless the context clearly dictates otherwise. Also, as used in the description herein, the meaning of “in” includes “in” and “on” unless the context clearly dictates otherwise.
6
[0034] The recitation of ranges of values herein is merely intended to serve as a
shorthand method of referring individually to each separate value falling within the range. Unless otherwise indicated herein, each individual value is incorporated into the specification as if it were individually recited herein. All methods described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The use of any and all examples, or exemplary language (e.g. “such as”) provided with respect to certain embodiments herein is intended merely to better illuminate the invention and does not pose a limitation on the scope of the invention otherwise claimed. No language in the specification should be construed as indicating any non-claimed element essential to the practice of the invention.
[0035] Groupings of alternative elements or embodiments of the invention disclosed
herein are not to be construed as limitations. Each group member can be referred to and claimed individually or in any combination with other members of the group or other elements found herein. One or more members of a group can be included in, or deleted from, a group for reasons of convenience and/or patentability. When any such inclusion or deletion occurs, the specification is herein deemed to contain the group as modified thus fulfilling the written description of all groups used in the appended claims.
[0036] Exemplary embodiments will now be described more fully hereinafter with
reference to the accompanying drawings, in which exemplary embodiments are shown. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. These embodiments are provided so that this disclosure will be thorough and complete and will fully convey the scope of the invention to those of ordinary skill in the art. Moreover, all statements herein reciting embodiments of the invention, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, it is intended that such equivalents include both currently known equivalents as well as equivalents developed in the future (i.e., any elements developed that perform the same function, regardless of structure).
[0037] The present disclosure relates to the field of defect detection in industrial assets.
In particular, the present disclosure provides systems and methods for remotely detecting defects in an industrial equipment based asset by using image processing techniques.
7
[0038] An aspect of the present disclosure pertains to a system including a non-transitory
storage device having embodied therein one or more routines operable to detect a defect in an
industrial equipment based asset; and one or more processors coupled to the non-transitory
storage device and operable to execute the one or more routines, wherein the one or more
routines include: an image receive module, which when executed by the one or more processors,
receives an assessment image pertaining to the industrial equipment based asset from a
computing device; a reference image retrieve module, which when executed by the one or more
processors, retrieves at least one reference image based on an industry associated with the
industrial equipment based asset; an image based feature extraction module, which when
executed by the one or more processors, extracts a feature value of each pixel of the assessment
image and the at least one reference image by processing the said assessment image and the said
at least one reference image; a defect detection module, which when executed by the one or more
processors, detects the defect in the industrial equipment based asset by determining a difference
image from the assessment image and the at least one reference image, wherein the difference
image is obtained by comparing the feature value of each pixel of the assessment image with the
feature value of corresponding pixel of the at least one reference image; and a defect based
notification module, which when executed by the one or more processors, notifies the computing
device on detection of the defect in the industrial equipment based asset.
[0039] In an embodiment, the defect based notification module can notify geographical
location of the industrial equipment based asset associated with the defect to the computing
device.
[0040] In an embodiment, the feature value can be calculated based on image parameters
such as color, brightness, contrast, saturation, and sharpness.
[0041] In an embodiment, the reference image retrieve module can retrieve the at least
one reference image from a storage device configured with a server.
[0042] In an embodiment, the industrial equipment based asset can be an asset pertaining
to industry such as oil and gas, steel, power plant, wind energy, power utility, roadways,
railways, shipping, water services, mining and infrastructure.
[0043] In an embodiment, the defect detection module can detect the defect by providing
any or a combination of the assessment image, the at least one reference image and the
difference image to an artificial neural network trained to detect the defect.
8
[0044] In an embodiment, the defect detection module can associate a defect category
based on the detected defect with the industrial equipment based asset.
[0045] Another aspect of the present disclosure pertains to a method comprising steps of:
receiving, by one or more processors, an assessment image pertaining to the industrial equipment based asset from a computing device; retrieving, by the one or more processors, at least one reference image based on an industry associated with the industrial equipment based asset; extracting, by the one or more processors, a feature value of each pixel of the assessment image and the at least one reference image by processing the said assessment image and the said at least one reference image; detecting, by the one or more processors, a defect in the industrial equipment based asset by determining a difference image from the assessment image and the at least one reference image, wherein the difference image is obtained by comparing the feature value of each pixel of the assessment image with the feature value of corresponding pixel of the at least one reference image; and notifying, by the one or more processors, the computing device on detection of the defect in the industrial equipment based asset.
[0046] FIG. 1 illustrates an exemplary network architecture in which or with which
proposed system can be implemented in accordance with an embodiment of the present disclosure.
[0047] According to an embodiment of the present disclosure a defect detection system
102 (also referred to as the system 102, hereinafter) can detect a defect in an industrial equipment based asset (also referred to as the asset/industrial asset, hereinafter) by using image processing techniques. As illustrated, the system 102 can be communicatively coupled with one or more computing devices 106-1, 106-2,.., 106-N (individually referred to as the computing device 106 and collectively referred to as the computing devices 106, hereinafter) through a network 104. In an embodiment, the system 102 can be implemented using any or a combination of hardware components and software components such as a cloud, a server, a computing system, a computing device, a network device and the like. Further, the system 102 can interact with computing devices 106 through a website or an application that can reside in the computing devices 106. In an implementation, the system 102 can be accessed by website or application that can be configured with any operating system, including but not limited to, AndroidTM, iOSTM, and the like. Examples of the computing devices 106 can include, but are not limited to, a computing device associated with industrial equipment or an industrial equipment based asset, a
9
smart camera, a smart phone, a portable computer, a personal digital assistant, a handheld device and the like.
[0048] Further, the network 104 can be a wireless network, a wired network or a
combination thereof that can be implemented as one of the different types of networks, such as
Intranet, Local Area Network (LAN), Wide Area Network (WAN), Internet, and the like.
Further, the network 104 can either be a dedicated network or a shared network. The shared
network can represent an association of the different types of networks that can use variety of
protocols, for example, Hypertext Transfer Protocol (HTTP), Transmission Control
Protocol/Internet Protocol (TCP/IP), Wireless Application Protocol (WAP), and the like.
[0049] According to various embodiments of the present disclosure, the system 102 can
provide for an Artificial Intelligence (AI) based automatic defect detection by using image processing analytics for large industrial equipment based assets pertaining to various industries such as oil and gas, power generating plants, extra-high voltage (EHV) transmission lines, wind turbines, shipping industries, ports, roadways, railways and other infrastructure sectors. The system 102 can detect defects such as rust, missing/loose nuts and bolts, missing disc/broken insulators, potholes on roads, cracks in an industrial asset, turbine blade defects, loose/broken strands on Aluminium conductor steel-reinforced cable (ACSR) conductor or railway catenary, etc. It would be appreciated that various embodiments of the present disclosure can aid in monitoring the industrial equipment based asset, prioritizing the maintenance, asset performance and determining longevity of the asset.
[0050] In an aspect, the system 102 can receive an assessment image pertaining to the
industrial equipment based asset from the computing device 106. In an embodiment, the system 102 can receive a batch/collection of assessment images pertaining to industrial equipment based asset and can consider one assessment image from the batch of assessment images at a time for defect detection. The industrial equipment based asset can be an asset pertaining to any industry such as oil and gas, steel, power plant, wind energy, power utility, roadways, railways, shipping, water services, mining, infrastructure, and the like.
[0051] In an aspect, the system 102 can retrieve a reference image or a set of reference
images such that the assessment image and the reference image can be compared using image processing and analytics techniques for detection of the defect. In an aspect, a server 108 can be operatively coupled with the system 102 that can store various images from which the reference
10
image can be selected. The reference image can be based on an industry that is associated with the industrial equipment based asset. For example, the user can provide the industry associated with the assessment image to the system 102 and the system 102 can then retrieve the reference image based on the selected industry from the server 108.
[0052] In an embodiment, the system 102 can detect a defect by using image processing
and analytics approaches using Artificial Intelligence/Deep Learning techniques such as neural
network, convolutional neural network, Keras, TensorFlow, and the like that can be based on
programming languages such as PHP, Python, HTML, Django, Angular JS, etc.
[0053] In an embodiment, on detection of the defect, the system 102 can mark the defect
on the assessment image and can notify the computing device that the said defect is found. Further, the system 102 can associate a category with the defect and can determine extent of the defect. The system 102 can also notify said category of the defect and its corresponding extent such that suitable action can be taken for rectification. For example, when rusting in an asset is determined by the system 102, the system 102 can determine amount of rusting and can associate the defect with a category named “rusting”, such that the user can take suitable steps for rectifying the defect. In an embodiment, the system 102 can even indicate the geographical location of the industrial asset that is associated with the defect to the user/computing device. Thus, the user can remotely analyse the assets and even know the category, extent and location that can aid the user in taking suitable measure in order to rectify the defect for smooth functioning of industrial applications.
[0054] FIG. 2 illustrates exemplary functional modules of the proposed system in
accordance with an exemplary embodiment of the present disclosure.
[0055] As illustrated, the system 102 can include one or more processor(s) 202. The one
or more processor(s) 202 can be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, logic circuitries, and/or any devices that manipulate data based on operational instructions. Among other capabilities, the one or more processor(s) 202 are configured to fetch and execute computer-readable instructions stored in a memory 204 of the system 102. The memory 204 can store one or more computer-readable instructions or routines, which may be fetched and executed to create or share the data units over a network service. The memory 204 can include any non-transitory storage device
11
including, for example, volatile memory such as RAM, or non-volatile memory such as EPROM, flash memory, and the like.
[0056] The system 102 can also include an interface(s) 206. The interface(s) 206 may
include a variety of interfaces, for example, interfaces for data input and output devices, referred to as I/O devices, storage devices, and the like. The interface(s) 206 may facilitate communication of the system 102 with various devices coupled to the system 102. The interface(s) 206 may also provide a communication pathway for one or more components of the system 102. Examples of such components include, but are not limited to, processing engine(s) 208 and data 210.
[0057] The processing engine(s) 208 can be implemented as a combination of hardware
and programming (for example, programmable instructions) to implement one or more
functionalities of the processing engine(s) 208. In examples described herein, such combinations
of hardware and programming may be implemented in several different ways. For example, the
programming for the processing engine(s) 208 may be processor executable instructions stored
on a non-transitory machine-readable storage medium and the hardware for the processing
engine(s) 208 may include a processing resource (for example, one or more processors), to
execute such instructions. In the present examples, the machine-readable storage medium may
store instructions that, when executed by the processing resource, implement the processing
engine(s) 208. In such examples, the system 102 can include the machine-readable storage
medium storing the instructions and the processing resource to execute the instructions, or the
machine-readable storage medium may be separate but accessible to system 102 and the
processing resource. In other examples, the processing engine(s) 208 may be implemented by
electronic circuitry. The data 210 can include data that is either stored or generated as a result of
functionalities implemented by any of the components of the processing engine(s) 208.
[0058] In an example, the processing engine(s) 208 can include an image receive module
212, a reference image retrieve module 214, an image based feature extraction module 216, a defect detection module 218, a defect based notification module 220 and other module(s) 222. The other module(s) 220 can implement functionalities that supplement applications or functions performed by the system 102 or the processing engine(s) 208.
[0059] In an aspect, the image receive module 212 of the proposed system 102 can
receive an assessment image or a batch/collection of assessment images pertaining to the asset
12
from a computing device such as a smart camera or any other smart device installed in an
industry that can be capable of capturing images of the asset. In an embodiment, there can be a
computing device installed in the industry that can collate all images from various smart devices
and then provide these images to the system 102. In an embodiment, there can be a user who can
coordinate the providing of images to the system 102 by the computing device. In another
embodiment, the computing device can automatically provide images to the system 102. For
example, the computing device can provide images in real time to the system 102 or at a regular
interval of say 1 hour, such that automatic analysis for defect detection can be performed.
[0060] In an aspect, the reference image retrieve module 214 of the proposed system 102
can retrieve at least one reference image. In an embodiment, plurality of reference images can be stored in a database configured with a server such that the stored images can be segregated based on the industry. Thus, when a user uploads the assessment image, the user can select a corresponding industry and based on the industry associated with the assessment image, the reference image retrieve module 214 can determine the industrial asset associated with the assessment image and retrieve the reference image based on the industrial asset. In an alternate embodiment, after determination of the asset, the reference image retrieve module 214 can search the Internet using image search engines like GoogleTM Images to retrieve the reference image.
[0061] In an embodiment, the image based feature extraction module 216 of the proposed
system 102 can extract various features/image parameters of the reference image and the assessment image to obtain relevant information from the said images. The features may represent a pixel or a whole object in an image. Examples of features/image parameters can include color components, length, area, circularity, gradient magnitude, gradient direction, gray-level intensity value, sharpness, brightness, saturation, contrast, etc. The feature extraction module 216 can compute said features using image processing and computer vision techniques or by determining original pixel intensities. In an example, neural network based processing can be applied to said images to extract features. Based on several features extracted from the said images, in an aspect, the feature extraction module 216 can determine a feature value for each pixel of the assessment image and the reference image. In an exemplary embodiment, the feature value can be in form of a vector, for example, v = [R; G; B]; can be a feature vector containing color components of a pixel.
13
[0062] In an aspect, the defect detection module 218 can determine a difference image by
comparing the feature value of each pixel of the assessment image and the reference image. For example, the difference image can be an image indicating the grayscale difference at each corresponding position of the assessment image and the reference image. In an embodiment, the difference image can be assessed for defect detection such that if feature value/difference value of the difference image is above a specified value, the defect detection module 218 can associate the assessment image with a defect and/or a defect category. Exemplary defect categories can be rusting, missing nuts and bolts, missing disc, cracks in an industrial asset, etc. Further, by using various image processing and artificial intelligence techniques, the defect detection module 218 can calculate intensity/extent of the defect. In an exemplary embodiment, the defect detection module 218 can detect the defect by providing any or a combination of the assessment image, the reference image and the difference image to an artificial neural network that is previously trained to detect the defect. Also the defect detection module 218 can use various other AI techniques such as Convolutional Neural Networks (CNN), algorithms related to image processing based on Hue, Saturation, Value (HSV), Open Source Computer Vision (OpenCV) algorithm, Gaussian blur, Canny Edge detection, wire shape detection, morphological operations, etc. It would be appreciated by the one skilled in the art that the system 102 can perform machine learning using said various AI techniques such that after performing said learning, the system 102 can be enabled to detect a defect even without using reference images.
[0063] In an aspect, the defect based notification module 220 can notify the computing
device on detection of the defect in the asset such that suitable action for rectification of the defect can be taken. In an embodiment, along with the defect, the associated category, and the extent of defect, the defect based notification module 220 can also notify geographical location of the industrial equipment based asset associated with the defect to the computing device. Thus, the user can know the location of the asset where the defect has taken place. In an embodiment, the defect based notification module 220 can send notification on mobile phone of a user, such that real time monitoring of defects can take place.
[0064] In an embodiment, the defect based notification module 220 can retrieve the
geographical location associated with the industrial equipment based asset by accessing a geo-tagged data of the assessment image. For instance, the defect based notification module 220 can
14
retrieve latitude and longitude of the industrial equipment based asset by accessing the geo-tagged data of the assessment image.
[0065] Thus, embodiments of the present disclosure provide a complete solution for
remotely monitoring the defects that occur in an industrial asset.
[0066] FIG. 3 is a flow diagram illustrating a process for detecting a defect in an
industrial equipment based asset in accordance with an embodiment of the present disclosure.
[0067] In an aspect, a method for detecting a defect in an industrial equipment based
asset can include a step 302 for receiving an assessment image pertaining to the industrial
equipment based asset from a computing device. In an embodiment, the computing device can be
configured at a user end such that the computing device can collate assessment images from
various image capturing devices for detection of a defect. In another embodiment, a user can
upload an assessment image or a batch/collection of assessment images for defect detection.
[0068] In an embodiment, the method can include a step 304 for retrieving a reference
image based on an industry associated with the industrial equipment based asset. In an embodiment, when the user uploads the assessment image, the user can select a corresponding industry and based on the industry associated with the assessment image, such that the asset can be determined and retrieving the reference image can be performed.
[0069] In an embodiment, the method can include a step 306 for extracting a feature
value of each pixel of the assessment image and the reference image. The feature value can be
determined based on various image parameters/features such as color components, length, area,
circularity, gradient magnitude, gradient direction, gray-level intensity value, sharpness,
brightness, saturation, contrast, etc using image processing and computer vision techniques. In an
exemplary embodiment, the feature value can be in form of a vector, say a feature vector, for
example, v = [R; G; B] can be a feature vector containing color components of a pixel.
[0070] In an aspect, the method can include a step 308 for detecting a defect in the asset
by determining a difference image from the assessment image and the reference image. The difference image can be formed by comparing the feature value of each pixel of the assessment image and the reference image. In an embodiment, the difference image can be assessed for defect detection such that if feature value/difference value of the difference image is above a specified value, the assessment image can be associated with a defect. In an embodiment, on
15
detection of the defect the assessment image can be associated with a defect category such as
rusting, missing nuts and bolts, missing disc, cracks in an industrial asset, etc.
[0071] In an aspect, the method can include a step 310 for notifying the computing
device on detection of the defect in the industrial equipment based asset. Further, the computing
device can also be notified of geographical location of the industrial equipment based asset
associated with the defect such that the user can know the location of the asset where the defect
has taken place and appropriate measures for rectification of defects can be performed.
[0072] Embodiments of the present disclosure include various steps, which have been
described above. A variety of these steps may be performed by hardware components or may be tangibly embodied on a computer-readable storage medium in the form of machine-executable instructions, which may be used to cause a general-purpose or special-purpose processor programmed with instructions to perform these steps. Alternatively, the steps may be performed by a combination of hardware, software, and/or firmware.
[0073] FIG. 4 illustrates exemplary representation of a user interface for detecting a
defect by the proposed system in accordance with an embodiment of the present disclosure.
[0074] As illustrated, the user can be provided with options such as “upload images”,
“processing”, “view analysis” and “download report”. Using “upload images”, the user can provide a batch/collection of images to the system 102. Once the images are uploaded, the user can use “processing” for detecting defect in an asset. Further, after processing the user can view the performed analysis using “view analysis” and can also download the report using “Download Reports” such that the downloaded reports can be accessed subsequently by the user to perform analytics pertaining to defects taking place in an industry. In analysis and reports the system 102 can indicate the assessment images in which the defect is found after marking the said defect and can provide extent and category associated with the defect. For instance, if the user uploads 100 images, and after processing the defect is found in 5 images, the system 102 can show analysis of the 5 images in which the defect is found along with marking of the defect, the extent and the category.
[0075] FIG. 5A-D illustrates exemplary representations of assessment images and
reference images that the proposed system can use to perform defect detection in accordance with an embodiment of the present disclosure.
16
[0076] In an example, the user can upload image 510 for defect detection. The system
can use 520 as a reference image and can notify the computing device of the user that the uploaded image 510 includes an asset that has been corroded. The system can also indicate the level of corrosion such that the asset can be replaced if required.
[0077] In another example, the user can upload image 530 for defect detection. The
system can retrieve image 540 as a reference image for performing analysis of the assessment image 530. The system can notify the user about the missing bolt in the asset pertaining to the assessment image 530 such that the missing bolt can be installed into the asset for proper functioning.
[0078] FIG. 6 illustrates an exemplary computer system in which or with which
embodiments of the present invention can be utilized in accordance with embodiments of the present disclosure.
[0079] As shown in FIG. 6, computer system includes an external storage device 610, a
bus 620, a main memory 630, a read only memory 640, a mass storage device 650, communication port 660, and a processor 670. A person skilled in the art will appreciate that computer system may include more than one processor and communication ports. Examples of processor 670 include, but are not limited to, an Intel® Itanium® or Itanium 2 processor(s), or AMD® Opteron® or Athlon MP® processor(s), Motorola® lines of processors, FortiSOC™ system on a chip processors or other future processors. Processor 670 may include various modules associated with embodiments of the present invention. Communication port 660 can be any of an RS-232 port for use with a modem based dialup connection, a 10/100 Ethernet port, a Gigabit or 10 Gigabit port using copper or fiber, a serial port, a parallel port, or other existing or future ports. Communication port 660 may be chosen depending on a network, such a Local Area Network (LAN), Wide Area Network (WAN), or any network to which computer system connects.
[0080] Memory 630 can be Random Access Memory (RAM), or any other dynamic
storage device commonly known in the art. Read only memory 640 can be any static storage device(s) e.g., but not limited to, a Programmable Read Only Memory (PROM) chips for storing static information e.g., start-up or BIOS instructions for processor 670. Mass storage 650 may be any current or future mass storage solution, which can be used to store information and/or instructions. Exemplary mass storage solutions include, but are not limited to, Parallel Advanced
17
Technology Attachment (PATA) or Serial Advanced Technology Attachment (SATA) hard disk drives or solid-state drives (internal or external, e.g., having Universal Serial Bus (USB) and/or Firewire interfaces), e.g. those available from Seagate (e.g., the Seagate Barracuda 7102 family) or Hitachi (e.g., the Hitachi Deskstar 7K1000), one or more optical discs, Redundant Array of Independent Disks (RAID) storage, e.g. an array of disks (e.g., SATA arrays), available from various vendors including Dot Hill Systems Corp., LaCie, Nexsan Technologies, Inc. and Enhance Technology, Inc.
[0081] Bus 620 communicatively couples processor(s) 670 with the other memory,
storage and communication blocks. Bus 620 can be, e.g. a Peripheral Component Interconnect (PCI) / PCI Extended (PCI-X) bus, Small Computer System Interface (SCSI), USB or the like, for connecting expansion cards, drives and other subsystems as well as other buses, such a front side bus (FSB), which connects processor 670 to software system.
[0082] Optionally, operator and administrative interfaces, e.g. a display, keyboard, and a
cursor control device, may also be coupled to bus 620 to support direct operator interaction with computer system. Other operator and administrative interfaces can be provided through network connections connected through communication port 660. External storage device 610 can be any kind of external hard-drives, floppy drives, IOMEGA® Zip Drives, Compact Disc - Read Only Memory (CD-ROM), Compact Disc - Re-Writable (CD-RW), Digital Video Disk - Read Only Memory (DVD-ROM). Components described above are meant only to exemplify various possibilities. In no way should the aforementioned exemplary computer system limit the scope of the present disclosure.
[0083] Thus, it will be appreciated by those of ordinary skill in the art that the diagrams,
schematics, illustrations, and the like represent conceptual views or processes illustrating systems and methods embodying this invention. The functions of the various elements shown in the figures may be provided through the use of dedicated hardware as well as hardware capable of executing associated software. Similarly, any switches shown in the figures are conceptual only. Their function may be carried out through the operation of program logic, through dedicated logic, through the interaction of program control and dedicated logic, or even manually, the particular technique being selectable by the entity implementing this invention. Those of ordinary skill in the art further understand that the exemplary hardware, software,
18
processes, methods, and/or operating systems described herein are for illustrative purposes and, thus, are not intended to be limited to any particular named.
[0084] While embodiments of the present invention have been illustrated and described,
it will be clear that the invention is not limited to these embodiments only. Numerous
modifications, changes, variations, substitutions, and equivalents will be apparent to those skilled
in the art, without departing from the spirit and scope of the invention, as described in the claim.
[0085] In the foregoing description, numerous details are set forth. It will be apparent,
however, to one of ordinary skill in the art having the benefit of this disclosure, that the present invention may be practiced without these specific details. In some instances, well-known structures and devices are shown in block diagram form, rather than in detail, to avoid obscuring the present invention.
[0086] As used herein, and unless the context dictates otherwise, the term "coupled to" is
intended to include both direct coupling (in which two elements that are coupled to each other contact each other)and indirect coupling (in which at least one additional element is located between the two elements). Therefore, the terms "coupled to" and "coupled with" are used synonymously. Within the context of this document terms "coupled to" and "coupled with" are also used euphemistically to mean “communicatively coupled with” over a network, where two or more devices are able to exchange data with each other over the network, possibly via one or more intermediary device.
[0087] It should be apparent to those skilled in the art that many more modifications
besides those already described are possible without departing from the inventive concepts herein. The inventive subject matter, therefore, is not to be restricted except in the spirit of the appended claims. Moreover, in interpreting both the specification and the claims, all terms should be interpreted in the broadest possible manner consistent with the context. In particular, the terms “comprises” and “comprising” should be interpreted as referring to elements, components, or steps in a non-exclusive manner, indicating that the referenced elements, components, or steps may be present, or utilized, or combined with other elements, components, or steps that are not expressly referenced. Where the specification claims refers to at least one of something selected from the group consisting of A, B, C …. and N, the text should be interpreted as requiring only one element from the group, not A plus N, or B plus N, etc.
19
[0088] While the foregoing describes various embodiments of the invention, other and
further embodiments of the invention may be devised without departing from the basic scope thereof. The scope of the invention is determined by the claims that follow. The invention is not limited to the described embodiments, versions or examples, which are included to enable a person having ordinary skill in the art to make and use the invention when combined with information and knowledge available to the person having ordinary skill in the art.
ADVANTAGES OF THE PRESENT DISCLOSURE
[0089] The present disclosure provides system and method for detecting a defect in an
industrial equipment based asset.
[0090] The present disclosure provides system and method for detecting defects in an
industrial equipment based asset by incorporating image processing techniques.
[0091] The present disclosure provides system and method for detecting defects in an
industrial equipment based asset that can detect industrial defects in real time.
[0092] The present disclosure provides system and method for detecting defects in an
industrial equipment based asset that is cost and time effective.
We Claim:
A system comprising:
a non-transitory storage device having embodied therein one or more routines operable to detect a defect in an industrial equipment based asset; and
one or more processors coupled to the non-transitory storage device and operable to execute the one or more routines, wherein the one or more routines include:
an image receive module, which when executed by the one or more processors, receives an assessment image pertaining to the industrial equipment based asset from a computing device;
a reference image retrieve module, which when executed by the one or more processors, retrieves at least one reference image based on an industry associated with the industrial equipment based asset;
an image based feature extraction module, which when executed by the one or more processors, extracts a feature value of each pixel of the assessment image and the at least one reference image by processing the said assessment image and the said at least one reference image;
a defect detection module, which when executed by the one or more processors, detects the defect in the industrial equipment based asset by determining a difference image from the assessment image and the at least one reference image, wherein the difference image is obtained by comparing the feature value of each pixel of the assessment image with the feature value of corresponding pixel of the at least one reference image; and
a defect based notification module, which when executed by the one or more processors, notifies the computing device on detection of the defect in the industrial equipment based asset.
The system of claim 1, wherein the defect based notification module notifies geographical location of the industrial equipment based asset associated with the defect to the computing device.
The system of claim 1, wherein the feature value is calculated based on image parameters such as color, brightness, contrast, saturation, and sharpness.
The system of claim 1, wherein the reference image retrieve module retrieves the at least one reference image from a storage device configured with a server.
The system of claim 1, wherein the industrial equipment based asset is an asset pertaining to industry such as oil and gas, steel, power plant, wind energy, power utility, roadways, railways, shipping, water services, mining and infrastructure.
The system of claim 1, wherein the defect detection module detects the defect by providing any or a combination of the assessment image, the at least one reference image and the difference image to an artificial neural network trained to detect the defect.
The system of claim 1, wherein the defect detection module associates a defect category based on the detected defect with the industrial equipment based asset.
A method comprising steps of:
receiving, by one or more processors, an assessment image pertaining to the industrial equipment based asset from a computing device;
retrieving, by the one or more processors, at least one reference image based on an industry associated with the industrial equipment based asset;
extracting, by the one or more processors, a feature value of each pixel of the assessment image and the at least one reference image by processing the said assessment image and the said at least one reference image;
detecting, by the one or more processors, a defect in the industrial equipment based asset by determining a difference image from the assessment image and the at least one reference image, wherein the difference image is obtained by comparing the feature value of each pixel of the assessment image with the feature value of corresponding pixel of the at least one reference image; and
notifying, by the one or more processors, the computing device on detection of the defect in the industrial equipment based asset.
The method of claim 8, wherein the feature value is calculated based on image parameters such as color, brightness, contrast, saturation, and sharpness.
The method of claim 8, wherein the geographical location of the industrial equipment based asset associated with the defect is notified to the computing device.
| Section | Controller | Decision Date |
|---|---|---|
| # | Name | Date |
|---|---|---|
| 1 | 201811001316-AMMENDED DOCUMENTS [07-11-2019(online)].pdf | 2019-11-07 |
| 1 | 201811001316-STATEMENT OF UNDERTAKING (FORM 3) [11-01-2018(online)]_16.pdf | 2018-01-11 |
| 2 | 201811001316-Annexure (Optional) [07-11-2019(online)].pdf | 2019-11-07 |
| 2 | 201811001316-STATEMENT OF UNDERTAKING (FORM 3) [11-01-2018(online)].pdf | 2018-01-11 |
| 3 | 201811001316-FORM FOR STARTUP [11-01-2018(online)].pdf | 2018-01-11 |
| 3 | 201811001316-FORM 13 [07-11-2019(online)].pdf | 2019-11-07 |
| 4 | 201811001316-MARKED COPIES OF AMENDEMENTS [07-11-2019(online)].pdf | 2019-11-07 |
| 4 | 201811001316-FORM FOR SMALL ENTITY(FORM-28) [11-01-2018(online)]_2.pdf | 2018-01-11 |
| 5 | 201811001316-Written submissions and relevant documents (MANDATORY) [07-11-2019(online)].pdf | 2019-11-07 |
| 5 | 201811001316-FORM FOR SMALL ENTITY(FORM-28) [11-01-2018(online)].pdf | 2018-01-11 |
| 6 | 201811001316-FORM-26 [21-10-2019(online)].pdf | 2019-10-21 |
| 6 | 201811001316-FORM 1 [11-01-2018(online)].pdf | 2018-01-11 |
| 7 | 201811001316-HearingNoticeLetter-(DateOfHearing-24-10-2019).pdf | 2019-10-14 |
| 7 | 201811001316-EVIDENCE FOR REGISTRATION UNDER SSI(FORM-28) [11-01-2018(online)]_11.pdf | 2018-01-11 |
| 8 | 201811001316-EVIDENCE FOR REGISTRATION UNDER SSI(FORM-28) [11-01-2018(online)].pdf | 2018-01-11 |
| 8 | 201811001316-ABSTRACT [18-09-2019(online)].pdf | 2019-09-18 |
| 9 | 201811001316-CLAIMS [18-09-2019(online)].pdf | 2019-09-18 |
| 9 | 201811001316-EVIDENCE FOR REGISTRATION UNDER SSI [11-01-2018(online)].pdf | 2018-01-11 |
| 10 | 201811001316-COMPLETE SPECIFICATION [18-09-2019(online)].pdf | 2019-09-18 |
| 10 | 201811001316-DRAWINGS [11-01-2018(online)]_3.pdf | 2018-01-11 |
| 11 | 201811001316-CORRESPONDENCE [18-09-2019(online)].pdf | 2019-09-18 |
| 11 | 201811001316-DRAWINGS [11-01-2018(online)].pdf | 2018-01-11 |
| 12 | 201811001316-DECLARATION OF INVENTORSHIP (FORM 5) [11-01-2018(online)].pdf | 2018-01-11 |
| 12 | 201811001316-DRAWING [18-09-2019(online)].pdf | 2019-09-18 |
| 13 | 201811001316-COMPLETE SPECIFICATION [11-01-2018(online)]_6.pdf | 2018-01-11 |
| 13 | 201811001316-FER_SER_REPLY [18-09-2019(online)].pdf | 2019-09-18 |
| 14 | 201811001316-COMPLETE SPECIFICATION [11-01-2018(online)].pdf | 2018-01-11 |
| 14 | 201811001316-Correspondence-270819.pdf | 2019-08-29 |
| 15 | 201811001316-FORM-26 [30-01-2018(online)].pdf | 2018-01-30 |
| 15 | 201811001316-OTHERS-270819.pdf | 2019-08-29 |
| 16 | 201811001316-Power of Attorney-050218.pdf | 2018-02-09 |
| 16 | 201811001316-Power of Attorney-270819.pdf | 2019-08-29 |
| 17 | 201811001316-FER.pdf | 2019-08-22 |
| 17 | 201811001316-Correspondence-050218.pdf | 2018-02-09 |
| 18 | 201811001316-Proof of Right (MANDATORY) [21-08-2019(online)].pdf | 2019-08-21 |
| 18 | abstract.jpg | 2018-02-19 |
| 19 | 201811001316-8(i)-Substitution-Change Of Applicant - Form 6 [13-08-2019(online)].pdf | 2019-08-13 |
| 19 | 201811001316-Proof of Right (MANDATORY) [11-07-2018(online)].pdf | 2018-07-11 |
| 20 | 201811001316-ASSIGNMENT DOCUMENTS [13-08-2019(online)].pdf | 2019-08-13 |
| 20 | 201811001316-OTHERS-160718.pdf | 2018-07-17 |
| 21 | 201811001316-Correspondence-160718.pdf | 2018-07-17 |
| 21 | 201811001316-EVIDENCE FOR REGISTRATION UNDER SSI [13-08-2019(online)].pdf | 2019-08-13 |
| 22 | 201811001316-FORM 18A [01-08-2019(online)].pdf | 2019-08-01 |
| 22 | 201811001316-FORM FOR STARTUP [13-08-2019(online)].pdf | 2019-08-13 |
| 23 | 201811001316-FORM28 [13-08-2019(online)].pdf | 2019-08-13 |
| 23 | 201811001316-PA [13-08-2019(online)].pdf | 2019-08-13 |
| 24 | 201811001316-PA [13-08-2019(online)].pdf | 2019-08-13 |
| 24 | 201811001316-FORM28 [13-08-2019(online)].pdf | 2019-08-13 |
| 25 | 201811001316-FORM 18A [01-08-2019(online)].pdf | 2019-08-01 |
| 25 | 201811001316-FORM FOR STARTUP [13-08-2019(online)].pdf | 2019-08-13 |
| 26 | 201811001316-Correspondence-160718.pdf | 2018-07-17 |
| 26 | 201811001316-EVIDENCE FOR REGISTRATION UNDER SSI [13-08-2019(online)].pdf | 2019-08-13 |
| 27 | 201811001316-ASSIGNMENT DOCUMENTS [13-08-2019(online)].pdf | 2019-08-13 |
| 27 | 201811001316-OTHERS-160718.pdf | 2018-07-17 |
| 28 | 201811001316-8(i)-Substitution-Change Of Applicant - Form 6 [13-08-2019(online)].pdf | 2019-08-13 |
| 28 | 201811001316-Proof of Right (MANDATORY) [11-07-2018(online)].pdf | 2018-07-11 |
| 29 | 201811001316-Proof of Right (MANDATORY) [21-08-2019(online)].pdf | 2019-08-21 |
| 29 | abstract.jpg | 2018-02-19 |
| 30 | 201811001316-Correspondence-050218.pdf | 2018-02-09 |
| 30 | 201811001316-FER.pdf | 2019-08-22 |
| 31 | 201811001316-Power of Attorney-050218.pdf | 2018-02-09 |
| 31 | 201811001316-Power of Attorney-270819.pdf | 2019-08-29 |
| 32 | 201811001316-FORM-26 [30-01-2018(online)].pdf | 2018-01-30 |
| 32 | 201811001316-OTHERS-270819.pdf | 2019-08-29 |
| 33 | 201811001316-COMPLETE SPECIFICATION [11-01-2018(online)].pdf | 2018-01-11 |
| 33 | 201811001316-Correspondence-270819.pdf | 2019-08-29 |
| 34 | 201811001316-COMPLETE SPECIFICATION [11-01-2018(online)]_6.pdf | 2018-01-11 |
| 34 | 201811001316-FER_SER_REPLY [18-09-2019(online)].pdf | 2019-09-18 |
| 35 | 201811001316-DECLARATION OF INVENTORSHIP (FORM 5) [11-01-2018(online)].pdf | 2018-01-11 |
| 35 | 201811001316-DRAWING [18-09-2019(online)].pdf | 2019-09-18 |
| 36 | 201811001316-DRAWINGS [11-01-2018(online)].pdf | 2018-01-11 |
| 36 | 201811001316-CORRESPONDENCE [18-09-2019(online)].pdf | 2019-09-18 |
| 37 | 201811001316-COMPLETE SPECIFICATION [18-09-2019(online)].pdf | 2019-09-18 |
| 37 | 201811001316-DRAWINGS [11-01-2018(online)]_3.pdf | 2018-01-11 |
| 38 | 201811001316-CLAIMS [18-09-2019(online)].pdf | 2019-09-18 |
| 38 | 201811001316-EVIDENCE FOR REGISTRATION UNDER SSI [11-01-2018(online)].pdf | 2018-01-11 |
| 39 | 201811001316-ABSTRACT [18-09-2019(online)].pdf | 2019-09-18 |
| 39 | 201811001316-EVIDENCE FOR REGISTRATION UNDER SSI(FORM-28) [11-01-2018(online)].pdf | 2018-01-11 |
| 40 | 201811001316-EVIDENCE FOR REGISTRATION UNDER SSI(FORM-28) [11-01-2018(online)]_11.pdf | 2018-01-11 |
| 40 | 201811001316-HearingNoticeLetter-(DateOfHearing-24-10-2019).pdf | 2019-10-14 |
| 41 | 201811001316-FORM 1 [11-01-2018(online)].pdf | 2018-01-11 |
| 41 | 201811001316-FORM-26 [21-10-2019(online)].pdf | 2019-10-21 |
| 42 | 201811001316-Written submissions and relevant documents (MANDATORY) [07-11-2019(online)].pdf | 2019-11-07 |
| 42 | 201811001316-FORM FOR SMALL ENTITY(FORM-28) [11-01-2018(online)].pdf | 2018-01-11 |
| 43 | 201811001316-MARKED COPIES OF AMENDEMENTS [07-11-2019(online)].pdf | 2019-11-07 |
| 43 | 201811001316-FORM FOR SMALL ENTITY(FORM-28) [11-01-2018(online)]_2.pdf | 2018-01-11 |
| 44 | 201811001316-FORM FOR STARTUP [11-01-2018(online)].pdf | 2018-01-11 |
| 44 | 201811001316-FORM 13 [07-11-2019(online)].pdf | 2019-11-07 |
| 45 | 201811001316-STATEMENT OF UNDERTAKING (FORM 3) [11-01-2018(online)].pdf | 2018-01-11 |
| 45 | 201811001316-Annexure (Optional) [07-11-2019(online)].pdf | 2019-11-07 |
| 46 | 201811001316-STATEMENT OF UNDERTAKING (FORM 3) [11-01-2018(online)]_16.pdf | 2018-01-11 |
| 46 | 201811001316-AMMENDED DOCUMENTS [07-11-2019(online)].pdf | 2019-11-07 |
| 1 | searchstrategy_21-08-2019.pdf |