Claims:1. A processor implemented method (200) comprising:
obtaining, by an input module, one or more aerial images pertaining to a truss of interest (202);
performing segmentation of either the one or more aerial images or a 3D reconstruction thereof by:
detecting, by a line detector, maximal number of linear edges of maximal number of beams in the truss of interest, from the one or more aerial images or the 3D reconstruction thereof (204);
interpolating and clustering, by a medial line generator, the detected linear edges of the beams to maximize length thereof and form long medial lines representing the beams (206);
detecting, by a canonical polygon detector, canonical polygons formed by joining or crossing of the beams (208); and
classifying, by a truss classifier, regions having clusters of abutting canonical polygons from the detected canonical polygons, as regions having true positive or false positive truss elements based on shape density of the abutting canonical polygons (210).

2. The processor implemented method of claim 1, wherein the step of detecting maximal number of linear edges of maximal number of beams is based on Line Segment Detector (LSD) technique.

3. The processor implemented method of claim 1, wherein the step of interpolating and clustering the detected linear edges of detected beams comprises:
interpolating, with hypothetical lines, short gaps between the detected linear edges having identical orientation within a pre-defined difference in orientation; and
clustering the detected linear edges and the hypothetical lines at different locations on each of the beams based on the orientation, sideways intercept and frontal intercept, wherein the sideways intercept is a minimum distance between detected linear edges or the hypothetical lines and frontal intercept is the minimum distance between endpoints of the detected linear edges or the hypothetical lines.

4. The processor implemented method of claim 3, wherein the pre-defined difference in orientation is 2 degrees.

5. The processor implemented method of claim 1, wherein the step of detecting canonical polygons comprises:
arranging polygonal shapes formed by joining or crossing of the beams into a geometric lattice;
identifying centroid and vertex farthest from the centroid of each of the polygonal shapes;
iteratively creating an annular region by considering next bigger circle with one unit more radius than a circle in a previous iteration, both circles having center as centroid of each of the polygonal shapes, and performing iterations till the radius equals the distance of the centroid from the farthest vertex;
locating points within each of the polygonal shapes by iteratively testing subset of the points within an annulus created in a current iteration, identifying an interior of each of the polygonal shapes;
identifying the polygonal shapes having all points in the interior thereof as part of the interior of one or more of other polygonal shapes based on the located points on the annulus; and
detecting the canonical polygons from the polygonal shapes, the canonical polygons being level 1 polygons in the geometric lattice that do not subsume any other polygon.

6. The processor implemented method of claim 1, wherein the step of classifying the canonical polygons comprises:
computing the shape density as a ratio of (i) number of abutting canonical polygons in the regions having clusters of the canonical polygons and (ii) number of pixels in the interior of the region; and
classifying the region having maximum value of the shape density as the region having the true positive truss elements.

7. The processor implemented method of claim 1 further comprising:
superimposing and comparing the true positive truss elements with true positive truss elements obtained by performing segmentation of images of the truss of interest captured previously; and
classifying one or more of the true positive truss elements as damaged or healthy based on the comparison.

8. A system (100) comprising:
one or more data storage devices (102) operatively coupled to one or more hardware processors (104) and configured to store instructions configured for execution by the one or more hardware processors to:
obtain one or more aerial images pertaining to a truss of interest;
perform segmentation of either the one or more aerial images or a 3D reconstruction thereof by:
detecting maximal number of linear edges of maximal number of beams in the truss of interest, from the one or more aerial images or the 3D reconstruction thereof;
interpolating and clustering the detected linear edges of the beams to maximize length thereof and form long medial lines representing the beams;
detecting canonical polygons formed by joining or crossing of the beams; and
classifying regions having clusters of abutting canonical polygons from the detected canonical polygons, as regions having true positive or false positive truss elements based on shape density of the abutting canonical polygons.

9. The system of claim 8, wherein the one or more hardware processors are further configured to detect maximal number of linear edges of maximal number of beams based on Line Segment Detector (LSD) technique.

10. The system of claim 8, wherein the one or more hardware processors are further configured to perform interpolating and clustering of the detected linear edges by:
interpolating, with hypothetical lines, short gaps between the detected linear edges having identical orientation within a pre-defined difference in orientation; and
clustering the detected linear edges and the hypothetical lines at different locations on each of the beams based on the orientation, sideways intercept and frontal intercept, wherein the sideways intercept is a minimum distance between detected linear edges or the hypothetical lines and frontal intercept is the minimum distance between endpoints of the detected linear edges or the hypothetical lines.

11. The system of claim 10, wherein the pre-defined difference in orientation is 2 degrees.

12. The system of claim 8, wherein the one or more hardware processors are further configured to perform detecting of canonical polygons by:
arranging polygonal shapes formed by joining or crossing of the beams into a geometric lattice;
identifying centroid and vertex farthest from the centroid of each of the polygonal shapes;
iteratively creating an annular region by considering next bigger circle with one unit more radius than a circle in a previous iteration, both circles having center as centroid of each of the polygonal shapes, and performing iterations till the radius equals the distance of the centroid from the farthest vertex;
locating points within each of the polygonal shapes by iteratively testing subset of the points within an annulus created in a current iteration, identifying an interior of each of the polygonal shapes;
identifying the polygonal shapes having all points in the interior thereof as part of the interior of one or more of other polygonal shapes based on the located points on the annulus; and
detecting the canonical polygons from the polygonal shapes, the canonical polygons being level 1 polygons in the geometric lattice that do not subsume any other polygon.

13. The system of claim 8, wherein the one or more hardware processors are further configured to perform classifying the canonical polygons by:
computing the shape density as a ratio of (i) number of abutting canonical polygons in the regions having clusters of the canonical polygons and (ii) number of pixels in the interior of the region; and
classifying the region having maximum value of the shape density as the region having the true positive truss elements.

14. The system of claim 8, wherein the one or more hardware processors are further configured to:
superimpose and compare the true positive truss elements with true positive truss elements obtained by performing segmentation of images of the truss of interest captured previously; and
classify one or more of the true positive truss elements as damaged or healthy based on the comparison. , Description:FORM 2

THE PATENTS ACT, 1970
(39 of 1970)
&
THE PATENT RULES, 2003

COMPLETE SPECIFICATION
(See Section 10 and Rule 13)

Title of invention:
DETECTION AND SEGMENTATION OF OUTDOOR TRUSSES
USING AERIAL IMAGERY

Applicant:
Tata Consultancy Services Limited
A company Incorporated in India under the Companies Act, 1956
Having address:
Nirmal Building, 9th floor,
Nariman point, Mumbai 400021,
Maharashtra, India

The following specification particularly describes the embodiments and the manner in which it is to be performed.

TECHNICAL FIELD
[001] The embodiments herein generally relate to image analysis, and more particularly to systems and methods for detection and segmentation of outdoor trusses using aerial imagery.

BACKGROUND
[002] Remote video surveillance of large outdoor systems for structural health monitoring and inspection is now-a-days gaining wide popularity. Such vast systems include national infrastructural systems like bridges, transmission towers, and the like. Many of these systems are designed as truss structures owing to well-known mechanical reasons. Recent popular trend towards periodic health inspection of these truss structures is via acquisition of images or videos and their subsequent analysis for any possible faults. Automated analysis for fault detection has been identified as the aim of such solutions. Especially in aerial images / videos of such systems acquired via drones, self-occlusion by truss components, illumination variations arising from time-of-day, and shadow effects altogether make the problem of segmenting such images extremely challenging and accordingly solutions known in the art are unable to effectively address all of these issues as desired. Furthermore, known solutions are extremely complex and may not provide desired segmentation in near-real-time.

SUMMARY
[003] Embodiments of the present disclosure present technological improvements as solutions to one or more of the above-mentioned technical problems recognized by the inventors in conventional systems.
[004] In an aspect, there is provided a processor implemented method comprising: obtaining, by an input module, one or more aerial images pertaining to a truss of interest; performing segmentation of either the one or more aerial images or a 3D reconstruction thereof by: detecting, by a line detector, maximal number of linear edges of maximal number of beams in the truss, from the one or more aerial images or the 3D reconstruction thereof; interpolating and clustering, by a medial line generator, the detected linear edges of the beams to maximize length thereof and form long medial lines representing the beams; detecting, by a canonical polygon detector, canonical polygons formed by joining or crossing of the beams; and classifying, by a truss classifier, regions having clusters of abutting canonical polygons from the detected canonical polygons, as regions having true positive or false positive truss elements based on shape density of the abutting canonical polygons.
[005] In another aspect, there is provided a system comprising: one or more data storage devices operatively coupled to the one or more processors and configured to store instructions configured for execution by the one or more processors to: obtain one or more aerial images pertaining to a truss of interest; perform segmentation of either the one or more aerial images or a 3D reconstruction thereof by: detecting maximal number of linear edges of maximal number of beams in the truss, from the one or more aerial images or the 3D reconstruction thereof; interpolating and clustering the detected linear edges of the beams to maximize length thereof and form long medial lines representing the beams; detecting canonical polygons formed by joining or crossing of the beams; and classifying regions having clusters of abutting canonical polygons from the detected canonical polygons, as regions having true positive or false positive truss elements based on shape density of the abutting canonical polygons.
[006] In yet another aspect, there is provided a computer program product comprising a non-transitory computer readable medium having a computer readable program embodied therein, wherein the computer readable program, when executed on a computing device, causes the computing device to: obtain one or more aerial images pertaining to a truss of interest; perform segmentation of either the one or more aerial images or a 3D reconstruction thereof by: detecting maximal number of linear edges of maximal number of beams in the truss, from the one or more aerial images or the 3D reconstruction thereof; interpolating and clustering the detected linear edges of the beams to maximize length thereof and form long medial lines representing the beams; detecting canonical polygons formed by joining or crossing of the beams; and classifying regions having clusters of abutting canonical polygons from the detected canonical polygons, as regions having true positive or false positive truss elements based on shape density of the abutting canonical polygons.
[007] In an embodiment of the present disclosure, the one or more hardware processors are further configured to detect maximal number of linear edges of maximal number of beams based on Line Segment Detector (LSD) technique.
[008] In an embodiment of the present disclosure, the one or more hardware processors are further configured to perform interpolating and clustering of the detected linear edges by: interpolating, with hypothetical lines, short gaps between the detected linear edges having identical orientation within a pre-defined difference in orientation; and clustering the detected linear edges and the hypothetical lines at different locations on each of the beams based on the orientation, sideways intercept and frontal intercept, wherein the sideways intercept is a minimum distance between detected linear edges or the hypothetical lines and frontal intercept is the minimum distance between endpoints of the detected linear edges or the hypothetical lines.
[009] In an embodiment of the present disclosure, the pre-defined difference in orientation is 2 degrees.
[010] In an embodiment of the present disclosure, the one or more hardware processors are further configured to perform detecting of canonical polygons by: arranging the polygonal shapes formed by joining or crossing of the beams into a geometric lattice; identifying centroid and vertex farthest from the centroid of each of the polygonal shapes; iteratively creating an annular region by considering next bigger circle with one unit more radius than a circle in a previous iteration, both circles having center as centroid of each of the polygonal shapes, and performing iterations till the radius equals the distance of the centroid from the farthest vertex; locating points within each of the polygonal shapes by iteratively testing subset of the points within an annulus created in a current iteration, identifying an interior of each of the polygonal shapes; identifying the polygonal shapes having all points in the interior thereof as part of the interior of one or more of other polygonal shapes based on the located points on the annulus; and detecting the canonical polygons from the polygonal shapes, the canonical polygons being level 1 polygons in the geometric lattice that do not subsume any other polygon.
[011] In an embodiment of the present disclosure, the one or more hardware processors are further configured to perform classifying the canonical polygons by: computing the shape density as a ratio of (i) number of abutting canonical polygons in the regions having clusters of the canonical polygons and (ii) number of pixels in the interior of the region; and classifying the region having maximum value of the shape density as the region having the true positive truss elements.
[012] In an embodiment of the present disclosure, the one or more hardware processors are further configured to superimpose and compare the true positive truss elements with true positive truss elements obtained by performing segmentation of images of the truss of interest captured previously; and classify one or more of the true positive truss elements as damaged or healthy based on the comparison
[013] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the embodiments of the present disclosure, as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS
[014] The embodiments herein will be better understood from the following detailed description with reference to the drawings, in which:
[015] FIG.1 illustrates an exemplary block diagram of a system for detection and segmentation of outdoor trusses using aerial imagery, in accordance with an embodiment of the present disclosure;
[016] FIG.2 illustrates an exemplary flow diagram of a method for detection and segmentation of outdoor trusses using aerial imagery, in accordance with an embodiment of the present disclosure;
[017] FIG.3A and FIG.3B illustrate an output of a line detector and a medial line generator respectively, on an aerial image of an electrical pylon, in accordance with an embodiment of the present disclosure;
[018] FIG.4A and FIG.4B illustrate an output of a line detector and a medial line generator respectively, on an aerial image of background vegetation near an electrical pylon, in accordance with an embodiment of the present disclosure;
[019] FIG.5A and FIG.5B illustrate output of a canonical polygon detector, on an aerial image of an electrical pylon and an aerial image of background vegetation near an electrical pylon respectively, in accordance with an embodiment of the present disclosure;
[020] FIG.6 illustrates a geometric lattice of polygon subsumptions, in accordance with an embodiment of the present disclosure;
[021] FIG.7A and FIG.7B illustrate an output of a truss classifier on an on an aerial image of an electrical pylon and an aerial image of background vegetation near an electrical pylon respectively, in accordance with an embodiment of the present disclosure; and
[022] FIG.8 illustrates an aerial image of an exemplary damaged electrical pylon.
[023] It should be appreciated by those skilled in the art that any block diagram herein represent conceptual views of illustrative systems embodying the principles of the present subject matter. Similarly, it will be appreciated that any flow charts, flow diagrams, state transition diagrams, pseudo code, and the like represent various processes which may be substantially represented in computer readable medium and so executed by a computing device or processor, whether or not such computing device or processor is explicitly shown.

DETAILED DESCRIPTION
[024] The embodiments herein and the various features and advantageous details thereof are explained more fully with reference to the non-limiting embodiments that are illustrated in the accompanying drawings and detailed in the following description. The examples used herein are intended merely to facilitate an understanding of ways in which the embodiments herein may be practiced and to further enable those of skill in the art to practice the embodiments herein. Accordingly, the examples should not be construed as limiting the scope of the embodiments herein.
[025] The words "comprising," "having," "containing," and "including," and other forms thereof, are intended to be equivalent in meaning and be open ended in that an item or items following any one of these words is not meant to be an exhaustive listing of such item or items, or meant to be limited to only the listed item or items.
[026] It must also be noted that as used herein and in the appended claims, the singular forms "a," "an," and "the" include plural references unless the context clearly dictates otherwise. Although any systems and methods similar or equivalent to those described herein can be used in the practice or testing of embodiments of the present disclosure, the preferred, systems and methods are now described.
[027] Some embodiments of this disclosure, illustrating all its features, will now be discussed in detail. The disclosed embodiments are merely exemplary of the disclosure, which may be embodied in various forms.
[028] Before setting forth the detailed explanation, it is noted that all of the discussion below, regardless of the particular implementation being described, is exemplary in nature, rather than limiting.
[029] Classical works on image segmentation assume that foreground object of interest have no interstitial spaces between members. Systems and methods of the present disclosure aim to cater particularly to such a class of objects, for instance, trusses. The expression “truss” in the context of the present disclosure refers to electricity pylons, bridges, scaffoldings and such other structures that employ a truss structure and have interstitial spaces between truss elements. In aerial outdoor imagery, background is complex since structures are usually seen along with vegetation, buildings and others outdoor objects. Further, some of these objects can occlude the object of interest. Shadow effects caused by the structure itself, shadow effects caused by other objects and illumination variations make segmentation of images challenging. The present disclosure facilitates segmentation of such images in near real-time manner.
[030] Referring now to the drawings, and more particularly to FIGS. 1 through 8, where similar reference characters denote corresponding features consistently throughout the figures, there are shown preferred embodiments and these embodiments are described in the context of the following exemplary system and method.
[031] FIG.1 illustrates an exemplary block diagram of a system 100 for detection and segmentation of outdoor trusses using aerial imagery, in accordance with an embodiment of the present disclosure. In an embodiment, the system 100 includes one or more processors 104, communication interface device(s) or input/output (I/O) interface(s) 106, and one or more data storage devices or memory 102 operatively coupled to the one or more processors 104. The one or more processors 104 that are hardware processors can be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, graphics controllers, logic circuitries, and/or any devices that manipulate signals based on operational instructions. Among other capabilities, the processor(s) is configured to fetch and execute computer-readable instructions stored in the memory. In an embodiment, the system 100 can be implemented in a variety of computing systems, such as laptop computers, notebooks, hand-held devices, workstations, mainframe computers, servers, a network cloud and the like.
[032] The I/O interface device(s) 106 can include a variety of software and hardware interfaces, for example, a web interface, a graphical user interface, and the like and can facilitate multiple communications within a wide variety of networks N/W and protocol types, including wired networks, for example, LAN, cable, etc., and wireless networks, such as WLAN, cellular, or satellite. In an embodiment, the I/O interface device(s) can include one or more ports for connecting a number of devices to one another or to another server.
[033] The memory 102 may include any computer-readable medium known in the art including, for example, volatile memory, such as static random access memory (SRAM) and dynamic random access memory (DRAM), and/or non-volatile memory, such as read only memory (ROM), erasable programmable ROM, flash memories, hard disks, optical disks, and magnetic tapes. In an embodiment, one or more modules (102A through 102E) of the system 100 can be stored in the memory 102.
[034] FIG.2 illustrates an exemplary flow diagram of a method 200 and FIG.3 illustrates an exemplary flow chart for the method 200 for detection and segmentation of outdoor trusses using aerial imagery, in accordance with an embodiment of the present disclosure. In an embodiment, the system 100 comprises one or more data storage devices or memory 102 operatively coupled to the one or more processors 104 and is configured to store instructions configured for execution of steps of the method 200 by the one or more processors 104.
[035] The method 200 of the present disclosure is a four stage process for detection and segmentation of truss like structures in aerial outdoor images / videos, typically acquired using Unmanned Aerial Vehicles (UAV) or drones (low-altitude imagery). The method 200 relies on polygonal density or shape density to segment truss like structures. The polygons are grown by interpolating and extrapolating lines detected from the images. Since there are lots of segmentation complexities like illumination variations, complex backgrounds and occlusion, traditional line detection does not provide perfect output, leading to missed detection of some segments of these lines. This eventually affects the segmentation result. The method 200 employs a scale-independent, pyramid-based approach for robust line detection. Further computations involved in the method 200 are carried out over collections of line, which considerably reduces the complexity.
[036] In an embodiment, at step 202, an input module 102A may be configured to obtain one or more aerial images pertaining to a truss of interest. In an embodiment, the one or more aerial images may be still images captured in the form of an image directly or may be extracted from captured videos.
[037] In an embodiment, the system 100 may perform segmentation on either the one or more aerial images or on a 3D reconstruction of the images.
[038] Edges of beams or girders of trusses are lines that intersect or join each other in both 2D projection and 3D. Such crisscrossing lines, at various angles comprise many abutting polygon formations. As is known for wide-angle aerial frames, a region (2D) or a volume (3D) of any truss structure has very high number of polygons per unit area or volume. Hence in the present disclosure, 2D or 3D density of such polygonal shapes are estimated to distinctly classify such regions. Since trusses have, having extensive shape concavities or interstitial spaces, estimating shape density provides a reliable classification. In accordance with the present disclosure, only the linear edges representative of beams are used to detect polygonal shapes, it is imperative that maximal number of beams (true positive in terms of number of beams) are detected along with maximal area/volume of each beam region. Else, certain intersections of lines and hence some count of polygons may be missed. Accordingly, in an embodiment, at step 204, a line detector 102B may be configured to detect maximal number of linear edges of maximal number of beams in the truss, from the one or more aerial images or the 3D reconstruction of the aerial images.
[039] Since the images are outdoor images, illumination levels can differ and they cannot be controlled, being dependent on time of day which impacts orientation of the sun (photometric effect). Certain objects such as trees can cast shadows on the object of interest (truss). Due to low illumination in such regions, edge gradients of part of object that is overlaid with shadow are weak, leading to presence of diffuse edge segments along the object boundary. Again, background of the image may be quite heterogeneous. In 2D images, in certain parts, the background luminance of certain background artifact closely matches foreground luminance. This not only manifests in diffuse edge segments, but if region methods are considered, it also results in contour leaks. In accordance with an embodiment of the present disclosure, pyramid method is employed to robustly detect edges by reducing the variance of edge gradients using smoothening at multiple levels.
[040] In an embodiment, the step of detecting maximal number of linear edges of maximal number of beams is based on Line Segment Detector (LSD) technique. LSD starts with a Gaussian pyramid generated from an original image, which is down sampled N-1 times, and blurred N times, to obtain N layers. Then, from each layer in the pyramid, lines are extracted using LSD algorithm. FIG.3A illustrates an output of the line detector 102B on an aerial image of an electrical pylon while FIG.4A illustrates an output of the line detector 102B on an aerial image of background vegetation near an electrical pylon, in accordance with an embodiment of the present disclosure.
[041] It may be noted from FIG.3A and FIG. 4A that broken linear edges are detected in the output of the linear line detector 102B. Such breakage of edges occurs in localities of extreme diffusion, and is a known problem in outdoor imagery. Active contour methods known in the art pose a challenge here because the background clutter is very fine-grained (e.g. a leaf on a tree whose projection is in quite vicinity of a beam projection), and initial contour for such iterative active contour method cannot be made arbitrarily close to the desired contour to avoid presence of other spurious edges within the initializer. Since it is not practically possible to detect entire edge length of a beam as an ideal line, in an embodiment, at step 206, a medial line generator 102C may be configured to interpolate and cluster the detected linear edges of the beams to maximize length thereof and form long medial lines representing the beams.
[042] In an embodiment, to maximize length of the detected linear edges, short gaps between the detected linear edges having identical orientation within a pre-defined difference in orientation are interpolated with hypothetical lines. In an embodiment, the pre-defined difference in orientation is 2 degrees to account for image noise. Many of the stronger beams of a truss are actually two flat beams welded and abutted along the major length (L-angle beam). Hence ideally, up to 3 edges are detected in the projection of such beams. Practically, for each edge, different edgelets are obtained at different locations on the same beam. It may be noted that if the edgelets are combined into a single longer medial line by interpolating within a bounding box of a group of adjacent edgelets, then the number of polygon formations that these beams participate in, does not change. Accordingly, in accordance with the present disclosure, the detected linear edges and the hypothetical lines at different locations on each of the beams are clustered based on the orientation, sideways intercept and frontal intercept. In accordance with the present disclosure, the sideways intercept is a minimum distance between detected linear edges or the hypothetical lines and the frontal intercept is the minimum distance between endpoints of the detected linear edges or the hypothetical lines. FIG.3B illustrates an output of the medial line generator 102C on an aerial image of an electrical pylon while FIG.4B illustrates an output of the medial line generator 102C on an aerial image of background vegetation near an electrical pylon, in accordance with an embodiment of the present disclosure.
[043] It may be seen from FIG.3B and FIG.4B that that the medial lines that represent various beams of the truss either join or cross each other at various orientations. A join, when projected, implies a point of concurrence of two lines. A crossing, when projected, implies a point of intersection of two lines. A sample set of three or more beams, every two of which concur or intersect in the projected image, forms an irregular polygon. Since there are many beams, sometimes running into few hundreds, there exist many subsets of three of more beams having such nature in the projected image. Thus a truss region is typified by existence of high degree of polygons in its interior.
[044] In an embodiment, at step 208, a canonical polygon detector 102D may be configured to detect canonical polygons formed by joining or crossing of the beams. FIG.5A and FIG.5B illustrate output of a canonical polygon detector, on an aerial image of an electrical pylon and an aerial image of background vegetation near an electrical pylon respectively, in accordance with an embodiment of the present disclosure. As can be seen, all the polygonal shapes may be organized into a mathematical structure known as geometric lattice. It may be noted that the number of polygons are always much greater than the number of canonical polygons. Also by definition of set partition lattice, the lattice partial order is based on subsumption relation, i.e. two polygons are connected if one polygon subsumes the other. FIG.6 illustrates a geometric lattice of polygon subsumptions, in accordance with an embodiment of the present disclosure. As seen in FIG.6, level-1 polygons, in a truss/set of polygons, are those (small) polygons, which do not subsume any other polygon and are referred to as the canonical polygons. Level 2 represents union of level 1 polygons. Level 0 and Level 3 represent infimum and supremum in the exemplary geometric lattice of FIG.6.
[045] In an embodiment, the step of detecting canonical polygons comprises firstly arranging the polygonal shapes formed by joining or crossing of the beams into a geometric lattice as illustrated in FIG.6. Then centroid and vertex farthest from the centroid of each of the polygonal shapes are identified. An annular region is iteratively created by considering next bigger circle with one unit more radius than a circle in previous iteration, both circles having center as centroid of each of the polygonal shapes, and the iterations are performed till the radius equals the distance of the centroid from the farthest vertex. To establish subsumption, the property that all the points in the interior of one of the polygons must be part of interior of the other polygon is used. In accordance with the present disclosure, points within each of the polygonal shapes are located by iteratively testing subset of the points within an annulus created in a current iteration for identifying an interior of each of the polygonal shapes. Subsequently, the polygonal shapes having all points in the interior thereof are identified as part of the interior of one or more of other polygonal shapes based on the located points on the annulus and polygons in the geometric lattice that do not subsume any other polygon are detected as canonical polygons at level 1 of the geometric lattice.
[046] In accordance with the present disclosure, the region corresponding to a truss is distinctively represented by a high presence of polygonal shapes that abut each other. Further, a truss differs from a similar rectilinear structure (false positive) of a scaffolding in the sense that a truss is designed to bear much more mechanical stress along many directions, and hence in the same volume, there are significantly many more beams that are riveted together to distribute high stresses, than in the scaffolding. Therefore, in accordance with the present disclosure, region density of polygons in a truss region is estimated using number of canonical polygons. In an embodiment, at step 210, a truss classifier 102E may be configured to classify regions having clusters of abutting canonical polygons from the detected canonical polygons at step 208, as regions having true positive or false positive truss elements based on shape density of the abutting canonical polygons.
[047] In an embodiment, the step of classifying the canonical polygons comprises: computing the shape density as a ratio of (i) number of abutting canonical polygons in the regions having clusters of the canonical polygons and (ii) number of pixels in the interior of the region; and classifying the region having maximum value of the shape density as the region having the true positive truss elements.
[048] FIG.7A and FIG.7B illustrate an output of a truss classifier on an on an aerial image of an electrical pylon and an aerial image of background vegetation near an electrical pylon respectively, in accordance with an embodiment of the present disclosure.
[049] In an embodiment, the method 200 of the present disclosure may be represented as given herein below.
Detection of truss structures in aerial images
Fast line segmentation based on pyramid method to detect maximal edges of maximal number of beams
Representation of beams by medial lines
Interpolation of gaps between detected line segments
Clustering of line segment of each beam face around medial line
Detection of Polygons
Enumeration of all polygons based on line intersection
Shortlisting of canonical polygons
Shape density based classification
Estimation of shape density using canonical polygons
Density based removal of false positives
[050] The biggest damage to truss-like systems arise from vibration and other dynamic excitations due to storms, ice- shedding etc. FIG.8 illustrates an aerial image of an exemplary damaged electrical pylon. Such damage manifest because these systems age under open environmental conditions, leading to mechanical issues such as fracture of panel joints, brace breakage, and the like. Ultimate effects include pull-down, collapse and bending/breakage of a part of tower such as cross-arms as illustrated in FIG.8. Aerial surveys are conducted not only to detect and estimate degree of such damages but also to monitor for building-up conditions such as cracks appearing in the joints, missing rivet bolts etc., which may eventually lead to mechanical faults and disruption in service. In an embodiment, the system 100 may be configured to detect such damages by superimposing and comparing the true positive truss elements with true positive truss elements obtained by performing segmentation of images of the truss of interest captured previously and then classifying one or more of the true positive truss elements as damaged or healthy based on the comparison. Superimposing and comparing true positive truss elements obtained in a current image with true positive truss elements obtained by segmentation of images captured say, a few months ago ensures that occlusions such as growth of a tree (post capturing of previous image) and such other artifacts in the background are eliminated during segmentation and template matching provides a reliable conclusion on possible damage to the structure, if any.
[051] As seen from the description herein above, the present disclosure caters particularly to objects having interstitial spaces between members by providing systems and methods for detection and segmentation of images that are not effected by well-known challenges pertaining to illumination, shadows, complex background, and the like. Supervised segmentation methods known in the art is also dependent on a huge learning overhead in the form of labeled training data and involves high computational complexity. Besides reducing complexity, the method of the present disclosure provides unsupervised segmentation with enhanced reliability for detection of damages, if any, in critical infrastructure that employ such structures and need to be monitored for reliability and efficiency of operation. The present disclosure represents a truss by a region marked by high presence of overlapping planar polygons. This is possible because a standard sideways aerial inspection truly captures all the beam joints. The method of the present disclosure enumerates points forming the interior of polygonal regions based on a fast method of growing such set of points, starting from a seed point located at the centroid of the polygon thereby obviating known problems of polygon filling in computer graphics. The method of the present disclosure employs shape density to classify and differentiate truss like structures from all other false positives. It may be noted by a person skilled in the art that the method of the present disclosure may be employed not only on 2D images but also for segmentation of 3D models reconstructed from 2D images.
[052] The written description describes the subject matter herein to enable any person skilled in the art to make and use the embodiments of the present disclosure. The scope of the subject matter embodiments defined here may include other modifications that occur to those skilled in the art. Such other modifications are intended to be within the scope if they have similar elements that do not differ from the literal language of the claims or if they include equivalent elements with insubstantial differences from the literal language.
[053] The scope of the subject matter embodiments defined here may include other modifications that occur to those skilled in the art. Such other modifications are intended to be within the scope if they have similar elements that do not differ from the literal language of the claims or if they include equivalent elements with insubstantial differences from the literal language.
[054] It is, however to be understood that the scope of the protection is extended to such a program and in addition to a computer-readable means having a message therein; such computer-readable storage means contain program-code means for implementation of one or more steps of the method, when the program runs on a server or mobile device or any suitable programmable device. The hardware device can be any kind of device which can be programmed including e.g. any kind of computer like a server or a personal computer, or the like, or any combination thereof. The device may also include means which could be e.g. hardware means like e.g. an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a combination of hardware and software means, e.g. an ASIC and an FPGA, or at least one microprocessor and at least one memory with software modules located therein. Thus, the means can include both hardware means and software means. The method embodiments described herein could be implemented in hardware and software. The device may also include software means. Alternatively, the embodiments of the present disclosure may be implemented on different hardware devices, e.g. using a plurality of CPUs.
[055] The embodiments herein can comprise hardware and software elements. The embodiments that are implemented in software include but are not limited to, firmware, resident software, microcode, etc. The functions performed by various modules comprising the system of the present disclosure and described herein may be implemented in other modules or combinations of other modules. For the purposes of this description, a computer-usable or computer readable medium can be any apparatus that can comprise, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The various modules described herein may be implemented as software and/or hardware modules and may be stored in any type of non-transitory computer readable medium or other storage device. Some non-limiting examples of non-transitory computer-readable media include CDs, DVDs, BLU-RAY, flash memory, and hard disk drives.
[056] Further, although process steps, method steps, techniques or the like may be described in a sequential order, such processes, methods and techniques may be configured to work in alternate orders. In other words, any sequence or order of steps that may be described does not necessarily indicate a requirement that the steps be performed in that order. The steps of processes described herein may be performed in any order practical. Further, some steps may be performed simultaneously.
[057] The preceding description has been presented with reference to various embodiments. Persons having ordinary skill in the art and technology to which this application pertains will appreciate that alterations and changes in the described structures and methods of operation can be practiced without meaningfully departing from the principle, spirit and scope.