Abstract: Embodiments of the present disclosure provide a method and system for enhanced image feature extraction. Unlike the conventional Speeded-up Robust Features (SURF™) the method disclosed identifies higher number of Points of Interest (POIs) in an image without compromising on time required by (SURF™). This is achieved by replacing the computationally heavy Hessian matrix (a second order derivative matrix) used by (SURF™) during Response map generation (RMG) process by a light weight normal Diagonally Dominant Matrix (DDM), which is a unit matrix of order 3x3. Further, unlike the well-known Scale Invariant Features Transform (SIFT™) standard technique, which is computationally very heavy and cannot be implemented on low power devices, the method disclosed herein consumes less time, and is computationally light weight and thus easy to implement on low power devices with good feature extraction capability. [To be published with 1B]
FORM 2
THE PATENTS ACT, 1970
(39 of 1970)
&
THE PATENT RULES, 2003
COMPLETE SPECIFICATION (See Section 10 and Rule 13)
Title of invention: METHOD AND SYSTEM FOR ENHANCED IMAGE FEATURE
EXTRACTION
Applicant
Tata Consultancy Services Limited
A company Incorporated in India under the Companies Act, 1956
Having address:
Nirmal Building, 9th floor,
Nariman point, Mumbai 400021,
Maharashtra, India
Preamble to the description
The following specification particularly describes the invention and the manner in which it is to be performed.
TECHNICAL FIELD [001] The embodiments herein generally relate to image processing and, more particularly, to a method and system for enhanced image feature extraction.
BACKGROUND
[002] Feature extraction is a critical step in image processing applications, where object recognition is one of the important process. Feature extraction refers to a process of extracting valuable information referred to as features from an input image. The extracted features carry important and unique attributes of the image, any object can be detected based on its features in different images. This extraction should ideally be feasible when the image shows the object with different transformations, such as mainly scale and rotation, or when parts of the object are occluded. Conventional well-known techniques for feature extraction, which are industry standards, such as Speeded-up Robust Features (SURF™) and Scale Invariant Features Transform (SIFT™) have been used. However, each technique has its own limitations. The SIFT™ is mathematically complicated and computationally heavy. It is based on Histogram of Gradients. That is, the gradients of each pixel in the patch need to be computed, which is time consuming and computationally intensive. Thus, SIFT™ is not effective for low powered devices. The SURF™ consumes lesser computational resources, hence a choice for low powered devices, but provides lower number of POIs. As can be understood, as the number of POIs increase the feature extraction is stronger, providing more accuracy in the end applications of image processing. Furthermore, SURF™ is not stable to rotation and performance further drops with variation in image illumination. Thus, accuracy of POIs obtained is low. Further, SURF™ uses a second ordered derivative matrix known as the Hessian matrix for the POIs. Though it performs computationally better than SIFT™ but still requires second order computations and consumes computational resources.
SUMMARY
[003] Embodiments of the present disclosure present technological improvements as solutions to one or more of the above-mentioned technical problems recognized by the inventors in conventional systems.
[004] For example, in one embodiment, a method for enhanced image feature extraction is provided. The method includes receiving and preprocessing an image, wherein the preprocessing comprises converting the image into a grayscale image and generating an integral image from the grayscale image.
[005] Further the method includes applying a Response Map Generation (RMG) process on the integral image to construct a scale space of the integral image comprising Points of Interest (POIs) across different scales of the integral image, wherein the RMG process comprises: performing column wise and row wise first order differentiation of the integral image to obtain a first order derivative matrix of the integral image; generating an offset matrix by multiplying the first order derivative matrix by a normal Diagonally Dominant Matrix (DDM), wherein the normal DDM is a lightweight unit matrix of order 3x3; comparing a value of each of a plurality of elements of the offset matrix with an offset threshold; and identifying one or more elements among the plurality of elements, having the value above the offset threshold, as points of interest (POIs).
[006] Further the method includes assigning a reproducible orientation to each of the POIs to provide rotation invariance during a feature descriptor generation for each of the POIs; and generating the feature descriptor for each of the POIs, wherein the feature descriptor describes a pixel intensity distribution based on a scale independent neighborhood, and, wherein the reproducible orientation enables rotation independent feature descriptor generation.
[007] In another aspect, a system for enhanced image feature extraction is provided. The system comprises a memory storing instructions; one or more Input/Output (I/O) interfaces; and one or more hardware processors coupled to the memory via the one or more I/O interfaces, wherein the one or more hardware processors are configured by the instructions to receive and preprocess an image,
wherein the preprocessing comprises converting the image into a grayscale image and generating an integral image from the grayscale image.
[008] Further the system is configured to apply a Response Map Generation (RMG) process on the integral image to construct a scale space of the integral image comprising Points of Interest (POIs) across different scales of the integral image, wherein the RMG process comprises: performing column wise and row wise first order differentiation of the integral image to obtain a first order derivative matrix of the integral image; generating an offset matrix by multiplying the first order derivative matrix by a normal Diagonally Dominant Matrix (DDM), wherein the normal DDM is a lightweight unit matrix of order 3x3; comparing a value of each of a plurality of elements of the offset matrix with an offset threshold; and identifying one or more elements among the plurality of elements, having the value above the offset threshold, as points of interest (POIs).
[009] Further the system is configured to assign a reproducible orientation to each of the POIs to provide rotation invariance during a feature descriptor generation for each of the POIs; and generate the feature descriptor for each of the POIs, wherein the feature descriptor describes a pixel intensity distribution based on a scale independent neighborhood, and, wherein the reproducible orientation enables rotation independent feature descriptor generation.
[0010] In yet another aspect, there are provided one or more non-transitory machine-readable information storage mediums comprising one or more instructions, which when executed by one or more hardware processors causes a method for enhanced image feature extraction is provided. The method includes receiving and preprocessing an image, wherein the preprocessing comprises converting the image into a grayscale image and generating an integral image from the grayscale image.
[0011] Further the method includes applying a Response Map Generation (RMG) process on the integral image to construct a scale space of the integral image comprising Points of Interest (POIs) across different scales of the integral image, wherein the RMG process comprises: performing column wise and row wise first order differentiation of the integral image to obtain a first order derivative matrix
of the integral image; generating an offset matrix by multiplying the first order derivative matrix by a normal Diagonally Dominant Matrix (DDM), wherein the normal DDM is a lightweight unit matrix of order 3x3; comparing a value of each of a plurality of elements of the offset matrix with an offset threshold; and identifying one or more elements among the plurality of elements, having the value above the offset threshold, as points of interest (POIs).
[0012] Further the method includes assigning a reproducible orientation to each of the POIs to provide rotation invariance during a feature descriptor generation for each of the POIs; and generating the feature descriptor for each of the POIs, wherein the feature descriptor describes a pixel intensity distribution based on a scale independent neighborhood, and, wherein the reproducible orientation enables rotation independent feature descriptor generation.
[0013] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.
BRIEF DESCRIPTION OF THE DRAWINGS
[0014] The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate exemplary embodiments and, together with the description, serve to explain the disclosed principles:
[0015] FIG. 1A is a functional block diagram of a system, for enhanced image feature extraction, in accordance with some embodiments of the present disclosure.
[0016] FIG. 1B is an architectural overview of a feature extraction module of the system of FIG. 1, in accordance with some embodiments of the present disclosure.
[0017] FIG. 2A and 2B are flow diagrams illustrating a method for enhanced image feature extraction, using the system of FIG. 1, in accordance with some embodiments of the present disclosure.
[0018] FIGS. 3A through 3D (collectively referred as FIG. 3) and FIGS. 4A through 4D (collectively referred as FIG. 4) are example illustrations for
comparative analysis of the enhanced image feature extraction provided by the system of FIG. 1 with state of art techniques, in accordance with some embodiments of the present disclosure.
[0019] It should be appreciated by those skilled in the art that any block diagrams herein represent conceptual views of illustrative systems and devices embodying the principles of the present subject matter. Similarly, it will be appreciated that any flow charts, flow diagrams, and the like represent various processes which may be substantially represented in computer readable medium and so executed by a computer or processor, whether or not such computer or processor is explicitly shown.
DETAILED DESCRIPTION OF EMBODIMENTS [0020] Exemplary embodiments are described with reference to the accompanying drawings. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. Wherever convenient, the same reference numbers are used throughout the drawings to refer to the same or like parts. While examples and features of disclosed principles are described herein, modifications, adaptations, and other implementations are possible without departing from the scope of the disclosed embodiments.
[0021] Embodiments of the present disclosure provide a method and system for enhanced image feature extraction. Unlike the conventional Speeded-up Robust Features (SURF™) the method disclosed identifies higher number of Points of Interest (POIs) in an image compromising on time required by (SURF™). This is achieved by replacing the computationally heavy Hessian matrix (a second order derivative matrix) used by (SURF™) during Response map generation (RMG) process by a light weight normal Diagonally Dominant Matrix (DDM), which is a unit matrix of order 3x3. Also, the method disclosed herein provides better stability against rotation or oriented images and performance does not drop with variation in image illumination.
[0022] Further, unlike the well-known Scale Invariant Features Transform (SIFT™) technique which is computationally very heavy and cannot be
implemented on low power devices, the method disclosed herein consumes less time, and is computationally light weight and thus easy to implement on low power devices with good feature extraction capability.
[0023] Referring now to the drawings, and more particularly to FIGS. 1A through 4D, where similar reference characters denote corresponding features consistently throughout the figures, there are shown preferred embodiments and these embodiments are described in the context of the following exemplary system and/or method.
[0024] FIG. 1A is a functional block diagram of a system, for enhanced image feature extraction, in accordance with some embodiments of the present disclosure.
[0025] In an embodiment, the system 100 includes a processor(s) 104, communication interface device(s), alternatively referred as input/output (I/O) interface(s) 106, and one or more data storage devices or a memory 102 operatively coupled to the processor(s) 104. The system 100 with one or more hardware processors is configured to execute functions of one or more functional blocks of the system 100.
[0026] Referring to the components of system 100, in an embodiment, the processor(s) 104, can be one or more hardware processors 104. In an embodiment, the one or more hardware processors 104 can be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuitries, and/or any devices that manipulate signals based on operational instructions. Among other capabilities, the one or more hardware processors 104 are configured to fetch and execute computer-readable instructions stored in the memory 102. In an embodiment, the system 100 can be implemented in a variety of computing systems including laptop computers, notebooks, hand-held devices such as mobile phones, workstations, mainframe computers, servers, and the like.
[0027] The I/O interface(s) 106 can include a variety of software and hardware interfaces, for example, a web interface, a graphical user interface, and the like and can facilitate multiple communications within a wide variety of networks
N/W and protocol types, including wired networks, for example, LAN, cable, etc., and wireless networks, such as WLAN, cellular and the like. In an embodiment, the I/O interface (s) 106 can include one or more ports for connecting to a number of external devices or to another server or devices.
[0028] The memory 102 may include any computer-readable medium known in the art including, for example, volatile memory, such as static random access memory (SRAM) and dynamic random access memory (DRAM), and/or non-volatile memory, such as read only memory (ROM), erasable programmable ROM, flash memories, hard disks, optical disks, and magnetic tapes.
[0029] Further, the memory 102 includes a database 108 that stores an image(s), received via the I/O interface 106. Further, the memory 102 includes modules such as a feature extraction module 110. The feature extraction module 110 processes the received image (s) and provides enhanced feature extraction. The feature extraction module 110 is explained in conjunction with FIG. 1B and flow diagram depicted in FIGS. 2A and 2B along with examples in FIG. 3 and FIG. 4. The database 108, may also store all input, images, preprocessed images, the POIs, and corresponding feature descriptors for each processed image. Further, the memory 102 may comprise information pertaining to input(s)/output(s) of each step performed by the processor(s) 104 of the system100 and methods of the present disclosure. In an embodiment, the database 108 may be external (not shown) to the system 100 and coupled to the system via the I/O interface 106.
[0030] FIG. 2A and 2B is a flow diagram illustrating a method 200 for enhanced image feature extraction, using the system of FIG. 1, in accordance with some embodiments of the present disclosure.
[0031] In an embodiment, the system 100 comprises one or more data storage devices or the memory 102 operatively coupled to the processor(s) 104 and is configured to store instructions for execution of steps of the method 200 by the processor(s) or one or more hardware processors 104. The steps of the method 200 of the present disclosure will now be explained with reference to the components or blocks of the system 100 as depicted in FIG. 1A, 1B and the steps of flow diagram as depicted in FIGS. 2A and 2B. Although process steps, method steps, techniques
or the like may be described in a sequential order, such processes, methods, and techniques may be configured to work in alternate orders. In other words, any sequence or order of steps that may be described does not necessarily indicate a requirement that the steps to be performed in that order. The steps of processes described herein may be performed in any order practical. Further, some steps may be performed simultaneously.
[0032] The steps of the method 200 (202 through 208) are executed by the feature extraction module 110 implemented via the one or more hardware processors 104 as depicted in architectural overview of the system 100 in FIG. 1B.
[0033] Referring to the steps of the method 200, at step 202 of the method 200, the one or more hardware processors 104 receive and preprocess an image, wherein the preprocessing comprises converting the image into a grayscale image and generating an integral image from the grayscale image. As understood and well known in the art. integral image is used as a quick and effective way of calculating the sum of values (pixel values) in a given image , or a rectangular subset of a grid (the given image). The integral image is used for the quick computation of box type convolution filters.
[0034] At step 204 of the method 200, the one or more hardware processors 104 apply a Response Map Generation (RMG) process on the integral image to construct a scale space of the integral image comprising Points of Interest (POIs) across different scales of the integral image.
[0035] Scale Space Representation: Objects in the real world have characteristics in various levels of details, such as contours at a low level and texture at a high level. An algorithm for automatic extraction of image features must obtain information about the different aspects of each object. Thus, most image descriptors use a scale-space structure that generates one-parameter family of images obtained from the original image, with each member representing a different detail level. This step ensures that objects with different sizes or levels of degradation can be recognized, as they present similar features at different scales. The method disclosed utilizes the SURF™ scale-space, which uses a series of pre-designed filters that can be applied in parallel, as opposed to the cascade of Gaussian filters employed by the
SIFT algorithm. To match interest points over different scales, a pyramidal scale space is built. Instead of downs sampling, each successive level of the pyramid is built by upscaling the image in parallel. Each scale is defined as the response of the image with the convolution of a box filter of a certain dimension (9x9, 15x15, 27x27, etc.).Further, the scale space is subdivided into octaves (sets of filter responses). Thus, the RMG process takes care for compensating for any scale variance in input image
[0036] The RMG process:
a) Performing (204a) column wise and row wise first order differentiation of the integral image to obtain a first order derivative matrix of the integral image. As well known in art, the image is differentiated in both X direction (column wise) and Y direction (row wise) to get a differential image, that identifies regions of abrupt changes in pixel values.
b) Generating (204b) an offset matrix by multiplying the first order derivative matrix by a normal Diagonally Dominant Matrix (DDM), wherein the normal DDM is a lightweight unit matrix of order 3x3. The conventional technique SURF™ utilizes the Hessian matrix during the response map generation phase. Rather than using a different measure for selecting the location and the scale (Hessian-Laplace detector), SURF™ relies on the determinant of Hessian matrix for both location and scale. Given a pixel, the Hessian of this pixel is expressed as given in equation 1 below:
To adapt to any scale, the image is filtered by a Gaussian kernel, so given a point X = (x, y), the Hessian matrix H(x, σ) in x at scale σ is defined as in equation 2 below, where Lxx (x, σ) is the convolution of the Gaussian second order derivative with the image I in point x, and similarly for Lxy (x, σ) is the convolution of the Gaussian second order derivative ) and Lyy (x, σ) is the convolution of the Gaussian second order derivative ).
The method disclosed herein removes some traces of this Hessian Matrix and employs the 3x3 normal diagonally dominant matrix as in expression 3 below, which improves the number of interest points detection without compromising on time compared to the SURF™ algorithm. There are few references to DDM in the literature, but they refer to DDM of Hessian matrix and not the normal DDM referred by the method herein.
Hessian Laplace uses Scale Invariant Interest Point Detector which maximizes the entropy within the region, but second ordered derivative of Hessian is computationally intensive. Replacing the Hessian matrix with DDM makes the computation fast and at the same time it increases the selection of the more neighborhood points in the region of interest as the first order derivative matrix already covers or captures more interest points. In accordance with the step 204b, the first order derivative matrix is multiplied with DDM instead of multiplying with the inverse of hessian matrix. The normal DDM replaces the Hessian matrix (second order derivative matrix) and maintains first order nature of the computation. This enables the method disclosed to keep the computation intact and fast. This is not possible with the second order derivative matrix, which being a second order computation is obviously computationally intensive.
c) Comparing (204c) a value of each of a plurality of elements of the offset matrix with an offset threshold. The offset threshold value and the thresholding process follows the standard SURF™ process. In one implementation, the offset threshold value is set to 0.5f, where f indicates a floating point value. This offset threshold value is used to get the extrema points as the POIs.
d) Identifying (204d) one or more elements among the plurality of elements, having the value above the offset threshold, as points of interest (POIs).
Based on the generation of scale response maps, maxima and minima are
used as POIs, interchangeably referred to as interest points, where the POIs
are points which have a well-defined position and can be robustly detected.
POIs are typically associated with a significant change of one or more image
properties simultaneously, for example, intensity, color, texture, and the like.
[0037] At step 206 of the method 200, the one or more hardware processors
104 assign a reproducible orientation to each of the POIs to provide rotation and
scale invariance during a feature descriptor generation for each of the POIs. The
method disclosed when applied on an image and its rotated and scaled siblings still
provides better performance as compared to conventional methods. The rotation and
scale invariance enables to extract almost same number of POIs of the image for the
rotated and scaled siblings. Thus, for any image processing application requiring
identification of matching between image and its siblings (as depicted in FIG. 3C
and 3D and FIGS. 4C and 4D) the method 200 enables better match.
[0038] At step 208 of the method 200, the one or more hardware processors 104 generate the feature descriptor for each of the POIs, wherein the feature descriptor describes a pixel intensity distribution based on a scale independent neighborhood, and wherein the reproducible orientation enables scale and rotation independent feature descriptor generation. .
[0039] Once the feature descriptors are generated by the method, the feature descriptors can be used for any image processing applications as in examples below. Thus, for say image stitching application, the method 200 identifies the POIs and corresponding feature descriptor for each of the POIs for a set of images frames and stitches the set of image frames based on the matching identified between the POIs of successive image frames. .Similarly, say for object detection application, the method 200 utilizes the feature descriptors corresponding to the identified POIs of the image to identify object of interest in the image in accordance to feature descriptors defined for the object. 1) Occlusion detection - Object tracking and detection has a variety of applications
especially in the field of security surveillance. Feature extraction as disclosed herein can be used in partial Occlusion detection to correctly identify occluded objects in the video.
2) Autonomous vehicle obstruction detection – Feature extraction as disclosed herein is applied to automated driving cars in order to detect object that are in their vicinity so as to detect obstructions in their path
3) Augmented reality - Feature extraction as disclosed herein is the essence of Augmented Reality. Augmented reality is a field in which 3D virtual objects are integrated into a 3-D real environment in real time. The Virtual Reality business is a booming one with virtual reality gaming and tourism gaining a lot of traction especially during the pandemic.
[0040] EXPERIMENTAL RESULTS: The system 100 is tested on few
images from publicly available open databases such as
https://pixabay.com/photos/stadttheater-freiburg-main-page-5002861/ (building image) and https://www.pexels.com/photo/lifeguard-station-at-beach-3560450/ (beach image).
[0041] FIGS. 3A through 3D (collectively referred as FIG. 3) and FIGS. 4A through 4D (collectively referred as FIG. 4) are example illustrations for comparative analysis of the enhanced image feature extraction provided by the system of FIG. 1 with state of art techniques, in accordance with some embodiments of the present disclosure. FIGS..3A through 4D depict comparison of the industry standard (SURF™) with the method 200 implemented using the system 100 for number of interest points or POIs detected to generate the feature descriptor.
[0042] As depicted in FIGS. 3 and 4, it can be noted that method 200 exceeds the POIs as compared to SURF. Few image spots are depicted in FIG. 3 to depict the comparison. However, it can be understood by person ordinary skilled in the art that it is practically difficult to depict all the additional POIs. However, the same is calculated using a POI calculator code and provided in table 1 below.
[0043] Further, table 1 below compares results of the SURF™ and the method 200 (system 100) when tested on the test images based on four parameters: a) number of POIs detected for the test images
b) number of POIs detected for rotated and scaled version of the test images
c) time taken for POI detection
d) number of matching POIs between POIs extracted for the test image and POIs extracted for the rotated and scaled version of the test images, for the SURF™ and the method.
TABLE 1:
1 2 3 4 IMAGE NAME INTE POI REST NTS TIME TAKEN MATCHING POINTS
SURF Method 200 SURF Method 200 SURF Method 200
beach 217 245 0.182442 0.181387 61 75
beachrotated scaled 137 147 0.115362 0.114782
building 1052 1165 0.784556 0.782312 183 228
building_rotated_ scaled 473 517 0.302488 0.301784
[0044] It can be observed from columns of the table 1, wherein the numerals are depicted in bold font, that the method 200 exceeds performance across 3 parameters: the number of POIs for the original test images, the number of POIs for the rotated and scaled versions of the test images and the number of matching POIs between original test image and its rotated and scaled version, while not compensating on time consumed for the POI detection as compared to SURF™. Moreover, the method shows improvement over time parameter.
[0045] The method disclosed herein, alternatively referred as improved SURF (iSURF), provides enhanced and efficient feature extraction by identifying a greater number of POIs without consuming additional time. The method and system provides a computationally light weight feature extraction solution, which is easy to implement on low power devices, with good feature extraction capability. Also, provides better stability against rotation or oriented images and performance does not drop with variations in image illumination.
[0046] The written description describes the subject matter herein to enable any person skilled in the art to make and use the embodiments. The scope of the subject matter embodiments is defined by the claims and may include other modifications that occur to those skilled in the art. Such other modifications are intended to be within the scope of the claims if they have similar elements that do not differ from the literal language of the claims or if they include equivalent elements with insubstantial differences from the literal language of the claims.
[0047] It is to be understood that the scope of the protection is extended to such a program and in addition to a computer-readable means having a message therein; such computer-readable storage means contain program-code means for implementation of one or more steps of the method, when the program runs on a server or mobile device or any suitable programmable device. The hardware device can be any kind of device which can be programmed including e.g. any kind of computer like a server or a personal computer, or the like, or any combination thereof. The device may also include means which could be e.g. hardware means like e.g. an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a combination of hardware and software means, e.g. an ASIC and an FPGA, or at least one microprocessor and at least one memory with software processing components located therein. Thus, the means can include both hardware means, and software means. The method embodiments described herein could be implemented in hardware and software. The device may also include software means. Alternatively, the embodiments may be implemented on different hardware devices, e.g. using a plurality of CPUs.
[0048] The embodiments herein can comprise hardware and software elements. The embodiments that are implemented in software include but are not limited to, firmware, resident software, microcode, etc. The functions performed by various components described herein may be implemented in other components or combinations of other components. For the purposes of this description, a computer-usable or computer readable medium can be any apparatus that can comprise, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
[0049] The illustrated steps are set out to explain the exemplary embodiments shown, and it should be anticipated that ongoing technological development will change the manner in which particular functions are performed. These examples are presented herein for purposes of illustration, and not limitation. Further, the boundaries of the functional building blocks have been arbitrarily defined herein for the convenience of the description. Alternative boundaries can be defined so long as the specified functions and relationships thereof are appropriately performed. Alternatives (including equivalents, extensions, variations, deviations, etc., of those described herein) will be apparent to persons skilled in the relevant art(s) based on the teachings contained herein. Such alternatives fall within the scope of the disclosed embodiments. Also, the words “comprising,” “having,” “containing,” and “including,” and other similar forms are intended to be equivalent in meaning and be open ended in that an item or items following any one of these words is not meant to be an exhaustive listing of such item or items, or meant to be limited to only the listed item or items. It must also be noted that as used herein and in the appended claims, the singular forms “a,” “an,” and “the” include plural references unless the context clearly dictates otherwise.
[0050] Furthermore, one or more computer-readable storage media may be utilized in implementing embodiments consistent with the present disclosure. A computer-readable storage medium refers to any type of physical memory on which information or data readable by a processor may be stored. Thus, a computer-readable storage medium may store instructions for execution by one or more processors, including instructions for causing the processor(s) to perform steps or stages consistent with the embodiments described herein. The term “computer-readable medium” should be understood to include tangible items and exclude carrier waves and transient signals, i.e., be non-transitory. Examples include random access memory (RAM), read-only memory (ROM), volatile memory, nonvolatile memory, hard drives, CD ROMs, DVDs, flash drives, disks, and any other known physical storage media.
[0051] It is intended that the disclosure and examples be considered as exemplary only, with a true scope of disclosed embodiments being indicated by the following claims.
We Claim:
1. A processor implemented method for image feature extraction, the method comprising:
receiving and preprocessing (202), via one or more hardware processors, an image, wherein the preprocessing comprises converting the image into a grayscale image and generating an integral image from the grayscale image;
applying (204), via the one or more hardware processors, a Response Map Generation (RMG) process on the integral image to construct a scale space comprising Points of Interest (POIs) across different scales of the integral image, wherein the RMG process comprises:
performing (204a) column wise and row wise first order differentiation of the integral image to obtain a first order derivative matrix of the integral image;
generating (204b) an offset matrix by multiplying the first order derivative matrix by a normal Diagonally Dominant Matrix (DDM), wherein the normal DDM is a lightweight unit matrix;
comparing (204c) a value of each of a plurality of elements of the offset matrix with an offset threshold; and
identifying (204d) one or more elements among the
plurality of elements, having the value above the offset
threshold, as points of interest (POIs)
assigning (206), via the one or more hardware processors, a
reproducible orientation to each of the POIs to provide rotation invariance
during a feature descriptor generation for each of the POIs; and
generating (208), via the one or more hardware processors, the feature descriptor for each of the POIs, wherein the feature descriptor
describes a pixel intensity distribution based on a scale independent neighborhood, and wherein the reproducible orientation enables rotation independent feature descriptor generation.
2. The method as claimed in claim 1, wherein the normal DDM is a 3X3 rank matrix.
3. The method as claimed in claim 1, wherein the method further comprising identifying the POIs and corresponding feature descriptor for each of the POIs for a set of images and stitching the set of images based on the matching identified between the POIs of successive images.
4. The method as claimed in claim 1, wherein the method further comprises utilizing the feature descriptors corresponding to the identified POIs of the image to identify object of interest in the image in accordance to feature descriptors defined for the object.
5. A system (100) for image feature extraction, the system (100) comprising:
a memory (102) storing instructions;
one or more Input/Output (I/O) interfaces (106); and
one or more hardware processors (104) coupled to the memory (102) via the
one or more I/O interfaces (106), wherein the one or more hardware
processors (104) are configured by the instructions to:
receive and preprocess an image, wherein the preprocessing comprises converting the image into a grayscale image and generating an integral image from the grayscale image;
apply a Response Map Generation (RMG) process on the integral image to construct a scale space comprising Points of Interest (POIs) across different scales of the integral image, wherein the RMG process comprises:
performing column wise and row wise first order differentiation of the integral image to obtain a first order derivative matrix of the integral image;
generating an offset matrix by multiplying the first order derivative matrix by a normal Diagonally Dominant Matrix (DDM), wherein the normal DDM is a lightweight unit matrix;
comparing a value of each of a plurality of elements of the offset matrix with an offset threshold; and
identifying one or more elements among the plurality
of elements, having the value above the offset threshold, as
points of interest (POIs);
assign a reproducible orientation to each of the POIs to provide
rotation invariance during a feature descriptor generation for each of the
POIs; and
generate the feature descriptor for each of the POIs, wherein the feature descriptor describes a pixel intensity distribution based on a scale independent neighborhood, and wherein the reproducible orientation enables scale and rotation independent feature descriptor generation.
6. The system as claimed in claim 1, wherein the normal DDM is a 3X3 rank matrix.
7. The system as claimed in claim 1, wherein the one or more hardware processors (104) are further configured to identify the POIs and corresponding feature descriptor for each of the POIs for a set of images and stitching the set of images based on the matching identified between the POIs of successive images.
8. The method as claimed in claim 1, wherein the method further comprises utilizing the feature descriptors corresponding to the identified POIs of the
image to identify object of interest in the image in accordance to feature descriptors defined for the object
| # | Name | Date |
|---|---|---|
| 1 | 202121010557-STATEMENT OF UNDERTAKING (FORM 3) [12-03-2021(online)].pdf | 2021-03-12 |
| 2 | 202121010557-REQUEST FOR EXAMINATION (FORM-18) [12-03-2021(online)].pdf | 2021-03-12 |
| 3 | 202121010557-PROOF OF RIGHT [12-03-2021(online)].pdf | 2021-03-12 |
| 4 | 202121010557-FORM 18 [12-03-2021(online)].pdf | 2021-03-12 |
| 5 | 202121010557-FORM 1 [12-03-2021(online)].pdf | 2021-03-12 |
| 6 | 202121010557-FIGURE OF ABSTRACT [12-03-2021(online)].jpg | 2021-03-12 |
| 7 | 202121010557-DRAWINGS [12-03-2021(online)].pdf | 2021-03-12 |
| 8 | 202121010557-DECLARATION OF INVENTORSHIP (FORM 5) [12-03-2021(online)].pdf | 2021-03-12 |
| 9 | 202121010557-COMPLETE SPECIFICATION [12-03-2021(online)].pdf | 2021-03-12 |
| 10 | 202121010557-FORM-26 [22-10-2021(online)].pdf | 2021-10-22 |
| 11 | Abstract1.jpg | 2022-02-17 |
| 12 | 202121010557-FER.pdf | 2022-09-21 |
| 13 | 202121010557-FER_SER_REPLY [06-12-2022(online)].pdf | 2022-12-06 |
| 14 | 202121010557-DRAWING [06-12-2022(online)].pdf | 2022-12-06 |
| 15 | 202121010557-COMPLETE SPECIFICATION [06-12-2022(online)].pdf | 2022-12-06 |
| 16 | 202121010557-CLAIMS [06-12-2022(online)].pdf | 2022-12-06 |
| 17 | 202121010557-US(14)-HearingNotice-(HearingDate-18-02-2025).pdf | 2025-01-27 |
| 18 | 202121010557-FORM-26 [31-01-2025(online)].pdf | 2025-01-31 |
| 19 | 202121010557-Correspondence to notify the Controller [14-02-2025(online)].pdf | 2025-02-14 |
| 20 | 202121010557-Response to office action [18-02-2025(online)].pdf | 2025-02-18 |
| 21 | 202121010557-Written submissions and relevant documents [05-03-2025(online)].pdf | 2025-03-05 |
| 22 | 202121010557-PatentCertificate27-03-2025.pdf | 2025-03-27 |
| 23 | 202121010557-IntimationOfGrant27-03-2025.pdf | 2025-03-27 |
| 1 | imagefeatureextractionE_20-09-2022.pdf |