DESC:FIELD
The present disclosure relates to a system and method of performing tasks based on multi-modal gestures recognition. More specifically, to the system and method for performing the tasks based on neural networks.

DEFINITIONS OF TERMS USED IN THE SPECIFICATION
As used in the present disclosure, the following terms are generally intended to have the meaning as set forth below, except to the extent that the context in which they are used indicate otherwise.
The expression ‘micro gestures’ used hereinafter in the specification refers to, but is not limited to, patterns of vibrations caused by movement of the user.
The expression ‘macro gestures’ used hereinafter in the specification refers to, but is not limited to, actions captured based on orientation and spatial movement of body of the user.
These definitions are in addition to those expressed in the art.

BACKGROUND
The background information herein below relates to the present disclosure but is not necessarily prior art.
Input devices are peripheral hardware equipments attached to a computer. These peripheral hardware equipments provide both basic and additional functionality to a user. Commonly known input devices may include, but not limited to, scanners, cameras, keyboards, mouse, and other pointing and reading devices.
It may be noted that the input devices may wear down, over a period of time, due to their continuous use, leading to decreased performance of the input devices. Every input device requires a wired or wireless connection to the computer. This assembly of input devices is required to be kept in the proximity of the computer for proper functioning, thereby taking up a lot of space for its ideal working.
Further, the input devices and touch screens need direct physical contact of the user. As these contacts are exposed to the environment, this leads to degradation in the performance of the computer system. Moreover, there is a possibility of abuse and damage to the input devices exposed to the public. Further, the input devices lead to hygiene problems, since a lot of users work on the same device in public environments, such as, office, internet cafes etc. These factors reduce the usefulness of the system designed to cater to a wide range of users.
Further, speech recognition is yet another mode of inputting information, but is very limited in a noisy environment.
Therefore, there is a need to provide a system for capturing user input and performing tasks inputted by a user that alleviates the aforementioned challenges.

OBJECTS
Some of the objects of the present disclosure, which at least one embodiment herein satisfies, are as follows:
It is an object of the present disclosure to ameliorate one or more problems of the prior art or to at least provide a useful alternative.
An object of the present disclosure is to provide a system and method for performing tasks based on multi-modal gesture recognition provided by a user.
Another object of the present disclosure is to provide a system and method for enabling a user to control devices through gestures.
Yet another object of the present disclosure is to provide a system and method for eliminating the use of traditional input devices.
Still another object of the present disclosure is to provide a system and method that is easier to operate.
Other objects and advantages of the present disclosure will be more apparent from the following description, which is not intended to limit the scope of the present disclosure.

SUMMARY
A system for performing tasks based on multi-modal gestures, wherein the system includes a wearable unit, an image capturing unit, a repository, and a control unit. A plurality of sensors are mounted on the wearable unit, the sensors are configured to capture micro gestures provided by a user, and further configured to generate micro gesture signals corresponding to the captured micro gestures. The image capturing unit is configured to capture macro gestures provided by the user, and further configured to generate macro gesture signals corresponding to the captured macro gestures. The repository is configured to store a pre-defined set of analyzing rules, a first list of micro gestures and a pre-defined task corresponding to each of the micro gestures, a second list of macro gestures and a pre-defined task corresponding to each of the macro gestures. The control unit is configured to cooperate with the repository, the wearable unit, and the image capturing unit. The control unit is further configured to condition the generated macro and micro gesture signals to generate digital micro and macro gestures, perform analysis of the digital micro and macro gestures based on the pre-defined set of analyzing rules to identify accuracy level of each of the digital micro and macro gestures, and perform at least one task corresponding to either of the digital micro gesture, the macro gesture, or both based on the identified accuracy level of each of the digital micro and macro gestures.
In an embodiment, the plurality of sensors mounted on the wearable unit is configured to capture micro gestures based on patterns of vibrations caused by movement of the user.
In another embodiment, the image capturing unit is configured to capture macro gestures based on at least one of orientation and spatial movement of body of the user.
In still another embodiment, the control unit includes a signal conditioning module configured to receive and convert the macro and micro gesture signals to the digital micro and macro gestures.
In yet another embodiment, the control unit includes an analyzer configured to perform analysis of the digital gestures to identify the accuracy level of each of the digital micro and macro gestures, generate a control signal based on the identified accuracy level of each of the digital micro and macro gestures, and transmit the control signal to a user device.
In an embodiment, the analyzer includes a prediction module configured to predict the accuracy level of each of the digital micro and macro gestures by employing convolutional neural network.
In another embodiment, the analyzer includes a memory configured to store a first threshold value, a second threshold value, and a third threshold value, wherein the third threshold value is greater than the second threshold value, the second threshold value is greater than the first threshold value.
In still another embodiment, the analyzer includes a comparator configured to cooperate with the prediction module and the memory and is further configured to compare the accuracy level of the digital micro and macro gestures with the threshold values.
In yet another embodiment, the analyzer includes a control component configured to cooperate with the comparator to generate the control signal based on the comparison of the micro and macro digital gesture with the threshold values.
In an embodiment, the control component is configured to cooperate with the repository, and further configured to:
• extract a pre-defined task corresponding to the micro gesture, when the accuracy level is above the third threshold value;
• generate the control signal based on the extracted pre-defined task; and
• transmit the control signal to the user device for performing the pre-defined task corresponding to the micro digital gesture.
In another embodiment, the control component is configured to cooperate with the repository, and is further configured to:
• extract a pre-defined task corresponding to combination of the digital micro and macro gestures, when the accuracy level of the digital micro gesture is below the third threshold value and the digital micro and macro gestures is above the second threshold value;
• generate the control signal based on the extracted pre-defined task; and
• transmit the control signal to the user device for performing the pre-defined task corresponding to the combination of the digital micro and macro gestures.
In still another embodiment, the control component is configured to cooperate with the repository, and further configured to:
• extract a pre-defined task corresponding to digital gesture with higher accuracy level, when the accuracy level of both of the digital gestures is above the first threshold value and between the second and first threshold values;
• generate the control signal based on the extracted pre-defined task; and
• transmit the control signal to the user device for performing the pre-defined task corresponding to the digital gesture with the higher accuracy level.
In another embodiment, the system includes a learning module configured to facilitate the user to add at least one new set of the micro and macro gestures via a user interface.
In still another embodiment, the learning module is configured to store the added new set of micro and macro gestures and corresponding task in the repository.
A method for performing tasks based on multi modal gestures, the method comprising steps of:
• capturing micro gestures provided by a user and generating micro gesture signals corresponding to the captured micro gestures, by a wearable unit;
• capturing macro gestures provided by a user and generating macro gesture signals corresponding to the captured macro gestures, by an image capturing unit;
• conditioning, by a control unit, the generated macro and micro gesture signals to generate digital micro and macro gestures;
• performing analysis, by the control unit, of the digital micro and macro gestures based on pre-defined set of analyzing rules to identify accuracy level of each of the digital micro and macro gestures; and
• performing, by the control unit, at least one task corresponding to either of the digital micro gesture, the macro gesture, or both based on the identified accuracy level of each of the digital micro and macro gestures.
In an embodiment, the step of performing analysis includes sub steps of:
• predicting, by a prediction module, the accuracy level of each of the digital micro and macro gestures by employing convolutional neural network;
• storing, in a memory, a first threshold value, a second threshold value, and a third threshold value, wherein the third threshold value is greater than the second threshold value, the second threshold value is greater than the first threshold value;
• comparing, by a comparator, the accuracy level of the each of digital micro and macro gestures with the threshold values; and
• generating, by a control component, a control signal based on the comparison of the digital micro and macro gestures with the threshold values.
In another embodiment, the step of generating includes sub steps of:
• extracting a pre-defined task corresponding to the micro gesture, when the accuracy level is above the third threshold value;
• generating the control signal based on the extracted pre-defined task; and
• transmitting the control signal to a user device for performing the pre-defined task corresponding to the micro digital gesture.
In still another embodiment, the step of generating includes sub steps of:
• extracting a pre-defined task corresponding to combination of the digital micro and macro gestures, when the accuracy level of the digital micro gesture is below the third threshold value and the digital micro and macro gestures is above the second threshold value;
• generating the control signal based on the extracted pre-defined task; and
• transmitting the control signal to the user device for performing the pre-defined task corresponding to the combination of the digital micro and macro gestures.
In yet another embodiment, the step of generating includes sub steps of:
• extracting a pre-defined task corresponding to digital gesture with higher accuracy level, when the accuracy level of both of the digital gestures is above the first threshold value and between the second and first threshold values;
• generating the control signal based on the extracted pre-defined task; and
• transmitting the control signal to the user device for performing the pre-defined task corresponding to the digital gesture with the higher accuracy level.

BRIEF DESCRIPTION OF ACCOMPANYING DRAWING
A system and method for performing tasks in a user device based on multi modal gestures recognition, of the present disclosure will now be described with the help of the accompanying drawing, in which:
Figure 1 illustrates a block diagram of a system for performing tasks in a user device based on multi modal gestures recognition, in accordance with an embodiment of the present disclosure in communication with a user device;
Figure 2 illustrates an exemplary embodiment of setup of the system of Figure 1;
Figures 3A and 3B illustrate a desktop with an image capturing unit and a wearable unit; and
Figure 4 illustrates a flow of a method for performing tasks in a user device based on multi modal gestures recognition, in accordance with an embodiment of the present disclosure.

LIST OF REFERENCE NUMERALS USED IN DETAILED DESCRIPTION AND DRAWING
100 – System
102 – Wearable unit
104 – Image capturing unit
106 – Repository
108 – Control unit
110 – Signal conditioning module
112 – Analyzer
114 – Prediction module
116 – Memory
118 – Comparator
120 – Control component
122 – User interface

DETAILED DESCRIPTION
Embodiments, of the present disclosure, will now be described with reference to the accompanying drawing.
Embodiments are provided so as to thoroughly and fully convey the scope of the present disclosure to the person skilled in the art. Numerous details are set forth, relating to specific components, and methods, to provide a complete understanding of embodiments of the present disclosure. It will be apparent to the person skilled in the art that the details provided in the embodiments should not be construed to limit the scope of the present disclosure. In some embodiments, well-known processes, well-known apparatus structures, and well-known techniques are not described in detail.
The terminology used, in the present disclosure, is only for the purpose of explaining a particular embodiment and such terminology shall not be considered to limit the scope of the present disclosure. As used in the present disclosure, the forms "a,” "an," and "the" may be intended to include the plural forms as well, unless the context clearly suggests otherwise. The terms “comprises,” “comprising,” “including,” and “having,” are open ended transitional phrases and therefore specify the presence of stated features, integers, steps, operations, elements, modules, units and/or components, but do not forbid the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. The particular order of steps disclosed in the method and process of the present disclosure is not to be construed as necessarily requiring their performance as described or illustrated. It is also to be understood that additional or alternative steps may be employed.
When an element is referred to as being "mounted on," “engaged to,” "connected to," or "coupled to" another element, it may be directly on, engaged, connected or coupled to the other element. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed elements.
The present disclosure envisages a system and a method for performing tasks in a user device based on recognition of multi-modality in gestures provided by a user.
Referring to Figure 1, the system 100 for performing tasks in a user device based on recognition of multi-modality in gestures provided by a user is disclosed. The system 100 includes a wearable unit 102, an image capturing unit 104, a repository 106, and a control unit 108.
In an embodiment, the wearable unit 102 includes a start button that generates a start signal when pressed by the user. Upon actuation, the wearable unit 102 is configured to communicate a start signal to the control unit 108. Upon receipt of the start signal, the control unit 108 controls the execution of the tasks on a user interface of the user device.
In an embodiment, the user device can be selected from the group consisting of, but is not limited to, a mobile phone, a smartphone, an iPad, a tablet, and a palmtop. In another embodiment, the user interface can be selected from the group consisting of, but is not limited to, at least one switch, a touch pad, a touch screen, and at least one push button.
The wearable unit 102 includes a wearable device (not shown in the figure) and a plurality of sensors (not shown in the figure) mounted on the wearable unit 102. The plurality of sensors is configured to capture micro gestures provided by the user, and are further configured to generate micro gesture signals corresponding to the captured micro gestures. In an embodiment, the micro gestures are based on patterns of vibrations caused by movement of the user. For example, the vibration associated to movements and orientation of fingers of the user, such as, tapping of an index finger against thumb, snapping, and simple rubs using fingers and the like.
In an embodiment, the plurality of sensors, mounted on the wearable unit 102, is configured to capture the micro gestures.
In an embodiment, the wearable unit 102 can be selected from the group consisting of, but not limited to, a smart watch, a smart bracelet, or any other wearable electronic device that captures high-frequency vibrations.
Further, the image capturing unit 104 is configured to capture macro gestures provided by the user, and further configured to generate macro gesture signals corresponding to the captured macro gestures. In an embodiment, the macro gestures are based on orientation and spatial movement of body of the user. For example, capturing a visual feed based on orientation and spatial movement of hands, such as, palm moving across a pre-defined frame, fist working as a grab gesture. In another embodiment, the gestures can be provided by any body part of the user that is moveable, such as, hands, fingers, shoulders and the like. In another embodiment, the image capturing unit 104 is configured to capture a video or an image of the macro gestures.
In an embodiment, the image capturing unit 104 can be selected from the group consisting of, not limited to, built-in camera of the user device, a high-definition camera, a web-cam and the like.
The micro gesture signals and the macro gesture signals generated by the wearable unit 102 and the image capturing unit 104 respectively are then transmitted to the control unit 108. In an embodiment, the wearable unit 102 and the image capturing unit 104 communicate with the control unit 108 through a network (not shown), as will be appreciated by people skilled in the art.
In one implementation, the network may be a wireless network, a wired network or a combination thereof. The network may be implemented as one of the different types of networks, such as intranet, local area network (LAN), wide area network (WAN), the internet, and the like. The network may either be a dedicated network or a shared network. The shared network represents an association of the different types of networks that use a variety of protocols, for example, Hypertext Transfer Protocol (HTTP), Hypertext Transfer Protocol Secure (HTTPS), Transmission Control Protocol/Internet Protocol (TCP/IP), Wireless Application Protocol (WAP), and the like, to communicate with one another. Further the network may include a variety of network devices, including routers, bridges, servers, computing devices, storage devices, and the like.
The repository 106 is configured to store a pre-defined set of analyzing rules, a first list of micro gestures and a pre-defined task corresponding to each micro gesture, a second list of macro gestures and a pre-defined task corresponding to each macro gesture.
The control unit 108 is configured to cooperate with the repository 106, and is further configured to receive the micro gesture signals and the macro gesture signals in cooperation with the wearable unit 102, and the image capturing unit 104.
The control unit 108 is further configured to condition the generated macro gesture signals and the micro gesture signals to generate digital micro gestures and macro gestures. In an embodiment, the conditioning includes, but is not limited to, filtering, cleaning, normalization, transformation, feature extraction, selection and a combination thereof.
The control unit 108 performs analysis of the digital micro gestures and the digital macro gestures based on the pre-defined set of analyzing rules to identify accuracy level of each of the digital micro and macro gestures, and further performs at least one task corresponding to either of the digital micro gesture, the macro gesture, or both based on the identified accuracy level of each of the digital micro and macro gestures.
In an embodiment, the control unit 108 can be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuitries, and/or any devices that manipulate signals based on operational instructions. Among other capabilities, the control unit 108 is configured to fetch and execute the set of pre-determined instructions from a memory (not shown in the figure) which when executed provides for one or more executable commands.
In another embodiment, the control unit 108 includes a signal conditioning module 110, and an analyzer 112. The signal conditioning module 110 is configured to receive and convert the macro gesture signals and the micro gesture signals to the digital micro gestures and the digital macro gestures. Further, the analyzer 112 is configured to perform analysis of the digital gestures to identify the accuracy level of each of the digital micro and the macro gestures. The analyzer 112 generates a control signal based on the identified accuracy level of each of the digital micro and macro gestures, and transmits the control signal to the user device.
In an embodiment, the analyzer 112 includes a prediction module 114, a memory 116, a comparator 118, and a control component 120. The prediction module 114 is configured to predict the accuracy level of each of the digital micro gestures and the digital macro gestures by employing convolutional neural network. The memory 116 is configured to store a first threshold value, a second threshold value, and a third threshold value. In an embodiment, the third threshold value is greater than the second threshold value which is greater than the first threshold value.
The analyzer 112 further includes comparator 118 configured to cooperate with the prediction module 114 and the memory 116 and is further configured to compare the accuracy level of the digital micro gestures and the digital macro gestures with the above said threshold values.
The control component 120 is configured to cooperate with the comparator 118 to generate the control signal based on the comparison of the micro and macro digital gesture with the threshold values.
In first embodiment, the control component 120 is configured to cooperate with the repository 106 and is further configured to extract a pre-defined task corresponding to the micro gesture, when the accuracy level is greater than the third threshold value.
In accordance with the first embodiment, the control component 120 is further configured to generate the control signal based on the extracted pre-defined task and transmit the control signal to the user device for performing the pre-defined task corresponding to the digital micro gestures.
In second embodiment, the control component 120 is configured to cooperate with the repository 106 and is further configured to extract a pre-defined task corresponding to combination of the digital micro gestures and the digital macro gestures, when the accuracy level of the digital micro gesture is less than the third threshold value and the digital micro gestures and the macro gestures is greater than the second threshold value.
In accordance with the second embodiment, the control component 120 is further configured to generate the control signal based on the extracted pre-defined task and transmit the control signal to the user device for performing the pre-defined task corresponding to the combination of the digital micro and macro gestures.
In third embodiment, the control component 120 is configured to cooperate with the repository 106 and is further configured to extract a pre-defined task corresponding to digital gesture with higher accuracy level, when the accuracy level of both of the digital gestures is greater than the first threshold value and between the second threshold values and first threshold values.
In accordance with the third embodiment, the control component 120 is further configured to generate the control signal based on the extracted pre-defined task and transmit the control signal to the user device for performing the pre-defined task corresponding to the digital gesture with the higher accuracy level.
For example, user performs micro gesture of rubbing finger and macro gesture of moving hand upwards from a downwards position intended to increase default volume of the user device. The control unit performs the desired analysis, wherein the prediction module 114 predicts the accuracy level of both the digital micro and macro gestures by employing convolutional neural network. The comparator 118 initially compares the accuracy level of the digital micro gesture with third threshold value, assuming it to be 90%. If the accuracy level of the digital micro gesture is above said third threshold value, the comparator 118 will not compare the accuracy level of the digital macro gesture and control component 120 will generate control signal corresponding to digital micro gesture.
If the accuracy level of the digital micro gesture is below the third threshold value and above the second threshold value, assuming it to be 75%, the comparator 118 checks the accuracy level of digital macro gesture. If the accuracy level of digital macro gesture is also greater than the second threshold value, a pre-determined task corresponding to a combination of digital micro gestures and digital macro gestures is extracted from the repository 106.
In an embodiment, the control component 120 includes a crawlor and extractor (not shown in the figure) configured to crawl the first list of micro gestures and the second list of macro gestures to extract pre-defined task based on the accuracy level.
The system 100, in an embodiment, includes a learning module configured to facilitate the user to add new sets of the micro and macro gestures via a user interface 122. In an embodiment, the learning module is configured to train CNN with multiple pre-defined macro and micro gestures. In another embodiment, the learning module is configured to store the newly added set of micro and macro gestures and corresponding task in the repository 106. In still another embodiment, the user interface 122 is configured to implement the task corresponding to the control signal.
Figure 2 illustrates an exemplary embodiment of a setup of the system 100 for performing tasks based on multi-modal gesture recognition. The setup represents an image capturing unit such as a camera 202 configured to view the user’s body. The camera 202 is further configured to identify the user’s body movements.
The figure further represents a vibration sensitive wearable device 204 that is present on the user body to identify vibration patterns. For example, the user can wear the vibration sensitive wearable device 204 on the wrist. The vibration sensitive wearable device 204 is configured to transmit vibration data to a user device 206.
Figures 3A and 3B illustrate a desktop 302 with an image capturing unit 304 and a vibration sensitive wearable 306.
Referring to Figure 4, a method 400 for performing tasks based on multi modal gestures, wherein the method 400 comprises the steps of:
• capturing micro gestures 402 provided by a user and generating micro gesture signals corresponding to the captured micro gestures, by a wearable unit 102;
• capturing macro gestures 404 provided by a user and generating macro gesture signals corresponding to the captured macro gestures, by an image capturing unit 104;
• conditioning 406, by a control unit 108, the generated macro and micro gesture signals to generate digital micro and macro gestures;
• performing analysis 408 of the digital micro and macro gestures, by the control unit 108, based on pre-defined set of analyzing rules to identify accuracy level of each of the digital micro and macro gestures; and
• performing 410 at least one task, by the control unit 108, corresponding to either of the digital micro gesture, the macro gesture, or both based on the identified accuracy level of each of the digital micro and macro gestures.
In an embodiment, the step of performing analysis 408 includes sub steps of:
• predicting 408-1, by a prediction module 114, the accuracy level of each of the digital micro and macro gestures by employing convolutional neural network;
• storing 408-2, in a memory 116, a first threshold value, a second threshold value, and a third threshold value, wherein the third threshold value is greater than the second threshold value, the second threshold value is greater than the first threshold value;
• comparing 408-3, by a comparator 118, the accuracy level of the each of digital micro and macro gestures with the threshold values; and
• generating 408-4, by a control component 120, a control signal based on the comparison of the digital micro and macro gestures with the threshold values.
In another embodiment, the step of generating 408-4 includes sub steps of:
• extracting a pre-defined task corresponding to the micro gesture, when the accuracy level is above the third threshold value;
• generating the control signal based on the extracted pre-defined task; and
• transmitting the control signal to a user device for performing the pre-defined task corresponding to the micro digital gesture.
In still another embodiment, the step of generating 408-4 includes sub steps of:
• extracting a pre-defined task corresponding to combination of the digital micro and macro gestures, when the accuracy level of the digital micro gesture is below the third threshold value and the digital micro and macro gestures is above the second threshold value;
• generating the control signal based on the extracted pre-defined task; and
• transmitting the control signal to the user device for performing the pre-defined task corresponding to the combination of the digital micro and macro gestures.
In yet another embodiment, the step of generating 408-4 includes sub steps of:
• extracting a pre-defined task corresponding to digital gesture with higher accuracy level, when the accuracy level of both of the digital gestures is above the first threshold value and between the second and first threshold values;
• generating the control signal based on the extracted pre-defined task; and
• transmitting the control signal to the user device for performing the pre-defined task corresponding to the digital gesture with the higher accuracy level.
The functions described herein may be implemented in hardware, software executed by a processor, firmware, or any combination thereof. If implemented in software executed by a processor, the functions may be stored on or transmitted over as one or more instructions or code on a computer - readable medium. Other examples and implementations are within the scope and spirit of the disclosure and appended claims. For example, due to the nature of software, functions described above can be implemented using software executed by a processor, hardware, firmware, hardwiring, or combinations of any of these. Features implementing functions may also be physically located at various positions, including being distributed such that portions of functions are implemented at different physical locations.
In addition, any disclosure of components contained within other components or separate from other components should be considered exemplary because multiple other architectures may potentially be implemented to achieve the same functionality, including incorporating all, most, and/or some elements as part of one or more unitary structures and/or separate structures.
The functions described herein may be implemented in hardware, software executed by a processor, firmware, or any combination thereof. If implemented in software executed by a processor, the functions may be stored on or transmitted over as one or more instructions or code on a computer- readable medium. Other examples and implementations are within the scope and spirit of the disclosure and appended claims. For example, due to the nature of software, functions described above can be implemented using software executed by a processor, hardware, firmware, hardwiring, or combinations of any of these. Features implementing functions may also be physically located at various positions, including being distributed such that portions of functions are implemented at different physical locations.
In addition, any disclosure of components contained within other components or separate from other components should be considered exemplary because multiple other architectures may potentially be implemented to achieve the same functionality, including incorporating all, most, and/or some elements as part of one or more unitary structures and/or separate structures.

TECHNICAL ADVANCES AND ECONOMICAL SIGNIFICANCE
The present disclosure described herein above has several technical advantages including, but not limited to, the realization of a system and method for performing tasks in a user device based on multi modal gestures recognition that:
• enables user to control devices through multi-modal gestures;
• eliminates the use of traditional input devices; and
• easier to operate.
The embodiments herein and the various features and advantageous details thereof are explained with reference to the non-limiting embodiments in the following description. Descriptions of well-known components and processing techniques are omitted so as to not unnecessarily obscure the embodiments herein. The examples used herein are intended merely to facilitate an understanding of ways in which the embodiments herein may be practiced and to further enable those of skill in the art to practice the embodiments herein. Accordingly, the examples should not be construed as limiting the scope of the embodiments herein.
The foregoing description of the specific embodiments so fully revealed the general nature of the embodiments herein that others can, by applying current knowledge, readily modify and/or adapt for various applications such specific embodiments without departing from the generic concept, and, therefore, such adaptations and modifications should and are intended to be comprehended within the meaning and range of equivalents of the disclosed embodiments. It is to be understood that the phraseology or terminology employed herein is for the purpose of description and not of limitation. Therefore, while the embodiments herein have been described in terms of preferred embodiments, those skilled in the art will recognize that the embodiments herein can be practiced with modification within the spirit and scope of the embodiments as described herein.
The use of the expression “at least” or “at least one” suggests the use of one or more elements or ingredients or quantities, as the use may be in the embodiment of the disclosure to achieve one or more of the desired objects or results.
Any discussion of documents, acts, materials, devices, articles or the like that has been included in this specification is solely for the purpose of providing a context for the disclosure. It is not to be taken as an admission that any or all of these matters form a part of the prior art base or were common general knowledge in the field relevant to the disclosure as it existed anywhere before the priority date of this application.
The numerical values mentioned for the various physical parameters, dimensions or quantities are only approximations and it is envisaged that the values higher/lower than the numerical values assigned to the parameters, dimensions or quantities fall within the scope of the disclosure, unless there is a statement in the specification specific to the contrary.
While considerable emphasis has been placed herein on the components and component parts of the preferred embodiments, it will be appreciated that many embodiments can be made and that many changes can be made in the preferred embodiments without departing from the principles of the disclosure. These and other changes in the preferred embodiment as well as other embodiments of the disclosure will be apparent to those skilled in the art from the disclosure herein, whereby it is to be distinctly understood that the foregoing descriptive matter is to be interpreted merely as illustrative of the disclosure and not as a limitation. ,CLAIMS:WE CLAIM:
1. A system (100) for performing tasks based on multi modal gestures, said system (100) comprising:
• a wearable unit (102) configured to capture micro gestures provided by a user, and further configured to generate micro gesture signals corresponding to said captured micro gestures;
• an image capturing unit (104) configured to capture macro gestures provided by said user, and further configured to generate macro gesture signals corresponding to said captured macro gestures;
• a repository (106) configured to store a pre-defined set of analyzing rules, a first list of micro gestures and a pre-defined task corresponding to each of said micro gestures, a second list of macro gestures and a pre-defined task corresponding to each of said macro gestures; and
• a control unit (108) configured to cooperate with said repository (106), said wearable unit (102), and said image capturing unit (104), and further configured to:
o condition said generated macro and micro gesture signals to generate digital micro and macro gestures;
o perform analysis of said digital micro and macro gestures based on said pre-defined set of analyzing rules to identify accuracy level of each of said digital micro and macro gestures; and
o perform at least one task corresponding to at least one of said digital micro gesture, and said macro gesture based on the identified accuracy level of each of said digital micro gestures and macro gestures.
2. The system as claimed in claim 1, wherein said wearable unit (102) includes a wearable device and a plurality of sensors mounted on said wearable unit (102), wherein said wearable unit (102) is configured to capture micro gestures based on patterns of vibrations caused by movement of said user.
3. The system as claimed in claim 1, wherein said image capturing unit (104) is configured to capture macro gestures based on at least one of orientation and spatial movement of body of said user.
4. The system as claimed in claim 1, wherein said control unit (108) comprises:
• a signal conditioning module (110) configured to receive and convert said macro and micro gesture signals to said digital micro and macro gestures; and
• an analyzer (112) configured to perform analysis of said digital gestures to identify the accuracy level of each of said digital micro and macro gestures, generate a control signal based on the identified accuracy level of each of said digital micro and macro gestures, and transmit said control signal to a user device.
5. The system as claimed in claim 4, wherein said analyzer (112) includes:
• a prediction module (114) configured to predict the accuracy level of each of said digital micro and macro gestures by employing convolutional neural network;
• a memory (116) configured to store a first threshold value, a second threshold value, and a third threshold value, wherein said third threshold value is greater than said second threshold value, said second threshold value is greater than said first threshold value;
• a comparator (118) configured to cooperate with said prediction module (114) and said memory (116) and is further configured to compare the accuracy level of said digital micro and macro gestures with said threshold values;
• a control component (120) configured to cooperate with said comparator (118) to generate said control signal based on said comparison of said micro and macro digital gesture with said threshold values.
6. The system as claimed in claim 5, wherein said control component (120) is configured to cooperate with said repository (106), and further configured to:
• extract a pre-defined task corresponding to said micro gesture, when the accuracy level is above the third threshold value;
• generate said control signal based on said extracted pre-defined task; and
• transmit said control signal to said user device for performing said pre-defined task corresponding to said micro digital gesture.
7. The system as claimed in claim 5, wherein said control component (120) is configured to cooperate with said repository (106), and further configured to:
• extract a pre-defined task corresponding to combination of said digital micro and macro gestures, when the accuracy level of said digital micro gesture is below the third threshold value and said digital micro and macro gestures is above the second threshold value;
• generate said control signal based on said extracted pre-defined task; and
• transmit said control signal to said user device for performing said pre-defined task corresponding to said combination of said digital micro and macro gestures.
8. The system as claimed in claim 5, wherein said control component (120) is configured to cooperate with said repository (106), and further configured to:
• extract a pre-defined task corresponding to digital gesture with higher accuracy level, when the accuracy level of both of the said digital gestures is above the first threshold value and between the second and first threshold values;
• generate said control signal based on said extracted pre-defined task; and
• transmit said control signal to said user device for performing said pre-defined task corresponding to said digital gesture with said higher accuracy level.
9. The system as claimed in claim 1, wherein said system includes a learning module configured to facilitate said user to add at least one new set of said micro and macro gestures via a user interface (122), and wherein said learning module is configured to store said added new set of micro and macro gestures and corresponding task in said repository (106).
10. A method (400) for performing tasks based on multi modal gestures, said method (400) comprising steps of:
• capturing micro gestures (402) provided by a user and generating micro gesture signals corresponding to said captured micro gestures, by a wearable unit (102);
• capturing macro gestures (404) provided by a user and generating macro gesture signals corresponding to said captured macro gestures, by an image capturing unit (104);
• conditioning (406), by a control unit (108), said generated macro and micro gesture signals to generate digital micro and macro gestures;
• performing analysis (408), by said control unit (108), of said digital micro and macro gestures based on pre-defined set of analyzing rules to identify accuracy level of each of said digital micro and macro gestures; and
• performing (410), by said control unit (108), at least one task corresponding to either of said digital micro gesture, said macro gesture, or both based on the identified accuracy level of each of said digital micro and macro gestures.
11. The method as claimed in claim 10, wherein said step of performing analysis (408) includes sub steps of:
• predicting (408-1), by a prediction module (114), the accuracy level of each of said digital micro and macro gestures by employing convolutional neural network;
• storing (408-2), in a memory (116), a first threshold value, a second threshold value, and a third threshold value, wherein said third threshold value is greater than said second threshold value, said second threshold value is greater than said first threshold value;
• comparing (408-3), by a comparator (118), the accuracy level of said each of digital micro and macro gestures with said threshold values; and
• generating (408-4), by a control component (120), a control signal based on said comparison of said digital micro and macro gestures with said threshold values.
12. The method as claimed in claim 11, wherein said step of generating (408-4) includes sub steps of:
• extracting a pre-defined task corresponding to said micro gesture, when the accuracy level is above the third threshold value;
• generating said control signal based on said extracted pre-defined task; and
• transmitting said control signal to a user device for performing said pre-defined task corresponding to said micro digital gesture.
13. The method as claimed in claim 11, wherein said step of generating (408-4) includes sub steps of:
• extracting a pre-defined task corresponding to combination of said digital micro and macro gestures, when the accuracy level of said digital micro gesture is below the third threshold value and said digital micro and macro gestures is above the second threshold value;
• generating said control signal based on said extracted pre-defined task; and
• transmitting said control signal to said user device for performing said pre-defined task corresponding to said combination of said digital micro and macro gestures.
14. The method as claimed in claim 11, wherein said step of generating (408-4) includes sub steps of:
• extracting a pre-defined task corresponding to digital gesture with higher accuracy level, when the accuracy level of both of the said digital gestures is above the first threshold value and between the second and first threshold values;
• generating said control signal based on said extracted pre-defined task; and
• transmitting said control signal to said user device for performing said pre-defined task corresponding to said digital gesture with said higher accuracy level.