Claims:WE CLAIM:
1. A method for controlling an Internet of Things (IoT) device using multi-modal gesture commands, the method comprising:
receiving, by an application server, one or more multi-modal gesture commands comprising at least one of one or more personalized gesture commands and one or more personalized voice commands of a user;
detecting, by the application server, the one or more multi-modal gesture commands using at least one of a gesture grammar database and a voice grammar database;
determining, by the application server, one or more control parameters and IoT device status information associated with a plurality of IoT devices in response to the detection;
identifying, by the application server, the IoT device that the user intends to control from the plurality of IoT devices based on user requirement, the IoT device status information, and line of sight information associated with the user; and
controlling, by the application server, the identified IoT device based on the one or more control parameters and the IoT device status information.
2. The method of claim 1, further comprising initiating a conversation with the user to receive additional information if at least one of the user requirement, the determined one or more control parameters and the IoT device status information are insufficient for identifying the IoT device that the user intends to control.
3. The method of claim 2, further comprising determining a mode of controlling the identified IoT device based on at least one of the additional information, the line of sight information, the user requirement, and the IoT device status information, wherein the mode of controlling the identified IoT device comprises a gesture command mode, a voice command mode, and a hybrid mode.
4. The method of claim 1, wherein the one or more multi-modal gesture commands are captured by each of the plurality of IoT devices using one or more sensors, wherein the one or more sensors comprise an image sensor, an audio sensor, and a haptic sensor.
5. The method of claim 4, further comprising controlling the identified IoT device from a remote location based on the one or more multi-modal gesture commands captured by each of the plurality of IoT devices using the one or more sensors.
6. The method of claim 1, wherein detection comprises performing at least one of: one or more image processing techniques and speech processing techniques on the received one or more multi-modal gesture commands.
7. The method of claim 1, wherein detection further comprises comparing the one or more personalized gesture commands with a gesture grammar database; and comparing one or more personalized voice commands with a voice grammar database to determine a match.
8. The method of claim 7, wherein each of the plurality of IoT devices are pre-configured by:
receiving a voice input from the user, wherein the voice input may be processed using speech processing techniques to identify the user;
assigning a unique name to each of the plurality of IoT devices based on an output of one or more natural language processing techniques implemented on the received voice input;
defining one or more multi-modal gesture commands to control each of the plurality of IoT devices, wherein the one or more multi-modal gesture commands comprise at least one of one or more personalized gesture commands and one or more personalized voice commands, wherein the gesture grammar database is created based on the defined one or more personalized gesture commands using one or more deep learning techniques; and wherein the voice grammar database is created based on the defined one or more personalized voice commands using the one or more deep learning techniques.
9. An application server to control an Internet of Things (IoT) device using multi-modal gesture commands, the application server comprising:
a processor; and
a memory communicatively coupled to the processor, wherein the memory stores processor instructions, which, on execution, causes the processor to:
receive one or more multi-modal gesture commands comprising at least one of one or more personalized gesture commands and one or more personalized voice commands of a user;
detect the one or more multi-modal gesture commands using at least one of a gesture grammar database and a voice grammar database;
determine one or more control parameters and IoT device status information associated with a plurality of IoT devices in response to the detection;
identify the IoT device that the user intends to control from the plurality of IoT devices based on user requirement, the IoT device status information, and line of sight information associated with the user; and
control the identified IoT device based on the one or more control parameters and the IoT device status information.
10. The application server of claim 9, wherein the processor is further configured to initiate a conversation with the user to receive additional information if at least one of the user requirement, the determined one or more control parameters and the IoT device status information are insufficient for identifying the IoT device that the user intends to control.
11. The application server of claim 10, wherein the processor is further configured to determine a mode of controlling the identified IoT device based on at least one of the additional information, the line of sight information, the user requirement, and the IoT device status information, wherein the mode of controlling the identified IoT device comprises a gesture command mode, a voice command mode, and a hybrid mode.
12. The application server of claim 9, wherein the one or more multi-modal gesture commands are captured by each of the plurality of IoT devices using one or more sensors, wherein the one or more sensors comprise an image sensor, an audio sensor, and a haptic sensor.
13. The application server of claim 12, wherein the processor is further configured to control the identified IoT device from a remote location based on the one or more multi-modal gesture commands captured by each of the plurality of IoT devices using the one or more sensors.
14. The application server of claim 9, wherein detection comprises performing at least one of: one or more image processing techniques and speech processing techniques on the received one or more multi-modal gesture commands.
15. The application server of claim 9, wherein detection further comprises comparing the one or more personalized gesture commands with a gesture grammar database; and comparing one or more personalized voice commands with a voice grammar database to determine a match.
16. The application server of claim 15, wherein each of the plurality of IoT devices are pre-configured by:
receiving a voice input from the user, wherein the voice input may be processed using speech processing techniques to identify the user;
assigning a unique name to each of the plurality of IoT devices based on an output of one or more natural language processing techniques implemented on the received voice input;
defining one or more multi-modal gesture commands to control each of the plurality of IoT devices, wherein the one or more multi-modal gesture commands comprise at least one of one or more personalized gesture commands and one or more personalized voice commands, wherein the gesture grammar database is created based on the defined one or more personalized gesture commands using one or more deep learning techniques; and wherein the voice grammar database is created based on the defined one or more personalized voice commands using the one or more deep learning techniques.

Dated this 27th day of March, 2017

Swetha S. N
Of K&S Partners
Agent for the Applicant , Description:TECHNICAL FIELD
The present subject matter is related, in general to controlling IoT (Internet of Things) devices, and more particularly, but not exclusively to a method and a system for controlling an IoT device using multi-modal gesture commands.