Claims:We Claim

1. A method for classifying a hand gesture based on an electronic signal that is received by a depth camera device, by means of the depth camera device, the method comprising:
obtaining 10 a depth image of a target object; characterized in that
determining 12 whether the image of the target object is greater than a minimum critical displacement and lesser than a maximum critical displacement from the depth camera device;
subtracting 14 an image of a background data object from the image of the target object to obtain a difference image of the target object if the image of the target object is greater than a minimum critical displacement and lesser than a maximum critical displacement from the depth camera device;
determining 16 if the target object is that of a human hand; and
determining 18 hand movement by flow estimator if the target object is that of a human hand in order to determine if the target object is one of stationary and in a state of motion.

2. The method for classifying a hand gesture in accordance with Claim 1 wherein movement of the target object being detected is considered as a dynamic hand gesture 25, if the target object is in a state of motion.

3. The method for classifying a hand gesture in accordance with Claim 2 further comprising:
determining 20 a centroid movement of the human hand;
tracking 22 the centroid movement of the human hand in each frame of the depth camera device;
determining 24 the centroid movement of the human hand in the x-direction and in the y-direction.

4. The method for classifying a hand gesture in accordance with Claim 3 wherein based on the movement of the centroid of the human hand in the x-direction and the movement of the centroid of the human hand in the y-direction, it is determined 26 that the gesture of the human hand is one of swipe left, swipe right, swipe up, and swipe down.

5. The method for classifying a hand gesture in accordance with Claim 4 wherein the output of the determination of the gesture of the human hand being one of swipe left, swipe right, swipe up, and swipe down is transmitted to a voting engine 28.

6. The method for classifying a hand gesture in accordance with Claim 1 wherein if movement of the target object is not detected, it is considered as a static hand gesture 30, wherein the target object is stationary.

7. The method for classifying a hand gesture in accordance with Claim 6 further comprising:
checking 32 for extrema count and location in the hand contour;
determining 34 the hand gesture to be one of: one extrema peak on a right hand side of the hand contour currosponding to thumb right, no extrema corresponding to fist, two extrema peaks corresponding to palm down, one extrema on peak corresponding to index finder pointed outwardly, and one extrema on left hand side of hand contour corresponding to thumb left; and
transmitting the determination of the hand gesture to be one of one extrema peak on a right hand side of the hand contour, no extrema, two extrema peaks, one extrema on peak, and one extrema on left hand side of hand contour to a voting engine 28.

8. The method for classifying a hand gesture in accordance with Claim 1 further comprising:
determining 20 a centroid movement of the human hand;
track 22 the centroid movement of the human hand in each frame of the depth camera device;
determine 24 the centroid movement of the human hand in the x-direction and in the y-direction;
input 36 comprising the three channel image in which the first channel is the flow of the centroid in the x-direction, the second channel comprising the flow of the centroid in the y-direction, and the third channel comprising the threshold input image which contains the reference shape of the hand for each frame of the depth camera device;
convert 40 the first channel which is the flow of the centroid in the x-direction, the second channel comprising the flow of the centroid in the y-direction, and the third channel comprising the threshold input image which contains the reference shape of the hand into a format that can me inputted into a neural network;
transmit 42 the first channel which is the flow of the centroid in the x-direction, the second channel comprising the flow of the centroid in the y-direction, and the third channel comprising the threshold input image which contains the reference shape of the hand to a tensorflow prediction system;
determine 44 a confidence value for occurrence of the all possible hand gestures;
transmit the confidence value for occurrence of all possible hand gestures to a voting engine 28.

9. The method for classifying a hand gesture in accordance with Claims 5, 7, and 8 further comprising determining 50 a maximum confidence that a given hand gesture is a static hand gesture.

10. The method for classifying a hand gesture in accordance with Claims 5, 7, and 8 further comprising determining 50 a maximum confidence that a given hand gesture is a dynamic hand gesture.
, Description:Complete Specification:

The following specification describes and ascertains the nature of this invention and the manner in which it is to be performed.
Field of the invention
[0001] This invention relates to a method for classifying a hand gesture based on an electronic signal that is received by a depth camera device, and more specifically to a method for classifying a hand gesture for controlling an electronic device by means of the depth camera device.

Background of the invention
[0002] US 6788809 B1 describes a system and method for recognizing gestures. The method comprises obtaining image data and determining a hand pose estimation. A frontal view of a hand is then produced. The hand is then isolated from the background. The resulting image is then classified as a type of gesture. In one embodiment, determining a hand pose estimation comprises performing background subtraction and computing a hand pose estimation based on an arm orientation determination. In another embodiment, a frontal view of a hand is then produced by performing perspective unwarping and scaling. The system that implements the method may be a personal computer with a stereo camera coupled thereto.

Brief description of the accompanying drawing
[0003] Figure 1 illustrates a flow chart depicting a method of classifying a hand gesture for controlling an electronic device in one embodiment of the invention.

Detailed description of the embodiments
[0004] Figure 1 illustrates a flow chart 1 illustrating the method of classifying a hand gesture for controlling an electronic device in one embodiment of the invention. The method for classifying the hand gesture based on an electronic signal received by a depth camera device by means of the depth camera device comprises obtaining 10 an image of a target object, and determining 12 whether the image of the target object is greater than a minimum critical displacement and lesser than a maximum critical displacement from the depth camera device. The method further comprises subtracting 14 an image of a background data object from the image of the target object to obtain a difference image of the target object if the image of the target object is greater than a minimum critical displacement and lesser than a maximum critical displacement from the depth camera device, determining 16 if the target object is that of a human hand, and determining 18 hand movement by flow estimator if the target object is that of a human hand in order to determine if the target object is one of stationary and in a state of motion.

[0005] The method for classifying a hand gesture based on an electronic signal that is received by a depth camera device, by means of the depth camera device comprises obtaining 10 an image of a target object. More specifically, the depth camera device is used to capture and obtain an image of the target object for further post processing. Once the image of the target object is captured and obtained by the depth camera device, the depth camera device determines 12 whether the image of the target object is greater than a maximum critical displacement and lesser than a minimum critical displacement from the depth camera device. More specifically, if the target object is greater than the maximum critical displacement, then the target object is close to or in contact with a control surface. Therefore the image of the target object is discarded from the memory of the depth camera device. If the target object is lesser than the minimum critical displacement, then the target object is close to or in contact with the depth camera device. Therefore, the image of the target object is discarded from the memory of the depth camera device.

[0006] If the target object is greater than the minimum critical displacement and lesser than the maximum critical displacement, then the hand gesture is considered for controlling the electronic device. An image of a background data object is subtracted 14 from the image of the target object to obtain a difference image of the target object if it is determined that the image of the target object is greater than a minimum critical displacement and lesser than a maximum critical displacement from the depth camera device. Therefore, all the background noise may be eliminated and the image of the hand gesture is under focus for performing a decision to control the electronic device. Therein, the depth camera device determines 16 if the target object is that of a human hand by determining if the target object has peaks and valleys. After it has been ascertained that the target object is that of a human hand, the depth camera device detects hand movement by means of a flow estimator 18 if the target object is that of a human hand. If the depth camera device detects hand movement by means of the flow estimator, it is therein determined that the target object is in a state of motion. However, if the depth camera device detects no hand movement by means of the flow estimator 18, it is therein determined that the target object is stationary.

[0007] When the target object is in a state of motion, the movement of the target object is considered as a dynamic hand gesture 25. If the target object is stationary, the movement of the target object is considered as a static hand gesture 30. If the target object is considered as a dynamic hand gesture 25, it is required to determine the centroid movement of the human hand in the x-direction and in the y-direction to determine the type of hand gesture such as swipe hand left, swipe hand right, swipe hand up, and swipe hand down. The method to determine the centroid movement of the human hand in the x-direction and in the y-direction to determine the type of hand gesture is to primarily determine the centroid movement of the human hand 20, and tracking the centroid movement of the human hand in each frame 22 of the depth camera device. Therein, the depth camera device determines the centroid movement of the human hand in the x-direction and in the y-direction 24 respectively.

[0008] Based on the movement of the centroid of the human hand in the x-direction and in the y-direction, it is determined 26 by the depth camera device that the gesture of the human hand is one of swipe left, swipe right, swipe up, and swipe down respectively. The output of the determination 26 of the gesture of the human hand being one of swipe left, swipe right, swipe up, and swipe down is transmitted to a voting engine 28 to determine 26 whether the gesture of the human hand is swipe left, swipe right, swipe up, or swipe down.

[0009] If the target object is considered as a static hand gesture 30, it is required to determine 34 the type of hand gesture such as one extrema peak on a right hand side of the hand contour, no extrema, two extrema peaks, one extrema on peak, and one extrema on left hand side of hand contour respectively. The method for classifying the hand gesture further comprises checking for extrema count and location in the hand contour. By determining the extrema count and location in the hand contour, it is determined by the depth camera device that the hand gesture is one of one extrema peak on a right hand side of the hand contour, no extrema, two extrema peaks, one extrema on peak, and one extrema on left hand side of hand contour. The determination of the hand gesture to be one of one extrema peak on a right hand side of the hand contour, no extrema, two extrema peaks, one extrema on peak, and one extrema on left hand side of hand contour is then transmitted to a voting engine 28 to determine whether the gesture of the human hand is one of one extrema peak on a right hand side of the hand contour, no extrema, two extrema peaks, one extrema on peak, and one extrema on left hand side of hand contour.

[0010] Once it is determined whether the hand gesture is one of a static hand gesture and a dynamic hand gesture, it is required to validate the hand gesture by means of an alternate approach which will corroborate with confidence the occurrence of a specific hand gesture. The method for validating whether a hand gesture is one of a static hand gesture or a dynamic hand gesture comprises determining 20 a centroid movement of the human hand, and tracking 22 the centroid movement of the human hand in each frame of the depth camera device. The neural network is inputted with a three channel image which comprises a first channel as a flow of the centroid in the x-direction, a second channel comprises a flow of the centroid in the y-direction, and a third channel comprises a threshold input image which contains a reference shape of the hand. Once the depth camera device determines the three channel image which comprises the first channel as the flow of the centroid in the x-direction, the second channel comprises the flow of the centroid in the y-direction, and the third channel comprises the threshold input image which contains the reference shape of the hand, the first channel as the flow of the centroid in the x-direction, the second channel comprising the flow of the centroid in the y-direction, and the third channel comprising the threshold input image which contains the reference shape of the hand is converted 40 into a format that can be inputted into a neural network.

[0011] The first channel as the flow of the centroid in the x-direction, the second channel comprising the flow of the centroid in the y-direction, and the third channel comprising the threshold input image which contains the reference shape of the hand is transmitted to a tensorflow prediction system which would help predict 44 a confidence value for all the possible static and dynamic hand gestures. The confidence value from the neural network, the flow estimator dynamic hand gestures, and the static hand gestures is therein transmitted to a voting engine 28 to determine a maximum confidence of the occurrence of each hand gesture. The voting engine 28 then determines a maximum confidence for the occurrence of each hand gesture. If the given hand gesture is a static hand gesture 30, it is compared with the results of the flow estimator 18 to confirm that the given hand gesture is indeed a static hand gesture. If the given hand gesture is a dynamic hand gesture 25, it is compared with the results of the flow estimator 18 to confirm that the given hand gesture is indeed a dynamic hand gesture. Once the two results match and it is determined that the given hand gesture is a static hand gesture, then if the hand gesture is one extrema on right hand side of the hand contour, then a first control on the electronic device is transmitted by the depth camera device to perform a first specific function corresponding to that hand gesture. If the hand gesture is no extrema, then a second control on the electronic device is transmitted by the depth camera device to perform a second specific function corresponding to that hand gesture. If the hand gesture is one extrema on peak, then a third control on the electronic device is transmitted by the depth camera device to perform a third specific function corresponding to that hand gesture.

[0012] Similarly, once the two results match and it is determined that the given hand gesture is a dynamic hand gesture 25, then if the hand gesture is swipe left, then a control is transmitted to the electronic device perform a specific function corresponding to that hand gesture. If the hand gesture is swipe right, then a control is transmitted to the electronic device to perform a specific function corresponding to that hand gesture. If the hand gesture is swipe up, then a control is transmitted to the electronic device to perform a specific function corresponding to that hand gesture. If the hand gesture is swipe down, then a control is transmitted to the electronic device to perform a specific function corresponding to that hand gesture. From this stage, the depth camera device control loops back to the step of obtaining an image 10 of a target object and the process repeats once more.

[0013] It must be understood that the embodiments explained above are only illustrative and do not limit the scope of the disclosure. Many modifications in the embodiments with regard to dimensions of various components are envisaged and form a part of this invention. The scope of the invention is only limited by the scope of the claims.