CLIAMS:CLAIMS
We claim:-

1. Method and system for real time hand gesture recognition of sign language alphabet and numerals for physically impaired people which combines vision based 2D & 3D hand gesture, motion tracking and face recognition with extended audiovisual facilities and the combine vision based hand gesture tracking system with face recognition provides a text input modality that does not require additional devices and allows users to give characters/symbols or any assigned operations to the computer or any machine /robot just by waving their hands in air towards it which extends the use of 2D & 3D hand gestures to allow for convenient and unrestricted text input the system uses MATLAB platform for programming and consists of three modules: vision based data acquisition, feature extraction, and hand gesture recognition and is practicable to integrate into Remote PC, ROBOT, TV, Virtual Games and other consumer electronics.

2. In the system as claimed in claim 1 for the purpose of Image acquisition:
a) the input images are captured by a webcam placed on a table or laptop;
b) The system is demonstrated on a conventional PC/ Laptop computer running on Intel Pentium Dual Processor with 4GB of RAM;
c) Each image has a spatial resolution of 620 x 480 pixels and a gray scale resolution of 32 bit;
d) The system developed can process hand gestures at an acceptable speed;
e) Given a variety of available image processing techniques and recognition algorithms, we have designed our preliminary process on detecting the image as part of our image processing;
f) The system starts by capturing a hand image from signer with a webcam setup towards certain angle with black background;
g) The next process will convert the RGB image into grey scale with either black (0) or white (1);
h) The edge of each object is then computed against the black background;
i) The object can then be segmented and differs greatly in contrast to the background images.

3. In the system as claimed in claim 1 for Pre-processing:
a) Changes in contrast can be detected by operators that calculate the gradient of an image;
b) One way to calculate the gradient of an image is the Sobel operator, which creates a binary mask using a user-specified threshold value;
c) The binary gradient mask shows lines of high contrast in the image;
d) These lines do not quite delineate the outline of the object of interest;
e) Compared to the original image, the gaps in the lines surrounding the object in the gradient mask can be seen;
f) The proposed system recognizes characters and symbols that are given in midair with the hand acting as the hand main object, from a Video camera or webcam;
g) Therefore, the gesture and motion of the hand is of interest;
h) To compute this motion, we used a Dynamic time wrapping approach to reconstruct gesture;
i) Based on the user's pose, we extract the palm motion and gesture of the hand.

4. In the system as claimed in claim 3 in order to deal with sensor noise and occasional misclassifications, we applied a median filter to smooth the gesture.

5. In the system as claimed in claim 1 for the purpose of Image Segmentation:
a) Image Pre-processing is necessary for getting good results;
b) In this algorithm, we take the RGB image as input image;
c) Image segmentation is typically performed to locate the hand object and boundaries in image;
d) It assigns label to every pixel in image such that the pixels which share certain visual characteristics will have the same label.

6. In the system as claimed in claim 1 for Feature extraction:
a) When gesture is given in air, users should tend to give on a video camera in front of them;
b) We exploit this video by projecting in to different frames of the hand gesture on a 2D plane, in front of the users, which has the advantage that it reduces the feature space;
c) The beginning and ending of hand gesture can be easily detected because system provides different time slot and notify the same to users, user move their hands, correspondingly;
d) We found that there were hand and hand movements while giving as is the case in traditional hand gesture;
e) Character and symbol recognition from hand gesture are both based on Centroids, which is an established technique in hand gesture recognition.

7. In the system as claimed in claim 1 for Hand gesture recognition:
a) Specially, the system is based on left to right Centroids;
b) First, for each character, a separate c is trained with training data of multiple people with the extracted features as observations;
c) Then, symbols can be modeled as a concatenation of character Centroids, i.e. for any given arbitrary symbol, and Centroids can be constructed;
d) A database denotes the set of symbols that can be actually recognized;
e) We examine two recognition tasks, the recognition of face and the recognition of character from hand gesture.
,TagSPECI:FORM 2
THE PATENT ACT 1970
(39 OF 1970)
AND
The patent rules, 2003
COMPLETE SPECIFICATION
(See section 10: rule 13)

1. TITLE OF INVENTION
Real Time Hand Gesture Recognition for Physically Impaired People

2 APPLICANT

Name Nationality Address
Sandip Foundation’s Sandip Institute of Technology & Research Centre Indian Sandip institute of Technology & Research Centre, Mahiravani, Trimbak road, Nashik, Maharashtra

3. PREAMBLE TO THE DESCRIPTION

COMPLETE
Following specification particularly describes the invention and the manner in which it is to be performed.

4. DESCRIPTION.

Technical field of invention:

Present invention in general relates to a method for real time hand gesture recognition of Indian sign language alphabet and numerals for physically impaired people and in particular to a system that combines vision based 2D & 3D hand gesture, motion tracking and face recognition.

Prior art:

Sign language is an efficient tool to support communication between aurally handicapped persons. By using the sign language, an aurally handicapped person can communicate directly with another aurally handicapped person being close to him or her with hand gestures, body gestures, face expressions etc. For remote communication transmission of the sign language is also possible using video cameras or videophone devices. Nevertheless, this approach requires a lot of training, but still does not allow communication with persons that are not capable of communicating in the sign language.

Personal communication skills are vital to a successful life. However, many millions of people suffer from impaired speaking and listening abilities. A significant majority of these people use hand sign language to communicate, such as the Sign Language, where letters are formed by various hand/finger/thumb/wrist combinations. It, therefore, becomes very difficult to converse with someone who doesn't know any such sign language. If users of Sign Language had a device that could readily translate from sign language to written or audible words, the process of communication would become much easier. Therefore, it would be very helpful for people with speaking disabilities to have, in particular, a Sign Language interpreter device to translate their finger spelling into readable text or audible speech.

Sign language translator is disclosed in US 20020152077 A1 is a method and apparatus for translation of hand positions into symbols. A glove for wearing on an operator's hand includes bend sensors disposed along the operator's thumb and each finger. Additional bend sensors are located between selected fingers and along the wrist. A processor generates a hand position signal using bend sensor signals read from the bend sensors and transmits the hand position signal to an output device. The output device receives the hand position signal and generates a symbol representative of the hand position signal using the received hand position signal and a lookup table of hand position signals associated with a set of symbols. The output device then produces either a visual or audio output using the symbol.

EP 1780625 A1 describes a Data input device and method and computer program product. It relates to the input of data to a terminal and relates in particular to the input and communication of data for aurally handicapped persons, in particular for deaf-mute people or under conditions where data input via speech or noise-generating keyboards is not suitable. Selected ones of five sensing means are actuated by selected ones of five fingers of a human hand during a data input time period to generate a 5-bit binary signal. The 5-bit binary signal is analysed and decoded into data in accordance with a predetermined encoding scheme; and the data are inputted into the terminal. In a first state the fingers actuate a sensing means. This encodes a first logical state. Non-actuation of the sensing means encodes a second logical state, which is the complementary logical state of the first logical state. Thus the logical states of all five fingers encode data as a 5-bit binary signal. The 5-bit binary signal is decoded in accordance with this predetermined encoding scheme. The method can be used for communicating with deaf and mute people or for silent data input and communication.

US 5659764 A discloses a sign language interpretation apparatus for performing sign language recognition and sign language generation generates easily read sign language computer graphics (CG) animation by preparing sign language word CG patterns on the basis of actual motion of the hand through the use of a glove type sensor to generate natural sign language CG animation, and by applying correction to the sign language word CG patterns. Further, in the sign language interpretation apparatus, results of translation of inputted sign language or voice language are confirmed and modified easily by the individual input persons, whereby results of translation of the inputted sign language or voice language are displayed in a combined form desired by the user to realize smooth communication. Also, candidates obtained as a result of translation are all displayed and can be selected easily by the input person with a device such as a mouse. Further, when a polysemous word is available, the word is displayed while being changed in its display form, and other expressions are confirmed and modified with the mouse.

Sign language recognition system and method is disclosed in US 8428643 B2. A sign language recognition system includes a menu generating module, an image processing module, and text processing module. The image processing module controls an image capturing unit to capture a gesture image of a target person when an image recognition icon is selected, extracts sign language data from the gesture image, and transmits the sign data packet to a server for analysing the sign language data to generate sign recognition information. The text processing module packs text data to be sent to a reception device, and a reception language and a phone number of the reception device to generate a text data packet when a text recognition icon is selected, and sends the text data packet to the server for analysing the text data to generate a sign recognition image recognizable by the reception device.

EP 848 552 B1 discloses a sign language telephone system comprising a camera, a sign language input means providing positional data of hand gestures and a videophone device. A sign language recognition unit is used to recognize and translate the input sign language. Recognition of hand gestures is, however, difficult and not efficient.

DE 102 33 233 A1 discloses another method for recognizing dynamic gestures, wherein differential information of gestures is obtained by capturing images of movable body parts, in particular of the fingers, at regular, equidistant time intervals.

US 5982853 A disclose Telephone for the deaf and method. An electronic communications system for the deaf includes a video apparatus for observing and digitizing the facial, body and hand and finger signing motions of a deaf person, an electronic translator for translating the digitized signing motions into words and phrases, and an electronic output for the words and phrases. The video apparatus desirably includes both a video camera and a video display which will display signing motions provided by translating spoken words of a hearing person into digitized images. The system may function as a translator by outputting the translated words and phrases as synthetic speech at the deaf person's location for another person at that location, and that person's speech may be picked up, translated, and displayed as signing motions on a display in the video apparatus.

Communication system for deaf, deaf-blind, or non-vocal individuals using instrumented glove is disclosed in US 5047952 A. It includes an instrumented glove for obtaining electrical signals indicative of a hand configuration of a first individual. Strain gage sensors in the glove flex with movement of the hand. Each sensor includes a tension strain gage and a compression strain gage which is serially connected and form two legs in a bridge circuit. Signals from the bridge circuit are amplified and digitized and applied to a computer which includes an adaptive pattern recognition algorithm which is responsive to hand-state vectors for recognizing letter beacons in hand-space. A second individual communicates with the first individual through the computer system using a portable keyboard. The output devices for communicating to the first and second individuals depend on the visual, vocal and hearing capabilities of the individuals and can be selected from a voice synthesizer, LCD monitor, or braille display.

Sign language telephone device is disclosed in US 6477239 B1. A sign language telephone device is offered which enables an aurally handicapped person who uses the sign language to converse with a normal person at a distant place who does not know the sign language. The sign language telephone device is placed on the side of the aurally handicapped person, and hand gestures of the sign language inputted from a sign language input means are recognized as the sign language, and the recognized sign language is translated to Japanese. The translated Japanese word train is converted to synthesized voices and it is transmitted to a videophone on the side of a normal person. The voices from the videophone are recognized, and the recognized Japanese is translated to the sign language to generate sign language animations and they are displayed on the screen of a TV set on the side of the aurally handicapped person. According to the present invention, it is made possible for an aurally handicapped person to have conversation easily with a normal person at a distant place who does not know the sign language through an existing network.

Accordingly, what is needed is a simple, cost-effective, hardware/software system of hand gesture recognition to minimize the barrio in the communication of aurally handicapped person to the normal one. Hence the present invention provides a method for real time hand gesture recognition of Indian sign language alphabet and numerals for physically impaired people. Such system lowers the communication gap between the deaf or mute community and the normal world.

Object:

1. Primary object of the present invention is to provide a method for real time hand gesture recognition of sign language alphabet and numerals for physically impaired people.
2. Another object of the present invention is to provide a system that combines vision based 2D & 3D hand gesture, motion tracking and face recognition with extended audiovisual facilities.
3. Yet another object of the present invention is to combine a vision based hand gesture tracking system with face recognition to provide a text input modality that does not require additional devices.
4. Yet another object of the present invention is to minimize the communication gap between the deaf or mute community and the normal world.
5. Yet another object of the present invention is to allow users to give characters/symbols or any assigned operations to the computer or any machine /robot just by waving their hands in air towards it.
6. Yet another object of the present invention is to provide us an innovative, natural, user friendly way of interaction with the computer which is more familiar to the human beings.
7. Yet another object of the present invention is to provide method is able to identify the images of the signer which are captured dynamically during testing phase.
8. Yet another object of the present invention is to interact with the physically challenged world.
9. Yet another object of the present invention is to help paralyzed person in day-to-day activities.
10. Yet another object of the present invention is to help ill/injured person to communicate with others.
11. Yet another object of the present invention is to communicate with old ones when on bed rest.
Other objects, features and advantages will become apparent from detail description and appended claims to those skilled in art.

STATEMENT:

Accordingly following invention provides a method for real time hand gesture recognition of sign language alphabet and numerals for physically impaired people. The present system combines vision based 2D & 3D hand gesture, motion tracking and face recognition with extended audiovisual facilities. The combine vision based hand gesture tracking system with face recognition provides a text input modality that does not require additional devices. This method aims to lower the communication gap between the deaf or mute community and the normal world. It allows users to give characters/symbols or any assigned operations to the computer or any machine /robot just by waving their hands in air towards it. This extends the use of 2D & 3D hand gestures to allow for convenient and unrestricted text input. The platform used for the programming is MATLAB. The proposed hand gesture recognition system consists of three modules: vision based data acquisition, feature extraction, and hand gesture recognition. This proposed system presents a low-cost hand gesture HCI system for the users of visually and Hearing Impaired. The proposed system is practicable to integrate into Remote PC, ROBOT, TV, Virtual Games and other consumer electronics.

BRIEF DESCRIPTION OF DRAWING:
This invention is described by way of example with reference to the following drawing where,
Figure 1 of sheet 1 shows the gesture hand gesture matching block diagram.
Where,
1 denotes Training
2 & 9 denotes Image Analysis Enhancement
3 & 8 denotes Feature Detector
4 denotes Template Name
5 denotes Database
6 denotes Template (L/N)
7 denotes Matcher
10 denotes Testing.

Figure 2 of sheet 2 shows the sequence diagram.
Where,
11 denotes Real-Time Images
12 denotes Camera Captures Image
13 denotes Find position of Hand Gesture
14 denotes Hand Gesture Recognized?
15 denotes Search for Information of Hand Gesture
16 denotes Display Information on PC
17 denotes No
18 denotes Yes.
Figure 3 A of sheet 3 shows Representation of ISL Alphabet

Figure 3 B of sheet 3 shows Representation of ISL Numerals

In order that the manner in which the above-cited and other advantages and objects of the invention are obtained, a more particular description of the invention briefly described above will be referred, which are illustrated in the appended drawing. Understanding that these drawing depict only typical embodiment of the invention and therefore not to be considered limiting on its scope, the invention will be described with additional specificity and details through the use of the accompanying drawing.

Detailed description:

Sign language is widely used by physically impaired people who cannot speak and hear or who can hear but cannot speak and is the only medium of communication for those people. It is nothing but the combination of various gestures formed by different hand shapes, movements and orientations of hands or body, facial expressions and lip-patterns for conveying messages. These gestures are widely used by the deaf and dumb people to express their thoughts. Usually physically impaired people need the help of sign language interpreters for translating their thoughts to normal people and vice versa. But it becomes very difficult to find a well experienced and educated translator for the sign language every time and everywhere in daily life, but human-computer interaction system for this can be installed anywhere possible. So a system recognizing the sign language gestures automatically is necessary which will help to minimize the gap between deaf people and normal people in the society. The development of a natural input device for creating sign language documents would make such documents more readable for deaf people. Moreover hearing people have difficulties in learning sign language and likewise the majority of those people who were born deaf or who became deaf early in life, have only a limited vocabulary of accordant spoken language of the community in which they live. Hence a system of translating sign language to spoken language would be of great help for deaf as well as for hearing people.
The motivation for developing such helpful application came from the fact that it would prove to be of utmost importance for socially aiding people and it would help increasingly for social awareness as well. Further, if we keep aside this world of computers and just take into consideration human- human interaction, we can realize that we are utilizing a broad range of gesture in personal communication. In fact gesturing is so deeply rooted in our communication that people often continue gesturing when speaking on the telephone. The significant use of gestures in daily life motivates the use of gestural interface in modern era.
As sign language is well structured code gesture, each gesture has a meaning assigned to it. There are number of sign languages spreader across the world. The sign language used by those deaf and mute at a particular place is dependent on the culture and spoken language at that place.

American Sign Language (ASL), British Sign Language (BSL), Japanese Sign Language family (Japanese, Taiwanese and Korean Sign Languages), French Sign Language family (French, Italian, Irish, Russian and Dutch Sign Languages), Australian Sign Language, etc. are the examples of regionally different sign languages. Indian sign language (ISL) is used by the deaf and dumb community in India and like countries. It consists of both word level gestures and finger spelling which is used to form words with letter by letter coding. The words for which no signs exist can be expressed with the use of letter by letter signing. It helps in recognizing the words for which the signer does not know the gestures or to emphasis or clarifies a particular word. So the finger spelling has key importance in sign language recognition. ISL differs in the syntax, phonology, morphology and grammar from other country’s sign languages. Designing a hand gesture recognition system for ISL is more challenging than other sign languages due to the following reasons.

1. Unlike other sign languages (American Sign Language, German Sign language) Indian Sign Language uses both hands to make sign.
2. Some signs involve overlapping of both the hands and complicated hand shapes.
3. One hand moves faster than the other at times in dynamic hand gestures.

Since ISL got standardized only recently and also since tutorials on ISL gestures were not available until recently, there are very few research work that has happened in ISL recognition. Here we propose a method for hand gesture recognition of Indian sign language alphabet and numerals. The signs considered for recognition include 26 letters of the English alphabet and the numerals from 0-9. Indian sign language alphabet and numerals are shown in Fig. 1 and Fig. 2 respectively.

Methodology:
With current advances in technology, we see a rapidly increasing availability, but also demand, for intuitive HCI. Devices are not only controlled by mouse and keyboard anymore, but we are now using gesture controlled devices in public areas and at our homes. Distant hand gestures in particular removed the restriction to operate a device directly; we can now interact freely with machines while moving around. In this work, we are interested in HCI that does not force users to touch a specific device or to wear special sensors, but that allows for unrestricted use. While there is a great variety of HCI techniques to interact with distant virtual objects, e.g. to select menu entries, there is still a lack for intuitive and unrestricted text input. Although there are ways to input text by using a virtual keyboard on a display or by speech recognition, there are situations where both are not suitable: the rest requires interaction with a display.

Users must speak which is not always possible demanding on the surroundings, especially for physically challenged people. With this work, we extend the available text input modalities by introducing an intuitive hand gesture recognition system. In the remainder of this work, we present a system that combines vision based 2D & 3D hand gesture, motion tracking and face recognition with extended audiovisual facilities. It allows users to give characters/symbols or any assigned operations to the computer or any machine /robot just by waving their hands in air towards it. This extends the use of 2D & 3D hand gestures to allow for convenient and unrestricted text input. The platform used for the programming is MATLAB. Initially, we used R2012a version but it raised problem in executing the command “imtool” so we moved to R2012b but that raised problem in executing the commands of video capturing and taking snapshots from running video. Then finally we moved to R2010a version which supports both of these commands and it runs in 64- bit computers.

Related Work:
Hand gesture recognition is not limited to paper or digital surfaces, but has also been extended to the third dimension. Hand gesture recognition is a problem that has elicited significant attention and research as computational capabilities, camera performance, and computer-vision- style learning algorithms have rapidly improved over the past few years. Such research is driven by the tremendous growth and variety in development of applications that require some form of gesture comprehension: these include remote hardware control, game controls, affective computing, and other endeavors in enhanced HCI. There are, however, fundamental limitations to most current systems for gesture detection based off training on a set of predefined gestures. Non-uniform lighting conditions and less- than-ideal camera resolution and depth of color limit the number and accuracy of possible gesture classifications in practice. Moreover, the modeling and analysis of hand gestures is complicated by the variegated treatment required for adequate detection of static gestures, which represent a combination of different finger states, orientations, and angles of finger joint that are often hidden by self-occlusion.
In computer vision (CV) based solutions such as ours, hand gestures are captured by web cameras which offer resolutions that allow only a general sense of the figure state to be detected. On the other hand, modeling gestures as temporal objects (emphasizing an understanding the movement of the hand as a pointer to the nature of the gesture) allows for greater accuracy and better differentiation of gestures. Additionally, the problem of hand-gesture recognition usually occurs in contexts where gestures involving finger conformation are accompanied by movement of the hands relative to the body and background.

In our work, we will combine a vision based hand gesture tracking system with face recognition to provide a text input modality that does not require additional devices. To the best of our knowledge, we are unaware of vision based hand gesture recognition of whole symbols based on concatenated individual character models.

VISION BASED HAND GESTURE RECOGNITION:
The proposed hand gesture recognition system consists of three modules: vision based data acquisition, feature extraction, and hand gesture recognition that will now be introduced in more detail.

Proposed system:
This proposed system presents a low-cost hand gesture HCI system for the users of visually and Hearing Impaired. The proposed system is practicable to integrate into Remote PC, ROBOT, TV, Virtual Games and other consumer electronics, benefiting from its three advantages:
a) The proposed system adds only a little to the hardware cost as just a USB webcam and audio player are used.
b) The proposed system can run secured and authenticable on mainstream and even highly populous area due to its face recognition feature.
c) The control commands issued by proposed system are converted to standard text display and Audio files simultaneously. Only drawbacks of this project are that it works only when it is trained in different environments.

Image acquisition:
The input images are captured by a webcam placed on a table or laptop. The system is demonstrated on a conventional PC/ Laptop computer running on Intel Pentium Dual Processor with 4GB of RAM. Each image has a spatial resolution of 620 x 480 pixels and a gray scale resolution of 32 bit. The system developed can process hand gestures at an acceptable speed. Given a variety of available image processing techniques and recognition algorithms, we have designed our preliminary process on detecting the image as part of our image processing. Hand detection preprocessing workflow is showed in Fig.

The system starts by capturing a hand image from signer with a webcam setup towards certain angle with black background. The next process will convert the RGB image into grey scale with either black (0) or white (1). The edge of each object is then computed against the black background. The object can then be segmented and differs greatly in contrast to the background images.

Pre-processing:
Changes in contrast can be detected by operators that calculate the gradient of an image. One way to calculate the gradient of an image is the Sobel operator, which creates a binary mask using a user-specified threshold value. The binary gradient mask shows lines of high contrast in the image. These lines do not quite delineate the outline of the object of interest. Compared to the original image, the gaps in the lines surrounding the object in the gradient mask can be seen.

The proposed system recognizes characters and symbols that are given in midair with the hand acting as the hand main object, from a Video camera or webcam. Therefore, the gesture and motion of the hand is of interest. To compute this motion, we used a Dynamic time wrapping approach to reconstruct gesture. Based on the user's pose, we extract the palm motion and gesture of the hand. There are also other ways to acquire this data, e.g. with sensors based on structured light like the Microsoft Kinect. But a vision based system has the advantage that there is much broader variety of sensors to choose from with varying resolutions and frame rates which can be important for the overall performance . To deal with sensor noise and occasional misclassifications, we applied a median filter to smooth the gesture.

Image Segmentation:
Image Pre-processing is necessary for getting good results. In this algorithm, we take the RGB image as input image. Image segmentation is typically performed to locate the hand object and boundaries in image. It assigns label to every pixel in image such that the pixels which share certain visual characteristics will have the same label.

Feature extraction:
When gesture is given in air, users should tend to give on a video camera in front of them. We exploit this video by projecting in to different frames of the hand gesture on a 2D plane, in front of the users, which has the advantage that it reduces the feature space. The beginning and ending of hand gesture can be easily detected because system provides different time slot and notify the same to users, user move their hands, correspondingly. We found that there were hand and hand movements while giving as is the case in traditional hand gesture. Character and symbol recognition from hand gesture are both based on Centroids, which is an established technique in hand gesture recognition.

Hand gesture recognition:
Specially, the system is based on left to right Centroids. First, for each character, a separate c is trained with training data of multiple people with the extracted features described above as observations. Then, symbols can be modeled as a concatenation of character Centroids, i.e. for any given arbitrary symbol, and Centroids can be constructed. A database denotes the set of symbols that can be actually recognized. We examine two recognition tasks, the recognition of face and the recognition of character from hand gesture. The general block diagram for matching of Hand gesture is given below.

Implication:
Sign language is a useful tool to ease the communication between the deaf or mute community and the normal people. Yet there is a communication barrier between these communities with normal people. This project aims to lower the communication gap between the deaf or mute community and the normal world.
This project was meant to be a prototype to check the feasibility of recognizing sign language using hand gestures. With this project the deaf or mute people can use the gestures to perform sign language and it will be converted in to speech so that normal people can easily understand. The main feature of this project is that the gesture recognizer is a standalone system, which is applicable in daily life.

Hardware Specification Details:
i Ball Face2Face C8.0 web camera with interpolated 8.0 MP still image resolution, 4.0MP Video resolution and 5G Wide angle lens provides smooth video and Let’s enjoy the clarity of web video.

Features / Specifications:
Features:
Interpolated 8.0 Mega Pixel Still Image Resolution
Interpolated 4.0 Mega Pixel Video Resolution
High quality 5G wide angle lens
6 LEDs for night vision, with brightness controller
Snapshot button for still image capture
Built-in high sensitive USB microphone
Built-in 10 Photo frames and 16 special effects for more fun
4X Digital Zoom and Auto Face Tracking Function
Multi-utility camera base for use on Monitors, LCDs and Laptops
Image Sensor: High quality 1/4 CMOS sensor
Max. Video Resolution: Max. upto 2304 x 1728 pixels
Max. Image Resolution: Max. upto 3264 x 2448 pixels
Frame Rates: 30 frames per second
Color Depth: 24-Bit True Color
Interface: USB 2.0, backward compatible to USB 1.1
Focus: 5 cm to Infinity
Microphone: Built-in high sensitive USB microphone
Snap Shot Button: Built-in snap shot button
White Balance: Auto
Auto Exposure: Auto
Auto Compensation: Auto
Auto Tracking: Auto face tracking function
Zoom: 4X Digital Zoom
Video Effects: 10 photo frames and 16 special effects
OS Compatibility: Windows XP / Vista / 7 & 8
Bundled Software: Driver for Windows with 10 Photo frames, 16 Special effects & Auto Face Tracking and more.
Effective Pixels: 480K pixels (Interpolated 8M pixels still image & 4M pixels video)
Night Vision: 6 LED's for night vision, with brightness controller
Low Light Boost: Automatic low light boost.

Software: MATLAB: MATLAB (matrix laboratory) is a multi-paradigm numerical computing environment and fourth-generation programming language. Developed by Math Works, MATLAB allows matrix manipulations, plotting of functions and data, implementation of algorithms, creation of user interfaces, and interfacing with programs written in other languages, including C, C++, Java, and FORTRAN.

Although MATLAB is intended primarily for numerical computing, an optional toolbox uses the MuPAD symbolic engine, allowing access to symbolic computing capabilities. An additional package, Simulink, adds graphical multi-domain simulation and Model-Based Design for dynamic and embedded systems. In 2004, MATLAB had around one million users across industry and academia. MATLAB users come from various backgrounds of engineering, science, and economics. MATLAB is widely used in academic and research institutions as well as industrial enterprises. MATLAB is a high-level language and interactive environment for numerical computation, visualization, and programming. Using MATLAB, you can analyze data, develop algorithms, and create models and applications. The language, tools, and built-in math functions enable you to explore multiple approaches and reach a solution faster than with spreadsheets or traditional programming languages, such as C/C++ or Java. You can use MATLAB for a range of applications, including signal processing and communications, image and video processing, control systems, test and measurement, computational finance, and computational biology. More than a million engineers and scientists in industry and academia use MATLAB, the language of technical computing.

Key Features:
• High-level language for numerical computation, visualization, and application development
• Interactive environment for iterative exploration, design, and problem solving
• Mathematical functions for linear algebra, statistics, Fourier analysis, filtering, optimization, numerical integration, and solving ordinary differential equations
• Built-in graphics for visualizing data and tools for creating custom plots
• Development tools for improving code quality and maintainability and maximizing performance
• Tools for building applications with custom graphical interfaces
• Functions for integrating MATLAB based algorithms with external applications and languages such as C, Java, .NET, and Microsoft Excel.

Additional advantages and modification will readily occur to those skilled in art. Therefore, the invention in its broader aspect is not limited to specific details and representative embodiments shown and described herein. Accordingly various modifications may be made without departing from the spirit or scope of the general invention concept as defined by the appended claims and their equivalents.

Adv. Swapnil Gawande (IN/PA/1587)

CLAIMS
We claim:-

1. Method and system for real time hand gesture recognition of sign language alphabet and numerals for physically impaired people which combines vision based 2D & 3D hand gesture, motion tracking and face recognition with extended audiovisual facilities and the combine vision based hand gesture tracking system with face recognition provides a text input modality that does not require additional devices and allows users to give characters/symbols or any assigned operations to the computer or any machine /robot just by waving their hands in air towards it which extends the use of 2D & 3D hand gestures to allow for convenient and unrestricted text input the system uses MATLAB platform for programming and consists of three modules: vision based data acquisition, feature extraction, and hand gesture recognition and is practicable to integrate into Remote PC, ROBOT, TV, Virtual Games and other consumer electronics.

2. In the system as claimed in claim 1 for the purpose of Image acquisition:
a) the input images are captured by a webcam placed on a table or laptop;
b) The system is demonstrated on a conventional PC/ Laptop computer running on Intel Pentium Dual Processor with 4GB of RAM;
c) Each image has a spatial resolution of 620 x 480 pixels and a gray scale resolution of 32 bit;
d) The system developed can process hand gestures at an acceptable speed;
e) Given a variety of available image processing techniques and recognition algorithms, we have designed our preliminary process on detecting the image as part of our image processing;
f) The system starts by capturing a hand image from signer with a webcam setup towards certain angle with black background;
g) The next process will convert the RGB image into grey scale with either black (0) or white (1);
h) The edge of each object is then computed against the black background;
i) The object can then be segmented and differs greatly in contrast to the background images.

3. In the system as claimed in claim 1 for Pre-processing:
a) Changes in contrast can be detected by operators that calculate the gradient of an image;
b) One way to calculate the gradient of an image is the Sobel operator, which creates a binary mask using a user-specified threshold value;
c) The binary gradient mask shows lines of high contrast in the image;
d) These lines do not quite delineate the outline of the object of interest;
e) Compared to the original image, the gaps in the lines surrounding the object in the gradient mask can be seen;
f) The proposed system recognizes characters and symbols that are given in midair with the hand acting as the hand main object, from a Video camera or webcam;
g) Therefore, the gesture and motion of the hand is of interest;
h) To compute this motion, we used a Dynamic time wrapping approach to reconstruct gesture;
i) Based on the user's pose, we extract the palm motion and gesture of the hand.

4. In the system as claimed in claim 3 in order to deal with sensor noise and occasional misclassifications, we applied a median filter to smooth the gesture.

5. In the system as claimed in claim 1 for the purpose of Image Segmentation:
a) Image Pre-processing is necessary for getting good results;
b) In this algorithm, we take the RGB image as input image;
c) Image segmentation is typically performed to locate the hand object and boundaries in image;
d) It assigns label to every pixel in image such that the pixels which share certain visual characteristics will have the same label.

6. In the system as claimed in claim 1 for Feature extraction:
a) When gesture is given in air, users should tend to give on a video camera in front of them;
b) We exploit this video by projecting in to different frames of the hand gesture on a 2D plane, in front of the users, which has the advantage that it reduces the feature space;
c) The beginning and ending of hand gesture can be easily detected because system provides different time slot and notify the same to users, user move their hands, correspondingly;
d) We found that there were hand and hand movements while giving as is the case in traditional hand gesture;
e) Character and symbol recognition from hand gesture are both based on Centroids, which is an established technique in hand gesture recognition.

7. In the system as claimed in claim 1 for Hand gesture recognition:
a) Specially, the system is based on left to right Centroids;
b) First, for each character, a separate c is trained with training data of multiple people with the extracted features as observations;
c) Then, symbols can be modeled as a concatenation of character Centroids, i.e. for any given arbitrary symbol, and Centroids can be constructed;
d) A database denotes the set of symbols that can be actually recognized;
e) We examine two recognition tasks, the recognition of face and the recognition of character from hand gesture.

Adv. Swapnil Gawande (IN/PA/1587)

Sandip Foundation’s Sandip Institute of
Technology & Research Centre Sheet 3/1

Figure 1

Adv. Swapnil Gawande (IN/PA/1587)

Sandip Foundation’s Sandip Institute of
Technology & Research Centre Sheet 3/2

Figure 2

Adv. Swapnil Gawande (IN/PA/1587)

Sandip Foundation’s Sandip Institute of
Technology & Research Centre Sheet 3/3

Fig. 3 A

Fig. 3 B.

Adv. Swapnil Gawande (IN/PA/1587)
ABSTRACT

Sign language is the language of communication for deaf and dumb people. Most of these physically impaired communities are dependent on sign language translators to express their thoughts to rest of the world. This causes isolation of these people in society. Hence, Sign Language Recognition is one of the most growing fields of research today. A sign language is composed of various gestures formed by physical movement of body parts i.e. hand, arms or facial expressions. In this paper, a method is proposed that makes the use of hand gestures for recognition of Indian Sign Language. Hand Gesture recognition system provides us an innovative, natural, user friendly way of interaction with the computer which is more familiar to the human beings. The proposed method is able to identify the images of the signer which are captured dynamically during testing phase. To implement this approach we have utilized a simple web camera to capture hand gesture images. Artificial neural network is used for recognizing different signs and translate them into text and voice format. Following invention is described in detail with the help of Figure 1 of sheet 1 showing the gesture hand gesture matching block diagram and Figure 2 of sheet 2 showing sequence diagram.