Abstract: An interactive device interfaced between a set top box and at least one display device, said device comprising: • blending means co-operating with said set top box to receive video content, said blending means adapted to blend pre determined textual content with said video content to form blended content; • graphical interface cooperating with blending means to receive said blended content, said graphical interface further adapted to output the received blended content on to said display device; • a user interface adapted to enable a user to interact with said interactive device; • a machine vision unit cooperating with blending means to receive said blended content, said machine vision unit adapted to detect the coordinates of a frame of an active page containing said blended content; • storage means adapted to store the coordinates of said frame of an active page; • pre processing means adapted to recognize the active page displayed on said display device, said pre processing means cooperating with storage means to receive the coordinates of said frame of the active page, said preprocessing means adapted to separate the background image and foreground image of said frame and enhance said foreground image; • character recognition means cooperating with preprocessing means to receive said enhanced foreground image, said character recognition means further adapted to extract the alpha numeric character set embedded in said enhanced foreground image; • communication means cooperating with character recognition means, said communication means further adapted to formulate the message text and address and transmit the text message to said address over a computer network; • receiving means cooperating with user interface to receive the signals.
FORM-2
THE PATENTS ACT, 1970
(39 of 1970)
&
THE PATENTS RULES, 2003
COMPLETE SPECIFICATION
(See section 10 and rule 13)
SYSTEM FOR SMS VALUE ADDED SERVICE FOR ACTIVE TELEVISION SHOWS RECEIVED VIA A SET TOP BOX
TATA CONSULTANCY SERVICES LTD.
an Indian Company of Nirmal Building, 9th floor, Nariman Point, Mumbai 400 021,
Maharashtra, India
The following specification particularly describes the invention and the manner in which it is to be performed.
FIELD OF THE INVENTION
The present invention relates to the field of telecommunication. Particularly, the present invention relates to the field of recognition of active services shown on a TV channel.
DEFINITIONS OF TERMS USED IN THE SPECIFICATION
Active Page - Active Page includes the visual content being displayed on a display screen at a particular instance of time.
Frame - Frame is that region of active page, where the textual content is superimposed on video content.
Set top box (STB) - A set-top box (STB) is a device that connects to a television and an external source of signal, turning the signal into content which is then displayed on the television screen.
Interactive device - An interactive device includes any device, gadget,
equipment or instrument which is interfaced between the set top box and a display device and is further configured to provide certain functions in addition to the functions provided by the set top box. For instance the services provided by the interactive device may include automatic sending of message, facility to browse the internet on the television set etc.
SMS - SMS (Short Message Service) is the text communication service
component that allows the exchange of short text messages between the devices coupled to a computer network or mobile communication network.
Message - A message is an object of communication. A message contains
data in text format. The data in text format may include consumer information such as consumer ID and product ID, user confirmation code, request to access a particular service or product and the like.
URL - URL stands for Universal Resource Locator. A universal resource
locator denotes the address of a web page located on World Wide Web.
These definitions are in addition to those expressed in the art.
BACKGROUND OF THE INVENTION AND PRIOR ART
Today, active services are the next big thing in DTH (Direct-To-Home) services. Active services are interactive services which offer host of products and offers to the consumer's at the convenience of their homes while watching the television.
Many active services are being offered by DTH providers including sports, online banking, shopping, jobs and the like. These services enable the consumers to avail various facilities at home using the SMS facility (Short Messaging Service) for instance; the consumers can apply for jobs and buy
products by sending a message having the code of the service as seen on the television screen to a predetermined mobile number. Many a times, the services also provide consumers with URLs (Uniform Resource Locator) where the consumers can log in and view the detailed specification of the products. However, it becomes cumbersome for the consumers to take note of the URLs and the SMS details which are flashed on the television screen for a few seconds. In addition, the consumers have to switch from one medium (television) to another (computer/laptop/mobile) to avail the services. It is seen that even though the consumers note the details of the service, very few of them actually log on to the URL or send a message.
United States patent numbered US7346917 granted to Cyberview Technologies describes a set top box containing a trusted transactional cache and corresponding transactional protocols. The set top box provides for efficient and secure e-commerce transactions to be committed to a remote server. The set top box makes use of a network interface to access the computer network. The prior art document discloses a method of conducting an online transaction between a set top box and a remote server comprising the steps of coupling the set top box and the remote server via a network interface, initiating a transaction at the set top box, sending the transaction to the remote server via the network interface under a control of the set top box and receiving the corresponding service or the corresponding acknowledgement. However above mentioned United States prior art document does not enable the user to send messages automatically from the set top box
United States Patent Application 2010/0058416 discloses a method and system for providing web based content feed for a set top box. A network infrastructure coupled to the set top box is utilized for sending, receiving of signals and also for receiving web based content. The network infrastructure may provide wired or wireless connectivity to the set top box. The document also discloses that the set top box module relies upon the said network infrastructure for communications. However the invention discloses receiving web based content feed on the set top box, which has been coupled to a network infrastructure. However, above mentioned United States prior art document does not provide the user with the facility of automatically sending messages through the set top box.
United States Patent 6349410 discloses a method of coordinating the display of an incoming signal stream on a display with web browsing. Disclosed method involves integration of broadcast television and web content. The invention combines the features of a computer system with a broadcast TV pause/relay device and a cable TV set top box. The set top device is connected to a network such as internet and also to a broadcast television signal source. The web based content are received through the internet and the TV content are received either in analog or digital form from the broadcast TV signal source. The set top box is capable of receiving, storing, rendering the web content as well as TV content. The remote control associated with the said set top box houses input controls named TV and INTERNET amongst other input controls. The input controls named FORWARD and BACKWARD, available with the remote control are used to traverse between television broadcast content, static web content and streaming internet content. According to the disclosed system, when the user
. can watch the television broadcast content by pressing 'TV button available on the remote control, and while watching the television broadcast content, if he desires to view web content available on a URL that has been integrated onto television broadcast stream, he could do so by first pressing the 'PAUSE' button on the remote control to pause the television broadcast signal and then press the 'INTERNET' button to go to the most recently available web content. However above mentioned United States prior art document does not disclose providing user with the facility of sending messages automatically through the set top box.
United States Patent 6181326 provides a set top box which is capable of providing web based content on the television screen based on the URL being provided as input by the viewer. The set top box is capable of receiving a remote control signal consisting of infrared radiation or radio wave from a remote input unit such as optical character reader, bar code reader, handwriting input remote controller and keyboard input remote controller. The set top box provides user the option of providing the URL as input through a keyboard or through a television remote control or through a bar code reader or through an OCR. The microprocessor of the set top box is configured to display the character data on the television screen and ask for a user confirmation. If the user confirms that the character data is indeed the address of the URL he desired to access, the microprocessor instructs the modem, which has been connected to the internet or a network of computers through the telephone line to access the external terminal corresponding to the said URL. The data available with the accessed URL is then displayed on the television display unit. However above mentioned United States prior art
document does not provide user with the facility of automatically sending the message through the set top box.
United States Patent 7673320 provides a system that automatically detects the commercial messages associated with the television broadcast, automatically acquires the details of the reference site corresponding to a particular commercial message and sets up the connection to the reference site of the corresponding commercial message automatically. The information transmitting/receiving device automatically detects the commercial messages associated with the television broadcast. The consumer terminal (display unit) receives the television broadcast feed as well as the commercial message in real time. A commercial message server provided as a part of the system provides the option of recording the incoming commercial messages for later viewing. In the event of viewer expressing interest in a particular commercial message, the user terminal notifies the access information furnishing device, which establishes a connection to the corresponding URL through internet. After getting a connection to the corresponding URL through the internet, the detailed information sourced at the remote URL is provided to the user terminal through the network connection. However the invention provides set top box that provides web based content on the television. But above mentioned United States prior art document does not provide the user with the facility of sending messages through the set top box to the desired destination.
Chinese Patent Application 101673386 provides a community television business system which comprises of a service front terminal, a transmission
network and a user terminal. The disclosed system provides an online retail shop for buyers and sellers by establishing a community television business platform. The system broadcasts the merchandise multimedia information to the users of the set top box through the IP network or the HFC (Hybrid fiber coaxial) network and subsequently sends the booking information selected by the user automatically to the corresponding merchant/distributing mechanism in the form of short messages (SMS) through the wireless network or telephone network or through SMS gateway. However the functionalities provided by the invention are restricted to a web based B2C electronic commerce model. According to the invention whenever a particular user selects a particular product or service for purchase, a text message (SMS) containing the booking information (preferably customer details such as customer ID, customer telephone number, money payable by customer and product details such as product category, product price etc) is automatically sent to the corresponding service provider for further processing. Even though the SMS is sent to the preferred destination through the set top box, the customer information and product information are extracted from a database in response to user selecting a product for purchase, and sends the corresponding booking information in the form of an SMS. The Chinese document does not disclose extracting the message text and address embedded in live television content, and the above mentioned Chinese prior art document does not provide user with the facility of automatically sending the message to the preferred destination through the set top box.
There is therefore a need for a system which will enable users to send messages over the computer network and browse the internet while they are
watching their favorite television shows. Moreover, there is need for a system which will help in increasing the interactivity of the consumers and . increase the usage of the facilities provided by the active services.
Currently, there are value added services on STB provided by the DTH providers, but the automation of sending messages (SMS) is missing, hence the user needs another device to manually enter the text and send the message. Therefore there is a need for a system to automate the entire process and provide a single point of interface for availing all active services.
The interactive device of instant invention is different from the prior and related art, in that it provides the user with the facility of automatically sending messages through the interactive device, without the requirement of user having to type the text message and the corresponding address on the mobile phone. The invention includes an interactive device coupled to a computer network through which the messages are sent in response to user pressing a pre determined key located on the remote control device. None of the above mentioned patents and inventions taken either singly or in combination is seen to describe the instant invention as claimed
OBJECTS OF THE INVENTION
It is an object of the present invention to provide a single system which will facilitate sending of SMS (Short Message Services) and browsing Internet through the television while watching active television shows.
It is another object of the present invention to provide a system which enhances the user interactivity while watching active television shows.
It is still another object of the present invention to provide a system which generates revenue for the telecom service providers by simplifying the method of sending an SMS or a message while watching active television
shows.
It is yet another object of the present invention to provide a system which allows the users to instantly subscribe for the active services available with the service provider.
It is yet another object of the present invention to automate the process of subscribing to active services advertised on the television.
It is yet another object of the present invention to automatically recognize the active page using remote control key press event tracking.
It is yet another object of the present invention to recognize the active page based on image processing based approach.
SUMMARY OF THE INVENTION
In accordance with the present invention, there is provided a system for efficiently sending message while watching active/ live television programs or playing games using the television connected to a Set Top Box (STB).
The invention envisages a system for recognition of the text messages and their addresses shown in active shows or live program on televisions which
are connected to an STB and sending the recognized text message to the recognized address.
Moreover, the present invention provides a system which enables consumers to visit the website of the URL shown in the active television shows and send a message to the address/ mobile number simultaneously while watching active television shows.
In accordance with the present invention, "A system for providing SMS value added service on an STB" denoted by 'interactive device' comprises: 1. An interactive device interfaced between a set top box and at least one display device, said device comprising:
• blending means co-operating with said set top box to receive video content, said blending means adapted to blend pre determined textual content with said video content to form blended content;
• graphical interface cooperating with blending means to receive said blended content, said graphical interface further adapted to output the received blended content on to said display device;
• a user interface adapted to enable a user to interact with said interactive device;
• a machine vision unit cooperating with blending means to receive said blended content, said machine vision unit adapted to detect the coordinates of a frame of an active page containing said blended content;
• storage means adapted to store the coordinates of said frame of an active page;
• pre processing means adapted to recognize the active page displayed on said display device, said pre processing means cooperating with
storage means to receive the coordinates of said frame of the active page, said preprocessing means adapted to separate the background image and foreground image of said frame and enhance said foreground image;
• character recognition means cooperating with preprocessing means to receive said enhanced foreground image, said character recognition means further adapted to extract the alpha numeric character set embedded in said enhanced foreground image;
• communication means cooperating with character recognition means, said communication means further adapted to formulate the message text and address and transmit the text message to said address over a computer network;
• receiving means cooperating with user interface to receive the signals
Typically the storage means includes at least one database adapted to store the coordinates corresponding to the location on the active page wherein textual content has been superimposed on video content.
Typically the user interface includes receiving means adapted to receive signals from the remote control device. Typically the receiving means includes an infra red receiver.
Typically, pre processing means includes at least one microprocessor adopted to perform image processing in order to recognize the active page currently being displayed. The microprocessor is further adopted to separate the foreground image and background image contained in the extracted
frame using the process of binarization. Typically the microprocessor enhances the foreground image contained in the extracted frame using the process of touching character segmentation.
Typically the character recognition means includes an optical character recognition system.
Typically the communication means includes a network interface coupled to a computer network, said network interface further adapted to transmit messages from interactive device to the preferred destination through computer network.
In accordance with another aspect of present invention, a method of automatically sending a message from interactive device to the designated address in response to user pressing a specific key on the remote control device is provided, the method comprising the following steps:
• receiving video content from said set top box;
• blending the textual content with the received video content to form blended content;
• determining the coordinates of a frame of said active page containing said blended content;
• storing said coordinates on at least one database;
• extracting the frame containing the blended content based on said coordinates;
• improving quality of said extracted frame;
• separating the background image and the foreground image of said extracted frame;
• segmenting the foreground image of said frame in order to provide for efficient recognition of characters contained in said foreground image;
• extracting the alphanumeric character set contained in said foreground image;
• dividing said alphanumeric character set into message text and address;
• sending a message from interactive device to designated address in response to user pressing a designated key located on said remote control device.
Typically the method includes the step of carrying out the process of image processing of active page currently displayed, in order to determine the coordinates of the frame wherein textual content has been superimposed on video content.
Typically the method includes the step of carrying out the process of noise removal in order to improve the signal quality of extracted frame.
Typically the method includes the step of performing binarization on the extracted frame in order to improve the signal quality of the frame. The binarization is achieved using the technique of adaptive thresholding
Typically the method includes the step of performing segmentation on the extracted frame. Typically the process of segmentation is carried out by utilizing at least one of the methods of morphological filter based segmentation, pixel projection, profile projection and multifactor based segmentation.
Typically the method of extracting alphanumeric characters includes using an optical character recognition device capable of recognizing the alpha numeric character superimposed on the video content.
Typically the method of sending messages from interactive device to designated address includes sending messages via a network interface which has been coupled to the computer network.
In accordance with one aspect of the present invention, the system for SMS value added service for active television shows brings value to any active interface which needs to send a message in a convenient way while watching active shows, live programs and the like on television.
BRIEF DESCRIPTION OF THE ACCOMPANYING DRAWINGS
The present invention will now be described in relation to the accompanying
drawings, in which:
FIGURE 1 illustrates the interconnection between the STB, the interactive device, the remote control and the television set in accordance with the present invention;
FIGURE 2 illustrates a schematic of the system for SMS value added service for active television shows (interactive device) in accordance with the present invention; and
FIGURE 3 is a flowchart showing the steps for sending an SMS for availing value added services on active television shows in accordance with the . present invention.
DESCRIPTION OF THE INVENTION
The system for SMS value added services will now be described with reference to the accompanying drawings which do not limit the scope and ambit of the invention. The description provided is purely by way of example and illustration.
FIGURE 1 is a schematic showing the interconnection between the STB represented by block 10, the interactive device, represented by block 12, the remote control represented by block 14, and the television, represented by block 16.
In accordance with the present invention, the analog video/audio output of the STB 10 is connected to the input of the interactive device 12. And the analog video/audio output of the interactive device 12 is connected to the TV 16. The interface of the STB 10 to the TV broadcast network (not shown) could be via a cable analog tuner, a satellite analog tuner or an aerial analog tuner for example. All or some of the above mentioned connection modules could be provided as replaceable modules. The interface of the interactive device 12 to internet (not shown) may be via a modem (56K modem, for
example), integrated Services Digital Network (ISDN), cable modem (return channel), Digital Subscriber Line (DSL), Universal Serial Bus2, residential Ethernet or residential wireless, to name only a few illustrative examples. One or several of the above mentioned modules could be provided as replacement modules. The interface of the STB 10 to the TV 16 could be via NTSC/Pal or Secam, VGA (standard PC monitor interface), CRT monitor, LCD monitor or Plasma monitor, for example. The remote control device 14 translates the remote key press events into infra red light signals that are perceived by the IR receiver of interactive device 12. The remote control device 14 is configured to work with STB 10 as well as interactive device 12, the signals from the remote control are received by the IR receiver of the interactive device 12, and the signals meant for STB 10 are transferred to STB 10 through the IR blaster of interactive device 12.
Figure 2 is a system level block diagram showing the schematic of the interactive device for automatic detection of text messages from the active programs/ live television programs. A network interface, represented by block 22 of figure 2, is provided on the interactive device 12 for attaching an Internet connection. The network interface 22 could make use of an USB (Universal Serial Bus) based modem, integrated Services Digital Network (ISDN), cable modem (return channel), Digital Subscriber Line (DSL), Universal Serial Bus2, residential Ethernet, residential wireless or an Ethernet connection and the like to connect to the computer network .
In accordance with one aspect of the present invention, a single remote control 14 interacts with the STB 10 and the interactive device 12. The remote control signals are received by the IR receiver, represented by block
20 of figure 2, of the interactive device 12. The remote control functionalities which are meant for the STB 10 are forwarded to the STB 10 using the IR blaster, represented by 24 of figure 2 of the interactive device 12. Typically, STB's provide a variety of active programs including 'Active Mall', 'Active Astrology', 'Active Matrimony', 'Active Helpdesk', 'Active Jobs', 'Active Movies' and the like. These active programs are a collection of pages which provide consumers with services and products which can be availed by sending a text message to designated address/ mobile number. From the set of images of the active pages, we can make the following observations:
• the location of the relevant text region is fixed for a particular active page;
• there is a considerable contrast difference between the relevant text region and the background; and
• the characters to be recognized are of standard font type and size.
In accordance with the present invention, the following interaction takes place between the STB, the interactive device, the television and the consumer for sending a message while watching an active program on television. The interactive device 12 processes the video it receives from the STB 10, after processing the interactive device blends the textual content with the video content thereby superimposing the textual content over the video content. The video output is therefore modified by the interactive device and is passed on to Analog AV input of the TV 16 for display. For instance, on the active page the interactive device blends text 'Confirm to SMS' and sends the modified video on the television for display. Further the interactive device 12 processes the images on the analog video for extraction
of message text. Once the user, by pressing the designated key on the remote control confirms his intention to send a message, the interactive device sends the message using the network interface.
For superimposing the textual content over the video content, the interactive device 12 includes blending means which has been represented by block 26 of Figure 2. The blending means 26 includes a microprocessor that receives the analog video input from the STB 10 and superimposes textual content over the video content. The text thus superimposed advertises a particular type of product or service and it prompts the user to press a pre determined key located on the remote control 14 in order to confirm his interest towards availing the advertised product or service. The blended video is further sent to the television 16 for display. The blending means 26 is further configured to blend the static or dynamic web based content with the analog video input. The blending means 26 transfers the blended signal to the graphic interface of television 16 for display. The graphic interface of television 16 is adapted to display the static or dynamic web content along with the television broadcast content thereby enabling the user to concurrently watch the active television content as well as browse the internet.
The blended signal which is output from the blending means 26 is also received by the machine vision unit, represented by block 28 of figure 2. The machine vision unit 28 performs offline detection of the region of the active page wherein the video content has been super imposed by textual content. The machine vision unit 28 processes the active page and determines the bounding box coordinates of every region which contains the superimposed textual data. Machine vision unit 28 first calculates the normalized
coordinates of top left (x' 0,y' °) and bottom right (x' l,y' ') position for the bounding box for each video frame based on the image resolution using the
formula:
Where x active 0 and y ac!ive 0 represents the top left coordinate of active page. The absolute coordinates are represented as (xo,'y°) and (X1,Y1) respectively. widthactive and heightactive represents the width and height of active page respectively. The machine vision unit 28 further stores the calculated relative position of all relevant text region for each active pages in a repository, represented by block 30 of figure 2. The repository 30 of figure 2 is a database that stores the location co-ordinates of the relevant video frame for each active page. The frame is further referred by the pre-processing unit represented by block 32 of Figure 2.
The pre-processing Unit 32 contains a microprocessor that has been configured to track the remote key press events in order to recognize the active page being displayed. As an alternative to the method of tracking remote key press events, the microprocessor of pre processing unit 32 could employ the method of image processing in order to detect the active page being displayed. After detecting the active page being displayed, the microprocessor of pre-processing Unit 32 extracts from the repository (database) 30, the corresponding coordinates of the video frame that contains
the superimposed textual data. After determining the coordinates of the relevant video frame, the micro processor of pre processing unit 32 extracts the video frame and performs the steps of noise removal and binarization. The step of noise removal is undertaken in order to improve the quality of the signal corresponding to the extracted video frame. The extracted video frame is further subjected to the step of binarization which includes separation of the foreground image and back ground image corresponding to the extracted video frame. The foreground image and the background image are separated in order to facilitate a better character recognition. The microprocessor of pre-processing unit 32 also performs color enhancement and contrast enhancement since efficient binarization requires considerable contrast difference between the foreground region and the background region. There are several ways to achieve binarization so that the foreground and the background can be separated. However, as both the characters present in the relevant text region as well as the background are not of a fixed gray level value, adaptive thresholding is utilized by pre processing unit 32 in order to achieve binarization of extracted video frame. To obtain the threshold value, the micro processor of pre processing unit 32 makes use of one or several of following approaches, namely, Gaussian mixture model (GMM), Independent component analysis (ICA), Color histogram based approach and otsu's method. After the completion of binarization, the microprocessor of pre-processing unit 32 performs touching character segmentation to facilitate more accurate detection of the text region by the OCR engine, represented by block 34 of figure 2.
A major barrier to achieve good character recognition is the presence of touching characters. A good touching character segmentation scheme is
inevitable for the betterment of the recognition rate. In order to obtain the touching character segmentation, micro processor of pre-processing unit 32 uses one or several of the following approaches, namely morphological filter based segmentation, pixel projection, profile projection and multifactor based segmentation
Typically, the textual content superimposed on the video content being displayed as active pages contain only alpha numeric characters. The OCR Engine 34 performs detection and extraction of the text (alphanumeric characters) from the processed frame and sends the detected text (message and the address) to the communication means, represented by block 36 of figure 2. The communication means 36 formulates the text message and forwards it to the network interface 22 along with the address for sending the message, once the communication means 36 receives the confirmation for sending the message via the IR receiver 20 i.e. once the consumer presses the designated key on the remote control, thereby confirming his intention of sending the message, the text message is sent via network interface 22 which is coupled t a computer network (not shown). Further, if the text content being superimposed on the video content contains a URL (Universal Resource Locator), the communication means 22 formulates the URL and forwards it to the browsing means, represented by block 38 of Figure 2.
The browsing means 38 receives the URL from the communication means 36 and via the internet connection connected to the network interface 22 the website indicated by the URL is accessed by the browsing means 38. On receiving the confirmation from the consumer via a designated key press the browsing means transfers the web page to the blending means 26 which
blends the webpage along with the active analog video content for display on the television 16.
Figure 3 shows the method for automatically sending a message to address mentioned on the active page. In accordance with the present invention, the entire process of extraction of text and sending the message upon confirmation from the consumer comprises the steps depicted in figure 3. Block 50 indicates the step localizing the region of interest or the frame from the active page. Frame is the region of the active page that contains the message text and the address. Prior to the step of localizing the region containing the message text and address, the process of carrying out the process of image processing in order to determine the active page currently displayed is conducted. Alternatively the process of remote key press event tracking could also be undertaken in order to determine the active page currently displayed. Block 52 of figure 3 represents the process of removing noise from the extracted frame in order to improve the signal quality of the frame. Block 54 of figure 3 represents the step of carrying out the process of binarization of extracted frame in order to separate the foreground and the background images. The process of separating the background image and the foreground image is accomplished by utilizing at least one of the methods of morphological filter based segmentation, pixel projection, profile projection and multifactor based segmentation. Block 56 of figure 3 represents the process of segmenting the foreground image contained in the extracted frame using the process of touching character segmentation. The process of touching character segmentation is undertaken in order to provide for efficient recognition of characters contained in the foreground image. The process of touching character segmentation is accomplished utilizing at least
one of the methods of Gaussian mixture model, Independent component analysis, Color histogram based approach and otsu's method. Block 58 of figure 3 represents the step of recognizing the alphanumeric character set contained in the foreground image. The process of recognizing the alphanumeric characters includes the step of using an optical character recognition device which has been configured to recognize the alpha numeric character set superimposed on the video content. Block 60 of figure 3 denotes the step of
sending a message containing the recognized text message to the recognized, pre determined address. The message is sent to the pre determined address using a network interface 22 that has been coupled to a computer network.
In accordance with the present invention, the active pages contain the message text and the address where the message has to be sent, for instance if the active page contains "SMS to buy. Please SMS the following code to 12345. For games XY ABC123". The present invention extracts the SMS address as "12345" and the content is extracted as "XY ABC123". After the consumer presses the designated button on the remote control, the message is automatically sent by the present invention and the purchase gets completed. The present invention is not limited to Set top box and can be applied for online live videos as well.
TECHNICAL ADVANCEMENTS
The technical advancements offered by the present invention include the realization of:
• providing a single system which will facilitate sending of a text message while watching active television shows;
• Providing a system that recognizes the message text and message address superimposed on active video content and transmits the message automatically to the recognized address on receiving a confirmation from the user;
• providing a system which enhances the user interactivity while watching active television shows;
• providing a system which generates revenue for the telecom service providers by simplifying the method of sending a text message while watching active television shows; and
• providing a user-centric system which aims at enhancing the user experience while watching active shows on television.
While considerable emphasis has been placed herein on the particular features of this invention, it will be appreciated that various modifications can be made, and that many changes can be made in the preferred embodiments without departing from the principles of the invention. These and other modifications in the nature of the invention or the preferred embodiments will be apparent to those skilled in the art from the disclosure herein, whereby it is to be distinctly understood that the foregoing descriptive matter is to be interpreted merely as illustrative of the invention and not as a limitation.
We Claim:
1. An interactive device interfaced between a set top box and at least one display device, said device comprising:
• blending means co-operating with said set top box to receive video content, said blending means adapted to blend pre determined textual content with said video content to form blended content;
• graphical interface cooperating with blending means to receive said blended content, said graphical interface further adapted to output the received blended content on to said display device;
• a user interface adapted to enable a user to interact with said interactive device;
• a machine vision unit cooperating with blending means to receive said blended content, said machine vision unit adapted to detect the coordinates of a frame of an active page containing said blended content;
• storage means adapted to store the coordinates of said frame of an active page;
• pre processing means adapted to recognize the active page displayed on said display device, said pre processing means cooperating with storage means to receive the coordinates of said frame of the active page, said preprocessing means adapted to separate the background image and foreground image of said frame and enhance said foreground image;
• character recognition means cooperating with preprocessing means to receive said enhanced foreground image, said character
recognition means further adapted to extract the alpha numeric character set embedded in said enhanced foreground image;
• communication means cooperating with character recognition means, said communication means further adapted to formulate the message text and address and transmit the text message to said address over a computer network;
• receiving means cooperating with user interface to receive the signals
2. Interactive device of claim 1, wherein said blending means includes at least one microprocessor.
3. Interactive device of claim 1, wherein said user interface includes a remote control device.
4. The remote control device as claimed in claim 3, further adapted to generate signals in response to user pressing the keys located on said device.
5. Interactive device of claim 1, wherein said storage means includes at least one database;
6. The interactive device of claim 1, wherein said pre processing means includes at least one microprocessor adapted to perform image processing in order to recognize the active page.
7. The interactive device of claim 1, wherein pre processing means includes at least one microprocessor further adapted to separate the foreground image and background image of said frame.
8. The interactive device of claim 1, wherein pre processing means includes at least one microprocessor further adapted to enhance the foreground image contained in the extracted frame.
9. The interactive device of claim 1, wherein character recognition means includes an optical character recognition system.
10.The interactive device of claim 1, wherein communication means includes a network interface coupled to a computer network, said network interface further adapted to transmit messages from interactive device to preferred destination through computer network.
11. The interactive device of claim 1, wherein said receiving means includes infra red receiver.
12.A method for sending a message from interactive device to the designated address in response to user pressing a specific key on a remote control device, the method comprising the steps of:
• receiving video content from said set top box;
• blending the textual content with the received video content to form blended content;
• determining the coordinates of a frame of said active page containing said blended content;
• storing said coordinates on at least one database;
• extracting the frame containing the blended content based on said coordinates;
• improving quality of said extracted frame;
• separating the background image and the foreground image of said extracted frame;
• segmenting the foreground image of said frame in order to provide for efficient recognition of characters contained in said foreground image;
• extracting the alphanumeric character set contained in said foreground image;
• dividing said alphanumeric character set into message text and address;
• sending a message from interactive device to designated address in response to user pressing a designated key located on said remote control device.
13.The method of claim 12, wherein said method includes the step of performing image processing in order to determine the active page currently displayed.
14. The method of claim 12, wherein said method includes the step of performing remote key press event tracking in order to determine the active page currently displayed.
15.The method of claim 12, wherein said method includes the step of carrying out the process of noise removal in order to improve the signal quality of said extracted frame.
16.The method of claim 12, wherein said method includes the step of performing adaptive thresholding on said extracted frame in order to separate the background image and foreground image.
17.The method of claim 12, wherein said method includes the step of utilizing at least one of the methods of Gaussian mixture model, Independent component analysis, Color histogram based approach and otsu's method in order to separate said background image and foreground image.
18. The method of claim 12, wherein said method includes the step of utilizing at least one of the methods of morphological filter based segmentation, pixel projection, profile projection and multifactor
based segmentation in order to achieve segmentation of said
foreground image. 19.The method of claim 12 comprising, using an optical character
recognition device adapted to recognize the alpha numeric character
set blended with video content. 20.The method of claim 12 further comprising the method wherein a
message is sent to the designated address via network interface
coupled to a computer network.
| # | Name | Date |
|---|---|---|
| 1 | 2093-MUM-2009-FORM 18(30-11-2010).pdf | 2010-11-30 |
| 1 | 2093-MUM-2009-RELEVANT DOCUMENTS [28-09-2023(online)].pdf | 2023-09-28 |
| 2 | 2093-MUM-2009-CORRESPONDENCE(30-11-2010).pdf | 2010-11-30 |
| 2 | 2093-MUM-2009-RELEVANT DOCUMENTS [26-09-2022(online)].pdf | 2022-09-26 |
| 3 | 2093-MUM-2009-RELEVANT DOCUMENTS [29-09-2021(online)].pdf | 2021-09-29 |
| 3 | 2093-MUM-2009-CORRESPONDENCE(IPO)-(FER)-(14-12-2015).pdf | 2015-12-14 |
| 4 | Petition Under Rule 137 [18-05-2016(online)].pdf | 2016-05-18 |
| 4 | 2093-MUM-2009-RELEVANT DOCUMENTS [29-03-2020(online)].pdf | 2020-03-29 |
| 5 | OTHERS [18-05-2016(online)].pdf | 2016-05-18 |
| 5 | 2093-MUM-2009-RELEVANT DOCUMENTS [23-03-2019(online)].pdf | 2019-03-23 |
| 6 | Other Document [18-05-2016(online)].pdf | 2016-05-18 |
| 6 | 2093-MUM-2009-ABSTRACT(14-9-2010).pdf | 2018-08-10 |
| 7 | Examination Report Reply Recieved [18-05-2016(online)].pdf | 2016-05-18 |
| 7 | 2093-MUM-2009-CLAIMS(14-9-2010).pdf | 2018-08-10 |
| 8 | Description(Complete) [18-05-2016(online)].pdf | 2016-05-18 |
| 8 | 2093-MUM-2009-CORRESPONDENCE(10-9-2014).pdf | 2018-08-10 |
| 9 | 2093-MUM-2009-CORRESPONDENCE(14-9-2010).pdf | 2018-08-10 |
| 9 | Correspondence [18-05-2016(online)].pdf | 2016-05-18 |
| 10 | 2093-MUM-2009-CORRESPONDENCE(8-12-2009).pdf | 2018-08-10 |
| 10 | Claims [18-05-2016(online)].pdf | 2016-05-18 |
| 11 | 2093-MUM-2009-CORRESPONDENCE(9-8-2011).pdf | 2018-08-10 |
| 11 | Abstract [18-05-2016(online)].pdf | 2016-05-18 |
| 12 | 2093-MUM-2009-CORRESPONDENCE-160516.pdf | 2018-08-10 |
| 12 | 2093-MUM-2009-PatentCertificate01-09-2017.pdf | 2017-09-01 |
| 13 | 2093-mum-2009-correspondence.pdf | 2018-08-10 |
| 13 | 2093-MUM-2009-IntimationOfGrant01-09-2017.pdf | 2017-09-01 |
| 14 | 2093-MUM-2009-DESCRIPTION(COMPLETE)-(14-9-2010).pdf | 2018-08-10 |
| 14 | 2093-MUM-2009-RELEVANT DOCUMENTS [28-03-2018(online)].pdf | 2018-03-28 |
| 15 | TCS-2093MUM2009- POA.pdf | 2018-08-10 |
| 16 | 2093-mum-2009-description(provisional).pdf | 2018-08-10 |
| 16 | RTOA 2093_MUM_2009.pdf | 2018-08-10 |
| 17 | Form 3.pdf | 2018-08-10 |
| 17 | 2093-MUM-2009-DRAWING(14-9-2010).pdf | 2018-08-10 |
| 18 | F-1-2-page.pdf | 2018-08-10 |
| 18 | 2093-mum-2009-drawing.pdf | 2018-08-10 |
| 19 | 2093-MUM-2009-FORM 1(8-12-2009).pdf | 2018-08-10 |
| 19 | CLAIMS-MARK+CLEAN.pdf | 2018-08-10 |
| 20 | 2093-mum-2009-form 1.pdf | 2018-08-10 |
| 20 | abstract1.jpg | 2018-08-10 |
| 21 | 2093-mum-2009-form 2(14-9-2010).pdf | 2018-08-10 |
| 21 | ABS-MARK+CLEAN.pdf | 2018-08-10 |
| 22 | 2093-MUM-2009-FORM 2(TITLE PAGE)-(14-9-2010).pdf | 2018-08-10 |
| 22 | 2093-MUM-2009_EXAMREPORT.pdf | 2018-08-10 |
| 23 | 2093-mum-2009-form 2(title page).pdf | 2018-08-10 |
| 23 | 2093-MUM-2009-GENERAL POWER OF ATTORNEY-160516.pdf | 2018-08-10 |
| 24 | 2093-MUM-2009-FORM 5(14-9-2010).pdf | 2018-08-10 |
| 25 | 2093-mum-2009-form 3.pdf | 2018-08-10 |
| 25 | 2093-mum-2009-form 2.pdf | 2018-08-10 |
| 26 | 2093-mum-2009-form 26.pdf | 2018-08-10 |
| 26 | 2093-MUM-2009-FORM 3(10-9-2014).pdf | 2018-08-10 |
| 27 | 2093-mum-2009-form 26.pdf | 2018-08-10 |
| 27 | 2093-MUM-2009-FORM 3(10-9-2014).pdf | 2018-08-10 |
| 28 | 2093-mum-2009-form 2.pdf | 2018-08-10 |
| 28 | 2093-mum-2009-form 3.pdf | 2018-08-10 |
| 29 | 2093-MUM-2009-FORM 5(14-9-2010).pdf | 2018-08-10 |
| 30 | 2093-mum-2009-form 2(title page).pdf | 2018-08-10 |
| 30 | 2093-MUM-2009-GENERAL POWER OF ATTORNEY-160516.pdf | 2018-08-10 |
| 31 | 2093-MUM-2009-FORM 2(TITLE PAGE)-(14-9-2010).pdf | 2018-08-10 |
| 31 | 2093-MUM-2009_EXAMREPORT.pdf | 2018-08-10 |
| 32 | 2093-mum-2009-form 2(14-9-2010).pdf | 2018-08-10 |
| 32 | ABS-MARK+CLEAN.pdf | 2018-08-10 |
| 33 | 2093-mum-2009-form 1.pdf | 2018-08-10 |
| 33 | abstract1.jpg | 2018-08-10 |
| 34 | 2093-MUM-2009-FORM 1(8-12-2009).pdf | 2018-08-10 |
| 34 | CLAIMS-MARK+CLEAN.pdf | 2018-08-10 |
| 35 | 2093-mum-2009-drawing.pdf | 2018-08-10 |
| 35 | F-1-2-page.pdf | 2018-08-10 |
| 36 | Form 3.pdf | 2018-08-10 |
| 36 | 2093-MUM-2009-DRAWING(14-9-2010).pdf | 2018-08-10 |
| 37 | RTOA 2093_MUM_2009.pdf | 2018-08-10 |
| 37 | 2093-mum-2009-description(provisional).pdf | 2018-08-10 |
| 38 | TCS-2093MUM2009- POA.pdf | 2018-08-10 |
| 39 | 2093-MUM-2009-DESCRIPTION(COMPLETE)-(14-9-2010).pdf | 2018-08-10 |
| 39 | 2093-MUM-2009-RELEVANT DOCUMENTS [28-03-2018(online)].pdf | 2018-03-28 |
| 40 | 2093-mum-2009-correspondence.pdf | 2018-08-10 |
| 40 | 2093-MUM-2009-IntimationOfGrant01-09-2017.pdf | 2017-09-01 |
| 41 | 2093-MUM-2009-CORRESPONDENCE-160516.pdf | 2018-08-10 |
| 41 | 2093-MUM-2009-PatentCertificate01-09-2017.pdf | 2017-09-01 |
| 42 | 2093-MUM-2009-CORRESPONDENCE(9-8-2011).pdf | 2018-08-10 |
| 42 | Abstract [18-05-2016(online)].pdf | 2016-05-18 |
| 43 | 2093-MUM-2009-CORRESPONDENCE(8-12-2009).pdf | 2018-08-10 |
| 43 | Claims [18-05-2016(online)].pdf | 2016-05-18 |
| 44 | 2093-MUM-2009-CORRESPONDENCE(14-9-2010).pdf | 2018-08-10 |
| 44 | Correspondence [18-05-2016(online)].pdf | 2016-05-18 |
| 45 | 2093-MUM-2009-CORRESPONDENCE(10-9-2014).pdf | 2018-08-10 |
| 45 | Description(Complete) [18-05-2016(online)].pdf | 2016-05-18 |
| 46 | Examination Report Reply Recieved [18-05-2016(online)].pdf | 2016-05-18 |
| 46 | 2093-MUM-2009-CLAIMS(14-9-2010).pdf | 2018-08-10 |
| 47 | 2093-MUM-2009-ABSTRACT(14-9-2010).pdf | 2018-08-10 |
| 47 | Other Document [18-05-2016(online)].pdf | 2016-05-18 |
| 48 | OTHERS [18-05-2016(online)].pdf | 2016-05-18 |
| 48 | 2093-MUM-2009-RELEVANT DOCUMENTS [23-03-2019(online)].pdf | 2019-03-23 |
| 49 | Petition Under Rule 137 [18-05-2016(online)].pdf | 2016-05-18 |
| 49 | 2093-MUM-2009-RELEVANT DOCUMENTS [29-03-2020(online)].pdf | 2020-03-29 |
| 50 | 2093-MUM-2009-RELEVANT DOCUMENTS [29-09-2021(online)].pdf | 2021-09-29 |
| 50 | 2093-MUM-2009-CORRESPONDENCE(IPO)-(FER)-(14-12-2015).pdf | 2015-12-14 |
| 51 | 2093-MUM-2009-CORRESPONDENCE(30-11-2010).pdf | 2010-11-30 |
| 51 | 2093-MUM-2009-RELEVANT DOCUMENTS [26-09-2022(online)].pdf | 2022-09-26 |
| 52 | 2093-MUM-2009-FORM 18(30-11-2010).pdf | 2010-11-30 |
| 52 | 2093-MUM-2009-RELEVANT DOCUMENTS [28-09-2023(online)].pdf | 2023-09-28 |