FORM -2
THE PATENTS ACT, 1970 (39 of 1970) & THE PATENTS RULES, 2003
PROVISIONAL
Specification
(Sec Section 10 and rule 13)
LOCALIZATION AND DETECTION OF TEXT REGIONS OF VIDEO
SEQUENCES


TATA CONSULTANCY SERVICES LTD.,
an Indian Company of Nirmal Building, 9Ih floor, Nariman Point, Mumbai 400 021, Maharashtra, India
THE FOLLOWING SPECIFICATION DESCRIBES THE INVENTION

This invention relates to localization and detection of text regions of video sequences.
In particular, this invention envisages a novel way of text images detection of a video using compressed domain information of a H.264 Codec.
In particular, this invention relates to automatic detection of text region from an input video sequence.
In particular, this invention relates to a non-threshold based technique for detection of text region in an input sequence.
Background of the Invention:
Different commercial agencies spend millions of dollars for advertisement and sponsorship to different sports events like tennis, soccer, cricket and many other popular sports. They usually place their logo and trademarks in billboards placed in the ground. In cricket it is also very popular to paint a trademark or logo in bowler run up and behind the stumps to get a better visibility. But it is difficult for the advertisements agencies to track the benefit or business they are getting out of those sponsorships. Till now, they usually verify the visibility of their brand by annotating the video manually. But these manual methods are tedious, labor intensive and thus costly. So if these trademarks and logos can be recognized automatically from the streamed/broadcasted videos, it will be helpful for the marketing cell of those organizations to monitor their visibility against the advertisement cost. Moreover, if any less computationally expensive methods can be proposed so that it can recognize the trademarks, it can give rise to a new business model concept of e-commerce called channel hyper linking.

In this concept, in the client end, a hyper link is placed on the portion of the videos containing the trademark and logos so that the viewer can directly browse to the website of the company and place an order.
Textual part of a video carries good amount of semantic information of the video data. Different applications like key word based video search, automatic indexing of videos, multilingual subtitle generation, automatic video indexing, and channel hyper linking can be developed by recognizing the text embedded into the video. In the current invention the use case of channel hyper linking is considered. This invention describes a practical and reliable solution/approach to achieve automated tracking and localization of text regions from a streamed video. In the present invention, the features obtained from H.264 compressed domain data are used. the current invention focuses on the problem of identification and localization of the text regions from a video stream in real time only.
In accordance with this invention there is envisaged a novel way of text images detection of a video using compressed domain information of a block based hybrid video Codec.
In particular, this invention envisages a novel way of text images detection of a video using compressed domain information of a H.264 Codec. In accordance with this invention a system is envisaged where the entire video is split into several smaller logical units, each having a text region using compressed domain features of H.264.
This invention is therefore designed to detect the text regions and also locate the position of the text regions in real time.

This invention provides a method for text region detection from a compressed H.264 input video data using compressed domain information of H.264 hybrid video codec comprising the following steps:
(a) calculating the first order difference of DC coefficients of integer transformed residual part of the luma component in the P frames/fields;
(b) calculating the vertical edge strengths computed from de-blocking filter; and
(c) Detecting a text region by evaluating the parameters calculated in the steps (a) and (b).
Typically, step (a) involves calculating the first order difference of DC coefficients of Luma and chroma value for each 4x4 sub block.
Particularly, step (a) includes the steps of: Obtaining a region at which
(a) a change is found in the DC values of integer transformed coefficients of Luma and chroma component;
(b) applying a prescribed low pass filter to smoothen the data, and
(c) Defining a threshold value based on which the candidate text region can be detected.
Typically, step (b) includes the step of: using the de-blocking filter based features in text region detection.

Particularly, step (b) includes the step of: using the vertical edge strength of the de-blocking filter.
The method of the invention further comprises the step of : estimating a candidate text region using the methods described herein above.
Particularly, the method of the invention includes the step of filtering all candidate text regions based on (a) temporal and (b)spatial information.
Particularly, step (a) checks the continuity of text region in consecutive frames., typically by removing ail spurious candidate regions using a pre determined logic.
Particularly, step (b) includes the step of using the coordinate information of candidate text regions to identify the text regions
More specifically, step (b) uses median and mode information of coordinate information of candidate text regions to identify the text regions
Brief Description of the Accompanying Drawings:
The invention will now be described with reference to the accompanying drawings, in which
Figure 1 gives an overview of the method in accordance with this invention Detailed Description of the Invention:

Different types of texts can be found from spoils videos like, trademarks painted in billboards, trademark on the jersey of a player, trademarks painted in the ground, score cards, and the texts coming because of video editing. In the current invention the text regions are identified from compressed domain features of H.264 and identify the trademarks from scorecards . Our proposed method is based on two features which can be directly obtained from compressed domain H.264 stream during decoding it. In this section we shall first discuss about these features, why they are used in the proposed method and finally about the decision making process based on these features. The overview of the process is described in Figure 11 of the accompanying drawings..
The two features used for text region identification from compressed domain information are described below:
DC component of transformed Luma coefficient based features: In H.264 4x4 Integer transformation is used which is different from the 8x8 DCT transformation of MPEG series video codec.
DC components of the integer transformed luma coefficients is a representative of the original video at a lower resolution. In H.264, unlike previous video codecs, 4x4 block size is used. So it gives the iconic representation of the video more precisely. The pseudo code used in the proposed method is given below:
Get the Luma DC value ( dc1) for each 4x4 sub block from decoder
Compute the first order difference (ax(dc1)anday (dc1)) of 'with neighboring
sub blocks in x and y direction.
From observation it is found that the difference is very high for a high contrast
region.

Obtain such for different (not including the test sequences)
Run K-Means algorithm (with K = 2) on them and find the centroid the
high valued cluster.
If is greater than the experimentally obtained threshold value
mark that MB as a candidate text in this frame and store this MB number in an array (a1)
De-blocking filter based feature:
The main focus for placing trademarks into billboards and ground is to get a visibility and thus they are designed to be easily read. As a consequence the texts results in a strong edge at the boundaries of text and background. But in this approach an additional time complexity is required for detecting edges. One of the new features of H.264 which was not there in any previous video CODEC is the deblocking (DB) filter. This invention uses the edge information extracted from the decoder during the process of decoding without computing the edges explicitly.
A conditional filtering is applied to all 4x4 luma block edges of a picture, except for edges at the boundary of the picture and any edges for which the DB filter is disabled by disable_deblocking_filter_idc, as specified in the slice header. This filtering process is performed on a macroblock after the picture construction process prior to DB filter process for the entire decoded picture, with all macroblocks in a picture processed in order of increasing macroblock addresses. For each macroblock and each component, vertical edges are filtered first. The process starts with the edge on the left-hand side of the macroblock proceeding through the edges towards the right-hand side of the macroblock in their geometrical order. Then horizontal edges are filtered, starting with the

edge on the top of the macroblock proceeding through the edges towards the
bottom of the macroblock in their geometrical order.
The pseudo code for selecting candidate frames using this feature is given
below:
Get the strength of DB filter for each MB.
If it is a strong vertical edge mark that MB as a candidate one
Identifying the text regions:
The proposed methodology is based on the following assumptions based on the
observations form different sports video clippings.
(i) Text regions should have a high contrast
(ii) Texts should be aligned horizontally
(iii) Texts have a strong vertical edge with background
(iv) Texts of trademarks and scorecards persists in the video for at least 0.2
seconds Some of the texts in billboard are not perceptible and therefore only the perceptible texts in the video are considered in this invention The above mentioned features (DC based and de-blocking based) together takes care of the observation (i) and (ii). This invention uses the first assumption to identify the MBs with high contrast only and the second assumption is used to identify the strong vertical edges. The pseudo code for removing the non textual part is as below:
Step I. For each candidate MB, identify the X and Y coordinate top left
position for each MB ( cx and cy) Step 2. Find the frequency (fy) of candidate MB in each row.
Step 3. Remove all MBs from a, If fr<2

Step 4. Check for continuity of MBs in each row: For this check the
column number (C() for candidate MBs in a row.
Step 5. If cx(1 + 1)-c,(i) > 2unmark the MB from a1
Where cx(i)' is the column number for i candidate MB in a particular row Step 6. To ensure that time domain filtering one frame is stored into
buffer and display the '"frame while decoding the( i-1)th frame. Step 7. Unmark all candidate MBs in '"frame if there is no candidate MB
in adjacent (i-1)th frame and (i+1) frame. Step 8. Finally all marked candidates MBs are considered as text content
in the video.
Although the invention has been described in terras of particular embodiments and applications, one of ordinary skill in the art, in light of this teaching, can generate additional embodiments and modifications without departing from the spirit of or exceeding the scope of the chained invention. Accordingly, it is to be understood that the drawings and descriptions herein are offered by way of example to facilitate comprehension of the invention and should not be construed to limit the scope thereof.
Dated this 1 7lh day of October 2008.
Mohan Dewan of R K Dewan&Co Applicants' Patent Attorneys.