FORM 2
THE PATENTS ACT, 1970
(39 of 1970)
&
THE PATENT RULES, 2003
COMPLETE SPECIFICATION
(See Section 10 and Rule 13)
Title of invention:
SKIM BASED READING FOR VISUALLY IMPAIRED
Applicant:
TATA Consultancy Services Limited
A company Incorporated in India under The Companies Act, 1956
Having address:
Nirmal Building, 9th Floor,
Nariman Point, Mumbai 400021,
Maharashtra, India
The following specification describes the invention and the manner in which it is to be performed.

FIELD OF THE INVENTION
The present invention relates to the field of eye-tracking and gaze data collection. More particularly, the invention relates to the field of generating a pruned data set from a digital document for visually impaired users based upon the gaze data acquired from a group of sighted readers.
BACKGROUND OF THE INVENTION
According to a survey conducted by United States Census Bureau (USCB) and World Health organization the overall world human population has reached to 7 billion out of which 285 million are visually impaired. The overall population of visually impaired people contributes about 4 percent of the total world population. Most of the visually impaired/ blind users rely on their other senses to gain knowledge and access information from various sources. Due to visual disabilities the digital data available through internet and other electronic systems is not directly accessible to these users. One implementation of making this data available for such communities is through the use of screen readers that enables reading of digital document and articles available in audio format. These articles are spoken out through a text-to-speech engine installed on the internet browser of the blind user's electronic device. However, as these text-to-speech readers access the documents and articles in a sequential manner, this result in lack of flexibility for the end-user to get acquainted with the document as per her preferences. For example, by using this technique, the user cannot skip any irrelevant section or passage and finds hard time in navigating through the digital document.
On the other hand a sighted reader may access a digital document randomly and view the document as a whole. Further the sighted reader can identify important

parts in a digital document based on the layout and structure of the digital document/article and accordingly skim through this document. A sighted reader may easily skim through a document, skipping and hurrying through irrelevant portions of the document. On the contrary a visually impaired user cannot skim through content of the digital documents using a screen reader. Hence it is difficult for the visually impaired user to ascertain whether a document is relevant to him or not without spending considerable time on it. Thus, there is a need for creating a pruned data set from the digital document for the visually impaired users by analyzing the data skimmed by the sighted users at pre-defined time intervals.
In a background art, document summarization techniques exist, which summarize the document for visually impaired users. These may be done using algorithmic techniques or may require extensive human effort in rewriting a pruned document. However, a summary may not be the most appropriate alternative to skimming through an actual document since it involves an algorithmic effort which might skip relevant sections without taking into consideration the reader's interest. The summarization can also be done as a single human activity, wherein an individual summarizes the document based on her understanding and intelligence level. This approach may fail to capture what the social community thinks about the document. Moreover the summarization technique fails to capture the evolving interest of the users such as people's interests in music/politics/etc. at that point of time. This may play an important role in what is important in the document being read.
While skimming a sighted reader may automatically focus on such interesting data in an online article which may differ at different time depending on the events which occur in our day to day life. The evolving interest of the people is not captured by the summarization technique available in the art. A social input in refining the pruned dataset visually impaired user is still lacking in the existing art. In other words the

summarization technique may not capture evolving interests of people over a period of time. A blind user might not always be interested in the summary of the whole document but would rather prefer hearing a gist of the actual content in the document which most of the sighted reader prefer to read. There is no text reader available which can accept the inputs and feedbacks from visually impaired users and accordingly refine the summarized data.
Hence in the light of above lacunae observed in the art, there is a long-felt need to develop a system and method which can capture human intelligence in the form of gaze data from a plurality of sighted readers when they are reading a digital document and accordingly generate a pruned data set associated with a digital document instead of summarizing the digital document. Further there is a need to accept inputs from visually impaired user in order to effectively skim through the digital document as per the requirements specified by the visually impaired user. There is a need to frequently modify the pruned data based upon the frequently captured gaze data from new sighted readers in order to identify the evolving interest of the social community and replicate it in the pruned data set. Further, there is a need to capture feedback received from the blind users and accordingly modify the pruned data content.
OBJECTS OF THE INVENTION
The primary object of a present invention is to enable a communication device to capture human intelligence in the form of gaze data from a plurality of sighted readers and accordingly assign significance score.
Another object of the invention is to generate a pruned data content based upon the gaze data captured from a plurality of sighted readers.

Yet another object of the invention is to capture the evolving interest of the sighted readers over a period of time and accordingly update the pruned data content.
Still another object of the invention is to accept feedback from visually impaired users and accordingly summarize the pruned data content using a screen reader.
SUMMARY OF THE INVENTION:
Before the present systems and methods, enablement are described, it is to be understood that this application is not limited to the particular apparatus, systems, and methodologies described herein, as there can be multiple possible embodiments which are not expressly illustrated in the present disclosures. It is also to be understood that the terminology used in the description is for the purpose of describing the particular versions or embodiments only, and is not intended to limit the scope of the present application.
In one embodiment, the present invention discloses a method for skimming through a digital document based on collecting gaze input from a plurality of sighted readers. An eye tracking device attached to the local machine of a sighted reader collects the gaze input from the sighted readers while the sighted reader is skimming through the digital document. Analysis is performed on the gaze input to identify the most relevant portions in the digital document and assigning a significant score to each of the section. In a similar manner the gaze input from a plurality of sighted readers is collected and statistical analysis is performed on their gaze input in order to determine gaze pattern which signifies the sections of in the digital document the sighted readers found to be interesting. This skimmed data is made available to visually impaired users through a screen reader, so that they can get a quick

understanding of the document without sequentially going through the entire document.
In one of the embodiment, this document is made available in voice format to blind users. When the blind users are interested in skimming through the digital document instead of reading the entire document, the pruned data is made available to the visually impaired users using a text to speech reader. The screen reader accepts input with respect to the duration of skimming from the visually impaired user and accordingly schedules the skimming path through the pruned data content. Further the visually impaired users are enabled to skip through various important sections using a shortcut key.
In one of the embodiment of the present invention the pruned data is updated periodically at a pre-defined time interval based on the feedback received from the visually impaired users and the gaze data updates generated during the course of time. Based on the feedback received from the visually impaired user and updated gaze data, the pruned data is continually updated. The system also captures the evolving interest of the sighted readers across various instances and updates this evolution of interests into the pruned data set.
BRIEF DESCRIPTION OF DRAWINGS
The foregoing summary, as well as the following detailed description of embodiments, is better understood when read in conjunction with the appended drawings. For the purpose of illustrating the invention, there is shown in the present document example constructions of the invention. However, the invention is not limited to the specific methods and apparatus disclosed in the document and the drawings:

Figure 1 is a block diagram illustrating the various computing devices connected over a communication network involved in the process of gaze data capturing and analysis.
Figure 2 is a system diagram illustrating different units and modules involved in the process of gaze data extraction and pruned data creation.
Figure 3 illustrates a flow diagram involved in the process of pruned data content creation from the captured gaze data.
Figure 4 illustrates an exemplary embodiment for the actual working of the system for generating pruned data content for a digital document.
DETAILED DESCRIPTION:
The description has been presented with reference to an exemplary embodiment of the invention. Persons skilled in the art and technology to which this invention pertains will appreciate that alterations and changes in the described method and system of operation can be practiced without meaningfully departing from the principle spirit and scope of this invention.
In accordance to one embodiment of the present invention, a method for generating pruned data content for a digital document by capturing gaze data using an eye tracking device is disclosed. For this purpose, the eye tracking device is attached to a communication device of first set of users. The first set of users belong a social community of sighted readers voluntarily participating to provide their eye tracked data/ gaze input in order to generate a pruned data content for the digital document being read. The said digital document is made available to the first set of users in the

form of digital text or article using a communication network. The digital document is divided into a plurality of sections wherein each section signifies a group of associated context.
In one embodiment of the present invention, the form factor of the communication devices facilitating access to the digital document for first set of users is identified. Based upon the identified form factor of each of the communication devices, gaze input from the first set of users is standardized in order to generate a standardized gaze pattern from all the communication devices. The eye tracking device primarily determines whether a user is thoroughly reading the document or skimming through the digital content. A known algorithm is applied to determine whether a first set of user is skimming through the digital document or reading thoroughly. While the first set of users are skimming through the digital document, the eye tracking devices capture gaze data and stores it into a Meta document on the communication device of the first set of users. In a similar manner a plurality of Meta documents associated the digital document are generated by a plurality of first set of users and stored on the local memory of their respective communication device. The gaze data collected in the Meta document by the eye tracking device is processed by the processing unit present on the communication device in order to determine what the user finds relevant in a document. Based upon the time spent by the user to read a particular section of the digital document a significance score is computed for each section and stored in the Meta document.
In one embodiment of the present invention, all the Meta documents present on the communication device are sent to a central processing server of the further processing. Statistical measures are applied on all Meta document associated with a single digital document to extract which sections are most relevant in the said digital document for most of the first set of users. This statistical computation is performed

based on the significance score assigned in the Meta document. The computation of the significance score for each section of the digital document is done based on the time spent by the user while reading that section. A list of sections according to their relevance is generated and is maintained in a sequential order in the form of pruned data content.
In one embodiment of the present invention, the central processing server is further attached to a central repository which stores the pruned data content associated with the digital document. For each section of the digital document the associated significance score from each Meta document is aggregated to generate an aggregated significance score. This aggregated significance score and the associated section of the digital document are then stored in the central repository. In a similar manner pruned data content is generated and stored for various other digital documents available to the first of users through the communication network. Further processing of the gaze data is performed in order to eliminate grammatical errors and introduce shortcuts in the generated pruned data content.
In one embodiment of the present invention, in the course of time new Meta documents are received from first set of users as new users read the digital document and the eye tracking device captures their gaze data. The evolving interest of the first set of users is captured based upon the continuous received new Meta documents. The central processing server performs statistical analysis and accordingly modifies the pruned data content. As a result the pruned data content generated in the course of time is not personalized but signifies evolving interest of the first set of users.
In one embodiment of the present invention, a screen reader is installed on to the communication devices of second set of users. The second set of users includes user with visually disabilities such as blindness, dyslexia, low vision and the like.

Whenever any second set of user wishes to skim through said digital document, the pruned data content associated with that digital document is transferred to the communication devise of the second set of user. Inputs are taken from the second set of users regarding the duration with which they want to skim through the digital document using known inputting means such as voice detection, keyboard input. Based upon the duration specified by the second set of user such as 30 second, 1 minute, 2 minutes, and 3 minutes etc., the most relevant pruned data content which can be presented in audio form within the specified duration is read by the screen reader.
In one embodiment of the present invention, feedback data is collected from a plurality of second set of users using a set of questionnaire or voting. The second set of users can provide their inputs such as like, dislike of a particular section. Based upon this feedback data and newly received Meta documents, the pruned data content is periodically updated.
In one embodiment of the present invention, the pruned data content can be further used to highlight the important aspects in the document itself for other sighted users in the communication network. The gaze feedback received from the users can help authors of the digital document to improve and elaborate on the digital document. Further the overall system can act as human participatory noise identification systems in digital document by identifying relevant and irrelevant words from a document based on gaze pattern generated by plurality of users. Compute metrics like latent semantic index (LSI) and term frequency-inverse document frequency (TF-IDF) can be generated which can further be used by search engines to only find relevant results and ignore irrelevant content in the digital document.

In one embodiment of the present invention, the captured gaze data can be used for enhancing advertising business. A surface area could be bid for the best screen space for displaying advertising content. A platform on which various business domains could participate to display products on a screen, and they bid laying/displaying their products based on gaze statistics. Further the system could be used to display a grid of touch sensitive devices displaying various products in a shopping mall. The items on the grid of screen would dynamically change the placement or position of the product display based on the gaze time of a particular touch sensitive device. The system can also act as a backup storage for storing the most relevant text in a digital document in case of disaster management.
Referring to figure 1 is a block diagram illustrating the various computing devices connected over a communication network (107) involved in the process of gaze data capturing and analysis. The said communication network (107) connects a communication device of sighted readers (101) with a central processing server (103) which is further coupled to a central repository (105). The communication network (107) also connects communication devices of visually impaired users (109) to the central processing server (103).
Further referring to figure 1, the said communication devices of sighted readers (101) are enabled with an eye tracking device such as webcam, USB camera, and other dedicated eye tracking hardware device available in the market in order to capture gaze data of sighted readers. This captured gaze data from each communication device of sighted readers (101) is stored in a Meta document and transmitted to the central processing server (103) using the communication network (107). The central processing server (103) examines the received gaze data in the form of a Meta document and accordingly generates a pruned data content which is further stored in the central repository (105).

In one exemplary embodiment of the present invention, the communication devices of visually impaired users (109) are enabled with a screen reader. Whenever a visually impaired user want to skim through a digital document, the pruned data content associated with that digital document is transferred from the central repository (105) to the communication device of visually impaired user (109). The screen reader attached to communication device of the visually impaired user (109) converts the pruned data content into an audio signal for the visually impaired users.
Referring to figure 2 is a system diagram illustrating different units and modules involved in the process of gaze data extraction and pruned data creation. A plurality of communication devices of sighted reader (201.1) to (201.n) are coupled to the central processing server (211) using a communication network. Each of the plurality of communication devices of sighted reader (201.1) to (201.n) are enabled with a display unit (203), an eye tracking device (205), a memory unit storing a Meta document (207) and a gaze data acquisition module (209). A digital document virtually divided into a plurality of pre-identified sections is displayed on the display unit (203) of the said device (201.1) and gaze data is captured by the eye tracking device (205) while the user is skimming through the digital document. This gaze data is processed by the gaze data acquisition module (209) in order to obtain a gaze pattern of the said sighted readers and a significance score is assigned to each pre-identified sections of the digital document. The significance score is assigned to each of the section based on the duration spent by the sighted reader to read through each section. The said gaze pattern and significance score is stored in the Meta document (207).
In one of the exemplary embodiments of the present invention, the central processing server (211) comprises of a data computation module (213) and an optimization module (214). The Meta documents (207) generated in the communication device of

sighted users (201.1) to (201.n) is transferred to the central processing server (211) for further processing. The data computation module (213) of the central processing server (211) applies statistical computations on the significance score to identify the most relevant pre-identified section of the digital document and generates pruned data content (217) from said Meta documents (207). An aggregated significance score is generated and assigned to each pre-identified section by aggregating the individual significance scores from each Meta document (207). This aggregated significance score is also stored in the pruned data content (217).
In one exemplary embodiment of the present invention, the central processing server (211) is further connected to central repository (215) storing pruned data contents (217) for a plurality of digital documents. In a similar manner pruned data content (217) for the digital documents available to the sighted users is generated and stored in the central repository (215).
In one exemplary embodiment of the present invention, a communication device of visually impaired users (219) comprising of a screen reader (223) and input module (221) is electronically coupled to the central processing server (215). Whenever a visually impaired user request to skim through a digital document, the pruned content (217) associated with the digital document is retrieved from the central repository (215) by the central processing server (211). The input module (221) of the communication device of the visually impaired user (219) accepts input with regards to the time in which he wants to summarize the digital document. Based on this input, appropriate sections of the digital document which can be read in the specified duration are shortlisted and read by the screen reader (223). The visually impaired user can skip through the sections of the digital document being read using shortcut keys.

In one exemplary embodiment of the present invention, feedback from the visually impaired user is captured using the input module (221) and transferred to the central processing server (211). The optimization module (214) in the central processing server (211) updates the pruned data content (217) associated with the digital document based upon the received feedback. The central processing server (211) keeps on updating the pruned data content (217) based upon the newly received gaze data from the sighted readers (201) in the form of Meta document (207). Due to the continuous updating of the pruned data content (217) the evolving interest of the sighted readers gets captured in the pruned data content (217).
Figure 3 illustrates a flow diagram involved in the process of pruned data content creation from the captured gaze data. At step (301), gaze data is captured from a plurality of sighted users using an eye tracking device attached to their personal communication device. For this purpose the digital document being read is divided into a plurality of pre-identified sections. At step (303), based upon the time spent by each user to read/skim through each section a significance score is assigned to each of the sections of the digital document. A Meta document is created for each sighted user storing the sections and their corresponding significance score. At step (305), statistical analysis is performed on the plurality of Meta documents received from the sighted readers and an aggregated significance score is generated corresponding to each section of the digital document. At step (307), this aggregated significance score is stored as pruned data content in the central repository attached to the central processing server. At step (309), whenever a visually impaired user wishes to skim through a digital document, the pruned data content associated with the said digital document is transferred to the communication device of the visually impaired user. The screen reader installed on the communication device reads out the most relevant section in the digital document considering the aggregated significance score extracted from the pruned data content and the duration of reading specified by the

visually impaired user. At step (311), feedback of the visually impaired user is captured using an input module present on the communication device of the visually impaired user. At step (313), the pruned data content is modified based on the feedback received from the visually impaired users. The pruned data content is further modified based upon the newly received Meta documents from the sighted readers.
WORKING EXAMPLE
Figure 4 illustrates a working example depicting the processor implemented steps in order to generate pruned data content. A digital document D is made available to a plurality of sighted users on their personal computers. For instance at time T, N number of people have skimmed through the digital document. An eye tracking device is attached to personal computers of each sighted user in order to capture gaze data while the user is reading the digital document. The digital document is divided into a plurality of sections depending on gazing pattern of these N sighted users. A Meta Document Di is created which stores the number of sighted readers reading a particular section and the time spent by each sighted reader while reading that section. The said Meta document Di is temporal in nature and it would be constantly evolving. The Meta document would also capture an age preference and change in interest as time evolves.
In one exemplary embodiment of the present invention, the process to calculate D' (pruned data set) starts by identifying which part of the digital document is read most frequently by the sighted readers and accordingly the digital document D is divided
into plurality of sections as dl, d2, d3, d4 dk. Each section would contain one or
more sentences which are gazed by the sighted users at a time. The grouping of sentences in a single section is not restricted with the logical association between

them. For the purpose of determining a section from a digital document D, the eye tracking device is used to detect which part of the digital document is gazed while skimming through the digital document D.
In an alternate embodiment of the present invention, the process to calculate D' (pruned data set) starts by analyzing the index of the digital document and accordingly dividing the digital document D into plurality of sections as dl, d2, d3,
d4 dk. Depending upon the structure of the digital document, the size of each
section can be varied accordingly. Each section would contain one or more sentences which talk about same concept and are logically grouped together to constitute a single section. The eye tracking device is used to detect which part of the digital document is gazed while reading this document.
In one exemplary embodiment of the present invention, the time spent by sighted reader has consumed time tl, t2 ...tk respectively for reading these k sections respectively. A Meta document is created which stores these sections in the decreasing order of time duration spent on the section while reading the digital document. Therefore for the document D, for every sighted reader Ui we would have a Meta document Di which would represent sections in decreasing order of time
spent on each section. Di = (si, s2 sK) where sj = one of the dl, d2...dk
depending on the time duration spent by the sighted reader.
In one exemplary embodiment of the present invention, a pruned data set D' is computed for all the sections in the digital document D. D' = (d'l, d'2....d'k), where d'l is the average of time spent by all sighted users while going through section dl of the digital document D. When a visually impaired user wants to skim through the document by specifying the time duration time t which he can spend over the document, a computation unit first calculates the number of sections ki that can be

read in time t. the most relevant sections from D' which have top time durations. These top sections are read to the user and feedback from the visually impaired users is taken in order to refine the pruned data content D'. With the course of time as the number of sighted users increase, the number of meta documents Di also increase and periodically the pruned data content D' is updated.
ADVANTAGES OF THE INVENTION
The present invention has following advantages:
The present invention enables effective capturing of gaze pattern from a plurality of sighted user and accordingly assigns significance score to each section of the document.
The present invention enables generation of pruned data content for a digital document using gaze data captured from a plurality of sighted users.
The present invention further enables capturing of evolving interest of a plurality of users based upon the variation in the gaze data pattern.
The present invention further enables adaptive skimming for visually impaired users through the digital document.
The present invention further enables acceptance feedback from a plurality of users and accordingly modifying the pruned data content.

WE CLAIM:
1. A method for generating a pruned data content for a digital document characterized by translating a human intelligence into a significance score to each pre-identified section of said digital document, the method comprising processor implemented steps of:
a. receiving an input gaze data from an eye-tracking device attached
with a computing devices of a first set of users reading said digital
document to obtain a gazing pattern thereof;
b. applying a significance score to each pre-identified section of said
digital document based on the said gazing pattern and storing the
resulting section of the digital document along with said score
value thereof in a meta document for each first set of user;
c. identifying and sorting the significant sections according to the
generated significance score value based upon the plurality of
meta documents created thereof to generate pruned data content;
d. storing said pruned data content for the digital document in a
central repository and arranging the pruned data content in a
descending order of aggregated significance score;
e. facilitating a second set of users to selectively access the pruned
data contents according to degree of significance and at least in
one data format; and
f. updating, at a predetermined interval, the significance score,
ranking and corresponding order of the pruned data content at the
central repository.

2. The method of claim 1, wherein said first set of users refers to a group of social community accessing the digital document through network communication means.
3. The method of claim 1, wherein said significant sections in the digital document is identified by tracking the first set of user's skimming habits such as pausing to read few sections, skipping to ignore a particular section in the digital document and combinations thereof.
4. The method of claim 1, wherein the captured gaze data is further processed in order to eliminate grammatical errors and introduce shortcuts in the generated pruned data content.
5. The method of claim 1, wherein while accessing the pruned data contents, the users is prompted to select a significance level of the pruned data contents.
6. The method of claim 1, wherein said pruned data contents are accessible to the second set of users in the form of text, audio signals generated through a screen-reader attached with computing devices, video signals or combination thereof.
7. The method of claim 1, wherein irregularities with the display of the first users including form factor thereof are computed such that a gaze pattern for the said digital document read on a specific display is consolidated to assign a fare score to the read section of the digital document.
8. The method of claim 1, wherein said meta document is temporal in nature and is adapted to a demographic, age preference, change in interest of the first set of users at a predefined time intervals.

9. The method of claim 1, wherein said second set of users may include visually impaired users.
10. The method of claim 1, wherein said pruned data content is updated on the basis of evolving interests and reading habits of the first set of users analyzed through gaze pattern thereof over a period of time.
11. The method of claim 1, wherein said pruned data content is updated on the basis of feedback received from the second set of users on the relevance of the penultimate pruned data content.
12. The method of claim 1, wherein the significance score for a section in the digital document is determined based upon a statistical consolidation of the number of first set of users reading the section, average time spent on reading the section and feedback received therefor the section from the second set of users.
13. A system for generating a pruned data content for a digital document characterized by translating a human intelligence into a significance score to each pre-identified section of said digital document, the system comprising:
a. an eye tracking device for capturing gaze data from a computing
devices of said first set of user;
b. a central processing server coupled to said computing devices of
first set of user to consolidate said gazed data;
c. a gaze data acquisition module to obtain a gaze pattern of first set
of users for a pre-identified sections of the digital document and
populating said gaze patterns into a meta documents;

d. a data computation module enabled to generate a set of pruned
data content from said meta document by assigning a significance
score thereto each pre-identified section of the digital document;
e. a central repository connected to the central processing server
storing gaze data from plurality of said first set of users in the
form of pruned data content;
f. a screen reader adapted to convert the pruned data content into
voice signals for a second set of users; and
g. an optimization module adapted to refine the pruned data content
on the basis of variation in interest of the first set of users evolved
over a period of time and the feedback received from the second
set of users.
14. The system of claim 13, wherein the eye tracking device is attached to the computing device of the first set of users while screen reader is installed on the computing device of the second set of users.
15. The system of claim 13, wherein the central repository is continuously updated with gaze data collected from said plurality of sighted readers and evolving interest of the sighted readers is identified by the central processing server based upon the variation in gaze data captured over a period of time.
16. The system of claim 13, wherein the central repository is connected to the central processing server and adapted to store gaze data captured from sighted readers, evolving interest of users, pruned data and feedback received from visually impaired users.

17. The system of claim 13, wherein the data computation module examines the gaze data to identify the most gazed sections in the digital document and accordingly generates significance score associate therewith.
18. The system of claim 13, wherein the pruned data content is modified periodically based on the variation in gaze data received over a period of time from a plurality of first set of users.