FIELD OF INVENTION
[0001] The present subject matter relates in general to feedback capture and, particularly but not exclusively, to automatic feedback capture and aggregation for websites.
BACKGROUND
[0002] In order to maintain and provide an efficient, informative and useful website, website owners and/or providers may collect feedback from their users and customers concerning various aspects of the website, such as content, layout, etc. Important benefits achieved from receiving such user feedback to website owners can include improving the layout and content of the website, in order to attract new customers, and retain existing customers.
[0003] Certain user feedback techniques involve gathering and registering various parameters from a user’s system, such as an Internet Protocol (IP) address, web browser version, operating system version, and geographical location of that IP address). These gathered parameters can then be stored in a central data repository, which can be utilized at a later stage to personalize the content of the website. Other user feedback techniques rely on providing one or more interactive elements, such as buttons, on the website through which the users can manually indicate their interest or satisfaction regarding the content, layout, etc. Such inputs can then be stored in a central data repository and then utilized at a later point of time.
[0004] Recent advancements in user feedback techniques utilize special hardware which tracks the users gaze as the user consumes the content of the website. Corresponding attention levels during said consummation are mapped to an interest level to ascertain whether or not the user is satisfied and or content with the website. SUMMARY
[0005] This summary is provided to introduce concepts related to feedback capture and aggregation. This summary is not intended to identify essential features of the claimed subject matter nor is it intended for use in determining or limiting the scope of the claimed subject matter.
[0006] In one implementation, the present subject matter includes a method for feedback capture and aggregation. In one implementation, the method includes obtaining a plurality of interest value parameters from at least one peripheral device, where each of the plurality of interest value parameters is based on an interaction of an individual user with the websites. The
3
method further includes processing the interest value parameters to generate an overall interest value, where the overall interest value is indicative of an interest level of the individual user, for content on the websites.
[0007] In one implementation, the method includes analyzing a video stream based on an Active Appearance Model (AAM) fitting and detecting a facial region of the individual user, and determining at least one expression model based on the analyzing.
[0008] In another implementation, the present subject matter relates to a system for capture and aggregation of user feedback. In one implementation, the system includes a processor and a memory coupled to the processor, the memory including an advanced analysis module configured to obtain a plurality of interest value parameters based on an individual user’s interaction with websites, and a regression rating module configured to process the interest value parameters to generate an overall interest value of a user.
[0009] In one implementation, the regression rating module is further configured to operate on a probabilistic model, and wherein the probabilistic model is based on a perception variable, an emotional state variable, and an expression of the user.
[0010] In a further implementation, the present subject matter relates to a computer-readable medium having embodied thereon a computer program for executing a method. The method includes obtaining a plurality of interest value parameters including at least one of a video stream, an expression model, mouse-actions, text inputs, and social network inputs, where the interest value parameters are based on an individual user’s interaction with the websites. The method also includes processing the interest value parameters to generate an overall interest value, where the overall interest value is indicative of a user’s interest level for content on a website. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] The detailed description is described with reference to the accompanying figures. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. The same numbers are used throughout the figures to reference like features and components. Some embodiments of system and/or methods in accordance with embodiments of the present subject matter are now described, by way of example only, and with reference to the accompanying figures, in which:
4
[0012] Fig. 1 illustrates a network environment implementing a primary feedback system, and an advanced feedback system, in accordance with an embodiment of the present subject matter.
[0013] Fig. 2 illustrates a primary feedback system, in accordance with an embodiment of the present subject matter.
[0014] Fig. 3 illustrates an advanced feedback system, in accordance with an embodiment of the present subject matter.
[0015] Fig. 4 illustrates a method for feedback capture and aggregation, in accordance with an embodiment of the present subject matter. DESCRIPTION OF EMBODIMENTS
[0016] Systems and methods for automatic user feedback aggregation are described. The systems and methods can be implemented in a variety of computing devices, such as, laptops, desktops, workstations, tablet-PCs, smart phones, notebooks or portable computers, tablet computers, mainframe computers, mobile computing devices, entertainment devices, computing platforms, internet appliances and similar systems. Although the description herein is with reference to certain networks, the systems and methods may be implemented in other networks and devices, albeit with a few variations, as will be understood by a person skilled in the art.
[0017] In order to better understand the needs and interests of web users and website customers, user feedback is required. The effectiveness of a website and businesses associated either directly, or indirectly with the website, can depend on the manner in which the content of the website can be suitably tailored according to the aggregated user feedback. User feedback on the internet generally involves obtaining information regarding how a user perceives content provided on the website. Generally, user feedback aggregation techniques involve obtaining statistical data, such as from a central data repository, regarding website user details.
[0018] For example, one user feedback aggregation technique may involve requiring a website owner to use a third party service to track the number of users that visit the website. The third party service provider generally tallies a number of clicks, and a number of unique Internet Protocol (IP) hits received by a host server of the website to ascertain how many unique users accessed the website in a given duration of time. In addition, the website owner may also receive
5
a duration for which the user remained on the website from the third party service provider. This user feedback aggregation technique provides basic statistical results to the owner of the website.
[0019] Another user feedback aggregation technique involves providing an interactive interface on the website, via which users of the website can be provided with the option to input how they perceive the content presented therein. For example, the website can be provided with multiple ‘like’ or ‘+’ buttons, associated with different content on the website. The clicking of the ‘like’ or ‘+’ button associated with a video or any other content on the website, will indicate that the user positively approves the video or likes the video. The inputs received can be stored on a central data repository, which can be accessed by a customer who wishes to attain such data for his website. This user feedback aggregation technique requires active and conscious inputs from the user, and therefore relies on response provided by the user himself.
[0020] Yet another user feedback aggregation technique involves the utilization of specialized hardware, where the hardware is configured to capture the gaze of a user as he visually scans through the web-content, for example, on a display screen. Such hardware is configured to assess attention levels of the user while the user is absorbing the content. The attention levels once assessed can be mapped to an interest level. This technique may involve substantially cost intensive hardware. Furthermore, such mechanism are based on the assumption that a specific attention level of the user implies interest, or may assess attention at a peripheral level, such as by measuring pupil dilation of the users eyes, while the users attention is directed at the display screen. For example, the users gaze may be directed at content not related to the desired content, but the hardware will most probably recognize this as a relevant attention level.
[0021] According to an implementation of the present subject matter, systems and methods for automatic user feedback aggregation are described. In one implementation, the present subject matter describes a manner by which user feedback can be passively and automatically aggregated. By the passive nature of the manner in which user feedback is aggregated, the present subject matter eliminates the need for active and/or manual user participation while the user consumes content. The user feedback aggregated thus, can be indicative of a more fine grained understanding and determining the users’ preferences while performing activities, such as browsing a website, watching a video, reading an article, or the like. Such a precise understanding of user feedback can be beneficial for content providers in
6
providing tailored content on their websites, which provide better quality of service and enhance the user’s web browsing experience.
[0022] In one implementation, the methods and systems according to the present subject matter are based on continuously capturing a user’s facial expression, along with the user’s body language, and other multi-modal inputs, while the user is absorbing content, in order to interpret and better understand the user’s preferences. In one example, the multi-modal inputs can include but are not limited to mouse-actions, keystrokes, and ratings from a social networking website. In one example, the mouse-actions can include mouse pointer movements and clicks received from the mouse, and keystrokes can include inputs from a text input device, such as a keyboard. In a further example, the ratings from a social networking website can include receiving inputs pertaining to similar content as consumed by the client on the website. In one example, a video capture device, such as a user’s webcam, can be utilized to capture a video stream of the user’s facial expression over a period of time for which the user is absorbing content.
[0023] In one implementation, the video stream thus captured is analyzed to mine and further analyze the user’s facial expressions. In such a case, the mined facial expressions undergo a fitting process, where the expressions are compared and mapped onto one or more reference expression models to generate an expression model of the user. In one example, facial region detection of the user can be facilitated by utilizing Haar wavelets, and subsequently, an analysis of the detected facial region can be performed. In one implementation, the analysis of the detected facial region is based on an Active Appearance Model (AAM) process. The AAM process involves determining a substantially strong fit of the user’s facial expression, for example, against one or more reference expression models. Once the facial expressions are suitably mapped, the facial region of the user from the video stream can be removed or masked in order to maintain privacy of the user.
[0024] In another implementation, text inputs received, for example, from the keyboard of the user can pass through a privacy filter to preserve privacy of the users. In one example, the textual inputs are parsed to filter out any references to the user, or to entities that are related to the user. In one example, all proper nouns can be masked in the textual inputs using Named Entity Recognizers.
7
[0025] In a further implementation, the mouse-actions of the user can be analyzed. In one example, regions on the display module, such as a computer monitor, where specific mouse actions were performed can have one or more associated screen captures. These screen captures can be analyzed and mapped with reference representations. For example, the blue image associated with the popular social networking website Facebook® ‘like’ button and a mouse action click can imply positive interest of the user. In yet another implementation, inputs from the social networking website that the user is part of can be captured in addition to the inputs provided above. For example, ratings provided by the user pertaining to similar content can be captured by techniques, such as Optical Character Recognition (OCR), text recognition, and Scale Invariant Feature Transforms (SIFT) matching.
[0026] In one implementation, the video stream (with the masked facial region), along with expression model of the user, and the filtered multi-modal inputs can be forwarded to a secure location where an advanced analysis of the above mentioned inputs can be facilitated. In one example, the advanced analysis can calculate the most probable value for a user’s emotional state when the website content was consumed. For example, if the user viewed a video of a particular duration, the output of the advanced analysis can be a rating of the emotional state of the user during the duration of the video. In one example, the video stream along with the expression model, can contain AAM parameters, which can be further utilized in understanding the different expressions on the face of the user as he consumed the content of interest. Furthermore, the faceless video stream can be analyzed, for example, using depth estimation techniques. Once the depth values are determined, a skeleton fitting process can be utilized to obtain specific state abstractions, such as folded arms and leaning forward. These abstractions can then be fed into an analysis module, such as a regression engine, which can be configured to analyze all the above mentioned inputs to identify an expressive state of the user, such as expression of interest, or disinterest. In one implementation, the regression engine can be configured to operate on a probabilistic model to generate an overall interest value. Furthermore, based on a probability model, an emotional state of the user can be calculated. The emotional state of the user can be calculated when the user is actually consuming the content, based on the expressive state of the user. An automatic content rating can then be developed to obtain a user feedback for associated content on the website.
8
[0027] It should be noted that the description and figures merely illustrate the principles of the present subject matter. It will thus be appreciated that those skilled in the art will be able to devise various arrangements that, although not explicitly described or shown herein, embody the principles of the present subject matter and are included within its spirit and scope. Furthermore, all examples recited herein are principally intended expressly to be only for pedagogical purposes to aid the reader in understanding the principles of the present subject matter and the concepts contributed by the inventor(s) to furthering the art, and are to be construed as being without limitation to such specifically recited examples and conditions. Moreover, all statements herein reciting principles, aspects, and embodiments of the present subject matter, as well as specific examples thereof, are intended to encompass equivalents thereof.
[0028] As would be appreciated by a person skilled in the art, a fine grained understanding of a user interest level can be facilitated, without the need for manual intervention by the user. This also implies that the methods and systems according to the present subject matter can be effectively utilized by a user with no special knowledge or expertise, with a substantially zero learning curve. It will also be appreciated by those skilled in the art that the words during, while, and when as used herein are not exact terms that mean an action takes place instantly upon an initiating action but that there may be some small but reasonable delay, such as a propagation delay, between the initial action and the reaction that is initiated by the initial action. Additionally, the word “connected” is used throughout for clarity of the description and can include either a direct connection or an indirect connection.
[0029] Fig. 1 illustrates a network environment 100 implementing an advanced feedback system 102, according to an embodiment of the present subject matter. In the network environment 100, the advanced feedback system 102 is connected to a network 104. Through the network 104, the advanced feedback system 102 is further connected to a database 106, and one or more client devices 108-1, 108-2...108-N, collectively referred to as client devices 108. Examples of the client devices 108 include, but are not limited to personal computers, desktop computers, smart phones, PDAs, and laptops. Communication links between the client devices 108 and the advanced feedback system 102 are enabled through a desired form of connections, for example, via dial-up modem connections, cable links, digital subscriber lines (DSL), wireless or satellite links, or any other suitable form of communication. Each of the client devices 108 can
9
include or be connected to an associated image/video capturing device, such as a webcam (not numbered).
[0030] The advanced feedback system 102 can be implemented as any computing device connected to the network 104. For instance, the advanced feedback system 102 may be implemented as mainframe computers, workstations, personal computers, desktop computers, multiprocessor systems, laptops, network computers, minicomputers, servers, and the like. In addition, the advanced feedback system 102 may include multiple servers to perform mirrored tasks for users, thereby relieving congestion or minimizing traffic.
[0031] Moreover, the network 104 may be a wireless network, a wired network, or a combination thereof. The network 104 can also be an individual network or a collection of many such individual networks interconnected with each other and functioning as a single large network, e.g., the internet or an intranet. The network 104 can be implemented as one of the different types of networks, such as intranet, local area network (LAN), wide area network (WAN), the internet and such. The network 104 may either be a dedicated network or a shared network, which represents an association of the different types of networks that use a variety of protocols, for example, Hypertext Transfer Protocol (HTTP), Transmission Control Protocol/Internet Protocol (TCP/IP), etc., to communicate with each other. Further, the network 104 may include network devices, such as network switches, hubs, routers, host bus adapters (HBAs), for providing a link between the advanced feedback system 102 and the client devices 108. The network devices within the network 104 may interact with the advanced feedback system 102 and the client devices 108 through communication links.
[0032] In one implementation, the client devices 108 further include a primary feedback system 110. The primary feedback system 110 can be implemented within one or more of the client devices 108 or can be implemented at a secure third party location connected to the client devices 108 via the network 104, without deviating from the scope of the present subject matter. The primary feedback system 110 further includes a basic analysis module 112 and a filtration module 114. In one example, the primary feedback system 110 can be configured to receive interest value parameters that can include but are not limited to, a video stream, text inputs from a text input device, mouse-actions from a mouse of a desktop PC, and inputs from a social networking website, hereinafter referred to simply as social network inputs. As would be
10
appreciated, interest value parameters are such which are gathered during the course of interaction with a website. Based on activities performed during the course of interaction with the website, it can be determined whether the user found the content of the website useful or to his/her liking.
[0033] Returning to the operation of the primary feedback system 110, the basic analysis module 112 can be configured to obtain the interest value parameters during a period when the user is consuming content, i.e., either by reading or interacting with the website. Once the interest value parameters are obtained, the basic analysis module 112 can perform certain basic analysis functions on the interest value parameters. For example, the basic analysis module 112 can be configured to analyze a video stream of the interest value parameters in order to determine expression models therein. This is described further at a later portion of the detailed description.
[0034] Once the interest value parameters are processed by the basic analysis module 112, the filtration module 114 can be configured to filter the interest value parameters based on one or more rules. In one example, the rules can be configured such that the interest value parameters are filtered to preserve anonymity of the user. For example, the filtration module 114 may obscure or hide the face of the user in the video stream. In a further example, the filtration module 114 can be configured to mask all proper nouns in the text inputs.
[0035] Once obtained, the filtered interest value parameters can be communicated to the advanced feedback system 102. The advanced feedback system 102 can include an advanced analysis module 116. In one implementation, the advanced analysis module 116 can be configured to process the filtered interest value parameters to generate an overall interest value, indicative of a user’s interest level for content on the website. In an example, the advanced analysis can include operating on a probabilistic model, where the various filtered interest value parameters can be modeled according to a Conditional Random Field (CRF) processing. The manner in which the overall interest value is generated is described at a later stage in the detailed description.
[0036] Once generated, the interest values can be stored and accessed at a secure database, such as the database 106. Thereafter, users or subscribers can selectively be provided access to the interest values for market research purposes, feedback statistics studies, or the like.
11
The feedback capture and aggregation method according to the present subject matter provides an in-depth understanding of user preferences on a website, and can be substantially beneficial to business and website owners in better provided tailored products and services to their clients.
[0037] Fig. 2 illustrates the primary feedback system 110, in accordance with an embodiment of the present subject matter. In said implementation, the primary feedback system 110 includes one or more processor(s) 202, interface(s) 204, and a memory 206 coupled to the processor 202. The processor 202 can be a single processing unit or a number of units, all of which could also include multiple computing units. The processor 202 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuitries, and/or any devices that manipulate signals based on operational instructions. Among other capabilities, the processor 202 is configured to fetch and execute computer-readable instructions and data stored in the memory 206.
[0038] The interfaces 204 may include a variety of software and hardware interfaces, for example, interface for peripheral device(s), such as a webcam, a keyboard, a mouse, an external memory, and a printer. Further, the interfaces 204 may enable the advanced feedback system 102 to communicate with other computing devices, such as web servers and external data repositories in the communication network (not shown in the figure). The interfaces 204 may facilitate multiple communications within a wide variety of protocols and networks, such as a network, including wired networks, e.g., LAN, cable, etc., and wireless networks, e.g., WLAN, cellular, satellite, etc. The interfaces 204 may include one or more ports for connecting the advanced feedback system 102 to a number of computing devices.
[0039] The memory 206 may include any computer-readable medium known in the art including, for example, volatile memory, such as static random access memory (SRAM) and dynamic random access memory (DRAM), and/or non-volatile memory, such as read only memory (ROM), erasable programmable ROM, flash memories, hard disks, optical disks, and magnetic tapes. The memory 206 also includes module(s) 208 and data 210.
[0040] The module(s) 208 include routines, programs, objects, components, data structures, etc., which perform particular tasks or implement particular abstract data types. In one implementation, the module(s) 208 includes the basic analysis module 112, the filtration module
12
114, and other module(s) 212. The other module(s) 212 may include programs or coded instructions that supplement applications and functions of the primary feedback system 110.
[0041] On the other hand, the data 210, inter alia serves as a repository for storing data processed, received, and generated by one or more of the module(s) 208. The data 210 includes for example, basic analysis data 214, filtration data 216, and other data 218. The other data 218 includes data generated as a result of the execution of one or more modules in the module(s) 208.
[0042] In operation, the basic analysis module 112 can be configured to receive the interest value parameters. Further, the basic analysis module 112 may determine the peripheral devices that are associated with one or more of the client devices 108, say client device 108-1. The peripheral devices can include but are not limited to a video capturing device, such as a webcam, input devices, such as a keyboard, a mouse, and an optical track pad. As would be appreciated by a person skilled in the art, any user of the client device 108-1 would interact with a website through the use of one or more of such peripheral devices, as illustrated above. The examples are only illustrative and should not be construed to be a limitation. While the user is consuming content from said website, the basic analysis module 112 can be configured to monitor and receive one or more interest value parameters, such as the video stream from the video capturing device, the text inputs from a text input device, such as a keyboard, the mouse-actions, or one or more information related to various social networking websites.
[0043] In one example, the basic analysis module 112 can be configured to activate the video capture device associated with the client device 108-1 to determine the expression on the user’s face. Depending on the expression of the user’s face, whether the user perceived the content of the website positively or negatively, can be determined. For example, the basic analysis module 112 analyzes the video stream by utilizing haar wavelets to detect a facial region of the user. Once the facial region of the user is detected, the basic analysis module 112 can be configured to perform an Active Appearance Model (AAM) fitting. In such a case, the AAM fitting includes recognizing various facial features such as the eyes, mouth, nose, and eyebrows of the user, based on one or more models.
[0044] In one implementation, the basic analysis module 112 can be configured to plot reference points on the recognized boundaries, and compare the recognized boundaries to reference shape models, for example, the reference shape models that can be stored in the basic
13
analysis data 214. Based on the comparison, the basic analysis module 112 further determines which portion of the face corresponds to which respective facial feature.
[0045] Once the facial features are identified, the basic analysis module 112 can be configured to determine one or more expression models from the facial features. Each of the expression models is indicative of various expressions on the facial region of the user. Furthermore, during the course of said consuming, the basic analysis module 112 can log variations in the position or the posture of the user, for example, a seating posture, and the facial expression. In said example, while the user is watching a video on the website, the user may react to certain content in a variety of ways. The user might smile, or frown, or laugh or be indifferent. The user may also raise his hands, or fold his hands, or get up and walk away from the desktop PC during the course of the video. In all these cases, the basic analysis module 112 can be configured to log the different variations in the various seating postures, and the facial expressions. Depending on which expression model the face captured by the video capturing device conforms to, the emotion being expressed by the user during the time the user is consuming the content on the website, can be determined. In one example, the basic analysis module 112 can be configured to store the determined expression models in the basic analysis data 214.
[0046] In one implementation, the basic analysis module 112 can be configured to receive text inputs and the mouse actions from the client device 108, such as the desktop PC, during the course of the consuming. For example, in case the user provides a text input, such as a comment, in response to some content observed on a website, the basic analysis module 112 can be configured to determine said written input in the form of keystrokes. Moreover, the basic analysis module 112 can be compared to store the text inputs, in the basic analysis data 214.
[0047] Similarly, the basic analysis module 112 can be configured to receive the mouse-actions from a pointing device, such as a touchpad, or mouse of the client device 108. In one implementation, the basic analysis module 112 can be configured to take screen captures where an associated mouse action has occurred. For example, if a user clicks a certain button on the website, the basic analysis module 112 can be configured to capture the click as well as take the screen capture of the occurrence. In one implementation, the screen captures can be further analyzed based on Scale Invariant Feature Transforms (SIFT) matching and Optical Character
14
Recognition (OCR) techniques. In one example, the basic analysis module 112 can compare the mouse action and its associated screen capture with reference interest representations. For example, the owner of the website can specify interest representations associated with one or more buttons, where activation of particular buttons on the website signifies a positive response (i.e., the Facebook® ‘like’ button), or a negative response, and these details can be stored in the primary feedback system 110. If the mouse-action along with the screen capture indicates one of the specified interest representations, the basic analysis module 112 can mark the mouse-action as pertinent. Furthermore, this functionality of the basic analysis module 112 can be said to be adaptive in that as various combinations of mouse actions and associated screen captures are received, they can be analyzed to determine whether or not they signify a positive or negative response with regard to website content. Moreover, the basic analysis module 112 can be compared to store information pertaining to the mouse actions and screen captures, as well as the comparison information, in the basic analysis data 214.
[0048] In a further implementation, the basic analysis module 112 can further obtain information from social networking websites which the user is associated with. For example, if the user is a member of Facebook®, the basic analysis module 112 may obtain various inputs provided by other users from Facebook® pertaining to particular content that the user is consuming on the website. In one example, the social network inputs can include, but are not limited to, text inputs from discussions and messages relating to the same content on the website (by the user or other users in a friend circle), in a manner similar to the manner in which the text inputs are received. In said example, topic models and sentiment analysis can be utilized in order to determine whether or not a particular discussion or message pertains to the same content on the website. Here, topic models of interest can be created, and sentiment analysis can be used as a text interpretation technique to see if a particular discussion or message falls into the topic model of interest. Furthermore, ratings from friends circles on the social networking website pertaining to the same content can also be received by the primary feedback system 110. In one example, the basic analysis module 112 can be configured to store the information generated from the receiving of the social network inputs in the basic analysis data 214.
[0049] In one implementation, the filtration module 114 can be configured to filter the interest value parameters as received and analyzed by the basic analysis module 112, based on a
15
plurality of filtration parameters. The filtration parameters can be implemented based on one or more rules, say provided by the administrator of the website. The filtration parameters are provided such that privacy of the users can be maintained while collecting feedback from them. In one example, the filtration module 114 can be configured to blur out a facial region from the video stream in order to preserve a visual identity of the user. The filtration module 114 can be configured to detect the facial region of the user from the video stream by utilizing the boundary locations recognized by the basic analysis module 112. Furthermore, the filtration module 114 can be configured to store the video stream minus the facial region of the user in the filtration data 216.
[0050] In one implementation, the filtration module 114 can be further configured to mask the text inputs received by the basic analysis module 112 in order to protect the privacy of the user. In one example, the filtration module 114 can be configured to use text detection techniques, such as named entity recognizers, and OCR to recognize proper nouns from the text inputs. Furthermore, in this manner, numbers, such as phone numbers, bank account numbers, and other such information can be masked from the text inputs. As described earlier, the text inputs can be received both from text inputs on the website itself, as well as from the social networking website that the user is associated with. The filtration module 114 can be configured to then store said masked text inputs in the filtration data 216.
[0051] In an implementation, the filtration module 114 can be further configured to compile the ratings from the friends’ circles from the social networking websites to form a histogram, and disassociate links to the friends profile to preserve anonymity. In one example, the filtration module 114 can be configured to store the social network ratings histograms in the filtration data 216. The filtered interest value parameters are stored, and can now be utilized for the advanced processing, which is described in the subsequent portion of the draft.
[0052] Fig. 3 illustrates the advanced feedback system 102, in accordance with an embodiment of the present subject matter. In said embodiment, the advanced feedback system 102 includes one or more processor(s) 302, interface(s) 304, and a memory 306 coupled to the processor 302. The functionality of the processor 302 is synonymous with the processor 202 and is not repeated here for purposes of clarity. Moreover, the interface(s) 304, and the memory 306
16
are synonymous with the interface(s) 204, and the memory 306 respectively. The memory 306 also includes module(s) 308 and data 310.
[0053] The module(s) 308 include routines, programs, objects, components, data structures, etc., which perform particular tasks or implement particular abstract data types. In one implementation, the module(s) 308 includes the advanced analysis module 116, a regression rating module 312, and other module(s) 314. The other module(s) 312 may include programs or coded instructions that supplement applications and functions of the advanced feedback system 102.
[0054] On the other hand, the data 310, inter alia serves as a repository for storing data processed, received, and generated by one or more of the module(s) 308. The data 310 includes for example, advanced analysis data 316, regression rating data 318, and other data 320. The other data 320 includes data generated as a result of the execution of one or more modules in the module(s) 308.
[0055] In one implementation, the advanced analysis module 116 can be configured to obtain the interest value parameters and the expression models from the primary feedback system 110. For example, once the interest value parameters have been filtered, say by the filtration module 114, the advanced analysis module 116 processes such filtered interest value parameters to determine an overall interest value. In one example, the interest value is indicative of user feedback, where said user feedback can be positive or negative regarding content provided on a website as discussed earlier.
[0056] In one implementation, the advanced analysis module 116 can be configured to obtain the interest value parameters in the form of continuous variables from the primary feedback system 110. For example, the video stream minus the facial region of the user can be obtained as a continuous value for the duration of the time that the user consumed the content on the website. Furthermore, in another example, the expression models can be presented as a temporal value for the same duration. As described earlier, the expression models are derived from the AAM fitting process.
[0057] The advanced analysis module 116 can be configured to obtain the masked text inputs, the mouse actions, as well as the social network inputs, such as the rating histogram. In one example, the advanced analysis module 116 can be configured to store the obtained interest
17
value parameters in a raw state and store the interest value parameters in the advanced analysis data 316. In another example, the advanced analysis module 116 can be configured to selectively categorize the obtained interest value parameters to a plurality of abstract states (hereinafter interchangeably referred to as abstracted data) and store the abstracted interest value parameters in the advanced analysis data 316. For example, the expression models can be categorized into abstracted expression states, where ‘raised eyebrows’ can be abstracted to mean ‘mild surprise’ and ‘downward curved lips’ be abstracted to mean ‘dissatisfaction’. Similarly, the raw data contained the video stream can be processed to obtain similar abstracted states from the various postures of the user.
[0058] During operation, the advanced analysis module 116 can be configured to utilize the raw data, or the abstracted data, based on a feedback requirement, in order to generate the interest value. For example, if a substantially in-depth user feedback is desired, the advanced analysis module 116 can be configured to utilize the raw data, and in another case, if a substantially less resource intensive used feedback assessment is desired, the advanced analysis module 116 can be configured to utilize the abstracted data.
[0059] In one implementation, the advanced analysis module 116 can be configured to perform depth estimation in order to analyze the video stream. In one example, the depth estimation involves measuring textures and gradients in the video to obtain actual data in the form of a depth map. The advanced analysis module 116 can then be configured to compare the depth map to a plurality of statistical models to determine a match. Furthermore, the depth analysis can also include determining variations in position of the user over time, for example, by measuring a distance of the user from the camera, and temporal variations thereof. In one example, the advanced analysis module 116 can be configured to store data pertaining generated from the depth analysis in the advanced analysis data 316.
[0060] In an implementation, the advanced analysis module 116 can be configured to perform a skeleton fit on the depth map in order to generate specific state abstractions of the user’s body during the course of the content consuming. The specific state abstractions can include but is not limited to, sitting back, leaning forward, folded arms, and standing up. In one example, the advanced analysis module 116 can be configured to store the specific state abstractions in the advanced analysis data 316.
18
[0061] In one implementation, the regression rating module 312 can be configured to process the interest value parameters in order to generate an interest value indicative of the users’ interest of the consumed content. In one example, the regression rating module 312 can be configured to process the video stream, namely the depth map, and the state abstractions, along with the filtered text inputs, the mouse actions (including the screen captures), and the social network inputs in order to generate the interest value.
[0062] In one example, the processing includes assigning weighing factors to each of the interest value parameters in order to generate an overall interest value of the user for the duration that the user consumed the content on the website. According to the present subject matter, the regression rating module 312 can be configured to assess whether each of the temporal interest value parameters indicates an interest or not. For example, certain expression models, such as a smile can indicate a positive interest value, and certain expression models, such as a frown or a neutral expression, can indicate a negative interest value. Moreover, certain body postures, such as folded arms, or a user getting up and walking away, can signify a negative interest value, and a user raising his arms can signify a positive interest value. Similarly, a mouse action can signify a positive interest when the user clicks on an assigned ‘like’ button.
[0063] In one implementation, the regression rating module 312 can be configured to operate on a probabilistic model. In one example, the probabilistic model is based on a perception of a user (perception variable), an emotional state of the user (emotional state variable), and an expression of the user. The user consuming the content on the website perceives the content in a manner unique to him. Furthermore, based on an emotional state of the user, the user displays an expression, which in this case, is an externally visible factor in the probabilistic model. For example, on one hand, a user ‘A’ consuming content on a website, may display a particular expression based on his emotional state at the time of consuming said content, and his manner of perception. On the other hand, a further user ‘B’ consuming the same content on the website may display another expression, based on his emotional state and manner of perception. According to the present subject matter, in one implementation, the regression rating module 312 can be configured to assign weighing factors to each user’s emotional state based on the social network inputs, and measure the interest value for each user independently. In this manner, an accurate and substantially in-depth user feedback method is facilitated,
19
irrespective of the manner in which each user perceives content on a website. In one example, the regression rating module 312 can be configured to store the weighing factors for each user in the regression rating data 318.
[0064] In one implementation, the regression rating module 312 can be configured to determine the emotional state variable of a user, based on the social network inputs. In said implementation, the regression rating module 312 can be configured to access the social network inputs, such as the text inputs, and the user ratings to determine a history of emotional states of the user, with regard to a particular content. In one example, if views and feedback for particular content is common to a plurality of friends on the social networking website, say for example, a group of friends are discussing a new electronic product, the regression rating module 312 can be configured to gather the social network inputs from the plurality of friends regarding said content, in order to collate a common emotional state variable for the users. The regression rating module 312 can be configured to then assign the weighing factors based on the individual inputs as described previously.
[0065] Furthermore, in one implementation, the regression rating module 312 can be configured to follow the pattern of a Conditional Random Field (CRF) modeling, in order to generate an overall interest value for a user, over a period of time. For example, the regression rating module 312 can be configured to calculate a probability distribution, by estimation and interference, of the plurality of interest value parameters along with the emotional state variable. In one example, the generated interest value can be a temporal value of interest values, based on the various interest value parameters. In a further example, the regression rating module 312 can be configured to store the interest value for each user in the regression rating data 318. In another example, the regression rating module 312 can be configured to store the interest value for each user in a secure database, such as the database 106.
[0066] Fig. 4 illustrates a method 400 for feedback capture and aggregation, according to one embodiment of the present subject matter. The method 400 may be implemented in a variety of computing systems, mentioned in description of fig. 1, in several different ways. For example, the method 400, described herein, may be implemented using the primary feedback system 110, and the advanced feedback system 102, as described above.
20
[0067] The method 400, completely or partially, may be described in the general context of computer executable instructions. Generally, computer executable instructions can include routines, programs, objects, components, data structures, procedures, modules, functions, etc., which perform particular functions or implement particular abstract data types. A person skilled in the art will readily recognize that steps of the method can be performed by programmed computers. Herein, some embodiments are also intended to cover program storage devices, e.g., digital data storage media, which are machine or computer readable and encode machine-executable or computer-executable programs of instructions, wherein said instructions perform some or all of the steps of the described method 400.
[0068] The order in which the method 400 is described is not intended to be construed as a limitation, and any number of the described method blocks can be combined in any order to implement the method, or an alternative method. Additionally, individual blocks may be deleted from the method without departing from the spirit and scope of the subject matter described herein. Furthermore, the methods can be implemented in any suitable hardware, software, firmware, or combination thereof. It will be understood that even though the method 400 is described with reference to the primary feedback system 110, and the advanced feedback system 102, the description may be extended to other systems as well.
[0069] At block 402, interest value parameters can be received from a user while the user is browsing content on a website. The interest value parameters can include but are not limited to a video stream from a camera, mouse-actions, text inputs, and social network inputs. In one example, if the user is using a PC desktop to browse the internet, the video stream can be received from a webcam connected to the desktop PC. The mouse-actions, as described earlier, can be a combination of mouse pointer movements and clicks received from the mouse. Furthermore, as described earlier, screen captures can be provided along with the mouse clicks to determine whether or not each of the mouse-actions corresponds to an interest or not. The text inputs can be keystrokes received from a text input device, such as a keyboard. Moreover, the social network inputs can include, as described earlier, text inputs in the form of user messages and discussions relating to the same content as on the website, and friend circle ratings, collectively relating to the same content. In one example, a basic analysis module 112 of a
21
primary feedback system 110 can be configured to receive the interest value parameters as described earlier.
[0070] At block 404, the video stream can be analyzed. In one implementation, the analysis includes detecting a facial region of the user, for example, by utilizing haar wavelets. Upon detecting the facial region of the user, an Active Appearance Model (AAM) fitting can take place in real time. In one example, the AAM fitting can include recognizing locations and boundaries of various body and facial features, such as eyes, nose, mouth and eyebrows of the user in the video stream. Based on the recognized locations and boundaries, one or more expression models can be determined. For example, the recognized locations and boundaries can be mapped and compared against reference shape models that are indicative of various facial expressions, to determine the expression models. In one example, the basic analysis module 112 of the primary feedback system 110 can be configured to analyze the video stream as previously described.
[0071] At block 406, one or more privacy filters may be applied on the interest value parameters in order to obtain filtered interest value parameters. The filtered interest value parameters are designed to cater to a privacy requirement. In other words, the privacy filters can be applied in such a manner that an identity of the user is not disclosed during processing of the interest value parameters. Furthermore, each of the interest value parameters can be filtered according to rules, in order to cater to the privacy requirement. For example, the facial region of the user can be blurred, all proper nouns and numbers can be masked from the text inputs, and links to friends’ profiles can be disassociated from the social network inputs. In one example, a filtration module 114 of the primary feedback system 110 can be configured to apply the privacy filters, in a manner previously described.
[0072] At block 408, the filtered interest value parameters, and the expression models can be forwarded to a further processing system in order to perform further analysis. In one example, an advanced feedback system 102 can be configured to perform the further analysis and finally generate the interest value for each user. In said example, an advanced analysis module 116 can be configured to obtain the filtered interest value parameters, and the expression models from the primary feedback system 110 as described previously.
22
[0073] At block 410, the filtered interest value parameters can be processed based on a probabilistic model. In one example, the processing can involve performing an expression analysis on the expression models to obtain expressive states of the user. The expressive states can be indicative of an interest, or disinterest of the user while consuming the content on the website. In said example, the expression analysis can be performed on raw data, i.e., AAM parameters of the AAM fitting. In another example, the expression analysis can be performed on abstracted data. As described earlier, depending on an end requirement, the raw data can be utilized for a substantially in-depth user feedback, and the abstracted data can be used for a comparatively quicker and less resource heavy user feedback.
[0074] Furthermore, the video stream minus the face region of the user can be processed using depth estimation, followed by a skeleton fitting process. In one example, the depth estimation involves measuring textures and gradients in the video to generate a depth map. In one example, the advanced analysis module 116 of the advanced feedback system 102 can be configured to perform depth estimation in a manner described earlier.
[0075] In an implementation, the skeleton fitting includes utilizing the depth map to generate specific state abstractions of the user’s body during the consuming of content. The state abstractions based on the skeleton fit can include but is not limited to, sitting back, leaning forward, folded arms, and standing up. In one example, the advanced analysis module 116 can be configured to perform the skeleton fit in a manner described earlier.
[0076] During the processing, the text inputs, the social network inputs, the mouse-actions, the abstracted states from the expression models, and the skeleton fit information can be regression rated to generate an overall interest value of the user. In one example, the regression rating can involve utilizing a probabilistic model, which includes a Conditional Random Field (CRF) modeling. The CRF modeling in turn, can include calculating a probability distribution, by estimation and interference, of the plurality of interest value parameters along with an emotional state variable of the user when he was consuming content on the website as described earlier. In one example, the generated interest value can be a temporal value, based on the various interest value parameters consumed over a period of time. In one example, a regression rating module 312 of the advanced feedback system 102 can be configured to process the interest value parameters in the above described manner to generate the overall interest value.
23
[0077] Although implementations for feedback capture and aggregation have been described in language specific to structural features and/or methods, it is to be understood that the appended claims are not necessarily limited to the specific features or methods described. Rather, the specific features and methods are disclosed as exemplary implementations for the capture and aggregation of feedback.
24

I/We claim:
1. A computer-implemented method for feedback capture and aggregation for websites, the method comprising:
obtaining from at least one peripheral device, a plurality of interest value parameters, wherein each of the plurality of interest value parameters is based on an interaction of an individual user with the websites; and
processing the plurality of interest value parameters to generate an overall interest value, wherein the overall interest value is indicative of an interest level of the individual user, for content on the websites.
2. The method as claimed in claim 1, wherein the plurality of interest value parameters comprise at least one of a video stream, an expression model, mouse-actions, text inputs, and social network inputs.
3. The method as claimed in any one of the previous claims further comprising:
analyzing the video stream based on an Active Appearance Model (AAM) fitting and detecting a facial region of the individual user; and
determining at least one expression model based on the analyzing.
4. The method as claimed in any one of the previous claims further comprising applying at least one privacy filter on the plurality of interest value parameters to obtain filtered interest value parameters, wherein the applying comprises blurring the facial region of the user as being displayed in the video stream.
5. The method as claimed in any one of the previous claims further comprising masking text portions from the text inputs provided by the user while interacting with the websites.
6. The method as claimed in any one of the previous claims further comprising disassociating the social network inputs from user profiles from where the social network inputs originate.
7. The method as claimed in any one of the previous claims, wherein the processing the plurality of interest value parameters further comprises:
performing an expression analysis on the at least one expression model to obtain at least one expressive state of the user, wherein the at least one expressive state is
25
indicative of one of an interest, and disinterest of the user during the individual user’s interaction with the websites;
processing the video stream based on depth estimation and skeleton fitting to obtain at least one abstraction state of the user’s body, wherein the abstraction state is indicative of at least one body posture of the user during consuming of the content on the websites.
8. The method as claimed in any one of the previous claims further comprising generating the overall interest value, wherein the generating comprises rating at least one of the interest value parameters and the abstraction states, based on a probabilistic model.
9. An advanced feedback system (102) for capture and aggregation of user feedback, the advanced feedback system (102) comprising:
a processor (302); and
a memory (306) coupled to the processor (302), the memory (306) comprising:
an advanced analysis module (116) configured to obtain a plurality of interest value parameters based on an individual user’s interaction with websites; and
a regression rating module (312) configured to process the plurality of interest value parameters to generate an overall interest value of a user.
10. The advanced feedback system (102) as claimed in claim 9, wherein the advanced analysis module (116) is further configured to analyze one of a raw data and abstracted data in order to process the plurality of interest value parameters.
11. The advanced feedback system (102) as claimed any one of the claims 9 and 10, wherein the advanced analysis module (116) is further configured to:
perform a depth analysis on a video stream to obtain a depth map; and
carry out a skeleton fit based on the depth map, in order to generate specific state abstractions of the user’s body.
12. The advanced feedback system (102) as claimed in any one of the claims 9 to 11, wherein the regression rating module (312) is further configured to operate on a probabilistic model, and
26
wherein the probabilistic model is based on a perception variable, an emotional state variable, and an expression of the user.
13. The advanced feedback system (102) as claimed in any one of the claims 9 to 12, wherein the regression rating module (312) is further configured to generate the overall interest value based on a Conditional Random Field (CRF) modeling.
14. A computer-readable medium having embodied thereon a computer program for executing a method comprising:
obtaining a plurality of interest value parameters comprising at least one of a video stream, an expression model, mouse-actions, text inputs, and social network inputs, wherein each of the plurality of interest value parameters is based on an interaction of an individual user with the websites; and
processing the plurality of interest value parameters to generate an overall interest value, wherein the overall interest value is indicative of an interest level of the individual user, for content on the websites.