The present disclosure relates to an information processing apparatus, information processing method, an information processing system, a display device and a reservation system.
BACKGROUND
[0002]
　By photographing the attendees participated in the entertainment analyzing the results, apparatus for performing evaluation of entertainment it has been proposed (e.g., see Patent Document 1).
CITATION
Patent Document
[0003]
Patent Document 1: JP 2004-272529 JP
Summary of the Invention
Problems that the Invention is to Solve
[0004]
　In this field, it is necessary to shoot such attendees while the entertainment is being performed, the photographer or the like of the attendees will be done without permission is not desirable from the viewpoint of protecting personal information.
[0005]
　Accordingly, the present disclosure, the reaction information indicating the reaction of the user to a given presentation information, it is another object to provide an information processing apparatus that used when the user has permission.
Means for Solving the Problems
[0006]
　The present disclosure is, for example,
　the reaction information indicating the reaction of the user to the presentation information, with the reaction information utilization unit for use of the reaction information is available if it is granted
　an information processing apparatus.
[0007]
　The present disclosure is, for example,
　the reaction information utilization unit, a response information indicating the reaction of the user to the presentation information, use of the reaction information is available if it is granted
　an information processing method.
[0008]
　The present disclosure is, for example,
　the reaction information acquisition unit that acquires response information indicating the reaction of the user to the presentation information,
　with the reaction information utilizing unit for utilizing the reaction information if the use of the reaction information is licensed
　information processing system it is.
[0009]
　The present disclosure is, for example,
　in a series of reservation processing for reserving a seat indicated by the seat information, response information indicating the seat information when viewing a predetermined presentation information, a reaction of the user when viewing the presentation information an input screen for whether to grant the access, a display control unit that performs control to display on the display unit and the information for prompting the permission
　is a display device.
　The present disclosure is, for example
　has a display device described above, and an information processing apparatus connected to a display device,
　the information processing apparatus, and license information indicating whether permission is output from the display device Booked with the reservation information managing unit that manages association with seat information
　is reservation system.
The invention's effect
[0010]
　According to at least an embodiment of the present disclosure, it is possible to provide an information processing apparatus such as the reaction information of the user is used when the user grants. Here, the advantages described in the present invention is not necessarily limited, it may be any of the effects described in the present disclosure. Also, but the present disclosure be construed as being limited by the illustrated effect.
BRIEF DESCRIPTION OF THE DRAWINGS
[0011]
[1] Figure 1 is a block diagram showing a configuration example of an information processing system according to an embodiment.
FIG. 2 is a flowchart showing a flow of processing in an information processing system according to an embodiment.
FIG. 3 is a flowchart showing a flow of reservation processing according to the embodiment.
[4] FIG. 4 is a diagram showing an example of a screen in the reservation process according to an embodiment.
FIG. 5 is a diagram showing an example of a screen in the reservation process according to an embodiment.
FIG. 6 is a diagram showing an example of a screen in the reservation process according to an embodiment.
[7] FIG. 7 is a diagram showing an example of a screen in the reservation process according to an embodiment.
[8] FIG. 8 is a diagram showing an example of a screen in the reservation process according to an embodiment.
[9] FIG. 9 is a diagram showing an example of a screen in the reservation process according to an embodiment.
[10] FIGS 10A and 10B are views for explaining an example of arrangement of a camera module according to an embodiment.
[11] FIG 11 is a diagram for explaining another example of arrangement of a camera module according to an embodiment.
[12] FIGS 12A and 12B are diagrams for explaining a setting example of a camera module according to an embodiment.
[13] FIG 13 is a diagram for explaining an example of arrangement of the biosensor according to one embodiment.
[14] FIGS. 14A ~ 14C are diagrams for explaining several examples of emotion estimation method according to the emotion estimation part according to an embodiment.
[15] FIG 15 is a diagram showing an example of the emotion information.
[16] FIG 16 is a flowchart showing a flow of processing performed after watching the movie.
[17] FIG 17 is a diagram for explaining a process of generating service information according to one embodiment.
[18] FIGS 18A ~ FIG. 18C is a diagram for explaining a plurality of examples of the service information.
[19] FIG 19 is a diagram for explaining another example of the service information.
[20] FIGS. 20A and 20B are views for explaining another example of the service information.
[21] FIG 21 is a diagram for describing a modified example.
DESCRIPTION OF THE INVENTION
[0012]
　It will be described below with reference to the accompanying drawings embodiments and the like of the present disclosure. The description will be made in the following order.
<1. One
embodiment> 1. Information processing system
1-1. Description of the terms
1-2. Example of the configuration of the system
1-3. Configuration of reservation terminal Example
1-4. Configuration server of the example
1-5. Configuration Example of the reaction information acquisition device
1-6. Overview of processing in the information processing system
2. Reservation processing
2-1. Flow of reservation processing
2-2. Screen example in the reservation processing
3. Acquisition of emotional information in the movie when viewing
3-1. Arrangement of the sensor unit
3-2. The operation of the emotion estimation unit
4. Generation of service information
4-1. The flow of processing after the movie viewing
4-2. Operation example of the service information generation unit
4-3. Specific examples of the service information
5. Processing in the case where there is no consent of the use of reaction information
<2. Modification>
　In addition, like the embodiment to be described below are preferred specific examples of the present disclosure, but the present disclosure is not limited to these embodiments and the like.
[0013]
<1. One
embodiment> [1. Information Processing System]
(1-1 discussion. Terminology)
　First, a description will be given of key terms used in the present embodiment.
　The "presentation information", is presented to the user, viewed by the user, appreciation, refers to the information to be watching, and the like. In the present embodiment, specifically, from the content as the presentation information is described as an example a movie.
　The "sensor information" refers to information obtained by the camera module and a biological sensor or the like of the sensor device (data).
　The "response information" means information indicating the reaction of the user to the presentation information, specifically refers to information obtained by analyzing the sensor information. In the present embodiment, by using the information corresponding to the emotional as response information. Therefore, there may be referred to as a "reaction information" and "emotion information". However, "response information" includes a status change of the reaction of the presence or absence or the user not only emotion, a broader concept "emotion information".
　"Licensed information" refers to information indicating the presence or absence of a license for the use of "response information".
　The "service information" refers to information that is presented to the user who licensed the use of "response information".
[0014]
(1-2. Configuration Example of System)
　FIG. 1 illustrates a configuration example of an information processing system according to the present embodiment (information processing system 100). The information processing system 100, for example, a reserved terminal 1, and a server 2 which is an example of an information processing apparatus, and a reaction information acquisition device 3. For example, the reservation system is constituted by reserved terminal 1 and the server 2.
[0015]
(1-3. Configuration Example of the reservation terminal)
　Next, a description will be given schematic configuration example of each device. First, a configuration example of the reservation terminal 1. Reservation terminal 1, may be a terminal to be installed in public places and shops, a personal computer or a smart phone with the user U, may be a tablet computer, or the like. In the present embodiment, the reservation terminal 1 is described with the smart phone with the user U as an example.
[0016]
　Reservations terminal 1 which is an example of a display device, for example, the reservation terminal control unit 11 is an example of a display control unit, a display unit 12, and a communication unit 13. Reservation terminal control unit 11, for example, a CPU is constituted by (Central Processing Unit), and controls each unit of the reservation terminal 1. Display unit 12 is an LCD (Liquid Crystal Display) or an organic EL (Electro Luminescence) panel or the like. Display unit 12 according to the present embodiment is configured the operation input to the display unit 12 as admissible touchscreen. The communication unit 13 performs communication with the server 2 and the like via the Internet or the like. Note that the reservation terminal 1 may have a known configuration to other configurations described above.
[0017]
(1-4. Configuration Example of Server)
　server 2, for example, the server control section 21, a reservation information management unit 22, a service information generation unit 23 is an example of response information using unit, have the communication section 24 are doing. Server control unit 21 is constructed of, for example, by CPU, and controls each unit of the server 2.
[0018]
　Reservation information management unit 22 is a storage unit or the like for managing the reservation information based on the information entered in the reservation terminal 1. Reservation information management unit 22, for example, the ID of the user U, the attribute information of user U, the seat information indicating the seat by the user U has reserved, correspondingly manages the license information. The attribute information of the user U, an information indicating an attribute of the user U, for example, is the user U of gender, nationality, residential area, race, age and the like. Incidentally, the attribute information of the user U may be one which is entered when a seat reservation, may be one which is previously registered before booking.
[0019]
　Reaction information according to this embodiment is information indicating the reaction of the user U for movies, and more specifically, a feeling of a user U at the time of viewing of the movie. Emotions, other such as emotion, sleepiness, empathy, relaxation, excitement, a concept that includes a mental state, such as concentration. Emotion is not limited to what is directly detected as sensor information, or may be estimated based on the sensor information. For example, as a model mapping the plurality of emotion, Russell emotion ring model is known. The Russell emotion ring model, based on the two indicators of free state and the awake state, and estimates a variety of emotions. Emotions may be classified in accordance with such emotion estimation model.
[0020]
　Service information generation unit 23 generates the service information provided to the user U who licensed reaction information. Service information is generated based on the reaction information of the user U. It will be described in detail later in the service information. User U who licensed reaction information can obtain service information, for example, after viewing or while watching a movie. The communication unit 24 performs communications with the reservation terminal 1 and the reaction information acquisition apparatus 3 or the like via the Internet or the like.
[0021]
(1-5. Configuration Example of the reaction information acquisition apparatus)
　reaction information acquisition unit 3 includes a controller 31, a sensor section 32, and a feeling estimation unit 33, a communication unit 34. For example, the reaction information estimation unit is configured by a sensor section 32 and the emotion estimation part 33. Control unit 31 is constructed of, for example, by CPU, and controls each unit of the reaction information acquisition device 3. Sensor unit 32 is a sensor that measures a state (reaction) of the user U at the time for example of movie viewing. Sensor portion 32 according to this embodiment, for example, a camera module 32A, but has a biological sensor 32B, a configuration in which based only on the camera module 32A, may be configured based only on biometric sensor 32B. The camera module 32A is a device for capturing the user U to watch movies, a lens 321, a CCD (Charge Coupled Device), an image sensor 322 formed of CMOS (Complementary Metal Oxide Semiconductor) or the like, an image processing unit 323, It has a configuration including the camera controller 324 which controls the entire camera module 32A. Biological sensor 32B is a sensor that measures biological information of a user U. The biological information of the user U, the body temperature of the user U, heartbeat, pulse, blood pressure, perspiration, brain wave, and movement of the body or the like.
[0022]
　Emotion estimation part 33, based on the sensor information measured by the sensor unit 32, estimates the emotion of the user U (analysis) to. The communication unit 34 performs communication with the server 2 and the like via the Internet or the like.
[0023]
(1-6. Processing overview in the information processing system)
　the process performed by the information processing system 100 will be described in brief with reference to the flowchart shown in FIG. In step ST1, reservation processing for movies reservation using the user reservation terminal 1 is performed. User U designates a seat which is viewing position with specifying the movie viewing target. The user U is further to input permission information indicating whether or not to grant the use of the reaction information. The use of the reaction information, for example, say that the reaction information to grasp the preference trend of cultivating new customers and movies, and is used as a marketing information related products sales of movie. Reservation information input via the reservation terminal 1 is transmitted to the server 2 through communication. Server 2 manages the reservation information transmitted from the reservation terminal 1. Then, the process proceeds to step ST2.
[0024]
　In step ST2, to watch a movie that the user U has reserved in a movie theater. Reaction of the user U is estimated by the reaction information acquisition apparatus 3 installed in a movie theater when viewing a movie. The estimation result is transmitted from the reaction information acquisition device 3 to the server 2. Then, the process proceeds to step ST3.
[0025]
　At step ST3, the whether or not the user U has licensed the reaction information is determined. In this determination process, if the user U does not licensed reaction information, the process ends. On the other hand, if the user U is licensed reaction information, the process flow of operation proceeds to step ST4.
[0026]
　In step ST4, the server 2 refers to the seat information and permission information, and generates the service information to the user U who licensed reaction information. This service information is presented to the user U. Then, the processing is terminated.
[0027]
[2. Reservation Process
(2-1. Reservation processing flow)
　Next, detailed contents of each process. Figure 3 is a flowchart showing a flow of reservation processing according to the present embodiment. In step ST11, the login screen is displayed on the display unit 12 of the reservation terminal 1. Then, the process proceeds to step ST12. In step ST12, specification of movies and show times is made. Then, the process proceeds to step ST13. In step ST13, the selection of the seat is made. Then, the process proceeds to step ST14. In step ST14, user impact envisioned by licensed reaction information is displayed, in particular displayed advantage that the user obtained Merritt is. Then, the process proceeds to step ST15. In step ST15, the presence or absence of the license is confirmed. Then, the process proceeds to step ST16. In step ST16, reservation of the movie is completed.
[0028]
(2-2 screen example in. Reservation process)
　will be described a specific example of the processing. Here, a description will be given screen example in the reservation process. Screen example to be described below is displayed on the display unit 12 according to the control of the reservation terminal control unit 11.
[0029]
　The user U connects to the reservation page by using a dedicated site or a specific application using the reservation terminal 1. Figure 4 shows an example of the login screen 41 displayed on the display unit 12 of the reservation terminal 1 in step ST11. In the login screen 41, for example, and the input items 41A for the member, and the input item 41B for non-members is displayed. Input items 41A for the member includes a display 410A to enter the membership number, and a display 410B for entering the password. In the present embodiment, when the members are pre attribute information of the members registration, when logged in member can be omitted inputs attribute information required non-member. User U, after the necessary input to the log-in screen 41, to select the display 410C of the "Next".
[0030]
　In the login screen 41 when the purchase without nonmember That log is selected, for example, customer information input screen 42 shown in FIG. 5 is displayed. In the purchaser information input screen 42 shown in FIG. 5, the name of the order to enter, phone number, contact information such as e-mail address, the field for entering the payment method and the like are displayed the information necessary for the purchase of movie . Furthermore, the customer information input screen 42, the attribute information of user U (e.g., sex, age) of field for inputting the display.
[0031]
　Figure 6 shows an example of a movie selection screen 43 for specifying a movie in step ST12. The movie selection screen 43, for example, a movie theater name (for example, Tokyo movie theater AB), the movie name (movie AAA), exhibition day (May 1 (Monday)), multiple screenings start time, running time, booking information including the presence or absence of display. User U selects the next available movie using the movie selection screen 43.
[0032]
　Figure 7 shows an example of a seat selection screen 44 displayed on the display unit 12 during the processing of the seat selected in step ST13. In the seat selection screen 44, for example, the arrangement of the seats, the seat display 44A including a seat or the like of the position and Selected of booking Vacancies are displayed. User U, to select the seat you want to see the seat display 44A reservation. Incidentally, the user U who has logged multiple seats corresponding to one of the account corresponding to the user U may be reserved more specifically. When a plurality of seats are reserved may be presumed that a predetermined relationship between a plurality of users U who uses the seat there. For example, 2 when the seat has been reserved in the first account, the user U to use the respective seats, lovers, estimated to be a couple or friends, if three or more seats 1 account is reserved the user U to use the respective seats, may be presumed to be a family member or friend. In the case of book two or more seats 1 account, not only the user U that has an account, it is also possible to determine the input of the attribute information of the other users U about the reservation.
[0033]
　Figure 8 shows an example of a screen displayed on the display unit 12 during the process in step ST13 (the screen 45). On the screen 45, as the service information, personality diagnosis after watching the movie, specifically, viewing type diagnosis, viewing in emotional map, the display 45A of the effect that can provide compatibility check, etc. are displayed. User U the desired service information from the display 45A is selected. It should be noted that the compatibility check, a plurality of seats may be possible to select in the case which has been reserved in one account.
[0034]
　In the screen 45, to generate the service information, emotion information is acquired by the emotion estimation of the user U is made at the time of viewing of the movie, emotion information display 45B to the effect that may be utilized for marketing information is displayed. More specifically, or the user U is photographed, it may be displayed or made to measure the biological information. Moreover, the screens 45, described and the effect is not performed out-of-target use, display or Disclaimer description such that the substance is the information available in a form that does not steward string from the user U by statistical processing, consent of the user U is If not, not performed emotion estimation, display 45C including a display to the effect that the service information is not presented is displayed.
[0035]
　Some user U, or are taken when viewing a movie, may be present user U feel psychological aversion to being acquired biometric information. Accordingly, the user U, in addition to the presentation of service information (indication as to incentives) display prompting the permission of the use of emotion information are preferably displayed. In the screen 45 according to the present embodiment, for example, when licensed emotion information, display 45D is displayed in addition to 100 points of presentation of service information is added.
[0036]
　Of course, emotion estimated at the time of viewing of the movie, not to be forced to the user U. If you do not want the presentation of service information, it is sufficient to check the check box that is included in the display 45E to the effect that do not agree. Furthermore, In addition, even when receiving a service of the presentation of the service information, it may be displayed that does not take personal information to specifically identify the user U individuals.
[0037]
　In addition, the screen 45 on the service information, as shown in FIG. 9, may be a trial play screen of the quiz concerning the movie. In the screen example 46 shown in FIG. 9, quizzes to predict the most rises scene in a movie is displayed. Answer quiz is determined based on the emotion information of a plurality of users U in the viewing of the movie, the results to the user U who licensed emotion information is fed back after watching the movie. Screen example shown in FIG. 8 and FIG. 9 may be displayed in chronological order or simultaneously. In addition, instead of a quiz about the movie, contents, etc., to encourage the input of prior expectations for the movie may be displayed. By providing the trial play screen before cinema viewing, it is possible to expect the effect of promoting the permission of the use of the emotion information to the user U, Upon presenting the reaction based on the emotion presumption in movie viewing after movie viewing, before viewing can present comparisons, with the expected value, it is possible to draw the interest of more user U. Furthermore, expectations and the various projections of the front view, comparison information with the results of actual emotion estimation, a useful information in view of the foregoing marketing.
[0038]
　Reservation information inputted using the reserved terminal 1 is transmitted from the communication unit 13. The reservation information is received by the communication unit 24 of the server 2 is managed by the reservation information managing unit 22 of the server 2.
[0039]
[3. Acquiring emotion information in film when viewing]
(3-1. Layout example of the sensor unit)
　after the reservation process, the user U is viewing actually a movie. 10A and 10B show an example of the arrangement of the camera module 32A of the sensor unit 32. The camera module 32A is disposed within a predetermined theaters 51 in cinemas. For example, as shown in FIG. 10A, the camera module 32A is disposed toward the audience 52 side from the front of the theater 51 (screen side). As shown in FIG. 10B, the audience 52 a plurality of areas (for example, two areas) by sharing the camera module 32A for each area may be located. Thus the resolution of each user to be the one camera module photographing target can be ensured. Further, a plurality of camera module 32A may be photographed area in common from a different angle. Accordingly, even when there is movement to the user, it can be recognized, such as accurate expression. It may be used a plurality of camera modules 32A is disposed in proximity to an array for one area. Based on the image obtained by the camera module 32A, the user U is photographed present in the shooting range, the face recognition of the user U is made.
[0040]
　The camera module 32A may be provided for each seat as shown in FIG. 11. In Figure 11, the seat 52A and the front of the seat 52B that of the total seat is shown. In general, the seat of the movie theater are provided such that the height of the seat 52A on the rear side is higher. The camera module 32A is provided near the seat back 52B, the user U of the seat 52A may also be captured by the camera module 32A. The camera module 32A may be provided a plurality for each seat. By providing the camera module 32A in every seat, more precisely it is possible to detect the user's expression or the like.
[0041]
　Setting values ​​of the camera module 32A may be fixed or may be variable in synchronism with the progress of the movie. The setting values ​​of the camera module 32A as an example in the case of a variable, for example, as shown in FIG. 12, it is possible to change the setting value according to the brightness of the movie. The set value, for example, include a dynamic range. 12A is a diagram showing a brightness that changes with the progress of the movie schematically, a vertical axis indicates the horizontal axis represents time represents the lightness. Figure 12B is a diagram temporally corresponding to FIG. 12A, shows a dynamic range that is changed with a change in brightness. For example, processing for enlarging a dynamic range in accordance with the control of the camera control unit 324 when the brightness exceeds the threshold value TH is performed. Accordingly, even after changing the brightness by movie scenes, it can detect facial expressions exactly user U. The camera module 32A may be to detect changes in brightness in real time by a sensor or the like, may be detected a change in brightness on the basis of the captured image, it is previously measured by chance test reproducing such movies it may be read out a change of the stored brightness.
[0042]
　Setting value is not to be limited to the dynamic range, sensitivity of the image sensor 322, the frame rate of the image sensor 322, the shutter speed, resolution, or a combination thereof or the like. The change other than the brightness, for example, considered the movement of the user, when the movement of the user is detected, it is possible to change the shutter speed as a set value. In addition, during the playback of the movie, it is not necessary to the camera module 32A is performed always shooting, it may be to carry out the shooting only period that has been set. In addition, shooting may be a single color rather than color, may be used to detect the infrared.
[0043]
　Biological sensor 32B is body temperature of the user U, heartbeat, pulse, blood pressure, perspiration, brain wave, and detects the movement of the body or the like. As shown in FIG. 13, the biological sensor 32B, for example, is incorporated in the glasses 53A for 3D that is lent at a movie theater. The sensor mounted in the biological sensor 32B, for example body temperature, pulse rate, perspiration, brain wave, the detection of such motion is suitable. Biological sensor 32B has been incorporated into the seat 53B, for example a pressure sensor, temperature sensor, a pulse wave sensor, a heart sensor, brain wave sensor, and may be a combination of these. The biological sensor 32B may be a device to be lent to the user U who licensed reaction information. As the camera module 32A shown in FIG. 11, the biological sensor 32B are those provided in front of the seat (for example, thermography and, using reflections wave sensor) it may be. In recent years, equipment wristband or smartphone with the user U, inside clothing shirts, etc. have a function for measuring biological information of the user U. The reaction information acquisition device 3 may be obtained by short-range wireless communication such as the biometric information of the user U from such equipment. Log of other emotions estimation service which the user is commonly used may be used. Seat In this case, the user U, may be presented an indication for prompting a connection with the device having the user U and the reaction information acquisition unit 3, which constitute the device having the user U, the reaction information acquisition device 3 establishing a short-range wireless communication in accordance with proximity of equal, it may be automatically obtains biometric information of the user U through the short-range wireless communication.
[0044]
(3-2. Operation of emotion estimation part)
　sensor information obtained by the camera module 32A and the biometric sensor 32B is supplied to the emotion estimation part 33. Emotion estimation part 33 estimates the user's emotional based on the sensor information. Method of estimating the emotion based on the sensor information can be applied a known method. For example, the emotion estimation part 33 is included in the sensor information, and estimates the emotion of the user U from the feature point and the posture of the face of the photographed user. More specifically, the emotion estimation part 33 performs a face recognition based on the sensor information, the expression analysis of the recognized face, from the result to determine the element values required for emotion estimation. For example obtains the smile degree indicating the degree of smile, and to estimate the degree of emotion in the absolute value (e.g., "Joy"), if the smile degree exceeds a predetermined threshold, the user has an "joy" it may determine that the state. Alternatively, or ratio of multiple elements emotions, by identifying the dominant factor feelings in a plurality of elements emotion, the final emotional may be estimated. Alternatively, regardless of the element emotion may estimate any emotion directly from expression of the recognized face using machine learning and the like. Here, when estimating the emotion, by referring to the biometric information contained in the sensor information, it is possible to estimate the emotion more accurately. Alternatively, without performing photographing by the camera module 32A, it is also possible to perform emotion estimation based on only the information of the biometric sensor 32B.
[0045]
　Figure 14A ~ 14C are shows an example of results of emotion estimation by emotion estimation unit 33. For example, as shown in FIG. 14A, there emotions (eg, surprise) a change in the estimated value corresponding to the are the result of emotion estimation. As will be described as an example surprise FIG 14A ~ FIG 14C, in the same manner, changes in the other emotions are also estimated. Results of emotion estimation for each predetermined frame rate, can be estimated in real time, for example, as a curve shown in FIG. 14A, the strength of the degree of emotion is derived. Presentation of results of emotion estimation is not limited to changes in the degree of emotion. For example, as shown in FIG. 14B, estimates corresponding to the surprise of feeling, the reference value period exceeds the STA (see FIG. 14A), it may be presented as being surprising state. By thus presented, it is possible to see the emotion estimation result in two values ​​whether the surprise state. Further, as shown in FIG. 14C, the average value of the estimated value corresponding to the surprise of feeling for each scene may be the result of emotion estimation. By presented in this manner, it is possible to evaluate the emotion of the user U on a scene-by-scene basis. At this time, the reference value STA shown in Figure 14A, may be different for each scene.
[0046]
　Estimates corresponding to a certain emotion may be a relative value to the average of all users for the scene of the movie. Average value used for the comparison is, for example, a value obtained corresponding to a specific scene (reference scene) that is made generally easier to recall the specific emotion in the movie. Reference scene may be provided respectively for each emotion to be estimated target. For example, for "joy" is, many of the user U and has been predetermined reference scene the scene determined that easy to embrace the emotions of joy, for "sadness", the determination and easy to embrace many of the user U sadness of emotion the predetermined scene which are capable of the reference scene. Also, the estimated value corresponding to a certain emotion may be a variation in the average value and unit time in a unit not per scene time.
[0047]
　Incidentally, the posture of the user U, some looking like, there may be a case where emotional estimation based on sensor information can not be appropriately performed. For example, in the case where the camera module 32A is the face of the user U are captured, if the face of the user is hidden by hats and sunglasses, can not be performed appropriately emotion estimation by emotion estimation unit 33. In such a case, the metadata indicating that it was unable to emotion estimation is stored in association with the corresponding scene. Alternatively, the metadata indicating the accuracy of emotion estimation may be stored.
[0048]
　Results of emotion estimation by emotion estimation unit 33, the emotion information is obtained. Figure 15 shows an example of a portion of the emotion information. Emotion information, for example, movies (type of content), scene, seat information, the estimated value corresponding to a predetermined emotion is data associated. In the specific example shown in FIG. 15, AA3 as information for identifying a movie is described, the scene number 1, 2, and 3 is described, 0035 is described as a seat information, estimates corresponding to the surprise of feeling (for example, 82 .2) is described, estimates corresponding to the feelings of sadness (e.g., 20.5) is described. Emotion estimation unit 33 supplies the emotion information to the communication unit 34. The communication unit 34 supplies the supplied emotion information to the server 2.
[0049]
　Incidentally, when the reaction information obtained in sequence, may be associated with different time stamps scene. Further, when the reaction information obtained in sequence, rather than the time information of the movie, other metadata (for example, a chapter, production intent information filmmaker) may be associated with.
[0050]
[4. Generation of Service Information
(4-1. Processing Flow after movie viewing)
　FIG. 16 is a flowchart showing the flow of processing the user U is performed after watching a movie. In step ST21 viewing of the movie is finished. Emotion information based on sensor information at the time of viewing of the user U of the movie is estimated, emotion information is transmitted to the server 2. When transmitting emotion information may be superimposed the random data in order to ensure security (sometimes referred to as a differential privacy, and the like.). Then, the process proceeds to step ST22.
[0051]
　In step ST22, the service information generation unit 23 of the server 2 generates the service information based on the emotion information. Then, the process proceeds to step ST23. Step ST23, the service information that is generated is delivered. For example, the service information generated is delivered to the reservation terminal 1. Delivery to reservation terminal 1, for example, transmits the guidance regarding the service information for the reserved terminal 1 by e-mail, the user U to the address on the network that are described in the guide is made by accessing. After the movie viewing to a user may be the service information is displayed on the display unit 12 of the reservation terminal 1 by holding the reservation terminal 1 in the installed terminal to the cinema, using a specific application service distribution of information may be performed. Moreover, it is not limited to electronic media, service information printed on paper may be provided to the user U.
[0052]
(4-2. Service information generator operation example)
　will be specifically described the processing in film after viewing. Emotion information from the response information acquisition device 3 to the server 2 is transmitted, the emotion information is received by the communication unit 24 of the server 2. Emotion information is supplied from the communication unit 24 to the service information generation unit 23.
[0053]
　Figure 17 shows a an example of reservation information managed by an example and the reservation information management unit 22 of the emotion information sent from the reaction information acquisition device 3. Since the emotion information is similar to that illustrated in FIG. 15, and a redundant description is omitted. Reservation information, for example, a seat information indicating a seat reserved by the user U, the attribute of the user U who reserved the seat, and license information is information and contacts associated. License information is, for example, a logical value "1", a flag "0", in this embodiment a "1" is licensed on the use of reaction information "present" and then the reaction information to "0" license for the use is a "no". Contacts, for example, is a mail address of reservation terminals 1, it may be in the e-mail address, etc. of other devices. Moreover, the account information of the user U may be associated used when booking process on the seat information.
[0054]
　For example, the seat information 0035 of reserved seats, "less than 20 generations" as an attribute of the user U, "male" is associated, "1" is associated as license information, associating a predetermined email address It is. For example, the seat information 0036 of a reserved seat, "30's" as an attribute of the user U, "female" is associated with, "0" is associated as license information, predetermined mail address is associated with ing.
[0055]
　Service information generation unit 23 refers to the reservation information, to extract the seat information corresponding to the available license from the emotional information. Then, for example, by using the emotion information corresponding to the same seat information, an attribute of the user U that has been described in the reservation information, it generates the marketing information. Marketing information may include, for example, in what what scenes of any movie user U attribute is information indicating whether showed any emotion. Such marketing information, filmmakers, it is possible to recognize what kind of movie or scene is accepted liable for any region and attributes, making it possible to reflect the marketing information in the production of the next movie it can. In particular, in the present embodiment, publicly disclosed movie estimates the emotion information about, and, since the so benefits can be obtained by licensed emotion information, potential use much emotion information large, it is possible to obtain significant marketing information easily. Incidentally, generally not published movie, in a limited such premiere environment, also to provide a service of the present embodiment, the effect is obtained that the preference for the user of the movie can be acquired along with emotion information. The herein marketing information may also include information corresponding to later-described service information.
[0056]
(4-3. Specific examples of the service information)
　in the present embodiment, so as to provide service information to the user as a merit by licensed emotion information. Service information generating unit 23 refers to the reservation information, extracts the seat information corresponding to available license from the emotion information, and generates the service information using the emotion information corresponding to the seat information. Service information, for example, a characteristic of the user U based on the response information. Specific examples of the service information will be described with reference to FIGS. 18A ~ FIG 18C.
[0057]
　Service information shown in FIG. 18A, with respect to the time series of movies, emotional changes A, emotion B and emotional C, is more intended specifically associating the graph showing the change in the estimated value of each emotion. It should be noted, emotion A ~ emotion C is, joy, surprise, etc., be anything. Time series of movies, may be indicated by the time information, for clarity for the user U, in this example, a typical scene (Scene 1-4) thumbnail images (reduced images) displayed by IM1 ~ IM4 ing. The graph may be what is called a heat map display for indicating a change in color intensity rather than the change of the line. The graph also as shown in Figure 14B or FIG. 14C, or presents a predetermined emotion nestled if it exceeds a certain reference value, may or presented for each scene. By carrying out such a presentation, the user U can be tied to the time series and the scene of the movie, look back a change of their own emotions.
[0058]
　Although service information illustrated in FIG. 18B is substantially the same as the service information shown in FIG. 18A, but a graph showing changes in emotion, using icons corresponding to the emotion (for example, an icon indicating facial expressions) there.
[0059]
　Service information may be obtained by displaying the emotion was uplifting scene in the rankings format. For example, as illustrated in FIG. 18C, the upper three scene estimation value is high which corresponds to the surprise of emotions, as the upper three scenes and the service information was higher estimates corresponding to the feelings of pleasure It is presented. Each scene is represented respectively by the thumbnail images IM5 ~ IM10.
[0060]
　Example shown in FIG. 18, but which was an absolute characteristic based only on the emotion information about a user U, presenting information relative to the positioning of its own emotion information in the emotion information of other users U it may be a characteristic. For example, it may be a feeling map shown example in Figure 19. In the emotional map of FIG. 19, six of emotion as an example (anger, surprise, sadness, excitement, discomfort, fun) and is defined, in the emotional map, the extent of each emotion that user U felt at the time of viewing of the movie There has been mapped by the graph 61. Further, the emotion map, the average degree of each emotion by other users U felt is mapped graphically 62. By looking at this emotional map, there is a user U is able to recognize the difference of perception for the movie of the other user U. Data to be compared may be a user's emotional information having a same attribute as the user U, may be a whole emotion information, or both. The service information is not limited to the emotion map, the user U may be those significantly reacted scenes with other users U showed a scene that significantly react contrasting. Incidentally, it may be displayed together is illustrated service information in FIGS. 18 and 19. Note that comparison of such other user, it may be performed in the method presented in Fig. 18A to C.
[0061]
　Figure 18 and additional information may be included in the service information shown in FIG. 19. Such additional information, include the presentation of the recommendations of the movie based on the emotion information. Here we recommend movies, in contrast to the movie and the emotion information, can be selected from the category that is estimated that the user prefers to harbor such feelings. For example, for a user U felt a lot of anger during viewing of the movie, when there is a tendency that the user that harbor similar feelings like an action movie presents the action movies to the user U as recommended movie. Similarly, the user U felt a lot of sadness can be presented to love movies as recommended movie. Additional information may be presented as a result of personality diagnosis. For example, personality diagnosis result of a "thick to information" for user U felt a lot of sadness, may be presented as additional information. In addition, for the user U is not much change in the emotional personality diagnosis result of to be "calm" is, may be presented as additional information.
[0062]
　Feedback on the service information may be possible. For example, as shown in FIG. 20A, the thumbnail images IM11 the user U corresponds to the most surprised scene based on emotion information is displayed on the display section 12 as the service information. Display scene, really display 71 for selecting whether the user U is most surprised scene is displayed at a position adjacent to the thumbnail image IM11. The user U is selected using the results own subjective perception matching and whether, for example, reserved terminal 1 indicated by the service information. Selection result is transmitted from the reservation terminal 1 to the server 2 as the feedback information. Note that this selection may not be selected without being limited by force users U. Feedback information transmitted to the server 2 corresponds to a corresponding emotion information. If the feedback information is obtained, it may be emotion information is corrected by the feedback information. The feedback information may correspond or user U that may be utilized in processes such as improvement of learning model for predicting an emotion information of the user U with the same attributes as the user U.
[0063]
　As described above, with one account with the user U has, for example, when two seats reserved is likely to view movies lovers and couples. In such a case, it may present information related to the user U to use the respective two seats as service information. 20A and 20B show an example of such a service information. For example, for each of the 2 seats are emotion information estimates, trend emotions of the user U in each seat is displayed by the thumbnail image IM12 ~ IM14 as scenes sympathy same or close to that scene. Further, each user U (embraced strongly certain feeling) to separate reacted scene is represented respectively by the thumbnail images IM15 and IM16. Further, the display 72 is displayed about the good compatibility. For example, becomes higher probability of good compatibility greater the number of empathy scene (%) is displayed.
[0064]
　In addition, two users U is not a lover or husband and wife, there may be a case of friends and parents. So, a group of relationship (husband and wife, lover, friend, parent-child, etc.) are displayed together, the user relationship to actually appropriate may be allowed to select. Selection result may be performed to diagnose the compatibility with different algorithms depending on. In addition to this, and movies that are considered user U of each other can relate, and extracted based films are considered to be preferred in order to understand each other the emotion information, it may be presented the result as the service information.
[0065]
　At the time of booking process, in the case of pre-expected quiz about the movie it has been carried out, the answer to it may be presented as a service information.
[0066]
　As described above, the user U can receive presentation of service information by licensed emotion information. User U it is possible to look back on the changes in the self-feelings at the time of viewing of the movie by receiving the service information. The user U can obtain characteristics and feelings himself holding for movies, compatibility, etc. with the other party, a variety of information corresponding to the service information. This way, the emotion information to be obtained, to provide to the user as the service information represented by these can provide motivation to licensed emotion information to the user, more users against, it is possible to prompt the permission of the use of emotional information. Of course, the user if the do not want the service information may need to license the use of emotional information in the reservation process.
[0067]
[5. Processing when there is no permission of use of the reaction Information
　will now be described processing performed when there is no permission of use of the reaction information. If there is no permission of use of the reaction information, the corresponding data is discarded. For example, the service information generation unit 23 of the server 2 reads out the seat information permission information by reference to the reservation information corresponding to "0", (erased) discards data of emotion information corresponding to the seat information. In addition, the server 2, license information "1" may be available only the data of emotion information corresponding to the seat information tied. The server 2 may only be used for information that is granted.
[0068]
　Reservation information may be supplied to the reaction information acquisition device 3 from the server 2. Then, the emotion estimation unit 33 refers to the reservation information, the license information is the result perform emotion estimation on only the sensor information supplied from the sensor unit 32 provided corresponding to the seat information of "1" it may be output. Further, reservation information may be supplied to the sensor unit 32. Then, the sensor unit 32, only when permission information corresponding to the seat itself is provided is "1", the sensor information may be output to the emotion estimation unit 33 a. By determining the contents of the license information in a reaction information acquisition unit 3, it is possible to reduce the processing load in the sensor unit 32 and the emotion estimation part 33.
[0069]
　Incidentally, the reaction information without attaching the attribute of the user, and store mixed with other data, in other words, in a manner not known as personal information (and meaningless) may be stored (personal information unconsolidated reduction). Alternatively, also by statistically processing of the obtained information, data can be utilized in a form that does not lead to be private. Thus, it is not always necessary to discard the reaction information about users that did not licensed. At this time, we perform information use in conjunction with the attribute information regarding the user who licensed reaction information, the information use in a form not steward string to be private, as described above with respect to users who have not permission performed such may be provided a difference in handling between the reaction information with users that did not grant the user who has permission.
[0070]
<2. Modification>
　Although the embodiment of the present disclosure has been specifically described, the contents of the present disclosure is not intended to be limited to the above-described embodiment, and various modifications can be made based on the technical idea of the present disclosure it is.
[0071]
　In one embodiment described above, a movie as an example of the presentation information and examples, but is not limited thereto. For example, presenting information, theater, vaudeville, live performances, sporting events, main playback before the publicity of the movie, it may be in the exhibition, such as works of art. In particular, also be able to provide the technology for the exhibition, such as works of art, artist, it is possible to know what was holding what kind of emotion in what works viewers. In this case, for example, input of the license information is made at the time of purchase of appreciation ticket.
[0072]
　In addition, the present disclosure is not in the content presentation-type event such as a movie, can also be applied to a participatory event. Is the participatory event, mention may be made of horse racing, a bicycle race of watching and theme park attractions. For example, based on the response information of the audience, to create a distribution of the reaction information in the stadium as shown in Figure 21. For example, audience location where hatching 81 is attached feels uplifted, mood audience locations hatching 82 is attached to the opposite is shown that the calm. In this way, the support team and cheer to the subject (horse racing if the horse), reaction information for each audience of attributes is visualized. Thus, for example, the audience although age is raised audience teenage 30s can be visualized state are calm. Use the reaction information, time changes and so far the audience of excitement, heat map of reaction information in the audience may be created. Reaction information visualization or delivered in real time to the mobile terminal or displayed on the screen at the stadium, the user who licensed the reaction information with. As a result, the user U, it is possible to recognize the upsurge situation in stadiums and theme parks.
[0073]
　Each function in the information processing system 100 according to an embodiment described above, it can be any device plays. For example, the server 2 is to have a feeling estimation unit 33, may be emotion information in the server 2 is estimated. In this case, only the sensor information is transmitted from the reaction information acquisition device 3 to the server 2. To reduce the bandwidth in a communication, the characteristic data of the sensor information (e.g., characteristic points such as the face recognition) only may be transmitted.
[0074]
　Further, it may be configured with the service information generation unit 23 reaction information acquisition device 3. The reaction information acquisition unit 3 generates the service information, the generated service information may be transmitted to the reservation terminal 1. Further, may be configured with all the functions of the server 2 is the reaction information acquisition device 3 may be configured by the information processing system 100 is comprised of a reserved terminal 1 and the reaction information acquisition device 3. By appropriately changing the configuration like this, the reaction information acquisition device 3 may be the case that functions as an information processing apparatus.
[0075]
　BD (Blu-ray Disc) (registered trademark), the playback equipment to play the DVD (Digital Versatile Disc), etc., there are those having a digest reproduction referred to as the ability to play by reducing the main. Disk author, may be created a disk to determine the delimiter (chapter) in the location and main to digest played using the reaction information available has been licensed. For example, places that surprised many users, excited place, disk may be created to digest reproduction is performed in the center of the excitement the point. The user who licensed the use of the reaction information, may be able to create a disc that reflects the reaction information of their own. Not limited to a disk, the reaction information in the reproduction of the content distributed via a network may be reflected.
[0076]
　Camera module 32A in the above-described embodiment is not intended to be installed in a movie theater, a smart phone having a user U, the mobile phone may be an imaging device such as a tablet computer. Such an imaging apparatus, for example may be captured of the user U is performed by mounting the supporting devices provided in front of the seat. Image taken by the imaging device is supplied to the emotion estimation part 33 by communication. Incidentally, with the image pickup device is mounted in such supporting devices, the use of the reaction information may be determined to have been granted.
[0077]
　Emotion estimation unit 33, a specific emotion surprise and excitement such as a target, it may be determined whether the emotion is the target. To every movie scene may change the emotion that target, may change the emotion that targets in accordance with the contents of the presentation information.
[0078]
　Service information instead of the display, or may be presented by sound together with the display.
[0079]
　The present disclosure, in addition can also be realized as an apparatus that constitutes a system comprising a plurality of devices, the system can also be implemented method, a program or the like.
[0080]
　The present disclosure may also take the following configurations.
(1)
　The reaction information indicating the reaction of the user to the presentation information, with the reaction information utilizing unit for utilizing when use of the response information is licensed
　information processing apparatus.
(2)
　The information processing apparatus according to obtain the reaction information with the reaction information acquisition unit (1).
(3)
　the reaction information acquisition unit includes a sensor unit, based on the sensor information output from the sensor unit, the information processing apparatus according to (2) having a reaction information estimation unit for estimating the response information.
(4)
　The sensor unit monitors the user existing in a predetermined range, if the user is licensed for the reaction information, and outputs the sensor information about the user to the reaction information estimation unit ( the information processing apparatus according to 3).
(5)
　the reaction information estimation unit estimates the reaction information for each user using the sensor information of all users that exist in a predetermined range, and outputs only the reaction information of a user who the license (3) or the information processing apparatus according to (4).
(6)
　the reaction information utilizing unit, the information processing apparatus according to any one of generating the service information presented only to the user who licensed the reaction information from (1) to (5).
(7)
　The service information processing apparatus according to a characteristic based on the response information of the user (6).
(8)
　The service information processing apparatus according to at least one of the relative properties (6) to the reaction information of characteristics and other users based on the response information of the user.
(9)
　the license information indicating the presence or absence of the license is an information processing apparatus according to any of the user is associated with the seat information reserved from (1) to (8).
(10)
　the seat information corresponding to the given user there is a plurality, and, if the license information indicates there permission, the reaction information using unit, the reaction of a plurality of users located in the seat information by using the information processing apparatus according to generate information relating to between the user (9).
(11)
　the reaction information processing apparatus according to any one of the information corresponding to the emotion of the user (1) to (10).
(12)
　the presentation information, the content the user is viewing, the information processing apparatus according to any one of any of the art sports and the user by the user to watch the watch (1) to (11) .
(13)
　response information utilization unit, a response information indicating the reaction of the user to the presentation information, utilized when the use of the reaction information is licensed
　information processing method.
(14)
　A reaction information acquisition unit that acquires response information indicating the reaction of the user to the presentation information,
　with the reaction information utilization unit for use to utilize the reaction information if it is granted the reaction information
　processing system.
(15)
　the reaction information acquisition unit includes a sensor unit, based on the sensor information output from the sensor unit, the information processing system according to and a reaction information estimation unit (14) for estimating the response information.
(16)
　in a series of reservation processing for reserving a seat indicated by the seat information, use of reaction information indicating the seat information when viewing a predetermined presentation information, a reaction of the user when viewing the presentation information has a whether the input screen or permission, a display control unit that performs control to display on the display unit and the information for prompting the granting
　display device.
(17)
　and the display device according to (16), and a data processing device connected to said display device,
　said information processing apparatus, and license information indicating the presence or absence of the license output from the display device has a reservation information management unit for managing in association with the seat information reserved
　reservation system.
DESCRIPTION OF SYMBOLS
[0081]
1 ... reservation terminal, 2 ... server, 3 ... reaction information acquisition device, 11 ... reserved terminal control section, 12 ... communication unit, 22 ... scheduling-information managing unit, 23 ... service information generating unit, 32 ... sensor section, 33 ... emotion estimation unit

WE claims

[Requested item 1]
　The reaction information indicating the reaction of the user to the presentation information, with the reaction information utilization unit for use of the reaction information is available if it is granted
　the information processing apparatus.
[Requested item 2]
　The information processing apparatus according to claim 1 having a reaction information acquisition unit for acquiring the reaction information.
[Requested item 3]
　The reaction information acquisition unit includes a sensor unit, based on the sensor information output from the sensor unit, the information processing apparatus according to claim 2 and a reaction information estimation unit for estimating the response information.
[Requested item 4]
　The sensor unit monitors the user existing in a predetermined range, if the user is licensed for the reaction information, to claim 3 for outputting the sensor information about the user to the reaction information estimation unit the information processing apparatus according.
[Requested item 5]
　The reaction information estimating unit, information of claim 3, estimating the reaction information for each user using the sensor information of all users that exist in a predetermined range, and outputs only the reaction information of a user who has the permission processing apparatus.
[Requested item 6]
　The reaction information utilizing unit, the information processing apparatus according to claim 1 for generating service information presented only to the user who licensed of the reaction information.
[Requested item 7]
　The service information processing apparatus according to claim 6 which is a characteristic based on the response information of the user.
[Requested item 8]
　The service information processing apparatus according to claim 6 at least one of a relative characteristic for the reaction information of a user reaction information based on the characteristics and other said user.
[Requested item 9]
　Permission information indicating whether the license is an information processing apparatus according to claim 1, wherein the user is associated with the seat information reserved.
[Requested item 10]
　The seat information corresponding to the given user there is a plurality, and, if the license information indicates there permission, the reaction information using unit may utilize the reaction information of a plurality of users located in the seat information and, the information processing apparatus according to claim 9 for generating information relating to between the users.
[Requested item 11]
　The reaction information processing apparatus according to claim 1 which is information corresponding to the emotion of the user.
[Requested item 12]
　The presentation information, the content the user is viewing, the information processing apparatus according to claim 1 sports and the user in which the user is watching is one of art to appreciate.
[Requested item 13]
　Reaction information utilization unit utilizes the reaction information indicating the reaction of the user to the presentation information, if the use of the reaction information is licensed
　information processing method.
[Requested item 14]
　A reaction information acquisition unit that acquires response information indicating the reaction of the user to the presentation information,
　with the reaction information utilization unit for use to utilize the reaction information if it is granted the reaction information
　processing system.
[Requested item 15]
　The reaction information acquisition unit, the information processing system of claim 14, having a sensor unit, based on the sensor information output from the sensor portion, and the reaction information estimation unit for estimating the response information.
[Requested item 16]
　In a series of reservation processing for reserving a seat indicated by the seat information, granting and seating information when viewing a predetermined presentation information, the use of reaction information indicating the reaction of the user when viewing the presentation information whether the input screen and has a display control unit that performs control to display on the display unit and the information for prompting the permission
　display device.
[Requested item 17]
　A display device according to claim 16, comprising an information processing apparatus connected to said display device,
　said information processing apparatus, and license information indicating the presence or absence of the license output from the display device, is reserved with the reservation information management unit for managing in association with the seat information
　reservation system.