Description
Title of Invention: CONTROL DEVICE, CONTROL METHOD AND PROGRAM
Technical Field
[0001] The present technology relates to a control device, a control method and a program,
and particularly to a control device, a control method and a program that enable proper
control of the output of a sound from an output device for outputting at least a sound
such as a TV (television receiver) or the like.
Background Art
[0002] For example, Patent Document 1 describes a TV that detects surrounding noise in
formation, determines whether there is a viewer or not from an image of a camera for
taking a surrounding image, and suppresses noise based on the noise information when
there is a viewer.
[0003] Furthermore, for example, Patent Document 2 describes a recording and reproducing
device that generates content analysis information based on an audio signal of
contents, generates environmental noise analysis information based on surrounding
noise and adjusts volume of an audio signal of the contents based on the content
analysis information and the environmental noise analysis information.
Citation List
Patent Literature
[0004] PTL 1: Japanese Patent Application Laid-Open No. 2010-268284
PTL 2: Japanese Patent Application Laid- Open No. 2010-154388
Summary of Invention
Technical Problem
[0005] When a user who is watching a program on a TV hears talking a voice around the TV
but the talking voice is not produced from the user, that is, the voice is a voice from a
telephone conversation which a family member living together or the like has at a
place out of the room where the TV is located, the telephone conversation is noise for
the user watching the program, and makes it difficult to hear the sound (voices) of the
program. Accordingly, in this case, it is desired to adjust the output of the sound of the
TV, that is, turn up the volume of the TV so that the user watching the program can
easily hear the sound of the program on the TV.
[0006] On the other hand, when the talking voice heard around the TV is a voice produced
from the user watching the program on the TV, the user watching the program
produces a voice by himself/herself. Therefore, it is not necessary to adjust the output
of the sound of the TV so as to allow the user watching the program to easily hear the
sound of the program on the TV.
[0007] As described above, even when a voice is heard around the TV, there are a case
where it is proper to adjust the output of the sound from the TV and a case where it is
proper not to adjust the output of the sound from the TV.
[0008] The present technology has been completed in view of such circumstances, and
enables proper control of the output of the sound from an output device for outputting
at least a sound such as a TV or the like.
Solution to Problem
[0009] According to an aspect of the present technology, there is provided an information
processing apparatus comprising: a processor that: receives captured image data and
captured sound data corresponding to an environment in which content is reproduced;
detects a user based on the captured image data; analyzes a situation of the en
vironment based on a result of the detection and the captured sound data; and controls
an audio volume corresponding to reproduced content based on a result of the
analyzing.
[0010] A method according to another aspect of the present technology includes the steps of:
receiving captured image data and captured sound data corresponding to an en
vironment in which content is reproduced; detecting a user based on the captured
image data; analyzing a situation of the environment based on a result of the detection
and the captured sound data; and controlling an audio volume corresponding to r e
produced content based on a result of the analyzing.
[001 1] In the aspects described above, the processor receives the captured image data from a
camera positioned in the environment in which content is reproduced and detects a
face based on the captured image data.
[0012] The control device may be an independent device or an internal block constituting
one device.
[0013] The program may be provided by being transmitted via a transmission medium or
recorded in a recording medium.
Advantageous Effects of Invention
[0014] According to the present technology, the output of the sound from the output device
can be properly controlled.
Brief Description of Drawings
[0015] [fig.l]Fig. 1 is a block diagram illustrating a configuration example of an embodiment
of a TV to which the present technology is applied.
[fig.2]Fig. 2 is a block diagram illustrating a configuration example of a sound output
controller 14.
[fig.3]Fig. 3 is a flowchart explaining processing carried out by the sound output
controller 14.
[fig.4]Fig. 4 is a flowchart explaining sound output control processing carried out by a
volume/gain controller 35.
[fig.5]Fig. 5 is a diagram explaining the sound output control executed by the volume/
gain controller 35.
[fig.6]Fig. 6 is a flowchart explaining processing of reflecting/registering a sound ad
justment key operation.
[fig.7]Fig. 7 is a flowchart explaining restoration processing.
[fig.8]Fig. 8 is a flowchart explaining the restoration processing.
[fig.9]Fig. 9 is a diagram explaining another example of the sound output control
executed by the volume/gain controller 35.
[fig. 10] Fig. 10 is a block diagram illustrating a configuration example of an em
bodiment of a computer to which the present technology is applied.
[00 16]

[0017] Fig. 1 is a block diagram illustrating a configuration example of an embodiment of a
TV to which the present technology is applied.
[0018] In Fig. 1, a TV is one of output devices for outputting at least a sound, and has a
tuner 11, a signal processor 12, a display unit 13, a sound output controller 14, a
speaker unit 15, a controller 16 and an operating unit 17.
[0019] The tuner 11 is supplied with an RF (Radio Frequency) signal of a television
broadcast signal received by an antenna (not shown).
[0020] The tuner 11 detects and demodulates the RF signal supplied thereto, achieves data
of a program of a channel that is selected by, for example, user's operation of the
operating unit 17, and supplies the data to the signal processor 12.
[0021] The signal processor 12 performs decoding and other necessary signal processing on
the data of the program from the tuner 11, supplies resultant video (image) data of the
program to the display unit 13 and also supplies audio (sound (voice)) data of the
program to the speaker unit 15.
[0022] The display unit 13 includes, for example, an LCD (Liquid Crystal Display), an
organic EL (Electro Luminescence) or the like, and displays images of the program
corresponding to the video data from the signal processor 12.
[0023] The sound output controller 14 takes an image around the TV and collects an envi
ronmental sound around the TV, and controls the output of the sound from the TV, that
is, for example, the volume or the like of (speakers 15L and 15R of) the speaker unit
15 based on the resultant taken image around the TV and the resultant environmental
sound around the TV.
[0024] The speaker unit 15 has the speaker 15L for outputting the sound of an L (Left)
channel and the speaker 15R for outputting the sound of an R (Right) channel, and
outputs the sound of the program corresponding to the audio data from the signal
processor 12 according to the control of the sound output controller 14.
[0025] The controller 16 controls each of blocks constituting the TV according to an
operation of the operating unit 17 or the like, for example.
[0026] The operating unit 17 is a button provided on a housing (main body) of the TV, a
remote commander, or the like, and is operated by the user to supply the controller 16
with the operation signal corresponding to the operation concerned.
[0027] Configuration Example of Sound Output Controller 14>
[0028] Fig. 2 is a block diagram illustrating a configuration example of the sound output
controller 14 of Fig. 1.
[0029] In Fig. 2, the sound output controller 14 has a camera 21, a microphone
(microphone) group 22, a controller 23 and a storage unit 24.
[0030] The camera 2 1 takes a surrounding image around the TV (particularly, at a side
opposite to the display screen of the display unit 13, for example) at a predetermined
rate, and supplies a resultant taken image to (a face detector 3 1 of) the controller 23.
[0031] The microphone group 22 has plural microphones, collects an environmental sound
around the TV and supplies the environmental sound to (a voice determination unit 33
of) the controller 23.
[0032] The controller 23 controls the output of the sound from the TV such as the volumes
of the speakers 15L and 15R of the speaker unit 15 based on the taken surrounding
image around the TV from the camera 2 1 and the environmental sound around the TV
from the microphone group 22.
[0033] That is, the controller 23 has the face detector 31, a face information acquisition unit
32, the voice determination unit 33, a situation analyzer 34, a volume/gain controller
35 and a speaker controller 36.
[0034] The face detector 31 detects a face (of a person) on a taken image supplied from the
camera 21, and supplies the face information acquisition unit 42 with face detection in
formation representing the detection result.
[0035] The face information acquisition unit 32 obtains face information concerning the face
on the taken image based on the face detection information or the like from the face
detector 31, and supplies the face information to the situation analyzer 34.
[0036] That is, the face information acquisition unit 32 recognizes the positions (for
example, the positions on the taken image) and the number of faces displayed on the
taken image as face information, for example, from the face detection information.
[0037] In addition, the face information acquisition unit 32 carries out individual identi
fication on a face displayed on the taken image to obtain information for identifying
who has the identified face (whose face is the identified face), and the age, gender and
the like of the person identified by the individual identification as face information. For
example, information necessary for individual identification of family members can be
input at the initialization stage of the TV or the like.
[0038] The voice determination unit 33 executes frequency analysis or the like on the envi
ronmental sound supplied from the microphone group 22 to detect the level (gain) of
the environmental sound and the direction to a sound source producing the envi
ronmental sound, and also determines whether the environmental sound is a human's
voice or a sound other than a human's voice (hereinafter referred to as "non-voice").
[0039] The voice determination unit 33 supplies the level of the environmental sound, the
direction to the sound source producing the environmental sound and the determination
result indicating whether the environmental sound is a voice or a non-voice to the
situation analyzer 34 as environmental sound information concerning the envi
ronmental sound.
[0040] The situation analyzer 34 analyzes the situation around the TV based on the face in
formation supplied from the face information acquisition unit 32 obtained from the
taken image, and the environmental sound information supplied from the voice deter
mination unit 33 obtained from the environmental sound, and supplies the situation
around the TV obtained as an analysis result (hereinafter referred to as "surrounding
situation") to the volume/gain controller 35.
[0041] The volume/gain controller 35 controls the speaker controller 36 according to the
surrounding situation from the situation analyzer 34, thereby controlling the output of
the sound of the TV, that is, for example, the volumes of the speakers 15L and 15R,
the gain of the sound output from the speakers 15L and 15R and the like.
[0042] The volume/gain controller 35 further controls the output of the sound of the TV
according to the user's operation of the operating unit 17 (Fig. 1).
[0043] That is, buttons (keys) of the main body as the operating unit 17 and buttons of the
remote commander are provided with sound adjustment keys operated to adjust the
output of the sound from the TV such as a volume key operated to turn up or down the
volume, a mute key operated to mute the sound and the like.
[0044] When the user operates the volume adjustment key of the operating unit 17, the
controller 16 supplies the volume/gain controller 35 with a volume operation signal
that is an operation signal corresponding to the operation of the volume adjustment
key.
[0045] The volume/gain controller 35 controls the output of the sound of the TV according
to the volume operation signal supplied from the controller 16 (from Fig. 1) as well as
the surrounding situation from the situation analyzer 34.
[0046] The volume/gain controller 35 can further control the output of the sound of the TV
according to information stored in the storage unit 24 as described later.
[0047] The speaker controller 36 controls the volumes of the speakers 15L and 15R
according to the control from the volume/gain controller 35.
[0048] External audio equipment is connected to the TV, and when the sound of a program
is output from a speaker of the external audio equipment, the speaker controller 36 can
control the volume of the speaker of the external audio equipment connected to the TV
according to the control from the volume/gain controller 35.
[0049] The storage unit 24 stores information necessary for the operation of the controller
23.
[0050] A partial storage area of the storage unit 24 serves as an environmental sound storage
unit 24A for temporarily storing the environmental sound collected by the microphone
group 22.
[0051] The environmental sounds collected by the microphone group 22 are supplied to the
environmental sound storage unit 24A via the controller 23. The environmental sound
storage unit 24A stores the environmental sound from the microphone group 22, and
when the environmental sound storage unit 24A stores the environmental sounds
amount of which corresponds to the storage capacity thereof, an environmental sound
which is newly supplied from the microphone group 22 is stored so as to be
overwritten on the oldest environmental sound.
[0052] Accordingly, the latest environmental sound for a predetermined time period (for
example, several tens seconds or the like) is stored in the environmental sound storage
unit 24A at all times.
[0053]
[0054] Fig. 3 is a flowchart explaining processing carried out by the sound output controller
14 of Fig. 2.
[0055] In step SI 1, the sound output controller 14 collects environmental sound information
and face information, and the processing goes to step S12.
[0056] That is, by the camera 21, a surrounding image around the TV is taken, and the
resultant taken image is supplied to the face detector 31.
[0057] The face detector 3 1 detects a face displayed on the taken image supplied from the
camera 21, and supplies face detection information representing the detection result to
the face information acquisition unit 32.
[0058] Based on the face detection information from the face detector 3 1 or the like, the face
information acquisition unit 32 obtains, for example, the positions, the number and the
like of faces displayed on a taken image as face information concerning the faces
displayed on the taken image, and supplies the obtained face information to the
situation analyzer 34.
[0059] On the other hand, an environmental sound around the TV are collected by the microphone
group 22 and supplied to the voice determination unit 33.
[0060] The voice determination unit 33 detects the level of the environmental sound
supplied from the microphone group 22 and the direction to a sound source producing
the environmental sound, and also determines whether the environmental sound is a
voice or a non-voice.
[0061] The voice determination unit 33 supplies the situation analyzer 34 with the level of
the environmental sound, the direction to the sound source producing the envi
ronmental sound and the determination result indicating whether the environmental
sound is a voice or a non-voice, as the environmental sound information.
[0062] The environmental sound collected by the microphone group 22 is supplied to the
voice determination unit 33, and also supplied to and temporarily stored in the envi
ronmental sound storage unit 24A as described above.
[0063] In step S12, the situation analyzer 34 analyzes the surrounding situation around the
TV based on the face information from the face information acquisition unit 32 and the
environmental sound information from the voice determination unit 33 and supplies
the analyzed surrounding situation to the volume/gain controller 35, and the processing
goes to step SI3.
[0064] In step S13, the volume/gain controller 35 controls the speaker controller 36
according to the surrounding situation from the situation analyzer 34 to perform the
sound output control for controlling the output of the sound from the TV, that is, for
example, the volumes of the speakers 15L and 15R, the gain of the sounds output from
the speakers 15L and 15R and the like, and the processing goes to step S14.
[0065] In step S14, the volume/gain controller 35 determines whether the user has operated
the sound adjustment key which is provided to the operating unit 17 and operated when
the output of the sound from the TV is adjusted.
[0066] When it is determined in step S14 that the sound adjustment key is not operated, the
processing skips step S15 and returns to step SI 1 to repeat the same processing.
[0067] When it is determined in step S14 that the sound adjustment key is operated, that is,
when the user has operated the sound adjustment key and thus a volume operation
signal corresponding to the operation of the sound adjustment key is supplied from the
controller 16 (Fig. 1) to the volume/gain controller 35, the processing goes to step S15,
the volume/gain controller 35 performs sound adjustment key operation reflecting/
registering processing by which the operation of the sound adjustment key is reflected
to the output of the sound of the TV and operation information on the operation
concerned is registered (stored) into the storage unit 24, and the processing returns to
step Sll.
[0068] Here, "the volume/gain controller 35 reflects the operation of the sound adjustment
key to the output of the sound of the TV in step S15" means that the volume/gain
controller 35 controls the output of the sound from the TV by controlling the speaker
controller 36 according to the operation of the sound adjustment key (the volume
operation signal).
[0069] Fig. 4 is a flowchart explaining the sound output control processing in accordance
with the surrounding situation carried out by the volume/gain controller 35.
[0070] In step S21, the volume/gain controller 35 determines based on the surrounding
situation whether the level of the environmental sound around the TV is not less than a
predetermined threshold value.
[0071] When it is determined in step S21 that the level of the environmental sound is less
than the threshold value, that is, when no environmental sound which disturbs
watching of a program occurs around the TV, the processing is returned.
[0072] When it is determined in step S21 that the level of the environmental sound is not
less than the threshold value, that is, when there is a probability that an environmental
sound disturbing watching of a program occurs around the TV, the processing goes to
step S22, and the volume/gain controller 35 determines based on the surrounding
situation whether the environmental sound level of which is not less than the threshold
value is a human's voice or not.
[0073] When it is determined in step S22 that the environmental sound level of which is not
less than the threshold value (hereinafter referred to as "large level environmental
sound") is a human's voice, the processing goes to S23, and the volume/gain controller
35 determines based on the surrounding situation whether a face is detected from a
taken image.
[0074] When it is determined in step S23 that no face is detected from the taken image, that
is, in case of a situation where a human's talking voice is heard around the TV but there
is no person watching a program around the TV, the processing goes to step S24, the
volume/gain controller 35 performs the sound output control for turning down the
volumes of the speakers 15L and 15R, and the processing returns.
[0075] For example, a situation where a user watching a program receives a phone call and
has a telephone conversation at a place which is slightly distant from the TV or a
situation where the user turns his/her face to a room located in the direction which is
entirely different from the direction to the TV (camera 21) and talks with his/her
family member in the room concerned is considered as the situation where a human's
voice is heard around the TV but no person exists around the TV.
[0076] In this case, the user is not watching the program, and the voice (sound) of the
program output from the TV may disturb a telephone call or a conversation which the
user has. Therefore, the volume/gain controller 35 performs the sound output control
for turning down the volumes of the speakers 15L and 15R. Accordingly, the sound
output from the TV can be prevented from disturbing the telephone call or the conversation
which the user has.
[0077] When the volumes of the speakers 15L and 15R during execution of the processing
of step S24 are originally small, the sound output control for adjusting the volumes
may not be executed in step S24.
[0078] On the other hand, when it is determined in step S23 that a face is detected from the
taken image, that is, when the face or faces of one or more persons are displayed on the
taken image taken by the camera 21, the processing goes to step S25, and the volume/
gain controller 35 determines based on the surrounding situation whether the direction
to a sound source producing an environmental sound that is a voice and the direction to
the position where the face is detected with respect to the TV are (regarded as being)
coincident with each other.
[0079] When it is determined in step S25 that the direction to the sound source producing
the environmental sound that is the voice is coincident with the direction to the
position where any face (of one or more faces) is detected, that is, in case of a situation
where the user is watching a program and the user watching the program himself/
herself has a conversation, the processing is returned.
[0080] Here, a situation where the user watching the program has a conversation about the
program (with another user) is considered as the situation where the user is watching
the program and the user has a conversation.
[0081] In this case, the volumes of the speakers 15L and 15R are not adjusted (kept
unchanged).
[0082] Furthermore, when it is determined in step S25 that the direction to the sound source
producing the environmental sound that is the voice is not coincident with the direction
to the position where the face is detected, that is, in case of a situation where the user is
watching a program, but a talking voice of a person other than the user is heard around
the TV, the processing goes to step S26, the volume/gain controller 35 performs the
sound output control for turning up the volumes of the speakers 15L and 15R, and the
processing returns.
[0083] That is, a situation where a user who is not watching a program is making some noise
at a place slightly distant from the TV (a place at which the user concerned is not
displayed on the taken image) though there is a user watching the program is
considered as the situation where the user is watching the program but a talking voice
of a person other than the user is heard around the TV.
[0084] In this case, the voice of the user who is making some noise without watching the
program may disturb the program watching by the user who is watching the program,
and thus the volume/gain controller 35 performs the sound output control for turning
up the volumes of the speakers 15L and 15R. Accordingly, the user who is watching
the program can watch the program without being disturbed by the user who is making
some noise without watching the program.
[0085] That is, when the user who is not watching the program produces a voice which
disturbs the watching of the program, the volumes of the speakers 15L and 15R are
adjusted to be turned up. Therefore, the user who is watching the program can watch
the program while hearing the sound with the same degree of easiness before and after
the user who is not watching the program produces a voice.
[0086] On the other hand, when it is determined in step S22 that the environmental sound
level of which is not less than the threshold value (large level environmental sound) is
not a human's voice, that is, when the large level environmental sound is a non- voice
such as a call sound of a telephone or an interphone or the like, the processing goes to
step S27, and the volume/gain controller 35 determines whether a registered sound
which is (regarded as being) coincident with the environmental sound that is the nonvoice
(large level environmental sound) exists in registered sounds described later
which are registered (stored) in the storage unit 24.
[0087] Here, when the user operates the sound adjustment key of the operating unit 17 (Fig.
1) and the volume/gain controller 35 performs the sound output control according to
the operation of the sound adjustment key, the operation information concerning the
operation of the sound adjustment key and the environmental sound just before the
sound adjustment key is operated are registered in the storage unit 24 in association
with each other.
[0088] The registered sound means an environmental sound which is registered in the
storage unit 24 in association with the operation information as described above.
[0089] When it is determined in step S27 that there is no registered sound which is co
incident with the environmental sound that is the non- voice, the processing goes to step
S26, and the volume/gain controller 35 performs the sound output control for turning
up the volumes of the speakers 15L and 15R so that the watching of the program by
the user is not disturbed by the environmental sound that is the non- voice, and the
processing returns.
[0090] When it is determined in step S27 that there is a registered sound coincident with the
environmental sound that is the non-voice, the processing goes to step S28, the
volume/gain controller 35 reads out the operation information associated with the
registered sound coincident with the environmental sound that is the non- voice from
the storage unit 24, and performs the sound output control for adjusting the volumes of
the speakers 15L and 15R according to (the operation of the sound adjustment key rep
resented by) the operation information, and the processing returns.
[0091] As described above, in the sound output controller 14, the surrounding situation
around the TV is analyzed based on the taken image obtained by taking the sur
roundings of the TV and the environmental sound around the TV, and the sound output
from the TV is controlled according to the surrounding situation. Therefore, the output
of the sound can be properly controlled.
[0092] That is, for example, when a talking voice is heard around the TV on which the user
is watching a program and the talking voice is from a telephone conversation which a
family member living together or the like has at a place out of a room where the TV is
placed, it is determined in step S25 of Fig. 4 that the direction to the sound source and
the direction to the position where the face is detected are not coincident with each
other, and the volume from the TV is adjusted to be larger. Therefore, the user who is
watching a program can be prevented from being disturbed in program watching by the
telephone conversation which the family member living together or the like has.
[0093] Furthermore, when the talking voice heard around the TV is a voice produced by the
user who is watching a program on the TV, it is determined in step S25 of Fig. 4 that
the direction to the sound source and the direction to the position where the face is
detected are coincident with each other, and the volume from the TV is not adjusted
(kept unchanged). Therefore, the adjustment of the volume from the TV in a case
where the user does not feel any need to adjust the volume can be prevented.
[0094] Fig. 5 is a diagram explaining the sound output control executed by the volume/gain
controller 35 of Fig. 2.
[0095] As described with reference to Figs. 3 and 4, the volume/gain controller 35 performs
the sound output control for adjusting the sound from the TV according to the sur
rounding situation around the TV. However, the sound output control can be
performed according to the distance (from the TV) to a user who is watching a
program (watching distance), a result of individual identification of a user who is
watching a program or the like in addition to the surrounding situation around the TV.
[0096] That is, in a case where the watching distance is long (not less than a predetermined
threshold value), an adjustment amount by which the volume is turned up can be set to
be larger than a preset default adjustment amount (turn up the volume more greatly),
for example, when the volume is turned up.
[0097] Furthermore, in a case where the watching distance is short (less than the prede
termined threshold value), the adjustment amount by which the volume is turned up
can be set to be smaller than the default adjustment amount, for example, when the
volume is turned up.
[0098] Still furthermore, for example when the age of a program- watching user which is
obtained as a result of the individual identification of the user is an advanced age of not
less than 80, the adjustment amount by which the volume is turned up can be set to be
larger than the default adjustment amount, for example, when the volume is turned up.
In this case, in place of the turn-up of the volume (or in addition to the turn-up of the
volume), the gain of frequency components which are difficult for advanced age
people to hear can be increased.
[0099] Fig. 6 is a flowchart explaining the sound adjustment key operation reflecting/reg
istering processing that is carried out by the volume/gain controller 35, by which the
operation of the sound adjustment key is reflected to the sound output from the TV and
the operation information concerning the operation concerned is registered (stored)
into the storage unit 24.
[0100] In step S41, the volume/gain controller 35 performs the sound output control for
adjusting the volume according to the operation of the sound adjustment key that is de
termined to be operated by the user in step S14 of Fig. 3, and the processing goes to
step S42.
[0101] That is, for example when the user operates the sound adjustment key so as to turn up
the volume by only two steps or the like, the volume/gain controller 35 performs the
sound output control for turning up the volume by only the two steps according to the
operation concerned.
[0102] In step S42, the volume/gain controller 35 reads out an environmental sound for a
fixed time period just before the sound adjustment key is operated (hereinafter referred
to as "environmental sound just before operation") from the environmental sound
storage unit 24A in which the latest environmental sound for a predetermined time
period is stored, and analyzes the environmental sound just before operation, and the
processing goes to step S43.
[0103] In step S43, the volume/gain controller 35 registers (an analysis result of) the envi
ronmental sound just before operation into the storage unit 24 as a registered sound in
association with the operation information concerning the operation of the sound ad
justment key, and the processing returns.
[0104] Here, as described with reference to steps S27 and S28 of Fig. 4, when a registered
sound which is (regarded as being) coincident with an environmental sound that is a
non-voice (large level environmental sound) exists in registered sounds registered in
the storage unit 24, the volume/gain controller 35 adjusts the volume according to the
operation information associated with the registered sound.
[0105] Accordingly, in a case where the user has ever adjusted the volume by only a prede
termined adjustment amount (including muting) through user's operation of the sound
adjustment key just after an environmental sound occurred, the same adjustment of the
volume which has ever been carried out is carried out without any user's operation of
the sound adjustment key when a similar environmental sound afterwards occurs.
[0106] That is, the TV can exhibits a so-called learning function as if the TV remembers the
operation of the sound adjustment key carried out by the user.
[0107] As described with reference to steps S27 and S28 of Fig. 4, in a case where the ad
justment of the volume is carried out in the volume/gain controller 35 according to the
operation information associated with the registered sound which is registered in the
storage unit 24 and coincident with the environmental sound, the volume can be
adjusted to be returned to the original one when the environmental sound coincident
with the registered sound vanishes afterwards.
[0108] Likewise, even in a case where the volume is adjusted in steps S24, S26 and S28 of
Fig. 4 or the volume is adjusted according to the user's operation of the sound ad
justment key in step S41 of Fig. 6, the volume may be subjected to restoration
processing as adjustment processing for returning the volume to an original one when
the surrounding situation returns to an original one before the volume is adjusted.
[0109] Fig. 7 is a flowchart explaining the restoration processing for returning the volume
adjusted in steps S24, S26 and S28 of Fig. 4 to the original one.
[0110] The restoration processing of Fig. 7 is started when the volume is adjusted in steps
S24, S26 and S28 of Fig. 4.
[0111] In the storage unit 24 (Fig. 2), the latest surrounding situation for a fixed time period
obtained by the situation analyzer 34 is stored, and a surrounding situation just before
an occurrence of a large level environmental sound which causes the adjustment of the
volume in steps S24, S26 and S28 of Fig. 4 (an original surrounding situation before
the large level environmental sound occurs) out of the latest surrounding situation is
stored until the restoration processing is finished.
[01 12] In the sound output controller 14 (Fig. 2), the face information acquisition unit 32
collects face information and the voice determination unit 33 collects environmental
sound information in step S51 as in step SI 1 of Fig. 3, and the processing goes to step
S52.
[0113] In step S52, the situation analyzer 34 analyzes the surrounding situation around the
TV on the basis of the face information obtained by the face information acquisition
unit 32 and the environmental sound information obtained by the voice determination
unit 33 as in step S12 of Fig. 3 and supplies the analyzed surrounding situation to the
volume/gain controller 35, and the processing goes to step S53.
[01 14] Here, the processing of steps S51 and 52 of Fig. 7 can be substituted by the
processing of steps Sll and S12 of Fig. 3.
[01 15] In step S53, the volume/gain controller 35 determines on the basis of the surrounding
situation from the situation determination unit 34 whether the environmental sound
level of which is determined to be not less than the threshold value in step S21 of Fig.
4 (the large level environmental sound) vanishes or not.
[0116] When it is determined in step S53 that the large level environmental sound does not
vanish, the processing returns to step S51.
[0117] When it is determined in step S53 that the large level environmental sound vanishes,
the processing goes to step S54, and the volume/gain controller 35 determines whether
the surrounding situation from the situation determination unit 34 has returned to the
situation stored in the storage unit 24 (original situation) before the environmental
sound level of which is not less than the threshold value (large level environmental
sound) occurs.
[0118] When it is determined in step S54 that the surrounding situation from the situation
determination unit 34 has not returned to the original situation before the large level
environmental sound occurs, the processing returns to step S51.
[01 19] Furthermore, when it is determined in step S54 that the surrounding situation from
the situation determination unit 34 has returned to the original situation before the
large level environmental sound occurs, the processing goes to step S55, the volume/
gain controller 35 performs the sound output control for returning the volume adjusted
in steps S24, S26 or S28 of Fig. 4 to the original volume, and the restoration
processing is finished.
[0120] There is a case where a user operates the sound adjustment key because of a change
in the surrounding situation. Even in a case where the user operates the sound ad
justment key because of a change in the surrounding situation and thus the volume is
adjusted according to the operation of the sound adjustment key in step S41 of Fig. 6,
the volume/gain controller 35 can return the volume to the original volume as in the
case of the restoration processing of Fig. 7 when the surrounding situation afterwards
returns to the original situation before the volume is adjusted (the situation before the
change).
[0121] Fig. 8 is a flowchart explaining the restoration processing for returning the volume
adjusted in step S41 of Fig. 6 to the original volume.
[0122] The restoration processing of Fig. 8 is started when the volume is adjusted in step
S41 of Fig. 6.
[0123] As in the case of Fig. 7, in the storage unit 24 (Fig. 2), the latest surrounding
situation for a fixed time period is stored, and the surrounding situation before the
change when the user operates the sound adjustment key because of the change in the
surrounding situation (when the sound adjustment key is operated just after the sur
rounding situation is changed) is selected from the latest surrounding situation and
stored until the restoration processing of Fig. 8 is finished.
[0124] As in the case of step SI 1 of Fig. 3, in the sound output controller 14 (Fig. 2), the
face information acquisition unit 32 collects the face information and the voice deter
mination unit 33 collects the environmental sound information in step S61, and the
processing goes to step S62.
[0125] In step S62, the situation analyzer 34 analyzes the surrounding situation around the
TV on the basis of the face information obtained by the face information acquisition
unit 32 and the environmental sound information obtained by the voice determination
unit 33 as in the case of step S12 of Fig. 3 and supplies the analyzed surrounding
situation to the volume/gain controller 35, and the processing goes to step S63.
[0126] Here, the processing of steps S61 and 62 of Fig. 8 may be substituted by the
processing of steps Sll and S12 of Fig. 3.
[0127] In step S63, the volume/gain controller 35 determines whether the surrounding
situation from the situation determination unit 34 has returned to the situation stored in
the storage unit 24 before the change (original situation).
[0128] When it is determined in step S63 that the surrounding situation from the situation
determination unit 34 has not returned to the original situation, the processing returns
to step S61.
[0129] When it is determined in step S63 that the surrounding situation from the situation
determination unit 34 has returned to the original situation, the processing goes to step
S64, the volume/gain controller 35 performs the sound output control for returning the
volume adjusted in step S41 of Fig. 6 to the original volume, and the restoration
processing is finished.
[0130] Fig. 9 is a diagram explaining another example of the sound output control executed
by the volume/gain controller 35 of Fig. 2.
[0131] In a case where the environmental sound is, for example, a calling sound of an in
terphone or a telephone and the environmental sound is determined to be a non-voice
in the voice determination unit 33 (the result of the determination of the environmental
sound indicates a non-voice), when no face is detected from a taken image (the number
of faces detected from the taken image is zero), a situation where a user only listens to
the sound of a program without watching the image and wants to stably listen to the
sound of the program is considered as a situation around the TV (surrounding
situation).
[0132] In this case, the volume/gain controller 35 can perform the sound output control for
turning up the volume.
[0133] When the environmental sound is a non-voice as described above and a face is
detected from the taken image (the number of faces detected from the taken image is
one or more), a situation where the user is watching a program and wants to stably
listen to the sound of the program is considered as the surrounding situation around the
TV.
[0134] In this case, the volume/gain controller 35 can also perform the sound output control
for turning up the volume.
[0135] In a case where the environmental sound is, for example, a human conversation or
the like and the voice determination unit 33 determines that the environmental sound is
a voice (the result of the determination of the environmental sound indicates a voice),
when no face is detected from a taken image (the number of faces detected from the
taken image is zero), a situation where a user is not watching any program and has a
conversation over the telephone or the like is considered as the surrounding situation
around the TV.
[0136] In this case, the volume/gain controller 35 can perform the sound output control for
turning down the volume so as to prevent the sound of a program output from the TV
from disturbing the user's conversation.
[0137] In a case where the environmental sound is a voice as described above, when a face
is detected from a taken image (the number of faces detected from the taken image is
one or more) and the direction to a sound source producing the voice that is an envi
ronmental sound is coincident with the direction to the position of the face (any face)
detected from the taken image, a situation where a user has a conversation over the
telephone or the like while watching a program is considered as the surrounding
situation around the TV.
[0138] In this case, the volume/gain controller 35 may keep the volume unchanged without
adjusting the volume, that is, may not perform the sound output control.
[0139] In a case where the environmental sound is a voice as described above, when a face
is detected from a taken image and the direction to a sound source producing the voice
that is an environmental sound is not coincident with the direction to the position of the
face (any face) detected from the taken image, a situation where there exist a user
watching a program and a user having a conversation without watching any program
(without watching the TV side) is considered as the surrounding situation around the
TV.
[0140] In this case, the volume/gain controller 35 can perform the sound output control for
turning up the volume so that the program watching by the user who is watching the
program is prevented from being disturbed by the conversation of the user who is not
watching the program.
[0141] Furthermore, irrespective of the environmental sound or the detection of the face
from the taken image, the sound output control for turning down the volume can be
performed during the nighttime (a time period set as nighttime) so as to prevent the
sound from traveling beyond the wall of the room where the TV is placed and reaching
another room, for example.
[0142] The sound output control for reducing or increasing the gain of a predetermined
frequency band of the sound of a program can be performed by the volume/gain
controller 35 in place of the sound output control for turning down or up the volume
(or together with such sound output control).
[0143] For example, when the environmental sound is a non-voice, a frequency band having
a high gain (hereinafter referred to as "high gain band") in the non-voice is detected by
analyzing the frequency characteristic of the non-voice, and the gain of the high gain
band of the sound of the program is increased, so that the non-voice as noise can be
equivalently subjected to noise-canceling.
[0144] Furthermore, as described above, when the sound is prevented from traveling beyond
the wall of the room where the TV is placed and reaching another room in the
nighttime, the sound output control for reducing (suppressing) the gain in a bass tone
of the sound of the program can be performed.
[0145] A manner for adjusting the gain of each frequency component of the sound of the
program can be changed in accordance with, for example, modes of the TV such as a
game mode and a theater (cinema) mode or a category of a content as a program (for
example, a sport program or the like) displayed (output) on the TV.
[0146]
[0147] Next, a series of processing described above may be carried out by hardware or
software. When the series of processing is carried out by software, a program con
stituting the software is installed in a general-purpose computer or the like.
[0148] Fig. 10 illustrates a configuration example of an embodiment of a computer in which
the program for executing the series of processing described above is installed.
[0149] The program may be recorded in a hard disk 105 or a ROM 103 as a recording
medium contained in the computer in advance.
[0150] Alternatively, the program may be stored (recorded) in a removable recording
medium 111. Such a removable recording medium 111 may be provided as so-called
package software. Here, a flexible disk, a CD-ROM (Compact Disc Read Only
Memory), a MO (Magneto Optical) disk, a DVD (Digital Versatile Disc), a magnetic
disk, a semiconductor memory and the like are known as the removable recording
medium 111.
[0151] In addition to the install of the program from the removable recording medium 111
described above into the computer, the program may be downloaded into the computer
via a communication network or a broadcasting network and installed into the hard
disk 105 contained in the computer. That is, for example, the program may be
wirelessly transmitted from a download site to the computer via an artificial satellite
for digital satellite broadcasting or transmitted to the computer via a network such as
LAN (Local Area network) or the Internet through a cable.
[0152] The computer contains a CPU (Central Processing Unit) 102, and an input/output
interface 110 is connected to the CPU 102 via a bus 101.
[0153] When the user operates an input unit 107 or the like to input an instruction through
the input/output interface 110, the CPU 102 executes a program stored in the ROM
(Read Only Memory) 103 according to the instruction. Alternatively, the CPU 102
loads a program stored in the hard disk 105 into a RAM (Random Access Memory)
104 and executes the program.
[0154] Accordingly, the CPU 102 carries out the processing in accordance with the
flowchart described above or the processing to be executed by the configuration of the
block diagram described above. Via the input/output interface 110, the CPU 102
outputs the processing result from the output unit 106, transmits the processing result
from a communication unit 108, records the processing result into the hard disk 105 or
the like as necessary for example.
[0155] The input unit 107 is constituted by a keyboard, a mouse, a microphone and the like.
The output unit 106 is constituted by an LCD (Liquid Crystal Display), a speaker and
the like.
[0156] Here, in this specification, the processing carried out according to the program by the
computer is not necessarily required to be carried out in time-series in accordance with
an order described as a flowchart. That is, the processing carried out according to the
program by the computer includes processing to be carried out in parallel or indi
vidually (for example, parallel processing or object-based processing).
[0157] The program may be processed by one computer (processor) or dispersively
processed by plural computers. Furthermore, the program may be transmitted to and
executed by a remote computer.
[0158] The embodiment of the present technology is not limited to the above-described em
bodiment, and various modifications may be made without departing from the subject
matter of the present technology.
[0159] That is, the present technology may be applied to an output device for outputting at
least sounds other than a TV.
[0160] In this embodiment, the program received by the tuner 11 is set as a processing
target. However, according to the present technology, in addition to programs received
by the tuner 11, contents recorded in a recorder, contents supplied from servers on a
network such as the Internet and the like may be set as processing targets.
[0161] When the user operates the sound adjustment key in step S24 or S26 of Fig. 4 to
return the adjustment of the volume to original one after adjustment of the volume, the
surrounding situation at that time can be registered in the storage unit 24 (learned). In
subsequent step S24 or S26, the adjustment of the volume may not be performed (can
be limited) when the surrounding situation is coincident with the situation stored in the
storage unit 24.
[0162] In a case where the volume is adjusted without user's operation of the sound ad
justment key in step S24, S26 or S28 of Fig. 4, it may make the user feel uncom
fortable if the volume is adjusted with no announcement. Therefore, a message in
dicating execution of the volume adjustment may be displayed on the display unit 13
(or output from the speaker unit 15 with voices).
[0163] Here, ON/OFF of the display (output) of the message indicating the execution of the
volume adjustment may be switched by user's operation of the operating unit 17.
[0164] Furthermore, in the processing of reflecting/registering the sound adjustment key
operation of Fig. 6, the operation information on the operation of the sound adjustment
key is registered in the storage unit 24 in association with the environmental sound for
a fixed time period just before the sound adjustment key is operated (the environmental
sound just before operation) in step S43. However, the registration of the operation in
formation into the storage unit 24 may be performed in association with not only the
environmental sound just before operation, but also a result of individual identification,
a time period, a source of a program displayed on the TV (input source) and the like.
[0165] In a case where the operation information on the operation of the sound adjustment
key is registered in the storage unit 24 in association with the environmental sound just
before operation, the result of individual identification and the like, a user watching a
program is recognized based on the result of individual identification or the like, and
the volume adjustment can be performed according to the operation of the sound ad
justment key which is different for each user even when the same environmental sound
just before operation occurs.
[0166] Here, in a case where the operation information on the operation of the sound ad
justment key is registered in the storage unit 24 in association with the result of in
dividual identification, the time period and the input source as well as the envi
ronmental sound just before operation, when an environmental sound just before
operation and the like which are wholly identical to a new environmental sound just
before operation and the like associated with the operation information have been
already registered in the storage unit 24, a set of the operation information and the new
environmental sound just before operation and the like is registered in the storage unit
24 so as to be overwritten on the identical environmental sound just before operation
and the like.
[0167] The present technology can be configured as follows.
[0168] (1) An information processing apparatus comprising:
a processor that:
receives captured image data and captured sound data corresponding to an en
vironment in which content is reproduced;
detects a user based on the captured image data;
analyzes a situation of the environment based on a result of the detection and the
captured sound data; and
controls an audio volume corresponding to reproduced content based on a result of
the analyzing.
(2) The information processing apparatus of (1), wherein
the processor receives the captured image data from a camera positioned in the environment
in which content is reproduced and detects a face based on the captured
image data.
(3) The information processing apparatus of (1), wherein
the processor receives the sound data from a microphone positioned in the environment
in which content is reproduced
(4) The information processing apparatus of (2), wherein
the processor detects a position corresponding to the detected face based on the
captured image data.
(5) The information processing apparatus of (2), wherein
the processor detects a plurality of faces based on the captured image data.
(6) The information processing apparatus of (2) wherein
the processor determines face information corresponding to the detected face, the face
information including at least one of an individual, age and gender.
(7) The information processing apparatus of (1), wherein
the processor determines a sound level corresponding to the captured sound data.
(8) The information processing apparatus of (1), wherein
the processor determines a direction corresponding to a source of the captured sound
data.
(9) The information processing apparatus of (1), wherein
the processor determines whether the captured sound data is a human's voice or a
sound other than a human's voice.
(10) The information processing apparatus of (1), wherein
the analyzing includes determining whether a sound level corresponding to the
captured sound data is greater than or equal to a predetermined threshold value.
(11) The information processing apparatus of (10), wherein
the processor controls the audio volume corresponding to the reproduced content to
remain unchanged when it is determined that the level is less than the predetermined
threshold value.
(12) The information processing apparatus of (10), wherein
the processor determines whether the captured sound data is a human's voice or a
sound other than a human's voice when it is determined that the level is greater than the
predetermined threshold value.
(13) The information processing apparatus of claim 12, wherein
the processor controls the audio volume corresponding to the reproduced content to be
lowered when it is determined that the captured sound data is a human's voice and a
face is not detected based on the captured image data.
(14) The information processing apparatus of (12), wherein
the processor determines a direction corresponding to a source of the captured sound
data when it is determined that the captured sound data is a human's voice and a face is
detected based on the captured image data.
(15) The information processing apparatus of (14), wherein
the processor controls the audio volume corresponding to the reproduced content to
remain unchanged when it is determined that the direction corresponding to the source
of the captured sound is coincident with a location of a face detected based on the
captured image data.
(16) The information processing apparatus of (14), wherein
the processor controls the audio volume corresponding to the reproduced content to
increase when it is determined that the direction corresponding to the source of the
captured sound is not coincident with a location of a face detected based on the
captured image data.
(17) The information processing apparatus of (12), wherein
the processor determines whether the captured sound data corresponds to an envi
ronmental sound registered in advance when it is determined that the captured sound
data is determined to be a sound other than a human's voice.
(18) The information processing apparatus of (17), wherein
the processor controls the audio volume corresponding to the reproduced content to
increase when it is determined that the captured sound data corresponds to an envi
ronmental sound that is registered in advance.
(19) The information processing apparatus of (17), wherein
the processor controls the audio volume corresponding to the reproduced content based
on previously stored settings corresponding to the environmental sound when it is de
termined that the captured sound data corresponds to the environmental sound stored in
advance.
(20) A method performed by an information processing apparatus, the method
comprising:
receiving captured image data and captured sound data corresponding to an en
vironment in which content is reproduced;
detecting a user based on the captured image data;
analyzing a situation of the environment based on a result of the detection and the
captured sound data; and
controlling an audio volume corresponding to reproduced content based on a result of
the analyzing.
(21) A non-transitory computer-readable medium including computer-program in
structions, which when executed by an information processing apparatus, cause the in
formation processing apparatus to perform a method comprising:
receiving captured image data and captured sound data corresponding to an environment
in which content is reproduced;
detecting a user based on the captured image data;
analyzing a situation of the environment based on a result of the detection and the
captured sound data; and
controlling an audio volume corresponding to reproduced content based on a result of
the analyzing.
Reference Signs List
11 Tuner
12 Signal processor
13 Display unit
14 Sound output controller
15 Speaker unit
15L, 15R Speaker
16 Controller
17 Operating unit
2 1 Camera
22 Microphone group
23 Controller
24 Storage unit
24A Environmental sound storage unit
31 Face detector
32 Face information acquisition unit
33 Voice determination unit
34 Situation analyzer
35 Volume/gain controller
36 Speaker controller
101 Bus
102 CPU
103 ROM
104 RAM
105 Hard disk
106 Output unit
107 Input unit
108 Communication unit
109 Drive
110 Input/output interface
111 Removable recording medium
PCT/JP2012/004588
Claims
An information processing apparatus comprising:
a processor that:
receives captured image data and captured sound data corresponding to
an environment in which content is reproduced;
detects a user based on the captured image data;
analyzes a situation of the environment based on a result of the
detection and the captured sound data; and
controls an audio volume corresponding to reproduced content based
on a result of the analyzing.
The information processing apparatus of claim 1, wherein
the processor receives the captured image data from a camera p o
sitioned in the environment in which content is reproduced and detects
a face based on the captured image data.
The information processing apparatus of claim 1, wherein
the processor receives the sound data from a microphone positioned in
the environment in which content is reproduced.
The information processing apparatus of claim 2, wherein
the processor detects a position corresponding to the detected face
based on the captured image data.
The information processing apparatus of claim 2, wherein
the processor detects a plurality of faces based on the captured image
data.
The information processing apparatus of claim 2 wherein
the processor determines face information corresponding to the
detected face, the face information including at least one of an in
dividual, age and gender.
The information processing apparatus of claim 1, wherein
the processor determines a sound level corresponding to the captured
sound data.
The information processing apparatus of claim 1, wherein
the processor determines a direction corresponding to a source of the
captured sound data.
The information processing apparatus of claim 1, wherein
the processor determines whether the captured sound data is a human's
voice or a sound other than a human's voice.
The information processing apparatus of claim 1, wherein
PCT/JP2012/004588
the analyzing includes determining whether a sound level corre
sponding to the captured sound data is greater than or equal to a prede
termined threshold value.
The information processing apparatus of claim 10, wherein
the processor controls the audio volume corresponding to the r e
produced content to remain unchanged when it is determined that the
level is less than the predetermined threshold value.
The information processing apparatus of claim 10, wherein
the processor determines whether the captured sound data is a human's
voice or a sound other than a human's voice when it is determined that
the level is greater than the predetermined threshold value.
The information processing apparatus of claim 12, wherein
the processor controls the audio volume corresponding to the r e
produced content to be lowered when it is determined that the captured
sound data is a human's voice and a face is not detected based on the
captured image data.
The information processing apparatus of claim 12, wherein
the processor determines a direction corresponding to a source of the
captured sound data when it is determined that the captured sound data
is a human's voice and a face is detected based on the captured image
data.
The information processing apparatus of claim 14, wherein
the processor controls the audio volume corresponding to the r e
produced content to remain unchanged when it is determined that the
direction corresponding to the source of the captured sound is co
incident with a location of a face detected based on the captured image
data.
The information processing apparatus of claim 14, wherein
the processor controls the audio volume corresponding to the r e
produced content to increase when it is determined that the direction
corresponding to the source of the captured sound is not coincident
with a location of a face detected based on the captured image data.
The information processing apparatus of claim 12, wherein
the processor determines whether the captured sound data corresponds
to an environmental sound registered in advance when it is determined
that the captured sound data is determined to be a sound other than a
human's voice.
The information processing apparatus of claim 17, wherein
WO 2013/014886 PCT/JP2012/004588
the processor controls the audio volume corresponding to the r e
produced content to increase when it is determined that the captured
sound data corresponds to an environmental sound that is registered in
advance.
[Claim 19] The information processing apparatus of claim 17, wherein
the processor controls the audio volume corresponding to the r e
produced content based on previously stored settings corresponding to
the environmental sound when it is determined that the captured sound
data corresponds to the environmental sound stored in advance.
[Claim 20] A method performed by an information processing apparatus, the
method comprising:
receiving captured image data and captured sound data corresponding
to an environment in which content is reproduced;
detecting a user based on the captured image data;
analyzing a situation of the environment based on a result of the
detection and the captured sound data; and
controlling an audio volume corresponding to reproduced content based
on a result of the analyzing.
[Claim 21] A non-transitory computer-readable medium including computer program
instructions, which when executed by an information
processing apparatus, cause the information processing apparatus to
perform a method comprising:
receiving captured image data and captured sound data corresponding
to an environment in which content is reproduced;
detecting a user based on the captured image data;
analyzing a situation of the environment based on a result of the
detection and the captured sound data; and
controlling an audio volume corresponding to reproduced content based
on a result of the analyzing.