Description
Title of Invention
INFORMATION PROCESSING APPARATUS, WFORMATION PROCESSING
5 METHOD, AND RECORDING MEDIUM
Technical Field
[OOOl]
10 The present disclosure relates to an information processing apparatus, an information
processing method, and a recording medium.
Background Ast
[0002]
In recent years, high functionality and/or multi-functionality of apparatus such as
15 electronic devices have been more advanced, and content of instructions that should
be assigned to an apparatus to be used by a user has become more diversified and
complicated. Under such circumstances, many apparatus adopt a graphical user
interface (GUI) as a user interface. Accordingly, a user can assign an instruction to
an apparatus more easily than in the case in which an apparatus adopting a character
20 user interface (CUI) is used. Howevel; for example, for users who are not familiar
with an operation using a pointing device such as a mouse, tile GUI to be operated
using the pointing device is not necessarily a user-friendly user interface.
[0003]
Under such circumstances, technology related to a user interface for improving
25 operability is being developed. As technology for displaying an image obtained by
combining a captured image with an object image related to an object associated with
a predetermined event on a display screen, for example, there is technology disclosed
in Patent Literature 1.
Citation List
30 Patent Literature
[0004]
Patent Literature 1 :
2/53
JP 2005-216061A
Summary of Invention
Technical Problem
5 [0005]
For example, as in the case in ~rliichth e technology disclosed in PTL 1 is used, a
gesture operation of a user can be fed back to the user by displaying an image
obtained by combining a captured image (or a mirror image based on the captured
image: hereinafter referred to as the same) with an object image related to an object
10 associated with a predetermined event on a display screen.
[0006]
In addition, for example, the user using an apparatus to which the techliology
disclosed in PTL 1 has been applied can cause the apparatus to perform a process
related to the predetermined event associated with the object by operating the object
15 according to the gesture operation. Here, for example, in the technology disclosed
in PTL 1, a position in which the object image is combined is changed according to
the user's position detected frotom the captured image. Consequently, for example,
the occurrence of a problem that it is difficult for the user to operate the object is
likely to be reduced, for example, using the technology disclosed in PTL 1.
20 [0007]
Howevel; for example, the technology disclosed in PTL 1 changes the position in
which the object image is combined according to the user's position detected from
the captured image. Consequently, it is difficult to apply, for example, the
technology disclosed in PTL 1 when an operation target image including an object
25 serving as an operation target (hereinafter referred to as "operation target object")
operable by the user such as a button or an icon, for example, as in a web page, is
combined with the captured image.
[0008]
Here, as a method of displaying a captured image for feeding back gesture
30 recognition to the user and an operation target image on a display screen, for
example, a method of combining the captured image with the operation target image
to display a combined image on the display screen or a method of displaying each of
the captured image and the operation target image as a separate image in one of
division regions into which the display screen is divided is considered.
[0009]
5 However, if the method of merely combining the captured image with the operation
target image to display the combined image on the display screen is used, there is a
problem in that visibility of an operation target object is degraded, for example,
because content of the captured image is superimposed on the operation target object
included in the operation target image. In addition, if the method of displaying each
10 of the captured image and the operation target image as the separate image in one of
the division regions into which the display screen is divided is used, there is a
problem in that visibility of the operation target object included in the operation
target image is degraded, for example, because a size or area of the division region in
which the operation target image is displayed is less than that of the entire display
15 screen.
[OOl 01
The present disclosure proposes a novel and improved information
processing apparatus, information processing method, and recording medium for
enabling a captured image and an operation target image to be displayed on a display
20 screen while preventing visibility of an operation target object from being degraded.
Solution to Problem
[OOll]
According to an embodiment of the present disclosure, there is provided an
25 information processing apparatus including a processing unit configured to control
combining of a captured image and an operation target image so as to generate a
combined image for feeding back gesture recognition to a user. A degree of
visualization of the captured image appears to be changed in the combined image.
[OOlZ]
30 According to an embodiment of the present disclosure, there is provided an
information processing method including the step of combining a captured image
with an operation target image so as to generate a combined image for feeding back
gesture recognition to a user. A degree of visualization of the captured image
appears to be changed in the combined image.
[00 131
5 According to an embodiment of the present disclosure, there is provided a computerreadable
recording medium recording a program for causing a computer to execute a
step of combining a captured image with an operation target image so as to generate
a combined image for feeding back gesture recognition to a user. A degree of
visualization of the captured image appears to be changed in the combined image.
10 [0014]
According to an embodiment of the present disclosure, there is provided an image
processing apparatus including an image processing unit for changing a degree of
visualization of a captured image, for feeding back gesture recognition to a user, to
be displayed on a display screen, based on the captured image or an operation target
15 image to be operated according to a result of the gesture recognition, and combining
the captured image whose degree of visualization has been changed with the
operation target image, and a display control unit for displaying a combined image
on the display screen.
[0015]
20 Fulthel; according to an embodiment of the present disclosure, there is provided an
image processing method including changing a degree of visualization of a captured
image, for feeding back gesture recognition to a user, to be displayed on a display
screen, based on the captured image or an operation target image to be operated
according to a result of the gesture recognition, combining the captured image whose
25 degree of visualization has been changed with the operation target image, and
displaying; a combined image on the display screen.
Advantageous Effects of Invention
[0016]
30 According to the present disclosure, it is possible to display a captured image and an
operation target image on a display screen while preventing visibility of an operation
target object from being degraded.
Brief Description of Drawings
[0017]
5 [Fig. lA]
Fig. 1A is an explanatory diagram illustrating an example of a combined image
displayed by an image processing apparatus according to this embodiment on a
display screen.
[Fig. IB]
10 Fig. 1B is an explanatory diagram illustrating an example of a combined image
displayed by the image processing apparatus according to this embodiment on a
display screen.
[Fig. 2A]
Fig. 2A is an explanatory diagram illustrating an example of a combined image
15 displayed by the image processing apparatus according to this embodiment on a
display screen.
[Fig. 2B]
Fig. 2B is an explanatory diagram illustrating an example of a combined image
displayed by the image processing apparatus according to this embodiment on a
20 display screen.
[Fig. 31
Fig. 3 is an explanatory diagram illustrating an example of a process of cropping a
region including a user from a captured image in the image processing apparatus
according to this embodiment.
25 [Fig. 4A]
Fig. 4A is an explanatory diagram illustrating an example of a combined image
displayed by the image processing apparatus according to this embodiment on a
display screen.
[Fig. 4B]
30 Fig. 48 is an explanatory diagram illustrating an example of a combined image
displayed by the image processing apparatus according to this embodiment on a
display screen.
[Fig. 51
Fig. 5 is an explanatory diagram illustrating an example of a combined image
displayed by the image processing apparatus according to this embodiment on a
5 display screen.
[Fig. 61
Fig. 6 is an explanatory diagram illustrating an example of an operation panel
displayed by the image processing apparatus according to this embodiment on a
display screen.
10 [Fig. 71
Fig. 7 is an explanatory diagram illustrating an example of a combined image
displayed by the image processing apparatus according to this embodiment on a
display screen.
[Fig. 81
15 Fig. 8 is an explanatory diagram illustrating an example of a process of displaying
the combined image as illustrated in Fig. 7 in the image processing apparatus
according to this embodiment.
[Fig. 91
Fig. 9 is an explanatory diagram illustrating an example of a combined image
20 displayed by the image processing apparatus according to this embodiment on a
display screen.
[Fig. 101
Fig. 10 is an explanatory diagram illustrating an example of an operation target
object according to this embodiment.
25 [FIG 111
FIG. 11 is an explanatory diagram illustrating an example of a combined image
displayed by the image processing apparatus according to this embodiment on the
display screen.
[FIG. 121
30 FIG. 12 is a flowchart illustrating an example of a process related to an image
processing method according to this embodiment.
[FIG. 131
FIG. 13 is a block diagram illustrating an example of a configuration of the image
processing apparatus according to this embodiment.
[FIG. 141
5 FIG. 14 is an explanatory diagram illustrating an example of a hardware
configuration of the image processing apparatus according to this embodiment.
Description of Embodiments
[0018]
10 Hereinafter, preferred embodiments of the present disclosure will be described in
detail with reference to the appended drawings. Note that, in this specification and
the appended drawings, structural elements that have substantially the same function
and structure are denoted with the same reference numerals, and repeated
explanation of these structural elements is omitted.
15 [0019]
In addition, description will be given in the following order.
1. Image Processing Method (Information Processing Method) According to This
Embodiment
2. Image Processing Apparatus (Inforn~ation Processing Apparatus) According to
20 This Embodiment
3. Program According to This Embodiment
[0020]
(Image Processing Method According to This Embodiment)
Before the description of a configuration of the image processing apparatus
25 (information processing apparatus, the same shall apply hereafter) according to this
embodiment, the image processing method (information processing method, the
same shall apply hereafter) according to this embodiment will be described. In
addition, the image processing apparatus according to this embodiment that performs
processes related to the image processing method according to this embodiment will
30 be described hereinafter.
[0021]
[I]Overview of Image Processing Method According to This Embodiment
As described above, when a captured image and an operation target image are merely
combined and displayed on a display screen, there is a problem in that visibility of an
operation target object included in the operation target image is degraded. In
5 addition, even when each of the captured image and the operation target image is
displayed as a separate image in one of division regions into wvhich the display screen
is divided, there is a problem in that visibility of the operation target object included
in the operation target image is degraded.
[0022]
10 The image processing apparatus according to this embodiment changes a degree of
visualization of the captured image to be displayed on the display screen based on
the captured image or the operation target image including the operation target
object, and combines the captured image whose degree of visualization has been
changed with the operation target image (a combining process). The image
15 processing apparatus according to this embodiment displays a combined image on
the display screen (a display control process).
[0023]
Here, for example, if an external imaging apparatus or the image processing
apparatus according to this embodiment is provided with an imaging unit (to be
20 described later), the captured image according to this embodiment is an image
obtained by performing mirror-image processing (left-right reversal processing) of an
image captured by the imaging unit (to be described later),). The captured image
according to this embodiment functions to feed back gesture recognition to the usee
An image in wvhich the captured image is combined is displayed on the display
25 screen, so that the user can visually recognize content of his/her gesture such as
causing hislher hand to slide, performing a pressing operation using hislber band, or
performing an enlargement operation, a reduction operation, or a moving operation
using a plurality of fingers. Here, the mirror-image processing may be performed
by the image processing apparatus according to this embodiment or an external
30 apparatus such as an imaging apparatus performing imaging. In addition, an
example of the captured image according to this embodiment is a moving image (or a
plurality of still images). Hereinafter, an example in which the ,captured image
according to this embodiment is a moving image including a plurality of frame
images will be described.
[0024]
5 The captured image according to this embodiment is not limited to an image
subjected to the mirror-image processing (left-right reversal processing). For
example, if a mirror image is captured by the imaging apparatus or the imaging unit
(to be described), the captured image according to this embodiment may be an actual
image that is captured.
10 [0025]
In addition, an example of the operation target image according to this embodiment
is an image including an operation target object operated according to the result of
gesture recognition. Here, an example of the operation target image according to
this embodiment is an image representing a web page, a menu screen of each device,
15 or an operation screen. In addition, an example of the operation target object
according to this embodiment is a button, an icon, or a link such as a hyperlink. A
process related to the gesture recognition may be performed by the image processing
apparatus according to this embodiment or an apparatus outside the image processing
apparatus according to this embodiment.
20 [0026]
In addition, an example of the display screen according to this embodiment is a
display screen in an external display apparatus or a display screen in a display unit
(to be described later) if the itnage processing apparatus according to this
embodiment includes the display unit (to be described later).
25 [0027]
The image processing apparatus according to this embodiment performs, for
example, (1) Combining process and (2) Display Control Process, as the processes
related to the image processing method according to this embodiment. Hereinaftel;
the processes related to the image processing method according to this embodiment
30 will be specifically described.
[0028]
(1) Combining Process
The image processing apparatus according to this embodiment changes a degree of
visualization of a captured image to be displayed on a display screen based on a
captured image or an operation target image. The image processing apparatus
5 according to this embodiment combines the captured image whose degree of
visualization has been changed with the operation target image.
[0029]
Here, an example of the change in the degree of visualization of the captured image
according to this embodiment is a change in a partial range in which the captured
10 image is partially visualized, a change in a degree of smoothing to which the
captured image is smoothed, or the like. In addition, an example of partial
visualization of the captured image according to this embodiment is a change in a
degree to which the captured image is transparent or a change in an area in which the
captured image is displayed. Specific examples of the change in the degree of
15 visualization of the captured image according to this embodiment will be described
later.
[0030]
(1-1) First Example of Process According to Change in Partial Range
The image processing apparatus according to this embodiment determines a partial
20 range, for example, based on a predetermined target recognized from a captured
image. The image processing apparatus according to this embodiment partially
visualizes the captured image, for example, by changing a degree to which the
captured image is transparent based on the determined partial range.
[0031]
25 Here, an example of the predetermined target according to this embodiment is an
object sucli as the user's face or the user's hand. The image processing apparatus
according to this embodiment, for example, recognizes a face region including the
user's face by detecting feature points such as the user's eye, nose, mouth, and
skeletal structure or detecting a region similar to a luminance distribution or structure
30 pattern of the face from the captured image. In addition, the image processing
apparatus according to this embodiment recognizes an object of a recognition target
such as the user's hand from the captured image, for example, by perfoulling various
image processing operations such as edge detection and pattern matching.
[0032]
A process of recognizing the predetermined target in the image processing apparatus
5 according to this embodiment is not limited to the above. For example, the image
processing apparatus according to this embodimellt may recognize a moving body as
the predetermined target from the captured image, and determine a region including
the recognized moving body as the partial range. Here, although the image
processing apparatus according to this embodiment detects the moving body, for
10 example, by detecting a difference between an image corresponding to a current
frame and an image corresponding to a previous frame or detecting a change in a
motion vector, a method of detecting the moving body is not limited to the above.
[0033]
Hereinafter, examples of the process related to the change in the partial range
15 according to this embodiment will be more specifically described.
[0034]
[A] First Example: When Face Is Recognized as Predetermined Target
When the face is recognized as the predetermined target, the image processing
apparatus according to this embodiment recognizes the face from a captured image
20 and determines a portion other than the recognized face as a partial range.
[0035]
Figs. lA, lB, 2A, and 2B are explanatory diagrams illustrating examples of a
combined image displayed by the image processing apparatus according to this
embodiment on a display screen. Here, Fig. 1A is a collceptual diagram of the
25 example illustrated in Fig. IB, and Fig. 2A is a conceptual diagram of the example
illustrated in Fig. 2B. In addition, Figs. lA, lB, 2A, and 28 illustrate examples of
an image when the image processing apparatus according to this embodiment
determines a portion other than the face recognized from the captured image as the
partial range.
30 [0036]
The image processing apparatus according to this embodiment visualizes the portion
other than the recognized face, for example, by performing a process of applying
semi-transparent gradation from the recognized face region. Here, the image
processing apparatus according to this embodiment may dynamically change a
degree of transparency, for example, when the semi-transparent gradation is applied,
5 according to a background portion (for example, a portion other than the recognized
face region) of the captured image.
[0037]
For example, the image processing apparatus according to this embodiment detects a
spatial frequency or hue in the background portion of the captured image and
10 dynamically changes the degree of transparency according to the detection result.
For example, the image processing apparatus according to this embodiment further
decreases the degree of transparency when the detected spatial frequency is higher
(when the image is more complex), and further decreases the degree of transparency
when the detected hue is darker. For example, as described above, the image
15 processing apparatus according to this embodiment can prevent the visibility of the
operation target object included in the operation target image from being degraded by
further decreasing the degree of transparency when the detected spatial frequency is
higher and further decreasing the degree of transparency when the detected hue is
darker.
20 [0038]
A process of dynamically changing the degree of transparency in the image
processing apparatus according to this embodiment is not limited to the above. For
example, the image processing apparatus according to this embodiment may specify
a position of the operation target object from the operation target image (or data
25 c0nstitutil.g the operation target image such as hypertext markup language (HTML)
data or extensible markup language (XML) data), and decrease the degree of
transparency in a region including the specified position. That is, the image
processing apparatus according to this embodiment can change a partial range by
determining the partial range in which the captured image is partially visualized
30 based on the operation target object included in the operation target image.
[0039]
Here, the image processing apparatus according to this embodiment may exclusively
perform one of a process of changing the partial range based on the operation target
object included in the operation target image and a process of changing the partial
range based on the predetermined target recognized from the captured image, and
5 may perform both of the processes. In addition, if the above-described two
processes are performed, the image processing apparatus according to this
embodiment may set priority for each process, for example, such as setting higher
priority for the process of changing the partial range based on the operation target
, object included in the operation target image.
10 [0040]
For example, as described above, the image processing apparatus according to this
embodiment can prevent the visibility of the operation target object included in the
operation target image from being degraded by dynamically changing the degree of
transpareracy based on the position of the operation target object.
15 [0041]
Although the image processing apparatus according to this embodiment, for example,
designates the entire captured image as a region of a target for determining the partial
range, the region of the target for determining the partial range according to this
embodiment is not limited to the above. For example, the image processing
20 apparatus according to this embodiment recognizes a user from the captured image,
and crops a partial region including the user recognized in the captured image from
the captured image. The image processing apparatus according to this embodiment,
for example, may designate the cropped region as the region of the target for
determining the partial range.
25 [0042]
More specifically, the image processing apparatus according to this embodiment, for
example, detects a face region from the captured image. The image processing
apparatus according to this embodiment determines a region having a predetermined
size col~espondingto a size of the detected face region as a region including the user
30 whose face has been detected, for example, by referring to a lookup table in which
the size of the face region is associated with a size of the region (cropped region)
including the user. A method of determining the region including the user
according to this embodiment is not limited to the above. For example, the image
processing apparatus according to this embodiment can determine a minimum region
including the user as the region including the user based on a result of face-region
5 detection and a result of an edge detection process.
[0043]
Fig. 3 is an explanatory diagram illustrating an example of a process of cropping a
region including a user from a captured image in the image processing apparatus
according to this embodiment. A illustrated in Fig. 3 illustrates the entire captured
10 image, and B illustrated in Fig. 3 illustrates a region AR' obtained by cropping and
scaling a region AR of a target for determining a partial range illustrated in A of Fig.
3.
[0044]
For example, as illustrated in Fig. 3, the image processing apparatus according to this
15 embodiment can perform further processing, such as scaling the region of the target
for determining the partial range cropped from the captured image, in addition to
cropping. Needless to say, the image processing apparatus according to this
embodiment may not perform the further processing such as scaling for the region of
the target for determining the partial range cropped from the captured image.
20 [0045]
In addition, the image processing apparatus according to this embodiment may
correct a position of combination, for example, such as combining the region AR' at
the center in a horizontal direction of the operation target image, for example, if the
region AR' (an exanlple of a region based on a region cropped fro111 the captured
25 image) illustrated in B of Fig. 3 is combined with the operation target image.
[0046]
Although one user is included in the captured image in Figs. 1A to 3, a plurality of
users may be included in the captured image according to this embodiment. The
image processing apparatus according to this embodiment crops a partial region
30 including a recognized user from the captured image for each recognized user if the
plurality of users have been recognized from the captured image. The image
processing apparatus according to this embodiment designates each cropped region
(or a region subjected to further processillg such as scaling for each region) as a
region of a target for determining a partial range corresponding to each recognized
user. In addition, even when the plurality of users have been recognized from the
5 captured image, the image processing apparatus according to this embodiment can
correct a position in which each cropped region is combined with the operation target
image.
[0047]
In addition, if the plurality of users have been recognized from the captured image, a
10 process related to gesture recognition may be performed by the image processing
apparatus according to this embodiment or the external apparatus, for example,
independently for each recognized usel; in cooperation according to operations of the
plurality of users, or exclusively. In addition, if the plurality of users have been
recognized from the captured image, the process related to the gesture recognition
15 may be performed by the image processing apparatus according to this embodiment
or the external apparatus, according to priority set for each user. Here, there are
various methods, for example, such as setting high priority for a first recognized user,
a last recognized user, or a user for which the size of a partial region including the
recognized user is largest, as the above-described priority setting method.
20 [0048]
[B] When Hand Is Recognized as Predetermined Target
If the hand is recognized as the predetermined target, the image processing apparatus
according to this embodiment recognizes the hand from a captured image and
determines a region including the recognized hand as a partial range. Here, the
25 image processing apparatus according to this embodiment, for example, may
recognize the hand from the entire captured image or recognize the hand from a
region cropped from the captured image as illustrated in Fig. 3.
[0049]
In addition, the image processing apparatus according to this embodiment visualizes
30 the region including the hand, for example, by performing a process of applying a
mask to a region other than the region including the recognized hand in the captured
image. In addition, the image processing apparatus according to this embodiment
may visualize the region including the recognized hand, for example, by performing
a process of applying semi-transparent gradation. Here, if the process of applying
the semi-transparent gradation is performed, the image processing apparatus
5 according to this embodiment may dynamically change a degree of transparency, for
example, when the semi-transparent gradation is applied, according to a background
portion of the captured image as in the process related to the above-described first
example.
[0050]
10 [C] Third Example: When Moving Body Is Recognized as Predetermined Target
If the moving body is recognized as the predetermined target, the image processing
apparatus according to this embodiment recognizes the moving body as the
predetermined target from a captured image, and determines a region including the
recognized moving body as a partial range. Here, the image processing apparatus
16 according to this embodiment, for example, may recognize the moving body from
the entire captured image, or recognize the moving body from a region cropped from
the captured image as illustrated in Fig. 3.
[0051]
Figs. 4A and 4B are explanatory diagrams illustrating examples of a combined image
20 displayed by the image processing apparatus according to this embodiment on a
display screen. Here, Fig. 4A is a conceptual diagram of the example illustrated in
Fig. 4B. In addition, Figs. 4A and 4B illustrate examples of an image when the
image processing apparatus according to this embodiment has determined a region in
which a moving body has been recognized from a captured image as a partial range.
25 [0052]
The image processing apparatus according to this embodiment visualizes the region
in which the moving body has been recognized, for example by visualizing the
region in which the moving body has been recognized as if a hand had been dragged
across frosted glass as illustrated in Figs. 4A and 4B. Needless to say, the image
30 displayed on a display screen when the moving body is recognized as the
predetermined target is not limited to the examples illustrated in Figs. 4A and 4B.
[0053]
(1-2) Second Example of Process Related to Change in Partial Range
An example in which the image processing apparatus according to this embodiment
determines the partial range based on the predetermined target recognized from the
5 captured image and partially visualizes the captured image based on the determined
partial range has been described as the first example of the process related to the
change in the partial range. However, the process related to the change in the
partial range in the image processing apparatus according to this embodiment is not
limited to the above-described example. For example, the image processing
10 apparatus according to this embodiment can change the partial range in which the
captured image is partially visualized by changing an area in which the captured
image is displayed.
[0054]
Fig. 5 is an explanatory diagram illustrating an example of a combined image
15 displayed by the image processing apparatus according to this embodiment on a
display screen. Here, Fig. 5 illustrates an example in which a partial range in which
a captured image is visualized is changed by applying a mask to the captured image.
[OOSS]
The image processing apparatus according to this embodiment decreases an area in
20 which the captured image is displayed, and changes the partial range in which the
captured image is visualized by applying the mask to the captured image, for
example, as illustrated in Fig. 5.
[0056]
The process in the image processing apparatus according to this embodiment is not
25 limited to the example illustrated in Fig. 5. Although the image processing
apparatus according to this embodiment, for example, applies a uniform mask to the
entire captured image in Fig. 5, the image processing apparatus according to this
embodiment may dynamically change, for example, the density or shape of the mask.
For example, the image processing apparatus according to this embodiment specifies
30 a position of an operation target object from an operation target image (or data
constituting an operation target image such as HTML data or XML data) and
decreases the density of the mask on a region including the specified position or
around the region. That is, the image processing apparatus according to this
embodiment can change the partial range by determining the partial range in which
the captured image is partially visualized based on the operation target object
5 included in the operation target image.
[0057]
For example, as described above, the image processing apparatus according to this
embodiment can prevent the visibility of the operation target object included in the
operation target image from being degraded by dynamically changing the density or
10 shape of the mask based on the position of the operation target object.
[0058]
In addition, the image processing apparatus according to this embodiment may apply
the mask to the operation panel (an example of the operation target object) displayed
on a part of the display screen without being limited to the application of the mask to
15 the entire captured image. Fig. 6 is an explanatory diagram illustrating an example
of the operation panel displayed by the image processing apparatus according to this
embodiment on the display screen. Needless to say, the operation panel displayed
by the image processing apparatus according to this embodiment on the display
screen is not limited to the example illustrated in Fig. 6.
20 [0059]
(1-3) Third Example of Process Related to Change in Partial Range
In the image processing apparatus according to this embodiment, the process related
to the change in the partial range is not limited to the above-described first and
second examples. Fig. 7 is an explanatory diagram illustrating an example of a
25 combined image displayed by the image processing apparatus according to this
embodiment on a display screen.
[0060]
For example, as a method of feeding back a gesture to a user so that the user can
view content without interference as much as possible if an image representing
30 content related to television broadcasting or the like is displayed on the display
screen, an on-screen display (OSD) is displayed on a part of the display screen, for
example, as illustrated in Fig. 7. Here, in Fig. 7, an example in which the operation
panel (an example of the operation target object) with which a part of the captured
image is combined in a part of the display screen is displayed as illustrated in A of
Fig. 7.
5 [0061]
Here, for example, if only an icon is displayed on the OSD, a situation in which
feedback for the user is not sufficient, for example, soch as "the case in which it is
not known how much the arm should move when a IeWright flick operation (an
example of the gesture) is performed" or "the case in which an operation failure
10 cause (for example, out of an angle of view of the imaging apparatus or the like) is
not known when an operation is not suitably perfomled," may occur.
[0062]
On the other hand, the image processing apparatus according to this embodiment
combines a part cropped from the captured image with the operation panel, for
15 example, as illustrated in A of Fig. 7. For example, as illustrated in Fig. 7, the
image processing apparatus according to this embodiment displays the operation
panel combined with the captured image on the display screen. Consequently, the
image as illustrated in Fig. 7 is displayed on the display screen, so that the user can
more easily understand that "the gesture is recognized if the hand is shaken to an
20 extent to which the hand is in contact with the icon" or that "the operation is not
suitably performed because an angle of view or recognition of the imaging apparatus
is out of range."
[0063]
Here, an example of a process of displaying the combined image as illustrated in Fig.
25 7 in the image processing apparatus according to this embodiment will be more
specifically described. Fig, 8 is an explanatory diagram illustrating the example of
the process of displaying the combined image as illustrated in Fig. 7 in the image
processing apparatus according to this embodiment. Fig. 8 illustrates an example of
a captured image. Hereinafter, an example in which the image processing apparatus
30 according to this embodiment recognizes a user's hand as a gesture recognition target
for recognizing the user's gesture will be described.
[0064]
When recognizing the hand (an example of the gesture recognition target) from the
captured image, the image processing apparatus according to this embodiment
determines a region (B illustrated in Fig. 8) defining a range in which the gesture is
5 recognized based on a region (A illustrated in Fig. 8) including the recognized hand.
The image processing apparatus according to this embodiment, for example,
designates a larger region including the region including the recognized hand as the
region defining the range in which the gesture is recognized. Here, although a size
of the larger region including the region including the recognized hand is determined,
10 for example, according to a size of the region including the recognized hand, a
method of determining the size of the larger region including the region including the
recognized hand is not limited to the above. For example, lower and upper limits of
the above-described size of the larger region including the region including the
recognized hand may be set. The user may set the above-described size of the
15 larger region including the region including the recognized hand. For example, the
region defining the range in which the gesture is recognized is determined as
described above, so thata region around the region including the recognized hand is
determined as the region defining the range in which the gesture is recognized.
[0065]
20 If the region defining the range in which the gesture is recognized is determined, the
image processing apparatus according to this embodiment trims the region defining
the range in which the above-described gesture is recognized from the captured
image, and combines the trimmed region with the operation panel. The image
processint; apparatus according to this embodiment displays the combined operation
25 panel on the display screen.
[0066]
Here, although the image processing apparatus according to this embodiment
displays the operation panel combined in a predefined position of the display screen,
the image processing apparatus according to this embodiment does not limit the
30 position in which the combined operation panel is displayed to the above. For
example, the image processing apparatus according to this embodiment may display
the combined operation panel in a position corresponding to a position of the hand
(an example of the gesture recognition target) first recognized from the captured
image.
[0067]
5 (1-4) Process Related to Change in Degree of Smoothing to Which Captured Image
Is Smoothed
Although the process related to the change in the partial range has been described
above as the process related to the image processing method in the image processing
apparatus according to this embodiment, the process related to the image processing
10 method according to this embodiment is not limited to the above. For example, the
image processing apparatus according to this embodiment may change a degree of
smoothing to which the captured image is smoothed as a process of changing a
degree of lay screen.
[0089]
25 Here, because the image processing apparatus according to this embodiment changes
the degree of visualization of the captured image in the above-described process
(combining process) (I), it is possible to prevent the visibility of the operation target
object from being degraded even when the captured image and the operation target
image are merely combined and displayed on the display screen.
30 [0090]
Therefore, the image processing apparatus according to tbis embodiment can display
the captured image and the operation target image on the display screen while
preventing the visibility of the operation target object from being degraded.
[0091]
If the image processing apparatus according to this embodiment determines the
5 partial range in which the captured image is partially visualized based on the
operation target object included in the operation target image in the above-described
process (combining process) (I), it is possible to display an image in which the
operation target object included in the operation target image is more clearly
displayed on the display screen. Therefore, in the above-described case, the itnage
10 processing apparatus according to this embodiment, for example, can further
improve the visibility of the operation target object (that is, an operation target object
constituting a GUI) included in the operation target image.
[0092]
In addition, if the image processing apparatus according to this embodiment
15 determines the partial range in which the captured itnage is partially visualized based
on the predetermined target recognized from the captured image in the abovedescribed
process (combining process) (I), a face of the user who performs the
operation is not displayed on the display screen, for example, as illustrated in Figs.
1A to 2B. Therefore, in the above-described case, because the image processing
20 apparatus according to this embodiment, for example, can implement an unrealistic
mirror-image expression for the user, it is possible to feed back unrealistic gesture
recognition to the user.
[0093]
Further, because the image processing apparatus according to this embodiment
25 changes the degree of visualization of the captured image in the above-described
process (combining process) (I), it is possible to control visualization of a region
unnecessary for an operation by a gesture even when the captured image and the
operation target image are merely combined and displayed on the display screen.
Therefore, because the image processing apparatus according to this embodiment, for
30 example, enables the user to more easily perform an operation by a more precise
gesture, it is possible to further improve the operability of the user.
[0094]
[2]Example of Process Related to Image Processing Method According to This
Embodiment
Fig. 12 is a flowchart illustrating an example of a process related to the image
5 processing method according to this embodiment: Hereinafter, an example in
which the image processing apparatus according to this embodiment performs the
process illustrated in Fig. 12 will be described. Here, in Fig. 12, for example, the
process of steps S100, S102, and S104 corresponds to the above-described process
(combining process) (I), and the process of step S106 corresponds to the above-
10 described process (display control process) (2).
[0095]
The image processing apparatus according to this embodiment determines whether or
not to perform feedback for the user (S100). Here, the process of step SlOO
corresponds to a process in which the image processing apparatus according to this
15 embodiment determines whether or not to start a process related to the image
processing method according to this embodiment.
[0096]
The image processing apparatus according to this embodiment determines to perform
the feedback for the user (that is, determines to start the process related to the image
20 processing method according to this embodiment), for example, when a
predetermined target has been recognized from the captured image. Here, an
example of the above-described predetermined target is a gesture recognition target
for recognizing the user's gesture such as a part of a body of the user such as the
user's hand or a shape of an object to be operated by the. usee
25 [0097]
The process of step SlOO according to this embodiment is not limited to the above.
For example, the image processing apparatus according to this embodiment may
determine whether or not to perform the feedback for the user based on an operation
signal corresponding to the user's operation on an operation unit (to be described
30 later) or an external operation signal corresponding to the user's operation on an
external operation apparatus such as a remote controlle~ In addition, the image
processing apparatus according to this embodiment may determine to perform the
feedback for the user, for example, when a predetermined gesture has been
recognized such as when the user shakes his or her hand. Although the image
processing apparatus according to this embodiment recognizes a predetermined
5 gesture, for example, by detecting the user's periodic operation based on a frequency
of a luminance change in the captured image, a gesture recognition method in the
image processing apparatus according to this embodiment is not limited to the above.
[0098]
When determining not to perform the feedback for the user in step S100, the image
10 processing apparatus according to this embodiment ends the process related to the
image processing method according to this embodiment. The process related to the
image processing method according this embodiment is iterated, for example,
periodically, aperiodically, or every time an image signal representing a captured
image is input, instead of not being re-performed once the process ends.
15 [0099]
In addition, when determining to perform the feedback for the user in step S100, the
image processing apparatus according to this embodiment changes a degree of
visualization of the captured image (S102). The image processing apparatus
according to this embodiment changes the degree of visualization of the captured
20 image, for example, by changing a degree to which the captured image is transparent,
changing an area in which the captured image is displayed, or changing a degree of
smoothing to which the captured image is smoothed.
[OlOO]
If a process of changing the degree of visualization of the captured image ends in
25 step S102, the image processing apparatus according to this embodiment combines
the captured image whose degree of visualization has been changed with the
operation target image (S104). Here, if a region based on a region cropped from the
captured image (or the region itself cropped from the captured image), for example,
such as the region AR' illustrated in B of Fig. 3, is conibined with the operation
30 target image, the image processing apparatus according to this embodiment may
correct a position in which the region is combined.
[ O l O l ]
If the process related to combination between the captured image whose degree of
visualization has been changed and the operation target image ends in step S104, the
image processing apparatus according to this embodiment displays a combitled
5 image on the display screen (S106). The image processing apparatus according to
this embodiment ends the process related to the image processing method according
to this embodiment.
[O 1021
The image processing apparatus according to this embodiment implements the
10 process (combining process) (1) and the process (display control process) (2) related
to the image processing method according to this embodiment, for example, by
performing the process illustrated in Fig. 12. Needless to say, the process related to
the image processing method according to this embodiment is not limited to the
process illustrrtted in Fig. 12.
15 [0103]
(Image Processing Apparatus According to This Embodiment)
Next, an example of a configuration of the image processing apparatus according to
this embodiment capable of performing the process related to the image processing
method according to this embodiment will be described.
20 [0104]
Fig. 13 is a block diagram illustrating an example of the configuration of the image
processing apparatus 100 according to this embodiment. The image processing
apparatus 100, for example, includes a communication unit 102 and a control unit
(processing unit) 104.
25 [OlOS]
In addition, the image processing apparatus 100 may include, for example, a read
only memory (ROM) (not illustrated) or a random access memory (RAM) (not
illustrated), a storage unit (not illustrated), an operation unit (not illustrated) capable
of being operated by the user, a display unit (not illustrated) that displays various
30 screens on a display screen, and the like. The image processing apparatus 100
establishes a connection between the above-described components by a bus, for
example, as a transmission path for data.
[Ol 061
Here, the ROM (not illustrated) stores programs to be used by the control unit 104
and control data such as arithmetic parameters. The RAM (not illustrated)
5 temporarily stores programs and the like to be executed by the control unit 104.
[OI 071
The storage unit (not illustrated) is a storage means provided in the image processing
apparatus 100, and, for example, stores data constituting the operation target image
or various data such as an application. Here, an example of the storage unit (not
10 illustrated) is a magnetic recording medium such as a hard disk or a nonvolatile
memoly such as an electrically erasable and programmable ROM (EEPROM) or a
flash memory. In addition, the storage unit (not illustrated) may be attachable to or
detachable from the image processing apparatus 100.
[0108]
15 [Hardware Configuration Example of Image Processing Apparatus 1001
Fig. 13 is an explanatory diagram illustrating an example of a hardware configuration
of the image processing apparatus 100 according to this embodiment. The image
processing apparatus 100 includes, for example, a micro processing unit (MPU) 150,
a ROM 152, a RAM 154, a recording medium 156, an inputloutput (110) interface
20 158, an operation input device 160, a display device 162, and a communication
interface 164. In addition, the image processing apparatus 100 establishes a
connection between the above-described components by a bus 166, for example, as a
transmission path for data.
[0109]
25 The MPU 150 functions as the control unit 104 that controls the entire image
processing apparatus 100, for example, constituted by an MPU or various processing
circuits. In addition, the MPU 150, for example, functions as an image processing
unit 110 and a display control unit 112 in the image processing apparatus 100 as will
be described later.
30 [OllO]
The ROM 152 stores programs to be used by the MPU 150 and control data such as
arithmetic parameters, and the RAM 154 primarily stores, for example, programs and
the like to be executed by the MPU 150.
[Olll]
The recording medium 156 functions as the storage unit (not illustrated), and, for
5 example, stores data constituting the operation target image or various data such as
an application. Here, an example of the recording medium 156 is a magnetic
recording medium such as a hard disk or a nonvolatile memory such as a flash
memoly. In addition, the recording medium 156 may be attachable to or detachable
from the image processing apparatus 100.
10 [01121
The I/O interface 158 is connected to, for example, the operation input device 160
and the display device 162. The operation input device 160 functions as an
operation unit (not illustrated), and the display device 162 functions as a display unit
(not illustrated). Here, an example of the L/O interface 158 is a universal serial bus
15 (USB) terminal, a digital visual interface (DVI) terminal, a high-definition
multimedia interface (HDMI) terminal, various processing circuits, or the like. In
addition, the operation input device 160 is provided, for example, on the image
processing apparatus 100, and is connected to the I/O interface 158 inside the image
processing apparatus 100. An example of the operation input device 160 is a button,
20 a direction key, a rotary selector such as a jog dial, or a combination thereof. In
addition, the display device 162 is provided, for example, on the image processing
apparatus 100, and is connected to the 110 interface 158 inside the image processing
apparatus 100. An example of the display device 162 is a liquid clystal display
(LCD), an organic electroluminescence (EL) display (which is also refersed to as an
25 organic light emitting diode (OLED) display), or the like.
[0113]
Needless to say, the 110 interface 158 can be connected to an external device such as
a display device (for example, an external display or the like) or an operation input
device (for example, a keyboard, a mouse, or the like) as an apparatus outside the
30 image processing apparatus 100. In addition, the display device 162 may be a
device in which a display and the user's operation are possible, for example, such as
a touch screen.
[0114]
The communication interface 164 is a communication means provided in the image
processing apparatus 100, and functions as the communication unit 102 for
5 performing wireless/wired communication with an imaging apparatus, a display
apparatus, or an external apparatus such as a server via the network (or directly).
Here, an example of the communication interface 164 is a communication autenna
and a radio frequency (RF) circuit (wireless communication), an Institute of
Electrical and Electronics Engineers (IEEE) 802.15.1 port and a
10 transmission/reception circuit (wireless communication), an IEEE 802.11b post and a
transmission/reception circuit (wireless communication), or a local area network
(LAN) terminal and a transmission/reception circuit (wired communication). In
addition, an example of the network according to this embodiment is a wired
network such as a LAN or a wide area network (WAN), a wireless network such as a
15 wireless WAN (WWAN) via a wireless LAN (WLAN) or a base station, or the
Internet using a communication protocol such as a transmission control
protocol/Internet protocol (TCPIIP).
[OllS]
For example, according to the configuration illustrated in Fig. 14, the image
20 processing apparatus 100 performs a process related to the display control method
according to this embodiment. The hardware configuration of the image processing
apparatus 100 according to this embodiment is not limited to the configuration
illustrated in Fig. 14. For example, the image processing apparatus 100 may be
provided with an imaging device that functions as an imaging unit (not illustrated) to
25 capture a still image or a moving image. When the imaging device is provided, the
image processing apparatus 100, for example, can process a captured image
generated by imaging in the imaging device.
[0116]
Here, examples of the imaging device according to this embodiment are a
30 lenstimaging element and a signal processing circuit. The lenslimaging element is
constituted by an image sensor using a plurality of imaging elements, for example,
such as a lens of an optical system and a complementary metal oxide se~niconductor
(CMOS). In addition, the signal processing circuit includes, for example, an
automatic gain control (AGC) circuit andfor an analog to digital converter (ADC),
converts an analog signal generated by the imaging element into a digital signal
5 (image data), and processes various signals. An example of signal processing to be
performed by the signal processing circuit is a white balance correction process, a
color tone correction process, a gamma correction process, a YCbCr conversion
process, an edge enhancement process, or the like.
[0117]
10 In addition, in a configuration in which a process is performed in a stand-alone
mannel; the image processing apparatus 100 may not be provided with the
communication device 164. In addition, it is possible to configure the image
processing apparatus I00 without the operation device 160 or the display device 162.
[0118]
15 An example of the configuration of the image processing apparatus 100 will be
described with reference back to Fig. 13. The communication unit 102 is a
communication means provided in the image processing apparatus 100, and performs
wireless/wired communication with the imaging apparatus, the display apparatus, or
the external apparatus such as a sewer via the network (or directly). In addition,
20 communication of the communication unit 102 is controlled, for example, by the
control unit 104. Here, although an example of the communication unit 102 is a
communication antenna and an RF circuit or a LAN terminal and a
transmission/reception circuit, the configuration of the communication unit 102 is not
limited to the above. For example, the communication unit 102 can have a
25 configuration corresponding to an arbitrary standard in which communication is
possible such as a USB terminal and a transmissioll/reception circuit or an arbitrary
configuration communicable with an external apparatus via the network.
[0119]
The control unit 104 is constituted, for example, by an MPU, and functions to control
30 the entire image processing apparatus 100. In addition, the control unit 104
includes, for example, the image processing unit 110 and the display control unit 112,
and functions to play a leading role in the process related to the image processillg
method according to this embodiment. More specifically, the control unit 104, for
example, controls combining of the captured image and the operation target itnage so
as to generate the combined image for feeding back gesture recogtlitiou to the usee
5 [0120]
The image processing unit 110 functions to play a leading role in the abovedescribed
process (combining process) (I), and changes a degree of visualization of a
captured image to be displayed on a display screen based on the captured image or an
operation target image, and combines the captured image whose degree of
10 visualization has been changed with the operation target image.
[0121]
The display control unit 112 functions to play a leading role in the above-described
process (display control process) (2), and displays the image combined by the image
processing unit 110 on the display screen.
15 [0122]
The control unit 104 plays a leading role in the process related to the image
processing method according to this embodiment, for example, by including the
image processing unit 110 and the display control unit 112.
[0123]
20 For example, according to the configuration illustrated in Fig. 13, the image
processing apparatus 100 perfornls the processes (for example, the above-described
process (combining process) (1) and the above-described process (display control
process) (2)) related to the image processing method according to this embodiment.
Therefore, the image processing apparatus 100 can display the captured itnage and
25 the operation target image on the display screen while preventing visibility of an
operation target object from being degraded, for example, according to the
configuration illustrated in Fig. 13.
[0124]
The configuration of the image processing apparatus according to this embodiment is
30 not limited to the configuration illustrated in Fig. 13. For example, the image
processing apparatus according to this embodiment can separately include the image
processing unit 110 and the display control unit 112 illustrated in Fig. 13 (for
example, each is implemented by a separate processing circuit).
[0125]
In addition, the image processing apparatus according to this embodiment may
5 include, for example, an imaging unit (not illustrated). When the imaging unit (not
illustrated) is provided, the image processing apparatus according to this embodiment
can process a captured image generated by imaging in the imaging unit (not
illustrated).
[0126]
10 For example, in a configuration in which a process is performed in a stand-alone
manner, the image processing apparatus according to this embodiment may not be
provided with the communication unit 102.
101271
As described above, the image processing apparatus according to this embodiment
15 performs, for example, the above-described process (combining process) (1) and the
above-described process (display control process) (2), as the process related to the
image processing method according to this embodiment. Here, because the image
processing apparatus according to this embodiment changes the degree of
visualization of the captured image in the above-described process (combining
20 process) (I), it is possible to prevent the visibility of the operation target object from
being degraded even when the captured image and the operation target image are
merely combined and displayed on the display screen.
[0128]
Therefore, the image processing apparatus according to this embodiment can display
25 the captured image and the operation target image on the display screen while
preventing the visibility of the operation target object from being degraded.
[0129]
In addition, when the image processing apparatus according to this embodiment
determines a partial range in which the captured image is partially visualized based
30 on the operation target object included in the operation target image in the abovedescribed
process (combining process) (I), it is possible to display an image in which
the operation target object included in the operation target image is more clearly
displayed on the display screen. Therefore, in the above-described case, the image
processing apparatus according to this embodiment, for example, can further
improve the visibility of the operation target object (that is, an operation target object
5 constituting a GUI) included in the operation target image.
[0130]
In addition, when the image processing apparatus according to this embodiment
determines the partial range in which the captured image is partially visualized based
on a predetermined target recognized from the captured image in the above-described
10 process (combining process) (I), the image processing apparatus according to this
embodiment, for example, can implement an unrealistic mirror-image expression for
the user, Therefore, in the above-described case, the image processing apparatus
according to this embodiment, for example, can feed back ulirealistic gesture
recognition to the user.
15 [0131]
Further, because the image processing apparatus according to this embodiment
changes a degree of visualization of the captured image in the above-described
process (combining process) (I), it is possible to control visualization of a region
unnecessary for an operation by a gesture even when the captured image and the
20 operation target image are merely combined and displayed on the display screen.
Therefore, because the image processing apparatus according to this embodiment, for
example, enables the user to more easily perform an operation by a more precise
gesture, it is possible to further improve the operability of the user.
[0132]
25 Although the image processing apparatus has been described as this embodiment,
this embodiment is not limited to this type. This embodiment can be applied to
various devices capable of processing an image and controlling a display of the
processed image on the display screen. The devices include a communication
apparatus such as a portable phone or a smart phone, a videolmusic reproduction
30 device (or video/music recording/reproduction device), a game device, a computer
such as a personal computer (PC) or a servel; a display apparatus such as a television
receiver, and an imaging apparatus such as a digital camera. In addition, this
embodiment, for example, can be embedded in the device as described above, and
applied to a processing integrated circuit (IC).
[0133]
5 In addition, the process related to the itnage processing method according to this
embodiment may be implemented by an image processing system including a
plurality of apparatus on a premise of a connection to a network, for example, such
as cloud computing (or communication between apparatus).
[0134]
10 (Program According to This Embodiment)
A program (for example, a program capable of performing the processes related to
the image processing method according to this embodiment such as the abovedescribed
process (combining process) (1) and the above-described process (display
control process) (2)) for causing a computer to function as the image processing
15 apparatus according to this embodiment can display a captured image and an
operation target image on a display screen while preventing visibility of an operation
target object from being degraded.
[0135]
Although the preferred embodiments of the present disclosure have been described
20 with reference to the accompanying drawings, a technical scope of the present
disclosure is not limited thereto. It should be understood by those skilled in the art
that various modifications, combinations, sub-combinations and alterations may
occur depending on design requirements and other factors insofar as they are within
the scope of the appended claims or the equivalents thereof.
25 [0136]
Although an example in which the program (computer program) for causing the
computer to function as the image processing apparatus according to this
embodiment is provided has been described above, a recording medium recording
each program described above can be further provided in this embodiment.
30 [0137]
The above-described configuration is an example of this embodiment, and, of course,
belongs to the technical scope of the present disclosure.
[0138]
Additionally, the following configurations are also within the technical scope of the
present disclosure.
5 (1)
An information processing apparatus including:
a processing unit configured to control combining of a captured image and
an operation target image so as to generate a combined image for feeding back
gesture recognition to a user,
10 vrherein a degree of visualization of the captured image appears to be
changed in the combined image.
(2)
The infonnation processing apparatus according to (I), wherein the degree
of the visualization in part of the captured image appears to be changed in the
15 combined image.
(3)
The infor~nation processing apparatus according to (2), wherein the part of
the captured image corresponds to a region of the combined image having a position
determined according to a position in the operation target image of an operation
20 target object in the combined image.
(4)
The information processing apparatus according to (I), wherein the degree
of the visualization of the captured image is changed based on an operation target
object included in the operation target image.
25 ( 5 )
The information processing apparatus according to (4),
wherein the degree of the visualization of the captured image includes a
decrease in a degree of visualization of a region of the captured image, and
wherein the region is determined according to a position of the operation
30 target object within the operation target image.
(6)
The information processing apparatus according to (I), wherein the degree
of the visualization of the captured image is changed based on a predetermined target
recognized from the captured image.
(7)
5 The information processing apparatus according to (6), wherein the degree
of the visualization of the captured image is changed by at least a size of a part
cropped from the captured image.
(8)
The information processing apparatus according to (6), wherein the
10 predetermined target is at least one of a face and a hand.
(9)
The information processing apparatus according to (6), wherein the
predetermined target is a web page.
(10)
15 The infomation processing apparatus according to (6), wherein changing
the degree of the visualization of the captured image includes changing the degree of
the visualizatioll of the partial region including the predetermined target in the
captured image to the degree of the visualization of the other region in the captured
image.
20 (11)
The infor~nationp rocessing apparatus according to (lo), wherein changing
the degree of the visualization of the captured image includes changing one or more
of luminance, saturation, and colors of the partial region and the other region.
(12)
25 The information processing apparatus according to (lo), wherein changing
the degree of the visualization of the captured image includes setting one of the
partial region and the other region as a monochromatic region and setting the other as
a color region.
(13)
30 The information processing apparatus according to (lo), wherein changing
the degree of the visualization of the captured image includes changing degrees of
blurring of the partial region and the other region.
(14)
The information processing apparatus according to (lo), wherein changing
the degree of the visualization of the captured image includes displaying only the
5 partial region.
(15)
The information processing apparatus according to (I), wherein the degree
of the visualization of the captured image is changed so that the captured image
appears to be classified by rank in the combined image.
10 (16)
The information processing apparatus according to (I), wherein changing
the degree of the visualization of the captured image includes changing a degree to
which the captured image is transparent.
(17)
15 The information processing apparatus according to (16), wherein changing
the degree to which the captured image is transparent includes dynamically changing
the degree to which the captured image is transparent.
(18)
The information processing apparatus according to (17), wherein
20 dynamically changing the degree to which the captured image is transparent includes
dynamically changing the degree to which the captured image is transparent based on
a spatial frequency of the captured image.
(19)
The information processing apparatus according to (17), wherein
25 dynamically changing the degree to which the captured image is transparent includes
dynamically changing the degree to which the captured image is transparent based on
a color shade of the captured image.
(20)
The information processillg apparatus according to (I), wherein changing
30 the degree of the visualization of the captured image includes cropping the captured
(21)
The information processing apparatus according to (20), wherein changing
the degree of the visualization of the captured image further includes scaling the
cropped image.
5 (22)
The information processing apparatus according to (I), wherein changing
the degree of the visualization of the captured image includes smoothing the captured
image.
(23)
10 The information processing apparatus according to (22), wherein smoothing
the captured image includes using an edge-preserving smoothing filter.
(24)
The information processing apparatus according to (I), wherein combining
a changed captured image with the operation target image includes changing a
15 display position of the changed captured image with respect to a display position of
the operation target image.
(25)
The information processing apparatus according to (I),
wherein the captured image includes a face part of the user and another part
20 of the user, and
wherein the face part is excluded from the combined image.
(26)
The information processing apparatus according to (I),
wherein the captured image includes an image of a plurality of users, and
wherein changing the degree of the visualization of the captured image
includes cropping the captured image so as to generate a region cropped for each of
the plurality of recognized users.
(27)
The information processing apparatus according to (I), wherein the
30 processing unit controls a display of the combined image.
(28)
Tlte information processing apparatus according to (I), further including:
a display unit,
wherein the processing unit controls a display of the combined image.
(29)
5 The information processing apparatus according to (I), further including:
an imaging unit.
(30)
The information processing apparatus according to (I), wherein the
operation target image is one of an image representing a web page, an image
10 representing a menu screen, and an image representing an operation screen.
(3 1)
An information processing method including the step of:
combining a captured image with an operation target image so as to generate
a combined image for feeding back gesture recognition to a usel;
15 wherein a degree of visualization of the captured image appears to be
changed in the combined image.
(32)
A computer-readable recording medium recording a program for causing a
computer to execute a step of:
20 combining a captured image with an operation target image so as to generate
a combined image for feeding back gesture recognition to a user,
wherein a degree of visualization of the captured image appears to be
changed in the combined image.
(33)
25 An image processing apparatus including:
an image processing unit for changing a degree of visualization of a captured image,
for feeding back gesture recognition to a user, to be displayed on a display screen,
based on the captured image or an operation target image to be operated according to
a result of the gesture recognition, and combining the captured image whose degree
30 of visualization has been changed with the operation target image; and
a display control unit for displaying a combined image on the display screen.
(34)
The image processing apparatus according to (33), wherein the image processing unit
changes a partial range in which the captured image is partially visualized, as a
change in the degree of visualization.
5 (35)
The image processing apparatus according to (34), wherein the image processing unit
determines the partial range based on the operation target object included in the
operation target image.
(36)
10 The image processing apparatus according to (34), wherein the image processing unit
determines the partial range based on a predetermined target to be recognized from
the captured image.
(37)
The image processing apparatus according to (36), wherein the image processing unit
15 recognizes a face as the predetermined target from the captured image, and
determines a portion other than the recognized face as the partial range.
(38)
The image processing apparatus according to (36), wherein the image processing unit
recognizes a hand as the predetermined target from the captured image, and
20 determines a region including the recognized hand as the partial range.
(39)
The image processing apparatus according to (36), wherein the image processing unit
recognizes a moving body as the predetermined target from the captured itnage, and
determines a region including the recognized moving body as the partial range.
25 (40)
The image processing apparatus according to (34), wherein the image processing unit
recognizes a user from the captured image,
crops a partial region including the recognized user in the captured image from the
captured image, and
30 designates the cropped region as a region of a target for determining the partial
(41)
The image processing apparatus according to (40), wherein the image processing unit
crops a partial region including a recognized user for each recognized user when a
plurality of users have been recognized from the captured image, and
5 designates each cropped region as a region of a target for determining the partial
range corresponding to each recognized user.
(42)
The image processing apparatus according to (34), wherein the image processing unit
changes a degree to which the captured image is transparent or an area in which the
10 captured image is displayed, as partial visualization of the captured image.
(43)
The image processing apparatus according to (33), wherein the image processing unit
changes a degree of smoothing to which the captured image is smoothed, as a change
in the degree of visualization.
15 (44)
The image processing apparatus according to any of (33) to (43), wherein the image
processing unit starts a process when a predetermined target has been recognized
from the captured image.
(45)
20 An image processing method including:
changing a degree of visualization of a captured image, for feeding back gesture
recognition to a user, to be displayed on a display screen, based on the captured
image or an operation target image to be operated according to a result of the gesture
recognition;
25 combining the captured image whose degree of visualization has been changed with
the operation target image; and
displaying a combined image on the display screen.
Reference Signs List
30 [Reference Signs List]
[0139]
100 Image processing apparatus
102 Comtnunication unit
104 Control unit
11 0 Image processing unit
5 112 Display control unit
CLAIMS
Claim 1
An information processing apparatus comprising:
a processing unit configured to control combining of a captured image and
5 an operation target image so as to generate a combined image for feeding back
gesture recognition to a user,
wherein a degree of visualization of the captured image appears to be
changed in the combined image.
10 Claim 2
The information processing apparatus according to claim 1, wherein the
degree of the visualization in part of the captured image appears to be changed in the
combined image.
15 Claim 3
The information processing apparatus according to claim 2, wherein the part
of the captured image corresponds to a region of the combined image having a
position determined according to a position in the operation target image of an
operation target object in the combined image.
20
Claim 4
The information processing apparatus according to claim 1, wherein the
degree of the visualization of the captured image is changed based on an operation
target object included in the operation target image.
25
Claim 5
The information processing apparatus according to claim 4,
wherein the degree of the visualization of the captured image includes a
decrease in a degree of visualization of a region of the captured image, and
30 wherein the region is determined according to a position of the operation
target object within the operation target image.
Claim 6
The information processing apparatus according to claim 1, wherein the
degree of the visualization of the captured image is changed based on a
5 predetermined target recognized from the captured image.
Claim 7
The information processing apparatus according to claim 6, wherein the
degree of the visualization of the captured image is changed by at least a size of a
10 part croppedfrom the captured image.
Claim 8
The information processing apparatus according to claim 6, wherein the
predetermined target is at least one of aface and a hand.
15
Claim 9
The information processing apparatus according to claim 1, wherein the
degree of the visualization of the captured image is changed so that the captured
image appears to be classified by rank in the combined image.
20
Claim 10
The information processing apparatus according to claim 1, wherein
changing the degree of the visualization of the captured image includes changing a
degree to which the captured image is transparent.
25
Claim 11
The information processing apparatus according to claim 10, wherein
changing the degree to which the captured image is transparent includes dynamically
changing the degree to which the captured image is transparent.
30
Claim 12
The information processing apparatus according to claim 1, wherein
changing the degree of the visualization of the captured image includes cropping the
captured image.
5 Claim 13
The information processing apparatus according to claim 12, wherein
changing the degree of the visualization of the captured image further includes
scaling the cropped image.
10 Claim 14
The information processing apparatus according to claim 1, wherein
combining a changed captured image with the operation target image includes
changing a display position of the changed captured image with respect to a display
position of the operation target image.
15
Claim 15
The information processing apparatus according to claim 1,
wherein the captured image includes an image of a plurality of users, and
wherein changing the degree of the visualization of the captured image
20 includes cropping the captured image so as to generate a region cropped for each of
the plurality of recognized users.
Claim 16
The information processing apparatus according to claim 1, further
25 comprising:
a display unit,
wherein the processing unit controls a display of the combined image.
Claim 17
30 The information processing apparatus according to claim 1, further
comprising:
an imaging unit.
Claim 18 -
The information processing apparatus according to claim 1, wherein the
5 operation target image is one of an image representing a web page, an image
representing a menu screen, and an image representing an operation screen.
Claim 19
An information processing method comprising the step of:
combining a captured image with an operation target image so as to generate
a combined image for feeding back gesture recognition to a user,
wherein a degree of visualization of the captured image appears to be
changed in the combined image.
Claim 20
A computer-readable recording medium recording a program for causing a
computer to execute a step of:
combining a captured image with an operation target image so as to generate
a combined image for feeding back gesture recognition to a user,
wherein a degree of visualization of the captured image appears tobe
changed in the combined image.