Technical Field
SP352945W000
[0001] The present technology relates to detection
apparatus and method and more particularly to detection
apparatus and method that can enhance the operability
1·1i th a simple configuration.
Background Art
[0002] For example, a user interface for moving a
cursor or pointer on a screen needs an operation means.
As a method of detecting a user's operation for moving
the cursor or the like, there are a method of detecting
a movement of an operation site of a user's arm, leg,
15 or finger that is captured by a camera based on a
position of the operation site in the image and a
method of detecting the movement based on a signal of a
gyro sensor attached to the user's arm, leg, or finger.
[0003] In the case where the user interface
20 operation for the cursor or pointer is performed using
the user's limb, finger, or the like as described
above, an external detector such as a camera and a gyro
sensor is necessary.
[0004] Furthermore, in the case 1·1here the operation
25 for moving the cursor or the pointer is performed using
the line of sight of the user, it is necessary to
SP352945WOOO
2
detect an eyeball movement of the user.
[0005] Examples of the method of measuring a
position or movement of the eyeball include a search
coil method using the fact that a potential is
5 generated in a coil placed in a magnetic field, the
potential being proportional to an angle formed by the
magnetic field and the coil. In the search coil method,
a detection coil is incorporated in a contact lens. It
is worn on an eyeball. A magnetic field coil that
10 applies horizontal and vertical magnetic fields is
externally placed. Induced electromotive force is
generated in the detection coil incorporated in the
contact lens due to the fields applied from the
outside. The induced electromotive force is detected.
15 In this manner, the eyeball movement is detected.
[0006] As another method of measuring the position
or movement of the eyeball, an EOG (Electrooculography)
method is also known. The EOG method uses the fact that
a cornea has a positive potential of from 10 to 30 ~V
20 in comparison with a retina. Electrodes are attached
around an eye. A potential difference is detected.
[0007] As other methods of measuring the position or
movement of the eyeball, a limbus tracking method, a
corneal reflection method, and a pupil corneal
25 reflection method are known.
[0008] The limbus tracking method uses the fact that
SP352945WOOO
3
the reflectance of infrared light emitted to an eye
differs between a sclera and iris and pupil. Light
reflected by an eyeball is captured by an externally
placed camera. An eyeball movement is thus detected.
5 [0009] Furthermore, the corneal reflection method
uses the fact that a virtual image on a cornea portion
that is formed of infrared LED light emitted to the eye
by an infrared light LED (Light Emitting Diode) is
moved in parallel along with an eyeball movement due to
10 a rotation center difference bet1-1een the cornea portion
and the eyeball. The virtual image of the infrared LED
light reflected by the eyeball is captured by the
externally placed camera. The eyeball movement is thus
detected.
15 [0010] The pupil corneal reflection method has the
same basic principles as the corneal reflection method.
However, the pupil corneal reflection method is
different from the corneal reflection method in that
the center of the pupil is used as a reference. That
20 is, the pupil corneal reflection method is a method in
which the center of the pupil is detected by the
externally placed camera and the eyeball movement is
detected based on a difference from the position of the
virtual image of the infrared LED light.
25 [0011] By the v1ay, as a compact image display
apparatus, a contact lens type display apparatus has
SP352945WOOO
4
been proposed (e.g., see Patent Document 1). This
display apparatus is used by being worn on an eyeball
of the user. Therefore, irrespective of "'here the user
is located, images can be presented to the user.
5 [0012) Patent Document 1: Japanese Patent No.
4752309
Summary of Invention
Problem to be solved by the Invention
[0013) Ho\•lever, v1hen the cursor or pointer is
10 displayed by the above-mentioned contact lens type
display apparatus and a user's operation is performed,
the user interface for moving the cursor or pointer
according.tothe above-mentioned method needs an
external detection apparatus for detecting the
15 operation means.
[0014) Worn on the eyeball of the user, the contact
lens type display apparatus is l·lirelessly used. The use
of the external detection apparatus for operating the
cursor or pointer becomes a burden because the user
20 that uses it has to carry the extra device.
[0015) For example, in accordance with the method of
detecting a movement captured by the camera, the user
has to be located in an angle of vie"' of the camera.
Thus, the activity range of the user is limited.
25 Therefore, it is difficult to take the display
apparatus outside. Furthermore, as an operation
5
10
SP352945WOOO
5
distance becomes longer, an operation area in a camera
screen becomes smaller. Therefore, the number of pixels
for detecting the movement of the user relatively
decreases. Thus, the detection accuracy is lowered.
[0016] In addition, the gyro sensor detects a
relative position. Therefore, in the method of
detecting the movement of the user by the gyro sensor,
it is necessary to specify a reference position for
every operation.
[0017] In the search coil method, the magnetic field
coil that applies the horizontal and vertical magnetic
fields should be externally placed. Furthermore, the
search coil method uses the electromotive force
generated by the detection coil being moved relative to
15 the magnetic field generated by the magnetic field
coil. Therefore, the position of the head of the user
has to be fixed ~o~ith respect to the magnetic field
coil.
[0018] The EOG method has a 1·1ide detection range and
20 is capable of detecting the eyeball movement even ~o~hen
the user closes eyes. Ho~o~ever, it is ~o~eak against
external electromagnetic noise and the detection
accuracy is low. Detection ~o~ith an accuracy of less
than 1 degree is impossible.
25 [0019] Regarding the limbus tracking method, the
corneal reflection method, and the pupil corneal
SP352945WOOO
6
reflection method, all of them have less burden on a
human body. However, these methods require the
externally placed camera. Furthermore, it is
susceptible to ambient light. Therefore, for increasing
5 the detection accuracy, it is necessary to prepare an
environment with less ambient light.
[0020] In addition, in the method of detecting the
eyeball movement by the camera capturing the user from
the outside, the eyeball cannot be detected when the
10 user closes eyes. Therefore, when the user is closing
eyes, the user interface cannot be operated.
[ 0021] As described above, in the above-mentioned
techniques, it has not been possible to enhance the
operability of the contact lens type display apparatus
15 \•lith a simple configuration without using the external
detection apparatus.
[0022] The present technology has been made in view
of the above-mentioned situation and it is an object to
enhance the operability with a simple configuration.
20 Means for solving the Problem
[0023] A detection apparatus according to an aspect
of the present technology is a detection apparatus that
is 1o1earable on an eyeball and includes a lightreceiving
element that receives light entering from the
25 eyeball.
[0024] The detection apparatus may be further
SP352945WOOO
7
provided with a light-emitting element that outputs
light. The light-receiving element may be provided near
the light-emitting element.
[0025] The light-emitting element may be formed of a
5 plurality of light-emitting sections, and the lightreceiving
element may be provided near the lightemitting
section.
[0026] The light-receiving element may receive light
that is output from the light-emitting section and
10 reflected by the eyeball. A signal processing unit that
detects light-receiving amounts of a plurality of
light-receiving elements arranged in regions of the
detection apparatus may be further provided.
[0027] The light-emitting section may be a display
15 pixel that displays information.
[0028] The detection apparatus may be configured to
cover an entire cornea portion when the detection
apparatus is worn on the eyeball.
[0029] In a state in 1·1hich the detection apparatus
20 is worn on the eyeball, at least one of the lightemitting
section and the light-receiving element may be
provided in a region of the detection apparatus that is
opposed to a region in a range in which a pupil of the
eyeball is movable.
25 [0030] A lateral width may be set to be larger than
a vertical width by 1•1hich the detection apparatus
SP352945~1000
8
covers the eyeball.
[0031] An element different from the light-emitting
element and the light-receiving element may be provided
near a lateral end of the detection apparatus.
5 [0032] The detection apparatus may have a structure
for fixing the detection apparatus with respect to a
head having the eyeball.
[0033] The signal processing unit may determine a
direction of the eyeball based on the light-receiving
10 amounts of the plurality of light-receiving elements.
[0034] The signal processing unit may calculate a
convergence amount of left and right eyes based on the
direction of the eyeball and a direction of an eyeball
that pairs with the eyeball and calculate a distance to
15 a gazed target object based on the convergence amount.
[0035] The signal processing unit may determine a
diameter of a pupil of the eyeball based on the lightreceiving
amounts of the plurality of light-receiving
elements.
20 [ 003 6] The signal processing unit may detect a
living-body state based on the light-receiving amounts
of the plurality of light-receiving elements.
[0037] The light-emitting section may emit light
having a predetermined 1·1avelength to the eyeball or
25 emit light having different wavelengths to the eyeball
in order, and the signal processing unit may detect the
SP352945WOOO
9
living-body state based on light-receiving amounts of
the light having the predetermined 1·1avelength or the
light having the different wavelengths that is emitted
to the eyeball, in the light-receiving elements.
5 [0038] The light-emitting section may be a display
pixel that displays information. The light-emitting
section may emit, after a period in which the
information is displayed, the light having the
predetermined wavelength or the light having the
10 different wavelengths to the eyeball,
[ 0039] A detection method according to an aspect of
the present technology is a detection method for a
detection apparatus including a light-receiving element
that receives light entering from an eyeball, and a
15 signal processing unit that detects a light-receiving
amount of the light-receiving element and being
'"ear able on the eyeball, the method including: a lightreceiving
step of receiving, by the light-receiving
element, light reflected by the eyeball; and a
20 detection step of detecting, by the signal processing
unit, light-receiving amounts of a plurality of lightreceiving
elements arranged in regions of the detection
apparatus,
[0040] The detection method may further include a
25 light-emitting step of outputting light by a lightemitting
element provided in the detection apparatus.
SP352945WOOO
10
The light-receiving element may receive, in the lightreceiving
step, light output from the light-emitting
element and reflected by the eyeball.
[0041] The detection method may further include a
5 calculation step of determining, by the signal
processing unit, a direction of the eyeball based on
the light-receiving amounts of the plurality of lightreceiving
elements.
[0042] In the detection method, the signal
10 processing unit may calculate, in the calculation step,
a convergence amount of left and right eyes based on
the direction of the eyeball and a direction of an
eyeball that pairs >lith the eyeball and calculate a
distance to a gazed target object based on the
15 convergence amount.
[0043] In an aspect of the present technology, in
the detection apparatus that includes the lightreceiving
element that receives light entering from the
eyeball and is 1·1earable on the eyeball, light reflected
20 by the eyeball is received by the light-receiving
element.
Effects of the Invention
[0044] According to an aspect of the present
technology, it is possible to enhance the operability
25 "ith a simple configuration.
Brief Description of Dra1·lings
5
SP352945WOOO
11
[0045] [Fig. 1] A diagram shm1ing a configuration
example of the outer appearance of a display apparatus.
[Fig. 2] A diagram showing a configuration
example of the display apparatus.
[Fig. 3] A diagram showing a configuration
example of a display region.
[Fig. 4] A diagram for describing detection
of a line of sight.
[Fig. 5] A diagram for describing detection
10 of a line of sight.
15
[Fig. 6] A diagram for describing detection
of a gaze position.
[Fig. 7] A diagram for describing processing
by a control apparatus.
[Fig. 8] A diagram shm1ing pulsations of a
heart.
[Fig. 9] A diagram showing an eyeball.
[Fig. 10] A diagram for describing a movement
range of a pupil.
20 [Fig. 11] A flowchart for describing
calibration processing.
[Fig. 12] A flowchart for describing line-ofsight
detection processing.
[Fig. 13] A flowchart for describing living-
25 body state detection processing.
[Fig. 14] A diagram showing of another
SP352945WOOO
12
configuration example of the display apparatus.
[Fig. 15] A diagram shoHing a Hearing state
of the display apparatus.
[Fig. 16] A diagram for describing convex
5 portions of the display apparatus.
[Fig. 17] A diagram for describing highfriction
portions of the display apparatus.
[Fig. 18] A diagram for describing regions of
the convex portions or high-friction portions of the
10 display apparatus.
[Fig. 19] A diagram showing another
configuration example of the display region.
[Fig. 20] A diagram showing another
configuration example of the display region.
15 [Fig. 21] A diagram showing another
configuration example of the display region.
[Fig. 22] A diagram sho~1ing another
configuration example of the display apparatus.
[Fig. 23] A diagram sho~1ing a configuration
20 example of a line-of-sight detection apparatus.
[Fig. 24] A diagram showing a configuration
example of a detection region.
[Fig. 25] A flowchart for describing
calibration processing.
25 [Fig. 26] A flowchart for describing line-ofsight
detection processing.
SP352945WOOO
13
Mode(s) for Carrying Out the Invention
[0046] Hereinafter, embodiments to 1·1hich the present
technology is applied will be described with reference
to the drawings.
5 [0047]

The present technology relates to a contact lens
type display apparatus. The contact lens type display
10 apparatus is wirelessly used by being worn on an
eyeball of the user. Therefore, l'lhen it is used for the
function as the display apparatus, the user can, for
example, freely l'lalk around ~1hile 1·1earing the display
apparatus. Hol'lever, performing, by an external
15 apparatus such as a camera and a detection apparatus,
an operation of selecting/moving a cursor, pointer, or
the like ~lith respect to information in a displayed
screen imposes a burden or limitation on the user.
[0048] In viel'l of this, in the present technology,
20 light-receiving elements are provided near a display
element that displays an image. Thus, a user interface
for operating a cursor, pointer, or the like can be
realized ~1ithout needing no external apparatuses other
than the display apparatus.
25 [0049] In the contact lens type display apparatus,
reflected light on an eyeball surface of light emitted
SP352945WOOO
14
by the display element is detected by the lightreceiving
elements. In this case, reflected light is
detected in a sclera or iris of the eyeball surface. In
contrast, in a pupil portion, light passes through the
5 eyeball and reflected light thereof is less. Thus, the
portion 1·1ith less reflected light is detected as the
pupil. Based on the detected pupil movement, a line of
sight is detected.
[0050) With this, it becomes possible to determine a
10 direction in which the eyeball is oriented. Thus, the
user interface for operating the cursor, pointer, or
the like can be provided 1·1i thout using the external
apparatus. Thus, it is possible to enhance the
operability of the display apparatus 1·1ith a simple
15 configuration.
[0051) Next, a specific embodiment of the contact
lens type display apparatus to v1hich the present
technology is applied will be described.
[0052) The contact lens type display apparatus is
20 worn on an eyeball of the user as shown in Fig. 1.
[0053) In Fig. 1, a contact lens type display
apparatus 11 is v10rn on a surface of an eyeball EY11 of
the user. The display apparatus 11 has such a shape
that it can be worn and removed on/from the eyeball
25 EY11 of the user like a so-called contact lens.
[0054) Such a display apparatus 11 is, for example,
SP352945WOOO
15
configured as shown in Fig. 2.
[0055] Specifically, the display apparatus 11 is
constituted of a display region 21, a feeding antenna
22, a signal antenna 23, a power generation unit 24, a
5 sensor 25, a signal processing unit 26, and a display
element drive unit 27.
[0056] Note that Fig. 2 is a diagram of the display
apparatus 11 as viewed in a left-to-right direction in
Fig. 1. That is, it is a diagram as the user wearing
10 the display apparatus 11 is viev1ed from the front. In
Fig. 2, the display apparatus 11 has a circular shape.
[0057] The display region 21 includes a display
element and light-receiving elements. The display
element is formed of a plurality of display pixels that
15 display information such as an image and a character
presented to the user. The light-receiving elements are
arranged adjacent to the display pixels and receive
light reflected by the eyeball surface of the user.
[0058] The feeding antenna 22 is provided
20 surrounding the display region 21 and receives induced
electromotive force due to a magnetic field or electric
field supplied from the outside. The signal antenna 23
transmits information supplied from the signal
processing unit 26, such as a result of the operation
25 of the user interface based on the line of sight of the
user, to the outside. The signal antenna 23 receives
SP352945WOOO
16
information transmitted from the outside, such as
information displayed by the display pixels, and
supplies it to the signal processing unit 26.
[0059] The p01-1er generation unit 24 rectifies an
5 induced current generated in the feeding antenna 22 due
to electromagnetic induction by the magnetic field or
the like from the outside, to thereby obtain and store
electric power and supplies the electric power to the
respective sections of the display apparatus 11. Note
10 that, in the case 1-1here the po1-1er generation unit 24
generates po1-1er by itself according to a predetermined
method or includes a rechargeable battery, the display
apparatus 11 does not need to be provided with the
feeding antenna 22.
15 [0060] The sensor 25 is formed of a gyro sensor, a
gravity sensor, or the like. The sensor 25 detects the
posture or movement of the user wearing the display
apparatus 11 and supplies a detection result thereof to
the signal processing unit 26. For example, the
20 movement of the head of the user is detected by the
sensor 25.
[0061] The signal processing unit 26 controls the
entire display apparatus 11. For example, based on
signals supplied from the light-receiving elements of
25 the display region 21, the signal processing unit 26
detects a difference bet1-1een light-receiving amounts of
SP352945WOOO
17
light in the light-receiving elements arranged in the
regions of the display apparatus 11, to thereby detect
the line of sight of the user. Furthermore, based on
the detection result supplied from the sensor 25, the
5 detection result of the line of sight, the information
received by the signal antenna 23, and the like, the
signal processing unit 26 controls the display element
drive unit 27 to display an image or the like in the
display region 21.
10 [0062] Specifically, for example, when the display
apparatus 11 is rotated relative to the eyeball of the
user, the sensor 25 is capable of detecting rotation
direction and rotation amount thereof. In vie1·1 of this,
the signal processing unit 26 controls the display
15 element drive unit 27 to rotate the image currently
displayed in the display region 21 to a direction
opposite to the rotation direction of the display
apparatus 11 relative to the eyeball, which is supplied
from the sensor 25, by an amount corresponding to the
20 rotation amount of the display apparatus 11. \'lith this,
even if the display apparatus 11 is rotated on the
eyeball of the user, the resulting rotation of the
image can be corrected and the image easy to vie1·1 can
be presented to the user.
25 [0063] The display element drive unit 27 drives the
display element of the display region 21 under the
SP352945WOOO
18
control of the signal processing unit 26 to display an
image or to supply the signals, l'lhich are supplied from
the light-receiving elements of the display region 21,
to the signal processing unit 26. Hereinafter, the
5 signals according to the light-receiving amounts of the
light-receiving elements, which are output from the
light-receiving elements of the display region 21, 1·rill
be referred to as light-receiving signals.
[0064] The display region 21 of the display
10 apparatus 11 is, for example, configured as shovm in
Fig. 3. Note that Fig. 3 shows a part of a crosssection
of the display apparatus 11 as the display
apparatus 11 is vievred in a depth direction in Fig. 1.
[0065] In Fig. 3, the display region 21 of the
15 display apparatus 11 includes display pixels 51-1 to
51-7 that display information such as an image and
light-receiving elements 52-1 to 52-7 that receive
reflected light entering from the eyeball surface of
the user. A single display device formed of the display
20 pixels 51-1 to 51-7 is a display element 53.
[0066] Hereinafter, in the case vrhere the display
pixels 51-1 to 51-7 do not particularly need to be
distinguished from one another, they will be also
simply referred to as display pixels 51. Furthermore,
25 in the case vrhere the light-receiving elements 52-1 to
52-7 do not particularly need to be distinguished from
SP352945WOOO
19
one another, they 1·1ill be also simply referred to as
light-receiving elements 52.
[ 0 0 67] The display element 53 is constituted of, for
example, a liquid-crystal display element or organic
5 electroluminescence (OLEO (Organic Light Emitting
Diode)) display element. In the example in Fig. 3, the
display pixels 51 and the light-receiving elements 52
are alternately arranged in a vertical direction on a
right side of the display apparatus 11, that is, the
10 side of the eyeball of the user in the figure. Thus,
for example, in Fig. 2, the display pixels 51 and the
light-receiving elements 52 are alternately arranged in
vertical and lateral directions in Fig. 2 in the
display region 21.
15 [0068] A lubricant layer 54 is provided on a left
side of the display pixels 51 and the light-receiving
elements 52 in the display apparatus 11 in the figure,
that is, an external 1wrld side of the display
apparatus 11. The lubricant layer 54 is, for example,
20 formed of a transparent synthetic resin. Due to the
lubricant layer 54, an eyelid of the user can smoothly
move when the user 1·1ears the display apparatus 11 on
the eye.
[0069] Referring to Fig. 3, the example in which the
25 display pixels 51 and the light-receiving elements 52
are in close contact has been described. However, the
SP352945W000
20
display pixels 51 and the light-receiving elements 52
do not necessarily need to be in close contact and a
clearance may be provided between the display pixels 51
and the light-receiving elements 52. Furthermore, in
5 Fig. 3, one light-receiving element 52 is provided for
one display pixel 51. However, one light-receiving
element 52 may be provided for a plurality of display
pixels 51.
[0070]
10 Next, user's line-of-sight detection by the
display apparatus 11 v1ill be described.
[ 0071] For example, as shown in Fig. 4, it is
assumed that the display apparatus 11 worn on the
eyeball EY11 of the user is provided with display
15 pixels 51-1 to 51-11 and light-receiving elements 52-1
to 52-12. Furthermore, a region of the display region
21 of the display apparatus 11 that is opposed to a
portion of a site of the eyeball EY11 of the user,
1;hich is different from a pupil BEll, for example, a
20 sclera or iris will be referred to as a region A. A
region of the display region 21 that is opposed to the
pupil BEll will be referred to as a region B.
[0072] When the display pixels 51 located in the
regions A and B emit light, light emitted from the
25 display pixels 51 travels to an eyeball EYll and
arrives at the eyeball EY11 as indicated by the solidSP352945WOOO
21
line arrow marks in the figure.
[0073] For example, some of the light output from
the display pixels 51, which have entered an opaque
site such as a sclera and an iris in the eyeball EYll,
5 are absorbed and reflected by a surface of the eyeball
EYll. Therefore, in the region A, some of the light
output from the display pixels 51 are, as indicated by
the dotted-line arroH marks, reflected by the surface
of the eyeball EYll and received (detected) by the
10 light-receiving elements 52.
[ 007 4] In contrast, the pupil BEll is transparent,
and hence some of the light output from the display
pixels 51, ~1hich have entered the pupil BEll, are
hardly reflected by the pupil BEll but arrive at the
15 retina in the eyeball EYll and are absorbed by the
retina. That is, as indicated by the solid-line arr01·1
marks in the figure, in the region B, the light outp0t
from the display pixels 51 is hardly reflected by the
surface of the eyeball EYll but absorbed by the retina.
20 Therefore, in the region B, the light output from the
display pixels 51 is hardly detected by the lightreceiving
elements 52.
[0075] In this manner, by detecting a difference
bet\o~een amounts of light from the display pixels 51
25 that is received by the light-receiving elements 52, a
·direction of the eyeball EYll indicating the direction
SP352945WOOO
22
in Hhich the eyeball EYll (pupil BEll) is oriented,
that is, the user's line-of-sight direction can be
determined. In particular, as long as the user's lineof-
sight direction at each point of time can be
5 determined, it is possible to detect an eyeball
movement, that is, a movement of the line of sight and
also to estimate mental state or feelings of the user
based on the movement of the line of sight.
[0076] Strictly speaking, little reflection occurs
10 in all surfaces of the eyeball EYll. Ho~
In the display apparatus 11, it is also possible
to determine, based on the light-receiving signal map,
25 a distance to the target object gazed by the user.
[0091) Now, referring to Fig. 6, a principle of
SP352945\'IOOO
27
detecting a distance to a gaze position based on
directions of left and right eyeballs of the user will
be described.
[0092] For example, as shm·m in Fig. 6, it is
5 assumed that the user wears a display apparatus DPllL
on a left eyeball EY21L (left eye) and a display
apparatus DPllR on a right eyeball EY21R (right eye)
and gazes a predetermined target object OBll or OB12.
[0093] Here, the display apparatuses DPllL and DPllR
10 are apparatuses each equivalent to the display
apparatus 11.
[0094] In Fig. 6, the target object OBll is located
at a gaze position ATll and the target object OB12 is
located at a gaze position AT12. A distance from the
15 user to the gaze position ATll is longer than a
distance from the user to the gaze position AT12. That
is, the gaze position ATll is further from the user
than the gaze position AT12.
[0095] In general, 1·1hen a person views a certain
20 object, the left and right eyes have a convergence
depending on a distance to the object. That is, the
left and right eyeballs rotate inward and rotation
angles thereof change depending on the distance to the
gazed object.
25 [ 0096] For example, in the example shown on the left
side in the figure, the user gazes the target object
SP352945WOOO
28
OB11 at the gaze position AT11 approximately in front
of the user. In this example, an angle formed by a
straight line linking a center of the pupil BE21L of
the left eyeball EY21L of the user and the target
5 object OB11 and a straight line linking a center of the
right eyeball EY21R of the pupil BE21R and the target
object 0811 is a convergence angle of the user viewing
the target object OB11. The convergence angle indicates
a convergence amount of the left and right eyes of the
10 user.
[0097] When the user is vievling the target object
OB11 in the above-mentioned manner, a light-receiving
signal map RM11L for the left eye is obtained by the
display apparatus DP11L worn on the left eyeball EY21L
15 and a light-receiving signal map RM11R for the right
eye is obtained by the display apparatus DP11R worn on
the right eyeball EY21R.
[0098] As can be seen from the light-receiving
signal map RM11L and the light-receiving signal map
20 RM11R, the left and right pupils BE21L and BE21R of the
user are oriented slightly inward as viewed from the
user.
[0099] Furthermore, for example, in the example
shovm on the right side in the figure, the user gazes
25 the target object OB12 at the gaze position AT12
approximately in front of the user. In this example, an
____ j
~j
SP352945WOOO
29
angle formed by a straight line linking a center of the
pupil BE21L of the left eyeball EY21L of the user and
the target object OB12 and a straight line linking a
center of the pupil BE21R of the right eyeball EY21R
5 and the target object OB12 is a convergence angle of
the user viewing the target object OB12.
[0100] When the user is viewing the target object
OB12 in the above-mentioned manner, a light-receiving
signal map RM12L for the left eye is obtained by the
10 display apparatus DP11L 1-10rn on the left eyeball EY21L
and a light-receiving signal map RM12R for the right
eye is obtained by the display apparatus DP11R worn on
the right eyeball EY21R.
[0101] As can be seen from the example shm·m in Fig.
15 6, as the distance from the user to the gazed target
object becomes shorter, the convergence angle of the
user viewing the target object becomes larger. For
example, in the example in Fig. 6, the convergence
angle in viewing the target object OB12 is larger than
20 the convergence angle in viewing the target object 0811.
[0102] Furthermore, along v1ith this change in
convergence angle, the user's pupil position (line-ofsight
position) in the light-receiving signal map also
changes. In the example in Fig. 6, it can be seen that,
25 in the light-receiving signal map, the pupil position
in vie1·1ing the target object OB12 is located on an
"'I
SP352945WOOO
30
inner side of the user (center side of user) in
comparison with the pupil position in vie1·1ing the
target object OB11.
[0103] In the display apparatus 11, the convergence
5 angle of the left and right eyes of the user can be
calculated based on the directions of the left and
right eyeballs paired that is obtained based on a
detection result of light at the light-receiving
elements 52, that is, the pupil position in the light-
10 receiving signal map. Based on the obtained convergence
angle, the distance to the gazed object and upper,
l01·1er, left, and right positions of the gazed object
can be determined.
[0104] If the distance to the gazed object can be
15 determined in this manner, not only the position in
left and right directions but also the position in the
depth direction can be distinguished. Therefore, for
example, t•hen images, buttons, or the like having a
parallax are displayed to the left and right eyes, an
20 operation Hith a depth feeling can be realized.
[0105] In the display apparatus DP11L and the
display apparatus DP11R, the convergence angle may be
calculated using only the light-receiving signal map
obtained based on the light-receiving signals of their
25 o1<1n light-receiving elements 52. Alternatively, the
convergence angle may be calculated using a light.
I
9
SP352945WOOO
31
receiving signal map for the left and right eyes.
[0106] In the case where the convergence angle is
calculated based oh the light-receiving signal map for
the left and right eyes, for example, the display
5 apparatus DPllL communicates Hith the display apparatus
DPllR and receives the light-receiving signal map
obtained by the display apparatus DPllR. Then, the
display apparatus DPllL calculates a convergence angle
based on the light-receiving signal map obtained by the
10 display apparatus DPllL and the light-receiving signal
map received from the display apparatus DPllR and
transmits the obtained convergence angle to the display
apparatus· DPllR. In the calculation of the convergence
angle, an eyeball direction indicating the pupil
15 position in the light-receiving signal map for the left
eye and an eyeball direction indicating the pupil
position in the light-receiving signal map for the
right eye is calculated.
[0107] With this, in the display apparatus DPllL and
20 the display apparatus DPllR, the common convergence
angle can be obtained. In this case, the convergence
angle is calculated by the signal processing unit 26
and the convergence angle is transmitted/received by
the signal antenna 23. Furthermore, the signal
25 processing unit 26 determines the distance to the gazed
object and the upper, lo"1er, left, and right positions
SP352945WOOO
32
of the gazed object using the convergence angle
depending on needs. The distance to the gazed object
and the like may be transmitted from the display
apparatus DPllL to the display apparatus DPllR.
5 [0108) In addition, in the case 1·1here each of the
left and right display apparatuses DPllL and DPllR
calculates the convergence angle based on a single
light-receiving signal map, the convergence angle is
calculated based on the pupil position in the light-
10 receiving signal map for the left or right eye.
Therefore, in the case where the target object is not
in front of the user, different convergence angles are
obtained by the display apparatuses DPllL and DPllR.
That is, left and right asymmetrical convergence angles
15 are obtained. Hov~ever, based on the left and right
convergence angles, it is possible to determine the
distance to the target object or the position in the
left, right, upper, and lower directions of the target
object.
20 [0109) Note that another apparatus may calculate a
convergence angle based on the light-receiving signal
map obtained by the display apparatus DPllL and the
light-receiving signal map obtained by the display
apparatus DPllR.
25 [0110) In such a case, for example, as shown in Fig.
7, a control apparatus 81 communicates with the display
SP352945W000
33
apparatuses DP11L and DP11R and receives the lightreceiving
signal maps.
[0111] Then, the control apparatus 81 calculates a
convergence angle based on the light-receiving signal
5 map received from the display apparatus DP11L and the
light-receiving signal map received from the display
apparatus DP11R and transmits the obtained convergence
angle to the display apparatuses DP11L and DP11R.
[0112] Note that the control apparatus 81 may be
10 configured to calculate the distance to the target
object or the position or the like of the target object
based on the convergence angle and transmit it to the
display apparatuses DP11L and DP11R.
[0113] As described above, according to the display
15 apparatus 11, it is possible to directly receive
reflected light from the eyeball and detect the eyeball
movement, that is, the eyeball direction (line-of-sight
position) at each point of time. In particular, due to
the provision of the plurality of light-receiving
20 elements 52 in the display region 21, it becomes
possible to accurately detect little movements of the
eyeball.
[0114]
By the 1·1ay, it is known that eyeball movements
25 include little movements called saccades. In particular,
the largest movements of involuntary eyeball movements
;i ;--;
SP352945WOOO
34
that occur during visual fixation are called
microsaccades.
[0115] Instead of being random, these little ocular
shifts may point to where mind of a person is secretly
5 focusing even if his/her gaze is directed else1·1here
revealing hidden thoughts and desires.
[0116] Therefore, if the display apparatus 11
detects microsaccades, it becomes possible to determine
not only a target object gazed by the user but also a
10 target object in l·lhich the user is potentially
interested.
[0117] Specifically, for example, the signal
processing unit 26 detects, based on the lightreceiving
signal map at each point of time, an eyeball
15 direction, that is, a line-of-sight position at each
point of time. Then, if the same line-of-sight position
is detected at most points of time in a predetermined
period, this line-of-sight position is a position gazed
by the user, that is, a position of the target object
20 v1hich the user focuses on.
[0118] Furthermore, the signal processing unit 26
detects one of the line-of-sight positions different
from the position gazed by the user, which is obtained
at a point of time at which the movement of the eyeball
25 direction is largest in the above-mentioned
predetermined period, as a position of the target
,--!
""1
SP352945WOOO
35
object in which the user is potentially interested.
That is, the largest eyeball movement is detected as a
microsaccade.
[0119)
5 The display apparatus 11 is also capable of
detecting a living-body state.
[0120) For example, the display apparatus 11 is
capable of de.tecting pulsations of the heart of the
user as the living-body state. Hereinafter, a principle
10 of detecting the pulsations will be described.
[0121) In the display apparatus 11, the display
pixels 51 output light having a predetermined
1·1avelength and light reflected by the eyeball surface
is received by the light-receiving elements 52. Then,
15 the signal processing unit 26 detects pulsations of the
heart of the user wearing the display apparatus 11
based on values of the light-receiving signals supplied
from the light-receiving elements 52 via the display
element drive unit 27.
20 [0122) For example, the pulsations of the heart are
periodically generated as shown in Fig. 8. A pulsation
time is short with respect to a cycle and a blood flow
is generated at a pulsation timing. Note that, in Fig.
8, the horizontal axis indicates a time and the
25 vertical axis indicates a value of a light-receiving
signal, that is, a blood flow rate.
;j
SP352945WOOO
36
[0123] As can be seen from Fig. 8, portions at 1·1hich
the blood flm·r rate shapely changes are pulsation
portions and the pulsations are periodically generated.
[0124] At a timing at vrhich the blood flow rate is
5 increased due to a pulsation, the blood flowing through
capillary vessels increases. Thus, it is possible to
detect a pulsation based on the presence/absence of a
blood flow. The capillary vessels extend through the
eyeball and have blood flovrs depending on pulsations of
10 the heart.
[0125] Oxyhemoglobin and deoxyhemoglobin that are
components in blood have different spectral
characteristics of light absorption. The
deoxyhemoglobin has a high absorption coefficient with
15 a wavelength shorter than 805 nm. The oxyhemoglobin has
a high absorption coefficient with a-wavelength longer
than 805 nm.
[0126] In view of this, the signal processing unit
26 controls the display element drive unit 27 to output,
20 from the display pixels 51, light having a
predetermined wavelength shorter than the wavelength of
805 nm and light having a predetermined wavelength
longer than the l·ravelength of 805 nm in order
(alternately). Furthermore, the signal processing unit
25 26 causes the light-receiving elements 52 to receive
light output from the display pixels 51 and reflected
SP352945WOOO
37
by the eyeball surface. Here, the light having a
wavelength shorter than 805 nm may be visible light.
[0127] Then, the signal processing unit 26
determines a difference betl-1een a value of a light-
5 receiving signal obtained when the short-vlavelength
light is output and a value of a light-receiving signal
obtained when the long-1·1avelength light is output, to
thereby determine which component of oxyhemoglobin and
deoxyhemoglobin is more contained in the blood. In
10 addition, the signal processing unit 26 detects a blood
flow (change in blood flow rate) based on a
determination result obtained based on the difference
between the light-receiving signals and changes in the
values of the light-receiving signals at respective
15 points of time in a predetermined period, that is,
variations over time in the intensity of reflected
light received by the light-receiving elements 52. Then,
the signal processing unit 26 determines a pulsation
based on the detection result of the blood flov1.
20 [0128] Hemoglobin in the blood has a strong
absorption spectrum ~
By the way, the display apparatus 11 has a
structure such that it covers an area larger than the
cornea portion of the eyeball shown in Fig. 9. In Fig.
5 9, a hashed portion of an eyeball EY31 expresses a
cornea portion COll.
[0138] The display apparatus 11 has size and shape
such that it covers the entire cornea portion COll when
the display apparatus 11 is worn on the eyeball EY31.
10 [0139] For example, as shown on the upper side in
Fig. 10, in the state in 11hich the user faces for11ard,
the pupil BEll of the eyeball EYll is also oriented
fonvard. Note that, in Fig. 10, portions corresponding
to those in Fig. 4 will be denoted by the same
15 reference symbols and descriptions thereof 1vill be
appropriately omitted.
[0140] In the example shmm on the upper side in Fig.
10, the pupil BEll is oriented forv1ard and light output
from the display pixels 51 of the display apparatus 11
20 travels to the eyeball EYll.
[0141] Note that solid-line arrow marks in the
figure indicate some of the light output from the
display pixels 51, 1·1hich pass through the pupil BEll
and arrive at the retina. Dotted-line arrow marks in
25 the figure indicate light beams when some of the light
output from the display pixels 51, 1·1hich do not pass
d
'I
SP352945WOOO
41
through the pupil BEll but are absorbed or reflected by
the eyeball EYll, arrive at the retina Hithout being
absorbed and reflected by the surface of the eyeball
EYll, for example.
5 [0142] As can be seen from the example on the upper
side in the figure, light output from the display
pixels 51 in a region of the display apparatus 11 that
is opposed to the pupil BEll arrives at the retina and
light output from the display pixels 51 in a region
10 opposed to a portion different from the pupil BEll does
not arrive at the retina.
[0143] Furthermore, ori the lov1er side in the figure,
shoHn is the state of the eyeball EYll 1·1hen the user
looks upward. In this example, in comparison 1·1i th the
15 example on the upper side in the figure, the pupil BEll
moves toward an end (circumference) of the display
apparatus 11 and light output from the display pixels
51 near an upper end of the display apparatus 11 passes
through the pupil BEll and arrives at the retina.
20 Furthermore, light output from the display pixels 51
near the center of the display apparatus 11 does not
pass through the pupil BEll and is reflected or
absorbed on/in the surface of the eyeball EYll.
[0144] In the display apparatus 11, the display
25 pixels 51 and the light-receiving elements 52 are
provided in an entire region of the entire region of
,j
1
SP352945WOOO
42
the display apparatus 11, 1·1hich is opposed to a region
in a range in 1·1hich the pupil BEll is movable along
1·1i th a movement of the eyeball EY11 as shown in Fig. 10,
more specifically, a range in which the pupil BEll
5 moves in daily life. In other words, irrespective of
which direction the eyeball EY11 is oriented, the
display pixels 51 and the light-receiving elements 52
are arranged near the region of the display apparatus
11 that is opposed to the pupil BEll.
10 [0145] With this, even if the eyeball EYll is moved
in the user interface operation, it becomes possible to
prevent reduction of information due to lack of
information displayed by the display pixels 51. That is,
irrespective of l'lhich direction the line of sight of
15 the user is oriented, it is possible to present the
information to the user by the display pixels 51 and
detect a user's line-of-sight position by the lightreceiving
elements 52. In this case, light for
displaying an image is emitted from a full field of the
20 user and the image is displayed in an entire visual
field of the user.
[0146] Note that, in a region of a l'lider range
(movement range of pupil) in comparison \'lith the center
of the display apparatus 11, at least the display
25 pixels 51 or the light-receiving elements 52 only need
to be provided. It is not necessarily necessary to
SP352945W000
43
provide the display pixels 51 and the light-receiving
elements 52 in an entire range in which the eyeball
moves.
[0147]
5 Next, operations of the display apparatus 11 will
be described.
[0148] For example, for the user interface operation
using an eyeball movement due to a movement of the line
of sight of the user, the display apparatus 11 can
10 perform calibration processing, to thereby accurately
correct a positional relationship bet•1een the user's
line-of-sight position and the position of information
displayed in the display region 21.
[0149] Hereinafter, referring to a flowchart in Fig.
15 11, calibration processing by the display apparatus 11
will be described. The calibration processing is
started, for example, 1·1hen the user wears the contact
lens type display apparatus 11 on the eyeball.
[0150] In Step Sll, the signal processing unit 26
20 controls the display element drive unit 27 to cause the
display pixels 51 to emit light. The display pixels 51
emit light under the control of the display element
drive unit 27 and output light for displaying a
predetermined image.
25 [0151] In Step S12, the light-receiving elements 52
start detection of light entering from the eyeball.
SP352945WOOO
44
That is, the light-receiving elements 52 receive light
that has entered the eyeball from the outside of the
display apparatus 11 or the display pixels 51 and been
reflected by the eyeball surface. Then, the light-
5 receiving elements 52 perform photoelectric conversion
and supplies light-receiving signals according to
light-receiving amounts to the signal processing unit
26 via the display element drive unit 27.
[0152) In Step Sl3, the signal processing unit 26
10 controls the display element drive unit 27 to cause the
display pixels 51 to display a calibration positioning
image. The display pixels 51 emit light under the
control of the display element drive unit 27, to
thereby display the calibration positioning image.
15 [0153) For example, an image or the like of a mark
for calibration is used as the calibration positioning
image. Calibration positioning images are displayed at
five positions of the center and upper, lower, left,
and right positions of the display region 21 in order.
20 [0154) In Step Sl3, the signal processing unit 26
selects a position in which the calibration positioning
image is not yet displayed from among the center and
the upper, lower, left, and right positions and
displays the calibration positioning image at the
25 selected position.
[0155) Furthermore, together with the calibration
SP352945WOOO
45
positioning images, a message for prompting the user to
vie1·1 the calibration positioning images and perform an
operation for determining the position may be displayed
in the display region 21 depending on needs.
5 [0156] When the calibration positioning images are
displayed, light for displaying the calibration
positioning images is output from the display pixels 51
and some of the light are reflected by the eyeball
surface and received by the light-receiving elements 52.
10 Then, the light-receiving elements 52 supply the lightreceiving
signals according to the amounts of light
received to the signal processing unit 26 via the
display element drive unit 27.
[0157] Furthermore, the user looks at the
15 calibration positioning image and performs a position
determination operation, for example, gazing the
calibration positioning image for a predetermined time
or longer or blinking.
[0158] In Step S14, the signal processing unit 26
20 determines a user's line-of-sight position based on the
light-receiving signals supplied from the lightreceiving
elements 52.
[0159] For example, if the motion of gazing the same
position for a predetermined time or longer is set as
25 the position determination operation performed by the
user, the signal processing unit 26 generates a light5
SP352945WOOO
46
receiving signal map based on light-receiving signals
and determines a user's line-of-sight position at each
point of time based on the obtained light-receiving
signal map.
[0160] Then, the signal processing unit 26 sets the
same one of the determined line-of-sight positions at
the points of time, which is successively detected for
a predetermined time or longer, as a line-of-sight
position v1ith respect to the calibration positioning
10 image. That is, the set line-of-sight position is
considered as a line-of-sight position when the user
vie1-1s the calibration positioning image.
[0161] Furthermore, for example, if the blinking
motion is set as the position determination operation
15 performed by the user, the signal processing unit 26
generates a light-receiving signal map based on lightreceiving
signals, detects user's blinking based on the
light-receiving signal map at each point of time, and
determines a user's line-of-sight position at each
20 point of time.
25
[0162] Then, the signal processing unit 26 sets the
user's line-of-sight position at the point of time 1·1hen
the blinking is detected, as the line-of-sight position
vlith respect to the calibration positioning image.
[ 0163 J The blinking detection is performed based on,
for example, light intensity detected by the lightSP352945WOOO
47
receiving elements 52, that is, a value (value of
light-receiving signal) at each position of the lightreceiving
signal map.
[0164] When the user opens the eyelid, light
5 received by the light-receiving elements 52 includes
ambient light in addition to light from the display
pixels 51. Therefore, between the state in which the
user opens the eyelid and the state in which the user
closes the eyelid, the light intensity received by the
10 light-receiving elements 52 differ. Therefore, the
user's blinking can be detected based on a change in a
light amount level detected by the light-receiving
elements 52, that is, the value of the light-receiving
signal. Note that, by considering a temporal change in
15 addition to the change in the light amount level, it is
possible to further enhance the detection accuracy of
the blinking.
[0165] In Step Sl5, the signal processing unit 26
determines whether or not the processing has been
20 performed with respect to all the positions. For
example, if calibration positioning images are
displayed at the center and the upper, lower, left, and
right positions of the display apparatus 11 and the
line-of-sight position is determined for each of the
25 positions, it is determined that the processing has
been performed with respect to all the positions.
SP352945WOOO
48
[0166] In Step S15, if the processing has not yet
been performed with respect to all the positions, the
processing returns to Step S13 and the above-mentioned
processing is repeated. That is, the calibration
5 positioning image is displayed at a next position and a
line-of-sight position is determined.
[0167] In contrast, if it is determined in Step S15
that the processing has been performed with respect to
all the positions, the signal processing unit 26 in
10 Step S16 performs calibration and the calibration
processing ends.
[0168] For example, the signal processing unit 26
determines, 1·li th respect to each position, a deviation
amount bet1-1een a display position of the calibration
15 positioning image and a line-of-sight position when the
calibration positioning image is displayed at that
position and performs calibration. That is, a
correction value for making the display position of the
image in the display region 21 coincide with the line-
20 of-sight position when the user actually gazes that
image is determined.
[0169] In the above-mentioned manner, the display
apparatus 11 displays the calibration positioning image
and performs calibration based on the display position
25 and the user's line-of-sight position. By performing a
calibration in this manner, it is possible to
SP352945W000
49
accurately correct the deviation between the display
position and the line-of-sight position and to enhance
the operability of the display apparatus 11.
[0170]
When the calibration processing is performed, the
user can activate an arbitrary application program and
execute desired processing.
[0171] For example, during execution of the
10 application program, the user can move the line of
sight and perform various operations. In such a case,
the display apparatus 11 performs the line-of-sight
detection processing to detect the user's line-of-sight
position and performs processing depending on a
15 detection result thereof.
[0172] Hereinafter, referring to a flowchart in Fig.
12, the line-of-sight detection processing by the
display apparatus 11 will be described.
[0173] In Step S41, the signal processing unit 26
20 controls the display element drive unit 27 and causes
the display pixels 51 to emit light. The display pixels
51 emit light according to the control of the display
element drive unit 27 and outputs light for displaying
a predetermined image. With this, for example, a button
25 or pointer for information selection is displayed in
the display region 21 depending on needs.
. i
t~
SP352945WOOO
50
[0174] In Step S42, the light-receiving elements 52
start detection of light entering from the eyeball.
That is, the light-receiving elements 52 receive light
that has entered the eyeball from the outside of the
5 display apparatus 11 or the display pixels 51 and been
reflected by the eyeball surface, performs
photoelectric conversion, and supply light-receiving
signals according to light-receiving amounts to the
signal processing unit 26 via the display element drive
10 unit 27.
[0175] In Step S43, the signal processing unit 26
determines a user's line-of-sight position based on the
light-receiving signals supplied from the lightreceiving
elements 52. That is, the signal processing
15 unit 26 generates a light-receiving signal map based on
the light-receiving signals and detects a pupil center
(eyeball direction) of the user based on the obtained
light-receiving signal map.
[0176] In Step S44, the signal processing unit 26
20 performs the selection processing based on the line-ofsight
position.
[0177] For example, if the pointer or cursor is
displayed in the display region 21, according to a
movement of the line-of-sight position, the signal
25 processing unit 26 controls the display element drive
unit 27 to drive the display pixels 51 to move the
5
SP352945W000
51
pointer or cursor displayed in the display region 21
(display pixels 51) . For example, control is performed
such that the pointer or the like is displayed at the
line-of-sight position.
[0178] Furthermore, if the pointer or cursor, that
is, the user's line-of-sight position is located in the
region of the selection target such as a button or icon
displayed in the display region 21, the signal
processing unit 26 determines that the selection target
10 thereof is selected. Note that if the user's line-ofsight
position is at the position of the selection
target for a predetermined time or longer, it may be
determined that the selection target is selected.
[0179] In addition, for example, even if an
15 indication means such as a pointer is not displayed in
the display region 21, 1·1hen the line-of-sight position
is located in the region of the selection target such
as a button, it may be determined that the selection
target is selected.
20 [0180] In addition to this, if the user blinks a
predetermined number of times, for example, one time or
a plurality of times in the state in which the user's
line-of-sight position is located at the selection
target such as a button or if the user closes the
25 eyelid for a predetermined time or longer in the state
in 1·1hich the line-of-sight position is located at the
SP352945WOOO
52
selection target such as a button, the selection target
may be selected. In this case, the signal processing
unit 26 detects, based on the light-receiving signal
map, blinking and a line-of-sight position or a time
5 for 1·1hich the user is closing eyes, such that selection
processing is performed on the selection target.
[0181] Furthermore, if images having a parallax are
presented to the paired left and right eyes of the user,
the selection processing is performed using the
10 distance from the user to the gaze position. In this
case, the signal antenna 23 receives an eyeball
direction or light-receiving signal map from a display
apparatus 11 worn on the other eyeball that pairs with
the one eyeball and supplies it to the signal
15 processing unit 26. Then, the signal processing unit 26
calculates the convergence angle based on the eyeball
direction (line-of-sight position) obtained in Step S43
and the received eyeball direction or light-receiving
signal map and calculates the distance to the gaze
20 position based on the obtained convergence angle.
[0182] In addition, the signal processing unit 26
may control the display pixels 51 via the display
element drive unit 27 and display the selected
selection target such as a button in color and shape
25 different from color and shape of other selection
targets that are not selected, that is, in a different
SP352945\'IOOO
53
display manner. With this, the user can easily kno\'1
l'lhich selection target is selected.
[0183] Note that the selection target is not limited
to the button or the like and any target, for example,
5 image and character information can be used as long as
it can be a selection target.
[0184] In Step S45, the signal processing unit 26
executes processing according to selection by the
selection processing in Step S44 and the line-of-sight
10 detection processing ends.
[0185] For example, the signal processing unit 26
executes the soft"1are or calculation associated l'li th
the selected selection target or controls the display
pixels 51 to display the image and character
15 information set as the selection target in an enlarged
manner. Furthermore, according to selection in the
selection processing, the diameter or the like of the
pupil may be determined based on the light-receiving
signal map as the information .used in an application
20 program, for example.
[0186] Note that, in Step S45, the processing
according to the detection result of the abovementioned
living-body state or the detection result of
microsaccades and selection by the selection processing
25 may be executed.
[0187] In the above-mentioned manner, the display
SP352945W000
54
apparatus 11 receives light from the display pixels 51
or the like by the light-receiving elements 52, detects
the line-of-sight position based on the obtained lightreceiving
signals, performs selection processing based
5 on the line-of-sight position, and executes processing
according to a selection result thereof.
[0188] In this manner, by detecting the line-ofsight
position based on the light-receiving signals
obtained by the light-receiving elements 52, it is
10 possible to easily determine the user's operation
"ithout needing the external apparatus other than the
display apparatus 11. In other "ords, it is possible to
enhance the operability of the display apparatus 11
1·1i th a simple configuration.
15 [0189] Furthermore, in the display apparatus 11,
even in the state in "hich the user closes eyes, it is
possible to detect the eyeball direction, that is, the
line-of-sight position at high accuracy. Here, the
detection accuracy of the line-of-sight position can be
20 made higher as the distance {pitch) between the lightreceiving
elements 52 proximate to each other in the
display region 21 is made shorter.
[0190]
25 In addition, in the display apparatus 11, the
living-body state can be detected.
SP352945WOOO
55
[0191) Hereinafter, referring to a flowchart in Fig.
13, the living-body state detection processing by the
display apparatus 11 will be described.
[0192) Note that the living-body state detection
5 processing is performed alternating with the line-ofsight
detection processing described with reference to
Fig. 12, for example. That is, the line-of-sight
detection processing is performed for a period for
which information of an image or the like is displayed
10 in the display region 21. After that period, the
living-body state detection processing is performed and
the living-body state is detected. Still after that,
the image or the like is displayed again in the display
region 21 and the line-of-sight detection processing is
15 performed. After that, the line-of-sight detection
processing and the living-body state detection
processing are alternately performed.
[0193) In Step S71, the signal processing unit 26
controls the display element drive unit 27 to cause the
20 display pixels 51 to emit light. The display pixels 51
emit light under the control of the display element
drive unit 27 and outputs light having a predetermined
wavelength band set in advance.
[0194) In Step S72, the light-receiving elements 52
25 detect light entering from the eyeball. That is, the
light-receiving elements 52 receive light that has
SP352945W000
56
entered the eyeball from the outside of the display
apparatus 11 or the display pixels 51 and been
reflected by the eyeball surface, perform photoelectric
conversion, and supply light-receiving signals
5 according to light-receiving amounts to the signal
processing unit 26 via the display element drive unit
27.
[0195] Note that the processes in Steps S71 and S72
are alternately performed a predetermined number of
10 times for each v1avelength of the light output from the
display pixels 51.
[0196] For example, if the pulsations of the heart,
the blood flow rate, the eye congestion degree, and the
like are detected as the living-body state, reception
15 and output of light having a predetermined wavelength
shorter than 805 nm as described above and reception
and output of light having a predetermined wavelength
shorter than 805 nm are alternately performed.
[0197] In the case of detecting the living-body
20 state, in the state in which the line-of-sight
detection processing is not performed, light from the
display pixels 51 is detected in the light-receiving
elements 52. Therefore, during detection of the livingbody
state, the light-receiving elements 52 are not
25 influenced by light for displaying an image, which is
output from the display pixels 51 v1hen the line-ofSP352945WOOO
57
sight detection processing is performed, and hence the
living-body state can be detected more accurately.
[0198) The example in which the light having two
particular wavelengths is output from the display
5 pixels 51 has been described. Alternatively, while
changing the wavelength of the light output from the
display pixels 51 over time, light beams having each of
three or more 1·1avelengths in a particular wavelength
band may be emitted to the eyeball in order.
10 [0199) In Step S73, the signal processing unit 26
determines a difference bet~lever, some processes may be performed by the control
apparatus 81 sh01m in Fig. 7.
[0208]

Furthermore, in the above, the example in 1·1hich
.I
<--j
SP352945WOOO
60
the shape as the display apparatus 11 is vieHed from
the front as shmm in Fig. 2 is circular has been
described, for example, as shoHn in Fig. 14, the shape
may be oval. Note that, in Fig. 14, portions
5 corresponding to those in Fig. 2 1·1ill be denoted by the
same reference symbols and descriptions thereof will be
appropriately omitted.
[0209] The display apparatus 101 shown in Fig. 14 is
constituted of a display region 21, a feeding antenna
10 22, a signal antenna 23, a power generation unit 24, a
sensor 25, a signal processing unit 26, and a display
element drive unit 27.
[0210] The display region 21 to the display element
drive unit 27 of the display apparatus 101 have the
15 same configurations and operations as the display
region 21 to the display element drive unit 27 of the
display apparatus 11. Only the shape of the outer
appearance of the entire apparatus and the shape of the
display region 21 are different from the display
20 apparatus 101 and the display apparatus 11.
[0211] Fig. 14 is a diagram as the display apparatus
101 is viewed in the same direction as that the user
v1earing the contact lens type display apparatus 101 is
vie1·1ed from the front. In Fig. 14, the display
25 apparatus 101 has an oval shape long in the lateral
direction. Therefore, in the state in 1·1hich the user
SP352945WOOO
61
wears the display apparatus 101 on the eyeball, in
comparison with the circular display apparatus 11, it
becomes difficult for the display apparatus 101 to
rotate with respect to the eyeball. With this, rotation
5 deviation of the display apparatus 101 with respect to
the eyeball can be suppressed.
[0212] Furthermore, in the example in Fig. 14, the
display region 21 of the display apparatus 101 has an
oval shape long in the lateral direction.
10 [ 0213] In addition, the display apparatus 101 has a
shape long in the lateral direction rather than the
vertical direction in the figure, and hence a region of
portions proximate in the left and right directions of
the display region 21 has a larger area than a region
15 of portions proximate in the upper and l01·1er directions
of the display region 21. These regions in the display
apparatus 101 that are not the display regions 21 are
regions outside the movement range of the pupil of the
user.
20 [0214] In view of this, in the display apparatus 101,
the power generation unit 24 to the display element
drive unit 27 are arranged in the region near the left
and right ends (lateral direction) of the display
apparatus 101, which are proximate in the left and
25 right directions of the display region 21 in the figure.
By arranging elements for driving the display apparatus
SP352945WOOO
62
101, such as the power generation unit 24 to the
display element drive unit 27, in the left and right
regions proximate in the left and right directions of
the display region 21 in this manner, it is possible to
5 prevent these elements from interrupting the image
display.
[0215] Fig. 15 shows a wearing structure as the
state in which the user wears the contact lens type
display apparatus 101 is vieviBd from the front of the
10 user.
[0216] The display apparatus 101 has a larger width
in the lateral direction rather than the vertical
direction in the figure. Therefore, in the state in
which the display apparatus 101 is loJOrn on the eyeball
15 of the user, the lateral width is larger than the
vertical width by v1hich the display apparatus 101
covers the eyeball. Furthermore, the position of the
end of the display apparatus 101 in the upper and l01·1er
directions is set to extend to the front of the
20 connection between the eyelid and the eyeball of the
user. The width in the left and right directions is set
to be a width in a range in which the eyeball moves to
the left and right. The width in the left and right
directions is larger.
25 [ 021 7] Furthermore, the display apparatus 101 has a
structure in which it is fixed to the head such that
SP352945W000
63
the display apparatus 101 is not moved Hith respect to
the head of the user.
[0218] For example, with respect to a movement of
the eyeball due to variations in the line of sight of
5 the user, Hhen the contact lens type display apparatus
101 is moved together with the eyeball, an absolute
position of information (image) currently displayed in
the display region 21 vii th respect to the head of the
user is also moved. The movement of the absolute
10 position of the information (image) Hith respect to the
head of the user is recognized as the movement of the
display position. Therefore, it is desirable to
constantly fix the position of the contact lens type
display apparatus 101 Hith respect to the head of the
15 user.
[0219] In vie VI of this, for example, as shovm in Fig.
16, convex portions are provided near an outer
circumference of the display apparatus 101. In the
example shown in Fig. 16, the display apparatus 101 is
20 worn to cover an entire cornea portion C021 of the
eyeball of the user.
[0220] Furthermore, in this example, the upper and
l01·1er ends of the display apparatus 101 are positioned
in front of the connections at which the eyelid and the
25 eyeball are connected to each other in the upper and
lower ends, that is, near a limbus. In a surface on the
SP352945WOOO
64
external world side of the upper and lower end portions
of the display apparatus 101, convex portion 141 and
convex portion 142 protruding to the external world
side are provided. The convex portion 141 and convex
5 portion 142 are held in contact with the eyelid
(palpebral conjunctiva). Thus, also when the eyeball
moves or the user blinks, the display apparatus 101 is
fixed not to be moved 1-lith respect to the head of the
10
user.
[0221] The example in 1·1hich the display apparatus
101 is provided with the convex portion 141 and the
convex portion 142 has been described. However, for
example, as shown in Fig. 17, by providing a highfriction
portion 151 and a high-friction portion 152
15 near an upper and lower outer circumference of the
display apparatus 101, the display apparatus 101 may be
fixed with respect to the head of the user.
[0222] The high-friction portion 151 and the highfriction
portion 152 are processed to have a higher
20 coefficient of friction with respect to the eyelid in
comparison with the center portion of the display
apparatus 101. Therefore, in the state in 1·1hich the
display apparatus 101 is 1"/0rn on the eyeball of the
user, due to the friction of the high-friction portion
25 151 and high-friction portion 152 and the eyelid
(palpebral conjunctiva) of the user, the display
SP352945W000
65
apparatus 101 is fixed not to be moved with respect to
the head of the user.
[0223] In this manner, if the display apparatus 101
is provided with the convex portion 141 and the convex
5 portion 142 or the high-friction portion 151 and the
high-friction portion 152, the convex portions and the
high-friction portions are provided in region SR11 and
region SR12 shm-m in Fig. 18.
[0224] Note that Fig. 18 is a diagram of the display
10 apparatus 101 as the display apparatus 101 is viewed
from the same direction as that 1·1hen the user wearing
the display apparatus 101 is viewed from the front.
Therefore, an upper side in the figure of the display
apparatus 101 corresponds to an upper side of the eye
15 of the user and a lower side in the figure of the
display apparatus 101 corresponds to a lower side of
the eye of the user.
[0225] In 'this case, the convex portion 141 shovm in
Fig. 16 or the high-friction portion 151 shown in Fig.
20 17 is provided in the region SR11 formed along an upper
end of the display apparatus 101. Furthermore, the
convex portion 142 shmm in Fig. 16 or the highfriction
portion 152 shown in Fig. 17 is provided in
the region SR12 formed along a lower end of the display
25 apparatus 101.
[0226] Here, the convex portion 141 and the convex
SP352945WOOO
66
portion 142 or the high-friction portion 151 and the
high-friction portion 152 are provided on a front side
in the figure of the display apparatus 101, that is,
the external world side.
5 [0227] The example in which the display apparatus
101 is provided 1·1ith the convex portions and the highfriction
portions has been described. Ho1·1ever, the
convex portions and the high-friction portions may be
provided along upper and lower ends of the display
10 apparatus 11 shown in Fig. 2.
[0228]

The example in which the display pixels 51 and. the
light-receiving elements 52 are provided in a close
15 contact state in the display region 21 as shmm in Fig.
3 has been described above. However, the transmissive
region for causing the ambient light from the outside
to pass through the display region 21 may be provided.
[0229] In such a case, the display region 21 is, for
20 example, configured as shovm in Fig. 19. Note that the
vertical direction and the lateral direction in Fig. 19
correspond to, for example, the vertical direction and
the lateral direction in Fig. 14. Furthermore, in Fig.
19, a single square region represents the display pixel
25 51, the light-receiving element 52, or the transmissive
region.
SP352945WOOO
67
[0230] Specifically, the black square represents a
region of a single display pixel 51. The double-hatched
square represents a region of a single light-receiving
element 52. The Hhite square represents a transmissive
5 region. Here, the transmissive region is a region
having a higher transmittance (transparency) against
light in comparison 1·li th the display pixel 51 and the
light-receiving element 52.
[0231] For example, a square indicated by an arro\'1
10 mark Q31 represents a region of a single display pixel
51. The upper, lower, left, and right portions in the
figure of the display pixels 51 are set as transmissive
regions. Furthermore, in the obliquely upper and l01•1er
portions of the display pixel 51 indicated by the arr01·1
15 mark Q31, the light-receiving element 52 is disposed.
Therefore, each of the display pixels 51 is surrounded
by four light-receiving elements 52 and four
transmissive regions.
[0232] By providing the transmissive regions for
20 causing light of the external world (ambient light) to
pass through the display apparatus 11 or the display
apparatus 101 around each display pixel 51 as described
above, it becomes possible for the user to look around
also l·lhen wearing the display apparatus 11 or the
25 display apparatus 101. Furthermore, in this example,
the light-receiving element 52 is disposed adjacent to
SP3529451'1000
68
each display pixel 51, and hence light output from the
display pixel 51 and reflected by the eyeball can be
received by the light-receiving element 52.
[0233]
5
Furthermore, the display region 21 may be
configured as shown in Fig. 20. Note that, in Fig. 20,
the single square region represents the display pixel
51, the light-receiving element 52, or the transmissive
10 region.
[0234] Specifically, the black square represents a
region of a single display pixel 51. The double-hatched
square represents a region of a single light-receiving
element 52. The white square represents a transmissive
15 region.
[0235] For example, the square indicated by the
arr01·1 mark Q41 represents a region of a single display
pixel 51 and the display pixel 51 is surrounded by the
transmissive regions.
20 [0236] Furthermore, the square indicated by the
arrow mark Q42 represents a region of a single display
pixel 51. In an obliquely upper right portion in the
figure of the display pixel 51, a single lightreceiving
element 52 is disposed. Other regions
25 proximate to the display pixel 51 indicated by the
arrow mark Q42 are transmissive regions.
SP352945WOOO
69
[0237] In the example shown in Fig. 20, the number
of light-receiving elements 52 provided in the display
region 21 is smaller than the number of display pixels
51 provided in the display region 21. More transmissive
5 regions are correspondingly provided. By providing
light-receiving elements 52 fewer than the display
pixels 51 as described above, it is possible to
increase light (ambient light) passing through the
display region 21 from the outside of the display
10 apparatus. Further, the user can look around in a
brighter view in comparison with the example shm·m in
Fig. 19.
[ 0238]

15 In addition, in the case where the display pixels
51 provided in the display region 21 have permeability,
the user can look around through the display pixels 51
without providing the transmissive regions in the
display region 21. In such a case, the display region
20 21 is, for example, configured as shm·m in Fig. 21.
[0239] Note that, in Fig. 21, the black region
represents a region of display pixels 51. The doublehatched
square represents a region of a single lightreceiving
element 52.
25 [0240] In this example, the light-receiving elements
52 are disposed adjacent to the display pixels 51.
SP352945\'IOOO
70
Furthermore, the transmittance of light in the display
pixel 51 is higher than the transmittance of light in
the light-receiving element 52. The user can look
around through the display pixels 51.
5 [0241]

In addition, a display apparatus 101 may be
provided with the pressure sensor, the open/close of
10 the eyelid of the user v1earing the display apparatus
101 may be detected, and the pressure 11hen the user
strongly closes the eyelid may be detected. In such a
case, the display apparatus 101 is, for example,
configured as shown in Fig. 22. Note that, in Fig. 22,
15 portions corresponding to those in Fig. 14 will be
denoted by the same reference symbols and descriptions
thereof 1·1ill be appropriately omitted.
[0242) The display apparatus 101 shown in Fig. 22 is
different from the display apparatus 101 in Fig. 14 in
20 that the display apparatus 101 shm-m in Fig. 14 is
further provided 1·1i th a pressure sensor 181 but
othen1ise has the same configuration as the display
apparatus 101 in Fig. 14.
[0243) That is, the display apparatus 101 in Fig. 22
25 is constituted of a display region 21, a feeding
antenna 22, a signal antenna 23, a power generation
SP352945WOOO
71
unit 24, a sensor 25, a signal processing unit 26, a
display element drive unit 27, and a pressure sensor
181.
[0244] The pressure sensor 181 is positioned near a
5 right end in the figure and is operated by receiving
electric power from the power generation unit 24.
Furthermore, the pressure sensor 181 detects a pressure
applied in a depth direction in the figure of the
display apparatus 101 and supplies a detection result
10 thereof to the signal processing unit 26 via the
display element drive unit 27.
[0245] The signal processing unit 26 detects the
close/open or the like of the eyelid of the user based
on a detection result of the pressure supplied from the
15 pressure sensor 181. For example, the signal processing
unit 26 determines selection of the selection target
v1hen the user closes the eyelid or determines selection
of the selection target 1<1hen the user closes the eyelid,
that is, a pressure higher than a predetermined value
20 is detected.
25
[0246] By the pressure sensor 181 detecting the
user's operation based on the detection result of the
pressure in this manner, it is possible to further
enhance the operability of the display apparatus 101.
[0247]

Note that the example in which the present
technology of detecting the user's line-of-sight
direction is applied to the display apparatus has been
5 described above. The present technology is not limited
to the display apparatus. The present technology is
applicable to all apparatuses that detects the user's
line-of-sight direction (eyeball direction) .
Hereinafter, an embodiment in which the present
10 technology is applied to the line-of-sight detection
apparatus that detects the user's line-of-sight
direction "'ill be described.
[0248] Fig. 23 is a diagram showing a configuration
example of a line-of-sight detection apparatus to which
15 the present technology is applied. Note that, in Fig.
23, portions corresponding to those in Fig. 2 will be
denoted by the same reference symbols and descriptions
thereof 1-lill be appropriately omitted.
[0249] A contact lens type line-of-sight detection
20 apparatus 211 has a shape such that it can be worn on
the eyeball of the user. In the state in 1·1hich a lineof-
sight detection apparatus 211 is worn on the eyeball,
the line-of-sight detection apparatus 211 covers the
entire cornea portion of the eyeball of the user.
25 [0250] The line-of-sight detection apparatus 211 is
constituted of a detection region 221, a feeding
SP352945WOOO
73
antenna 22, a signal antenna 2:5, a pO\·/er generation
unit 24, a sensor 25, a signal processing unit 26, and
a light-emitting element drive unit 222.
[0251] Note that Fig. 23 is a diagram of the line-
5 of-sight detection apparatus 211 as the line-of-sight
detection apparatus 211 is viewed from the same
direction as that 1·1hen the user 1·1earing the line-ofsight
detection apparatus 211 is viewed from the front.
In Fig. 23, the line-of-sight detection apparatus 211
10 has an oval shape.
[0252] The detection region 221 includes a lightemitting
element formed of a plurality of lightemitting
sections that emit light for the line-of-sight
detection to the eyeball surface of the user and a
15 light-receiving element that is disposed adjacent to
the light-emitting sections and receives light
reflected by the eyeball surface of the user.
Furthermore, as in the above-mentioned display region
21, in the detection region 221, at least one of the
20 light-emitting section and the light-receiving element
are provided in a region in the detection region 221
that is opposed to a region in a range in 1·1hich the
pupil of the eyeball of the user is movable.
[0253] The light-emitting element drive unit 222
25 drives the light-emitting element of the detection
region 221 under the control of the signal processing
5
SP352945WOOO
74
unit 26, emits light from each light-emitting section,
and supplies the light-receiving signals supplied from
the light-receiving elements of the detection region
221 to the signal processing unit 26.
[0254] In the line-of-sight detection apparatus 211,
the positions of the feeding antenna 22 to the signal
processing unit 26 are the same positions as in the
case of the display apparatus 101 in Fig. 14.
[0255] Furthermore, the detection region 221 of the
10 line-of-sight detection apparatus 211 is, for example,
configured as shown in Fig. 24. Note that Fig. 24 shows
a part of a cross-section of the line-of-sight
detection apparatus 211 as the line-of-sight detection
apparatus 211 is vie1·1ed from the lateral direction in
15 Fig. 23. Note that, in Fig. 24, portions corresponding
to those in Fig. 3 1·1ill be denoted by the same
reference symbols and descriptions thereof will be
appropriately omitted.
[0256] In Fig. 24, in the detection region 221 of
20 the line-of-sight detection apparatus 211, the lightemitting
sections 251-1 to 251-7 that emit light for
detecting the line-of-sight direction and the lightreceiving
elements 52-1 to 52-7 that receive reflected
light entering from the eyeball surface of the user are
25 provided. Then, a single light-emitting device formed
of the light-emitting sections 251-1 to 251-7 is set as
SP352945WOOO
75
a light-emitting element 252.
[0257] Hereinafter, in the case where the lightemitting
sections 251-1 to 251-7 do not need to be
particularly distinguished from one another, they will
5 be also simply referred to as light-emitting sections
251.
[0258] The light-emitting section 251 has a function
of emitting light for line-of-sight detection by
emitting light. However, the light-emitting section 251
10 has a function of displaying information as in the
display pixels 51 shmm in Fig. 3.
[ 0259] In the example in Fig. 24, the light-emitting
sections 251 and the light-receiving elements 52 are
alternately arranged on the right side in the figure of
15 the line-of-sight detection apparatus 211, that is, on
the side of the eyeball of the user in the vertical
direction. Therefore, for example, in Fig. 23, the
light-emitting sections 251 and the light-receiving
elements 52 are alternately arranged in the vertical
20 direction and the lateral direction in Fig. 23 in the
detection region 22.
[ 02 60 l Note that, in Fig. 24, the example in 1·1hich
the light-emitting sections 251 and the light-receiving
elements 52 are in close contact has been described.
25 The light-emitting sections 251 and the light-receiving
elements 52 do not necessarily need to be in close
SP352945WOOO
76
contact. A clearance bet1-1een the light-emitting
sections 251 and the light-receiving elements 52 may be
provided. Furthermore, in Fig. 24, one light-receiving
element 52 is provided for one light-emitting section
5 251. H01·1ever, one light-receiving element 52 may be
provided for a plurality of light-emitting sections 251.
[0261] In addition, the example in which the lightemitting
element 252 formed of the plurality of lightemitting
sections 251 is provided in the detection
10 region 221 has been described. However, the lightemitting
element 252 formed of a single light-emitting
section that emits light in the entire region of the
detection region 221 may be provided in the detection
region 221. In this case, in each region of the light-
15 emitting element 252, the light-receiving elements 52
for detecting the light-receiving amounts in these
regions only need to be provided.
[ 02 62] Furthermore, in the case 1·1here only light
entering the eyeball from the external 1o10rld is used to
20 detect the user's line-of-sight direction, it is not
necessarily necessary to provide the line-of-sight
detection apparatus 211 with the light-emitting element
252.
[ 02 63]
25 Next, operations of the line-of-sight detection
apparatus 211 1·1ill be described.
SP352945WOOO
77
[0264] For example, when the line-of-sight detection
apparatus 211 is worn by the user, the calibration
processing is started. Hereinafter, the calibration
processing by the line-of-sight detection apparatus 211
5 will be described with reference to a flowchart in Fig.
25.
[ 02 65] In Step SlOl, the signal processing unit 26
controls the light-emitting element drive unit 222 to
cause the light-emitting sections 251 to emit light.
10 The light-emitting sections 251 emit light under the
control of the light-emitting element drive unit 222
and outputs light for detecting the user's line-ofsight
direction.
[0266] In Step Sl02, the light-receiving elements 52
15 start detection of light entering from the eyeball.
That is, the light-receiving elements 52 receive light
that has entered the eyeball from the outside of the
line-of-sight detection apparatus 211 or the lightemitting
sections 251 and been reflected by the eyeball
20 surface perform photoelectric conversion, and supply
light-receiving signals according to light-receiving
amounts to the signal processing unit 26 via the lightemitting
element drive unit 222.
[ 02 67] Furthermore, 1·1hen the calibration processing
25 is started, the user looks in a direction set in
advance. For example, according to a sound guidance or
;-!
"1
SP352945WOOO
78
the like output from the control apparatus wirelessly
connected to the line-of-sight detection apparatus 211,
the user moves the line of sight in the upper, l01·1er,
left, or right direction as largely as possible.
5 [ 02 68] While the user moves the line of sight in the
upper, lower, left, or right direction as described
above, light for detecting the line-of-sight direction
is output from the light-emitting sections 251. The
light-receiving elements 52 receive light entering from
10 the eyeball surface, for example, light that has been
output from the light-emitting sections 251 and
reflected by the eyeball. Then, the light-receiving
elements 52 supply light-receiving signals according to
light-receiving amounts to the signal processing unit
15 26 via the light-emitting element drive unit 222.
[0269] In Step Sl03, the signal processing unit 26
determines a user's line-of-sight position based on the
light-receiving signals supplied from the lightreceiving
elements 52.
20 [0270] For example, when the user largely moves the
line of sight in the upper, lower, left, or right
direction, the signal processing unit 26 determines a
position of each of upper, l01·1er, left, and right ends
of the line of sight moved by the user as a line-of-
25 sight position. With this, the user can determine a
range in which the line of sight is movable based on
d
'"j
SP352945WOOO
79
the line-of-sight positions. Note that, during
calculation of the line-of-sight position, for example,
the same processing as Step Sl4 in Fig. 11 is performed.
[0271] In Step Sl04, the signal processing unit 26
5 performs calibration based on the determined line-ofsight
position and the calibration processing ends.
[0272] For example, it is assumed that, after
calibration, the detection result of the line of sight
by the line-of-sight detection apparatus 211 is used
10 for processing of moving a cursor on an external
display different from the line-of-sight detection
apparatus 211 by the user moving the line of sight.
[0273] In such a case, the signal processing unit 26
determines a range in which the line of sight of the
15 user is movable based on the line-of-sight position
with respect to each of the upper, lower, left, and
right positions, which is determined in the processing
of Step Sl03. Then, the signal processing unit 26
performs calibration by making each position of the
20 region, \•lhich is obtained by subtracting a margin from
the range in which the line of sight of the user is
movable, corresponding to each position of the display.
[0274] In the above-mentioned manner, the line-ofsight
detection apparatus 211 performs calibration
25 based on some line-of-sight positions. By performing
calibration in this manner, it is possible to obtain a
SP352945W000
80
correspondence or the like of a particular region of
the external display or the like and a region as a
moving destination of the line of sight of the user,
and hence to enhance the operability of the interface
5 operation performed by the user.
[0275]
For example, in the case ••here the line-of-sight
detection apparatus 211 and the external control
10 apparatus are vlirelessly connected to each other, when
the calibration processing is performed, the user can
activate an arbitrary application program and execute
desired processing.
[0276] For example, during execution of the
15 application program, the user can move the line of
sight and perform various operations. In such a case,
the line-of-sight detection apparatus 211 performs
line-of-sight detection processing to detect the user's
line-of-sight position and outputs a detection result
20 thereof to the external control apparatus.
25
[0277] Hereinafter, line-of-sight detection
processing by the line-of-sight detection apparatus 211
will be described with reference to a flowchart in Fig.
26.
[0278] In Step S131, the signal processing unit 26
controls the light-emitting element drive unit 222 to
SP352945WOOO
81
cause the light-emitting sections 251 to emit light.
The light-emitting sections 251 emit light under the
control of the light-emitting element drive unit 222
and output light for detecting the user's line-of-sight
5 direction.
[0279] In Step S132, the light-receiving elements 52
start detection of light entering from the eyeball.
That is, the light-receiving elements 52 receive light
that has entered from the outside of the line-of-sight
10 detection apparatus 211 or the light-emitting sections
251 and been reflected by the eyeball surface, perform
photoelectric conversion, and supply light-receiving
signals according to light-receiving amounts to the
signal processing unit 26 via the light-emitting
15 element drive unit 222.
[0280] In Step S133, the signal processing unit 26
determines a user's line-of-sight position based on the
light-receiving signals supplied from the lightreceiving
elements 52. That is, the signal processing
20 unit 26 generates a light-receiving signal map based on
the light-receiving signals and determines a user's
line-of-sight position by detecting a pupil center
(eyeball direction) of the user based on the obtained
light-receiving signal map.
25 [0281] In Step S134, the signal processing unit 26
outputs the determined line-of-sight position and
SP352945W000
82
terminates the line-of-sight detection processing.
[0282] For example, the signal processing unit 26
supplies the determined line-of-sight position to the
signal antenna 23 and causes the control apparatus to
5 send it. The control apparatus executes processing
according to the line-of-sight position, for example,
moving the cursor or the like according to the line-ofsight
position received from, for example, the line-ofsight
detection apparatus 211.
10 [0283] In the above-mentioned manner, in the lineof-
sight detection apparatus 211, light from the lightemitting
sections 251 or the like is received by the
light-receiving elements 52. The line-of-sight position
is detected based on the obtained light-receiving
15 signals. A detection result thereof is output.
[0284] By detecting the line-of-sight position based
on the light-receiving signals obtained by the lightreceiving
elements 52 in this manner, it is possible to
easily determine the user's operation \•li thout needing
20 the external apparatus other than the line-of-sight
detection apparatus 211. In other words, it is possible
to enhance the operability with a simple configuration.
[0285] Here, the processing in which the line-ofsight
detection apparatus 211 detects the user's line-
25 of-sight position has been described. However, the
line-of-sight detection apparatus 211 may cause the
SP352945WOOO
83
light-emitting sections 251 to output light having a
particular wavelength from the light-emitting sections
251 for detecting the living-body state, calculating a
convergence amount of the left and right eyes or a
5 distance to a target object, or determining a diameter
of the pupil.
[ 02 8 6] In addition, in the above, the example in
which, using the fact that the value of the lightreceiving
signal in the pupil region in the light-
10 receiving signal map is smaller than the value of the
light-receiving signal in the region of the sclera or
iris portion, the pupil region, that is, the line-ofsight
position is detected based on the light-receiving
signal map has been described.
15 [0287] In other words, using a difference betv1een a
spectral reflectance of the retina at 1·1hich light
transmitted through the pupil arrives and a spectral
reflectance of the sclera or iris portion, the pupil
region of the user is detected based on the light-
20 receiving signal map.
[0288] Therefore, depending on the wavelength of
light emitted from the display pixels 51, the lightemitting
sections 251, or the like, the reflectance of
the retina is in some cases higher than the reflectance
25 of the sclera or iris portions. In those cases, in the
light-receiving signal map, the value of the lightSP352945WOOO
84
receiving signal in the pupil region is higher than the
value of the light-receiving signal in the region of
the sclera or iris portion.
[0289] Also in the case where light by which the
5 reflectance of the retina is higher than the
reflectance of the sclera or iris portion is output
from the display pixels 51 or the light-emitting
sections 251 as described above, it is possible to
detect the pupil region based on the light-receiving
10 signal map in the signal processing unit 26. In this
case, the signal processing unit 26 detects a region of
the light-receiving signal map, in l'lhich the value of
the light-receiving signal is high, as the pupil region.
[0290] In any case, the signal processing unit 26 is
15 capable of detecting the pupil region (line-of-sight
position) based on the value of the light-receiving
signal in each region of the light-receiving signal map.
At this time, v1hether to set the region in which the
value of the light-receiving signal is high or the
20 region in Hhich the value of the light-receiving signal
is lol'l as the pupil region only needs to be determined
depending on the wavelength of the light output from
the display pixels 51 or the light-emitting sections
251, the spectral reflectance properties of the sclera,
25 iris, and retina, and the like.
[0291] Note that embodiments of the present
rl
5
SP352945WOOO
85
technology are not limited to the above-mentioned
embodiments and various modifications can be made
1·1i thout departing from the gist of the present
technology.
[0292) For example, the present technology can take
a cloud computing configuration in ~o~hich a single
function is shared with a plurality of apparatuses over
a netl'lork and commonly processed.
[0293) Furthermore, the steps described l'lith
10 reference to the fl01·1charts above can be executed by a
single apparatus and can be othen1ise shared 1vi th and
executed by a plurality of apparatuses.
[0294) In addition, in the case \'/here a single step
includes a plurality of processes, the plurality of
15 processes included in the single step can be executed
by the single apparatus and can be otherwise shared
v1ith and executed by the plurality of apparatuses.
[0295) In addition, the present technology may also
take the follol'ling configurations.
20 [0296)
[1)
A detection apparatus that is wearable on an
eyeball, including
a light-receiving element that receives light
25 entering from the eyeball.
[ 2)
SP352945WOOO
86
The detection apparatus according to [1], further
including
a light-emitting element that outputs light, in
which
5 the light-receiving element is provided near the
light-emitting element.
[3]
The detection apparatus according to [2], in 1·1hich
the light-emitting element is formed of a
10 plurality of light-emitting sections, and
the light-receiving element is provided near the
light-emitting section.
[4]
The detection apparatus according to [3], in which
15 the light-receiving element receives light that is
output from the light-emitting section and reflected by
the eyeball, further including
a signal processing unit that detects lightreceiving
amounts of a plurality of light-receiving
20 elements arranged in regions of the detection apparatus.
[ 5]
The detection apparatus according to [3] or [4],
in 1·1hich
the light-emitting section is a display pixel that
25 displays information.
[ 6]
-SP352945W000
87
The detection apparatus according to any one of
[2] to [5], in which
the detection apparatus is configured to cover an
entire cornea portion 1·1hen the detection apparatus is
5 worn on the eyeball.
[ 7]
The detection apparatus according to any one of
[3] to [6], in which
in a state in which the detection apparatus is
10 •10rn on the eyeball, at least one of the light-emitting
section and the light-receiving element is provided in
a region of the detection apparatus that is opposed to
a region in a range in which a pupil of the eyeball is
movable.
15 [ 8]
The detection apparatus according to any one of
[2] to [7], in which
a lateral width is set to be larger than a
vertical •1idth by which the detection apparatus covers
20 the eyeball.
[ 9]
The detection apparatus according to [ 8], in 1·1hich
an element different from the light-emitting
element and the light-receiving element is provided
25 near a lateral end of the detection apparatus.
[10]
SP352945WOOO
88
The detection apparatus according to [2] to [9),
in l·lhich
the detection apparatus has a structure for fixing
the detection apparatus with respect to a head having
5 the eyeball.
[11)
The detection apparatus qccording to [4), in which
the signal processing unit determines a direction
of the eyeball based on the light-receiving amounts of
10 the plurality of light-receiving elements.
[12)
The detection apparatus according to [11), in
v1hich
the signal processing unit calculates a
15 convergence amount of left and right eyes based on the
direction of the eyeball and a direction of an eyeball
that pairs \•lith the eyeball and calculates a distance
to a gazed target object based on the convergence
amount.
20
25
[13)
The detection apparatus according to [4), in which
the signal processing unit determines a diameter
of a pupil of the eyeball based on the light-receiving
amounts of the plurality of light-receiving elements.
[14)
The detection apparatus according to [4), in which
5
SP352945WOOO
89
the signal processing unit detects a living-body
state based on the light-receiving amounts of the
plurality of light-receiving elements.
[ 15]
The detection apparatus according to [14], in
which
the light-emitting section emits light having a
predetermined wavelength to the eyeball or emits light
having different 1-1avelengths to the eyeball in order,
10 and
the signal processing unit detects the living-body
state based on light-receiving amounts of the light
having the predetermined 1-1avelength or the light having
the different 1-1avelengths that is emitted to the
15 eyeball, in the light-receiving elements.
[ 16]
The detection apparatus according to [15], in
1-1hich
the light-emitting section is a display pixel that
20 displays information and emits, after a period in 1·1hich
the information is displayed, the light having the
predetermined 1-1avelength or the light having the
different wavelengths to the eyeball.
[17]
25 A detection method for a detection apparatus
including
d ,
SP352945WOOO
90
a light-receiving element that receives light
entering from an eyeball, and
a signal processing unit that detects a
light-receiving amount of the light-receiving element
5 and being wearable on the eyeball, the method
including:
a light-receiving step of receiving, by the lightreceiving
element, light reflected by the eyeball; and
a detection step,of detecting, by the signal
10 processing unit, light-receiving amounts of a plurality
of light-receiving elements arranged in regions of the
detection apparatus.
[18]
The detection method according to [17], further
15 including
a light-emitting step of outputting light by a
light-emitting element provided in the detection
apparatus, in l'lhich
the light-receiving element receives, in the
20 light-receiving step, light output from the lightemitting
element and reflected by the eyeball.
[ 19]
The detection method according to [18], further
including
25 a calculation step of determining, by the signal
processing unit, a direction of the eyeball based on
SP352945W000
91
the light-receiving amounts of the plurality of lightreceiving
elements.
[ 20 l
The detection method according to [19], in which
5 the signal processing unit calculates, in the
calculation step, a convergence amount of left and
right eyes based on the direction of the eyeball and a
direction of an eyeball that pairs 1·1i th the eyeball and
calculates a distance to a gazed target object based on
10 the convergence amount.
Description of Reference Numerals
[0297]
11 display apparatus
21 display region
15 23 signal antenna
25 sensor
26 signal processing unit
27 display element drive unit
51-1 to 51-7, 51 display pixel
20 52-1 to 52-7, 52 light-receiving element
53 display element
81 control apparatus
101 display apparatus
141 convex portion
25 142 convex portion
151 high-friction portion
152 high-friction portion
181 pressure sensor
92
211 line-of-sight detection apparatus
251-1 to 251-7, 251 light-emitting section
5 252 light-emitting element
SP352945WOOO
SP352945WOOO
93

Claims
[ 1] A detection apparatus that is 1·1earable on an
eyeball, comprising
a light-receiving element that receives light
5 entering from the eyeball,
[2] The detection apparatus according to claim 1,
further comprising
a light-emitting element that outputs light,
wherein
10 the light-receiving element is provided near the
light-emitting element.
[3] The detection apparatus according to claim 2,
v1herein
the light-emitting element is formed of a
15 plurality of light-emitting sections, and
the light-receiving element is provided near the
light-emitting section.
[4] The detection apparatus according to claim 3,
wherein
20 the light-receiving element receives light that is
output from the light-emitting section and reflected by
the eyeball, further comprising
a signal processing unit that detects lightreceiving
amounts of a plurality of light-receiving
25 elements arranged in regions of the detection apparatus.
[5] The detection apparatus according to claim 3,
SP352945WOOO
94
·wherein
the light-emitting section is a display pixel that
displays information.
[6] The detection apparatus according to claim 2,
5 v1herein
the detection apparatus is configured to cover an
entire cornea portion 1·1hen the detection apparatus is
worn on the eyeball.
[7] The detection apparatus according to claim 3,
10 wherein
in a state in v1hich the detection apparatus is
worn on the eyeball, at least one of the light-emitting
section and the light-receiving element is provided in
a region of the detection apparatus that is opposed to
15 a region in a range in which a pupil of the eyeball is
movable.
[8] The detection apparatus according to claim 2,
wherein
a lateral width is set to be larger than a
20 vertical width by which the detection apparatus covers
the eyeball.
[9] The detection apparatus according to claim 8,
wherein
an element different from the light-emitting
25 element and the light-receiving element is provided
near a lateral end of the detection apparatus.
SP352945WOOO
95
[10] The detection apparatus according to claim 2,
wherein
the detection apparatus has a structure for fixing
the detection apparatus lvith respect to a head having
5 the eyeball.
[11] The detection apparatus according to claim 4,
\'/herein
the signal processing unit determines a direction
of the eyeball based on the light-receiving amounts of
10 the plurality of light-receiving elements.
[12] The detection apparatus according to claim 11,
v1herein
the signal processing unit calculates a
convergence amount of left and right eyes based on the
15 direction of the eyeball and a direction of an eyeball
that pairs 1·1i th the eyeball and calculates a distance
to a gazed target object based on the convergence
amount.
[13] The detection apparatus according to claim 4,
20 1·1herein
the signal processing unit determines a diameter
of a pupil of the eyeball based on the light-receiving
amounts of the plurality of light-receiving elements.
[14] The detection apparatus according to claim 4,
25 1·1herein
the signal processing unit detects a living-body
5
SP352945WOOO
96
state based on the light-receiving amounts of the
plurality of light-receiving elements.
[15] The detection apparatus according to claim 14,
\·lherein
the light-emitting section emits light having a
predetermined wavelength to the eyeball or emits light
having different wavelengths to the eyeball in order,
and
the signal processing unit detects the living-body
10 state based on light-receiving amounts of the light
having the predetermined wavelength or the light having
the different wavelengths that is emitted to the
eyeball, in the light-receiving elements.
[16] The detection apparatus according to claim 15,
15 wherein
the light-emitting section is a display pixel that
displays information and emits, after a period in which
the information is displayed, the light having the
predetermined v1avelength or the light having the
20 different v1avelengths to the eyeball.
25
[17] A detection method for a detection apparatus
including
a light-receiving element that receives light
entering from an eyeball, and
a signal processing unit that detects a
light-receiving amount of the light-receiving element
SP352945WOOO
97
and being wearable on the eyeball, the method
comprising:
a light-receiving step of receiving, by the lightreceiving
element, light reflected by the eyeball; and
5 a detection step of detecting, by the signal
processing unit, light-receiving amounts of a plurality
of light-receiving elements arranged in regions of the
detection apparatus.
[18] The detection method according to claim 17,
10 further comprising
a light-emitting step of outputting light by a
light-emitting element provided in the detection
apparatus, wherein
the light-receiving element receives, in the
15 light-receiving step, light output from the lightemitting
element and reflected by the eyeball.
[19] The detection method according to claim 18,
further comprising
a calculation step of determining, by the signal
20 processing unit, a direction of the eyeball based on
the light-receiving amounts of the plurality of lightreceiving
elements.
[20] The detection method according to claim 19,
\'/herein
25 the signal processing unit calculates, in the
calculation step·, a convergence amount of left and
SP352945\'IOOO
98
right eyes based on the direction of the eyeball and a
direction of an eyeball that pairs with the eyeball and
calculates a distance to a gazed target object based on
the convergence amount.
5
Dated this 11th day of January, 2016
[RANJNA MEHTA DUTT]
[INIPA- 190A]
OFREMFRYANDSAGAR
ATTORNEY FOR THE APPLICANT(S)
SP352945WOOO
99
Abstract
The present technology relates to detection
apparatus and method by Hhich the operability can be
enhanced with a simple configuration.
5 A contact lens type line-of-sight detection apparatus
has a shape such that it is Hearable on an eyeball of a
user. Furthermore, in the line-of-sight detection
apparatus, a plurality of light-emitting sections that
output light and a plurality of light-receiving
10 elements that receive light reflected by an eyeball
surface. The light-receiving elements receive light
that has been output from the light-emitting sections
and reflected by the eyeball surface and output lightreceiving
signals according to amounts of light
15 received. The signal processing unit detects a line of
sight of the user based on the light-receiving signals
of the light-receiving elements. The present technology
can be applied to a contact lens type line-of-sight
detection apparatus or a display apparatus.