EXTRACTED FROM WIPO:
FORMULAS AND TABLES ARE NOT COPIED:
FORM 2
THE PATENTS ACT, 1970
(39 of 1970)
&
THE PATENTS RULES, 2003
COMPLETE SPECIFICATION
(See section 10, rule 13)
“IMAGE CAPTURE ELEMENT, IMAGE CAPTURE DEVICE, AND
INFORMATION PROCESSING METHOD”
SONY CORPORATION, of 1-7-1, Konan, Minato-ku, Tokyo
108-0075, Japan
The following specification particularly describes the invention and the manner in which it is to
be performed.
1
SP371667WO00
DESCRIPTION
IMAGING ELEMENT, IMAGING DEVICE, AND INFORMATION
PROCESSING METHOD
5 TECHNICAL FIELD
[0001]
The present technology relates to an imaging
element, an imaging device, and an information processing
method, and more particularly to an imaging element, an
10 imaging device, and an information processing method
capable of generating a more accurate restored image.
BACKGROUND ART
[0002]
15 Conventionally, an imaging element is typically
used in combination with an imaging lens that focuses
light on the imaging element. The imaging lens guides
light from an object plane to each pixel of the imaging
element so as to reproduce light intensity distribution
20 on the object plane, so that the imaging element can
obtain a detection signal at a level corresponding to the
light intensity distribution in each pixel and can obtain
a captured image of the object as a whole.
[0003]
25 However, in this case, the physical size increases.
Therefore, imaging elements without using an imaging lens
have been conceived (see, for example, Patent Documents 1
to 3 and Non-Patent Document 1). A detection image is
generated in such an imaging element. A restored image
30 is restored by solving multidimensional simultaneous
linear equations (performing a matrix operation) using
2
SP371667WO00
the detection image and a restoration matrix according to
characteristics of the imaging element. This restoration
matrix is generated on the basis of, for example, design
values of an imaging element, and the like. Ideally, a
5 correct restored image can be restored using a correct
restoration matrix.
CITATION LIST
PATENT DOCUMENT
10 [0004]
Patent Document 1: International Publication No.
2016/123529
Patent Document 2: PCT Japanese Translation Patent
Publication No. 2016-510910
15 Patent Document 3: International Publication No.
2018/012492
NON-PATENT DOCUMENT
[0005]
20 Non-Patent Document 1: M. Salman Asif and four others,
“Flatcam: Replacing lenses with masks and computation”,
“2015 IEEE International Conference on Computer Vision
Workshop (ICCVW)”, 2015, pages 663-666
25 SUMMARY OF THE INVENTION
PROBLEMS TO BE SOLVED BY THE INVENTION
[0006]
However, in reality, the characteristics of the
imaging element may have a gap from design values due to
30 individual differences during manufacturing. That is,
there is a possibility that a correct restored image
3
SP371667WO00
cannot be obtained because the restored image is restored
using a restoration matrix that does not correspond to
(has a gap from) the actual characteristics.
[0007]
5 The present disclosure has been made in view of
such circumstances and is to enable generation of a more
accurate restored image.
SOLUTIONS TO PROBLEMS
10 [0008]
An imaging element according to one aspect of the
present technology is an imaging element including a
plurality of pixel output units each having a
configuration capable of independently setting incident
15 angle directivity for incident light incident without
through both of an imaging lens and a pinhole, in which
the plurality of pixel output units includes a plurality
of image restoration pixel output units arranged in a
matrix, at least some of the pixel output units having
20 the incident angle directivity both in a row direction
and in a column direction of the matrix, and a
unidirectional pixel output unit having the incident
angle directivity only in the row direction of the matrix
or only in the column direction of the matrix.
25 [0009]
An imaging device according to another aspect of
the present technology is an imaging device including an
imaging element including a plurality of pixel output
units each having a configuration capable of
30 independently setting incident angle directivity for
incident light incident without through both of an
4
SP371667WO00
imaging lens and a pinhole, the plurality of pixel output
units including a plurality of image restoration pixel
output units arranged in a matrix, at least some of the
pixel output units having the incident angle directivity
5 both in a row direction and in a column direction of the
matrix, and a unidirectional pixel output unit having the
incident angle directivity only in the row direction of
the matrix or only in the column direction of the matrix;
and a generation unit configured to generate, using an
10 output of the unidirectional pixel output unit, a
restoration matrix to be used when generating a restored
image from a detection image that is an output of the
plurality of image restoration pixel output units.
[0010]
15 An information processing method according to still
another aspect of the present technology is an
information processing method including generating a
restoration matrix to be used when generating a restored
image from a detection image that is an output of a
20 plurality of image restoration pixel output units, using
an output of a unidirectional pixel output unit, of an
imaging element including a plurality of pixel output
units each having a configuration capable of
independently setting incident angle directivity for
25 incident light incident without through both of an
imaging lens and a pinhole, the plurality of pixel output
units including the plurality of image restoration pixel
output units arranged in a matrix, at least some of the
pixel output units having the incident angle directivity
30 both in a row direction and in a column direction of the
matrix, and the unidirectional pixel output unit having
5
SP371667WO00
the incident angle directivity only in the row direction
of the matrix or only in the column direction of the
matrix.
[0011]
5 The imaging element according to one aspect of the
present technology includes a plurality of pixel output
units each having a configuration capable of
independently setting incident angle directivity for
incident light incident without through both of an
10 imaging lens and a pinhole, and the plurality of pixel
output units includes a plurality of image restoration
pixel output units arranged in a matrix, at least some of
the pixel output units having the incident angle
directivity both in a row direction and in a column
15 direction of the matrix, and a unidirectional pixel
output unit having the incident angle directivity only in
the row direction of the matrix or only in the column
direction of the matrix.
[0012]
20 The imaging device according to another aspect of
the present technology includes an imaging element
including a plurality of pixel output units each having a
configuration capable of independently setting incident
angle directivity for incident light incident without
25 through both of an imaging lens and a pinhole, the
plurality of pixel output units including a plurality of
image restoration pixel output units arranged in a
matrix, at least some of the pixel output units having
the incident angle directivity both in a row direction
30 and in a column direction of the matrix, and a
unidirectional pixel output unit having the incident
6
SP371667WO00
angle directivity only in the row direction of the matrix
or only in the column direction of the matrix; and a
generation unit configured to generate, using an output
of the unidirectional pixel output unit, a restoration
5 matrix to be used when generating a restored image from a
detection image that is an output of the plurality of
image restoration pixel output units.
[0013]
In the information processing method according to
10 still another aspect of the present technology, a
restoration matrix to be used when generating a restored
image from a detection image that is an output of a
plurality of image restoration pixel output units is
generated using an imaging element including a plurality
15 of pixel output units each having a configuration capable
of independently setting incident angle directivity for
incident light incident without through both of an
imaging lens and a pinhole, the plurality of pixel output
units including the plurality of image restoration pixel
20 output units arranged in a matrix, at least some of the
pixel output units having the incident angle directivity
both in a row direction and in a column direction of the
matrix, and a unidirectional pixel output unit having the
incident angle directivity only in the row direction of
25 the matrix or only in the column direction of the matrix,
and using an output of the unidirectional pixel output
unit.
EFFECTS OF THE INVENTION
30 [0014]
According to the present technology, an image of an
7
SP371667WO00
object can be captured or information can be processed.
Furthermore, according to the present technology, a more
accurate restored image can be generated.
5 BRIEF DESCRIPTION OF DRAWINGS
[0015]
Fig. 1 is a diagram for describing principles of
imaging in an imaging device to which the technology of
the present disclosure is applied.
10 Fig. 2 is a diagram for describing a difference in
configuration between a conventional imaging element and
an imaging element of the present disclosure.
Fig. 3 is a diagram for describing a first
configuration example of an imaging element.
15 Fig. 4 is a diagram for describing the first
configuration example of an imaging element.
Fig. 5 is a diagram for describing principles of
occurrence of incident angle directivity.
Fig. 6 is a diagram for describing changes in the
20 incident angle directivity using an on-chip lens.
Fig. 7 is a diagram for describing design of the
incident angle directivity.
Fig. 8 is a diagram for describing a relationship
between an object distance and a coefficient expressing
25 the incident angle directivity.
Fig. 9 is a diagram for describing a relationship
between a narrow-angle-of-view pixel and a wide-angle-ofview
pixel.
Fig. 10 is a diagram for describing a relationship
30 between a narrow-angle-of-view pixel and a wide-angle-ofview
pixel.
8
SP371667WO00
Fig. 11 is a diagram for describing a relationship
between a narrow-angle-of-view pixel and a wide-angle-ofview
pixel.
Fig. 12 is a diagram for describing a modification.
5 Fig. 13 is a diagram for describing a modification.
Fig. 14 is a diagram for describing a modification.
Fig. 15 is a diagram for describing an example of
changing an angle of view by applying a modification.
Fig. 16 is a diagram for describing an example of
10 combining pixels of a plurality of angles of view when
changing an angle of view by applying a modification.
Fig. 17 is a diagram for describing a modification.
Fig. 18 is a diagram for describing a modification.
Fig. 19 is a diagram for describing a modification.
15 Fig. 20 is a diagram illustrating an example of a
mask pattern by a light-shielding film.
Fig. 21 is a diagram for describing an example of
an error caused by a positional gap of the lightshielding
film.
20 Fig. 22 is a diagram for describing an example of
an error caused by a height gap of an on-chip lens.
Fig. 23 is a diagram for describing an example of
an error caused by a height gap of an on-chip lens.
Fig. 24 is a diagram illustrating a main
25 configuration example of a light-receiving unit.
Fig. 25 is a flowchart for describing an example of
a flow of generation processing.
Fig. 26 is a flowchart for describing an example of
a flow of imaging processing.
30 Fig. 27 is a diagram illustrating a main
configuration example of a calibration device to which
9
SP371667WO00
the technology of the present disclosure is applied.
Fig. 28 is a diagram illustrating an example of a
state of calibration.
Fig. 29 is a diagram illustrating an example of a
5 state of calibration.
Fig. 30 is a diagram illustrating an example of a
state of calibration.
Fig. 31 is a diagram illustrating an example of a
state of calibration.
10 Fig. 32 is a diagram illustrating an example of a
state of calibration.
Fig. 33 is a flowchart for describing an example of
a flow of calibration processing.
Fig. 34 is a diagram illustrating a main
15 configuration example of an imaging device to which the
technology of the present disclosure is applied.
Fig. 35 is a block diagram illustrating a main
configuration example of an imaging element.
Fig. 36 is a diagram illustrating an example of an
20 attachment.
Fig. 37 is a diagram illustrating a main
configuration example of an imaging device to which the
technology of the present disclosure is applied.
Fig. 38 is a diagram illustrating a main
25 configuration example of an imaging system to which the
technology of the present disclosure is applied.
Fig. 39 is a block diagram illustrating a main
configuration example of a computer.
30 MODE FOR CARRYING OUT THE INVENTION
[0016]
10
SP371667WO00
Hereinafter, modes for implementing the present
disclosure (hereinafter referred to as embodiments) will
be described. Note that the description will be given in
the following order.
5 1. Imaging Element Calibration
2. First Embodiment (Imaging Element)
3. Second Embodiment (Imaging Device)
4. Third Embodiment (Imaging Device)
5. Fourth Embodiment (Calibration System)
10 6. Appendix
[0017]
<1. Imaging Element Calibration>

Conventionally, an imaging element is typically
15 used in combination with an imaging lens that focuses
light on the imaging element. The imaging lens guides
light from an object plane to each pixel of the imaging
element so as to reproduce light intensity distribution
on the object plane, so that the imaging element can
20 obtain a detection signal at a level corresponding to the
light intensity distribution in each pixel and can obtain
a captured image of the object as a whole. However, in
this case, the physical size increases.
[0018]
25 Therefore, for example, imaging elements not using
an imaging lens (also referred to as imaging lens-free
imaging elements) as described in Patent Document 1,
Patent Document 2, and Non-Patent Document 1 have been
conceived. A detection image is generated in such an
30 imaging lens-free imaging element. A restored image is
restored by solving multidimensional simultaneous linear
11
SP371667WO00
equations (performing a matrix operation) using the
detection image and a restoration matrix according to
characteristics of the imaging element. This restoration
matrix is generated on the basis of, for example, design
5 values of an imaging element, and the like. Ideally, a
correct restored image can be restored using a correct
restoration matrix.
[0019]
However, in reality, the characteristics (incident
10 angle directivity and the like) of the imaging element
may have a gap from the design values due to individual
differences during manufacturing. That is, there is a
possibility that a correct restored image cannot be
obtained because the restored image is restored using a
15 restoration matrix that does not correspond to (has a gap
from) the actual characteristics.
[0020]
Therefore, an imaging element including a plurality
of pixel output units each having a configuration capable
20 of independently setting incident angle directivity for
incident light entering without passing through both of
an imaging lens and a pinhole, the plurality of pixel
output units including a plurality of image restoration
pixel output units arranged in a matrix, at least a part
25 of the pixel output units having the incident angle
directivity both in a row direction and in a column
direction of the matrix, and a unidirectional pixel
output unit having the incident angle directivity only in
the row direction of the matrix or only in the column
30 direction of the matrix, is used.
[0021]
12
SP371667WO00
By doing so, characteristics of the restoration
matrix can be more accurately and more easily calibrated.
That is, calibration can be performed to reduce a
difference between the actual characteristics of the
5 imaging element and the characteristics corresponding to
the restoration matrix. In other words, it is possible
to generate a restoration matrix further corresponding to
the characteristics of the actual imaging element and
generate a restored image using the restoration matrix.
10 Therefore, a more accurate restored image can be
generated.
[0022]
<2. First Embodiment>

15 Fig. 1 is a block diagram illustrating a main
configuration example of an imaging element to which the
present technology as described above is applied. An
imaging element 121 illustrated in Fig. 1 is configured
to perform processing regarding imaging of an object.
20 For example, the imaging element 121 captures an image of
the object and obtains data (electronic data) regarding
the captured image. At this time, the imaging element
121 can capture an image of the object without an imaging
lens, an optical filter such as a diffraction grating, or
25 a pinhole, and can obtain the data regarding the captured
image. For example, the imaging element 121 captures an
image of the object and obtains data (detection signal or
the like) from which data of the captured image can be
obtained by a predetermined operation.
30 [0023]
Note that, in the present specification, the
13
SP371667WO00
captured image is an image configured using pixel values
for forming an object image and visually recognizable by
a user. On the other hand, an image (called detection
image) configured using the detection signal that is a
5 detection result of incident light at a pixel unit output
of the imaging element 121 is an image that cannot be
visually recognized by the user as an image (that is, an
image from which the object cannot be visually
recognized) because the object image is not formed. That
10 is, the detection image is an image different from the
captured image. However, by performing a predetermined
operation for data of the detection image, as described
above, the captured image, that is, an image in which the
object image is formed and visually recognizable as an
15 image by the user (that is, an image from which the
object can be visually recognized) can be restored. This
restored captured image is referred to as a restored
image. That is, the detection image is an image
different from the restored image.
20 [0024]
Furthermore, in the present specification, an image
configuring the restored image and before synchronization
processing, color separation processing, and the like
(for example, demosaic processing) is also referred to as
25 a raw image. This raw image is also an image visually
recognizable by the user as an image (that is, the object
can be visually recognized), similarly to the captured
image. In other words, the detection image is an image
according to a color filter array but is an image
30 different from a raw image.
[0025]
14
SP371667WO00
However, in the case where the imaging element 121
has sensitivity only to invisible light such as infrared
light or ultraviolet light, the restored image (raw image
or captured image) may be an image that cannot be
5 visually recognized by the user as an image (an image
from which the object cannot be visually recognized).
Note that the above case depends on a wavelength range of
detected light. Therefore, the restored image can be
made to an image from which the object can be visually
10 recognized by converting the wavelength range into a
visible light range. In contrast, since the object image
is not formed in the detection image, the object image
cannot be made to an image from which the object can be
visually recognized only by converting the wavelength
15 range. Therefore, even in the case where the imaging
element 121 has sensitivity only to invisible light, an
image obtained by performing a predetermined operation
for the detection image as described above is referred to
as a restored image. Note that, hereinafter, the present
20 technology will be basically described using an example
in which the imaging element 121 receives visible light
unless otherwise specified.
[0026]
That is, the imaging element 121 can capture the
25 image of the object and obtain data regarding the
detection image.
[0027]
Such an imaging element 121 includes a plurality of
pixel output units each having a configuration capable of
30 independently setting incident angle directivity for
incident light entering without passing through both of
15
SP371667WO00
an imaging lens and a pinhole, and the plurality of pixel
output units includes a plurality of image restoration
pixel output units arranged in a matrix, at least a part
of the pixel output units having the incident angle
5 directivity both in a row direction and in a column
direction of the matrix, and a unidirectional pixel
output unit having the incident angle directivity only in
the row direction of the matrix or only in the column
direction of the matrix.
10 [0028]
By using such an imaging element 121, calibration
of the restoration matrix becomes possible, and a more
accurate restored image can be generated.
[0029]
15 As illustrated in Fig. 1, the imaging element 121
includes a light-receiving unit 111 and the other
processing unit 112. The light-receiving unit 111 and
the other processing unit 112 may be formed on the same
semiconductor substrate or may be formed on different
20 semiconductor substrates. For example, the semiconductor
substrate on which the light-receiving unit 111 is formed
and the semiconductor substrate on which the other
processing unit 112 is formed may be stacked.
[0030]
25 The light-receiving unit 111 includes a pixel
matrix including a plurality of pixels (pixel output
units), and in each pixel, receives light from the object
and photoelectrically converts the light, accumulates
electric charges according to the incident light, and
30 outputs the electric charge as a detection signal at
predetermined timing (outputs a detection image of an
16
SP371667WO00
analog signal).
[0031]
As illustrated in Fig. 1, the other processing unit
112 includes an A/D conversion unit 101, a restoration
5 matrix generation unit 102, a restoration matrix storage
unit 103, a restoration unit 104, an association unit
105, and an output unit 106.
[0032]
The A/D conversion unit 101 is configured to
10 perform processing regarding A/D conversion of a signal.
For example, the A/D conversion unit 101 performs A/D
conversion for the detection signal (detection image) of
the analog signal output from the light-receiving unit
111, and supplies the converted detection signal to the
15 restoration unit 104 and the association unit 105 as a
detection signal (detection image) of digital data.
[0033]
Note that the configuration of the A/D conversion
unit 101 is arbitrary. For example, the A/D conversion
20 unit 101 may be configured by one A/D conversion unit and
may perform A/D conversion for the detection signals of
all of pixels of the light-receiving unit 111, using the
one A/D conversion unit. Furthermore, for example, the
A/D conversion unit 101 may include an A/D conversion
25 unit for each column or each row of the pixel matrix of
the light-receiving unit 111, and perform A/D conversion
for the detection signals read from pixels in each column
or each row, using the A/D conversion unit of the each
column or the each row. Furthermore, for example, the
30 A/D conversion unit 101 may include an A/D conversion
unit for each area of the pixel matrix of the light17
SP371667WO00
receiving unit 111, and perform A/D conversion for the
detection signals read from pixels in each area, using
the A/D conversion unit of the each area. Furthermore,
for example, the A/D conversion unit 101 may include an
5 A/D conversion unit for each pixel of the light-receiving
unit 111, and perform A/D conversion for the detection
signal read from each pixel, using the A/D conversion
unit of the each pixel.
[0034]
10 The restoration matrix generation unit 102 is
configured to perform processing regarding generation
(calibration) of the restoration matrix. For example,
the restoration matrix generation unit 102 generates
(calibrates) the restoration matrix, using information
15 (hereinafter also referred to as calibration information)
regarding calibration of incident angle directivity of
each pixel of the imaging element 121. The restoration
matrix generation unit 102 supplies the generated
restoration matrix to the restoration matrix storage unit
20 103.
[0035]
Note that the calibration information is supplied
from an outside of the imaging element 121, for example.
Furthermore, the calibration information includes, for
25 example, information directly or indirectly indicating a
gap between the actual incident angle directivity of the
imaging element 121 and its design value.
[0036]
Furthermore, the method of generating the
30 restoration matrix is arbitrary. For example, the
restoration matrix generation unit 102 may acquire the
18
SP371667WO00
calibration information indicating the actual incident
angle directivity (a detection signal level according to
an incident angle of incident light) of each pixel of the
imaging element 121, and generate the restoration matrix
5 corresponding to the actual incident angle directivity of
the imaging element 121 on the basis of the calibration
information. Furthermore, for example, the restoration
matrix generation unit 102 may acquire the calibration
information indicating the gap between the actual
10 incident angle directivity (the detection signal level
according to an incident angle of incident light) of each
pixel of the imaging element 121 and the design value,
and calibrate the restoration matrix corresponding to the
design value on the basis of the calibration information
15 to generate the restoration matrix corresponding to the
actual incident angle directivity of the imaging element
121.
[0037]
Furthermore, the restoration matrix generation unit
20 102 may read the restoration matrix stored in the
restoration matrix storage unit 103 and calibrate
(update) the generated restoration matrix with the read
restoration matrix on the basis of the calibration
information. In that case, the restoration matrix
25 generation unit 102 supplies the updated restoration
matrix to the restoration matrix storage unit 103.
[0038]
The restoration matrix storage unit 103 includes a
storage medium (not illustrated) and performs processing
30 regarding storage of the restoration matrix. This
storage medium is arbitrary and may be, for example, a
19
SP371667WO00
hard disk or a semiconductor memory. The restoration
matrix storage unit 103 acquires the restoration matrix
supplied from the restoration matrix generation unit 102
and stores (retains) the restoration matrix in the
5 storage medium. Note that, in the case where the
restoration matrix storage unit 103 has already stored
the restoration matrix in the storage medium, the
restoration matrix storage unit 103 may overwrite
(update) the stored restoration matrix with the
10 restoration matrix supplied from the restoration matrix
generation unit 102. Furthermore, the restoration matrix
storage unit 103 reads the restoration matrix stored in
the storage medium as necessary (for example, at
predetermined timing or in response to a request), and
15 supplies the read restoration matrix to the restoration
unit 104 or the association unit 105, or both of the
restoration unit 104 and the association unit 105.
[0039]
The restoration unit 104 is configured to perform
20 processing regarding generation of a restored image. For
example, the restoration unit 104 acquires the
restoration matrix supplied from the restoration matrix
storage unit 103 and performs a predetermined operation
using the restoration matrix, for the detection image
25 (detection signal) supplied from the A/D conversion unit
101, thereby generating the restored image. That is, the
restoration unit 104 generates the restored image, using
the restoration matrix calibrated so as to correspond to
the actual incident angle directivity. The restoration
30 unit 104 supplies the generated restored image (correctly
restored restored image) to the output unit 106 as output
20
SP371667WO00
data. Note that the restoration unit 104 may apply
arbitrary image processing such as gamma correction (γ
correction) and white balance control to the restored
image. Furthermore, the restoration unit 104 may convert
5 a format of data of the restored image or compress the
data of the restored image by a predetermined compression
method such as the Joint Photographic Experts Group
(JPEG), Tagged Image File Format (TIFF), or Graphics
Interchange Format (GIF).
10 [0040]
The association unit 105 is configured to perform
processing regarding data association. For example, the
association unit 105 acquires the detection image
(detection signal) supplied from the A/D conversion unit
15 101. Furthermore, the association unit 105 acquires the
restoration matrix supplied from the restoration matrix
storage unit 103. Moreover, the association unit 105
associates the detection image with the restoration
matrix. The association unit 105 supplies the associated
20 data (detection image and restoration matrix) to the
output unit 106 as output data.
[0041]
Here, the term “associate” means, for example,
making one information usable (linkable) when processing
25 the other information (data, commands, programs, and the
like). That is, pieces of information associated with
each other may be collected as one file or may be
individual information. For example, information B
associated with information A may be transmitted on a
30 different transmission path from the information A.
Furthermore, for example, information B associated with
21
SP371667WO00
information A may be recorded on a different recording
medium (or another recording area of the same recording
medium) from the information A. Note that this
“association” may be a part of information instead of
5 entire information. For example, an image and
information corresponding to the image may be associated
with each other in an arbitrary unit such as a plurality
of frames, one frame, or a part in a frame.
[0042]
10 More specifically, for example, actions such as
assigning the same ID (identification information) to a
plurality of pieces of information, recording a plurality
of pieces of information to the same recording medium,
storing a plurality of pieces of information in the same
15 folder, storing a plurality of pieces of information in
the same file (assigning one to the other as metadata),
embedding a plurality of pieces of information in the
same stream, for example, embedding metadata in an image
like a digital watermark are included in the “associate”.
20 [0043]
The output unit 106 outputs the output data
(restored image) supplied from the restoration unit 104
or the output data (detection image and restoration
matrix) supplied from the association unit 105 to the
25 outside of the imaging element 121.
[0044]
Note that either one of the restoration unit 104 or
the association unit 105 may be omitted. That is, only
one of the restoration unit 104 and the association unit
30 105 may be formed in the imaging element 121.
[0045]
22
SP371667WO00

Next, the imaging element 121 will be described
with reference to Figs. 2 to 20.
[0046]
5
In the present specification, the present
technology will be described using the term “pixel” (or
“pixel output unit”). In the present specification, the
“pixel” (or “pixel output unit”) refers to a division
10 unit including at least one physical configuration
capable of receiving incident light of the imaging
element 121 independently of the other pixels, in a
region (also referred to as pixel region) where the
physical configuration for receiving light is formed.
15 The physical configuration capable of receiving light is,
for example, a photoelectric conversion element, and is,
for example, a photodiode (PD). The number of physical
configurations (for example, photodiodes) formed in one
pixel is arbitrary and may be one or plural. The type,
20 size, shape, and the like are also arbitrary.
[0047]
Furthermore, the physical configuration in units of
“pixels” includes not only the above-described “physical
configuration capable of receiving light” but also all
25 physical configurations regarding receiving incident
light, such as an on-chip lens, a light-shielding film, a
color filter, a flattening film, and an anti-reflection
film. Moreover, a configuration such as a readout
circuit may be included. That is, the physical
30 configuration in pixel units may be any configuration.
[0048]
23
SP371667WO00
Furthermore, the detection signal read from the
“pixel” (that is, the physical configuration in pixel
units) may be referred to as “a detection signal in pixel
units (or in pixel output units)” or the like. Moreover,
5 the detection signal in pixel units (or in pixel output
units) is also referred to as “pixel unit detection
signal (or pixel output unit detection signal)”.
Furthermore, the pixel unit detection signal is also
referred to as “pixel output”. Moreover, a value of the
10 pixel output is also referred to as “output pixel value”.
[0049]
The imaging element 121 includes a plurality of the
pixel output units each configured to receive incident
light incident without through both of an imaging lens
15 and a pinhole, and output one detection signal indicating
an output pixel value modulated according to an incident
angle of the incident light. Therefore, the value
(output pixel value) of the detection signal in pixel
units of the imaging element 121 can have incident angle
20 directivity indicating directivity with respect to the
incident angle of the incident light from the object
independently of the others. For example, each pixel
unit (pixel output unit) of the imaging element 121 may
have a configuration capable of independently setting the
25 incident angle directivity of the output pixel value, the
incident angle directivity indicating directivity with
respect to the incident angle of the incident light from
the object. In other words, in the imaging element 121,
the incident angle directivities of the output pixel
30 values of at least two pixel units can be made different
from each other.
24
SP371667WO00
[0050]
Note that, as described above, the number of
“physical configurations that can receive light” included
in the “pixel (or pixel output unit)” is arbitrary.
5 Therefore, the pixel unit detection signal may be a
detection signal obtained by a single “physical
configuration capable of receiving light” or may be a
detection signal obtained by a plurality of the “physical
configurations capable of receiving light”.
10 [0051]
Furthermore, a plurality of the pixel unit
detection signals (output pixel values) can be combined
into one signal (one value) in an arbitrary stage. For
example, the output pixel values of a plurality of pixels
15 may be added in a state of an analog signal or may be
converted into a digital signal and then added.
[0052]
Furthermore, after the detection signal is read
from the imaging element 121, that is, in the detection
20 image, a plurality of the detection signals can be
combined into a single signal, or a single detection
signal can be divided into a plurality of detection
signals. That is, the resolution (the number of data) of
the detection image is variable. For example, to improve
25 sensitivity, the detection signals of a plurality of
pixels having the same incident angle directivity can be
added together.
[0053]
By the way, hereinafter, description will be given
30 on the assumption that the imaging element 121 includes a
pixel region where a plurality of pixels is arranged in a
25
SP371667WO00
matrix (a pixel array is formed) unless otherwise
specified, for convenience of description. Note that an
array pattern of pixels (or pixel output units) of the
imaging element 121 is arbitrary and is not limited to
5 this example. For example, the pixels (or pixel output
units) may be arranged in a honeycomb manner.
Furthermore, for example, the pixels (or pixel output
units) may be arranged in one row (or one column). That
is, the imaging element 121 may be a line sensor.
10 [0054]
Note that the wavelength range in which (the pixels
of) the imaging element 121 has sensitivity is arbitrary.
For example, (the pixels of) the imaging element 121 may
have sensitivity to visible light, may have sensitivity
15 to invisible light such as infrared light or ultraviolet
light, or may have sensitivity to both the visible light
and invisible light. For example, in a case where the
imaging element detects far-infrared light that is
invisible light, a thermograph (an image representing
20 heat distribution) can be generated using a captured
image obtained by the imaging element. However, in the
case of the imaging element with an imaging lens, glass
is difficult to transmit far-infrared light. Therefore,
an expensive imaging lens formed using a special material
25 is required, which may increase a manufacturing cost.
Since the imaging element 121 can capture an image of the
object without through an imaging lens or the like and
obtain data regarding the captured image, the pixels of
the imaging element 121 are made able to detect far30
infrared light, thereby suppressing an increase in
manufacturing cost. That is, an image of far-infrared
26
SP371667WO00
light can be captured at a lower cost (a thermograph can
be obtained at a lower cost). In other words, the
restored image may be an image of visible light or an
image of invisible light (for example, (far) infrared
5 light or ultraviolet light).
[0055]

As described above, the imaging element 121 can
obtain detection signals for a plurality of pixel output
10 units (a plurality of pixel output unit detection
signals). Then, the incident angle directivities of at
least two pixel output unit detection signals can be made
different from each other.
[0056]
15 Here, the “incident angle directivity” refers to a
light-receiving sensitivity characteristic according to
the incident angle of the incident light, that is,
detection sensitivity to the incident angle of the
incident light. For example, there are some cases where
20 the detection sensitivity varies depending on the
incident angle even if the incident light has the same
light intensity. Such deviation of the detection
sensitivity (including a case where there is no
deviation) is referred to as “incident angle
25 directivity”.
[0057]
For example, when incident lights having the same
light intensity enter the physical configurations of two
pixel output units at the same incident angle, signal
30 levels (detection signal levels) of the detection signals
of the pixel output units may be different values from
27
SP371667WO00
each other depending on the incident angle directivities.
(Each pixel output unit of) the imaging element 121 has a
physical configuration having such characteristics.
[0058]
5 The incident angle directivity may be implemented
by any method. For example, the incident angle
directivity may be implemented by providing a lightshielding
film in front of (on the light incident side
of) a photoelectric conversion element (photodiode or the
10 like) of an imaging element having a basic structure
similar to a typical imaging element such as a
complementary metal oxide semiconductor (CMOS) image
sensor.
[0059]
15 When an image is captured only by a typical imaging
element including pixels having the same incident angle
directivity, light with approximately the same light
intensity enters all the pixels of the imaging element,
and a formed image of the object cannot be obtained.
20 Therefore, generally, an imaging lens or a pinhole is
provided in front of the imaging element (on the light
incident side). For example, by providing an imaging
lens, light from the object plane can be formed on an
imaging plane of the imaging element. Therefore, the
25 imaging element can obtain a detection signal at a level
corresponding to the formed image of the object at each
pixel (that is, the imaging element can obtain the formed
captured image of the object). However, in this case,
the size physically becomes large, and it may be
30 difficult to reduce the size of the device. Furthermore,
in the case of providing a pinhole, the size can be
28
SP371667WO00
reduced as compared with the case of providing an imaging
lens. However, since the amount of light entering the
imaging element is reduced, measures such as making an
exposure time long or increasing a gain are
5 indispensable, and blurring tends to occur in capturing
an image of the object at a high speed or natural color
expression may not be able to be achieved.
[0060]
In contrast, the imaging element 121 includes a
10 plurality of the pixel output units each configured to
receive incident light incident without through both of
an imaging lens and a pinhole, and output one detection
signal indicating an output pixel value modulated
according to an incident angle of the incident light.
15 Note that the incident angle directivities of the pixel
output units may be made different from each other (the
light-receiving sensitivity characteristic according to
the incident angle of the incident light may differ for
each pixel output unit), some pixels may have the same
20 light-receiving sensitivity characteristic, or some
pixels may have different light-receiving sensitivity
characteristics
[0061]
For example, in Fig. 2, assuming that a light
25 source forming an object plane 131 is a point light
source, light beams with the same light intensity emitted
from the same point light source enter all the pixels but
the light beams are incident at a different incident
angle for each pixel in the imaging element 121. Then,
30 since the pixels of the imaging element 121 have
different incident angle directivities, the pixels detect
29
SP371667WO00
the light beams with the same light intensity, with
different sensitivities. That is, a detection signal at
a different signal level is detected for each pixel.
[0062]
5 More specifically, the sensitivity characteristic
according to the incident angle of the incident light
received by each pixel of the imaging element 121, that
is, the incident angle directivity according to the
incident angle in each pixel is expressed by a
10 coefficient representing the light-receiving sensitivity
according to the incident angle, and the signal level
(also referred to as detection signal level) of the
detection signal according to the incident light in each
pixel is obtained by multiplying the light intensity by
15 the coefficient set corresponding to the light-receiving
sensitivity according to the incident angle of the
incident light.
[0063]
More specifically, as illustrated in the upper left
20 part in Fig. 2, detection signal levels DA, DB, and DC at
positions Pa, Pb, and Pc are expressed by the following
equations (1) to (3), respectively.
[0064]
[Math. 1]
25
30
SP371667WO00
[0065]
Here, α1 is a coefficient set according to the
incident angle of a light beam from a point light source
PA on the object plane 131 to be restored at the position
5 Pa on the imaging element 121. Furthermore, β1 is a
coefficient set according to the incident angle of a
light beam from a point light source PB on the object
plane 131 to be restored at the position Pa on the
imaging element 121. Moreover, γ1 is a coefficient set
10 according to the incident angle of a light beam from a
point light source PC on the object plane 131 to be
restored at the position Pa on the imaging element 121.
[0066]
As illustrated in the equation (1), the detection
15 signal level DA at the position Pa is expressed by a sum
(combined value) of a product of the light intensity “a”
of the light beam from the point light source PA at the
position Pa and the coefficient α1, a product of the
light intensity “b” of the light beam from the point
20 light source PB at the position Pa and the coefficient
β1, and a product of the light intensity “c” of the light
beam from the point light source PC at position Pa and
the coefficient γ1. Hereinafter, the coefficients αx,
βx, and γx (x is a natural number) are collectively
25 referred to as a coefficient set.
[0067]
Similarly, a coefficient set α2, β2, and γ2 in the
equation (2) is a coefficient set set according to the
incident angle of the light beams from the point light
30 sources PA, PB, and PC on the object plane 131 to be
restored at the position Pb on the imaging element 121.
31
SP371667WO00
That is, as illustrated in the above-described equation
(2), the detection signal level DB at the position Pb is
expressed by a sum (combined value) of a product of the
light intensity “a” of the light beam from the point
5 light source PA at the position Pb and the coefficient
α2, a product of the light intensity “b” of the light
beam from the point light source PB at the position Pb
and the coefficient β2, and a product of the light
intensity “c” of the light beam from the point light
10 source PC at position Pb and the coefficient γ2.
Furthermore, the coefficients α3, β3, and γ3 in the
equation (3) are a coefficient set set according to the
incident angle of the light beams from the point light
sources PA, PB, and PC on the object plane 131 to be
15 restored at the position Pc on the imaging element 121.
That is, as illustrated in the above-described equation
(3), the detection signal level DC at the position Pc is
expressed by a sum (combined value) of a product of the
light intensity “a” of the light beam from the point
20 light source PA at the position Pc and the coefficient
α3, a product of the light intensity “b” of the light
beam from the point light source PB at the position Pc
and the coefficient β3, and a product of the light
intensity “c” of the light beam from the point light
25 source PC at position Pc and the coefficient γ3.
[0068]
As described above, these detection signal levels
are mixture of the light intensities of the light beams
emitted from the point light sources PA, PB, and PC, and
30 thus are different from a formed image of the object.
That is, the detection signal level illustrated in the
32
SP371667WO00
upper right part in Fig. 2 is not the detection signal
level corresponding to an image in which the image of the
object is formed (captured image), and thus is different
from the pixel value illustrated in the lower right part
5 in Fig. 2 (in general, the signal levels do not match).
[0069]
Note that the pixel values at the positions Pa, Pb,
and Pc as illustrated in the lower right part in Fig. 2
can be obtained by configuring simultaneous equations
10 using the coefficient set α1, β1, and γ1, the coefficient
set α2, β2, and γ2, the coefficient set α3, β3, and γ3,
and the detection signal levels DA, DB, and DC, and
solving the simultaneous equations (1) to (3) with a, b,
and c as variables. As a result, a restored image (an
15 image in which the image of the object is formed), which
is a set of pixel values, is restored.
[0070]
With such a configuration, the imaging element 121
can output, in each pixel, one detection signal
20 indicating the output pixel value modulated according to
the incident angle of the incident light without
requiring an optical filter using an imaging lens, a
diffraction grating, and the like, or a pinhole or the
like. As a result, the optical filter using an imaging
25 lens, a diffraction grating, and the like, or the pinhole
or the like is not an indispensable configuration.
Therefore, the height of the imaging device, that is, the
thickness in the light incident direction in the
configuration for implementing the imaging function can
30 be reduced.
[0071]
33
SP371667WO00

The left part in Fig. 3 illustrates a front view of
a part of a pixel array unit of a typical imaging
element, and the right part in Fig. 3 illustrates a front
5 view of a part of a pixel array unit of the imaging
element 121. Note that Fig. 3 illustrates an example of
the pixel array units in which the numbers of pixels in a
horizontal direction × a vertical direction are six
pixels × 6 pixels. However, the configuration of the
10 numbers of pixels is not limited to the example.
[0072]
The incident angle directivity can be formed by,
for example, a light-shielding film. In the typical
imaging element 121, pixels 121a having the same incident
15 angle directivity are arranged in an array manner, as
illustrated in the left example in Fig. 3. Meanwhile, in
the imaging element 121 in the right example in Fig. 3, a
light-shielding film 121b, which is one of modulation
elements, is provided for each pixel 121a so as to cover
20 a part of a light-receiving region of the photodiode.
The incident light entering each pixel 121a is optically
modulated according to the incident angle. Then, for
example, by providing the light-shielding film 121b in a
different range for each pixel 121a, the light-receiving
25 sensitivity with respect to the incident angle of the
incident light becomes different for each pixel 121a, and
each pixel 121a has different incident angle directivity.
[0073]
For example, a pixel 121a-1 and a pixel 121a-2 have
30 different ranges to be shielded from light by provided
light-shielding films 121b-1 and 121b-2 (at least one of
34
SP371667WO00
regions (positions) to be shielded from light or areas to
be shielded from light are different). That is, in the
pixel 121a-1, the light-shielding film 121b-1 is provided
so as to shield light in a part of a left side of a
5 light-receiving region of the photodiode by a
predetermined width, and in the pixel 121a-2, the lightshielding
film 121b-2 is provided so as to shield light
in a part of a right side of a light-receiving region by
a width wider in the horizontal direction than the light10
shielding film 121b-1. Similarly, in the other pixels
121a, the light-shielding films 121b are provided so as
to shield light in different ranges in the lightreceiving
region for respective pixels, and are randomly
arranged in the pixel array.
15 [0074]
Note that the amount of receivable light becomes
smaller as the ratio of covering the light-receiving
region of each pixel becomes larger. Therefore, the
range of the light-shielding film 121b is desirably set
20 to the area by which a desired amount of light can be
secured. The area of the light-shielding film 121b may
be limited to about 3/4 of the maximum light-receivable
range, for example. By doing so, an amount of light
equal to or larger than a desired amount can be secured.
25 Note that a minimum amount of light can be received by
each pixel as long as a range not shielded from light
with a width corresponding to a wavelength of received
light is provided. That is, for example, in the case of
a B pixel (blue pixel), the wavelength is about 500 nm.
30 The minimum amount of light can be received unless the B
pixel is shielded from light by the width corresponding
35
SP371667WO00
to the wavelength or larger.
[0075]

A configuration example of the imaging element 121
5 in this case will be described with reference to Fig. 4.
The upper part in Fig. 4 is a side-sectional view of the
imaging element 121 and the middle part in Fig. 4 is a
top view of the imaging element 121. Furthermore, the
side-sectional view in the upper part in Fig. 4 is an AB
10 cross section in the middle part in Fig. 4. Moreover,
the lower part in Fig. 4 illustrates a circuit
configuration example of the imaging element 121.
[0076]
The imaging element 121 having the configuration
15 illustrated in Fig. 4 includes a plurality of the pixel
output units each configured to receive incident light
incident without through both of an imaging lens and a
pinhole, and output one detection signal indicating an
output pixel value modulated according to the incident
20 angle of the incident light. For example, the imaging
element 121 has a configuration for making the incident
angle directivities indicating directivities with respect
to the incident angle of the incident light from the
object different from each other, the incident angle
25 directivities being of the output pixel values of at
least two pixel output units among the plurality of pixel
output units. Furthermore, the imaging element 121 in
this case has a configuration in which the plurality of
pixel output units is capable of independently setting,
30 for each pixel output unit, the incident angle
directivity indicating the directivity with respect to
36
SP371667WO00
the incident angle of the incident light from the object.
[0077]
In the imaging element 121 in the upper part in
Fig. 4, the incident light is incident from an upper side
5 to a lower side in Fig. 4. Adjacent pixels 121a-15 and
121a-16 are so-called back-illuminated-type pixels
provided with a wiring layer Z12 in the lowermost layer
and a photoelectric conversion layer Z11 on the wiring
layer Z12 in Fig. 4.
10 [0078]
Note that in a case where it is not necessary to
distinguish the pixels 121a-15 and 121a-16, they are
simply referred to as pixel(s) 121a, and other
configurations are also similarly referred to.
15 Furthermore, Fig. 4 is a side view and a top view of two
pixels configuring the pixel array of the imaging element
121. It goes without saying that a larger number of
pixels 121a are arranged but illustration is omitted.
[0079]
20 Moreover, the pixels 121a-15 and 121a-16
respectively include photodiodes 121e-15 and 121e-16 in
the photoelectric conversion layer Z11. Furthermore, onchip
lenses 121c-15 and 121c-16 and color filters 121d-15
and 121d-16 are respectively formed on the photodiodes
25 121e-15 and 121e-16 from above.
[0080]
The on-chip lenses 121c-15 and 121c-16 focus the
incident light on the photodiodes 121e-15 and 121e-16.
[0081]
30 The color filters 121d-15 and 121d-16 are optical
filters that transmit light having specific wavelengths
37
SP371667WO00
such as red, green, blue, infrared, and white. Note
that, in the case of white, the color filters 121d-15 and
121d-16 may be transparent filters or may be omitted.
[0082]
5 Light-shielding films 121p-15 to 121p-17 are formed
at boundaries between pixels in the photoelectric
conversion layer Z11 of the pixels 121a-15 and 121a-16
and suppress crosstalk between adjacent pixels. Note
that the light-shielding films 121p-15 to 121p-17 may be
10 omitted.
[0083]
Furthermore, light-shielding films 121b-15 and
121b-16, which are one of modulation elements, partially
shield a light-receiving surface S, as illustrated in the
15 upper and middle parts in Fig. 4. Since a part of the
light-receiving surface S is shielded by the lightshielding
film 121b, the incident light entering the
pixel 121a is optically modulated according to the
incident angle. Since the pixel 121a detects the
20 optically modulated incident light, the pixel 121a has
incident angle directivity. Different ranges are
respectively shielded by the light-shielding films 121b-
15 and 121b-16 on the light-receiving surface S of the
photodiodes 121e-15 and 121e-16 in the pixels 121a-15 and
25 121a-16. Thereby, different incident angle directivity
is set for each pixel. Note that it is not limited to
the case where the shielded ranges are different among
all the pixels 121a of the imaging element 121, and some
pixels 121a having the same shielded range may exist.
30 [0084]
With the configuration illustrated in the upper
38
SP371667WO00
part in Fig. 4, a right end of the light-shielding film
121b-15 and an upper end of the light-shielding film
121b-15 are connected and a left end of the lightshielding
film 121p-16 and an upper end of the light-
5 shielding film 121p-16 are connected to form L shapes as
viewed from the side.
[0085]
Furthermore, the light-shielding films 121b-15 to
CLAIMS
1. An imaging element comprising:
a plurality of pixel output units each having a
5 configuration capable of independently setting incident
angle directivity for incident light incident without
through both of an imaging lens and a pinhole, wherein
the plurality of pixel output units includes
a plurality of image restoration pixel output
10 units arranged in a matrix, at least some of the pixel
output units having the incident angle directivity both
in a row direction and in a column direction of the
matrix, and
a unidirectional pixel output unit having the
15 incident angle directivity only in the row direction of
the matrix or only in the column direction of the matrix.
2. The imaging element according to claim 1, wherein
the unidirectional pixel output unit is formed in a
20 region other than an effective pixel region that is a
region in which the image restoration pixel output units
are formed.
3. The imaging element according to claim 2, wherein
25 the unidirectional pixel output unit is formed in
an external region outside the effective pixel region.
4. The imaging element according to claim 3, wherein
the unidirectional pixel output unit having the
30 incident angle directivity only in the column direction
of the matrix is formed in the external region outside
179
SP371667WO00
the rectangular effective pixel region in the row
direction of the matrix.
5. The imaging element according to claim 3, wherein
5 the unidirectional pixel output unit having the
incident angle directivity only in the row direction of
the matrix is formed in the external region outside the
rectangular effective pixel region in the column
direction of the matrix.
10
6. The imaging element according to claim 3, wherein
at least one of the plurality of image restoration
pixel output units formed in the effective pixel region
has the same incident angle directivity as at least one
15 of a plurality of the unidirectional pixel output units
formed in the external regions in the row direction and
in the column direction of the matrix.
7. The imaging element according to claim 6, wherein
20 at least one of the image restoration pixel output
units in each row of the matrix has the same incident
angle directivity in the column direction of the matrix
as at least one of the unidirectional pixel output units
provided in the same row as the each row, of the external
25 region.
8. The imaging element according to claim 6, wherein
at least one of the image restoration pixel output
units in each column of the matrix has the same incident
30 angle directivity in the row direction of the matrix as
at least one of the unidirectional pixel output units
180
SP371667WO00
provided in the same column as the each column, of the
external region.
9. The imaging element according to claim 6, wherein
5 at least one of the image restoration pixel output
units in each row of the matrix has the same incident
angle directivity in the column direction of the matrix
as at least one of the unidirectional pixel output units
provided in another row different from the each row, of
10 the external region.
10. The imaging element according to claim 6, wherein
at least one of the image restoration pixel output
units in each column of the matrix has the same incident
15 angle directivity in the row direction of the matrix as
at least one of the unidirectional pixel output units
provided in another column different from the each
column, of the external region.
20 11. The imaging element according to claim 6, wherein
all of the image restoration pixel output units
formed in the effective pixel region have the same
incident angle directivity as at least one of a plurality
of the unidirectional pixel output units formed in the
25 external regions in the row direction and in the column
direction of the matrix.
12. The imaging element according to claim 6, wherein
some image restoration pixel output units in a
30 plurality of the image restoration pixel output units
formed in the effective pixel region have the same
181
SP371667WO00
incident angle directivity as at least one of a plurality
of the unidirectional pixel output units formed in the
external regions in the row direction and in the column
direction of the matrix.
5
13. The imaging element according to claim 12, wherein
the incident angle directivities in the row
direction and in the column direction of the matrix, of
the other image restoration pixel output units than the
10 some image restoration pixel output units formed in the
effective pixel region, are able to be estimated on a
basis of the incident angle directivity of the
unidirectional pixel output unit formed in the external
region.
15
14. The imaging element according to claim 3, wherein
at least one of the unidirectional pixel output
unit or the image restoration pixel output unit is
arranged in an arrangement order based on predetermined
20 regularity of the incident angle directivity.
15. The imaging element according to claim 1, wherein
the configuration capable of independently setting
the incident angle directivity for each pixel output unit
25 is configured using a light-shielding film.
16. The imaging element according to claim 1, further
comprising:
a generation unit configured to generate a
30 restoration matrix to be used when generating a restored
image from a detection image obtained in the image
182
SP371667WO00
restoration pixel output unit, using calibration
information that is information regarding calibration of
the incident angle directivity of the image restoration
pixel output unit generated using the unidirectional
5 pixel output unit.
17. The imaging element according to claim 16, further
comprising:
a restoration matrix storage unit configured to
10 store the restoration matrix generated by the generation
unit.
18. An imaging device comprising:
an imaging element including
15 a plurality of pixel output units each having a
configuration capable of independently setting incident
angle directivity for incident light incident without
through both of an imaging lens and a pinhole,
the plurality of pixel output units including
20 a plurality of image restoration pixel output
units arranged in a matrix, at least some of the pixel
output units having the incident angle directivity both
in a row direction and in a column direction of the
matrix, and
25 a unidirectional pixel output unit having the
incident angle directivity only in the row direction of
the matrix or only in the column direction of the matrix;
and
a generation unit configured to generate, using an
30 output of the unidirectional pixel output unit, a
restoration matrix to be used when generating a restored
183
SP371667WO00
image from a detection image that is an output of the
plurality of image restoration pixel output units.
19. The imaging device according to claim 18, wherein
5 the generation unit generates the restoration
matrix, using calibration information that is information
generated using the output of the unidirectional pixel
output unit, and is regarding calibration of the incident
angle directivity of the image restoration pixel output
10 unit.
20. An information processing method comprising:
generating, using an output of a unidirectional
pixel output unit, a restoration matrix to be used when
15 generating a restored image from a detection image that
is an output of a plurality of image restoration pixel
output units, of
an imaging element including
a plurality of pixel output units each having a
20 configuration capable of independently setting incident
angle directivity for incident light incident without
through both of an imaging lens and a pinhole,
the plurality of pixel output units including
the plurality of image restoration pixel
25 output units arranged in a matrix, at least some of the
pixel output units having the incident angle directivity
both in a row direction and in a column direction of the
matrix, and
the unidirectional pixel output unit having
30 the incident angle directivity only in the row direction
of the matrix or only in the column direction of the
184
SP371667WO00
matrix.
21. The information processing method according to
claim 20, wherein
5 the generating a restoration matrix includes
generating the restoration matrix using calibration
information generated using an output from the
unidirectional pixel output unit.
10 22. The information processing method according to
claim 21, further comprising:
measuring a positional gap of a light-shielding
film having a configuration capable of independently
setting the incident angle directivity of the imaging
15 element for each pixel output unit, and generating the
calibration information including information regarding
the positional gap of the light-shielding film.
23. The information processing method according to
20 claim 21, further comprising:
measuring a height gap of an on-chip lens of the
imaging element, and generating the calibration
information including information regarding the height
gap of an on-chip lens.
25
24. The information processing method according to
claim 21, further comprising:
in the unidirectional pixel output unit, measuring
a detection signal value for each of a plurality of
30 incident angles different from one another of the
incident light, and generating the calibration
185
SP371667WO00
information using each measurement result.
25. The information processing method according to
claim 21, further comprising:
5 in the unidirectional pixel output unit, measuring
a detection signal value for a predetermined incident
angle of the incident light, and generating the
calibration information using a measurement result and a
reference value.
10
26. The information processing method according to
claim 21, further comprising:
measuring a detection signal value of each of a
plurality of the unidirectional pixel output units, and
15 generating the calibration information using a
statistical value of measurement results.