The scope of the claims
[Claim 1]
an imaging unit that has a pixel region in which a plurality of pixels are arranged and that reads and outputs pixel signals from the pixels included in the pixel region;
a readout unit control unit that controls the readout unit set as part of the pixel region;
a first readout unit setting unit that sets a first readout unit for reading out the pixel signal from the pixel region for recognition processing that has learned the teacher data for each readout unit;
a second readout unit setting unit for setting a second readout unit for reading out the pixel signal from the pixel region for output to a subsequent stage;
an arbitration unit that arbitrates between the first readout unit and the second readout unit;
with
　The read unit control unit
setting the read unit through arbitration by the arbitration unit;
Imaging device.
[Claim 2]
The arbitration department
The arbitration is performed by the AND of the first readout unit and the second readout unit,
The imaging device according to claim 1 .
[Claim 3]
The arbitration department
　Perform the arbitration based on the result of the recognition process,
The imaging device according to claim 1 .
[Claim 4]
The arbitration department
If the result of the recognition process indicates recognition of a moving object, selecting the second readout unit,
The imaging device according to claim 3.
[Claim 5]
The arbitration department
selecting the second reading unit when the result of the recognition processing indicates a recognition certainty factor equal to or lower than a threshold;
The imaging device according to claim 3.
[Claim 6]
The arbitration department
performing the arbitration based on the pixel signal read from the second readout unit;
The imaging device according to claim 1 .
[Claim 7]
The arbitration department
selecting the second readout unit when the luminance based on the pixel signal exceeds a threshold;
The imaging device according to claim 6.
[Claim 8]
The arbitration department
　The arbitration is performed based on external information supplied from the outside of the imaging device,
The imaging device according to claim 1 .
[Claim 9]
The arbitration department
perform the arbitration according to the operation mode supplied from the outside;
The imaging device according to claim 8 .
[Claim 10]
The arbitration department
　The arbitration is performed according to the detection output of the other sensor device supplied from the outside,
The imaging device according to claim 8 .
[Claim 11]
an imaging unit that has a pixel region in which a plurality of pixels are arranged and that reads and outputs pixel signals from the pixels included in the pixel region;
a readout unit control unit that controls the readout unit set as part of the pixel region;
a first readout unit setting unit that sets a first readout unit for reading out the pixel signal from the pixel region for recognition processing that has learned the teacher data for each readout unit;
a second readout unit setting unit for setting a second readout unit for reading out the pixel signal from the pixel region for output to a subsequent stage;
an arbitration unit that arbitrates between the first readout unit and the second readout unit;
an imaging device comprising
a recognition unit that executes the recognition process,
an information processing device comprising
including
　The read unit control unit
setting the read unit through arbitration by the arbitration unit;
imaging system.
[Claim 12]
　Executed by the processor,
a readout unit control step for controlling a readout unit set as part of a pixel region in which a plurality of pixels are arranged, which the imaging unit has;
a first readout unit setting step of setting a first readout unit for reading out pixel signals from pixels included in the pixel region for recognition processing in which the teacher data for each readout unit is learned;
a second readout unit setting step of setting a second readout unit for reading out the pixel signal from the pixel region for output to a subsequent stage;
an arbitration step of arbitrating between the first readout unit and the second readout unit;
including
　The read unit control step is
setting the read unit through arbitration by the arbitration step;
Imaging method.
[Claim 13]
a readout unit control step for controlling a readout unit set as part of a pixel region in which a plurality of pixels are arranged, which the imaging unit has;
a first readout unit setting step of setting a first readout unit for reading out pixel signals from pixels included in the pixel region for recognition processing in which the teacher data for each readout unit is learned;
a second readout unit setting step of setting a second readout unit for reading out the pixel signal from the pixel region for output to a subsequent stage;
an arbitration step of arbitrating between the first readout unit and the second readout unit;
is executed by the processor, and
　The read unit control step is
setting the read unit through arbitration by the arbitration step;
Imaging program for.

Title of Invention: Imaging Apparatus, Imaging System, Imaging Method, and Imaging Program
Technical field
[0001]
The present disclosure relates to imaging devices, imaging systems, imaging methods, and imaging programs.
Background technology
[0002]
In recent years, along with the high performance of imaging devices such as digital still cameras, digital video cameras, compact cameras installed in multifunctional mobile phones (smartphones), etc., an image recognition function that recognizes a predetermined object contained in a captured image has been developed. is being developed.
prior art documents
patent literature
[0003]
Patent Document 1: JP 2017-112409 A
SUMMARY OF THE INVENTION
Problems to be Solved by the Invention
[0004]
In general, images suitable for recognition processing in the image recognition function are different from images suitable for human recognition. Therefore, when trying to improve the accuracy of recognition in an imaging device equipped with an image recognition function, it may be difficult for the image captured for recognition processing to provide sufficient information as an image for visual recognition. There is
[0005]
An object of the present disclosure is to provide an imaging device, an imaging system, an imaging method, and an imaging program capable of both imaging for recognition processing and imaging for visual recognition.
Means to solve problems
[0006]
An imaging device according to the present disclosure has a pixel region in which a plurality of pixels are arranged, an imaging unit that reads and outputs pixel signals from pixels included in the pixel region, and a readout unit that is set as part of the pixel region. and a first readout unit setting unit for setting a first readout unit for reading pixel signals from a pixel region for recognition processing in which teacher data for each readout unit is learned; a second readout unit setting unit for setting a second readout unit for reading pixel signals from the pixel region for output; and an arbitration unit for arbitrating between the first readout unit and the second readout unit. , the read unit control unit sets the read unit through arbitration by the arbitration unit.
Brief description of the drawing
[0007]
1 is a block diagram showing the configuration of an example of an imaging device applicable to each embodiment of the present disclosure; FIG.
2A is a schematic diagram showing an example of a hardware configuration of an imaging device according to each embodiment; FIG.
2B is a schematic diagram showing an example of the hardware configuration of an imaging device according to each embodiment; FIG.
3A is a diagram showing an example in which an imaging device according to each embodiment is formed by a laminated CIS having a two-layer structure; FIG.
3B is a diagram showing an example in which an imaging device according to each embodiment is formed by a laminated CIS having a three-layer structure; FIG.
4 is a block diagram showing an example configuration of a sensor section applicable to each embodiment; FIG.
5A is a schematic diagram for explaining a rolling shutter method; FIG.
5B is a schematic diagram for explaining a rolling shutter method; FIG.
5C is a schematic diagram for explaining a rolling shutter method; FIG.
6A is a schematic diagram for explaining thinning of lines in the rolling shutter method; FIG.
6B is a schematic diagram for explaining thinning of lines in the rolling shutter method; FIG.
6C is a schematic diagram for explaining thinning of lines in the rolling shutter method; FIG.
7A is a diagram schematically showing an example of another imaging method in the rolling shutter system; FIG.
7B is a diagram schematically showing an example of another imaging method in the rolling shutter system; FIG.
8A is a schematic diagram for explaining a global shutter method; FIG.
8B is a schematic diagram for explaining a global shutter method; FIG.
8C is a schematic diagram for explaining a global shutter method; FIG.
9A] A diagram schematically showing an example of a sampling pattern that can be realized in the global shutter method. [FIG.
9B] A diagram schematically showing an example of a sampling pattern that can be implemented in the global shutter method. [FIG.
10 is a diagram for schematically explaining image recognition processing by CNN; FIG.
11 is a diagram for schematically explaining image recognition processing for obtaining a recognition result from a part of an image to be recognized; FIG.
12A is a diagram schematically showing an example of identification processing by DNN when time-series information is not used; FIG.
12B is a diagram schematically showing an example of identification processing by DNN when time-series information is not used; FIG.
13A is a diagram schematically showing a first example of identification processing by DNN when time-series information is used; FIG.
13B is a diagram schematically showing a first example of identification processing by DNN when time-series information is used; FIG.
14A is a diagram schematically showing a second example of identification processing by DNN when time-series information is used; FIG.
14B is a diagram schematically showing a second example of identification processing by DNN when time-series information is used; FIG.
15A is a diagram for explaining the relationship between the driving speed of a frame and the readout amount of pixel signals; FIG.
15B is a diagram for explaining the relationship between the driving speed of a frame and the readout amount of pixel signals; FIG.
16 is a schematic diagram for schematically explaining recognition processing according to each embodiment of the present disclosure; FIG.
17 is a flowchart of an example of recognition processing by a recognition processing unit according to the first embodiment; FIG.
18 is a diagram showing an example of image data for one frame; FIG.
19 is a diagram for explaining the flow of machine learning processing executed by the recognition processing unit according to the first embodiment; FIG.
20A is a schematic diagram for explaining an application example of the first embodiment; FIG.
20B is a schematic diagram for explaining an application example of the first embodiment; FIG.
21 is a functional block diagram of an example for explaining functions of an imaging device according to a second embodiment; FIG.
22 is a schematic diagram showing in more detail an example of processing in a recognition processing unit according to the second embodiment; FIG.
23 is a functional block diagram of an example for explaining functions according to the second embodiment; FIG.
24 is a schematic diagram for explaining frame readout processing according to the second embodiment; FIG.
25 is a schematic diagram schematically showing recognition processing according to the second embodiment; FIG.
26 is a diagram for explaining an example of terminating recognition processing in the middle of frame reading; FIG.
27 is a diagram for explaining an example of terminating recognition processing in the middle of frame reading; FIG.
28 is a flow chart showing an example of recognition processing according to the second embodiment; FIG.
29A is a time chart showing an example of control of reading and recognition processing according to the second embodiment; FIG.
29B is a time chart showing an example of control of reading and recognition processing according to the second embodiment; FIG.
30 is an example time chart showing another example of control of reading and recognition processing according to the second embodiment. FIG.
31 is a flow chart showing an example of control according to the third embodiment; FIG.
32 is a schematic diagram schematically showing an example of output control processing according to the third embodiment; FIG.
33A is a functional block diagram showing an example of functions on the side of a recognition processing unit of an imaging device according to a third embodiment; FIG.
33B is a functional block diagram showing an example of functions on the visual recognition processing unit side of the imaging device according to the third embodiment; FIG.
34 is a flow chart showing an example of processing when a trigger signal is output according to time according to the third embodiment; FIG.
35 is a schematic diagram schematically showing an example of output control processing according to the first modification of the third embodiment; FIG.
36A is a functional block diagram showing an example of functions on the side of a recognition processing section of an imaging device according to the first modified example of the third embodiment; FIG.
36B is a functional block diagram showing an example of functions on the visual recognition processing unit side of the imaging device according to the first modified example of the third embodiment; FIG.
37 is a flowchart of an example showing processing according to the first modification of the third embodiment; FIG.
38 is a schematic diagram schematically showing an example of output control processing according to a second modification of the third embodiment; FIG.
39A is a functional block diagram showing an example of functions on the side of a recognition processing section of an imaging device according to a second modified example of the third embodiment; FIG.
39B is a functional block diagram showing an example of functions on the visual recognition processing unit side of an imaging device according to a second modification of the third embodiment; FIG.
40 is a flowchart of an example showing processing according to a second modification of the third embodiment; FIG.
41A is a functional block diagram showing an example of functions on the side of a recognition processing section of an imaging device according to a third modified example of the third embodiment; FIG.
41B is a functional block diagram showing an example of functions on the visual recognition processing unit side of an imaging device according to a third modified example of the third embodiment; FIG.
42 is a flowchart of an example showing processing according to a third modification of the third embodiment; FIG.
43 is a schematic diagram schematically showing an example of output control processing according to the fourth embodiment; FIG.
44 is a functional block diagram showing functions of an example of an imaging device according to a fourth embodiment; FIG.
45 is a flow chart showing an example of processing according to the fourth embodiment; FIG.
46 is a functional block diagram showing functions of an example of an imaging device according to a first modified example of the fourth embodiment; FIG.
47 is a flowchart of an example showing processing according to the first modification of the fourth embodiment; FIG.
48 is a schematic diagram schematically showing an example of output control processing according to the second modification of the fourth embodiment; FIG.
49 is a functional block diagram showing functions of an example of an imaging device 1 according to a second modified example of the fourth embodiment; FIG.
50 is a flowchart of an example of processing according to the first modification of the fourth embodiment; FIG.
51 is a flowchart of an example for explaining an overview of arbitration processing according to the fifth embodiment; FIG.
52 is a functional block diagram of an example for explaining the functions of the imaging device 1 applicable to the fifth embodiment; FIG.
53 is a schematic diagram for explaining arbitration processing according to the fifth embodiment; FIG.
54 is a flow chart showing an example of arbitration processing according to the fifth embodiment; FIG.
55 is a functional block diagram of an example for explaining functions of an imaging device applicable to the first modified example of the fifth embodiment; FIG.
56 is a schematic diagram for explaining a first example of arbitration processing according to the first modification of the fifth embodiment; FIG.
57 is a schematic diagram for explaining a second example of arbitration processing according to the first modification of the fifth embodiment; FIG.
58 is a flowchart of an example of arbitration processing according to the first modification of the fifth embodiment; FIG.
59 is a functional block diagram of an example for explaining functions of an imaging device applicable to the second modification of the fifth embodiment; FIG.
60 is a schematic diagram for explaining arbitration processing according to the second modification of the fifth embodiment; FIG.
61 is a flowchart of an example of arbitration processing according to the second modification of the fifth embodiment; FIG.
62 is a functional block diagram of an example for explaining the functions of an imaging device 1 applicable to the third modified example of the fifth embodiment; FIG.
63 is a flowchart of an example of arbitration processing according to the third modification of the fifth embodiment; FIG.
64 is a diagram illustrating a usage example of an imaging device to which the technique of the present disclosure is applied; FIG.
65 is a block diagram showing an example of a schematic configuration of a vehicle control system; FIG.
66 is an explanatory diagram showing an example of installation positions of the vehicle exterior information detection section and the imaging section. FIG.
MODE FOR CARRYING OUT THE INVENTION
[0008]
Hereinafter, embodiments of the present disclosure will be described in detail based on the drawings. In addition, in the following embodiments, the same parts are denoted by the same reference numerals, thereby omitting redundant explanations.
[0009]
Hereinafter, embodiments of the present disclosure will be described in the following order.
1. Configuration example according to each embodiment of the present disclosure
2. Examples of existing technology applicable to the present disclosure
2-1. Rolling shutter overview
2-2. Global shutter overview
2-3. About DNN (Deep Neural Network)
2-3-1. Overview of CNN (Convolutional Neural Network)
2-3-2. Overview of RNN (Recurrent Neural Network)
2-4. About drive speed
3. This disclosure Overview of
4. 1st embodiment
4-1. Example of operation by the recognition processing part
4-2. Concrete example of operation by the recognition processing unit
4-3. Application example of the first embodiment
5. Second embodiment
5-0-1. Configuration example according to the second embodiment
5-0-2. Example of processing in the recognition processing unit according to the second embodiment
5-0-3. Details of recognition processing according to the second embodiment
5-0-4. Control Example of Readout and Recognition Processing According to Second Embodiment
6. Third embodiment
6-0. Outline of the third embodiment
6-0-1. Example of trigger signal output by time
6-1. First modification of the third embodiment
6-2. Second Modification of Third Embodiment
6-3. Third modification of the third embodiment
7. Fourth embodiment
7-1. First modification of the fourth embodiment
7-2. Second modification of the fourth embodiment
8. Fifth embodiment
　　8-0-1. Concrete example of arbitration process
8-1. First Modification of Fifth Embodiment
8-2. Second Modification of Fifth Embodiment
8-3. Third Modification of Fifth Embodiment
9. Sixth embodiment
[0010]
[1. Configuration example according to each embodiment of the present disclosure]
The configuration of the imaging device according to the present disclosure will be schematically described. FIG. 1 is a block diagram showing the configuration of an example of an imaging device applicable to each embodiment of the present disclosure. In FIG. 1, the imaging device 1 includes a sensor unit 10, a sensor control unit 11, a recognition processing unit 12, a memory 13, a visual recognition processing unit 14, and an output control unit 15, each of which is a CMOS ( It is a CMOS image sensor (CIS) integrally formed using Complementary Metal Oxide Semiconductor. Note that the imaging device 1 is not limited to this example, and may be another type of optical sensor such as an infrared light sensor that performs imaging using infrared light.
[0011]
The sensor unit 10 outputs pixel signals according to the light irradiated onto the light receiving surface via the optical unit 30 . More specifically, the sensor unit 10 has a pixel array in which pixels each including at least one photoelectric conversion element are arranged in a matrix. A light-receiving surface is formed by pixels arranged in rows and columns in the pixel array. The sensor unit 10 further includes a drive circuit for driving each pixel included in the pixel array, and a signal that performs predetermined signal processing on a signal read from each pixel and outputs the signal as a pixel signal of each pixel. and a processing circuit. The sensor unit 10 outputs a pixel signal of each pixel included in the pixel area as digital image data.
[0012]
Hereinafter, in the pixel array of the sensor unit 10, an area in which pixels effective for generating pixel signals are arranged is called a frame. Frame image data is formed by pixel data based on pixel signals output from pixels included in the frame. Each row in the array of pixels of the sensor section 10 is called a line, and line image data is formed by pixel data based on pixel signals output from each pixel included in the line. Further, the operation of the sensor unit 10 to output pixel signals according to the light applied to the light receiving surface is called imaging. The sensor unit 10 is controlled in terms of exposure during imaging and gain (analog gain) for pixel signals in accordance with imaging control signals supplied from a sensor control unit 11, which will be described later.
[0013]
The sensor control unit 11 is composed of, for example, a microprocessor, controls reading of pixel data from the sensor unit 10, and outputs pixel data based on each pixel signal read from each pixel included in the frame. The pixel data output from the sensor control section 11 is passed to the recognition processing section 12 and the visual recognition processing section 14 .
[0014]
Also, the sensor control unit 11 generates an imaging control signal for controlling imaging in the sensor unit 10 . The sensor control unit 11 generates an imaging control signal, for example, according to instructions from the recognition processing unit 12 and the visual recognition processing unit 14, which will be described later. The imaging control signal includes information indicating the exposure and analog gain at the time of imaging in the sensor section 10 described above. The imaging control signal further includes control signals (vertical synchronizing signal, horizontal synchronizing signal, etc.) used by the sensor unit 10 to perform an imaging operation. The sensor control section 11 supplies the generated imaging control signal to the sensor section 10 .
[0015]
The optical unit 30 is for irradiating the light receiving surface of the sensor unit 10 with light from the subject, and is arranged at a position corresponding to the sensor unit 10, for example. The optical unit 30 includes, for example, a plurality of lenses, a diaphragm mechanism for adjusting the size of an opening for incident light, and a focus mechanism for adjusting the focus of light irradiated onto the light receiving surface. The optical section 30 may further include a shutter mechanism (mechanical shutter) that adjusts the time during which the light receiving surface is irradiated with light. The aperture mechanism, focus mechanism, and shutter mechanism of the optical unit 30 can be controlled by the sensor control unit 11, for example. Without being limited to this, the aperture and focus in the optical unit 30 can be controlled from outside the imaging device 1 . It is also possible to configure the optical unit 30 integrally with the imaging device 1 .
[0016]
Based on the pixel data passed from the sensor control unit 11, the recognition processing unit 12 performs recognition processing of the object included in the image based on the pixel data. In the present disclosure, for example, a DSP (Digital Signal Processor) reads out and executes a program that is pre-learned with teacher data and stored as a learning model in the memory 13, thereby performing recognition processing using a DNN (Deep Neural Network). A recognition processing unit 12 is configured as a machine learning unit that performs the above. The recognition processing unit 12 can instruct the sensor control unit 11 to read pixel data necessary for recognition processing from the sensor unit 10 . A recognition result by the recognition processing unit 12 is passed to the output control unit 15 .
[0017]
The visual recognition processing unit 14 executes processing for obtaining an image suitable for human visual recognition on the pixel data passed from the sensor control unit 11, and outputs image data composed of, for example, a group of pixel data. do. For example, the visual recognition processing unit 14 is configured by an ISP (Image Signal Processor) reading and executing a program stored in advance in a memory (not shown).
[0018]
For example, when each pixel included in the sensor unit 10 is provided with a color filter and the pixel data has color information of R (red), G (green), and B (blue), the visual recognition processing unit 14 performs demosaic processing. processing, white balancing, etc. can be performed. Further, the visual recognition processing unit 14 can instruct the sensor control unit 11 to read pixel data necessary for visual recognition processing from the sensor unit 10 . Image data obtained by subjecting pixel data to image processing by the visual recognition processing unit 14 is passed to the output control unit 15 .
[0019]
The output control unit 15 is configured by, for example, a microprocessor, and outputs one or both of the recognition result delivered from the recognition processing unit 12 and the image data delivered as the viewing processing result from the viewing processing unit 14 to the imaging device. Output to the outside of 1. The output control unit 15 can output image data to, for example, a display unit 31 having a display device. Thereby, the user can visually recognize the image data displayed by the display unit 31 . Note that the display unit 31 may be built in the imaging device 1 or may be configured outside the imaging device 1 .
[0020]
2A and 2B are schematic diagrams showing an example of the hardware configuration of the imaging device 1 according to each embodiment. 2A, one chip 2 includes the sensor unit 10, the sensor control unit 11, the recognition processing unit 12, the memory 13, the visual recognition processing unit 14, and the output control unit 15 among the components shown in FIG. For example. Note that the memory 13 and the output control unit 15 are omitted in FIG. 2A to avoid complication.
[0021]
In the configuration shown in FIG. 2A, the recognition result by the recognition processing unit 12 is output to the outside of the chip 2 via the output control unit 15 (not shown). 2A, the recognition processing unit 12 can acquire pixel data to be used for recognition from the sensor control unit 11 via the internal interface of the chip 2. FIG.
[0022]
2B, the sensor unit 10, the sensor control unit 11, the visual recognition processing unit 14, and the output control unit 15 of the configuration shown in FIG. not shown) is placed outside the chip 2 . Also in FIG. 2B, the memory 13 and the output control unit 15 are omitted to avoid complication, as in FIG. 2A described above.
[0023]
In the configuration of FIG. 2B, the recognition processing unit 12 acquires pixel data to be used for recognition via an interface for inter-chip communication. Further, although FIG. 2B shows that the recognition result by the recognition processing unit 12 is output directly from the recognition processing unit 12 to the outside, this is not limited to this example. That is, in the configuration of FIG. 2B , the recognition processing unit 12 may return the recognition result to the chip 2 and output it from the output control unit 15 (not shown) mounted on the chip 2 .
[0024]
In the configuration shown in FIG. 2A, the recognition processing unit 12 is mounted on the chip 2 together with the sensor control unit 11, and communication between the recognition processing unit 12 and the sensor control unit 11 can be performed at high speed by the internal interface of the chip 2. . On the other hand, in the configuration shown in FIG. 2A, the recognition processing unit 12 cannot be replaced, and it is difficult to change the recognition processing. On the other hand, in the configuration shown in FIG. 2B, since the recognition processing unit 12 is provided outside the chip 2, communication between the recognition processing unit 12 and the sensor control unit 11 is performed via an interface between chips. There is a need. Therefore, communication between the recognition processing unit 12 and the sensor control unit 11 is slower than in the configuration of FIG. 2A, and control may be delayed. On the other hand, replacement of the recognition processing unit 12 is easy, and various recognition processes can be realized.
[0025]
Hereinafter, unless otherwise specified, the imaging device 1 includes the sensor unit 10, the sensor control unit 11, the recognition processing unit 12, the memory 13, the visual recognition processing unit 14, and the output control unit 15 in one chip 2 of FIG. 2A. shall be adopted.
[0026]
In the configuration shown in FIG. 2A described above, the imaging device 1 can be formed on one substrate. The imaging device 1 is not limited to this, and may be a stacked CIS in which a plurality of semiconductor chips are stacked and integrally formed.
[0027]
As an example, the imaging device 1 can be formed with a two-layer structure in which semiconductor chips are stacked in two layers. FIG. 3A is a diagram showing an example in which the imaging device 1 according to each embodiment is formed by a laminated CIS having a two-layer structure. In the structure of FIG. 3A, the pixel section 20a is formed in the semiconductor chip of the first layer, and the memory+logic section 20b is formed in the semiconductor chip of the second layer. The pixel section 20 a includes at least the pixel array in the sensor section 10 . The memory+logic unit 20b includes, for example, the sensor control unit 11, the recognition processing unit 12, the memory 13, the visual recognition processing unit 14, the output control unit 15, and an interface for communicating between the imaging device 1 and the outside. . The memory+logic section 20b further includes a part or all of a drive circuit that drives the pixel array in the sensor section 10. FIG. Although not shown, the memory+logic unit 20b can further include a memory used by the visual recognition processing unit 14 to process image data, for example.
[0028]
As shown on the right side of FIG. 3A, the imaging device 1 is configured as one solid-state imaging device by bonding the semiconductor chips of the first layer and the semiconductor chips of the second layer while electrically contacting them.
[0029]
As another example, the imaging device 1 can be formed with a three-layer structure in which semiconductor chips are stacked in three layers. FIG. 3B is a diagram showing an example in which the imaging device 1 according to each embodiment is formed by a laminated CIS having a three-layer structure. In the structure of FIG. 3B, the pixel section 20a is formed in the semiconductor chip of the first layer, the memory section 20c is formed in the semiconductor chip of the second layer, and the logic section 2 is formed in the semiconductor chip of the third layer.0b'. In this case, the logic unit 20b' includes, for example, the sensor control unit 11, the recognition processing unit 12, the visual recognition processing unit 14, the output control unit 15, and an interface for communicating between the imaging device 1 and the outside. Further, the memory unit 20c can include the memory 13 and a memory used by the visual recognition processing unit 14 for processing image data, for example. The memory 13 may be included in the logic portion 20b'.
[0030]
As shown on the right side of FIG. 3B , the first layer semiconductor chip, the second layer semiconductor chip, and the third layer semiconductor chip are laminated while being in electrical contact with each other, whereby the imaging device 1 is integrated into one. configured as one solid-state imaging device.