JP7384204B2 - Imaging device and imaging method - Google Patents

Description

The present invention relates to an imaging device and an imaging method.

By emitting infrared rays or laser light towards the measurement range and detecting the light returning from the object existing within the measurement range, the detection time from light emission to detection (so-called flight time) indicates that the object exists within the measurement range. Can measure distance to objects. In such a TOF (Time of Flight) distance measuring method, when a plurality of objects exist within the measurement range, the distances of each object can also be determined. However, it is not possible to identify each object.
Patent Document 1 International Publication No. 2018/101187

General disclosure

(Item 1)
The imaging device may include a light emitting section that emits light.
The imaging device may include a plurality of groups each including a plurality of light receiving elements that receive reflected light emitted from the light emitting section.
The imaging device may include a processing unit that generates images for each group.
At least one of the timing of receiving the plurality of groups of reflected light and the exposure time may be set corresponding to different imaging ranges.
The processing unit may generate images of the object in the imaging range corresponding to each of the plurality of groups.
(Item 2)
The imaging range may include at least one of a distance range from the plurality of light receiving elements to the target section and a time range from when light is emitted from the light emitting section until the light is received by the plurality of light receiving elements.
(Item 3)
The imaging device may include a control unit that controls at least one of timing and exposure time for each light receiving element or for each group.
(Item 4)
The imaging device may include a light receiving surface on which a plurality of light receiving elements are arranged.
Each of the plurality of groups may have light receiving elements located in different regions on the light receiving surface.
(Item 5)
At least one of the plurality of groups may include a plurality of blocks on the light receiving surface, each of which includes at least one light receiving element.
(Item 6)
The timing of the light receiving element located in the lower region on the light receiving surface may be earlier than the timing of the light receiving element located in the upper region on the light receiving surface.
(Item 7)
The light emitting section and the plurality of light receiving elements may be provided on the moving body.
The control unit shifts the exposure time of the light receiving element located on the right area on the light receiving surface forward when the moving object moves forward to the left, and shifts the exposure time of the light receiving element located on the right area on the light receiving surface forward when the moving object moves forward to the right. The exposure time of the light receiving element located in the partial area may be shifted forward.
(Item 8)
The control unit may control the exposure time of the plurality of light receiving elements so that the exposure time of the plurality of groups is continuous among the plurality of groups.
(Item 9)
The control unit controls the exposure of the plurality of light-receiving elements such that the number of light-receiving elements included in the group whose exposure time is earlier among the plurality of groups is greater than the number of light-receiving elements included in the group whose exposure time is later. You can control time.
(Item 10)
The control unit may control the exposure times of the plurality of light receiving elements so that the exposure times of the plurality of groups start at the same time and end at mutually different times.
(Item 11)
The control unit controls the exposure time of the plurality of light receiving elements so that at least one of the plurality of groups is exposed to light before the light is emitted;
The processing unit may subtract the intensity of the output signal of the light receiving element included in at least one group from the intensity of the output signal of the light receiving element included in each of the remaining groups among the plurality of groups.
(Item 12)
The processing unit may identify an object included in the generated image.
(Item 13)
The processing unit may detect movement of the object based on the objects specified for each of the plurality of groups and the exposure time of each of the plurality of groups.
(Item 14)
The light emitting section and the plurality of light receiving elements may be provided on the moving body.
The control unit may control the exposure time of the plurality of light receiving elements according to the speed of the moving object.
(Item 15)
The control unit may shift the exposure time of at least some of the plurality of light receiving elements later as the speed of the moving object is faster.
(Item 16)
A plurality of light receiving elements or a plurality of groups may be arranged between other light receiving elements in a Bayer arrangement.

(Item 17)
The mobile object may include the imaging device according to any one of items 1 to 16.
The moving object may include a speed sensor that detects the speed of the moving object.
The moving body may include a moving direction sensor that detects the moving direction of the moving body.

(Item 18)
The imaging method may include emitting light.
The imaging method may include the step of receiving reflected light of the emitted light using a plurality of light receiving elements.
The imaging method may include the step of controlling exposure time for each of a plurality of light receiving elements or for each of a plurality of groups each having at least two light receiving elements.
The imaging method may include the step of generating at least one of a distance to an object within a measurement target range corresponding to an exposure time and an image of the object for each of a plurality of light receiving elements or for each of a plurality of groups.

Note that the above summary of the invention does not list all the features of the invention. Furthermore, subcombinations of these features may also constitute inventions.

1 shows a cross-sectional configuration of a back-illuminated light receiving device according to the present embodiment. An example of a group arrangement of light receiving elements of an imaging chip is shown. Another example of the group arrangement of the light receiving elements of the imaging chip is shown. Another example of the group arrangement of the light receiving elements of the imaging chip is shown. Another example of the group arrangement of the light receiving elements of the imaging chip is shown. The equivalent circuit of a pixel is shown. FIG. 2 is a circuit diagram showing a connection relationship of pixels in blocks included in a unit group. FIG. 2 is a block diagram showing the functional configuration of an image sensor. 1 shows the configuration of an imaging device according to this embodiment. An example of objects existing within the imaging range and their distances is shown. An example of the exposure time of each group of light receiving elements is shown. Another example of the exposure time of each group of light receiving elements is shown. Another example of the exposure time of each group of light receiving elements is shown. Another example of the exposure time of each group of light receiving elements is shown. 1 shows an example of the configuration of an imaging system when an imaging device according to the present embodiment is mounted on a vehicle. An example of the flow of imaging processing by an imaging device mounted on a vehicle is shown.

Hereinafter, the present invention will be explained through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. Furthermore, not all combinations of features described in the embodiments are essential to the solution of the invention.

FIG. 1 shows a cross-sectional configuration of a back-illuminated light receiving device 100 according to this embodiment. The light receiving device 100 includes an imaging chip 113, a signal processing chip 111, and a memory chip 112. These chips are stacked, and the chips are bonded by bonding between conductive bumps 109 such as Cu formed on each chip and bonding between oxide film layers formed on the top surface of each chip. There is. Here, the chips are electrically connected to each other by bonding the bumps 109 together. Note that the incident light is incident in the +Z direction, as indicated by the white arrow. In this embodiment, the surface of the imaging chip 113 on the side where the incident light enters is called the back surface, and the surface on the opposite side is called the front surface. Further, the vertical direction in the drawing is the Z-axis direction, the left-right direction is the X-axis direction, and the direction perpendicular to these is the Y-axis direction. Furthermore, in this embodiment, a target range imaged by the imaging device 500 described later is also referred to as an imaging range.

The imaging chip 113 receives incident light and outputs a pixel signal according to the amount of exposure. As the imaging chip 113, for example, a back-illuminated MOS image sensor may be employed. The imaging chip 113 has a PD layer 106 and a wiring layer 108.

The PD layer 106 has a plurality of light receiving elements 104 arranged two-dimensionally on the light receiving surface of the light receiving device 100. The light receiving element 104 is a photoelectric conversion element such as a photodiode (PD).

Note that the color filter 102 is provided on the back side of the PD layer 106 with a passivation film 103 interposed therebetween. The color filter 102 has a plurality of types that transmit different wavelength regions, and has a specific arrangement corresponding to each light receiving element 104. One pixel is formed by one color filter 102, one light receiving element 104, and one transistor 105, which will be described later. Note that in the case of monochromatic imaging, the color filter 102 does not need to be provided.

Further, a plurality of microlenses 101 are provided on the back side of the color filter 102. Each microlens 101 condenses incident light toward a corresponding light receiving element 104.

The wiring layer 108 is provided on the front surface side of the PD layer 106 and transmits pixel signals from the transistors 105 and the light receiving elements 104 provided corresponding to each light receiving element 104 in the PD layer 106 to the signal processing chip 111. It has wiring 107 for transmission. The wiring 107 may be multilayered and may include passive elements and active elements. A plurality of bumps 109 are provided on the front surface of the wiring layer 108.

Signal processing chip 111 includes a device that processes pixel signals from PD layer 106. These devices may be provided on both sides of the chip and may be connected by TSVs (Through Silicon Vias) 110. The signal processing chip 111 has a plurality of bumps 109 and is electrically connected to the plurality of bumps 109 of the imaging chip 113.

Memory chip 112 includes devices that store pixel signals. The memory chip 112 has a plurality of bumps 109 and is electrically connected to the plurality of bumps 109 of the signal processing chip 111.

2A to 2D show the group arrangement of the light receiving elements 104 of the imaging chip 113. The imaging chip 113 has a pixel area on the light receiving surface, and more than 20 million pixels, that is, the light receiving elements 104 are arranged in a matrix. As an example, nine adjacent pixels (3 pixels x 3 pixels) constitute one block 131b to 135b, and a plurality of unit groups 131 to 135 each include a plurality of arbitrarily arranged blocks 131b to 135b. . Note that the number of pixels (light receiving elements 104) included in one block 131b to 135b is not limited to this, and only one pixel may be included in one block 131b to 135b. Furthermore, any number of blocks 131b to 135b may be included in one unit group 131 to 135.

As described later, the exposure time, that is, the charge accumulation, of the plurality of light receiving elements 104 is controlled for each unit group 131 to 135. For example, the light receiving elements 104 belonging to unit groups 131 to 135 each have an exposure time of 0 to 100 nanoseconds, an exposure time of 100 to 200 nanoseconds, and an exposure time of 200 to 300 nanoseconds, based on the time point at which the illumination light is emitted. The shutter (unless otherwise specified, means an electronic shutter as explained with reference to FIG. 3) is opened to discharge the charge during the exposure time of 300 to 400 nanoseconds, and the exposure time of 400 to 500 nanoseconds. accumulate.

Each of the plurality of unit groups 131 to 135 has a plurality of blocks 131b to 135b of the light receiving element 104 located in different areas on the light receiving surface. In the example of the group arrangement shown in FIG. 2A, a set of four blocks 131b to 134b arranged in two rows and two columns are repeatedly arranged in the row and column directions. The light receiving elements 104 included in the four blocks 131b to 134b belong to unit groups 131 to 134, respectively. Thereby, the blocks 131b to 134b of the light receiving elements 104 belonging to each unit group 131 to 134 are arranged in a matrix with a distance of one block in each of the row and column directions. Therefore, by using the light receiving elements 104 belonging to the unit groups 131 to 134, the imaging range can be divided into a plurality of distances and images can be taken with the same resolution.

In the example of the group arrangement shown in FIG. 2B, the columns of four blocks 131b to 134b are arranged in sequence in the row direction, and these four columns are arranged repeatedly in the row direction. There is. The light receiving elements 104 included in the four blocks 131b to 134b belong to unit groups 131 to 134, respectively. Thereby, the blocks 131b to 134b of the light receiving elements 104 belonging to each unit group 131 to 134 are arranged in a stripe pattern continuously in the column direction and separated by a distance of three blocks in the row direction. Therefore, by using the light receiving elements 104 belonging to the unit groups 131 to 134, the imaging range can be divided into a plurality of distances, and images can be taken with higher resolution in the column direction than in the row direction. Note that the four blocks 131b to 134b, each successively arranged in the row direction, may be repeatedly arranged in order in the column direction. Thereby, images can be captured with higher resolution in the row direction than in the column direction.

In the example of the group arrangement shown in FIG. 2C, some of the blocks 134b belonging to the unit group 134 are replaced with blocks 135b belonging to the unit group 135 compared to the group arrangement shown in FIG. 2A. As a result, many light-receiving elements 104 belonging to unit groups 131 to 133 provide high resolution, fewer light-receiving elements 104 belonging to unit group 134 provide somewhat lower resolution, and even fewer light-receiving elements 104 belonging to unit group 134 provide lower resolution. It is possible to capture images with high resolution.

In the example of the group array shown in FIG. 2D, a so-called Bayer array consisting of four pixels, green pixels Gb and Gr, blue pixel B, and red pixel R arranged in 2 rows and 2 columns, is repeatedly arranged in the row and column directions. ing. Note that in this example, one block includes only one pixel. The green pixels Gb and Gr have a green filter as the color filter 102, and receive light in the green wavelength band of the incident light. The blue pixel B has a blue filter as the color filter 102 and receives light in the blue wavelength band. The red pixel R has a red filter as the color filter 102 and receives light in the red wavelength band. Further, light receiving elements 104 belonging to unit groups 131 to 133 arranged in the row direction are included between the columns in the Bayer array. As a result, the entire imaging range can be imaged in color using the Bayer-arranged pixels, and the entire imaging range can be separated into a plurality of distances and imaged using the pixels (light receiving elements 104) belonging to the unit groups 131 to 133, respectively.

Note that in the examples of group arrays shown in FIGS. 2A to 2C, blocks 131b to 135b belonging to unit groups 131 to 135, respectively, may include a Bayer array.

FIG. 3 shows an equivalent circuit of the pixel 150. Each pixel 150 includes a PD 104, a transfer transistor 152, a reset transistor 154, an amplification transistor 156, a selection transistor 158, and a load current source 309. These transistors are collectively referred to as transistors 105 in FIG. Further, the pixel 150 includes a reset wiring 300 to which an ON signal of the reset transistor 154 is supplied, a transfer wiring 302 to which an ON signal of the transfer transistor 152 is supplied, a power supply wiring 304 which receives power from the power supply Vdd, and a selection transistor 158. A selection wiring 306 to which an on signal is supplied, and an output wiring 308 to output a pixel signal are provided.

In the pixel 150, the source, gate, and drain of the transfer transistor 152 are connected to one end of the PD 104, the transfer wiring 302, and the gate of the amplification transistor 156, respectively. The source, gate, and drain of the reset transistor 154 are connected to the gate of the amplification transistor 156, the reset wiring 300, and the power supply wiring 304, respectively. A so-called floating diffusion FD is formed between the drain of the transfer transistor 152 and the source of the reset transistor 154. The source and drain of the amplification transistor 156 are connected to the drain of the selection transistor 158 and the power supply wiring 304, respectively. The gate and source of the selection transistor 158 are connected to the selection wiring 306 and the output wiring 308, respectively.

Load current source 309 supplies current to output wiring 308 . As a result, the output wiring 308 for the selection transistor 158 is formed by a source follower. Note that the load current source 309 may be provided on the imaging chip 113 side or on the signal processing chip 111 side.

The flow from charge accumulation to pixel output in the pixel 150 will be described. By applying a reset pulse to the reset transistor 154 through the reset wiring 300 and simultaneously applying a transfer pulse to the transfer transistor 152 through the transfer wiring 302, the potentials of the PD 104 and the floating diffusion FD are reset. When the application of the transfer pulse is removed, the PD 104 accumulates charges according to the amount of incident light received. Thereafter, when a transfer pulse is applied again without a reset pulse being applied, the accumulated charges are transferred to the floating diffusion FD, and the potential of the floating diffusion FD changes from the reset potential to the signal potential after charge accumulation. The operation from the release of the application of the transfer pulse to the application of the transfer pulse again is also called opening the shutter, and the period thereof is called the shutter time or the exposure time. Then, when a selection pulse is applied to the selection transistor 158 through the selection wiring 306, fluctuations in the signal potential of the floating diffusion FD are transmitted to the output wiring 308 via the amplification transistor 156 and the selection transistor 158. Thereby, a pixel signal corresponding to the reset potential and the signal potential is output from the unit pixel to the output wiring 308.

Note that the circuit configuration of the pixel 150 is not limited to the circuit configuration of the pixel 150 shown in FIG. A circuit configuration may be adopted in which an additional transistor is provided between the two, and the selection transistor 158 is optionally omitted.

FIG. 4 shows the connection relationship of pixels 150 in each of blocks 131b to 135b included in unit groups 131 to 135. Note that although reference numbers for each transistor are omitted for the purpose of making the drawings easier to read, each transistor in each pixel in FIG. 4 has the same configuration and function as each transistor arranged at a corresponding position in the pixel 150 in FIG. 3. . The blocks 131b to 135b are formed by, for example, nine adjacent pixels A to I of 3 pixels x 3 pixels. Note that the number of pixels included in the blocks 131b to 135b is not limited to this.

The reset transistors of the pixels included in the blocks 131b to 135b are turned on and off in common in block units. A reset wiring 300 that turns on and off the reset transistor of pixel A, a reset wiring 310 that turns on and off the reset transistor of pixel B, a reset wiring 320 that turns on and off the reset transistor of pixel C, and a dedicated wiring that turns on and off the reset transistors of other pixels D to I. The lines are connected to a common driver (not shown).

The transfer transistors of the pixels included in the blocks 131b to 135b are also turned on and off in common in block units. Transfer wiring 302 that turns on and off the transfer transistor of pixel A, transfer wiring 312 that turns on and off the transfer transistor of pixel B, transfer wiring 322 that turns on and off the transfer transistor of pixel C, and dedicated for turning on and off the transfer transistors of other pixels D to I. The lines are connected to a common control circuit (not shown).

The selection transistors of the pixels included in blocks 131b to 135b are also turned on and off individually for each pixel. A selection wiring 306 that turns on and off the selection transistor of pixel A, a selection wiring 316 that turns on and off the selection transistor of pixel B, a selection wiring 326 that turns on and off the selection transistor of pixel C, and a dedicated wiring that turns on and off the selection transistors of other pixels D to I. The lines are separately connected to a driver (not shown).

Note that the power supply wiring 304 is commonly connected to each pixel A to I included in the blocks 131b to 135b. Similarly, the output wiring 308 is commonly connected to each of the pixels A to I included in the blocks 131b to 135b. Furthermore, although the power supply wiring 304 is commonly connected between a plurality of blocks, the output wiring 308 is provided for each block.

By turning on and off the reset transistors and transfer transistors of blocks 131b to 135b in common, charge accumulation start time, accumulation end time, and transfer timing are synchronized with respect to each pixel A to I included in blocks 131b to 135b. The charge accumulation involved can be controlled. Furthermore, by individually turning on and off the selection transistors of the blocks 131b to 135b, the pixel signals of each pixel A to I can be individually outputted via the common output wiring 308.

Charge accumulation in each pixel A to I included in blocks 131b to 135b is controlled by a rolling shutter method or a global shutter method. In the rolling shutter method, the charge accumulation of pixels is controlled in a regular order for rows and columns, for example, by selecting pixels for each row and then specifying the column, "ABCDEFGHI" in the example of FIG. Pixel signals are output in this order. However, in the rolling shutter method, when a moving object is imaged, an image in which the moving object is obliquely distorted is generated for the pixels in the blocks 131b to 135b. In the global shutter method, charge accumulation in pixels is controlled at the same time for all pixels A to I.

FIG. 5 shows the functional configuration of the light receiving device 100. The light receiving device 100 includes a multiplexer 411, a signal processing circuit 412, a demultiplexer 413, and a pixel memory 414. Here, the multiplexer 411 is formed on the imaging chip 113. The signal processing circuit 412 is formed on the signal processing chip 111. Demultiplexer 413 and pixel memory 414 are formed on memory chip 112.

The multiplexer 411 sequentially selects each pixel A to I of the blocks 131b to 135b, and transmits each pixel signal to the signal processing circuit 412 via the output wiring 308. Note that an amplifier may be provided after the multiplexer 411 to amplify each pixel signal by a predetermined amplification factor (gain) and transmit the amplified signal to the signal processing circuit 412.

The signal processing circuit 412 performs correlated double sampling (CDS) and analog/digital (A/D) conversion on the pixel signal (analog signal), and sends the digitized pixel signal to the demultiplexer 413 via the output wiring 330. Send.

The demultiplexer 413 transmits the pixel signals digitized by the signal processing circuit 412 to the memories A to I of the pixel memory 414 corresponding to the pixels A to I, respectively.

The pixel memory 414 has memories A to I that store pixel signals, and stores the pixel signals of pixels A to I in the memories A to I, respectively.

Note that a signal processing circuit 412 may be provided for each pixel A to I, and the pixel signals from these may be subjected to CDS/A/D conversion in parallel and stored in the memories A to I.

The arithmetic circuit 415 processes the pixel signal stored in the pixel memory 414 and delivers it to the subsequent processing section 40. The arithmetic circuit 415 may be provided in the signal processing chip 111 or in the memory chip 112. Note that the arithmetic circuit 415 may be provided for each group, or may be provided in common for a plurality of groups.

FIG. 6 shows the configuration of an imaging device 500 according to this embodiment. The imaging device 500 is a device that divides the imaging range into a plurality of distances and captures images, and includes a light emitting section 10, a light receiving section 20, a control section 30, and a processing section 40.

The light emitting unit 10 is a unit that emits light, that is, illumination light, toward an imaging range. The light emitting unit 10 has a light source that generates light in the infrared or visible light range, uses this to generate modulated light such as pulsed light, and illuminates the imaging range by emitting it as illumination light.

The light receiving section 20 is a unit that receives reflected light from the imaging range. The light receiving section 20 includes an optical system 21 and a light receiving device 100.

The optical system 21 is composed of a plurality of lens elements, and guides light entering from the imaging range along the optical axis OA onto the light receiving surface of the light receiving device 100 to form a subject image. The optical system 21 may be configured to be detachable from the light receiving device 100.

As described above, the light receiving device 100 includes a plurality of light receiving elements 104 arranged on the light receiving surface. Each of the plurality of light-receiving elements 104 forms a pixel, receives light (subject image) formed by the optical system 21 during an exposure time, performs photoelectric conversion, and accumulates charge. The plurality of light receiving elements 104 are grouped into a plurality of unit groups 131 to 135, and the exposure time, that is, charge accumulation is controlled for each unit group 131 to 135. Pixel signals output from the plurality of light receiving elements 104 are transmitted to the processing section 40.

The control section 30 is a unit that controls the exposure time, ie, the charge accumulation, of the plurality of light receiving elements 104 of the light receiving device 100 for each of the plurality of light receiving elements 104 or for each of the plurality of unit groups 131 to 135. The control unit 30 is composed of, for example, a microprocessor and its peripheral circuits, and performs its functions by executing a control program stored in a nonvolatile memory (not shown). Note that some functions of the control unit 30 may be configured by an electronic circuit such as a timing generator.

Based on the pixel signals output from the plurality of light-receiving elements 104, the processing unit 40 receives images at distances within the imaging range corresponding to the exposure times of each of the plurality of unit groups 131 to 135, that is, image data. It is generated for each element 104 or for each of a plurality of unit groups 131 to 135. Here, a buffer memory (not shown) can be used as a workspace for image processing. The image data may be generated in a JPEG file format, for example, and in such a case, compression processing is performed after performing white balance processing, gamma processing, etc. Additionally, the processing unit 40 may perform binning processing to improve sensitivity. The generated image data may be recorded in a storage device (not shown) such as a nonvolatile flash memory, and may also be converted into a display signal and displayed on a display device (not shown) such as a liquid crystal monitor.

FIG. 7A shows an example of objects 1 to 3 existing within the imaging range and their distances. It is assumed that object 1 exists within a distance of 15 to 30 m from the position (0 m) of the light receiving surface of the imaging device 500, object 2 exists within a distance of 30 to 45 m, and object 3 exists within a distance of 45 to 60 m. .

FIG. 7B shows an example of the exposure time of each unit group of the light receiving elements 104. In this example, it is assumed that the light receiving elements 104 are grouped into four unit groups 131 to 134 in the group arrangement shown in FIG. 2A or 2B. The control unit 30 controls the exposure times of the plurality of light receiving elements 104 so that the exposure times of the unit groups 131 to 134 of the light receiving elements 104 are continuous among the unit groups 131 to 134. As an example, the control unit 30 controls the light receiving elements 104 belonging to the unit groups 131 to 134 to have an exposure time of 0 to 100 nanoseconds, 100 to 200 nanoseconds, based on the time when the illumination light is emitted by the light emitting unit 10, respectively. Open the shutter for a nanosecond exposure time, a 200-300 nanosecond exposure time, a 300-400 nanosecond exposure time. Note that after the exposure time of the unit groups 131 to 134 ends or after repeating the exposure multiple times, the output signal (pixel signal) of the light receiving element 104 of each unit group is read out, and the processing unit 40 processes it for each unit group. good.

Note that the exposure time being continuous between the unit groups 131 to 134 does not mean that the exposure time is continuous so that the exposure time of the next unit group starts at the same time as the end of the exposure time of the previous unit group; The exposure time may be continued between 134 and 134, with some overlap, or may be continued at intervals.

Here, light travels a distance of about 3 meters in 10 nanoseconds. Since the unit group 131 of the light-receiving elements 104 receives reflected light from objects existing within a distance range of 0 to 15 m, objects existing within that range are imaged. In the example of FIG. 7A, no object is imaged. Since the unit group 132 of the light-receiving elements 104 receives reflected light from an object existing within a distance of 15 to 30 m, the object 1 existing within that range is imaged. Since the unit group 133 of the light receiving elements 104 receives reflected light from an object existing within a distance of 30 to 45 m, the object 2 existing within that range is imaged. Since the unit group 134 of the light-receiving elements 104 receives reflected light from an object existing within a distance of 45 to 60 m, the object 3 existing within that range is imaged. In this way, objects located in a plurality of continuous distance ranges can be separated from each other and imaged according to the exposure time of each unit group.

Note that the exposure times of the light receiving elements 104 grouped into five unit groups 131 to 135 in the group arrangement shown in FIG. 2C may be controlled to be continuous as in the example of FIG. 7B. Furthermore, the control unit 30 controls the number of light receiving elements 104 included in groups 131 to 133 whose exposure time is earlier among the unit groups 131 to 135 to the number of light receiving elements 104 included in groups 134 to 135 whose exposure time is later. The exposure time of the light receiving element 104 may be controlled so that the number of light receiving elements is greater than the number of light receiving elements. As a result, objects located closer to each other (in the example of FIG. 7A, object 1 and object 2) can be imaged with higher resolution.

FIG. 7C shows another example of the exposure time of each unit group of the light receiving elements 104. In this example, it is assumed that the light receiving elements 104 are grouped into four unit groups 131 to 134 in the group arrangement shown in FIG. 2A or 2B. The control unit 30 controls the exposure times of the plurality of light receiving elements 104 so that the exposure times of the unit groups 131 to 134 start from the same start time and end at mutually different end times. As an example, the control unit 30 opens the shutters and starts exposure for all of the light receiving elements 104 belonging to the unit groups 131 to 134 at the same time as the illumination light is emitted by the light emitting unit 10, and The exposure is completed in sequence by closing the shutter when 100 nanoseconds, 200 nanoseconds, 300 nanoseconds, and 400 nanoseconds have passed, respectively. Note that after the exposure time of the unit groups 131 to 134 ends or after repeating the exposure a plurality of times, the output signals (pixel signals) of the light receiving elements 104 of each group are read out and processed by the processing unit 40 for each group.

Here, the processing unit 40 subtracts the intensity of the pixel signal from the light receiving element 104 belonging to the unit group 131 from the pixel signal from the light receiving element 104 belonging to the unit group 132. The intensity of the pixel signal from the light receiving element 104 belonging to the unit group 132 is subtracted from the pixel signal from the light receiving element 104 belonging to the unit group 133. The intensity of the pixel signal from the light receiving element 104 belonging to the unit group 133 is subtracted from the pixel signal from the light receiving element 104 belonging to the unit group 134 . However, for example, the average intensity of the pixel signals of the light receiving elements 104 included in one or more blocks of another group adjacent to the block to which the light receiving element 104 to be subtracted belongs is subtracted. Thereby, from the pixel signals of the light receiving elements 104 belonging to the unit group 131, an image of an object existing within a distance range of 0 to 15 m is obtained. In the example of FIG. 7A, no images of any objects are obtained. An image of the object 1 existing within a distance of 15 to 30 m is obtained from the pixel signals of the light receiving elements 104 belonging to the unit groups 131 to 132. An image of the object 2 existing within a distance of 30 to 45 m is obtained from the pixel signals of the light receiving elements 104 belonging to the unit groups 132 to 133. From the pixel signals of the light receiving elements 104 belonging to the unit groups 133 to 134, an image of the object 3 existing within a distance of 45 to 60 m is obtained. In this way, the closer an object is located, the higher the resolution of the image can be obtained.

FIG. 7D shows another example of the exposure time of each unit group of the light receiving elements 104. In this example, it is assumed that the light receiving elements 104 are grouped into four unit groups 131 to 134 in the group arrangement shown in FIG. 2A or 2B. The control unit 30 controls the exposure time of the plurality of light receiving elements 104 so that at least one of the unit groups 131 to 134 is exposed to light before the illumination light is emitted. As an example, the control unit 30 controls the light receiving elements 104 belonging to the unit groups 131 to 134 to have an exposure time of 0 to 100 nanoseconds, 100 to 200 nanoseconds, based on the time when the illumination light is emitted by the light emitting unit 10, respectively. Open the shutter for nanosecond exposure times, 200-300 nanosecond exposure times, -100 to 0 nanosecond exposure times. Note that after the exposure time of the unit groups 131 to 134 ends or after repeating the exposure multiple times, the output signal (pixel signal) of the light receiving element 104 of each unit group is read out and processed by the processing unit 40 for each unit group.

Here, the processing unit 40 subtracts the intensity of the pixel signal of the light receiving element 104 included in the unit group 134 from the intensity of the pixel signal of the light receiving element 104 included in each of the unit groups 131 to 133. However, for example, the average intensity of the pixel signals of the light receiving elements 104 included in one or more blocks of the unit group 134 adjacent to the block to which the light receiving element 104 to be subtracted belongs is subtracted. Here, the light receiving element 104 included in the unit group 134 receives only background light because the shutter is opened before the illumination light is emitted. Thereby, noise originating from background light etc. can be removed from the pixel signals of the light receiving elements 104 belonging to each of the unit groups 131 to 133.

Note that the control unit 30 may repeat the exposure time cycle one or more times after one cycle of the exposure time of the unit groups 131 to 135 is completed. For example, in the example shown in FIG. 7B, the control unit 30 executes the second cycle (400 to 800 nanoseconds) after the first cycle (0 to 400 nanoseconds) ends. That is, the control unit 30 sets the exposure time of 400 to 500 nanoseconds, the exposure time of 500 to 600 nanoseconds, the exposure time of 600 to 700 nanoseconds, and the exposure time of 700 to 700 nanoseconds for the light receiving elements 104 belonging to unit groups 131 to 134, respectively. Open the shutter for an exposure time of 800 nanoseconds. In the example shown in FIG. 7C, the control unit 30 executes the second cycle (400 to 800 nanoseconds) after the first cycle (0 to 400 nanoseconds) ends. That is, the control unit 30 sets the exposure time of 400 to 500 nanoseconds, the exposure time of 400 to 600 nanoseconds, the exposure time of 400 to 700 nanoseconds, and the exposure time of 400 to 700 nanoseconds for the light receiving elements 104 belonging to unit groups 131 to 134, respectively. Open the shutter for an exposure time of 800 nanoseconds. In the example shown in FIG. 7D, the control unit 30 executes the second cycle (300 to 600 nanoseconds) after the first cycle (0 to 300 nanoseconds) ends. That is, the control unit 30 controls the light receiving elements 104 belonging to the unit groups 131 to 133 during the exposure time of 300 to 400 nanoseconds, the exposure time of 400 to 500 nanoseconds, and the exposure time of 500 to 600 nanoseconds, respectively. Open the shutter. Note that after the exposure time of the unit groups 131 to 134 ends or after repeating the exposure multiple times, the output signal (pixel signal) of the light receiving element 104 of each unit group is read out, and the processing unit 40 processes it for each unit group. good. This makes it possible to obtain an image of an object existing in a wider range of distances.

FIG. 7E shows another example of the exposure time of each unit group of the light receiving elements 104. In this example, it is assumed that the light receiving elements 104 are grouped into four unit groups 131 to 134 in the group arrangement shown in FIG. 2A or 2B. Further, the control unit 30 sets the exposure time of each of the unit groups 131 to 134 of the light receiving elements 104 to 10 nanoseconds, for example, and sets the exposure time of the plurality of light receiving elements 104 so that the unit groups 131 to 134 are continuous. is controlled. When the exposure time is short in this way, in order to obtain a sufficient amount of light (that is, to accumulate sufficient charge) in each light receiving element 104, the control unit 30 controls the emission of illumination light by the light emitting unit 10 and the Subsequent exposure of the light receiving element 104 may be repeated. For example, the control unit 30 controls the light receiving elements 104 belonging to the unit groups 131 to 134 to have an exposure time of 0 to 10 nanoseconds, and an exposure time of 10 to 20 nanoseconds, based on the time point when the illumination light is emitted by the light emitting unit 10, respectively. The shutter is opened during an exposure time of seconds, an exposure time of 20 to 30 nanoseconds, and an exposure time of 30 to 40 nanoseconds, and each light receiving element 104 receives light. The control unit 30 repeats this exposure operation multiple times within one frame, for example, 100 to 100,000 times, and accumulates the amount of light received by each light receiving element 104. After repeating the emission of illumination light and the exposure operation of the light receiving element 104 within one frame, the output signal (pixel signal) of the light receiving element 104 of each unit group is read out, and the processing unit 40 outputs the output signal for each unit group. May be processed. With this method, a sufficient amount of light can be obtained in each light receiving element 104 while maintaining distance resolution.

Furthermore, when the amount of light received by each light receiving element 104 is insufficient, the processing unit 40 may perform image integration and/or pixel addition. Image integration is the accumulation and averaging of pixel values in multiple frames. For example, when shooting at 1000 fps, a distance image can be obtained at 30 fps by accumulating and averaging the pixel values over 33 frames. Pixel addition (average) is so-called binning, and is the addition (or average) of pixel values of a plurality of adjacent pixels to obtain a pixel value of one pixel.

FIG. 8 shows an example of the configuration of an imaging system when the imaging device 500 according to this embodiment is mounted on a vehicle 600. Vehicle 600 is an example of a moving object, and may be a train, a car, a motorcycle, or the like. Vehicle 600 includes an imaging device 500, a speed sensor 610, and a movement direction sensor 620. The imaging device 500 (at least the light emitting unit 10 and the light receiving unit 20) is mounted on the vehicle 600 so as to image the front. That is, the light emitting section 10 emits illumination light toward the front of the vehicle 600, and the light receiving section 20 receives reflected light that returns from the front of the vehicle 600. Speed sensor 610 is a sensor that detects the speed of vehicle 600 and may be mounted on vehicle 600. Moving direction sensor 620 is a sensor that detects the moving direction of vehicle 600. The moving direction sensor 620 may be, for example, a sensor that detects acceleration applied to the vehicle 600, a sensor that detects the rotation angle of the steering wheel of the vehicle 600, or the like. The detection results of the speed sensor 610 and the movement direction sensor 620 are transmitted to the imaging device 500.

The processing unit 40 of the imaging device 500 controls the exposure time of the light receiving element 104 located in the lower region on the light receiving surface to start earlier than the exposure time of the light receiving element 104 located in the upper region on the light receiving surface. Thereby, an object located on the road near the front of the moving vehicle 600 on the ground can be imaged with high resolution while grasping the distant situation.

The processing unit 40 of the imaging device 500 controls the exposure time of the plurality of light receiving elements 104 according to the detection result of the speed sensor 610.

When the speed of the vehicle 600 increases, the control unit 30 shifts the exposure time of more light receiving elements 104 among the plurality of light receiving elements 104 to a later time as the speed of the vehicle 600 increases. For example, for the unit groups 131 to 134 of the light receiving elements 104 in the group arrangement shown in FIG. 2A, the exposure time is shifted back by a time (for example, 1 to 100 nanoseconds) corresponding to the respective speeds. Alternatively, at least some of the light receiving elements 104 belonging to unit groups 131 to 132 are changed to unit groups 133 to 134, respectively, so that the number of light receiving elements 104 whose exposure time is earlier is decreased and the number of light receiving elements 104 whose exposure time is later is increased.

When the speed of vehicle 600 decreases, control unit 30 shifts the exposure time of more of the plurality of light receiving elements 104 forward as the speed of vehicle 600 decreases. For example, for the unit groups 131 to 134 of the light receiving elements 104 in the group arrangement shown in FIG. 2A, the exposure time is shifted forward by a time (for example, 1 to 100 nanoseconds) corresponding to the respective speeds. Alternatively, at least some of the light receiving elements 104 belonging to the unit groups 133 to 134 are changed to the unit groups 131 to 132, respectively, so that the number of light receiving elements 104 whose exposure time is later is reduced and the number of light receiving elements 104 whose exposure time is earlier is increased.

As a result, as the speed of the vehicle 600 is higher, the exposure time of the light receiving element 104 is shifted backward, and as the speed of the vehicle 600 is lower, the exposure time of the light receiving element 104 is shifted forward, regardless of the speed of the vehicle 600. An object can be detected a certain amount of time before it approaches vehicle 600.

Further, when the vehicle 600 moves forward to the left, the control unit 30 shifts the exposure time of more light receiving elements 104 located in the right region on the light receiving surface earlier (for example, by 1 to 100 nanoseconds). . Alternatively, at least some of the light receiving elements 104 belonging to unit groups 133 to 134 in the right region on the light receiving surface are changed to unit groups 131 to 132, respectively, and the number of light receiving elements 104 whose exposure time is earlier is increased. When the vehicle 600 moves forward to the right, the control unit 30 shifts the exposure time of more light receiving elements 104 located in the left region on the light receiving surface earlier (for example, by 1 to 100 nanoseconds). Alternatively, at least some of the light receiving elements 104 belonging to unit groups 133 to 134 in the left region on the light receiving surface are changed to unit groups 131 to 132, respectively, and the number of light receiving elements 104 whose exposure time is earlier is increased. Thereby, when the vehicle 600 moves to the left front, such as when turning left, an object located near the left front can be imaged with high resolution. Moreover, when the vehicle 600 moves to the right front, such as when turning right, an object located near the right front can be imaged with high resolution.

FIG. 9 shows an example of a flow of imaging processing by the imaging device 500 mounted on the vehicle 600. Imaging device 500 is turned on when the engine of vehicle 600 is started, and executes imaging processing until the engine is stopped. In the imaging device 500, it is assumed that the exposure time of the light receiving element 104 of the light receiving device 100 is set in advance to a standard exposure time, for example, according to the group arrangement shown in FIGS. 2A to 2D.

In step S102, the speed sensor 610 and the moving direction sensor 620 mounted on the vehicle 600 detect the speed and moving direction of the vehicle 600, respectively. Those detection results are transmitted to the imaging device 500 (control unit 30).

In step S104, the control unit 30 of the imaging device 500 determines whether the speed and moving direction of the vehicle 600 have changed. If there is a change, the process proceeds to step S106, and the control unit 30 changes the setting of the exposure time of the light receiving element 104. If there is no change, the process proceeds to step S108, and the control unit 30 maintains the setting of the exposure time of the light receiving element 104.

In step S106, the control unit 30 controls the exposure time of the light receiving elements 104 for each unit group. When the speed of the vehicle 600 increases, the control unit 30 shifts the exposure time of more light receiving elements 104 among the plurality of light receiving elements 104 to the later time as the speed of the vehicle 600 increases, as described above. When the speed of the vehicle 600 decreases, the control unit 30 shifts the exposure time of more light receiving elements 104 among the plurality of light receiving elements 104 forward as the speed of the vehicle 600 decreases, as described above. Thereby, regardless of the speed of vehicle 600, an object can be detected a certain period of time before it approaches vehicle 600 closest.

Further, when the vehicle 600 moves forward to the left, the control unit 30 shifts the exposure time of more light receiving elements 104 located in the right region on the light receiving surface to the front, as described above, so that the vehicle 600 moves to the right. When moving forward, the control unit 30 shifts the exposure time of more light receiving elements 104 located in the left region on the light receiving surface forward, as described above.

In step S108, control unit 30 controls light emitting unit 10 to emit illumination light toward the front of vehicle 600 (that is, the imaging range).

In step S110, the control unit 30 controls the light receiving unit 20 to receive reflected light from the front of the vehicle (i.e., the imaging range) using the plurality of light receiving elements 104 grouped into a plurality of unit groups. do. Here, the exposure time of the light receiving element 104 is controlled for each unit group in advance or in step S106.

In step S112, the processing unit 40 generates, for each unit group, an image (that is, image data) at a distance within the imaging range corresponding to the exposure time of each unit group, based on the pixel signal of the light receiving element 104. For example, when objects 1 to 3 existing within the imaging range shown in FIG. 7A are imaged by the light receiving elements 104 grouped into unit groups 131 to 134 shown in FIG. 2A or 2B, the unit groups of the light receiving elements 104 Since there is no object within the distance range of 0 to 15 m, nothing appears in the image data obtained from the pixel signal of 131. In the image data obtained from the pixel signals of the unit group 132 of the light receiving elements 104, an object 1 existing within a distance of 15 to 30 m appears. In the image data obtained from the pixel signals of the unit group 133 of the light receiving element 104, an object 2 existing within a distance of 30 to 45 meters appears. In the unit group 134 of the light receiving elements 104, objects 3 existing within a distance of 45 to 60 m appear. In this way, objects located in a plurality of distance ranges within the imaging range can be separated from each other and imaged according to the exposure time of each unit group.

In step S114, the processing unit 40 processes the image (ie, image data) obtained in step S112 to identify the object included therein. In the above example, the processing unit 40 specifies that the object 1 existing within a distance of 15 to 30 m from the image data of the unit group 132 is a pedestrian, and from the image data of the unit group 133, the processing unit 40 identifies that the object 1 is a pedestrian. It is specified that the object 2 that exists within the range of is a street tree, and it is specified that the object 3 that exists within the range of distance 45 to 60 m is a vehicle from the image data of the unit group 134.

Note that in the present embodiment, the exposure time of the light receiving element 104 is shifted forward or backward according to the speed of the vehicle 600, but instead of this, or in addition to this, a warning is provided according to the speed of the vehicle 600. Alternatively, the timing at which the vehicle 600 is automatically stopped may be controlled. For example, as the speed of the vehicle 600 increases, the control unit 30 changes the unit group for determining whether a warning or automatic stop should be issued to a unit group with a slower exposure time. When the speed of the vehicle 600 is low, the processing unit 40 issues a warning to the driver or automatically stops the vehicle 600 if it identifies that a pedestrian exists within a distance of 15 to 30 meters from the image data of the unit group 132. do. When the speed of the vehicle 600 is high, the processing unit 40 issues a warning to the driver or automatically stops the vehicle 600 if it identifies that a pedestrian exists within a distance of 45 to 60 meters from the image data of the unit group 134. do.

In such a case, the processing unit 40 determines in advance a target object that should cause a warning or automatic stop of the vehicle 600 when identified from the image data, and also creates a data table showing the relationship between the image and the type of object. Keep it. The processing unit 40 generates image data for each unit group based on the pixel signals of the light receiving element 104, and determines what the imaged object is from the image data by comparing the image and the object using a data table. The object may be identified, and if the identified object is the target object, a warning or automatic stop processing may be executed.

When step S114 is completed, the process returns to step S102 and steps S102 to S114 are repeated. That is, in step S108, the light emitting unit 10 repeatedly emits illumination light toward the imaging range, and in step S110, the light receiving unit 20 receives reflected light from the imaging range each time the light emitting unit 10 emits illumination light. , in step S112, each time the light receiving section 20 receives reflected light, the processing section 40 processes the pixel signals of the light receiving element 104 for each unit group to generate a plurality of images corresponding to a plurality of distances within the imaging range. Then, in step S114, the object included in each image is identified.

Note that in step S114, the processing unit 40 determines the object specified for each unit group of light receiving elements 104 and the exposure time of each unit group of light receiving elements 104 (that is, the imaging device 500 The movement of an object can be detected based on the distance from the object. For example, the processing unit 40 identifies a vehicle (object 3) within a distance of 45 to 60 meters from the image data of the unit group 134 in the first cycle, and also identifies the vehicle (object 3) within a distance of 45 to 60 meters from the image data of the unit group 134 in the next cycle. When a vehicle (object 3) is identified within a range of 60 m, it is detected that the vehicle (object 3) is running in front of vehicle 600 at the same speed and in the same direction. Further, the processing unit 40 identifies a vehicle (object 3) within a distance range of 45 to 60 m from the image data of the unit group 134 in the first cycle, and in the next cycle (referred to as cycle T), the processing unit 40 identifies the vehicle (object 3) from the image data of the unit group 134 When a vehicle (object 3) is identified from the data within a distance range of 30 to 45 m, it is detected that the vehicle (object 3) is approaching the vehicle 600 from in front at a relative speed (approximately 15 m/T). In a further subsequent cycle, when a vehicle (object 3) is identified within a distance of 15 to 30 meters from the image data of the unit group 132, the processing unit 40 detects a collision with the vehicle (object 3) by emitting a warning sound or by emitting a warning light. You may warn the driver by flashing, etc.

According to the imaging device 500 according to the present embodiment, the light-emitting section 10 that emits light, the light-receiving section 20 that has a plurality of light-receiving elements 104 that receives reflected light of the light emitted from the light-emitting section 10, and the plurality of light-receiving elements 104. A control unit 30 is provided to control the exposure time for each unit group 131 to 135 each having at least two light receiving elements 104, and each of the plurality of light receiving elements 104 and the plurality of groups 131 to 135 outputs a signal corresponding to the exposure time. Furthermore, it further includes a processing unit 40 that generates an image at a distance corresponding to the exposure time for each of the plurality of light receiving elements 104 or for each of the plurality of groups 131 to 135, based on the output signal. The control unit 30 controls the exposure time for each of the plurality of light receiving elements 104 or for each of the plurality of groups 131 to 135 each having at least two light receiving elements 104, so that the light emitting unit 10 emits light, The light receiving unit 20 receives reflected light using a plurality of light receiving elements 104, and the processing unit 40 divides images at a distance corresponding to the exposure time into each of the plurality of light receiving elements 104 or into a plurality of groups based on the output signal. By generating each unit group 131 to 135, it is possible to separate the images into a plurality of distances according to the exposure time of each unit group and image the object located at each distance.

Note that in the imaging device 500 according to the present embodiment, one or a plurality of adjacent light receiving elements 104 constitute one block 131b to 135b, and a plurality of units each including a plurality of arbitrarily arranged blocks 131b to 135b, respectively. The groups 131 to 135 are configured and the exposure time is controlled for each unit group 131 to 135. However, instead of configuring a block or unit group of light receiving elements 104, each of the plurality of light receiving elements 104 is controlled. It is also possible to control the exposure time. Furthermore, in the imaging device 500 according to the present embodiment, an example is shown in which the exposure time is controlled for each of the plurality of light receiving elements 104 or for each of the plurality of unit groups 131 to 135, and images are acquired at a distance corresponding to the exposure time. , instead of or in addition to this, distances corresponding to each exposure time may be measured.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the range described in the above embodiments. It will be apparent to those skilled in the art that various changes or improvements can be made to the embodiments described above. It is clear from the claims that such modifications or improvements may be included within the technical scope of the present invention.

The execution order of each process such as operation, procedure, step, and stage in the apparatus, system, program, and method shown in the claims, specification, and drawings specifically refers to "before" and "prior to". It should be noted that they can be implemented in any order unless explicitly stated as such, and unless the output of a previous process is used in a subsequent process. With regard to the claims, specification, and operational flows in the drawings, even if the terms "first," "next," etc. are used for convenience, this does not mean that the operations must be carried out in this order. isn't it.

DESCRIPTION OF SYMBOLS 10... Light emitting part, 20... Light receiving part, 21... Optical system, 30... Control part, 40... Processing part, 100... Light receiving device, 101... Microlens, 102... Color filter, 103... Passivation film, 104... Light receiving element, 105...Transistor, 106...PD layer, 107...Wiring, 108...Wiring layer, 109...Bump, 110...Silicon through electrode, 111...Signal processing chip, 112...Memory chip, 113...Imaging chip, 131-135...Unit group , 150... Pixel, 152... Transfer transistor, 154... Reset transistor, 156... Amplification transistor, 158... Selection transistor, 300, 310, 320... Reset wiring, 302, 312, 322... Transfer wiring, 304... Power supply wiring, 306, 316, 326... Selection wiring, 308, 330... Output wiring, 309... Load current source, 411... Multiplexer, 412... Signal processing circuit, 413... Demultiplexer, 414... Pixel memory, 415... Arithmetic circuit, 500... Imaging device, 600...Vehicle, 610...Speed sensor, 620...Movement direction sensor.

Claims (16)

a light emitting part that emits light;
a plurality of groups each having a plurality of light receiving elements that receive reflected light of the light emitted from the light emitting section;
a control unit that controls at least one of the timing of receiving the reflected light and the exposure time for each of the plurality of light receiving elements or for each of the plurality of groups so as to correspond to different imaging ranges;
a processing unit that generates, for each of the plurality of groups, an image of the object in the imaging range corresponding to each group ;
The light emitting unit and the plurality of light receiving elements are provided on a moving body, and the control unit is configured to control a right area on the light receiving surface where the plurality of light receiving elements are arranged when the moving body moves to the left front. An imaging device that shifts the exposure time of a light-receiving element positioned forward, and shifts the exposure time of a light-receiving element located in a left region on the light-receiving surface forward when the moving object moves forward to the right. The imaging range includes at least one of a distance range from the plurality of light receiving elements to the target object, and a time range from when light is emitted from the light emitting section until it is received by the plurality of light receiving elements. 1. The imaging device according to 1. The imaging device according to claim 1 or 2 , wherein each of the plurality of groups has light receiving elements located in different areas on the light receiving surface. The imaging device according to claim 3 , wherein at least one group among the plurality of groups includes a plurality of blocks on the light receiving surface, each of which includes at least one light receiving element. The imaging device according to claim 3 or 4 , wherein the timing of the light receiving element located in the lower region on the light receiving surface is earlier than the timing of the light receiving element located in the upper region on the light receiving surface. The imaging device according to any one of claims 3 to 5 , wherein the control unit controls the exposure time of the plurality of light receiving elements so that the exposure time of the plurality of groups is continuous among the plurality of groups. Device. The control unit controls the plurality of light-receiving elements so that the number of light-receiving elements included in a group with an earlier exposure time among the plurality of groups is greater than the number of light-receiving elements included in a group with a later exposure time. The imaging device according to claim 6 , wherein the imaging device controls the exposure time of the element. 8. The control unit controls the exposure time of the plurality of light receiving elements so that the exposure time of each of the plurality of groups starts at the same time and ends at mutually different times. The imaging device according to item 1. The control unit controls the exposure time of the plurality of light receiving elements so that at least one group among the plurality of groups is exposed before the light is emitted,
8. The processing unit subtracts the intensity of the output signal of the light receiving element included in the at least one group from the intensity of the output signal of the light receiving element included in each of the remaining groups of the plurality of groups. The imaging device described in . The imaging device according to any one of claims 3 to 9 , wherein the processing unit specifies an object included in the generated image. The imaging device according to claim 10 , wherein the processing unit detects movement of the object based on the object specified for each of the plurality of groups and the exposure time of each of the plurality of groups. The imaging device according to any one of claims 3 to 11 , wherein the control unit controls the exposure time of the plurality of light receiving elements according to the speed of the moving body. The imaging device according to claim 12 , wherein the control unit shifts the exposure time of at least some of the plurality of light receiving elements to a later time as the speed of the moving body increases. The imaging device according to any one of claims 1 to 13 , wherein the plurality of light receiving elements or the plurality of groups are arranged between other light receiving elements arranged in a Bayer array. a light emitting part that emits light;
a plurality of groups each having a plurality of light receiving elements that receive reflected light of the light emitted from the light emitting section;
a control unit that controls at least one of the timing of receiving the reflected light and the exposure time for each of the plurality of light receiving elements or for each of the plurality of groups so as to correspond to different imaging ranges;
a processing unit that generates, for each of the plurality of groups, an image of the object in the imaging range corresponding to each group;
The light emitting unit and the plurality of light receiving elements are provided in a moving body, and the control unit controls the exposure time of at least some of the plurality of light receiving elements to be later as the speed of the moving body is faster. An imaging device that controls exposure time of the plurality of light receiving elements in accordance with the speed of the moving object so as to shift the exposure time of the plurality of light receiving elements. A mobile object,
An imaging device according to any one of claims 1 to 15 ,
a speed sensor that detects the speed of the moving body;
a moving direction sensor that detects the moving direction of the moving body;
A mobile body equipped with.

Images