JP7490928B2 - Image sensor and image pickup device - Google Patents

Description

The present invention relates to an imaging element and an imaging apparatus.

There is a known imaging device in which the pixel section is divided into two regions and the pixels are read out (Patent Document 1). However, when a subject moving at a relatively high speed is imaged using a conventional imaging device, there is a problem in that the image is misaligned at the boundary between the two regions, resulting in a very unnatural image.

JP 2013-243781 A

According to a first aspect of the present invention, an imaging element includes a plurality of pixels arranged in a first direction and a second direction different from the first direction in a first region and provided on a first substrate, a plurality of pixels arranged in the first direction and the second direction in a second region arranged in the second direction with respect to the first region and provided on the first substrate, a plurality of pixels arranged in the first direction and the second direction in a third region arranged in the second direction with respect to the second region and provided on the first substrate, and a first readout operation for reading out signals of the pixels in the first region from one side to the other side of the first region in the second direction, and a second readout operation for reading out signals of the pixels in the second region from one side to the other side of the first region in the second direction . the pixel signal read by the second read operation in the first frame, the pixel signal read by the second read operation in the second frame , and the pixel signal read by the third read operation in the third frame. The pixel signal read by the second read operation in the second frame, and the pixel signal read by the third read operation in the third frame are output from the readout section.
According to a second aspect of the present invention, an imaging device includes the imaging element according to the first aspect, and a generation section that generates image data based on the image signal output from the imaging element.

1 is a block diagram showing a configuration of an imaging apparatus according to a first embodiment. 1 is a block diagram showing a configuration of a portion of an image sensor according to a first embodiment; 1 is a circuit diagram showing a configuration of a pixel of an image sensor according to a first embodiment. 5A to 5C are diagrams illustrating an example of a method for reading out pixel signals during moving image shooting by an imaging control unit of the imaging device according to the first embodiment. 5A to 5C are diagrams for explaining whether or not a subject image is distorted when the imaging device according to the first embodiment captures a moving image of a subject that moves at high speed. FIG. 11 is a block diagram showing a configuration of a portion of an image sensor according to a second embodiment. 13 is a diagram showing an example of a method for reading out pixel signals during moving image shooting by an imaging control unit of an imaging device according to a second embodiment. FIG. 13 is a diagram showing an example of a method for reading out pixel signals during moving image shooting by an imaging control unit of an imaging device according to Modification 1. FIG.

(First embodiment)
1 is a diagram showing an example of the configuration of an electronic camera 1 (hereinafter referred to as camera 1) which is an example of an imaging device according to a first embodiment. Camera 1 includes a photographing optical system (imaging optical system) 2, an image sensor 3, a control unit 4, a memory 5, a display unit 6, and an operation unit 7. The photographing optical system 2 has a plurality of lenses and an aperture, and forms a subject image on the image sensor 3. Note that the photographing optical system 2 may be detachable from the camera 1.

The imaging element 3 is, for example, a CMOS image sensor. The imaging element 3 captures an image of a subject formed by the photographing optical system 2. The imaging element 3 has a plurality of pixels, each having a photoelectric conversion unit, arranged in row and column directions. The photoelectric conversion unit is, for example, composed of a photodiode (PD). The imaging element 3 photoelectrically converts incident light to generate a pixel signal, and outputs the generated pixel signal to the control unit 4. The pixel signal is a signal generated based on the charge photoelectrically converted by the photoelectric conversion unit.

The memory 5 is a recording medium such as a memory card. Image data and the like are recorded in the memory 5. The control unit 4 writes data to the memory 5 and reads data from the memory 5. The display unit 6 displays images based on the image data, information related to shooting such as shutter speed and aperture value, and menu screens. The operation unit 7 includes various setting switches such as a video shooting button and a power switch, and outputs operation signals corresponding to the respective operations to the control unit 4.

The control unit 4 is composed of a CPU, ROM, RAM, etc., and controls each part of the camera 1 based on a control program. The control unit 4 also has a shooting instruction unit 41, an imaging control unit 42, a synthesis unit 43, and a video data generation unit 44. When the shooting instruction unit 41 detects that the video shooting button has been operated based on an operation signal output from the operation unit 7, it instructs the imaging control unit 42 to shoot a video.

The imaging control unit 42 supplies a control signal to the imaging element 3 to control the operation of the imaging element 3. When the shooting instruction unit 41 instructs the imaging element 3 to shoot a video, the imaging control unit 42 causes the imaging element 3 to repeatedly shoot a subject image for each frame at a predetermined interval and output a pixel signal. In this case, the imaging control unit 42 performs a so-called rolling shutter readout control in which the pixels of the imaging element 3 are sequentially selected row by row and pixel signals are read out from the selected pixels.

The imaging control unit 42 determines the timing for sequentially selecting pixels in a first pixel region (described later) and the timing for sequentially selecting pixels in a second pixel region. The imaging control unit 42 controls the imaging element 3 based on the determined timing.

The synthesis unit 43 generates an image signal for each frame by performing synthesis processing using pixel signals repeatedly output from the image sensor 3 during video shooting. As will be described in detail later, the synthesis unit 43 synthesizes the pixel signals of each pixel in the first pixel area generated in the first frame of two consecutive frames and the pixel signals of each pixel in the second pixel area generated in the second frame to generate an image signal for one frame (one screen).

When video shooting is instructed, the video data generation unit 44 performs various image processing on the image signals of each frame generated by the synthesis unit 43 to generate video data. Image processing includes well-known image processing such as tone conversion processing, color interpolation processing, and edge enhancement processing. When recording video data, the video data generation unit 44 performs compression processing on the generated video data and then records the compressed video data in the memory 5.

Figure 2 is a block diagram showing a partial configuration of the image sensor 3 according to the first embodiment. The image sensor 3 includes a pixel section (pixel region) 20, a vertical drive section 30, a first column circuit section 40a, a second column circuit section 40b, a first horizontal transfer section 50a, a second horizontal transfer section 50b, an output processing section 60, and an output section 70.

The pixel section 20 includes m rows and n columns of pixels 10, and the pixels 10 are arranged at a first interval in the row direction and at a second interval in the column direction. The number of pixels 10 is, for example, several million pixels to several hundred million pixels. The pixel section 20 is divided into two equal regions, a first pixel region 21a and a second pixel region 21b, in the column direction, i.e., in the vertical direction. Therefore, each of the first pixel region 21a and the second pixel region 21b includes (m×n/2) pixels 10. In the first pixel region 21a, a vertical signal line 25a is provided for each column of the pixels 10 arranged in the column direction, i.e., in the vertical direction. In the second pixel region 21b, a vertical signal line 25b is also provided for each column of the pixels 10 arranged in the column direction, i.e., in the vertical direction.

The vertical drive unit 30 is controlled by the imaging control unit 42 of the camera 1, and supplies control signals such as a signal TX, a signal RST, and a signal SEL (described later) to each pixel 10 to control the operation of each pixel 10. The vertical drive unit 30 sequentially selects the pixels 10 in the first pixel region 21a row by row, from the bottom row to the top row, and simultaneously selects the pixels 10 in the second pixel region 21b row by row, from the bottom row to the top row. In other words, the vertical drive unit 30 reads out pixel signals while simultaneously scanning the pixels in the first pixel region 21a and the pixels in the second pixel region 21b row by row in the same direction.

The pixel signals of the pixels 10 in the first pixel region 21a, which are selected in sequence by the vertical drive unit 30, are read out in sequence to the vertical signal line 25a connected to the pixels 10. Similarly, the pixel signals of the pixels 10 in the second pixel region 21b, which are selected in sequence by the vertical drive unit 30, are read out in sequence to the vertical signal line 25b connected to the pixels 10.

A first column circuit section 40a is provided for the first pixel region 21a, and the first column circuit section 40a is configured to include a plurality of current sources 41a, a plurality of amplifier sections 42a, and a plurality of analog/digital conversion sections (AD conversion sections) 43a. The current source 41a, the amplifier section 42a, and the AD conversion section 43a in the first column circuit section 40a are provided for each vertical signal line 25a in the first pixel region 21a. In detail, the current source 41a is connected to each pixel 10 via the vertical signal line 25a. The current source 41a generates a current for reading out a pixel signal from each pixel 10, and supplies the generated current to the vertical signal line 25a and each pixel 10. The amplifier section 42a amplifies the pixel signal input via the vertical signal line 25a with a predetermined gain (amplification rate) and outputs the amplified pixel signal to the AD conversion section 43a.

The AD conversion unit 43a converts the signals input from each pixel 10 via the amplifier unit 42a into digital signals and outputs the converted digital signals to the first horizontal transfer unit 50a. The first horizontal transfer unit 50a sequentially outputs the pixel signals converted into digital signals by each AD conversion unit 43a to the output processing unit 60.

A second column circuit section 40b is provided for the second pixel region 21b, and the second column circuit section 40b includes a plurality of current sources 41b, a plurality of amplifier sections 42b, and a plurality of AD conversion sections 43b. The current source 41b, the amplifier section 42b, and the AD conversion section 43b in the second column circuit section 40b are provided for each vertical signal line 25b in the second pixel region 21b. In detail, the current source 41b is connected to each pixel 10 via the vertical signal line 25b. The current source 41b generates a current for reading out a pixel signal from each pixel 10, and supplies the generated current to the vertical signal line 25b and each pixel 10. The amplifier section 42b amplifies the pixel signal input via the vertical signal line 25b with a predetermined gain, and outputs the amplified pixel signal to the AD conversion section 43b.

The AD conversion unit 43b converts the signal input from each pixel 10 via the amplifier unit 42b into a digital signal and outputs the converted digital signal to the horizontal transfer unit 50b. The horizontal transfer unit 50b sequentially outputs the pixel signals converted into digital signals by each AD conversion unit 43b to the output processing unit 60.

The output processing unit 60 includes a timing control unit 61 and a storage unit 62. The timing control unit 61, which will be described in detail later, performs a process of storing the pixel signals of the pixels 10 in the first pixel region 21a, which are sequentially input from the first horizontal transfer unit 50a, in the storage unit 62, and a process of reading out the pixel signals of the pixels 10 in the first pixel region 21a stored in the storage unit 62 at a predetermined timing and outputting them to the output unit 70. As a result, the storage unit 62 stores the pixel signals read out from each pixel 10 in the first pixel region 21a for each frame during video shooting.

The timing control unit 61 also performs processing to output the pixel signals of the pixels 10 in the second pixel region 21b input from the second horizontal transfer unit 50b to the output unit 70 at a predetermined timing. In this way, as described in detail later, the output processing unit 60 outputs the pixel signals read out from the first pixel region 21a in a certain frame to the output unit 70, and subsequently outputs the pixel signals read out from the second pixel region 21b in the next frame to the output unit 70. The output unit 70 has input/output circuits and the like compatible with high-speed interfaces such as SLVS and LVDS, and outputs (transmits) signals to the control unit 4 at high speed.

The imaging element 3 reads out pixel signals from the pixels 10 in the first pixel region 21a and pixel signals from the pixels 10 in the second pixel region 21b through separate paths. That is, the imaging element 3 can simultaneously read out pixel signals from two rows of pixels 10 by simultaneously selecting the pixels 10 in the first pixel region 21a and the second pixel region 21b on a row-by-row basis. Therefore, the imaging element 3 can read out pixel signals in approximately half the time compared to when pixel signals are read out from each pixel 10 through only one path, achieving twice the frame rate.

Fig. 3 is a circuit diagram showing the configuration of a pixel 10 of an image sensor 3 according to the first embodiment. As shown in Fig. 3, the pixel 10 has a photoelectric conversion unit 11, a transfer unit 12, a reset unit 13, a floating diffusion 14, an amplifier unit 15, and a selection unit 16. The photoelectric conversion unit 11 has a function of converting incident light into an electric charge and storing the photoelectrically converted electric charge.

The transfer unit 12 is controlled by a signal TX and transfers the charge photoelectrically converted by the photoelectric conversion unit 11 to the floating diffusion 14. That is, the transfer unit 12 forms a charge transfer path between the photoelectric conversion unit 11 and the floating diffusion 14. The capacitance FD of the floating diffusion 14 accumulates charge. The amplifier 15 outputs a pixel signal obtained by amplifying the signal due to the charge accumulated in the capacitance FD. In the example shown in FIG. 3, the amplifier 15 is composed of a transistor M3 whose drain terminal, gate terminal, and source terminal are connected to the power supply VDD, the floating diffusion 14, and the selection unit 16, respectively. The source terminal of the amplifier 15 is connected to the vertical signal line 25 (25a, 25b) via the selection unit 16. The amplifier 15 functions as a part of a source follower circuit, using the current source 41 (41a, 41b) in FIG. 2 connected to the vertical signal line 25 as a load current source.

The reset unit 13 is controlled by a signal RST to reset the charge in the capacitance FD and reset the potential of the floating diffusion 14 to a reset potential (reference potential). The selection unit 16 is controlled by a signal SEL to output a pixel signal from the amplification unit 15 to a vertical signal line 25. The transfer unit 12, the reset unit 13, and the selection unit 16 are each composed of, for example, a transistor M1, a transistor M2, and a transistor M4, respectively.

Figure 4 is a diagram showing an example of a method for reading out pixel signals during video shooting by the imaging control unit 42 of the imaging device 1 according to the first embodiment. The vertical axis indicates the rows of the pixels 10 in each of the first pixel region 21a and the second pixel region 21b, and the horizontal axis indicates the timing (time t) at which pixel signals are output from each of the pixels 10 in the first pixel region 21a and the second pixel region 21b. Figure 4 shows a schematic diagram of the transition of the rows of the pixels 10 selected by the vertical drive unit 30 for each of the first pixel region 21a and the second pixel region 21b, that is, the transition of the pixels 10 that output pixel signals. In the example shown in Figure 4, in order to simplify the explanation, an example is shown in which pixel signals are read out from four rows of pixels 10 in the first pixel region 21a and the second pixel region 21b.

In the Nth frame shown in FIG. 4, arrow A1 indicates that the readout of pixel signals from pixels 10 in the first row (bottom row) of the first pixel region 21a starts at time t1, and arrow A2 indicates that the readout of pixel signals from pixels 10 in the second row of the first pixel region 21a starts at time t2. Arrows A3 and A4 indicate that the readout of pixel signals from pixels 10 in the third and fourth rows (top row, i.e., last row) of the first pixel region 21a starts at times t3 and t4, respectively.

In the Nth frame, arrow B1 indicates that readout of pixel signals from pixels 10 in the first row (bottom row) of the second pixel region 21b starts at time t1. Arrows B2, B3, and B4 indicate that readout of pixel signals from pixels 10 in the second, third, and fourth rows (top row, i.e., last row) of the second pixel region 21b starts at times t2, t3, and t4, respectively.

Similarly, in the (N+1)th frame, arrows A1, A2, A3, and A4 indicate that readout of pixel signals from pixels 10 in the first, second, third, and fourth rows of the first pixel region 21a starts at times t5, t6, t7, and t8, respectively. Arrows B1, B2, B3, and B4 indicate that readout of pixel signals from pixels 10 in the first, second, third, and fourth rows of the second pixel region 21b starts at times t5, t6, t7, and t8, respectively, in the (N+1)th frame.
In the (N+2)th frame, arrows A1, A2, A3, and A4 indicate that readout of pixel signals from pixels 10 in the first, second, third, and fourth rows of the first pixel region 21a starts at times t9, t10, t11, and t12, respectively. Arrows B1, B2, B3, and B4 indicate that readout of pixel signals from pixels 10 in the first, second, third, and fourth rows of the second pixel region 21b starts at times t9, t10, t11, and t12, respectively, in the (N+2)th frame.

As described above, in each frame, the pixel signals of the first row pixels of the first pixel region 21a and the first row pixels of the corresponding second pixel region 21b are read out simultaneously. That is, in each frame, the pixels of the corresponding rows of the first pixel region 21a and the second pixel region 21b are read out simultaneously.

The time intervals from time t1 to t2, t2 to t3, t3 to t4, t4 to t5, t5 to t6, t6 to t7, t7 to t8, t8 to t9, t9 to t10, t10 to t11, t11 to t12, and t12 to t13 are the same constant interval (Δt).

That is, in each frame, the pixel signals from the pixels 10 in the first, second, third, and fourth rows of the first and second pixel regions 21a and 21b are read out at the same fixed interval (Δt). Furthermore, the interval between the readout time t4 of the pixels in the fourth row (last row) of the first and second pixel regions 21a and 21b in the Nth frame and the readout time t5 of the pixels in the first row of the first and second pixel regions 21a and 21b in the (N+1)th frame is also the above-mentioned fixed interval (Δt). Similarly, the interval between the readout time t8 of the pixels in the fourth row (last row) of the first and second pixel regions 21a and 21b in the (N+1)th frame and the readout time t9 of the pixels in the first row of the first and second pixel regions 21a and 21b in the (N+2)th frame is also the above-mentioned fixed interval (Δt).

Therefore, the pixel readouts A1, A2, A3, and A4 of each row of the first pixel region 21a in the Nth frame and the pixel readouts B1, B2, B3, and B4 of each row of the second pixel region 21b in the (N+1)th frame, which are surrounded by the dashed frame G1 in Fig. 4, are all performed at the same fixed interval (Δt). Similarly, the pixel readouts A1, A2, A3, and A4 of each row of the first pixel region 21a in the (N+1)th frame and the pixel readouts B1, B2, B3, and B4 of each row of the second pixel region 21b in the (N+2)th frame, which are surrounded by the dashed frame G2 in Fig. 4, are all performed at the same fixed interval (Δt).

The synthesis unit 43 shown in FIG. 1 synthesizes the pixel signal read from the first pixel region 21a of the Nth frame and the pixel signal read from the second pixel region 21b of the (N+1)th frame to generate an image signal for one frame. Similarly, the synthesis unit 43 synthesizes the pixel signal read from the first pixel region 21a of the (N+1)th frame and the pixel signal read from the second pixel region 21b of the (N+2)th frame to generate an image signal for one frame. In this way, as described above, the synthesis unit 43 synthesizes the pixel signal of each pixel of the first pixel region generated in the first frame and the pixel signal of each pixel of the second pixel region generated in the second frame of two consecutive frames to generate an image signal for one frame.

Next, the operation of the camera 1 according to this embodiment will be described. When the shooting instruction unit 41 instructs video shooting, the imaging control unit 42 reads out pixel signals from the first and second pixel areas 21a and 21b at the readout timing shown in FIG. 4. That is, in the Nth frame, the vertical drive unit 30 reads out pixel signals in order from the pixel 10 in the first row to the pixel 10 in the last row of each of the first and second pixel areas 21a and 21b.

The timing control unit 61 of the output processing unit 60 sequentially stores the pixel signals read from the first pixel area 21a in the Nth frame in the memory unit 62, and sequentially outputs the pixel signals read from the second pixel area 21b in the Nth frame to the control unit 4 via the output unit 70.

The vertical drive unit 30 also reads out pixel signals in the (N+1)th frame from the pixel 10 in the first row to the pixel 10 in the last row of each of the first and second pixel regions 21a and 21b. The timing control unit 61 of the output processing unit 60 sequentially outputs the pixel signals of the first pixel region 21a of the Nth frame stored in the memory unit 62 to the control unit 4 via the output unit 70, and after this output is completed, sequentially outputs the pixel signals of the second pixel region 21b of the (N+1)th frame to the control unit 4 via the output unit 70. The output of the pixel signals of the first pixel region 21a of the Nth frame and the pixel signals of the second pixel region 21b of the (N+1)th frame to the control unit 4 is performed, for example, during the time interval 4Δt of the (N+1)th frame. Of course, the output of the pixel signal of the first pixel region 21a in the Nth frame and the pixel signal of the second pixel region 21b in the (N+1)th frame to the control unit 4 is not limited to during the (N+1)th frame, but may span the (N+1)th and (N+2)th frames as long as it is within the time interval 4Δt.

In this way, the pixel signal of the first pixel region 21a in the Nth frame and the pixel signal of the second pixel region 21b in the (N+1)th frame, which are surrounded by a dashed frame G1 in Figure 4, are sent to the synthesis unit 43 of the control unit 4 as a single time-series signal.
Furthermore, the timing control unit 61 of the output processing unit 60 sequentially stores in the storage unit 62 the pixel signals read out from the first pixel region 21a in the (N+1)th frame.

In the (N+2)th frame, the vertical drive unit 30 reads out pixel signals from the first and second pixel regions 21a and 21b in the same manner as in the Nth and (N+1)th frames. The timing control unit 61 of the output processing unit 60 sequentially outputs the pixel signals of the first pixel region 21a of the (N+1)th frame stored in the memory unit 62 to the control unit 4 via the output unit 70, and after this output is completed, sequentially outputs the pixel signals of the second pixel region 21b of the (N+2)th frame to the control unit 4 via the output unit 70. The output of the pixel signals of the first pixel region 21a of the (N+1)th frame and the pixel signals of the second pixel region 21b of the (N+2)th frame to the control unit 4 is also performed, for example, during the time interval 4Δt of the (N+2)th frame. In this case, the output of the pixel signal of the first pixel region 21a in the (N+1)th frame and the pixel signal of the second pixel region 21b in the (N+2)th frame is not limited to during the (N+2)th frame, but may span the (N+2)th and (N+3)th frames as long as it is within the time interval 4Δt.

In this manner, the pixel signal of the first pixel region 21a in the (N+1)th frame and the pixel signal of the second pixel region 21b in the (N+2)th frame, which are surrounded by a dashed frame G2 in FIG. 4, are sent to the synthesis unit 43 of the control unit 4 as a single time-series signal.
Furthermore, the timing control unit 61 of the output processing unit 60 sequentially stores in the storage unit 62 the pixel signals read out from the first pixel region 21a in the (N+2)th frame.

The synthesis unit 43 of the control unit 4 synthesizes the pixel signal of the first pixel area 21a of the Nth frame with the pixel signal of the second pixel area 21b of the (N+1)th frame to generate an image signal for one frame. The synthesis unit 43 also synthesizes the pixel signal of the first pixel area 21a of the (N+1)th frame with the pixel signal of the second pixel area 21b of the (N+2)th frame to generate an image signal for one frame. The synthesis unit 43 generates image signals for multiple frames by performing a similar synthesis process on the pixel signals output from the imaging element 3 in each frame after the (N+2)th frame. The video data generation unit 44 performs various image processes on the image signals of each frame generated by the synthesis unit 43 to generate video image data.

Figure 5 is a diagram for explaining the presence or absence of distortion in the image of a subject when the camera 1 of this embodiment captures a video of a subject moving at high speed. Figure 5(a) is a diagram that shows a model of the shape of a subject 90 moving at high speed in the direction of the arrow. Figure 5(b) is a diagram that shows a model of a subject image of the subject 90 captured by the camera 1 of this embodiment. Figure 5(c) is a comparative example to the subject image of Figure 5(b), and is a diagram that shows a model of a subject image captured by a conventional imaging device.

In FIG. 5(b), the subject image 90A captured by the camera 1 is composed of an image 101 captured by the pixel signal of the first pixel region 21a in, for example, the Nth frame, and an image 102 captured by the pixel signal of the second pixel region 21b in, for example, the (N+1)th frame. The subject image 90A is deformed into a parallelogram due to so-called rolling shutter distortion (focal plane distortion) for the rectangular subject 90, but the side 101a of the captured image 101 and the side 102a of the captured image 102 are smoothly connected in a straight line.

The reason why side 101a of captured image 101 and side 102a of captured image 102 are smoothly connected in a straight line is as follows. That is, the readout timings A1, A2, A3, and A4 for each row in first pixel region 21a of the Nth frame and the readout timings B1, B2, B3, and B4 for each row in second pixel region 21b of the (N+1)th frame shown in FIG. 4 are all performed at regular intervals (Δt).

Figure 5(c) shows, as a comparative example, a subject image 90B in which the pixel section is divided vertically into a first and a second pixel region and pixel signals read out from the first and second pixel regions in the same frame are combined. In Figure 5(c), for example, the boundary between an image 101 captured by the pixel signal of the first pixel region in the Nth frame and an image 102 captured by the pixel signal of the second pixel region in the same Nth frame is shifted in the direction of movement of the subject. That is, the boundary between side 101b of image 101 and side 102b of image 102 is shifted, creating a step, resulting in a very unnatural image compared to the rectangular shape of subject 90.

In this way, in the image sensor 3 of this embodiment, pixel signals are read out from the first pixel region 21a and the second pixel region 21b using two paths. Furthermore, the synthesis unit 43 of the camera 1 synthesizes the pixel signal read out from the first pixel region 21a in the first frame and the pixel signal read out from the second pixel region 21b in the second frame. As a result, the camera 1 according to this embodiment can increase the frame rate and obtain images with reduced unnatural distortion.

According to the above-described embodiment, the following advantageous effects can be obtained.
(1) The imaging device 1 includes a first region (first pixel region 21a) having a plurality of pixels 10 arranged in a first direction and a second direction different from the first direction, a second region (second pixel region 21b) having a plurality of pixels 10 arranged in the first direction and the second direction with respect to the first region, a readout unit (imaging control unit 42, vertical drive unit 30) that repeats a readout operation multiple times in which signals of the pixels 10 in the first region are read out from one side to the other side of the first region in the second direction and signals of the pixels 10 in the second region are read out from one side to the other side of the second region in the second direction, and an image generation unit (combination unit 43) that generates an image from signals read out from the first region by the first readout operation and signals read out from the second region by the second readout operation following the first readout operation. This makes it possible to increase the frame rate and obtain an image with reduced unnatural distortion.

Second Embodiment
An imaging device according to a second embodiment will be described with reference to the drawings. The imaging device according to the second embodiment has a configuration similar to that of the imaging device 1 shown in FIG. 1. FIG. 6 is a block diagram showing a configuration of a part of an imaging element 3 according to the second embodiment. In the first embodiment, the pixel section 20 is divided into two, a first pixel region 21a and a second pixel region 21b. In contrast, in the second embodiment, the pixel section is divided into three or more pixel regions, and a column circuit section and a horizontal transfer section for each pixel region are provided on a substrate other than the substrate on which the pixel section 20 is provided. In the drawings, the same reference numerals are used for the same or corresponding parts to the first embodiment, and differences will be mainly described.

The imaging element 3 includes a first substrate 201 and a second substrate 202. The first substrate 201 and the second substrate 202 are each made of a semiconductor substrate or the like, and are stacked on top of each other. The first substrate 201 is provided with a pixel section 20 in which a plurality of pixels 10 are arranged in row and column directions. The pixel section 20 is divided into three equal regions: a first pixel region 21a, a second pixel region 21b, and a third pixel region 21c.

In the first pixel region 21a, a vertical signal line 25a is provided for each column of pixels 10. Similarly, in the second pixel region 21b, a vertical signal line 25b is provided for each column of pixels 10, and in the third pixel region 21c, a vertical signal line 25c is provided for each column of pixels 10. The pixel signals of the pixels 10 in the first pixel region 21a are sequentially read out to the vertical signal line 25a connected to the pixels 10. Similarly, the pixel signals of the pixels 10 in the second pixel region 21b are sequentially read out to the vertical signal line 25b, and the pixel signals of the pixels 10 in the third pixel region 21c are sequentially read out to the vertical signal line 25c.

The second substrate 202 is provided with a first column circuit section 40a and a first horizontal transfer section 50a, a second column circuit section 40b and a second horizontal transfer section 50b, and a third column circuit section 40c and a third horizontal transfer section 50c. The first column circuit section 40a and the first horizontal transfer section 50a are provided for the first pixel region 21a, the second column circuit section 40b and the second horizontal transfer section 50b are provided for the second pixel region 21b, and the third column circuit section 40c and the third horizontal transfer section 50c are provided for the third pixel region 21c.

In the first substrate 201 and the second substrate 202, the first pixel region 21a of the first substrate 201 and the first column circuit section 40a and the first horizontal transfer section 50a of the second substrate 202 correspond in position, i.e., they overlap each other. Similarly, the second pixel region 21b, the second column circuit section 40b, and the second horizontal transfer section 50b also correspond in position, and the third pixel region 21c, the third column circuit section 40c, and the third horizontal transfer section 50c also correspond in position.

The vertical signal line 25a and the amplifier section 42a of the first column circuit section 40a are connected via a through electrode, a bump, or the like, as shown diagrammatically by the dashed line 210. Similarly, the pixel 10 of the second pixel region 21b and the amplifier section 42b of the second column circuit section 40b, and the pixel 10 of the third pixel region 21c and the amplifier section 42c of the third column circuit section 40c are connected via a through electrode, a bump, or the like, as shown diagrammatically by the dashed line 210.

In the second embodiment, the imaging control unit 42 determines the timing for sequentially selecting pixels in the first pixel region 21a, the timing for sequentially selecting pixels in the second pixel region 21b, and the timing for sequentially selecting pixels in the third pixel region 21c. The vertical driving unit 30 scans the pixels 10 in the first pixel region 21a, the pixels 10 in the second pixel region 21b, and the pixels 10 in the third pixel region 21c simultaneously in the same direction for each row to read out pixel signals. The synthesis unit 43 synthesizes the pixel signals of each pixel in the first pixel region 21a of the first frame, the pixel signals of each pixel in the second pixel region 21b of the second frame, and the pixel signals of each pixel in the third pixel region 21c of the third frame to generate an image signal for one frame.

In this manner, in the image sensor 3 according to the present embodiment, the pixel section 20 is divided into three parts, the first, second and third pixel regions 21a, 21b and 21c, and three vertical signal lines 25a, 25b and 25c are provided. This shortens the length of each vertical signal line, and reduces the number of pixels 10 connected to each vertical signal line. This reduces the load on the vertical signal lines (parasitic capacitance, parasitic resistance, etc.), and shortens the time required to read out pixel signals from the pixels 10.

Figure 7 is a diagram showing an example of a method for reading out pixel signals during video shooting by the imaging control unit 42 of the imaging device 1 according to the second embodiment. The vertical axis indicates the rows of the pixels 10 in each of the first pixel region 21a, the second pixel region 21b, and the third pixel region 21c. The horizontal axis indicates the timing (time t) at which pixel signals are output from each of the pixels 10 in the first pixel region 21a, the second pixel region 21b, and the third pixel region 21c. Figure 7 shows a schematic diagram of the transition of the rows of the pixels 10 selected by the vertical drive unit 30 for each of the first, second, and third pixel regions 21a, 21b, and 21c, that is, the transition of the pixels 10 that output pixel signals. In the example shown in Figure 7, in order to simplify the explanation, an example is shown in which pixel signals are read out from four rows of pixels 10 in each of the first pixel region 21a, the second pixel region 21b, and the third pixel region 21c. Arrows C1, C2, C3, and C4 indicate that pixel signals from pixels 10 in the first, second, third, and fourth rows of the third pixel region 21c are being read out, respectively.

The time intervals from time t1 to t2, t2 to t3, t3 to t4, t4 to t5, t5 to t6, t6 to t7, t7 to t8, t8 to t9, t9 to t10, t10 to t11, t11 to t12, t12 to t13, t13 to t14, t14 to t15, t15 to t16, and t16 to t17 are the same constant interval (Δt).

That is, in each frame, the pixel signals from the pixels 10 in the first, second, third, and fourth rows of the first, second, and third pixel regions 21a, 21b, and 21c are read out at the same fixed interval (Δt). Furthermore, the interval between the readout time t4 of the fourth row (last row) of the first to third pixel regions 21a to 21c in the Nth frame and the readout time t5 of the first row of the first to third pixel regions 21a to 21c in the (N+1)th frame is also the above-mentioned fixed interval (Δt). Similarly, the interval between the readout time t8 of the pixels in the fourth row (last row) of the first to third pixel regions 21a to 21c in the (N+1)th frame and the readout time t9 of the pixels in the first row of the first to third pixel regions 21a to 21c in the (N+2)th frame is also the above-mentioned fixed interval (Δt). In addition, the interval between the readout time t12 of the pixels in the fourth row (last row) of the first to third pixel regions 21a to 21c in the (N+2)th frame and the readout time t13 of the pixels in the first row of the first to third pixel regions 21a to 21c in the (N+3)th frame is also the same constant interval (Δt) as described above.

Readouts A1 to A4 of the first pixel region 21a in the Nth frame, readouts B1 to B4 of the second pixel region 21b in the (N+1)th frame, and readouts C1 to C4 of the third pixel region 21c in the (N+2)th frame, which are surrounded by frame G1, are performed at the same intervals (Δt).Readouts A1 to A4 of the first pixel region 21a in the (N+1)th frame, readouts B1 to B4 of the second pixel region 21b in the (N+2)th frame, and readouts C1 to C4 of the third pixel region 21c in the (N+3)th frame, which are surrounded by frame G2, are also performed at the same intervals (Δt).

The synthesis unit 43 synthesizes the pixel signal of the first pixel region 21a of the Nth frame, the pixel signal of the second pixel region 21b of the (N+1)th frame, and the pixel signal of the third pixel region 21c of the (N+2)th frame to generate an image signal for one frame. Similarly, the synthesis unit 43 synthesizes the pixel signal of the first pixel region 21a of the (N+1)th frame, the pixel signal of the second pixel region 21b of the (N+2)th frame, and the pixel signal of the third pixel region 21c of the (N+3)th frame to generate an image signal for one frame. In this way, the synthesis unit 43 synthesizes the pixel signal of each pixel of the first pixel region of the first frame, the pixel signal of each pixel of the second pixel region of the second frame, and the pixel signal of each pixel of the third pixel region of the third frame of three consecutive frames to generate an image signal for one frame.

Next, the operation of the camera 1 according to this embodiment will be described. When the shooting instruction unit 41 instructs video shooting, the imaging control unit 42 reads out pixel signals from the first, second, and third pixel areas 21a, 21b, and 21c at the readout timing shown in FIG. 7. The vertical drive unit 30 reads out pixel signals in the Nth frame, in order, from the pixels 10 in the first row to the pixels 10 in the last row of each of the first, second, and third pixel areas 21a, 21b, and 21c. The timing control unit 61 of the output processing unit 60 sequentially stores the pixel signals read out from the first pixel area 21a and the second pixel area 21b in the Nth frame in the storage unit 62.

The vertical drive unit 30 reads out pixel signals in the (N+1)th frame, in order from the pixel 10 in the first row to the pixel 10 in the last row of each of the first, second, and third pixel regions 21a, 21b, and 21c. The timing control unit 61 sequentially stores the pixel signals read out from the first pixel region 21a and the second pixel region 21b in the (N+1)th frame in the memory unit 62.

In the (N+2)th frame, the vertical drive unit 30 sequentially reads out pixel signals from the first row pixel 10 to the last row pixel 10 of each of the first, second, and third pixel regions 21a, 21b, and 21c. The timing control unit 61 sequentially outputs the pixel signals of the first pixel region 21a of the Nth frame stored in the memory unit 62 to the control unit 4 via the output unit 70. After the output of the pixel signals of the first pixel region 21a of the Nth frame is completed, the timing control unit 61 sequentially outputs the pixel signals of the second pixel region 21b of the (N+1)th frame stored in the memory unit 62 to the control unit 4 via the output unit 70. After the output of the pixel signals of the second pixel region 21a of the (N+1)th frame is completed, the timing control unit 61 sequentially outputs the pixel signals of the third pixel region 21c of the (N+2)th frame to the control unit 4 via the output unit 70.

The pixel signal of the first pixel region 21a in the Nth frame, the pixel signal of the second pixel region 21b in the (N+1)th frame, and the pixel signal of the third pixel region 21c in the (N+2)th frame are output to the control unit 4, for example, during the time interval 4Δt of the (N+2)th frame. In this way, the pixel signal of the first pixel region 21a in the Nth frame, the pixel signal of the second pixel region 21b in the (N+1)th frame, and the pixel signal of the third pixel region 21c in the (N+2)th frame, which are surrounded by the frame G1, are sent to the control unit 4 as a group of time-series signals.
Furthermore, the timing control unit 61 sequentially stores in the storage unit 62 the pixel signals read out from the first pixel area 21a and the second pixel area 21b in the (N+2)th frame.

The vertical drive unit 30 reads out pixel signals from the first, second, and third pixel regions 21a, 21b, and 21c in the (N+3)th frame. The timing control unit 61 sequentially outputs the pixel signals of the first pixel region 21a of the (N+1)th frame stored in the memory unit 62 to the control unit 4 via the output unit 70. After the output of the pixel signals of the first pixel region 21a of the (N+1)th frame is completed, the timing control unit 61 sequentially outputs the pixel signals of the second pixel region 21b of the (N+2)th frame stored in the memory unit 62 to the control unit 4 via the output unit 70. After the output of the pixel signals of the second pixel region 21a of the (N+2)th frame is completed, the timing control unit 61 sequentially outputs the pixel signals of the third pixel region 21c of the (N+3)th frame to the control unit 4 via the output unit 70.

The pixel signal of the first pixel region 21a in the (N+1)th frame, the pixel signal of the second pixel region 21b in the (N+2)th frame, and the pixel signal of the third pixel region 21c in the (N+3)th frame are output to the control unit 4, for example, during the time interval 4Δt of the (N+3)th frame. In this way, the pixel signal of the first pixel region 21a in the (N+1)th frame, the pixel signal of the second pixel region 21b in the (N+2)th frame, and the pixel signal of the third pixel region 21c in the (N+3)th frame, which are surrounded by frame G2, are sent to the control unit 4 as a group of time-series signals.
In addition, the timing control unit 61 sequentially stores in the memory unit 62 the pixel signal read out from the first pixel area 21a in the (N+3)th frame and the pixel signal read out from the second pixel area 21b in the (N+3)th frame.

The synthesis unit 43 of the control unit 4 synthesizes the pixel signal of the first pixel area 21a of the Nth frame, the pixel signal of the second pixel area 21b of the (N+1)th frame, and the pixel signal of the third pixel area 21c of the (N+2)th frame to generate an image signal for one frame. The synthesis unit 43 also synthesizes the pixel signal of the first pixel area 21a of the (N+1)th frame, the pixel signal of the second pixel area 21b of the (N+2)th frame, and the pixel signal of the third pixel area 21c of the (N+3)th frame to generate an image signal for one frame. The synthesis unit 43 generates image signals for multiple frames by performing a similar synthesis process on the pixel signals output from the imaging element 3 in each frame after the (N+3)th frame. The video data generation unit 44 performs various image processes on the image signals of each frame generated by the synthesis unit 43 to generate video image data.

In the camera 1 of this embodiment, pixel signals are read out from the first pixel region 21a, the second pixel region 21b, and the third pixel region 21c using three paths. In addition, the synthesis unit 43 of the camera 1 synthesizes the pixel signal read out from the first pixel region 21a in the first frame, the pixel signal read out from the second pixel region 21b in the second frame, and the pixel signal read out from the third pixel region 21c in the third frame. As a result, the camera 1 of this embodiment can increase the frame rate and obtain an image with reduced unnatural distortion.

The following modifications are also within the scope of the present invention, and one or more of the modifications may be combined with the above-described embodiment.

(Variation 1)
In the above-described first embodiment, an example has been described in which the pixels 10 in the first pixel region 21a are sequentially selected and read from the bottom row to the top row, and the pixels 10 in the second pixel region 21b are sequentially selected and read from the bottom row to the top row. However, as shown in FIG. 8, the vertical driving unit 30 may sequentially select and read the pixels 10 in the first pixel region 21a from the top row to the bottom row, and simultaneously select and read the pixels 10 in the second pixel region 21b from the top row to the bottom row. In this case, the output processing unit 60 outputs the pixel signals read from the second pixel region 21b in the first frame to the control unit 4, and subsequently outputs the pixel signals read from the first pixel region 21a in the next second frame to the control unit 4. The combining unit 43 of the control unit 4 combines the pixel signals of each pixel in the second pixel region in the first frame with the pixel signals of each pixel in the first pixel region in the second frame to generate an image signal for one frame.

In the above-described second embodiment, the vertical drive unit 30 sequentially selects and reads out the pixels 10 in the first, second, and third pixel regions 21a, 21b, and 21c from the bottom row to the top row. However, the vertical drive unit 30 may read out the pixel signals while simultaneously scanning the pixels 10 in the first pixel region 21a, the pixels 10 in the second pixel region 21b, and the pixels 10 in the third pixel region 21c from the top row to the bottom row for each row. The output processing unit 60 outputs the pixel signal of the third pixel region 21c of the first frame to the control unit 4, and subsequently outputs the pixel signal of the second pixel region 21b of the next second frame to the control unit 4, and subsequently outputs the pixel signal of the first pixel region 21a of the next third frame to the control unit 4. The synthesis unit 43 of the control unit 4 synthesizes the pixel signals of each pixel in the third pixel region of the first frame, the pixel signals of each pixel in the second pixel region of the second frame, and the pixel signals of each pixel in the first pixel region of the third frame to generate an image signal for one frame.

(Variation 2)
In the above-mentioned second embodiment, an example in which the pixel section is divided into three equal parts has been described. However, the pixel section can be divided into four or more parts. When the pixel section is divided into four or more parts, it is desirable to provide the column circuit section and the horizontal transfer section for each pixel region from the fourth pixel region onwards on the second substrate 202.

(Variation 3)
Furthermore, the pixel section does not necessarily have to be divided into equal parts. For example, the number of pixel rows in the first pixel region and the number of pixel rows in the second pixel region in the pixel section may be different.

(Variation 4)
In the above-described first embodiment, an example has been described in which the pixel signal of the first pixel region in the Nth frame and the pixel signal of the second pixel region in the (N+1)th frame are output to the control unit 4 as a block of time-series signals in the output processing unit 60. However, the image sensor 3 may not be provided with the output processing unit 60, and the pixel signals from the first and second pixel regions sequentially output by the first and second horizontal transfer units 50a and 50b may be output to the control unit 4 via the output unit 70. In this case, the control unit 4 stores the pixel signals from the first and second pixel regions in a memory provided in the control unit 4. The synthesis unit 43 reads out the pixel signal of the first pixel region in the Nth frame and the pixel signal of the second pixel region in the (N+1)th frame from the memory, synthesizes them, and generates an image signal for one frame. The same applies to the second embodiment.

(Variation 5)
In the above-described embodiment, a description has been given of a case where moving images are captured using the imaging device 1. However, the present invention can also be applied to a case where the imaging device 1 captures still images by continuous shooting instead of or in addition to capturing moving images.

(Variation 6)
In the above-described embodiment, the photodiode is used as the photoelectric conversion unit, but a photoelectric conversion film may be used as the photoelectric conversion unit.

(Variation 7)
The imaging elements and imaging devices described in the above-mentioned embodiments and variations may be applied to cameras, smartphones, tablets, cameras built into PCs, in-vehicle cameras, cameras mounted on unmanned aerial vehicles (drones, radio-controlled aircraft, etc.), and the like.

Although various embodiments and modifications have been described above, the present invention is not limited to these. Other embodiments that are conceivable within the scope of the technical concept of the present invention are also included within the scope of the present invention.

1 Imaging device, 3 Imaging element, 4 Control unit, 10 Pixel, 20 Pixel unit, 21a First pixel area, 21b Second pixel area, 21c Third pixel area, 30 Vertical drive unit, 42 Imaging control unit, 43 Combining unit, 44 Video data image generation unit

Claims (6)

A plurality of pixels arranged in a first region in a first direction and a second direction different from the first direction and provided on a first substrate;
a plurality of pixels disposed in the first direction and the second direction in a second region disposed in the second direction with respect to the first region, the pixels being provided on the first substrate;
a plurality of pixels disposed in the first direction and the second direction in a third region disposed in the second direction with respect to the second region, the pixels being provided on the first substrate;
a readout section provided on a second substrate stacked on the first substrate , the readout section performing a first readout operation of reading out signals of pixels in the first region from one side to the other side of the first region in the second direction, a second readout operation of reading out signals of pixels in the second region from one side to the other side of the second region in the second direction , and a third readout operation of reading out signals of pixels in the third region from one side to the other side of the third region in the second direction;
an output section that outputs an image signal based on a signal of the pixel read out by the first read operation in a first frame, a signal of the pixel read out by the second read operation in a second frame, and a signal of the pixel read out by the third read operation in a third frame;
An imaging element comprising: 2. The imaging device according to claim 1,
The output unit is an imaging element that outputs one image signal based on the signal of the pixel read out by the first readout operation in a first frame, the signal of the pixel read out by the second readout operation in a second frame, and the signal of the pixel read out by the third readout operation in a third frame . 3. The imaging device according to claim 1,
The readout section is an image sensor that simultaneously performs the first readout operation , the second readout operation , and the third readout operation. 4. The imaging device according to claim 1,
The readout unit is an imaging element having, on the second substrate, a first readout unit provided below the first region and performing the first readout operation , a second readout unit provided below the second region and performing the second readout operation , and a third readout unit provided below the third region and performing the third readout operation . The imaging element according to any one of claims 1 to 4,
a generation unit that generates image data based on the image signal output from the imaging element;
An imaging device comprising: 6. The imaging device according to claim 5,
An instruction unit is provided for instructing video shooting or continuous shooting,
The reading unit performs the first reading operation , the second reading operation, and the third reading operation when the instruction unit instructs moving image shooting or continuous shooting.

Images