WO2020195936A1

WO2020195936A1 - Solid-state imaging device and electronic apparatus

Info

Publication number: WO2020195936A1
Application number: PCT/JP2020/011056
Authority: WO
Inventors: 渉新妻
Original assignee: ソニーセミコンダクタソリューションズ株式会社
Priority date: 2019-03-28
Filing date: 2020-03-13
Publication date: 2020-10-01
Also published as: JP2020167441A

Abstract

The present disclosure relates to a solid-state imaging device which makes it possible to reduce power consumption with a smaller circuit scale by using a pixel-parallel ADC method, and to an electronic apparatus. Provided is a solid-state imaging device comprising a pixel array unit in which a plurality of pixels are disposed two-dimensionally. Each pixel includes a photoelectric conversion unit and an AD conversion unit, the AD conversion unit converting a pixel signal obtained from photoelectric conversion performed by the photoelectric conversion unit. In the pixel array unit, when reading out AD conversion results from the plurality of pixels, some of the pixels are thinned. This technique can be applied in image sensors which use a pixel-parallel ADC method, for example.

Description

Solid-state image sensor and electronic equipment

The present disclosure relates to a solid-state image sensor and an electronic device, and more particularly to a solid-state image sensor and an electronic device capable of reducing power consumption with a smaller circuit scale by using a pixel parallel ADC method.

In the image sensor, as an ADC (Analog to Digital Converter) method that converts the pixel signal read from the pixels arranged two-dimensionally in the pixel array section from an analog signal to a digital signal, an ADC is used for each vertical row of pixels. A parallel arrangement method (hereinafter referred to as a column parallel ADC method (column ADC method)) is used.

Further, as an ADC method, a method in which an ADC is provided in each pixel arranged two-dimensionally in a pixel array portion (hereinafter, referred to as a pixel parallel ADC method) is known (see, for example, Patent Document 1). ..

International Publication No. 2016/009832

By the way, in the image sensor, when the pixel parallel ADC method is used as the ADC method, higher speed imaging is possible as compared with the case where the column parallel ADC method is used, but the circuit scale is smaller. It is required to reduce power consumption.

This disclosure has been made in view of such a situation, and makes it possible to reduce power consumption with a smaller circuit scale by using a pixel parallel ADC method.

The solid-state image sensor on one side of the present disclosure includes a pixel array unit in which a plurality of pixels are arranged in a two-dimensional manner, and the pixels AD a photoelectric conversion unit and a pixel signal obtained by photoelectric conversion by the photoelectric conversion unit. The pixel array unit includes an AD conversion unit for conversion, and is a solid-state image sensor in which some pixels are thinned out when reading AD conversion results from the plurality of pixels.

The electronic device on one aspect of the present disclosure includes a pixel array unit in which a plurality of pixels are arranged in a two-dimensional manner, and the pixels AD convert a photoelectric conversion unit and a pixel signal obtained by photoelectric conversion by the photoelectric conversion unit. The pixel array unit is an electronic device equipped with a solid-state image sensor in which some pixels are thinned out when the AD conversion results from the plurality of pixels are read out.

In the solid-state image sensor and the electronic device on one aspect of the present disclosure, two pixels including a photoelectric conversion unit and an AD conversion unit that AD-converts a pixel signal obtained by photoelectric conversion by the photoelectric conversion unit are included as a plurality of pixels. When reading the AD conversion results from the plurality of pixels in the pixel array unit arranged in a dimension, some pixels are thinned out.

The solid-state image sensor or electronic device on one aspect of the present disclosure may be an independent device or an internal block constituting one device.

It is a figure explaining a general motion detection method. It is a figure which shows the example of block matching. It is a figure which shows the example of thinning-out of a pixel. It is a figure which shows the example of the structure of the solid-state image sensor to which the technique which concerns on this disclosure is applied. It is a figure which shows the example of the operation of a repeater. It is a figure which shows the example of the arrangement of a cluster. It is a figure which shows the example of the composition of the pixel inside a cluster. It is a figure which shows the example of the arrangement of a repeater. It is a figure which shows the example of the configuration of the cluster inside the repeater. It is a figure which shows the example of the structure of a repeater. It is a figure which shows the example of the output of a thinning template and a matching block. It is a figure which shows the example of reading out the phase-shifted thinned-out image. It is a figure which shows the example of the area of the thinned-out image. It is a flowchart explaining the flow of the camera shake correction processing. It is a figure which shows the example of the effective pixel area after motion correction. It is a figure which shows the example of the three-dimensional structure of a solid-state image sensor. It is a figure which shows the example of the detailed structure of the ADC and the latch part of a pixel. It is a figure which shows the example of the circuit structure of the main part of a solid-state image sensor. It is a figure which shows the example of the structure of the electronic device equipped with the solid-state image sensor to which the technique which concerns on this disclosure is applied. It is a figure which shows the use example of the solid-state image sensor to which the technique which concerns on this disclosure is applied. It is a block diagram which shows an example of the schematic structure of a vehicle control system. It is explanatory drawing which shows an example of the installation position of the vehicle exterior information detection unit and the image pickup unit.

Hereinafter, embodiments of the technology (the present technology) according to the present disclosure will be described with reference to the drawings. The explanations will be given in the following order.

1. 1. Embodiment of this technology 2. Modification example 3. Configuration of electronic devices 4. Example of using a solid-state image sensor 5. Application example to moving body

<1. Embodiment of this technology>

FIG. 1 shows a general motion detection method in an electronic device.

In FIG. 1, the electronic device 900 is composed of a column-parallel ADC method (column AD method) image sensor 901, a DSP (Digital Signal Processor) 902, and a frame memory 903. Further, the DSP 902 is provided with a cache memory 911.

In the electronic device 900, when detecting a movement such as camera shake, the image sensor 901 captures an captured image larger than the angle of view, and the captured image is supplied to the DSP 902. The DSP 902 acquires an image corresponding to the search area while storing the captured image in the frame memory 903, executes processing such as block matching, and detects a motion vector.

Here, as the block matching method, for example, there is a representative point matching method. In this representative point matching method, the pixel values of the pixels corresponding to the template blocks held in the search area SA correspond to the matching blocks (MB ₁ , MB ₂ , MB ₃ , MB ₄ , ...). The motion vector MV is detected by taking the sum of the differences from the pixel values of the pixels (FIG. 2).

As described above, in the general motion detection method, the motion vector MV is detected by performing processing such as block matching using the frame memory 903 by the DSP 902, so that a dedicated circuit is required and the memory access is increased. As a result, the power consumption increases. In addition to the frame memory 903, the DSP 902 also requires a cache memory 911 for the matching block.

In addition, since the image sensor 901 of the column-parallel ADC system has a slow imaging speed, it is necessary to perform imaging by the image sensor 901 and processing such as block matching by the DSP 902 in parallel at the same time.

Therefore, when performing block matching processing, it is common practice to reduce the processing by using an image (thinned image) obtained by thinning out pixels in the horizontal and vertical directions with the image sensor 901. Specifically, as shown in FIG. 3, in the image sensor 901, when the pixels arranged two-dimensionally in the pixel array portion are represented by circular symbols (◯) in the figure, diagonal lines are drawn. The pixels represented by the circular circle are thinned out.

Here, as the ADC method of the image sensor, there is a pixel parallel ADC method in addition to the column parallel ADC method. In the image sensor, when the pixel parallel ADC method is used, faster imaging is possible as compared with the case where the column parallel ADC method is used, but it is less than when the column parallel ADC method is used. It is required to reduce power consumption on a circuit scale.

This technology solves the above-mentioned problems and makes it possible to reduce power consumption with a smaller circuit scale by using the pixel parallel ADC method. In particular, the present technology proposes a motion detection method capable of reducing power consumption with a smaller circuit scale when motion detection is performed. Hereinafter, details of the motion detection method to which the present technology is applied will be described with reference to FIGS. 4 to 18.

(Structure of solid-state image sensor)
FIG. 4 shows an example of the configuration of a solid-state image sensor to which the present technology is applied.

The solid-state image sensor 10 is an image sensor such as a back-illuminated CMOS (Complementary Metal Oxide Semiconductor) image sensor.

In FIG. 4, the solid-state imaging device 10 includes a pixel array unit 101, a repeater unit 102, a GC generation unit 103, a SRAM (Static RAM) 104, a signal processing unit 105, and a motion determination memory 106.

A plurality of pixels 111 are arranged two-dimensionally in the pixel array unit 101. Each pixel 111 has a photoelectric conversion unit such as a photodiode, a pixel circuit including a pixel transistor, and an AD conversion unit (ADC) that converts a pixel signal output from the pixel circuit from an analog signal to a digital signal.

That is, in the solid-state image sensor 10, as the ADC method, a pixel parallel ADC method in which an ADC is provided in each pixel 111 two-dimensionally arranged in the pixel array unit 101 is adopted.

Here, the reading of the AD conversion result from each pixel 111 (ADC) is performed via the repeater unit 102 in which a plurality of repeater 131 including a shift register in which flip flops (FF: Flip Flop) are connected in multiple stages is arranged. Further, this reading order is read in units of one pixel for each rectangular block (pixel block) called a cluster.

That is, each pixel 111 (ADC) is connected to the latch portion (latch circuit) in the cluster 121 on a one-to-one basis, and a plurality of latch portions are grouped into one cluster. Further, one cluster 121 is connected to one (flip-flop) of the shift registers of the repeater 131 configured as a repeater circuit.

The repeater unit 102 is a plurality of repeaters 131 arranged in parallel in a strip shape in the vertical direction (vertical direction in the drawing), and is configured to correspond to an image of one frame. A plurality of clusters 121 are vertically arranged so as to have a one-to-one relationship with flip-flops of shift registers arranged vertically in the repeater 131.

The GC generation unit 103 inputs a Gray code (GC: Gray Code) to a plurality of repeater 131s arranged in the repeater unit 102. The repeater 131 writes (Writes) and reads (Reads) the Gray code. FIG. 5 shows an example of the operation of the repeater 131.

In FIG. 5, the gray code from the GC generation unit 103 is input to the repeater 131 from the side opposite to the operating clock (CK) from the clock supply unit 133, and is sequentially transferred in a bucket relay manner to the data processing unit 134. Is output to. As a result, the Gray code is written to each latch portion, and the latch holding operation according to the inverting output (VCO) from the comparator of the ADC of the pixel 111 is performed.

That is, in the repeater 131, the Gray code transferred by the shift register is taken into the latch portion according to the inverting signal (VCO) of the comparator of the ADC of the pixel 111. Then, the latch data (Gray code) held in the latch portion is read via the repeater 131, converted into a binary code, and then the correlated double sampling (CDS: Correlated Double Sampling) is performed using the SRAM 104. Will be done.

In this way, the AD conversion result of the pixel 111 is once held in the latch portion in the cluster 121, then transferred to the shift register in the repeater 131, and sequentially output to the outside of the repeater portion 102. The reading order of the cluster 121 and the latch portion and the pixel 111 to be selected can be arbitrarily changed by controlling the selection signal from the control circuit (not shown).

In the solid-state image sensor 10, only one AD conversion result is selected from the plurality of latches and used as the output of the cluster 121, and when the AD conversion results of all the clusters 121 are output via the shift register, the first time. The reading is completed. Subsequently, when the AD conversion result is selected from another latch portion and the reading is completed for the number of times of the latch portion in the cluster 121, all the pixel signals of the image for one frame are read. ..

The signal processing unit 105 performs predetermined signal processing based on the image obtained from the pixel signal, and outputs the processing result to the outside. For example, in the signal processing unit 105, when motion detection is performed, signal processing for block matching calculation and the like are performed. Further, a correlation value for motion determination is recorded in the motion determination memory 106.

(Cluster configuration)
Here, a detailed configuration of the cluster 121 will be described with reference to FIGS. 6 and 7.

FIG. 6 shows pixels 111 arranged two-dimensionally in the pixel array unit 101 in a grid pattern, and one square corresponds to one pixel 111. Further, here, the XY coordinates of the pixels 111 arranged in the pixel array unit 101 are referred to as pixels (n, m), and the XY coordinates of the cluster 121 are referred to as clusters (j, k).

Here, an image for one frame is generated from the pixel signals output from the plurality of pixels 111 arranged in the pixel array unit 101, but the pixels 111 within the range of one cluster are, for example, the lower left cluster ( It corresponds to 0,0) and the square in the thick frame of the cluster (j-1, k-1) on the upper right.

Further, FIG. 7 shows the details of the coordinates of the pixel 111 inside the cluster 121. In FIG. 7, the range of one cluster is 4 × 32 pixels, that is, 4 pixels in the X direction (horizontal direction) and 32 pixels in the Y direction (vertical direction).

In FIG. 7, for convenience of explanation, the number of pixels in the row direction and the column direction of "4 pixels" and "32 pixels" are shown in the space by arranging them at intervals for each cluster. However, in reality, such a space does not exist. Further, in each cluster 121, the coordinates of the pixels 111 in the cluster 121 are also shown together with the coordinates of the cluster 121.

Specifically, the lower left cluster (0,0) is pixel (0,0) to (0,31), pixel (1,0) to (1,31), pixel (2,0) to (2). , 31), and pixels (3,0) to (3,31).

The clusters (1,0) adjacent to the right side of the clusters (0,0) are pixels (4,0) to (4,31), pixels (5,0) to (5,31), and pixels (6). , 0) to (6,31), and pixels (7,0) to (7,31), and the cluster (2,0) adjacent to the right side of the cluster (1,0) is the pixel (8,0). Consists of 0) to (8,31), pixels (9,0) to (9,31), pixels (10,0) to (10,31), and pixels (11,0) to (11,31) Will be done.

The clusters (0,1) adjacent to the upper side of the clusters (0,0) are pixels (0,32) to (0,63), pixels (1,32) to (1,63), pixels (2,32). ) To (2,63), and pixels (3,32) to (3,63).

In addition, the clusters (1,1) adjacent to the right side of the clusters (0,1) are pixels (4,32) to (4,63), pixels (5,32) to (5,63), and pixels (6). , 32) to (6,63), and pixels (7,32) to (7,63), and the cluster (2,1) adjacent to the right side of the cluster (1,1) is the pixel (8,1). Consists of 32) to (8,63), pixels (9,32) to (9,63), pixels (10,32) to (10,63), and pixels (11,32) to (11,63) Will be done.

Although it is not shown because it will be repeated any more, it can be expressed as follows by generalizing the relationship between the coordinates of the cluster 121 and the coordinates of the pixels 111 in the cluster 121.

That is, as shown in the upper right of the figure, when j = n / 4, k = m / 32, the clusters (j-1, k-1) are pixels (n-4, m-32) or ( n-4, m-1), pixel (n-3, m-32) to (n-3, m-1), pixel (n-2, m-32) to (n-2, m-1) , And pixels (n-1, m-32) to (n-1, m-1).

(Repeater configuration)
Next, a detailed configuration of the repeater unit 102 will be described with reference to FIGS. 8 to 10.

FIG. 8 shows the configuration of a plurality of repeaters 131 arranged in the repeater unit 102 and a repeater selector 132 provided in the subsequent stage. In FIG. 8, in the repeater section 102, j repeaters # 0 to # j-1 are arranged in parallel in a strip shape in the vertical direction (vertical direction in the drawing) as a repeater circuit.

Here, FIG. 9 shows the configuration of the cluster 121 inside the repeater 131. In FIG. 9, in repeater # 0, k clusters (0, 0) to (0, k-1) are arranged in the vertical direction.

In repeaters # 1 and # 2, k clusters (1,0) to (1, k-1) and k clusters (2,0) to (2, k-1) are vertically arranged. Each is placed. Although not shown because it is repeated, k clusters 121 are similarly arranged in the vertical direction in the repeaters # 3 to # j-1.

In this way, in the repeater section 102, j repeaters # 0 to # j-1 are arranged in parallel in the vertical direction in a strip shape, and in each repeater 131, k clusters 121 are bundled in the vertical direction. ..

Here, in the repeater unit 102, if one of the j repeaters # 0 to # j-1 is focused on the repeater 131, the repeater 131 of interest has a structure as shown in FIG. There is.

In FIG. 10, a shift register 141 in which a flip-flop 142 as a sequential circuit is connected in multiple stages is provided in the repeater 131 in the vertical direction. In the shift register 141, each of the k flip-flops 142 is connected to each of the k clusters # 0 to # k-1.

That is, each pixel 111 (ADC) arranged in the pixel array unit 101 is connected to the latch unit in the cluster 121 on a one-to-one basis, and a plurality of latch units are grouped into one cluster. Further, one cluster 121 is connected to one flip-flop 142 constituting the shift register 141 in the repeater 131.

Then, the repeater unit 102 is a plurality of repeater 131s arranged in parallel in a strip shape in the vertical direction, and is configured to correspond to an image of one frame. Further, the cluster 121 is connected to the flip-flop 142 of the shift register 141 arranged vertically in the repeater 131 on a one-to-one basis, and a plurality of clusters 121 are arranged vertically.

Returning to the description of FIG. 8, in the repeater unit 102, the repeater selector 132 selects the output from any repeater 131 among the repeaters # 0 to # j-1 according to the selection signal from the control circuit (not shown). And output to the latter stage.

For example, when motion detection is performed using the representative point matching method, the repeater selector 132 selects the output of the repeater 131 according to the horizontal thinning amount and the horizontal coordinate deviation in order to prepare the template block and the matching block. To.

In the example of FIG. 8, the repeater selector 132 is an output from the repeater 131 of an even number of repeaters # 0 to # j-1, that is, a cluster (0,0) to (0, k-1), a cluster ( The output of 2,0) to (2, k-1), ..., Cluster (j-2,0) to (0, k-1) is selected.

In the above description, the repeater 131 in the repeater section 102 is arranged in parallel in a strip shape in the vertical direction, but the repeater 131 may be arranged in parallel not only in the vertical direction but also in the horizontal direction. Good. Further, the arrangement of the clusters 121 in the repeater 131 is not limited to the vertical direction, and a plurality of clusters 121 may be arranged in the horizontal direction and the vertical direction. Further, the number of stages and the number of parallels of the shift register 141 in the repeater 131 are arbitrary.

(Example of motion detection)
Next, with reference to FIGS. 11 to 15, an example of motion detection using the representative point matching method as signal processing by the signal processing unit 105 (FIG. 4) will be described.

In the representative point matching method, the difference between the pixel value of the representative point of the first image and the pixel value of the sampling point of the second image is calculated for each of the plurality of detection blocks, and the sum of the differences is the cumulative correlation value (matching error). Also called). Then, a plurality of cumulative correlation values are calculated while shifting the relative positions of the first image and the second image, and the deviation of the relative positions is detected as the movement between the two blocks.

In this example, an example of motion detection in the range of 0 to +7 pixels in one pixel unit in the horizontal direction and in the range of 0 to +31 pixels in one pixel unit in the vertical direction is shown.

Therefore, in motion detection using the representative point matching method, a template block is prepared in which 8 pixels are thinned out in the horizontal direction and 32 pixels are thinned out in the vertical direction. On the other hand, as a search area, a matching block having the same thinning amount and whose coordinates are shifted by one pixel each in the horizontal direction and the vertical direction is prepared.

Then, in the solid-state image sensor 10, when the pixels 111 arranged in the pixel array unit 101 are used to prepare each pixel 111 of the template block and each pixel 111 of the 8 × 32 matching blocks, the cluster 121 described above is used. And the configuration of the repeater 131 can be used.

That is, in the pixel coordinate example of FIG. 7 described above, focusing on the cluster (2j, k), for example, the cluster (0,0), the cluster (2,0), the cluster (0,1), and the cluster ( If the pixel signal is read from the pixel (0,0), pixel (8,0), pixel (0,32), and pixel (8,32) located in the lower left of 2,1), the template block and all It is possible to prepare a matching block. In the example of FIG. 7, the pixel of interest is represented by inverting black and white.

Here, since a template block and a matching block that thin out 8 pixels in the horizontal direction and thin out 32 pixels in the vertical direction are prepared, the pixel signal is transmitted from the pixel 111 located at the lower left of each of the clusters (2j, k). Is read out. Then, in the repeater 131, the pixels 111 in each cluster 121 are sequentially selected for each clock, and all the pixels 111 are read out by selecting 4 × 32 times.

As described above, in the present technology, when the pixel parallel ADC method is adopted, the pixel signal from the pixel 111 is read out via the shift register 122 arranged in the vertical direction in each repeater 131 of the repeater unit 102. Further, the reading order is performed by selecting a plurality of pixels 111 arranged in the cluster 121 one by one.

Therefore, this output is an image in which the pixels 111 in the horizontal direction and the vertical direction are thinned out (thinned-out image), and corresponds to the pixel signal read out while repeating the phase shift. Then, by adapting this output to the cluster size suitable for the matching block and the pixel selection order (thinning phase order) suitable for the search method, motion detection using the representative point matching method is performed.

FIG. 11 shows an example of the output of the thinned out template block and the matching block.

In FIG. 11, the circular symbol (◯) in the drawing means the pixels 111 arranged two-dimensionally in the pixel array unit 101, and the pixels are represented by a plurality of circular symbols arranged in the horizontal and vertical directions. Of the total area where 111 is arranged, a part of the area is represented.

Further, in FIG. 11, three types of patterns are attached to the circular symbols representing the pixels 111, and the circular symbols having the same pattern are the pixels 111 arranged in different clusters. , Indicates that they are in the same phase. Specifically, for example, the upper left pixel 111 in the figure is the phase (0,0), the pixel 111 adjacent to the lower side thereof is the phase (0,1), and the pixel 111 adjacent to the right side thereof is the phase (1,0). It can be expressed as 0) and so on.

In FIG. 11, for convenience of explanation, three pixels 111 are illustrated by three types of patterns as the pixels 111 whose phase is shifted in the same cluster 121, but other pixels 111 in the same cluster 121 are illustrated. Is also read while repeating the phase shift.

In this way, by reading out the pixel signals while repeating the phase shift for each cluster 121, as shown in FIG. 12, thinned-out images (image frames) having different phases (thinned-out phases) are sequentially read out.

In FIG. 12, the direction from the left side to the right side in the figure is the direction of time, and the rectangular symbols in the figure represent the thinned images obtained from the pixel signals read while performing the phase shift in chronological order. ing. Further, in FIG. 12, three types of patterns are attached to the rectangular symbols, and these patterns correspond to the patterns attached to the circular symbols shown in FIG.

That is, in FIG. 12, among the thinned images arranged in time series, the _first thinned image TI ₁ is an image obtained from the pixel signal of the pixel 111 of the phase (0,0), and the second thinned image TI Reference numeral ₂ denotes an image obtained from the pixel signal of the pixel 111 having the phase (0, 1). The i-th thinned-out image TI _i is an image obtained from the pixel signal of the pixel 111 of the phase (1, 0).

FIG. 13 shows an example of the regions of the thinned-out images TI ₁ , TI ₂ , and TI _{i in} the search area SA. In FIG. 13, the thinned-out image TI ₁ is a region corresponding to the pixel 111 corresponding to the phase (0,0). Further, the thinned-out image TI ₂ is a region corresponding to the pixel 111 corresponding to the phase (0, 1), and is a region shifted downward by one pixel with respect to the region of the thinned-out image TI ₁ . Further, the thinned image TI _i is a region corresponding to the pixel 111 corresponding to the phase (1, 0), and is a region shifted to the right by one pixel with respect to the region of the thinned image TI ₁ .

In this way, by using the configuration of the cluster 121 and the repeater 131, the thinned-out image TI can be repeatedly read out while shifting the phase. Therefore, for example, the thinned-out image TI can be thinned out by 8 pixels in the horizontal direction and in the vertical direction. It is possible to prepare a template block for thinning out 32 pixels and a matching block for which the coordinates are shifted by 1 pixel each in the horizontal direction and the vertical direction with the same thinning amount.

When the pixel parallel ADC method is adopted, the imaging process including the ADC can be performed 100 times faster than when the column parallel ADC method is used. Therefore, the matching block is performed while shifting the phase. It is possible to realize a frame rate such as 10,000 fps by reading out the pixel signal for the device in a time-division manner.

The signal processing unit 105 holds the pixel signal of each pixel 111 corresponding to the template block in the memory, and corresponds to the pixel value of each pixel 111 corresponding to the matching block to be sequentially read and the held template block. The correlation value is obtained by taking the sum of the differences from the pixel values of each pixel 111.

Then, the signal processing unit 105 detects the deviation of the coordinates of the matching block having a strong correlation as the motion vector MV. In this motion detection, a correlation value for motion determination is used, and the smaller the absolute value of the sum of differences, the stronger the correlation. Here, the motion detection cycle (frequency) can be changed by changing the frame rate of imaging or pausing block matching.

Next, the flow of the image stabilization process using motion detection will be described with reference to the flowchart of FIG.

Here, it is assumed that a camera shake occurs while the user is shooting a subject by using an electronic device (electronic device 1000 in FIG. 19 described later) equipped with the solid-state image sensor 10. The electronic device includes, for example, an imaging device such as a smartphone, a digital still camera, or a digital video camera.

The signal processing unit 105 holds the thinned image TI including the pattern of the specific shape as a template block (S11), and sequentially acquires the thinned image TI sequentially read from the search area SA as a matching block (S12).

The signal processing unit 105 detects the motion by summing the difference between the pixel value of each pixel 111 corresponding to the held template block and the pixel value of each pixel 111 corresponding to the matching block acquired sequentially.

Then, by selecting the read start coordinates of the next imaging from the value of the motion vector MV detected in this way (S14), motion correction (camera shake correction) can be realized.

FIG. 15 shows an example of the effective pixel region after motion correction. In the example of FIG. 15, the pixels 111 arranged two-dimensionally in the pixel array unit 101 are represented in a grid pattern, and the entire grid-like region is the imageable pixel region IA. Further, a region smaller than the imageable pixel region IA is defined as an effective pixel region EA. That is, the solid-state image sensor 10 captures an captured image larger than the angle of view to detect motion.

At this time, in the solid-state image sensor 10, the signal processing unit 105 indicates that the value of the motion vector MV (movement amount in the horizontal direction and the vertical direction) detected by the camera shake has moved to the upper right direction in the drawing. If so, the read start coordinates for the next imaging are changed from pixel (0,0) to pixel (4,4). As a result, the effective pixel area EA is also shifted in the upper right direction in response to the movement of the subject due to camera shake, and the subject can be accommodated in the effective pixel area EA, so that camera shake correction is realized.

As described above, in the present technology, the pixel parallel ADC method is adopted as the ADC method of the solid-state imaging device 10, and the pixel signal (AD conversion result) from the pixel 111 (ADC) arranged in the pixel array unit 101 is obtained. When reading to the signal processing unit 105 via the repeater unit 102, the configuration of the cluster 121 and the repeater 131 can be used in the same manner as when the thinned image for block matching is read out in a time-divided manner.

Here, for example, in the case of performing motion detection using the representative point matching method, the solid-state image sensor 10 itself can be treated as if it were a frame memory when the template block and the matching block are prepared.

Therefore, in the solid-state image sensor 10, processing such as block matching is performed by the DSP 902 using a dedicated frame memory 903 (DRAM, SRAM, etc.) as shown in a general motion detection method (FIGS. 1 to 3). You don't have to do it. Further, in the solid-state image sensor 10, it is not necessary to provide a dedicated circuit, and the power consumption does not increase. That is, it is possible to provide a structure and an algorithm that realizes motion detection by block matching at low cost in the solid-state image sensor 10.

In addition, it is possible to incorporate a camera shake detection function (motion detection function not only for the full screen), a camera shake correction function (motion correction function not only for the full screen), and the like into the solid-state image sensor 10. Furthermore, the cost of the entire camera system and the power consumption can be reduced.

Further, since the solid-state image sensor 10 employs the pixel parallel ADC method as the ADC method, it is advantageous when a large number of thinned-out images having different phases are required.

Even in the case of normal block matching using a normal image instead of a thinned image, when the pixel parallel ADC method is used, the image can be imaged at a higher speed than when the column parallel ADC method is used. Therefore, it is considered to be sufficient in terms of performance. Further, since the solid-state image sensor 10 employs a pixel parallel ADC method and performs time-division readout at ultra-high speed, it is possible to acquire an captured image in addition to the thinned image for block matching in the imaging after the thinned out readout. ..

Further, in the solid-state image sensor 10, when the pixel parallel ADC method is used, the shutter is a batch shutter for all pixels that operates at a high frame rate, so that motion detection with less erroneous determination is performed with almost no influence of focal plane distortion. This makes it possible to perform highly accurate motion correction.

As described above, in this technology, the pixel parallel ADC method can be used to reduce the power consumption with a smaller circuit scale. In particular, in the motion detection method to which the present technology is applied, when the pixel parallel ADC method is used, motion detection with reduced power consumption can be realized with a smaller circuit scale.

(Structure of pixel parallel ADC)
FIG. 16 shows an example of the three-dimensional structure of the solid-state image sensor 10.

In FIG. 16, the solid-state image sensor 10 has a structure in which a pixel substrate 100A into which light is incident from the back surface side and a logic substrate 100B responsible for signal processing are bonded together. At least the pixel array unit 101 and the DAC (Digital to Analog Converter) 107 are formed on the pixel substrate 100A. At least the latch repeater unit 108 and the logic calculation unit 109 are formed on the logic board 100B.

A plurality of pixels 111 are arranged two-dimensionally in the pixel array unit 101. An ADC 151 corresponding to one single slope method is provided for each pixel 111, and the inverted output (VCO) of the comparator is connected to one latch circuit 152 provided in the latch repeater unit 108 (FIG. 17). ). Further, the latch repeater unit 108 includes a repeater unit 102.

That is, in the pixel array unit 101, when n × m pixels 111 are arranged in a two-dimensional manner, a pair of ADC 151 and a latch circuit 152 is provided in n × m pairs, and an inversion signal of the comparator of ADC 151 is provided. The gray code from the repeater unit 102 (repeater 131) is taken into the latch circuit 152 according to (VCO).

FIG. 18 shows an example of the circuit configuration of the main part of the solid-state image sensor 10.

The pixel 111 includes a pixel circuit 161, a differential amplifier circuit 162, a positive feedback circuit (PFB: Positive Feedback) 163, and a multiplex circuit (MUX: Multiplexer) 164. Further, in the pixel 111, the comparator 160 is composed of the differential amplifier circuit 162 and the positive feedback circuit 163. The comparator 160 constitutes a part of the ADC 151.

The pixel circuit 161 includes a photodiode 171 as a photoelectric conversion unit, a transfer transistor 172, a reset transistor 173, an FD (Floating Diffusion) 174, and an emission transistor 175.

The transfer transistor 172 transfers the electric charge generated by the photodiode 171 to the FD174. The reset transistor 173 resets the electric charge held in the FD174. The FD174 is connected to the gate of the transistor 177 of the differential amplifier circuit 162.

As a result, the transistor 177 of the differential amplifier circuit 162 also functions as an amplifier transistor of the pixel circuit 161. The discharge transistor 175 discharges the electric charge accumulated in the photodiode 171.

The differential amplifier circuit 162 includes

transistors

181 and 182 as a differential pair,

transistors

183 and 184 forming a current mirror, and transistors 185 as a constant current source for supplying a current corresponding to an input bias current (Vb). Consists of.

Of the

transistors

181 and 182 that form a differential pair, a reference signal (REF) output from the DAC 107 (FIG. 16) is input to the gate of the transistor 181 and a pixel circuit in the pixel 111 is input to the gate of the transistor 182. The pixel signal (SIG) output from 161 is input.

In the differential amplifier circuit 162, the reference signal (REF) input to the gate of the transistor 181 and the pixel signal (SIG) input to the gate of the transistor 182 are compared, and the reference signal (REF) and the pixel signal (SIG) are compared. The output signal (VCO) is output according to the comparison result with).

The positive feedback circuit 163 includes transistors 191 to 193 and a NOR circuit 194. Further, the NOR circuit 194 is configured to include transistors 195 to 198.

The connection point between the drain of the transistor 182 and the drain of the transistor 184 is the output end of the differential amplifier circuit 162, and is connected to the drain of the transistor 191 in the positive feedback circuit 163 via the

transistors

186 and 187. The output signal (VCO) output from the differential amplifier circuit 162 is input to the NOR circuit 194 in the positive feedback circuit 163, and is output as an inverted signal (VCO) of the comparator 160.

The inverting signal (VCO) from the comparator 160 (positive feedback circuit 163) and the control signal WORD are input to the multiplex circuit 164. In the multiplex circuit 164, the inversion signal (VCO) from the comparator 160 is output to the latch circuit 152 in the latch repeater unit 108 by controlling the control signal WORD.

The latch repeater unit 108 is composed of a repeater 131 and a latch circuit 152. Each pixel 111 (ADC 151) is connected to the latch circuit 152 in the cluster 121 on a one-to-one basis, and a plurality of latch circuits 152 are grouped into one cluster. Further, one cluster 121 is connected to one flip-flop 142 of the shift register 141 of the repeater 131 (FIG. 10).

Then, the AD conversion result from each pixel 111 (ADC 151) is held by the latch circuit 152 in the cluster 121, transferred to the shift register 141 in the repeater 131, and sequentially output to the outside of the latch repeater unit 108. Will be done.

<2. Modification example>

In the above description, the case where the thinned-out image obtained by utilizing the configuration of the cluster 121 and the repeater 131 is used for motion detection in the solid-state imaging device 10 is illustrated, but for example, the thinned-out image (display image according to) Is displayed on the display unit (for example, the display unit 1015 of FIG. 19), or the thinned image (data) is stored in the storage unit (for example, the storage unit 1016 of FIG. 19). It may be used for various purposes.

Further, in the above description, the case where the representative point matching method is used as a method for detecting the movement between images by image processing is illustrated, but the thinned-out image obtained by utilizing the configuration of the cluster 121 and the repeater 131 is used. Any other detection method may be used as long as the motion can be detected.

<3. Electronic device configuration>

FIG. 19 shows a configuration example of an electronic device equipped with a solid-state image sensor to which the present technology is applied.

The electronic device 1000 is, for example, an electronic device having an imaging function such as an imaging device such as a digital still camera or a video camera, or a mobile terminal device such as a smartphone or a tablet terminal.

The electronic device 1000 includes a lens unit 1011, a solid-state imaging device 1012, a signal processing unit 1013, a control unit 1014, a display unit 1015, a storage unit 1016, an operation unit 1017, a communication unit 1018, and a power supply unit 1019. Further, in the electronic device 1000, the signal processing unit 1013 to the power supply unit 1019 are connected to each other via the bus 1021.

The lens unit 1011 is composed of a zoom lens, a focus lens, and the like, and collects light from the subject. The light (subject light) focused by the lens unit 1011 is incident on the solid-state image sensor 1012.

The solid-state image sensor 1012 is a solid-state image sensor to which the present technology is applied (for example, the solid-state image sensor 10 described above). The solid-state imaging device 1012 performs photoelectric conversion of the light (subject light) received through the lens unit 1011 and AD-converts the pixel signal obtained as a result, and supplies the signal obtained as a result to the signal processing unit 1013.

The signal processing unit 1013 is composed of a signal processing circuit such as a DSP (Digital Signal Processor) circuit, and performs signal processing on a signal supplied from the solid-state imaging device 1012. For example, the signal processing unit 1013 generates image data of a still image or a moving image by performing signal processing on the signal from the solid-state imaging device 1012, and supplies the image data to the display unit 1015 or the storage unit 1016.

The control unit 1014 is configured as, for example, a CPU (Central Processing Unit), a microprocessor, an FPGA (Field Programmable Gate Array), or the like. The control unit 1014 controls the operation of each unit of the electronic device 1000.

The display unit 1015 is configured as a display device such as a liquid crystal panel or an organic EL (Electro Luminescence) panel. The display unit 1015 displays a still image or a moving image according to the image data supplied from the signal processing unit 1013.

The storage unit 1016 is configured as a recording medium such as a semiconductor memory or a hard disk, for example. The storage unit 1016 records the image data supplied from the signal processing unit 1013. Further, the storage unit 1016 supplies the recorded image data according to the control from the control unit 1014.

The operation unit 1017 is configured as a touch panel in combination with the display unit 1015 in addition to the physical buttons, for example. The operation unit 1017 outputs operation commands for various functions of the electronic device 1000 in response to an operation by the user. The control unit 1014 controls the operation of each unit based on the operation command supplied from the operation unit 1017.

The communication unit 1018 is configured as, for example, a communication interface circuit. The communication unit 1018 exchanges data with an external device by wireless communication or wired communication according to a predetermined communication method.

The power supply unit 1019 appropriately supplies various power sources that serve as operating power sources for the signal processing unit 1013 to the communication unit 1018 to these supply targets.

The electronic device 1000 is configured as described above.

This technology is applied to the solid-state image sensor 1012 as described above. By applying this technology to the solid-state imaging device 1012, in the electronic device 1000, the solid-state imaging device 1012 can be operated with a smaller circuit scale and low power consumption, and for example, camera shake correction using motion detection can be realized. Can be done.

<4. Example of using a solid-state image sensor>

FIG. 20 is a diagram showing a usage example of a solid-state image sensor to which the present technology is applied.

The solid-state image sensor 10 can be used in various cases for sensing light such as visible light, infrared light, ultraviolet light, and X-ray, as shown below. That is, as shown in FIG. 20, not only the field of appreciation for taking an image used for appreciation, but also, for example, the field of transportation, the field of home appliances, the field of medical / healthcare, the field of security, and the field of beauty. The solid-state imaging device 10 can also be used in devices used in the fields of fields, sports, agriculture, and the like.

Specifically, in the field of appreciation, for example, a device for taking an image to be used for appreciation, such as a digital camera, a smartphone, or a mobile phone having a camera function (for example, the electronic device 1000 in FIG. 19). Therefore, the solid-state imaging device 10 can be used.

In the field of traffic, for example, for safe driving such as automatic stop and recognition of the driver's condition, in-vehicle sensors that photograph the front, rear, surroundings, inside of the vehicle, etc., and the traveling vehicle and the road are monitored. The solid-state imaging device 10 can be used as a device used for traffic such as a surveillance camera and a distance measuring sensor that measures a distance between vehicles.

In the field of home appliances, for example, a device used for home appliances such as a television receiver, a refrigerator, and an air conditioner in order to photograph a user's gesture and operate the device according to the gesture. Can be used. Further, in the field of medical care / healthcare, the solid-state imaging device 10 is used in a device used for medical care or healthcare, such as an endoscope or a device for performing angiography by receiving infrared light. can do.

In the field of security, the solid-state image sensor 10 can be used in a device used for security such as a surveillance camera for crime prevention and a camera for personal authentication. Further, in the field of beauty, the solid-state image sensor 10 can be used in a device used for beauty such as a skin measuring device for photographing the skin and a microscope for photographing the scalp.

In the field of sports, the solid-state image sensor 10 can be used in a device used for sports, such as an action camera or a wearable camera for sports applications. Further, in the field of agriculture, the solid-state image sensor 10 can be used in a device used for agriculture, such as a camera for monitoring the state of a field or a crop.

<5. Application example to moving body>

The technology related to this disclosure (this technology) can be applied to various products. For example, the technology according to the present disclosure is realized as a device mounted on a moving body of any kind such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, a personal mobility, an airplane, a drone, a ship, and a robot. You may.

FIG. 21 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a moving body control system to which the technique according to the present disclosure can be applied.

The vehicle control system 12000 includes a plurality of electronic control units connected via the communication network 12001. In the example shown in FIG. 21, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an outside information detection unit 12030, an in-vehicle information detection unit 12040, and an integrated control unit 12050. Further, as a functional configuration of the integrated control unit 12050, a microcomputer 12051, an audio image output unit 12052, and an in-vehicle network I / F (interface) 12053 are shown.

The drive system control unit 12010 controls the operation of the device related to the drive system of the vehicle according to various programs. For example, the drive system control unit 12010 provides a driving force generator for generating the driving force of the vehicle such as an internal combustion engine or a driving motor, a driving force transmission mechanism for transmitting the driving force to the wheels, and a steering angle of the vehicle. It functions as a control device such as a steering mechanism for adjusting and a braking device for generating braking force of the vehicle.

The body system control unit 12020 controls the operation of various devices mounted on the vehicle body according to various programs. For example, the body system control unit 12020 functions as a keyless entry system, a smart key system, a power window device, or a control device for various lamps such as headlamps, back lamps, brake lamps, blinkers or fog lamps. In this case, the body system control unit 12020 may be input with radio waves transmitted from a portable device that substitutes for the key or signals of various switches. The body system control unit 12020 receives inputs of these radio waves or signals and controls a vehicle door lock device, a power window device, a lamp, and the like.

The vehicle outside information detection unit 12030 detects information outside the vehicle equipped with the vehicle control system 12000. For example, an imaging unit 12031 is connected to the vehicle exterior information detection unit 12030. The vehicle outside information detection unit 12030 causes the image pickup unit 12031 to capture an image of the outside of the vehicle and receives the captured image. The vehicle exterior information detection unit 12030 may perform object detection processing or distance detection processing such as a person, a vehicle, an obstacle, a sign, or characters on the road surface based on the received image.

The imaging unit 12031 is an optical sensor that receives light and outputs an electric signal according to the amount of the light received. The image pickup unit 12031 can output an electric signal as an image or can output it as distance measurement information. Further, the light received by the imaging unit 12031 may be visible light or invisible light such as infrared light.

The in-vehicle information detection unit 12040 detects the in-vehicle information. For example, a driver state detection unit 12041 that detects the driver's state is connected to the in-vehicle information detection unit 12040. The driver state detection unit 12041 includes, for example, a camera that images the driver, and the in-vehicle information detection unit 12040 determines the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 12041. It may be calculated, or it may be determined whether the driver is dozing.

The microcomputer 12051 calculates the control target value of the driving force generator, the steering mechanism, or the braking device based on the information inside and outside the vehicle acquired by the vehicle exterior information detection unit 12030 or the vehicle interior information detection unit 12040, and the drive system control unit. A control command can be output to 12010. For example, the microcomputer 12051 realizes ADAS (Advanced Driver Assistance System) functions including vehicle collision avoidance or impact mitigation, follow-up driving based on inter-vehicle distance, vehicle speed maintenance driving, vehicle collision warning, vehicle lane deviation warning, and the like. It is possible to perform cooperative control for the purpose of.

Further, the microcomputer 12051 controls the driving force generator, the steering mechanism, the braking device, and the like based on the information around the vehicle acquired by the vehicle exterior information detection unit 12030 or the vehicle interior information detection unit 12040. It is possible to perform coordinated control for the purpose of automatic driving that runs autonomously without depending on the operation.

Further, the microcomputer 12051 can output a control command to the body system control unit 12020 based on the information outside the vehicle acquired by the vehicle exterior information detection unit 12030. For example, the microcomputer 12051 controls the headlamps according to the position of the preceding vehicle or the oncoming vehicle detected by the external information detection unit 12030, and performs cooperative control for the purpose of antiglare such as switching the high beam to the low beam. It can be carried out.

The audio image output unit 12052 transmits the output signal of at least one of the audio and the image to the output device capable of visually or audibly notifying the passenger of the vehicle or the outside of the vehicle. In the example of FIG. 21, an audio speaker 12061, a display unit 12062, and an instrument panel 12063 are exemplified as output devices. The display unit 12062 may include, for example, at least one of an onboard display and a heads-up display.

FIG. 22 is a diagram showing an example of the installation position of the imaging unit 12031.

In FIG. 22, the vehicle 12100 has

imaging units

12101, 12102, 12103, 12104, 12105 as imaging units 12031.

The

imaging units

12101, 12102, 12103, 12104, 12105 are provided at positions such as, for example, the front nose, side mirrors, rear bumpers, back doors, and the upper part of the windshield in the vehicle interior of the vehicle 12100. The imaging unit 12101 provided on the front nose and the imaging unit 12105 provided on the upper part of the windshield in the vehicle interior mainly acquire an image in front of the vehicle 12100. The

imaging units

12102 and 12103 provided in the side mirrors mainly acquire images of the side of the vehicle 12100. The imaging unit 12104 provided on the rear bumper or the back door mainly acquires an image of the rear of the vehicle 12100. The images in front acquired by the

imaging units

12101 and 12105 are mainly used for detecting a preceding vehicle, a pedestrian, an obstacle, a traffic light, a traffic sign, a lane, or the like.

Note that FIG. 22 shows an example of the photographing range of the imaging units 12101 to 12104. The imaging range 12111 indicates the imaging range of the imaging unit 12101 provided on the front nose, the imaging ranges 12112 and 12113 indicate the imaging ranges of the

imaging units

12102 and 12103 provided on the side mirrors, respectively, and the imaging range 12114 indicates the imaging range of the

imaging units

12102 and 12103. The imaging range of the imaging unit 12104 provided on the rear bumper or the back door is shown. For example, by superimposing the image data captured by the imaging units 12101 to 12104, a bird's-eye view image of the vehicle 12100 as viewed from above can be obtained.

At least one of the imaging units 12101 to 12104 may have a function of acquiring distance information. For example, at least one of the image pickup units 12101 to 12104 may be a stereo camera composed of a plurality of image pickup elements, or may be an image pickup element having pixels for phase difference detection.

For example, the microcomputer 12051 has a distance to each three-dimensional object within the imaging range 12111 to 12114 based on the distance information obtained from the imaging units 12101 to 12104, and a temporal change of this distance (relative velocity with respect to the vehicle 12100). By obtaining, it is possible to extract as the preceding vehicle a three-dimensional object that is the closest three-dimensional object on the traveling path of the vehicle 12100 and that travels in substantially the same direction as the vehicle 12100 at a predetermined speed (for example, 0 km / h or more). it can. Further, the microcomputer 12051 can set an inter-vehicle distance to be secured in front of the preceding vehicle in advance, and can perform automatic braking control (including follow-up stop control), automatic acceleration control (including follow-up start control), and the like. In this way, it is possible to perform coordinated control for the purpose of automatic driving or the like in which the vehicle travels autonomously without depending on the operation of the driver.

For example, the microcomputer 12051 converts three-dimensional object data related to a three-dimensional object into two-wheeled vehicles, ordinary vehicles, large vehicles, pedestrians, utility poles, and other three-dimensional objects based on the distance information obtained from the imaging units 12101 to 12104. It can be classified and extracted and used for automatic avoidance of obstacles. For example, the microcomputer 12051 distinguishes obstacles around the vehicle 12100 into obstacles that can be seen by the driver of the vehicle 12100 and obstacles that are difficult to see. Then, the microcomputer 12051 determines the collision risk indicating the risk of collision with each obstacle, and when the collision risk is equal to or higher than the set value and there is a possibility of collision, the microcomputer 12051 via the audio speaker 12061 or the display unit 12062. By outputting an alarm to the driver and performing forced deceleration and avoidance steering via the drive system control unit 12010, driving support for collision avoidance can be provided.

At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared rays. For example, the microcomputer 12051 can recognize a pedestrian by determining whether or not a pedestrian is present in the captured image of the imaging units 12101 to 12104. Such pedestrian recognition includes, for example, a procedure for extracting feature points in an image captured by an imaging unit 12101 to 12104 as an infrared camera, and pattern matching processing for a series of feature points indicating the outline of an object to determine whether or not the pedestrian is a pedestrian. It is done by the procedure to determine. When the microcomputer 12051 determines that a pedestrian is present in the captured images of the imaging units 12101 to 12104 and recognizes the pedestrian, the audio image output unit 12052 outputs a square contour line for emphasizing the recognized pedestrian. The display unit 12062 is controlled so as to superimpose and display. Further, the audio image output unit 12052 may control the display unit 12062 so as to display an icon or the like indicating a pedestrian at a desired position.

The above is an example of a vehicle control system to which the technology according to the present disclosure can be applied. The technique according to the present disclosure can be applied to the imaging unit 12031 among the configurations described above. Specifically, the solid-state image sensor 10 of FIG. 4 can be applied to the image pickup unit 12031. By applying the technique according to the present disclosure to the imaging unit 12031, it is possible to realize motion detection at a high frame rate and with less erroneous determination due to the influence of focal plane distortion or the like.

It should be noted that the embodiment of the present technology is not limited to the above-described embodiment, and various changes can be made without departing from the gist of the present technology.

In addition, this technology can have the following configuration.

(1)
It is equipped with a pixel array unit in which a plurality of pixels are arranged two-dimensionally.
The pixel includes a photoelectric conversion unit and an AD conversion unit that AD-converts a pixel signal obtained by photoelectric conversion by the photoelectric conversion unit.
In the pixel array unit, a solid-state image sensor in which some pixels are thinned out when reading AD conversion results from the plurality of pixels.
(2)
The solid according to (1) above, further comprising a signal processing unit that performs predetermined signal processing based on a thinned image obtained from the AD conversion result read from the plurality of pixels excluding some of the pixels. Image sensor.
(3)
The solid-state image sensor according to (1) or (2) above, further comprising a sequence circuit for reading and a reading unit for reading the AD conversion result in cluster units according to pixel blocks.
(4)
The sequential circuit includes a flip-flop.
The read unit includes a plurality of repeater circuits including a shift register to which the flip-flops are connected in a number corresponding to the number of stages of the cluster.
The solid-state image sensor according to (3) above, wherein the cluster and the flip-flop are connected in pairs.
(5)
The solid-state image sensor according to (4), wherein the repeater circuit reads out the AD conversion result from the pixels in the cluster that are sequentially selected at a predetermined timing.
(6)
The reading unit further includes a number of latch circuits corresponding to the plurality of pixels.
The AD conversion unit included in the pixel and the latch circuit are connected in pairs.
The solid-state image sensor according to (4) or (5), wherein the inverted output of the comparator included in the AD conversion unit is input to the latch circuit.
(7)
The solid-state image sensor according to any one of (4) to (6) above, wherein the reading unit further includes a selector for selecting the repeater circuit according to the thinning amount from the plurality of repeater circuits.
(8)
The solid-state image sensor according to (2) above, wherein the signal processing unit detects motion based on the plurality of thinned-out images.
(9)
The motion detection includes motion detection using a representative point matching method.
The signal processing unit
Hold a specific thinned image as a template block,
The solid-state image sensor according to (8) above, wherein the motion vector is detected by correlating the pixel values of the matching block with the pixel values of the held template block, using the thinned images obtained in sequence as the matching block.
(10)
In the second imaging time after the first imaging, the AD conversion read from the plurality of pixels arranged in the pixel array unit based on the motion vector detected in the first imaging. The solid-state image sensor according to (9) above, wherein the reading start position of the result is selected and motion correction is performed.
(11)
The solid-state imaging device according to any one of (1) to (10), wherein in the pixel array unit, the plurality of pixels arranged in a two-dimensional manner are regularly thinned out in pixel block units.
(12)
It is equipped with a pixel array unit in which a plurality of pixels are arranged two-dimensionally.
The pixel includes a photoelectric conversion unit and an AD conversion unit that AD-converts a pixel signal obtained by photoelectric conversion by the photoelectric conversion unit.
The pixel array unit is an electronic device equipped with a solid-state image sensor in which some pixels are thinned out when reading AD conversion results from the plurality of pixels.

10 Solid-state imaging device, 100A pixel board, 100B logic board, 101 pixel array part, 102 repeater part, 103 GC generation part, 104 SRAM, 105 signal processing part, 106 motion judgment memory, 107 DAC, 108 latch repeater part, 111 pixels, 121 clusters, 131 repeaters, 132 repeater selectors, 141 shift registers, 142 flip-flops, 151 ADCs, 152 latch circuits, 161 pixel circuits, 162 differential amplification circuits, 163 positive feedback circuits, 164 multiple circuits, 1000 electronic devices. , 1012 Solid-state imaging device

Claims

It is equipped with a pixel array unit in which a plurality of pixels are arranged two-dimensionally.
The pixel includes a photoelectric conversion unit and an AD conversion unit that AD-converts a pixel signal obtained by photoelectric conversion by the photoelectric conversion unit.
In the pixel array unit, a solid-state image sensor in which some pixels are thinned out when reading AD conversion results from the plurality of pixels.
The solid-state imaging according to claim 1, further comprising a signal processing unit that performs predetermined signal processing based on a thinned-out image obtained from the AD conversion result read from the plurality of pixels excluding some of the pixels. apparatus.
The solid-state image sensor according to claim 1, further comprising a sequence circuit for reading and a reading unit for reading the AD conversion result in cluster units according to pixel blocks.
The sequential circuit includes a flip-flop.
The read unit includes a plurality of repeater circuits including a shift register to which the flip-flops are connected in a number corresponding to the number of stages of the cluster.
The solid-state image sensor according to claim 3, wherein the cluster and the flip-flop are connected in pairs.
The solid-state image sensor according to claim 4, wherein the repeater circuit reads the AD conversion result from the pixels in the cluster that are sequentially selected at a predetermined timing.
The reading unit further includes a number of latch circuits corresponding to the plurality of pixels.
The AD conversion unit included in the pixel and the latch circuit are connected in pairs.
The solid-state image sensor according to claim 4, wherein the inverted output of the comparator included in the AD conversion unit is input to the latch circuit.
The solid-state image sensor according to claim 4, wherein the reading unit further includes a selector that selects the repeater circuit according to the thinning amount from the plurality of repeater circuits.
The solid-state image sensor according to claim 2, wherein the signal processing unit detects motion based on the plurality of thinned-out images.
The motion detection includes motion detection using a representative point matching method.
The signal processing unit
Hold a specific thinned image as a template block,
The solid-state image sensor according to claim 8, wherein the thinned-out images obtained in sequence are used as matching blocks, and the motion vector is detected by correlating the pixel value of the matching block with the pixel value of the held template block.
In the second imaging time after the first imaging, the AD conversion read from the plurality of pixels arranged in the pixel array unit based on the motion vector detected in the first imaging. The solid-state image sensor according to claim 9, wherein the reading start position of the result is selected and motion correction is performed.
The solid-state image sensor according to claim 1, wherein in the pixel array unit, the plurality of pixels arranged in a two-dimensional manner are regularly thinned out in pixel block units.
It is equipped with a pixel array unit in which a plurality of pixels are arranged two-dimensionally.
The pixel includes a photoelectric conversion unit and an AD conversion unit that AD-converts a pixel signal obtained by photoelectric conversion by the photoelectric conversion unit.
The pixel array unit is an electronic device equipped with a solid-state image sensor in which some pixels are thinned out when reading AD conversion results from the plurality of pixels.