JP2020136686A - Solid-state image pickup device and driving method of solid-state image pickup device - Google Patents

Abstract

To provide a solid-state image pickup device having a wide dynamic range at one imaging, and a solid-state image pickup device.SOLUTION: A solid-state image pickup device includes: a pixel part having Y×X pixels arranged in a first direction and a second direction which is crossed to the first direction; an exposure control circuit ; and an exposure time calculation circuit. The pixel part includes: a first photoelectric conversion element provided so as to correspond to the Y×X pixels; and a second photoelectric conversion element provided to each of the Y×X pixels in the Y×X pixels. The second photoelectric conversion element is overlapped to the first photoelectric conversion element, and the first photoelectric conversion element outputs a brightness signal corresponding to a light reception intensity to the exposure time calculation circuit. The exposure time calculation circuit calculates an exposure time of the second photoelectric conversion element on the basis of the brightness signal. The exposure control circuit controls an exposure of the second photoelectric conversion element in each of the Y×X pixels per Y×X pixel unit.SELECTED DRAWING: Figure 1

Description

One embodiment of the present invention relates to a solid-state image sensor and a method for driving the solid-state image sensor.

In recent years, electronic devices equipped with a solid-state image sensor have become widespread. The solid-state image sensor is provided in an electronic device having a function for photographing a solid-state, such as a portable information terminal and a digital camera. The solid-state image sensor is also called an image sensor. Examples of the image sensor include a CMOS (Complementary Metal Oxide Semiconductor) image sensor, a CCD (Charge Coupled Device) image sensor, and the like.

The solid-state image sensor is, for example, a plurality of pixels arranged in a matrix in a row (row) direction and a column (column) direction, a row selection scanning circuit that selects pixels to be read in row units, and an analog-digital converter. Includes a converter (Analog to Digital Converter (ADC, AD converter)). The pixel has a light receiving element. The light receiving element is, for example, a photoelectric conversion element. The photoelectric conversion element is, for example, a photodiode (Photo Diode, PD). In the CMOS image sensor, for example, each pixel from the pixel in the first row to the pixel in the P (P is a natural number) row is selected by a row (row) selection scanning circuit. For each selected row, the light received by each pixel is converted into an analog signal (also called an analog voltage) by the photoelectric conversion element corresponding to each pixel. Each analog signal photoelectrically converted for each line is output from each pixel. The AD converter converts the analog signal output for each line into a digital signal. The digital signal in each pixel from the pixel in the first row to the pixel in the Pth row is image-processed in, for example, an image processing circuit to obtain one image for the subject.

The dynamic range (also referred to as light-dark ratio or illuminance ratio) is one of the indexes showing the performance of the solid-state image sensor. The wide dynamic range (Wide Dynamic Range, WDR) means that it is possible to clearly project the difference in brightness. That is, a solid-state image sensor having a wide dynamic range can image a subject having a large light-dark ratio, and can clearly image both a bright place and a dark place.

For example, the image processing apparatus described in Non-Patent Document 1 discloses a technique of synthesizing two images taken at different exposure times to generate one composite image. Further, the image pickup apparatus described in Non-Patent Document 2 has a plurality of light receiving elements whose exposure time can be changed independently, and generates one image from the plurality of images captured by each light receiving element. The technology to be used is disclosed.

Mirumiru Column, Part 10 The wide dynamic range function is [online], [Search on December 5, 2018], Internet <URL: https: // www. toa. co. jp / miru2 / volume / volume10. htm> Hirokazu Kobayashi, 6 outsiders, Development of Super CCD Honeycomb SRII, a wide dynamic range image sensor for single-lens reflex cameras, FUJIFILM RESERCH & DEVELOPMENT No. 50-2005 pp. 1-3, [online], Light emission on March 22, 2005, Material Research Headquarters, R & D Headquarters, Fuji Film Co., Ltd., [Search on December 5, 2018], Internet <URL: https: // www. FUJIFILM. co. jp / rd / report / rd050 / pack / pdf / ff_rd050_001. pdf>

However, in the technique disclosed in Non-Patent Document 1, since one image is generated from a plurality of images captured by the light receiving element, there is a problem that it is difficult to image in an environment where a moving object exists. is there. Further, in the techniques disclosed in Non-Patent Document 1 and Non-Patent Document 2, since one image is generated from a plurality of images captured by the light receiving element, a frame memory for synthesizing the images is required. There is a problem that the frame rate is lowered due to the large amount of data transfer.

The present invention has been made in view of the above problems, and an object of the present invention is to provide a solid-state image sensor and a solid-state image sensor having a wide dynamic range in one image capture instead of a plurality of images.

The solid-state image sensor according to an embodiment of the present invention has Y × X pixels arranged in a first direction and a second direction intersecting the first direction (Y and X are natural numbers set independently of each other). ), An exposure control circuit, and an exposure time calculation circuit, and the pixel unit is set to y × x (y and x are independently set) in the Y × X pixels. The second photoelectric conversion element has a first photoelectric conversion element provided corresponding to the pixels of the natural number) and a second photoelectric conversion element provided for each of the Y × X pixels. Overlapping with the first photoelectric conversion element, the first photoelectric conversion element supplies a brightness signal corresponding to the light receiving intensity to the exposure time calculation circuit, and the exposure time calculation circuit calculates the exposure time of the second photoelectric conversion element based on the brightness signal. Then, the exposure control circuit controls the exposure of the second photoelectric conversion element of each of the y × x pixels in units of the y × x pixels.

In the driving method of the solid-state image sensor according to the embodiment of the present invention, Y × X pieces (Y and X are set independently) arranged in the first direction and the second direction intersecting the first direction. A first photoelectric conversion element and a first photoelectric conversion element provided corresponding to y × x (natural numbers in which y and x are set independently) of a pixel portion having (natural number) pixels. A method for driving a solid-state image sensor including a second photoelectric conversion element, an exposure time calculation circuit, an exposure control circuit, and a signal processing circuit provided for each of the y × x pixels. 1 The photoelectric conversion element generates a brightness signal according to the light receiving intensity, the exposure time calculation circuit calculates the exposure time of the second photoelectric conversion element according to the brightness signal, and the exposure control circuit calculates the exposure time based on the exposure time. The exposure of the second photoelectric conversion element is controlled, and the signal processing circuit generates a signal corresponding to the exposure time of the second photoelectric conversion element provided in each of the y × x pixels.

According to one embodiment of the present invention, it is possible to provide a solid-state image sensor and a solid-state image sensor having a wide dynamic range in one imaging.

It is a top view which shows the structure of the solid-state image sensor which concerns on one Embodiment of this invention. It is sectional drawing which shows the structure of the solid-state image sensor which concerns on one Embodiment of this invention. It is sectional drawing which shows the structure of the solid-state image sensor which concerns on one Embodiment of this invention. It is a top view which shows the structure of the pixel part of the solid-state image sensor which concerns on one Embodiment of this invention. It is a top view which shows the structure of the pixel part of the solid-state image sensor which concerns on one Embodiment of this invention. It is a top view which shows the structure of the pixel part of the solid-state image sensor which concerns on one Embodiment of this invention. (A) is a schematic diagram showing the brightness and darkness of the pixels of the solid-state image sensor according to the embodiment of the present invention. (B) is a timing chart showing a driving method of the solid-state image sensor shown in (A). It is a flowchart which shows the driving method of the solid-state image sensor which concerns on one Embodiment of this invention. (A) is a schematic diagram showing the brightness and darkness of pixels in the t-1st frame of the solid-state image sensor according to the embodiment of the present invention. (B) is a schematic diagram showing the brightness and darkness of pixels in the t-th frame of the solid-state image sensor according to the embodiment of the present invention. (C) is a schematic diagram showing the brightness and darkness of pixels in the t + 1th frame of the solid-state image sensor according to the embodiment of the present invention. 9 is a timing chart showing a driving method of a solid-state image sensor according to an embodiment of the present invention in the t-1st frame shown in FIG. 9A and the tth frame shown in FIG. 9B. .. 9 is a timing chart showing a driving method of the solid-state image sensor according to the embodiment of the present invention in the t-th frame shown in FIG. 9B and the t + 1th frame shown in FIG. 9C.

Hereinafter, each embodiment of the present invention will be described with reference to the drawings. The disclosure is just an example. That is, a configuration that can be easily conceived by a person skilled in the art by maintaining the gist of the invention and appropriately modifying the invention is naturally included in the scope of the present invention. In order to clarify the explanation, the drawings may be schematically represented by the width, thickness, shape, etc. of each part as compared with the actual aspect. However, these are merely examples and do not limit the interpretation of the present invention.

In the following description, a configuration in which a photoelectric conversion element in which an electromotive force is generated by light is used as the light receiving element will be illustrated, but the present invention is not limited to this configuration. For example, as the light receiving element, a photoelectric conversion element whose electric conductivity changes with light may be used. Alternatively, as the light receiving element, a photoelectric conversion element of a type that converts the characteristics of the received light (for example, the wavelength of light) into an electrical signal may be used. Alternatively, as the light receiving element, an element that converts the received light into information other than electrical information may be used. In the following description, the received light may be referred to as exposed light, and the received light may be referred to as exposed light. In addition, receiving light may be referred to as exposing light. Further, the solid-state image sensor according to the embodiment of the present invention includes a transistor, a capacitive element, a resistance element, and the like. As the structure of a transistor, a capacitive element, a resistance element, etc., the film, the layer, and the material of each part forming the transistor, the capacitive element, the resistance element, etc., a known technique usually used in the technical field of the present invention may be adopted. it can. For example, the solid-state image sensor according to an embodiment of the present invention is a semiconductor device that utilizes semiconductor characteristics such as silicon, and a transistor included in the solid-state image sensor is formed of CMOS by a semiconductor device manufacturing process, which is a known technique. May be good.

It should be noted that the ordinal numbers such as "first", "second", and "third" in the present specification and the like are used only for the sake of brevity and should not be construed in a limited manner.

The following embodiments can be combined with each other as long as there is no technical contradiction.

1. 1. First Embodiment 1-1. Configuration of Solid-State Image Sensor 10 The outline of the solid-state image sensor 10 according to the embodiment of the present invention will be described with reference to FIGS. 1 to 3. FIG. 1 is a schematic view showing an outline of a solid-state image sensor 10 according to an embodiment of the present invention. FIG. 2 is a cross-sectional view showing the configuration of the solid-state image sensor 10 according to the embodiment of the present invention. FIG. 3 is a cross-sectional view showing the configuration of the solid-state image sensor 10 according to the embodiment of the present invention. Note that FIG. 2 is a schematic cross-sectional view taken along the lines A1 to A2 shown in FIG. 1, and is a plane parallel or substantially parallel along the X direction of the solid-state image sensor 10. FIG. 3 is a schematic cross-sectional view taken along the lines B1 to B2 shown in FIG. 2, and is a plane parallel or substantially parallel along the Y direction of the solid-state image sensor 10.

As shown in FIG. 1, the solid-state image sensor 10 is roughly classified into a pixel portion 101 and a peripheral portion 103. The pixel unit 101 has a plurality of luminance groups 150. The peripheral portion 103 is a peripheral region with respect to the pixel portion 101. The peripheral portion 103 is an area in which the image pickup timing control circuit 210, the second photoelectric conversion element exposure control circuit 220, the storage circuit 230, the exposure time calculation circuit 240, the readout circuit 300, and the image processing circuit 500 are arranged.

The luminance groups 150 are arranged in a matrix in the rectangular pixel portion 101. The brightness groups 150 are arranged in a matrix of N rows and M columns. That is, N × M luminance groups 150 are arranged in the pixel unit 101. Here, N and M are natural numbers that are set independently. The luminance group 150 has one first photoelectric conversion element 62 and y × x pixels 100. Here, y and x are natural numbers that are set independently. Further, in the present specification and the like, one first photoelectric conversion element 62 and y × x pixels or a set of pixels may be paraphrased as one luminance group 150. In the example of FIG. 1, the arrangement of the luminance group 150 is N = 3 and M = 4, but is not limited to this value. Further, in the example of FIG. 1, the luminance group 150 has 2 × 2 pixels 100 of y = x = 2, but is not limited to this example. Further, the first photoelectric conversion element 62 included in the luminance group 150 generates a first signal (or a first voltage) corresponding to the electric power generated based on the intensity of the exposed light. The first signal (or first voltage) is an analog signal (or analog voltage). The first signal (or first voltage) is also called a luminance signal. The intensity of light may be called brightness. The intensity of the exposed light is also called the exposure intensity or the light receiving intensity. The details of the configuration of the luminance group 150 will be described later.

Pixels 100 are arranged in a matrix of Y rows and X columns. That is, in the pixel unit 101, Y × X pixels 100 are arranged. Here, Y and X are natural numbers that are set independently. In the example of FIG. 1, the luminance group is arranged with 2 × 2 pixels 100 of y = x = 2, and 6 × 8 pixels 100 are arranged in the pixel portion 101, but the value is not limited to this value. Further, the pixel 100 has a second photoelectric conversion element 72. The second photoelectric conversion element 72 generates a second signal (or a second voltage) corresponding to the electric power generated based on the intensity of the exposed light. The second signal (or second voltage) is an analog signal (or analog voltage). In the present specification and the like, x is X or less and y is Y or less. Further, X is preferably a multiple of x and Y is preferably a multiple of y, but is not limited to this example.

Further, as shown in FIGS. 2 and 3, the pixel 100 includes a signal processing circuit 52 and a second photoelectric conversion element 72. Further, the pixel 100 may have any one of the red color filter 82, the green color filter 84, the blue color filter 86, and the optical element 92. Here, the signal processing circuit 52 may be electrically connected to the first photoelectric conversion element 62 and the second photoelectric conversion element 72. For example, by connecting the signal processing circuit 52 to the first photoelectric conversion element 62, the signal processing circuit 52 controls the timing of generating the first signal (luminance signal) executed by the first photoelectric conversion element 62. be able to. Further, by connecting the signal processing circuit 52 to the second photoelectric conversion element 72, the signal processing circuit 52 can control the timing of generating the second signal executed by the second photoelectric conversion element 72.

Although details will be described later, as shown in FIGS. 2 and 3, in the solid-state image sensor 10, the first photoelectric conversion element 62 and the second photoelectric conversion element 72 are arranged so as to overlap each other. The first photoelectric conversion element 62 can generate a luminance signal based on the light receiving intensity. That is, the first photoelectric conversion element 62 can detect the brightness. The second photoelectric conversion element 72 can expose the subject. Further, as will be described in detail later, when the solid-state imaging device 10 images a subject, the brightness is detected by the first photoelectric conversion element 62 before the subject is exposed by the second photoelectric conversion element 72. Next, based on the detected brightness, the exposure time when the subject is exposed by the second photoelectric conversion element 72 is calculated. Subsequently, the subject is exposed by the second photoelectric conversion element 72 based on the calculated exposure time. Since the solid-state image sensor 10 according to the embodiment of the present invention has two photoelectric conversion elements, the brightness can be detected in advance and the subject can be imaged based on the detected brightness. Therefore, the subject can be imaged in one image. It is possible to obtain an image that reflects the brightness and darkness of. That is, the solid-state image sensor 10 according to the embodiment of the present invention can widen the dynamic range by one imaging.

In this specification, an example is shown in which the first photoelectric conversion element 62 and the second photoelectric conversion element 72 are photodiodes. Further, the first photoelectric conversion element 62 may be referred to as a first PD 62, and the second photoelectric conversion element 72 may be referred to as a second PD 72.

The image pickup timing control circuit 210 is arranged at a position adjacent to the pixel portion 101 in the row direction in the peripheral portion 103. For example, a horizontal signal line (not shown) is connected to the image pickup timing control circuit 210. The horizontal signal line is connected to a plurality of pixels 100 arranged in the same row. A control signal for controlling each pixel 100 is input to the horizontal signal line. In FIG. 1, a plurality of control signals are collectively indicated by a control signal 214. Each of the control signals 214 is sequentially input line by line. For example, each of the control signals 214 is sequentially input line by line, such as the first line, the second line, the third line, and so on. However, each of the control signals 214 may not be input sequentially for each line as described above, but may be randomly input for each line, or may be input for each line at the same time. Further, the image pickup timing control circuit 210 is electrically connected to the second photoelectric conversion element exposure control circuit 220 by the wiring 212. The imaging timing control circuit 210 generates an imaging position control signal and supplies the imaging position control signal to the wiring 212. The imaging position control signal is supplied to the second photoelectric conversion element exposure control circuit 220. Although details will be described, the solid-state image sensor 10 according to the embodiment of the present invention can control the imaging timing of the second photoelectric conversion element 72 based on the imaging position control signal.

The readout circuit 300 is arranged at a position adjacent to the pixel portion 101 in the column direction in the peripheral portion 103. A first vertical signal line (not shown) is connected to the readout circuit 300. The first vertical signal line is connected to a plurality of pixels 100 or a plurality of first photoelectric conversion elements 62 arranged in the same row. A second signal corresponding to the electric power generated by the second photoelectric conversion element 72 provided in each pixel 100 is output from the pixel 100 arranged in the row selected by the row selection scanning circuit 200. The second signal output from each pixel 100 is supplied to the first vertical signal line. The second signal is image data corresponding to the exposed subject. Further, as will be described in detail later, the second signal is image data exposed based on the exposure time corresponding to each luminance group 150. That is, the second signal is image data corresponding to the exposed subject. The plurality of second signals are collectively indicated by the second signal 114. Further, a first signal corresponding to the electric power generated by the first photoelectric conversion element 62 may be supplied to the first vertical signal line. The plurality of first signals are collectively indicated by the first signal 112.

The readout circuit 300 includes, for example, an AD converter (not shown) and a horizontal transfer scanning circuit (not shown). The first signal 112 and the second signal 114 supplied to the first vertical signal line are converted into digital signals by the AD converter. The digital signal is transferred to the horizontal transfer scanning circuit. The horizontal transfer scanning circuit reads digital signals sequentially column by column. When the horizontal transfer scanning circuit reads out one row of digital signals, the first signal 112 and the second signal 114 of the row selected by the row selection scanning circuit (not shown) can be read out as digital signals. In the present specification and the like, the digital signal for the first signal 112 of each line is referred to as a first digital signal. The first digital signal of each line is collectively shown by the first digital signal 302. Similarly, the digital signal for the second signal 114 of each line is called the second digital signal, and the second digital signal of each line is collectively indicated by the second digital signal 304. The first digital signal 302 is a digital luminance signal corresponding to the luminance signal of each luminance group 150. The second digital signal 304 is digital image data corresponding to the subject. The first signal 112 and the second signal 114 may be transferred to the horizontal transfer scanning circuit as analog signals without being converted into digital signals by the AD converter, and may be sequentially read out for each column.

The exposure time calculation circuit 240 shows, for example, an example of being electrically connected to the readout circuit 300, the image processing circuit 500, and the storage circuit 230, but is not limited to this example. The arrangement of the exposure time calculation circuit 240, the electrical connection, and the like may be appropriately determined within a range that does not deviate from the configuration of the solid-state image sensor 10 according to the embodiment of the present invention. The first digital signal 302 is supplied to the exposure time calculation circuit 240. As described above, the first digital signal 302 is a digital luminance signal corresponding to the luminance signals of each luminance group 150. The exposure time calculation circuit 240 can calculate an exposure time suitable for exposing a subject by a plurality of second photoelectric conversion elements 72 of each luminance group 150 based on the first digital signal 302. The calculated exposure time data corresponding to each luminance group 150 is supplied to the wiring 242 and supplied to the storage circuit 230. The first signal 112 and the second signal 114 may not be converted into digital signals and may be processed as analog signals.

The storage circuit 230 shows, for example, an example of being electrically connected to the exposure time calculation circuit 240 and the second photoelectric conversion element exposure control circuit 220, but is not limited to this example. The arrangement of the storage circuit 230, the electrical connection, and the like may be appropriately determined within a range that does not deviate from the configuration of the solid-state image sensor 10 according to the embodiment of the present invention. The storage circuit 230 is supplied with exposure time data corresponding to each calculated luminance group 150. The storage circuit 230 can store the calculated exposure time data corresponding to each luminance group 150. The storage circuit 230 is, for example, a volatile memory. The storage circuit 230 may have a plurality of volatile memories. When the storage circuit 230 has a plurality of memories, the memory capacity of the storage circuit 230 increases. Therefore, even if the resolution of the solid-state image sensor 10 is high, a sufficient amount of exposure time data can be stored in the storage circuit 230. The storage circuit 230 may be built in the solid-state image sensor 10 or may be connected to the outside. For example, the storage circuit 230 may be removable, such as a DIMM or SO-DIMM. The exposure time data corresponding to each luminance group 150 stored in the storage circuit 230 is read out from the storage circuit 230. The read exposure time data is supplied to the wiring 232 and supplied to the second photoelectric conversion element exposure control circuit 220.

The second photoelectric conversion element exposure control circuit 220 shows an example in which the peripheral portion 103 is arranged at a position adjacent to the pixel portion 101 in the column direction, but is not limited to this example. The second photoelectric conversion element exposure control circuit 220 may be arranged at a position adjacent to the pixel portion 101 in the row direction in the peripheral portion 103. The second photoelectric conversion element exposure control circuit 220 shows, for example, an example of being electrically connected to the image pickup timing control circuit 210 and the storage circuit 230, but is not limited to this example. The arrangement, electrical connection, and the like of the second photoelectric conversion element exposure control circuit 220 may be appropriately determined within a range that does not deviate from the configuration of the solid-state image pickup device 10 according to the embodiment of the present invention.

Further, the second photoelectric conversion element exposure control circuit 220 can read out the exposure time data corresponding to each luminance group 150 calculated from the storage circuit 230. The second photoelectric conversion element exposure control circuit 220 is supplied with an imaging position control signal and exposure time data corresponding to each calculated luminance group 150. Based on the imaging position control signal and the calculated exposure time data corresponding to each luminance group 150, the second photoelectric conversion element exposure control circuit 220 controls the exposure of the pixels in each luminance group 150 (luminance group). Exposure signal) may be generated. In FIG. 1, the luminance group exposure signals are collectively shown by the luminance group exposure signal 224.

Further, for example, a second vertical signal line (not shown) may be connected to the second photoelectric conversion element exposure control circuit 220. The second vertical signal line is connected to a plurality of pixels 100 or a plurality of first photoelectric conversion elements 62 arranged in the same row. A luminance group exposure signal 224 is input to the second vertical signal line. The luminance group exposure signal 224 may be sequentially input for each column. For example, the luminance group exposure signal 224 may be sequentially input for each column, such as the first column, the second column, the third column, and so on. Further, the luminance group exposure signal 224 may not be sequentially input for each column as described above, but may be randomly input for each column, or may be input for each column at the same time.

Further, the second photoelectric conversion element exposure control circuit 220 can synchronize the luminance group exposure signal 224 with the control signal 214 by supplying the imaging position control signal from the imaging timing control circuit 210. The second photoelectric conversion element exposure control circuit 220 sets the exposure time corresponding to each luminance group 150 by supplying the exposure time data from the storage circuit 230, and sets the imaging timing of the second photoelectric conversion element 72. Can be controlled. That is, the plurality of second photoelectric conversion elements 72 included in the solid-state image sensor 10 according to the embodiment of the present invention are suitable for exposing a subject calculated based on the luminance signal generated by the first photoelectric conversion element 62. The subject can be exposed with a different exposure time.

The image processing circuit 500 shows an example in which the peripheral portion 103 is arranged at a position adjacent to the pixel portion 101 in the column direction, but the present invention is not limited to this example. The image processing circuit 500 may be arranged at a position adjacent to the pixel portion 101 in the row direction in the peripheral portion 103. The image processing circuit 500 shows an example of being electrically connected to the readout circuit 300, for example, but is not limited to this example. The arrangement of the image processing circuit 500, the electrical connection, and the like may be appropriately determined within a range that does not deviate from the configuration of the solid-state image sensor 10 according to the embodiment of the present invention.

In addition, the image processing circuit 500 included in the solid-state image sensor 10 according to the embodiment of the present invention can acquire the second digital signal 304, which is digital image data corresponding to the subject, by one imaging. That is, the solid-state image sensor 10 according to the embodiment of the present invention can obtain an image reflecting the brightness and darkness of the subject by one imaging, and can widen the dynamic range by one imaging.

Note that FIG. 1 illustrates a configuration in which the pixels 100 are arranged in a matrix, but the configuration is not limited to this configuration. For example, the pixels 100 may be arranged in a shape having a periodicity different from that of the matrix shown in FIG. 1, or may be arranged irregularly. Further, in FIG. 1, a configuration in which the pixel portion 101 is rectangular is illustrated, but the configuration is not limited to this configuration. For example, the pixel portion 101 may be polygonal, circular (including a perfect circle and an ellipse), or curved.

Further, in FIG. 1, in order to promote understanding, it is described that there is a gap between the pixel 100 and the adjacent pixel 100, but in reality, the pixel 100 is spread over the pixel portion 101 with almost no gap. Have been placed.

Further, in FIG. 1, in a plan view, the red color filter 82 is provided inside the second photoelectric conversion element 72, but the present invention is not limited to this example. In a plan view, the red color filter 82 may have the same size as the second photoelectric conversion element 72, or the second photoelectric conversion element 72 may be provided inside the red color filter 82. Good. The green color filter 84 and the blue color filter 86 are the same as the red color filter 82. The arrangement of each color filter and the second photoelectric conversion element 72 may be appropriately determined as long as the configuration of the solid-state image pickup device 10 according to the embodiment of the present invention is not deviated.

The cross-sectional structure of the solid-state image sensor 10 according to the embodiment of the present invention will be described with reference to FIGS. 2 and 3.

As shown in FIGS. 2 and 3, the solid-state image sensor 10 according to the embodiment of the present invention includes a signal processing unit 50, a first photoelectric conversion element unit 60, a second photoelectric conversion element unit 70, and a color filter unit 80. , And an optical element unit 90.

The signal processing unit 50 has a plurality of signal processing circuits 52. The first photoelectric conversion element unit 60 has a plurality of first photoelectric conversion elements 62. The first photoelectric conversion element unit 60 is provided on the signal processing unit 50. The plurality of first photoelectric conversion elements 62 are insulated from each other by, for example, an insulating film such as an inorganic insulating film or an organic insulating film. The second photoelectric conversion element unit 70 has a plurality of second photoelectric conversion elements 72. The second photoelectric conversion element unit 70 is provided on the first photoelectric conversion element unit 60. The plurality of second photoelectric conversion elements 72 are insulated from each other by, for example, an insulating film such as an inorganic insulating film or an organic insulating film. The color filter unit 80 has a plurality of red color filters 82, green color filters 84, and blue color filters 86, respectively. The color filter unit 80 is provided on the second photoelectric conversion element unit 70. The optical element unit 90 has a plurality of optical elements 92. The optical element 92 is, for example, a microlens. The arrangement of the second photoelectric conversion element unit 70 is not limited to the example provided above the first photoelectric conversion element unit 60 as described above, and may be provided below the first photoelectric conversion element unit 60. ..

In the side view, the width of the first photoelectric conversion element 62 is larger than the width of the second photoelectric conversion element 72. Since the width of the first photoelectric conversion element 62 is larger than the width of the second photoelectric conversion element 72, the first photoelectric conversion element 62 can have a larger light receiving area than the second photoelectric conversion element 72, and is therefore large. Easy to form capacity. Therefore, the first photoelectric conversion element 62 is hard to be saturated, and the first photoelectric conversion element 62 can detect a wide range of brightness. Therefore, the solid-state image sensor 10 according to the embodiment of the present invention can calculate the exposure time in a wide range.

Further, in the side view, a gap is provided between the second photoelectric conversion element 72 and the adjacent second photoelectric conversion element 72. Due to this gap, the first photoelectric conversion element 62 can more easily receive or expose the incident light 116 from the outside of the solid-state image sensor 10 to the solid-state image sensor 10, and the solid-state image sensor 10 according to the embodiment of the present invention has brightness. Is easier to detect.

In FIGS. 2 and 3, the widths of the second photoelectric conversion element 72, each color filter, and the optical element 92 are the same in the side view, but the width is not limited to this example. The widths of the second photoelectric conversion element 72, each color filter, and the optical element 92 may be appropriately determined as long as they do not deviate from the configuration of the solid-state image sensor 10 according to the embodiment of the present invention.

1-2. Configuration of Luminance Group 150 The configuration of the luminance group 150 will be described with reference to FIGS. 4 to 6. FIG. 4 is a plan view showing the configuration of the pixel portion 101 of the solid-state image sensor 10 according to the embodiment of the present invention. 5 and 6 are plan views showing the configuration of the brightness group 150 of the solid-state image sensor 10 according to the embodiment of the present invention. In the description of the configuration of the solid-state image sensor 10 according to the embodiment of the present invention using FIGS. 1 to 3, an example in which the arrangement of the luminance groups 150 is N = 3 and M = 4 has been described. In the description of the configuration of the luminance group 150 with reference to FIG. 6, an example in which N = n and M = m will be described. The description of the same or similar configuration as in FIGS. 1 to 3 may be omitted.

As shown in FIG. 4, the pixel unit 101 has a luminance group 150 of N rows and M columns. The brightness group 150 is arranged along a first direction and a second direction that intersects the first direction. For example, n luminance groups 150 are arranged in a direction parallel to the X direction, which is the first direction, and m luminance groups 150 are arranged in a direction parallel to the Y direction, which is the second direction. m and n are natural numbers set independently of each other. The numerical value described inside the luminance group 150 is the coordinates of the luminance group 150. For example, the coordinates of the luminance group 150 arranged in 2 rows and 1 column are shown as (2, 1). The coordinates of the luminance group 150 arranged in n rows and m columns are shown as (n, m).

As described above, the luminance group 150 has one first photoelectric conversion element 62 and y × x pixels 100. That is, one luminance group 150 has one first photoelectric conversion element 62 and y × x pixels or a set of pixels. In the example shown in FIG. 5, y = x = 8. As described above, since each pixel 100 has one second photoelectric conversion element 72, it may be said that the luminance group 150 has 8 × 8 second photoelectric conversion elements 72. Similar to the luminance group 150, the pixels 100 are arranged along a first direction and a second direction that intersects the first direction. For example, X pixels 100 are arranged parallel to the X direction, which is the first direction, and Y pixels are arranged parallel to the Y direction, which is the second direction. X and Y are natural numbers that are set independently. 5 and 6 show an example in which Y = X = 8, but the present invention is not limited to this example. The numerical value described inside the pixel 100 is the coordinate of the pixel 100, similarly to the numerical value described inside the luminance group 150. For example, the coordinates of the pixels 100 arranged in 4 rows and 7 columns are shown as (4, 7). The coordinates of the pixels 100 arranged in 8 rows and 8 columns are shown as (8, 8). The luminance group 150 shown in FIG. 5 is the luminance group 150 arranged in one row and one column in the pixel unit 101. Therefore, the pixels 100 arranged in the luminance group 150 arranged in one row and one column are the pixels 100 having coordinates X of 1 to 8 and Y of 1 to 8.

The luminance group 150 shown in FIG. 6 has the same configuration as the luminance group 150 described in FIG. 5, and is the luminance group 150 arranged in N rows and M columns in the pixel unit 101. Therefore, the pixel 100 arranged in the luminance group 150 arranged in the N rows and M columns is the pixel 100 having coordinates Y of 8n-7 to 8n and X of 8m-7 to 8m.

The solid-state image sensor 10 according to an embodiment of the present invention has the above-described configuration, and thus the brightness is detected by the first photoelectric conversion element 62 provided in each brightness group, and based on the detected brightness. Based on the calculated exposure time, the subject can be exposed by the second photoelectric conversion element 72 provided in each luminance group, and the subject can be imaged. Therefore, by using the solid-state image sensor 10 according to the embodiment of the present invention, the imager can obtain an image in which the brightness and darkness of the subject are reflected.

1-3. Driving Method of Solid-State Image Sensor 10 The driving method of the solid-state image sensor 10 according to the embodiment of the present invention will be described with reference to FIGS. 7 and 8. The description of the same or similar configuration as in FIGS. 1 to 6 may be omitted.

FIG. 7A is a schematic view showing the brightness and darkness of the pixel 100 of the solid-state image sensor 10 according to the embodiment of the present invention. FIG. 7B is a timing chart showing a driving method of the solid-state image sensor 10 shown in FIG. 7A. FIG. 8 is a flowchart showing a driving method of the solid-state image sensor 10 according to the embodiment of the present invention. In FIG. 7, the luminance group is described as luminance G.

As shown as an example in FIG. 7A, the luminance groups 150 are arranged in a matrix of 1 row and 3 columns (N = 1, M = 3). Further, as shown as an example in FIG. 7A, 2 × 2 pixels 100 are arranged in the luminance group, and 2 × 6 pixels 100 are arranged in the pixel portion 101. Further, since the pixel 100 has the second photoelectric conversion element 72, 2 × 2 second photoelectric conversion elements 72 are arranged in the luminance group, and 2 × 6 in the pixel portion 101. The second photoelectric conversion element 72 is arranged. Further, as shown as an example in FIG. 7A, the light exposed by the luminance group 150 in rows and columns (N = 1, M = 1) is assumed to be dark and is indicated by hatching on the left diagonal line. Similarly, the light exposed by the luminance group 150 in 1 row and 2 columns (N = 1, M = 2) is assumed to be halftone and is indicated by hatching on the right diagonal line, and 1 row and 3 columns (N = 1, M = 2). The light exposed by the luminance group 150 of M = 3) is assumed to be bright.

FIG. 7B shows two frames (tth luminance frame t and t + 1th luminance frame t + 1, or t-1st image frame t-1) in the driving method of the solid-state image sensor 10. The t-th image frame t) is shown as an example. Here, the luminance frame is a period related to the exposure of the first PD62, and the image frame is a period related to the exposure of the second PD72. In the following description, an image frame will be used as the frame in the driving method.

As shown in FIG. 8, the driving of the solid-state image sensor 10 is started. First, the solid-state image sensor 10 is initialized (step 31 (S31)). Here, initialization means that when the solid-state image sensor 10 is started, the unnecessary data is stored in the storage circuit 230 in order to prevent data unnecessary for new work of the solid-state image sensor 10 from remaining in the storage circuit 230. Including erasing. Initialization may include returning the solid-state image sensor 10 to its initial state. By initialization, the solid-state image sensor 10 can capture a new image. If it is not necessary to initialize the solid-state image sensor 10, such as when the solid-state image sensor 10 does not have data to be erased, step 31 may not be executed. Further, the step 31 may be executed in the t-1st image frame t-1 or may be executed before the t-1st image frame t-1.

In the first t-1th image frame t-1, step 33 (S33) shown in FIG. 8 is performed. In step 33 (S33), the first PD62 from the first column (M = 1) to the third column (M = 3) of the first row (N = 1) of the luminance group 150 is selected, and the selected first column. The first PD62 from the eyes (M = 1) to the third row (M = 3) is exposed, and each luminance signal generated based on the light exposed by each first PD62 is supplied to the exposure time calculation circuit 240. Let time T1 be the time when step 33 is executed. The time T1 may be rephrased as the time related to the exposure of the first PD62. That is, the solid-state image sensor 10 selects the luminance group 150 line by line, and each first PD62 of each luminance group 150 detects the luminance based on the exposed light. Each of the generated luminance signals is converted into a first digital signal 302 (FIG. 1) by the readout circuit 300 (FIG. 1), that is, a digital luminance signal corresponding to the luminance signal of each luminance group 150. The digital luminance signal is supplied to the exposure time calculation circuit 240 (FIG. 1).

After each luminance signal supplied to the exposure time calculation circuit 240 in the first t-1st image frame t-1 is supplied to the exposure time calculation circuit 240, the third image frame t-1 following the t-1st image frame t-1. By the time the second PD 72 is exposed in the t-th image frame t, steps 35 (S35) to 39 (S39) shown in FIG. 8 may be executed in the solid-state image sensor 10.

In step 35 (S35), the exposure time calculation circuit 240 calculates the exposure time based on the digital luminance signal. The calculation of the exposure time may be executed for each luminance group 150 for each row of the pixel unit 101, or may be executed for each luminance group 150 for a plurality of rows of data. Here, as shown in FIG. 7A, the light exposed by the luminance group 150 of 1 row and 1 column (N = 1, M = 1) is dark and 1 row and 2 columns (N = 1, M = 2). The light exposed by the luminance group 150 of) is halftone, and the light exposed by the luminance group 150 of 1 row and 3 columns (N = 1, M = 3) is bright. Therefore, although the details will be described later, as shown in FIG. 7B, when the second PD72 included in the pixel 100 corresponding to the luminance group 150 of 1 row and 1 column (N = 1, M = 1) is exposed. The exposure time T2 is longer than the exposure time T3 when the second PD 72 included in the pixel 100 corresponding to the luminance group 150 of 1 row and 2 columns (N = 1, M = 2) is exposed. Further, as will be described in detail later, as shown in FIG. 7B, when the second PD72 included in the pixel 100 corresponding to the luminance group 150 of 1 row and 2 columns (N = 1, M = 2) is exposed. The exposure time T3 is longer than the exposure time T4 when the second PD 72 included in the pixel 100 corresponding to the luminance group 150 of 1 row and 3 columns (N = 1, M = 3) is exposed.

In step 37 (S37), the calculated exposure time corresponding to each luminance group 150 is supplied to the storage circuit 230 (FIG. 1) and stored in the storage circuit 230. The calculated exposure time may be supplied to the storage circuit 230 with the exposure time corresponding to each luminance group 150 for each row as one unit, or the calculated exposure time may be supplied to each luminance group 150 of the data of a plurality of rows. The exposure time may be executed as one unit.

In step 39 (S39), the exposure time stored in the storage circuit 230 is read out from the storage circuit 230 and supplied to the second photoelectric conversion element exposure control circuit 220 (FIG. 1). The supply of the calculated exposure time from the storage circuit 230 to the second photoelectric conversion element exposure control circuit 220 may be executed with the exposure time corresponding to each luminance group 150 for each line as one unit, or a plurality of lines. The exposure time corresponding to each luminance group 150 of the data may be executed as one unit.

As described above, in the t-th image frame t shown in FIG. 7B, the subject is exposed by the second photoelectric conversion element 72 based on the exposure time (step 41 (S41) in FIG. 8). The second photoelectric conversion element exposure control circuit 220 sets the read exposure time by controlling the signal processing circuit 52, for example, and the solid-state image sensor 10 can image the subject.

Specifically, as shown in FIG. 7B, the solid-state image sensor 10 selects the pixel 100 in which Y is in the first row. When pixel 100 in the first row of Y is selected, X = 1 and X are based on the exposure time calculated from the luminance signal of the first PD62 of the luminance group of 1 row and 1 column (N = 1, M = 1). The second PD72 included in the = 2 pixel 100 is exposed. Then, each second signal (image data corresponding to the subject) generated by each second PD 72 is supplied to the readout circuit 300. At this time, the exposure time of 1 row and 1 column (N = 1, M = 1) is the exposure time T2. Similarly, the second PD72 included in the pixel 100 of X = 3 and X = 4 based on the exposure time calculated from the luminance signal of the first PD62 of the luminance group of 1 row and 2 columns (N = 1, M = 2). Is exposed. Then, each second signal (image data corresponding to the subject) generated by each second PD 72 is supplied to the readout circuit 300. At this time, the exposure time of 1 row and 2 columns (N = 1, M = 2) is the exposure time T3. Further, in the same manner, the first pixel 100 included in the pixels 100 of X = 5 and X = 6 based on the exposure time calculated from the luminance signal of the first PD62 of the luminance group of 1 row and 3 columns (N = 1, M = 3). 2PD72 is exposed. Then, each second signal (image data corresponding to the subject) generated by each second PD 72 is supplied to the readout circuit 300. At this time, the exposure time of 1 row and 3 columns (N = 1, M = 3) is the exposure time T4. Further, in the second PD72 from X = 1 to X = 6, the generation of the second signal according to the exposure time ends at the same or substantially the same time. By completing the generation of the second signal in the same or substantially the same manner, each second signal can be supplied to the readout circuit 300 at substantially the same timing.

Next, the solid-state image sensor 10 selects the pixel 100 in which Y is in the second row. When the pixel 100 in the second row is selected for Y, the drive and exposure time of each second PD72 are the same as when the pixel 100 in the first row is selected for Y described above. The description of is omitted.

In the t-th image frame t, step 33 for calculating the exposure time corresponding to the step 41 executed in the t + 1th image frame t + 1 following the t-th image frame t is also executed.

As described above, the plurality of second signals are collectively shown by the second signal 114 (FIG. 1). The second signal 114 supplied to the readout circuit 300 is converted into a second digital signal (collectively, a second digital signal 304) for the second signal 114 of each line. The second digital signal 304 is digital image data corresponding to the subject.

The image data captured as described above is image-processed by the image processing circuit 500 and output as an image of the subject. Then, the driving of the solid-state image sensor 10 is completed. In addition, in this specification, in order to promote understanding of the driving method, the driving method of the solid-state image sensor 10 according to the embodiment of the present invention is simply shown, but the example shown here. Not limited to. For example, in the t-th image frame t, the light receiving intensity of the first PD62 may be constant and the luminance signal may be constant, and the luminance signal of the first PD62 changes with time and the luminance signal changes with the change of the luminance signal. May change. Further, in each frame, the light receiving intensity of the first PD62 may be constant and the luminance signal may be constant, or the luminance signal of the first PD62 may change with time and the luminance signal may change as the light receiving intensity changes. Good. In the solid-state image sensor, any drive method may be used as long as the exposure time of each second photoelectric conversion element is controlled by the exposure time based on the luminance signal and an image reflecting the brightness and darkness of the subject can be obtained by one imaging.

As described above, the solid-state image pickup device 10 according to the embodiment of the present invention includes a luminance group having two photoelectric conversion elements, whereby a luminance signal is generated by the first photoelectric conversion element in the first frame. can do. Further, in the second frame, the exposure time of each second photoelectric conversion element is controlled by the exposure time based on the luminance signal, and the solid-state image sensor 10 according to the embodiment of the present invention can change the brightness of the subject by one imaging. The reflected image can be obtained. That is, since the solid-state image sensor 10 according to the embodiment of the present invention can optimize the exposure time for each luminance group, it prevents saturation of the analog signal output in the pixel 100 in which high luminance is detected. be able to. Therefore, the solid-state image sensor 10 according to the embodiment of the present invention can widen the dynamic range by one imaging.

1-4. Driving Method of Solid-State Image Sensor 10 The driving method of the solid-state image sensor 10 according to the embodiment of the present invention will be described with reference to FIGS. 9 to 11.

FIG. 9A is a schematic view showing the brightness and darkness of the pixel 100 in the t-1th image frame t-1 of the solid-state image sensor 10 according to the embodiment of the present invention. FIG. 9B is a schematic view showing the brightness and darkness of the pixel 100 in the t-th image frame t of the solid-state image sensor 10 according to the embodiment of the present invention. FIG. 9C is a schematic view showing the brightness and darkness of the pixel 100 in the t + 1th image frame t + 1 of the solid-state image sensor 10 according to the embodiment of the present invention. FIG. 10 shows a solid according to an embodiment of the present invention in the t-1st image frame t-1 shown in FIG. 9A and the tth image frame t shown in FIG. 9B. It is a timing chart which shows the driving method of the image pickup apparatus 10. FIG. 11 shows the solid-state image sensor 10 according to the embodiment of the present invention in the t-th image frame t shown in FIG. 9 (B) and the t + 1-th image frame t + 1 shown in FIG. 9 (C). It is a timing chart which shows the driving method. In FIGS. 10 and 11, the luminance group is referred to as luminance G.

In the configuration of the solid-state image sensor 10 in FIGS. 9 to 11, the brightness group 150 has 1 row and 3 columns (N =) as compared with the configuration of the solid-state image sensor 10 in the driving method described with reference to FIGS. 7 and 8. It has changed from 1, M = 3) to 2 rows and 4 columns (N = 2, M = 4). Further, in the driving method of FIGS. 9 to 11, the number of image frames has changed from two to three as compared with the driving method described with reference to FIGS. 7 and 8. 9 to 11 are the same as those of FIGS. 7 and 8 except for the number of arrays and the number of frames of the luminance group 150. Therefore, here, the main differences from those of FIGS. 7 and 8 are different. explain. The description of the same or similar configuration as in FIGS. 1 to 8 may be omitted.

As shown as an example in FIGS. 9 (A), 9 (B), and 9 (C), the luminance groups 150 are arranged in a matrix of 2 rows and 4 columns (N = 2, M = 4). .. Further, in the luminance group, 2 × 2 pixels 100 are arranged, and in the pixel portion 101, 4 × 8 pixels 100 are arranged. The second photoelectric conversion element 72 is also arranged in 2 × 2 in the luminance group and 4 × 8 in the pixel portion 101.

The brightness of the pixel 100 shown in FIGS. 9 (A), 9 (B) and 9 (C) is the same as in FIG. 7 (A). The brightness group 150 in which the light to be exposed is in the middle tone is indicated by hatching on the right diagonal line, and the brightness group 150 in which the light to be exposed is bright is shown without hatching.

As shown in FIG. 9A, the luminance group 150 of 1 row 1 column (N = 1, M = 1) and 2 rows 2 columns (N = 2, M = 2) exposes dark light. The brightness group 150 of 1 row and 2 columns (N = 1, M = 2), 1 row and 4 columns (N = 1, M = 4) and 2 rows and 3 columns (N = 2, M = 3) is exposed to light. It is a halftone. The brightness group 150 of 1 row and 3 columns (N = 1, M = 3), 2 rows and 1 column (N = 2, M = 1) and 2 rows and 4 columns (N = 2, M = 4) is exposed to light. bright.

As shown in FIG. 9B, the luminance group 150 of 1 row and 3 columns (N = 1, M = 3) exposes dark light. The brightness group 150 of 1 row and 2 columns (N = 1, M = 2), 2 rows and 1 column (N = 2, M = 1), and 2 rows and 3 columns (N = 2, M = 3) is the light to be exposed. Is in the middle tone. 1 row 1 column (N = 1, M = 1), 1 row 4 columns (N = 1, M = 4), 2 rows and 2 columns (N = 2, M = 2) and 2 rows and 4 columns (N = 2) In the brightness group 150 of M = 4), the light to be exposed is bright.

As shown in FIG. 9C, the luminance group 150 of 1 row and 3 columns (N = 1, M = 3) and 1 row and 4 columns (N = 1, M = 4) exposes dark light. The brightness group 150 of 1 row and 1 column (N = 1, M = 1), 1 row and 2 columns (N = 1, M = 2), and 2 rows and 1 column (N = 2, M = 1) is the light to be exposed. Is in the middle tone. The brightness group 150 of 2 rows and 2 columns (N = 2, M = 2), 2 rows and 3 columns (N = 2, M = 3) and 2 rows and 4 columns (N = 2, M = 4) is exposed to light. bright.

For example, focusing on the brightness group 150 of 1 row and 4 columns (N = 1, M = 4), the brightness of 1 row and 4 columns (N = 1, M = 4) in the t-1st image frame t-1. In group 150, the light to be exposed is halftone. In the t-th image frame t, the light to be exposed is bright in the luminance group 150 of 1 row and 4 columns (N = 1, M = 4). In the t + 1th image frame t + 1, the luminance group 150 of 1 row and 4 columns (N = 1, M = 4) exposes dark light. Therefore, when the solid-state image sensor 10 changes from the t-1st image frame t-1 to the tth image frame t, the light exposed by the luminance group 150 changes from halftone light to bright light. There is. Further, when the solid-state image sensor 10 changes from the t-th image frame t to the t + 1th image frame t + 1, the light exposed by the luminance group 150 changes from bright light to dark light. Brightness groups 150 other than 1 row and 4 columns (N = 1, M = 4) are also exposed to light for each image frame in the same manner as 1 row and 4 columns (N = 1, M = 4) luminance group 150. There are things that change and things that do not change. The solid-state image sensor 10 according to the embodiment of the present invention can generate a luminance signal of each luminance group 150 in each image frame by including the first PD 62 in the luminance group 150.

Here, focusing on the luminance group 150 of 1 row and 4 columns (N = 1, M = 4), the 1 row and 4 columns (N = 1, M = 4) described above using FIGS. 9 (A) to 9 (C) are used. The driving of each image frame of the luminance group 150 of M = 4) will be described with reference to FIGS. 10 and 11.

As shown in FIG. 10, in the t-1th image frame t-1, in the t-2nd image frame t-2 (not shown), the first row (N = 1) of the luminance group 150. ), A luminance signal is generated based on the light exposed by the first PD62 in the fourth row (M = 4). Next, the exposure time calculated based on the generated luminance signal is included in the pixels 100 of X = 7 and X = 8 included in the luminance group 150 of 1 row and 4 columns (N = 1, M = 4). 2PD72 is exposed. At this time, the light exposed by the luminance group 150 in 1 row and 4 columns (N = 1, M = 4) is halftone light (FIG. 9 (A)). As shown in FIG. 9A, the light exposed by the second PD72 in 1 row and 4 columns (N = 1, M = 4) is the second PD72 in 1 row and 2 columns (N = 1, M = 2). Similarly, it is a halftone light, and the light exposed by the second PD72 in 1 row and 3 columns (N = 1, M = 3) is bright light, and the light in 1 row and 1 column (N = 1, M = 2) is the first. The light exposed by the 2PD72 is dark light. Therefore, as shown in FIG. 10, the exposure time of the second PD72 of X = 7 and X = 8 in 1 row and 4 columns (N = 1, M = 4) is 1 row and 2 columns (N = 1, M = 4). It is the same as the exposure time of the second PD72 of X = 3 and X = 4 in 2), and is larger than the exposure time of the second PD72 of X = 5 and X = 6 in 1 row and 3 columns (N = 1, M = 3). It is long and shorter than the exposure time of the second PD72 of X = 1 and X = 2 in 1 row and 1 column (N = 1, M = 1).

As shown in FIG. 10, in the t-1st image frame t-1, the first PD62 in the fourth column (M = 4) included in the first row (N = 1) of the luminance group 150 is exposed. ing. The end of the time related to the exposure of the first PD62 is later than the end of the t-1th image frame t-1. As a result, even after the image processing of the image captured in the t-1st image frame t-1 is completed, the fourth column (M =) included in the first row (N = 1) of the luminance group 150. The first PD62 of 4) can be exposed. Therefore, the image of the image captured in the t-th image frame t based on the light exposed by the first PD62 in the fourth column (M = 4) included in the first row (N = 1) of the luminance group 150. Processing can be performed.

Subsequently, as shown in FIGS. 10 and 11, in the t-th image frame t, the t-1th image frame t-1 is included in the first row (N = 1) of the luminance group 150. A luminance signal is generated based on the light exposed by the first PD62 in the fourth row (M = 4). Next, the exposure time calculated based on the generated luminance signal is included in the pixels 100 of X = 7 and X = 8 included in the luminance group 150 of 1 row and 4 columns (N = 1, M = 4). 2PD72 is exposed. At this time, the light exposed by the luminance group 150 in 1 row and 4 columns (N = 1, M = 4) is bright light (FIG. 9 (B)). As shown in FIG. 9B, the light exposed by the second PD72 in 1 row and 4 columns (N = 1, M = 4) is the second PD72 in 1 row and 1 column (N = 1, M = 2). Similarly, it is bright light, and the light exposed by the second PD72 in 1 row and 2 columns (N = 1, M = 2) is halftone light, which is the 1st row and 3 columns (N = 1, M = 3). The light exposed by the 2PD72 is dark light. Therefore, as shown in FIG. 10, the exposure time of the second PD72 of X = 7 and X = 8 in 1 row and 4 columns (N = 1, M = 4) is 1 row and 1 column (N = 1, M = 4). It is the same as the exposure time of the second PD72 of X = 1 and X = 2 in 1), and the exposure time of the second PD72 of X = 3 and X = 4 in 1 row and 2 columns (N = 1, M = 2), and It is shorter than the exposure time of the second PD72 of X = 5 and X = 6 in 1 row and 3 columns (N = 1, M = 3).

As shown in FIGS. 10 and 11, in the t-th image frame t, the first PD62 in the fourth column (M = 4) included in the first row (N = 1) of the luminance group 150 is exposed. There is. The end of the time associated with the exposure of the first PD 62 is later than the end of the t-th image frame t. As a result, even after the image processing of the image captured in the t-th image frame t is completed, the fourth column (M = 4) included in the first row (N = 1) of the luminance group 150 1PD62 can be exposed. Therefore, the image of the image captured in the t + 1th image frame t + 1 based on the light exposed by the first PD62 in the fourth column (M = 4) included in the first row (N = 1) of the luminance group 150. Processing can be performed.

Subsequently, as shown in FIG. 11, in the t + 1th image frame t + 1, the fourth column (M =) included in the first row (N = 1) of the luminance group 150 in the tth image frame t. A luminance signal is generated based on the light exposed by the first PD62 in 4). Next, the exposure time calculated based on the generated luminance signal is included in the pixels 100 of X = 7 and X = 8 included in the luminance group 150 of 1 row and 4 columns (N = 1, M = 4). 2PD72 is exposed. At this time, the light exposed by the luminance group 150 in 1 row and 4 columns (N = 1, M = 4) is dark light (FIG. 9 (C)). As shown in FIG. 9C, the light exposed by the second PD72 in 1 row and 4 columns (N = 1, M = 4) is the second PD72 in 1 row and 3 columns (N = 1, M = 3). Similarly, it is dark light, and the light exposed by the second PD72 of 1 row and 1 column (N = 1, M = 1) and the 2nd PD72 of 1 row and 2 columns (N = 1, M = 2) is halftone light. is there. Therefore, as shown in FIG. 11, the exposure time of the second PD72 in 1 row and 4 columns (N = 1, M = 4) is the exposure time of the 2nd PD72 in 1 row and 3 columns (N = 1, M = 3). X = 1 in 1 row and 1 column (N = 1, M = 1) and the exposure time of the second PD72 in X = 2 and X = in 1 row and 2 columns (N = 1, M = 2). It is longer than the exposure time of the second PD72 of 3 and X = 4.

As shown in FIG. 11, in the t + 1th image frame t + 1, the first PD62 in the fourth column (M = 4) included in the first row (N = 1) of the luminance group 150 is exposed. The end of the time associated with the exposure of the first PD62 is later than the end of the t + 1th image frame t + 1. As a result, even after the image processing of the image captured in the t + 1th image frame t + 1 is completed, the fourth column (M = 4) included in the first row (N = 1) of the luminance group 150 1PD62 can be exposed. Therefore, based on the light exposed by the first PD62 in the fourth column (M = 4) included in the first row (N = 1) of the luminance group 150, the image is captured in the t + second image frame t + 2 (not shown). Image processing of the processed image can be performed.

In the description of the driving method of the solid-state image sensor 10 according to the embodiment of the present invention, the luminance group 150 of 1 row and 4 columns (N = 1, M = 4) has been focused on, but 1 row and 4 columns (1 row and 4 columns) Regarding the driving method of the luminance group 150 other than the luminance group 150 of N = 1, M = 4), 1 row and 4 columns (N = 1, M) except that the exposure time is optimized based on the generated luminance signal. Since it is the same as the driving method of = 4), the description here will be omitted. Further, since the driving method after the second PD72 generates the luminance signal is the same as the driving method described in "1-3. Driving method of the solid-state image sensor 10", the description thereof is omitted here.

As described above, the solid-state image sensor according to the embodiment of the present invention has a structure in which two photoelectric conversion elements are laminated, and a luminance signal can be generated by the first photoelectric conversion element. Further, since the solid-state image sensor according to the embodiment of the present invention can calculate the exposure time based on the luminance signal and control the exposure time of each second photoelectric conversion element by the calculated exposure time. An image that reflects the brightness and darkness of the subject can be obtained with a single image capture. Therefore, by using the solid-state image sensor according to the embodiment of the present invention, it is not necessary to synthesize the image data acquired by a plurality of imagings as in the prior art, and the image data acquired by the plurality of imagings is stored. Since there is no need for the existing frame memory, it is possible to provide a solid-state image sensor having a wide dynamic range with little decrease in the frame rate.

Although the present invention has been described above with reference to the drawings, the present invention is not limited to the above-described embodiment, and can be appropriately modified without departing from the spirit of the present invention. For example, a device in which a person skilled in the art appropriately adds, deletes, or changes the design based on the solid-state image sensor of each embodiment is also included in the scope of the present invention as long as it has the gist of the present invention. .. Further, the above-described embodiments can be appropriately combined as long as there is no contradiction with each other, and technical matters common to the respective embodiments are included in the respective embodiments even if there is no explicit description.

Further, even if the action / effect is different from the action / effect brought about by the embodiment of each of the above-described embodiments, those that are clear from the description of the present specification or those that can be easily predicted by those skilled in the art are referred to. Naturally, it is understood that it is brought about by the present invention.

10: Solid-state image sensor, 50: Signal processing unit, 52: Signal processing circuit, 60: First photoelectric conversion element unit, 62: First photoelectric conversion element, 70: Second photoelectric conversion element unit, 72: Second photoelectric conversion Element, 80: color filter section, 82: red color filter, 84: green color filter, 86: blue color filter, 90: optical element section, 92: optical element, 100: pixel, 101: pixel section, 103: peripheral section , 112: 1st signal, 114: 2nd signal, 116: incident light, 150: brightness group, 200: row selection scanning circuit, 210: image pickup timing control circuit, 212: wiring, 214: control signal, 220: second Photoelectric conversion element exposure control circuit, 224: brightness group exposure signal, 230: storage circuit, 232: wiring, 240: exposure time calculation circuit, 242: wiring, 300: readout circuit, 302: first digital signal, 304: second Digital signal, 500: Image processing circuit

Claims (9)

A pixel portion having Y × X pixels (Y and X are natural numbers set independently of each other) arranged in the first direction and the second direction intersecting the first direction, and
Exposure control circuit and
It has an exposure time calculation circuit and
The pixel part is
In the Y × X pixels, the first photoelectric conversion element provided corresponding to the y × x pixels (y and x are natural numbers set independently of each other) and
It has a second photoelectric conversion element provided for each of the Y × X pixels.
The second photoelectric conversion element overlaps with the first photoelectric conversion element.
The first photoelectric conversion element supplies a luminance signal corresponding to the light receiving intensity to the exposure time calculation circuit.
The exposure time calculation circuit calculates the exposure time of the second photoelectric conversion element based on the luminance signal.
The exposure control circuit controls the exposure of the second photoelectric conversion element of each of the y × x pixels in units of the y × x pixels.
Solid-state image sensor. It has a storage unit for storing the exposure time data.
The solid-state image sensor according to claim 1. The second photoelectric conversion element and the second photoelectric conversion element adjacent to the second photoelectric conversion element are separated from each other, and the first photoelectric conversion element is provided so that light can be exposed.
The solid-state image sensor according to claim 1. The solid-state imaging device according to claim 1, wherein the area of the first photoelectric conversion element is larger than the area of the second photoelectric conversion element. A pixel portion having Y × X pixels (Y and X are natural numbers set independently of each other) arranged in a first direction and a second direction intersecting the first direction, and y × x pixels. The first photoelectric conversion element provided corresponding to the pixel (y and x are natural numbers set independently of each other) and the y × x pixels overlapping the first photoelectric conversion element are provided. A method for driving a solid-state image sensor, which includes a second photoelectric conversion element, an exposure time calculation circuit, an exposure control circuit, and a signal processing circuit.
The first photoelectric conversion element generates a luminance signal according to the light receiving intensity, and the luminance signal is generated.
The exposure time calculation circuit calculates the exposure time of the second photoelectric conversion element according to the luminance signal.
The exposure control circuit controls the exposure of the second photoelectric conversion element based on the exposure time.
The signal processing circuit generates a signal corresponding to the exposure time of the second photoelectric conversion element provided in each of the y × x pixels.
How to drive a solid-state image sensor. The solid-state image sensor further includes a storage unit.
The exposure time data is stored in the storage unit,
The exposure control circuit reads out the exposure time data from the storage unit.
The method for driving a solid-state image sensor according to claim 5. The output of the luminance signal is executed in the first frame,
The generation of the signal according to the exposure time of the second photoelectric conversion element is executed in the second frame after the first frame.
The method for driving a solid-state image sensor according to claim 5. The time for the plurality of first photoelectric conversion elements arranged in the first direction to output a luminance signal, and the plurality of arrangements adjacent to the plurality of first photoelectric conversion elements in parallel with the first direction. It is different from the time when the first photoelectric conversion element outputs the luminance signal.
The method for driving a solid-state image sensor according to claim 5. The generation of signals according to the exposure time of the plurality of second photoelectric conversion elements arranged in the first direction ends at the same time.
The method for driving a solid-state image sensor according to claim 5.

Images