JP6579614B2 - Imaging device, imaging device - Google Patents

Description

  The present invention relates to an imaging device and an imaging apparatus in which phase difference pixels are discretely arranged among a plurality of pixels arranged in a matrix.

  Conventionally, imaging in which phase difference pixels are discretely arranged among a plurality of pixels arranged in a matrix so that electric charges output from the phase difference pixels can be used as information for phase difference detection The element has been put into practical use and commercialized.

  The phase difference pixels are arranged at a ratio within a predetermined arrangement range on the imaging surface of the imaging element, and are arranged at a ratio of one pixel to eight pixels, for example. And the predetermined arrangement range of the phase difference pixels tends to be expanded not only to a specific part in the imaging surface but also to almost the entire imaging surface in recent years.

  Here, the phase difference pixel is a pixel that photoelectrically converts only the light that has passed through a part of the pupil of the lens. Specifically, a part of the pixel opening of the normal pixel has a light-shielding structure so that the phase difference pixel can be changed from a specific direction. It is configured to generate charges that provide phase difference information by blocking incident light rays and photoelectrically converting only incident light rays from other specific directions.

  Accordingly, since the amount of charge generated by the phase difference pixel is in principle smaller than the amount of charge generated by the normal pixel, the S / N of the signal charge generated by the phase difference pixel is generated by the normal pixel. It becomes lower than the S / N of the signal charge.

  Incidentally, some image sensors operate in a mixed readout mode in which charges generated by a plurality of photoelectric conversion pixels are mixed and then read out. In such a mixed readout mode, for example, for moving image shooting, live view shooting, or contrast AF, which usually requires fewer pixels than a still image but needs to be read at a relatively fast constant frame rate (60 fps or the like). Used.

  When an operation of the mixed readout mode is performed in an image sensor including the phase difference pixel as described above, as a method of handling the phase difference pixel, a method for excluding it from pixel data for an image and an image There is a method of reading out pixel data mixed with normal pixels.

  The method of excluding these phase difference pixels from the target of the mix readout not only causes the processing circuit to be complicated and increases the power consumption, but also thins out the spatial information of the phase difference pixel position. Image quality deterioration such as generation of moire due to the phase difference pattern, coloring, and line shrinkage is caused. Such influence can be ignored when the ratio of the phase difference pixels in the whole pixel is extremely low, but it cannot be ignored for the recent image pickup device in which the ratio of the phase difference pixels is increasing.

  Therefore, when performing the mixed read mode operation, the charges generated by the phase difference pixels are also mixed and read out, and the image quality of the mix pixel mixed with the phase difference pixels is deteriorated (sensitivity is decreased, the subject is out of focus). For example, image quality deterioration due to phase difference pixels such as a jagged geometric pattern) generated on the top is corrected by digital processing in the subsequent stage.

  As a specific example, Japanese Patent Laying-Open No. 2015-23312 describes an imaging device that mixes and reads out charges generated by phase difference pixels.

Japanese Patent Laying-Open No. 2015-23312

  As a method corresponding to the point that the charge amount of the phase difference pixel becomes smaller than the charge amount of the normal pixel as described above, for example, the electronic shutter for the normal pixel and the electronic shutter for the phase difference pixel are controlled independently. It is conceivable that the exposure time of the phase difference pixel and the exposure time of the normal pixel are made different so that the optimum charge accumulation time is obtained for each pixel.

  However, if there is a moving subject within the shooting range, the amount of movement (blurring amount) of the subject differs between the phase difference pixels and the normal pixels that have different exposure times. Image data of an unnatural blur image will be obtained. Therefore, when such image data is mixed and read out so as to include phase difference pixels, whether or not the phase difference pixels are included in the mix pixels, and how many pixels are included in one mix pixel when included. An image reflecting a pattern indicating whether or not a phase difference pixel is included is output from the image sensor.

  The present invention has been made in view of the above circumstances, and it is possible to obtain a charge with an appropriate exposure amount from a normal pixel and obtain a charge with a preferable exposure amount from a phase difference pixel by performing phase difference detection, and further move An object of the present invention is to provide an imaging device and an imaging apparatus capable of obtaining an image that does not cause unnatural blur even when a subject is present.

An imaging device according to an aspect of the present invention includes an imaging device having a pixel group in which a plurality of pixels are arranged in a matrix, a plurality of phase difference pixels discretely arranged in the pixel group, and the pixel group A plurality of normal pixels that are pixels other than the phase difference pixels, and a normal exposure control unit that controls a timing at which the normal pixel group including the plurality of normal pixels starts and ends charge accumulation; wherein the plurality of the timing of the phase difference pixel group composed of a phase difference pixel has finished timing and charge accumulation starts charge accumulation, and a phase difference exposure control unit for controlling independently of the normal exposure control unit, wherein The phase difference exposure control unit sets a phase difference exposure time from the start of charge accumulation of the phase difference pixel group to the end thereof, and a normal exposure time from the start of charge accumulation of the normal pixel group to the end thereof. The first charge accumulated in the first exposure time and the second charge accumulated in the second exposure time are divided into the same first exposure time and the second exposure time. And are controlled to be read separately .

  An image pickup apparatus according to an aspect of the present invention includes the image pickup device, an AF processing unit that performs phase difference AF based on a signal of the phase difference pixel output from the image pickup device, and the normal pixel output from the image pickup device. Or an image processing unit that generates a recording image based on a signal obtained by mixing the normal pixel signal and the phase difference pixel signal.

  According to the imaging device and the imaging apparatus of the present invention, it is possible to obtain a charge with an appropriate exposure amount from a normal pixel, obtain a charge with a preferable exposure amount from the phase difference pixel by performing phase difference detection, and a moving subject. Even if it exists, an image can be obtained in which no unnatural blur occurs.

1 is a block diagram illustrating a configuration of an imaging apparatus according to Embodiment 1 of the present invention. FIG. 2 is a block diagram illustrating a configuration of an image sensor in the first embodiment. FIG. 3 is a diagram illustrating an example of an arrangement of normal pixels and a plurality of phase difference pixels in the pixel portion of the first embodiment. FIG. 3 is a diagram illustrating an outline of a circuit configuration related to a normal pixel and a phase difference pixel in the first embodiment. 4 is a timing chart showing the operation of each unit when a global shutter imaging operation is performed in the imaging device of the first embodiment. The block diagram which shows the structure of the image pick-up element in the said Embodiment 2 of this invention. In the said Embodiment 2, the figure which shows the outline | summary of the circuit structure relevant to a normal pixel and a phase difference pixel. 9 is a timing chart showing the operation of each unit when a rolling shutter imaging operation is performed in the imaging device of the second embodiment.

Embodiments of the present invention will be described below with reference to the drawings.
[Embodiment 1]

  1 to 5 show Embodiment 1 of the present invention, and FIG. 1 is a block diagram showing a configuration of an imaging apparatus.

  As shown in FIG. 1, the imaging device includes a lens 1, an imaging device 2, an image processing unit 3, an AF (autofocus) evaluation value calculation unit 4, a display unit 5, and a hand shake detection unit 7. A camera shake correction unit 8, an exposure control unit 9, a focus control unit 10, a camera operation unit 11, and a camera control unit 12. Although the memory card 6 is also illustrated in FIG. 1, the memory card 6 is configured to be detachable from the imaging apparatus, and thus may not have a configuration unique to the imaging apparatus.

  The lens 1 is an imaging optical system that forms an optical image of a subject on an imaging area of the imaging device 2. The lens 1 includes a focus lens for adjusting the focus position (focus position) and focusing, and a diaphragm for controlling the range of the light beam that passes therethrough. It also has a function.

  The imaging device 2 photoelectrically converts the optical image of the subject imaged by the lens 1 and outputs it as an image signal. In the present embodiment, the image pickup device 2 is described as a color image pickup device having a primary color Bayer array color filter, but other configurations may be used. Then, as will be described later with reference to FIG. 3, the imaging device 2 of the present embodiment includes discrete phase difference pixels 25 a (see FIG. 3) in the normal pixels 24 a (see FIG. 3) in the primary color Bayer array. It is arranged and configured. Further, the image sensor 2 is configured to be movable in a plane perpendicular to the photographing optical axis of the lens 1 and has a hand shake correction function.

  The image processing unit 3 performs various types of image processing on the image signal output from the image sensor 2. The image processing unit 3 generates a recording image based on the signal of the normal pixel 24a output from the image sensor 2 (at least using the signal of the normal pixel 24a or further using the signal of the phase difference pixel 25a). Perform image processing. Further, for example, when the normal pixel 24a and the phase difference pixel 25a are mixed and read, the image processing unit 3 amplifies the pixel signal decrease due to the pixel opening of the phase difference pixel 25a being smaller than the normal pixel 24a. , And correction of pixel signal variation (geometric pattern or the like) caused by the phase difference pixel 25a at the out-of-focus position.

  The AF evaluation value calculation unit 4 calculates an AF evaluation value based on the image signal output from the image sensor 2 and outputs the AF evaluation value to the camera control unit 12. Specifically, the AF evaluation value calculation unit 4 calculates a phase difference based on a signal read from the phase difference pixel 25a of the image sensor 2 and outputs it as an AF evaluation value. The AF evaluation value calculation unit 4 may further calculate a contrast value based on the image signal output from the image sensor 2 and output it as an AF evaluation value (that is, the phase difference AF). In addition, contrast AF may be performed.

  The display unit 5 displays an image based on a signal image-processed for display by the image processing unit 3. The display unit 5 performs live view display, still image display, operation reproduction display, and the like, and also displays various information related to the imaging apparatus.

  The memory card 6 is a recording medium for storing a signal (still image signal, moving image signal, etc.) subjected to image processing for recording by the image processing unit 3.

  The hand shake detection unit 7 includes an acceleration sensor, an angular velocity sensor, and the like, and detects hand shake of the imaging apparatus and outputs it to the camera control unit 12.

  Based on the control of the camera control unit 12, the camera shake correction unit 8 moves at least one of the lens 1 and the imaging device 2 so as to cancel the detected camera shake, and forms an image on the imaging region of the imaging device 2. This reduces the influence of camera shake on the optical subject image.

  Based on the shutter speed (exposure time) determined by the camera control unit 12, the exposure control unit 9 controls the element shutter of the image sensor 2 (the global shutter is used as the element shutter) under the control of the camera control unit 12. And a rolling shutter are included) to acquire an image. Furthermore, the exposure control unit 9 also controls the aperture included in the lens 1 based on the aperture value determined by the camera control unit 12. Further, the exposure control unit 9 outputs driving information of the image sensor 2 to the camera control unit 12.

  The focus control unit 10 drives the lens 1 to adjust the focus. That is, the focus control unit 10 drives the focus lens included in the lens 1 based on the control of the camera control unit 12 that has received the AF evaluation value from the AF evaluation value calculation unit 4 and forms an image on the image sensor 2. Make the subject image come into focus. The focus control unit 10 outputs lens drive information such as a lens position to the camera control unit 12.

  The AF evaluation value calculation unit 4, the focus control unit 10, and the camera control unit 12 described above constitute an AF processing unit that performs phase difference AF based on the signal of the phase difference pixel 25 a output from the image sensor 2.

  The camera operation unit 11 is an operation unit for performing various operation inputs to the imaging apparatus. The camera operation unit 11 includes a power switch for turning on / off the power of the imaging apparatus, a release button for inputting instructions for still image shooting, moving image shooting, and the like, a still image shooting mode, a moving image shooting mode, and a live view mode. An operation member such as a mode button for setting and the like is included.

  The camera control unit 12 includes lens drive information from the focus control unit 10, AF evaluation value from the AF evaluation value calculation unit 4, drive information from the exposure control unit 9, processing information from the image processing unit 3, and a hand shake detection unit. This image pickup apparatus includes an image processing unit 3, a memory card 6, a hand shake correction unit 8, an exposure control unit 9, a focus control unit 10 and the like based on hand shake information from 7, an operation input from the camera operation unit 11, and the like. It controls the whole.

  Then, the camera control unit 12 determines the shutter speed (exposure time) based on the automatic exposure control or based on the setting value input from the camera operation unit 11, and provides information on the determined shutter speed to the exposure control unit 9. Output to.

  FIG. 2 is a block diagram showing the configuration of the image sensor 2.

  The image sensor 2 includes a pixel exposure control unit 20, a pixel unit 23, a memory unit 26, a mix processing unit 30, an addition unit 31, and an analog / digital converter (ADC) 32.

  First, the pixel unit 23 includes a pixel group in which a plurality of pixels are arranged in a matrix, and the pixel group includes a normal pixel group 24 including a plurality of normal pixels 24a and a plurality of phase difference pixels 25a. And a phase difference pixel group 25.

  FIG. 3 is a diagram showing an example of the arrangement of the normal pixels 24a and the plurality of phase difference pixels 25a in the pixel unit 23.

  The plurality of phase difference pixels 25 a are discretely arranged in a pixel group arranged in a matrix in the pixel unit 23. Here, the phase difference pixel 25a is a pixel that photoelectrically converts only the light that has passed through a part of the pupil of the lens 1, and more specifically, a part of the pixel opening of the normal pixel 24a has a light shielding structure. It is configured to generate a charge that provides phase difference information by blocking light rays incident from a specific direction and photoelectrically converting only light rays incident from another specific direction. In the example shown in FIG. 3, the phase difference pixels 25 a are arranged over almost the entire surface of the pixel portion 23 at a ratio of 1 pixel per 4 pixels in the horizontal direction and 1 pixel per 2 pixels in the vertical direction. .

  The plurality of normal pixels 24 a are pixels other than the phase difference pixels 25 a in the pixel group arranged in a matrix in the pixel unit 23. In the example shown in FIG. 3, only the normal pixels 24 a and the phase difference pixels 25 a are arranged in the pixel unit 23, but pixels having other functions may be further arranged.

  The pixel exposure control unit 20 includes a normal exposure control unit 21 that controls the normal pixel group 24 and a phase difference exposure control unit 22 that controls the phase difference pixel group 25.

  The normal exposure control unit 21 is connected to the normal pixel group 24 via the control line 21a illustrated in FIG. 4, and the normal pixel group 24 including a plurality of normal pixels 24a starts charge accumulation and charge accumulation. Controls when to end.

  The phase difference exposure control unit 22 is connected to the phase difference pixel group 25 via a control line 22a provided independently of the control line 21a described in FIG. 4 and includes a plurality of phase difference pixels 25a. The timing at which the phase difference pixel group 25 starts charge accumulation and the timing at which charge accumulation ends is controlled independently of the normal exposure control unit 21.

  The memory unit 26 includes a normal memory unit 27, a first phase difference memory unit 28, and a second phase difference memory unit 29.

  The normal memory unit 27 holds the charges accumulated in the normal pixel group 24.

  The first phase difference memory unit 28 holds the charges accumulated in the phase difference pixel group 25 during the first exposure time T1 shown in FIG.

  The second phase difference memory unit 29 holds the charges accumulated in the phase difference pixel group 25 during the second exposure time T2 shown in FIG.

  The mix processing unit 30 mixes signals read from a plurality of pixels. Here, the mixing process is performed by, for example, adding (or weighting addition) or averaging (or weighting) a plurality of adjacent pixel signals of the same color (for example, 2 × 2 pixels or 3 × 3 pixels of the same color). Average). When the signal read from the phase difference pixel 25a is mixed, the mix processing unit 30 uses a signal related to the first charge accumulated during the first exposure time T1.

  The adder 31 adds the first charge accumulated during the first exposure time T1 and the second charge accumulated during the second exposure time T2.

  The ADC 32 converts the signal output from the mix processing unit 30 and the signal output from the addition unit 31 from an analog signal to a digital signal. For example, the ADC 32 is a column parallel AD converter (column parallel ADC). It is configured. However, the ADC 32 is not limited to the column parallel type ADC, and may have another configuration as long as the processing in units of lines can be completed in a desired time.

  In the configuration example shown in FIG. 2, the mix processing unit 30 and the addition unit 31 are provided in the previous stage of the ADC 32 (therefore, the mix processing unit 30 and the addition unit 31 are analog processing units). However, the configuration is limited to this configuration. For example, the mix processing unit 30 and the addition unit 31 may be provided after the ADC 32 (in this case, the mix processing unit 30 and the addition unit 31 are digital processing units).

  Next, FIG. 4 is a diagram showing an outline of a circuit configuration related to the normal pixel 24a and the phase difference pixel 25a. The circuit configuration related to the normal pixel 24a is shown on the left side of FIG. 4, and the circuit configuration related to the phase difference pixel 25a is shown on the right side of FIG.

  Each of the normal pixel 24a and the phase difference pixel 25a includes a photodiode PD that receives light and performs photoelectric conversion to generate charges.

  The photodiode PD of the normal pixel 24a is connected to the memory M0 provided in the normal memory unit 27 via the first switch SW1, and is connected to the voltage source VDD via the first switch SW1 and the second switch SW2. Yes. The first switch SW1 is controlled by a control line 21a from the normal exposure control unit 21, and the second switch SW2 is similarly controlled by a control line (not shown) from the normal exposure control unit 21.

  The photodiode PD of the phase difference pixel 25a is provided in the memory M1 and the second phase difference memory unit 29 provided in the first phase difference memory unit 28 via the first switch SW1 and the third switch SW3. It is connected to the memory M2. The photodiode PD of the phase difference pixel 25a is connected to the voltage source VDD through the first switch SW1 and the second switch SW2. The first switch SW1 related to the phase difference pixel 25a is controlled by a control line 22a from the phase difference exposure control unit 22, and the second switch SW2 is similarly controlled by a control line (not shown) from the phase difference exposure control unit 22. It is like that. The third switch SW3 is also controlled by a control line (not shown) from the phase difference exposure control unit 22, and connects the photodiode PD to the memory M1 and the memory M2 via the first switch SW1 so as to be switchable.

  In such a configuration, the photodiode PD is reset by turning on both the first switch SW1 and the second switch SW2. From this reset state, by turning off both the first switch SW1 and the second switch SW2, charge accumulation in the photodiode PD is started. Also, the memories M0, M1, and M2 are connected to the voltage source VDD through the control of the switches SW2 and SW3 and are reset.

  In addition, with respect to the normal pixel 24a, when only the first switch SW1 is turned on while the second switch SW2 is turned off, the charges accumulated in the photodiode PD of the normal pixel 24a are transferred to the memory M0.

  Further, regarding the phase difference pixel 25a, if the first switch SW1 is turned on while the second switch SW2 is turned off and the third switch SW3 is switched to the memory M1 side, the phase difference pixel 25a is accumulated in the photodiode PD of the phase difference pixel 25a. The charged charges are transferred to the memory M1. Also, if the first switch SW1 is turned on while the second switch SW2 is turned off, and the third switch SW3 is switched to the memory M2 side, the charge accumulated in the photodiode PD of the phase difference pixel 25a is stored in the memory M2. Forwarded to

  Next, FIG. 5 is a timing chart showing the operation of each unit when a global shutter imaging operation is performed in the imaging device 2.

  As described above, the normal pixel 24a and the phase difference pixel 25a are independently controlled via the independent control lines 21a and 22a. Using such a configuration, the pixel exposure control unit 20 performs control as shown in FIG.

  That is, all the normal pixels 24a are exposed at the same time by completing the reset RST of the photodiodes PD at the same time when the exposure time Texp determined by the camera control unit 12 is advanced from the rising timing of the vertical synchronization signal VD. To start.

  All the normal pixels 24a simultaneously complete the exposure by simultaneously transferring the charges MT stored in the photodiode PD to the normal memory unit 27 at the rising timing of the vertical synchronization signal VD.

  On the other hand, the pixel exposure control unit 20 determines the phase difference exposure time T from the start of the charge accumulation of the phase difference pixel group 25 to the end, and the normal exposure from the start of the normal pixel group 24 to the end of charge accumulation. It is set to be longer than the time (that is, the exposure time Texp described above).

  Then, the phase difference exposure control unit 22 determines the phase difference exposure time T from the start of the charge accumulation of the phase difference pixel group 25 to the end, and the time from the start of the normal pixel group 24 to the end of charge accumulation. The first charge accumulated in the first exposure time T1 and the second exposure time T2 are divided into a first exposure time T1 and a second exposure time T2 that are the same as the normal exposure time Texp. Control is performed so that the accumulated second charge is read separately.

  In other words, the phase difference exposure control unit 22 detects all the phase differences at a timing that is back by the second exposure time T2 from the timing at which the normal exposure control unit 21 starts charge accumulation (the reset RST end timing of the normal pixel 24a). By ending reset RST of the photodiodes PD of the pixels 25a all at once, exposure (charge accumulation) of all the phase difference pixels 25a is started simultaneously.

  Thereafter, the phase difference exposure control unit 22 performs the second exposure at the time when the second exposure time T2 has elapsed from the start of charge accumulation (that is, at the same time when the normal pixel group 24 starts exposure all at once). At time T2, transfer MT2 of the second charge accumulated in the photodiode PD to the second phase difference memory unit 29 (that is, reading of the phase difference pixel 25a) is performed simultaneously. The time point at which this second charge is read out is the time point at which the subsequent first charge accumulation starts.

  Then, the phase difference exposure control unit 22 performs the first exposure at the time when the first exposure time T1 has elapsed from the read timing of the second charge (that is, at the same time when the normal pixel group 24 finishes exposure all at once). During the exposure time T1, the transfer MT1 of the first charges accumulated in the photodiode PD to the first phase difference memory unit 28 (that is, reading of the phase difference pixels 25a) is performed simultaneously. The time point at which this first charge is read out is the time point when all the phase difference pixels 25a finish exposure at the same time.

  After each charge is held in the memory unit 26, the reading RD of the charge from the memory unit 26 is sequentially performed according to the AD conversion for each line of the ADC 32, and the data output from the image sensor 2 is performed.

  At this time, the mix processing unit 30 is held in the first phase difference memory unit 28 and the signal held in the normal memory unit 27 when performing the mix processing for, for example, a live view image or a moving image. Are mixed according to the mix read pattern. Thus, the mix processing unit 30 does not use the signal held in the second phase difference memory unit 28 for the mix processing.

  Further, the adding unit 31 adds the first charge accumulated during the first exposure time T1 and the second charge accumulated during the second exposure time T2 for each same phase difference pixel position. Thus, the AF evaluation value calculation unit 4 generates a signal for performing the phase difference AF. Accordingly, it is possible to obtain a phase difference signal having an appropriate exposure amount (or an exposure amount closer to the appropriate amount) while compensating for the sensitivity reduction of the phase difference pixel 25a having a pixel opening smaller than that of the normal pixel 24a.

  In the example shown in FIG. 5 described above, the second exposure time T2 is set to precede the first exposure time T1, but conversely, the second exposure time T2 is set after the first exposure time T1. The exposure time T2 may be set.

  In this case, the phase difference exposure control unit 22 sets the first exposure time T1 from the start of charge accumulation to the same timing as the timing at which the normal exposure control unit 21 starts charge accumulation. Control is performed so that the first charge is read when the time has elapsed, and the second charge is read when the second exposure time T2 has elapsed from the read timing of the first charge.

  According to the first embodiment, in the imaging device 2 having the normal pixel 24a and the phase difference pixel 25a, the normal exposure control unit 21 that controls the charge accumulation start / end timing of the normal pixel group 24, and the phase difference pixel Since the phase difference exposure control unit 22 that controls the charge accumulation start / end timing of the group 25 independently of the normal exposure control unit 21, the exposure amount suitable for the normal pixel 24a and the phase difference pixel 25a are suitable. It becomes possible to control the exposure amount independently.

  In addition, the phase difference exposure control unit 22 indicates the phase difference exposure time from when the phase difference pixel group 25 starts charge accumulation to the end, and the normal time period until the normal pixel group 24 starts charge accumulation and ends. The first charge accumulated in the first exposure time T1 and the second exposure time T2 are divided into the first exposure time T1 and the second exposure time T2 that are the same as the exposure time Texp. In order to control the second charge to be read separately, the first charge is used for an image so that the exposure time of the normal pixel 24a and the exposure time of the phase difference pixel 25a can be made the same. Even if a mixed reading is performed when there is a moving subject within the shooting range, image data of an unnatural blur image is not obtained. Furthermore, if the sum of the first charge and the second charge is output for phase difference detection, it is possible to compensate for a decrease in sensitivity of the phase difference pixel 25a having a smaller pixel aperture than the normal pixel 24a.

  Further, the phase difference exposure control unit 22 sets the timing that is the second exposure time T2 from the timing at which the normal exposure control unit 21 starts the charge accumulation as the timing at which the charge accumulation is started. When the second charge is read when the exposure time T2 elapses, and the first charge is read when the first exposure time T1 elapses from the read timing of the second charge, The phase difference detection signal can also be acquired simultaneously with the image signal.

  On the other hand, when the first exposure time T1 has elapsed from the start of charge accumulation, the phase difference exposure control unit 22 uses the same timing as the timing at which the normal exposure control unit 21 starts charge accumulation as the timing to start charge accumulation. In the case where the first charge is read and the second charge is read when the second exposure time T2 has elapsed from the first charge read timing, priority is given to the acquisition of the image signal. Can be done.

  Then, a normal memory unit 27 that holds charges accumulated by the normal pixel group 24, a first phase difference memory unit 28 that holds charges accumulated by the phase difference pixel group 25 during the first exposure time T1, and a phase difference pixel Since the group 25 further includes the memory unit 26 including the second phase difference memory unit 29 that holds the charge accumulated in the second exposure time T2, the mixing process and the adding process can be easily performed. Even when time is required for AD conversion of all pixels, a global shutter can be realized.

  In addition, when the mix processing unit 30 mixes the signal read from the phase difference pixel 25a, the exposure time of the normal pixel 24a to be mixed and the phase difference pixel 25a are used in order to use the signal related to the first charge. The exposure time can be the same.

  In addition, since the image pickup device 2 further includes the addition unit 31 that adds the first charge and the second charge, a signal having a more appropriate exposure amount is already generated when the signal for detecting the phase difference is output from the image pickup device 2. Thus, it is not necessary to perform addition processing outside the image sensor 2.

  Furthermore, by performing at least one of the mix processing and the addition processing, preferably both in the previous stage of AD conversion, the number of pixels to be AD converted can be reduced, and the processing load can be reduced.

  In the imaging apparatus, the AF processing unit performs the phase difference AF based on the signal of the phase difference pixel 25a output from the imaging element 2 (the signal obtained by adding the first charge and the second charge). Therefore, a signal with a high SN ratio can be used, and the accuracy of the phase difference AF can be improved.

In addition, in the imaging apparatus, an image can be obtained by generating an image for recording based on the signal of the normal pixel 24a output from the imaging element 2 by the image processing unit 3. At this time, when a mixed signal is output from the image sensor 2, since the charge of the normal pixel 24 a and the first charge are mixed, the recording image generated by the image processing unit 3 is added. There is no unnatural blur.
[Embodiment 2]

  6 to 8 show Embodiment 2 of the present invention, and FIG. 6 is a block diagram showing the configuration of the image sensor 2.

  In the second embodiment, the same parts as those in the first embodiment are denoted by the same reference numerals and the description thereof is omitted as appropriate, and only different points will be mainly described.

  In the imaging apparatus of the first embodiment described above, the global shutter imaging operation is performed. However, the imaging apparatus of the present embodiment performs a rolling shutter imaging operation.

  As shown in FIG. 6, the imaging device 2 includes a pixel exposure control unit 20, a pixel unit 23, a column parallel type ADC 32A, a mix processing unit 30A, and an adding unit 31A. In the present embodiment, unlike the first embodiment described above, the memory unit 26 is not provided. However, the ADC is not limited to the column parallel type ADC, and any other configuration may be used as long as the processing in units of lines can be completed in a desired time. For example, in the case of a configuration in which AD conversion is performed on a pixel basis, a memory for one line is generally provided, and pixel data in the memory is sequentially AD converted.

  The column parallel ADC 32A performs AD conversion in units of lines.

  Further, the mix processing unit 30A and the adding unit 31A are digital processing units because they are provided in the subsequent stage of the column parallel ADC 32A.

  Next, FIG. 7 is a diagram showing an outline of a circuit configuration related to the normal pixel 24a and the phase difference pixel 25a. The circuit configuration related to the normal pixel 24a is shown on the left side of FIG. 7, and the circuit configuration related to the phase difference pixel 25a is shown on the right side of FIG.

  In the present embodiment, the photodiodes PD of the normal pixel 24a and the phase difference pixel 25a are connected to the ADCs arranged in the column parallel ADC 32A instead of being connected to the memory in the first embodiment. Has been.

  Here, the charge accumulated in the normal pixel 24a and the first charge accumulated in the phase difference pixel 25a are AD-converted by the AD converter ADC1 in the column parallel ADC 32A. The first charge accumulated in the phase difference pixel 25a is AD-converted by the AD converter ADC2 in the column parallel ADC 32A by switching the third switch SW3.

  Next, FIG. 8 is a timing chart showing the operation of each unit when a rolling shutter imaging operation is performed in the imaging device 2.

  In this embodiment, since the rolling shutter imaging operation is performed, the reset RST and readout RD of the normal pixel 24a, the reset RST of the phase difference pixel 25a, the readout of the second charge RD2, and the readout of the first charge RD1 Are sequentially performed while shifting the timing in line units. The amount of timing deviation in line units depends on the conversion speed of the column parallel ADC 32A.

  Here, the phase difference exposure control unit 22 indicates the phase difference exposure time T from the start of the charge accumulation of the phase difference pixel group 25 to the end, and the time until the normal pixel group 24 starts the charge accumulation and ends. The first charge accumulated in the first exposure time T1 and the second exposure time T2 divided into a first exposure time T1 and a second exposure time T2 that are the same as the normal exposure time Texp. In the same manner as in the first embodiment, control is performed so that the second charge accumulated in the second charge is read separately.

  Also in the present embodiment, the second exposure time T2 is not limited to be set prior to the first exposure time T1, but the second exposure time T2 is set after the first exposure time T1. It doesn't matter.

  According to the second embodiment, the same effects as those of the first embodiment described above can be achieved in the rolling imaging operation.

  In this embodiment, the phase difference exposure control unit 22 arranges the start timing and end timing of the first exposure time T1 of all the phase difference pixels 25a arranged on a certain line on the certain line. The control is performed so as to be equal to the charge accumulation start timing and end timing of all the normal pixels 24a. Accordingly, it is possible to reduce an unnatural blur of an image when a mix reading is performed when there is a moving subject within the shooting range.

  In the above description, the imaging apparatus is mainly described. However, an imaging method that performs the same processing as the imaging apparatus may be used, and a processing program for causing a computer to perform the same processing as the imaging apparatus and the processing program are recorded. It may be a non-temporary recording medium that can be read by a computer.

  Further, the present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the constituent elements without departing from the scope in the implementation stage. In addition, various aspects of the invention can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, you may delete some components from all the components shown by embodiment. Furthermore, the constituent elements over different embodiments may be appropriately combined. Thus, it goes without saying that various modifications and applications are possible without departing from the spirit of the invention.

DESCRIPTION OF SYMBOLS 1 ... Lens 2 ... Imaging device 3 ... Image processing part 4 ... AF evaluation value calculating part 5 ... Display part 6 ... Memory card 7 ... Hand shake detection part 8 ... Hand shake correction part 9 ... Exposure control part 10 ... Focus control part 11 ... Camera operation unit 12 ... Camera control unit 20 ... Pixel exposure control unit 21 ... Normal exposure control unit 21a, 22a ... Control line 22 ... Phase difference exposure control unit 23 ... Pixel unit 24 ... Normal pixel group 24a ... Normal pixel 25 ... Phase difference pixel group 25a ... Phase difference pixel 26 ... Memory unit 27 ... Normal memory unit 28 ... First phase difference memory unit 29 ... Second phase difference memory unit 30, 30A ... Mix processing unit 31, 31A ... Addition unit 32 ... Analog digital Converter (ADC)
32A ... Column parallel ADC
ADC1, ADC2 ... AD converter M0, M1, M2 ... Memory PD ... Photodiode SW1, SW2, SW3 ... First, second and third switches VDD ... Voltage source

Claims (8)

In an image sensor having a pixel group in which a plurality of pixels are arranged in a matrix,
A plurality of phase difference pixels discretely arranged in the pixel group;
A plurality of normal pixels that are pixels other than the phase difference pixels in the pixel group;
A normal exposure control unit that controls the timing at which the normal pixel group including the plurality of normal pixels starts charge accumulation and the timing at which charge accumulation ends;
A phase difference exposure control unit that controls the timing at which the phase difference pixel group including the plurality of phase difference pixels starts charge accumulation and the timing at which charge accumulation ends, independently of the normal exposure control unit;
Equipped with,
The phase difference exposure control unit includes a phase difference exposure time from the start of charge accumulation of the phase difference pixel group to the end thereof, and a normal exposure time of the normal pixel group from start to end of charge accumulation. The first charge accumulated in the first exposure time and the second charge accumulated in the second exposure time are divided into the same first exposure time and the second exposure time. And an image pickup element that controls to read out separately . The phase difference exposure control unit sets the second retroactive timing from the timing at which the normal exposure control unit starts charge accumulation as the timing at which charge accumulation is started, and the second exposure time from the timing at which the second exposure time starts. Control is performed so that the second charge is read when an exposure time has passed, and the first charge is read when the first exposure time has passed from the read timing of the second charge. The imaging device according to claim 1. The phase difference exposure control unit uses the same timing as the timing at which the normal exposure control unit starts charge accumulation as the timing at which charge accumulation is started, and when the first exposure time has elapsed from the start of charge accumulation, 2. The control according to claim 1, wherein the first charge is read, and the second charge is read when the second exposure time has elapsed from the read timing of the first charge. Image sensor. A normal memory unit that holds charges accumulated in the normal pixel group; a first phase difference memory unit that holds charges accumulated in the first exposure time by the phase difference pixel group; and the phase difference pixel group The image pickup device according to claim 1, further comprising: a memory unit including a second phase difference memory unit that holds charges accumulated during the second exposure time. The phase difference exposure control unit is configured such that the start timing and the end timing of the first exposure time of all the phase difference pixels arranged on a certain line are the charge accumulation of all the normal pixels arranged on the certain line. The image pickup device according to claim 1, wherein control is performed so as to be equal to a start timing and an end timing. A mix processing unit for mixing signals read from a plurality of pixels;
  The image pickup device according to claim 1, wherein the mix processing unit uses a signal related to the first charge when the signal read from the phase difference pixel is mixed. The imaging device according to claim 1, further comprising an addition unit that adds the first charge and the second charge. The image sensor according to claim 1,
  An AF processing unit that performs phase difference AF based on a signal of the phase difference pixel output from the image sensor;
  An image processing unit that generates a recording image based on a signal of the normal pixel output from the image sensor or a signal obtained by mixing the signal of the normal pixel and the signal of the phase difference pixel;
  An imaging apparatus comprising:

Images