JP2012191591A - Imaging apparatus and smear correction method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus or the like which can sufficiently correct a smear even if a time interval of vertical transfer in a charge transfer-type solid-state imaging element is not constant.SOLUTION: The imaging apparatus includes: an imaging element 2 having an effective region, an OB region, a vertical transfer path and a horizontal transfer path; an imaging driving section 3 varying first vertical transfer time from second vertical transfer time at every prescribed number of vertical transfer times and transferring vertically a pixel signal to the imaging element 2; and a smear correcting section 7 calculating a smear correction signal based on the pixel signal of the OB region and the prescribed number of pixel signals, detecting a pattern in a vertical direction, which regular/high luminance smear pixel signals form, subtracting the smear correction signal from the regular smear pixel signal in accordance with the detected pattern, and subtracting a smear correction signal by which a smear correction coefficient corresponding to a ratio of the first vertical transfer time and the second vertical transfer time is multiplied from the high luminance smear pixel signal.

Description

  The present invention relates to an image pickup apparatus and a smear correction method, and more specifically, an image pickup apparatus that performs smear correction by reading a pixel signal from a charge transfer solid-state image pickup device, and a pixel signal read from the charge transfer solid-state image pickup device. The present invention relates to a smear correction method for performing correction.

  In recent years, since the number of pixels of a solid-state imaging device has been increased, it takes time to read out all pixels. Even if it takes a certain amount of time for a still image, it is desirable to read out all the pixels, but in the case of a moving image, there is a time defined by the frame rate. Getting higher priority. Furthermore, in the case of a moving image, the number of required pixels is generally smaller than that of a still image.

  Therefore, in a camera using such a high-pixel image sensor, it is common to employ a technique for increasing the frame rate by reducing the number of pixels read from the image sensor as a technique for shortening the output time of the video signal. It has become. As typical examples of such a technique, pixel thinning readout and pixel addition readout are known.

  The former pixel thinning readout is a technique for reducing the number of pixels to be read out by reading out only selected pixels at a rate of one line for a plurality of lines and one pixel for a plurality of pixels on one line, for example. .

  The latter pixel addition readout is a technique for adding the pixel signals of pixels of the same color adjacent in the horizontal direction and / or the vertical direction in the image sensor to reduce the number of pixels read from the image sensor.

  Comparing these two types of technologies, pixel decimation readout involves the problem that the photoelectric conversion amount is reduced by reducing the exposure time by increasing the frame rate and the S / N ratio is lowered, and the sampling loopback by the decimation of pixels. Since there is a problem that moire occurs, it is considered that pixel addition reading is superior to pixel thinning reading in terms of image quality.

  As a driving method of the solid-state imaging device for performing such pixel addition, for example, techniques described in Japanese Patent Application Laid-Open Nos. 2006-320019 and 2004-180284 are exemplified. In the techniques described in these publications, a driving technique for reading out pixel signals of a plurality of rows by one horizontal transfer is used.

  By the way, it has a vertical transfer path and a horizontal transfer path as represented by a CCD solid-state imaging device, and after transferring the charges generated in the photodiodes to the vertical transfer path at the same time, the charges are transferred while being forwarded. In various types of charge transfer type solid-state imaging devices, it is known that smear occurs when a high-luminance subject is present within the shooting angle of view.

  This smear occurs because light leaks from a photodiode on which an optical image of a high-luminance object is formed into a vertical transfer path or a signal line.

  When a plurality of frames of images are repeatedly captured like a moving image, the charge read from the photodiode to the vertical transfer path is transferred to the horizontal transfer path on the vertical line where the high-luminance subject exists. Since the potential well for holding electric charges always passes beside the pixel of the high-luminance object, it is affected by light leakage at this time. Therefore, the smear generated due to such a cause becomes a vertical high-intensity line that reaches from the upper end to the lower end of the image along the vertical line direction.

  A method for correcting such smear has been proposed in the past. For example, Japanese Patent Laid-Open No. 2000-50165 adds and averages a plurality of lines of dummy signals read from a light-shielded OB (optical black) portion, A method of subtracting from the signal of each line in the effective area is described.

JP 2006-320019 A JP 2004-180284 A JP 2000-50165 A

  The present applicant has found that when an image in which smear has occurred is subjected to pixel addition readout as described above, a horizontal stripe pattern occurs in the vertical line in which smear has occurred (FIG. 15 according to the present invention). reference).

  Then, when the cause of the occurrence of such a smear stripe pattern was studied, when performing pixel addition reading, the time interval for performing vertical transfer for one pixel is not constant, for example, a short time, a short time, a long time I found out that it was because the pattern was repeated. In other words, charges that have been waiting for vertical transfer for a short time at a position in the vertical transfer path adjacent to the pixel on which the high-brightness object is imaged are normal smears, whereas vertical transfer is longer at that position. The charge waiting for a long time is strongly influenced by smear and becomes a high brightness smear.

  When such smear is subjected to smear correction as described in Japanese Patent Laid-Open No. 2000-50165, normal smear is corrected relatively well, but high brightness smear cannot be ignored. The remainder of correction occurs. Therefore, in the technique described in the publication, even if smear correction is performed, a striped smear still remains.

  The present invention has been made in view of the above circumstances, and provides an imaging apparatus and a smear correction method that can correct smear satisfactorily even when the vertical transfer time interval in the charge transfer type solid-state imaging device is not constant. The purpose is that.

  In order to achieve the above object, an image pickup apparatus according to an aspect of the present invention includes pixels that perform photoelectric conversion and generate pixel signals in a two-dimensional array in a row direction and a column direction, and the arrayed pixels receive subject light. A vertical transfer path that vertically transfers the pixel signal transferred from the pixel, and a horizontal transfer of the pixel signal transferred from the vertical transfer path. In an imaging device that reads out a pixel signal from a charge transfer type solid-state imaging device having a horizontal transfer path and performs smear correction, the vertical transfer and the horizontal transfer are controlled, and the pixel signal in the vertical transfer path is continuously It is necessary to divide every predetermined number of pixel signals and move the first pixel signal of the divided predetermined number of pixel signals by one pixel in the vertical direction and transfer it to the horizontal transfer path. First A read control unit for controlling the vertical transfer time of the second pixel signal to be different from the second vertical transfer time required for moving the second pixel signal in the vertical direction by one pixel and transferring it to the horizontal transfer path; The smear correction signal is calculated based on the pixel signal of the OB area read by the readout control unit, and the pixel signal including the normal smear and the high level are calculated based on the signal value of the pixel signal of the OB area and the predetermined number. A vertical pattern formed by the pixel signal including the luminance smear is detected, and the smear correction signal is subtracted from the pixel signal in the effective area read by the read control unit in accordance with the detected pattern. Alternatively, the smear correction is performed by subtracting the smear correction coefficient multiplied by the smear correction coefficient corresponding to the ratio between the first vertical transfer time and the second vertical transfer time. And Mia correction unit is obtained by including a.

  In addition, the smear correction method according to another aspect of the present invention includes a pixel and an effective region in which pixels that generate photoelectric conversion and generate pixel signals are two-dimensionally arranged in a row direction and a column direction, and the arranged pixels receive subject light. A vertical transfer path that vertically divides the pixel signal transferred from the pixel, and a horizontal transfer path that horizontally transfers the pixel signal transferred from the vertical transfer path. Charge transfer type solid-state imaging device, and the vertical transfer and the horizontal transfer are controlled, and the pixel signal in the vertical transfer path is divided into a predetermined number of consecutive pixel signals, and the predetermined number divided The first vertical transfer time required for moving the first pixel signal in the vertical direction by one pixel and transferring it to the horizontal transfer path, and the second pixel signal for one pixel. Just move vertically above A smear correction method for performing smear correction on a pixel signal read from an image pickup apparatus provided with a read control unit that controls the second vertical transfer time required for transfer to a flat transfer path to be different. Calculating a smear correction signal based on the pixel signal of the OB area read by the readout control unit, and a pixel including normal smear based on the signal value of the pixel signal of the OB area and the predetermined number A step of detecting a vertical pattern formed by a signal and a pixel signal including a high-intensity smear, and the smear correction from the pixel signal of the effective area read by the read control unit according to the detected pattern Or subtracting the smear correction signal multiplied by the smear correction coefficient corresponding to the ratio between the first vertical transfer time and the second vertical transfer time. A step that is a method having.

  According to the imaging apparatus and the smear correction method of the present invention, it is possible to satisfactorily correct smear even when the vertical transfer time interval in the charge transfer type solid-state imaging device is not constant.

1 is a block diagram illustrating a configuration of an imaging apparatus according to a first embodiment of the present invention. FIG. 3 is a diagram illustrating a configuration of an image sensor according to the first embodiment. FIG. 3 is a diagram illustrating a horizontal synchronization signal HD and a horizontal transfer pulse output from the imaging drive unit to the imaging device in the first embodiment. FIG. 4 is a diagram showing a state at time t0 of pixel signal arrangement in the vertical transfer path and the horizontal transfer path in the first embodiment. FIG. 3 is a diagram illustrating a state at (A) time t1 and (B) time t2 of pixel signal arrangement in the vertical transfer path and the horizontal transfer path in the first embodiment. FIG. 5 is a diagram illustrating a state at (A) time t3 and (B) time t4 of pixel signal arrangement in the vertical transfer path and the horizontal transfer path in the first embodiment. FIG. 4 is a diagram illustrating a state at time t5 of pixel signal arrangement in the vertical transfer path and the horizontal transfer path in the first embodiment. FIG. 5 is a diagram illustrating a state at (A) time t6 and (B) time t7 of pixel signal arrangement in the vertical transfer path and the horizontal transfer path in the first embodiment. FIG. 6 is a diagram illustrating a state at (A) time t8 and (B) time t9 of pixel signal arrangement in the vertical transfer path and the horizontal transfer path in the first embodiment. FIG. 4 is a diagram showing a state at time t10 of pixel signal arrangement in the vertical transfer path and the horizontal transfer path in the first embodiment. FIG. 4 is a diagram illustrating a state at (A) time t11 and (B) time t12 of pixel signal arrangement in the vertical transfer path and the horizontal transfer path in the first embodiment. FIG. 4 is a diagram illustrating a state at (A) time t13 and (B) time t14 of pixel signal arrangement in the vertical transfer path and the horizontal transfer path in the first embodiment. FIG. 6 is a diagram illustrating a state at time t15 of pixel signal arrangement in the vertical transfer path and the horizontal transfer path in the first embodiment. FIG. 4 is a diagram illustrating a state at time t16 of pixel signal arrangement in the vertical transfer path and the horizontal transfer path in the first embodiment. FIG. 3 is a diagram showing a state in which a stripe pattern is generated in a smear when pixel addition reading is performed when a high-luminance subject is present in the first embodiment. The figure which shows a mode when the smear shown in FIG. 15 is processed by general smear correction in the said Embodiment 1. FIG. The figure which shows a mode when the smear shown in FIG. 15 is processed by the smear correction of this embodiment in the said Embodiment 1. FIG. 5 is a flowchart showing smear correction processing in the imaging apparatus according to the first embodiment.

Embodiments of the present invention will be described below with reference to the drawings.
[Embodiment 1]

  FIG. 1 to FIG. 18 show Embodiment 1 of the present invention. FIG. 1 is a block diagram showing the configuration of the imaging apparatus, and FIG. 2 is a diagram showing the configuration of the imaging device 2.

  As shown in FIG. 1, the imaging device includes a lens 1, an imaging device 2, an imaging drive unit 3, an imaging signal processing unit 4, an image signal processing unit 5, a display device 12, and an AE / AF process. A unit 13, a memory 14, an operation unit 16, and a control unit 17 are provided. Although the recording medium 15 is also illustrated in FIG. 1, the recording medium 15 is configured as a memory card that can be attached to and detached from the imaging apparatus, and thus may not have a configuration unique to the imaging apparatus. I do not care.

  The lens 1 forms an optical image of a subject on the imaging surface of the image sensor 2 and controls a focus lens for adjusting a focus position and a passage range of a light beam reaching the image sensor 2 from the lens 1. Consists of a diaphragm.

  The image sensor 2 is an image sensor that photoelectrically converts an optical image of a subject imaged by the lens 1 and outputs it as an image signal. As shown in FIG. 2, the imaging device 2 includes a plurality of pixels 21 that perform photoelectric conversion and generate pixel signals, and has an imaging surface in which the plurality of pixels 21 are two-dimensionally arranged in the row direction and the column direction. ing. The image pickup device 2 of the present embodiment is a so-called single-plate color image pickup device in which any one of a plurality of color filters is arranged for one of the arranged pixels. . In the following, a case where the color filter array is a primary color Bayer array will be described as an example. However, the color and array used as the color filter are not limited thereto. Further, the imaging surface described above is divided into an effective area 2ef for receiving subject light from the lens 1 and an OB (optical black) area 2ob for shielding the subject light (see FIG. 15 and the like). Here, the OB area 2ob is configured to include a plurality of horizontal lines. Further, as shown in FIG. 2, the imaging device 2 includes a vertical transfer path 22 that vertically transfers the pixel signal transferred from the pixel 21, and a horizontal transfer path that horizontally transfers the pixel signal transferred from the vertical transfer path 22. And a charge transfer type solid-state imaging device.

  The imaging drive unit 3 starts and ends the exposure by the photoelectric conversion unit in the pixel 21 of the image sensor 2, transfers the pixel signal from the photoelectric conversion unit to the vertical transfer path 22, and vertically transfers the pixel signal in the vertical transfer path 22. And a vertical transfer path 22 to the horizontal transfer path 23, and a horizontal transfer of the pixel signal in the horizontal transfer path 23, etc., to control the image capturing operation of the image sensor 2, and a read control unit Doubles as The imaging drive unit 3 further divides the pixel signal in the vertical transfer path 22 into a predetermined number of continuous pixel signals (in the example of FIG. 4, as shown by the symbols L1HD, L2Hd,... The pixel signal transferred from the vertical transfer path 22 is added in the horizontal transfer path 23 in units of a predetermined number of divided pixel signals, and the horizontal transfer path 23 When the pixel addition is completed, the horizontal transfer control is performed so that all the added pixel signals held in the horizontal transfer path 23 are read out. Here, when the image pickup device 2 is the above-described single-plate color image pickup device, the image pickup drive unit 3 controls the pixel addition to be performed between pixel signals of pixels of the same color. In this pixel addition reading, the image pickup drive unit 3 performs the vertical direction of the first pixel signal by one pixel among the predetermined number of divided pixel signals, as will be described later with reference to FIG. The first vertical transfer time required to move to the horizontal transfer path 23 and the second vertical transfer time required to transfer the second pixel signal by one pixel in the vertical direction and transfer to the horizontal transfer path 23 The vertical transfer time is controlled so as to be different. Here, the first vertical transfer time is the horizontal of the pixel signal of the horizontal line closest to the horizontal transfer path 23 in the vertical transfer path 22 and the pixel signal in the horizontal transfer path 23 to be subjected to pixel addition. This is the output time interval of two addition pulses (not shown) with the time to wait for horizontal transfer to align the pixel positions in the direction, but substantially at times T1 and T2 in FIG. You can think of it as equivalent. In addition, the second vertical transfer time includes two addition pulses (with the time for waiting for the horizontal transfer to be performed in order to read out all the pixel signals that have been held in the horizontal transfer path 23 and completed the pixel addition ( The output time interval (not shown) may be considered to substantially correspond to the time T3 in FIG.

  The imaging signal processing unit 4 amplifies the analog image signal output from the imaging element 2 in accordance with the sensitivity setting, and converts it into a digital signal.

  The image signal processing unit 5 performs various image signal processing on the image signal output from the imaging signal processing unit 4 based on the control of the control unit 17. The image signal processing unit 5 includes an image reconstruction unit 6, a smear correction unit 7, and an image processing unit 8.

  The image reconstruction unit 6 reconstructs an image from the pixel signal output from the imaging signal processing unit 4. That is, when performing pixel addition reading, as will be described later, the pixel addition signal read from the horizontal transfer path 23 and output from the imaging signal processing unit 4 is different from the pixel signal read from the raster scan, for example, in the reading order. Are different (see FIG. 13 and the like). Therefore, the image reconstruction unit 6 reconstructs an image having the correct pixel order. Further, the image reconstruction unit 6 also corrects defective pixels in the reconstructed image.

  The smear correction unit 7 performs smear correction of an image read from the image sensor 2 by the image pickup drive unit 3 and reconstructed by the image reconstruction unit 6. The smear detection unit 9, the smear calculation unit 10, And a smear subtracting unit 11.

  The smear detection unit 9 sequentially detects the signal value of the pixel signal in the OB region 2ob of the image reconstructed by the image reconstruction unit 6 in the horizontal line direction, and determines whether or not the pixel signal includes smear. Compared with a predetermined threshold value, a vertical line having a signal value exceeding this threshold value is detected as a smear vertical line. The smear detection unit 9 performs such detection of a smear vertical line on one horizontal line for a plurality of lines (or all lines) in the OB area 2ob. When the smear vertical line is detected, the smear detection unit 9 further divides the signal value of the pixel signal in the OB region 2ob on the smear vertical line and the pixel signal in the vertical transfer path 22 described above. Based on the predetermined number, a vertical pattern formed by a pixel signal including a normal smear and a pixel signal including a high luminance smear is detected. Here, the normal smear is transferred in the vicinity of the high-luminance subject image that is the smear generation source for the first vertical transfer time (corresponding to the times T1 and T2 in FIG. 3) when the pixel of interest is vertically transferred. It is thought that it was a smear that occurred because it was waiting to be done. On the other hand, when the pixel of interest is vertically transferred, the high-intensity smear is transferred in the vicinity of the high-intensity subject image that is the smear generation source for the above-described second vertical transfer time (corresponding to the time T3 in FIG. 3). It is thought that it was a smear that occurred because it was waiting for. Therefore, since the high-intensity smear has been affected by the leakage light from the high-intensity subject image for a longer time, the signal value is higher than that of the normal smear, and the ratio of the smear level between the normal smear and the high-intensity smear is It is considered that it is proportional to the ratio of the standby time in the vicinity of the high-luminance subject image, that is, the ratio between the first vertical transfer time and the second vertical transfer time. The normal / high luminance smear pattern in the vertical line direction detected in the OB area 2ob is considered to be applied as it is in the effective area 2ef.

  The smear calculation unit 10 calculates a smear correction signal based on the pixel signal of the OB area 2ob. Here, since the pixel signal in the OB area 2ob includes the OB level in addition to the smear level, it is of course possible to perform the smear correction simultaneously with the correction of the OB level. However, in order to clarify how the smear correction is performed, it is assumed here that the smear correction is performed by a process different from the correction of the OB level. Then, the smear calculation unit 10 performs smear correction using pixel signals in the OB area 2ob of a plurality of lines in which either one of normal smear or high-intensity smear (hereinafter described as being normal smear) occurs. Calculate the signal. Further, the smear calculation unit 10 calculates the ratio T3 / T1 between the first vertical transfer time T1 (generally represented by T1 here because T1 = T2) and the second vertical transfer time T3 as a high luminance smear. Is calculated as a correction coefficient k, and when the horizontal line input from the image reconstruction unit 6 is a horizontal line including a high-intensity smear, a value obtained by multiplying the above-described smear correction signal by the correction coefficient k is calculated. To do.

  The smear subtracting unit 11 generates a smear correction signal from the pixel signal in the effective region 2ef according to the vertical pattern formed by the pixel signal including the normal smear detected by the smear detecting unit 9 and the pixel signal including the high luminance smear. Alternatively, smear correction is performed by subtracting the smear correction signal multiplied by the correction coefficient k. That is, the smear subtracting unit 11 subtracts the smear correction signal when the pixel signal input from the image reconstruction unit 6 is a signal including normal smear, and the input pixel signal performs high luminance smear. In the case of a signal including the smear correction signal, the smear correction signal multiplied by the correction coefficient k is subtracted. If the pixel signal input from the image reconstruction unit 6 is not a signal including smear, the smear subtracting unit 11 outputs the input pixel signal as it is.

  The image processing unit 8 performs OB level correction processing, demosaicing processing, white balance processing, gradation conversion processing, compression / decompression processing, and the like on the image signal processed by the smear correction unit 7, An image signal for display is generated. The image processing unit 8 also generates a signal for AE / AF.

  The display device 12 displays an image based on the image signal processed for display by the image signal processing unit 5. The display device 12 performs live view (LV) display and still image display, and also displays various information related to the imaging device.

  The AE / AF processing unit 13 performs AF processing and AE processing based on the AE / AF signal received from the image processing unit 8.

  The memory 14 is for temporarily storing the image signal digitally converted by the imaging signal processing unit 4 and the image signal being processed by the image signal processing unit 5.

  The recording medium 15 is a non-volatile recording medium for storing the image signal processed for recording by the image signal processing unit 5.

  The operation unit 16 is for performing various operation inputs to the imaging apparatus. The operation unit 16 is used to set a power switch for turning on / off the power of the imaging apparatus, a release button for inputting an instruction for image shooting, a still image shooting mode, a moving image shooting mode, a live view mode, and the like. Operation members such as mode buttons are included.

  The control unit 17 is configured by a processor such as a CPU, for example, and controls the entire imaging apparatus including the imaging drive unit 3 and the image signal processing unit 5 in accordance with an operation input from the operation unit 16. is there.

  Next, with reference to FIGS. 3 to 14, pixel addition reading of the same color pixels of the image sensor 2 performed by the control of the imaging drive unit 3 will be described. FIG. 3 is a diagram showing the horizontal synchronization signal HD and the horizontal transfer pulse output from the image pickup drive unit 3 to the image pickup device 2, and FIG. FIG. 5 is a diagram showing the state of the pixel signal arrangement in the vertical transfer path 22 and the horizontal transfer path 23 at (A) time t1 and (B) time t2, and FIG. 6 is a diagram showing the vertical transfer path 22 and horizontal transfer. FIG. 7 is a diagram showing the state of pixel signal arrangement in the transfer path 23 at (A) time t3 and (B) time t4. FIG. 7 shows the state of pixel signal arrangement in the vertical transfer path 22 and horizontal transfer path 23 at time t5. FIG. 8 is a diagram showing the arrangement of pixel signals in the vertical transfer path 22 and the horizontal transfer path 23 at (A) time t6 and (B) time t7, and FIG. 9 is a diagram showing the vertical transfer path 22 and water. FIG. 10 is a diagram showing the state of pixel signal arrangement in the transfer path 23 at (A) time t8 and (B) time t9. FIG. 10 shows the state of pixel signal arrangement in the vertical transfer path 22 and horizontal transfer path 23 at time t10. FIG. 11 is a diagram showing the arrangement of pixel signals in the vertical transfer path 22 and the horizontal transfer path 23 at (A) time t11 and (B) time t12. FIG. 12 is a diagram showing the vertical transfer path 22 and horizontal transfer. FIG. 13 is a diagram showing the state of pixel signal arrangement in the path 23 at (A) time t13 and (B) time t14. FIG. 13 shows the state of pixel signal arrangement in the vertical transfer path 22 and horizontal transfer path 23 at time t15. FIG. 14 is a diagram illustrating a state of pixel signal arrangement in the vertical transfer path 22 and the horizontal transfer path 23 at time t16. FIG. 3 shows times t0 to t16 when the operations shown in FIGS. 4 to 14 are performed.

  Here, the horizontal three-pixel addition reading will be described. However, this is only an example, and the number of pixel additions is not limited to three, and may be two or four or more. Needless to say, the pixel addition in the vertical direction and the pixel addition in the vertical and horizontal directions may be used.

  At the stage where the pixel signals are transferred from the pixels 21 in which the three primary colors of RGB are arranged in the Bayer array to the vertical transfer path 22, the arrangement of the pixel signals in the vertical transfer path 22 is also in the Bayer array state. In addition, in the vertical transfer path 22 and the horizontal transfer path 23, a potential barrier is provided between the electrodes under which the pixel signals are held so that the pixel signals are not mixed. There exists an electrode under which no pixel signal is held.

  Such a Bayer arrangement is maintained even after the pixel signal in the horizontal transfer path 23 is transferred to the imaging signal processing unit 4 (time t0), for example, as shown in FIG. . Here, the numbers given after R, G, or B indicating the color information carried by the pixel signal indicate the numbers of the horizontal lines in the arrangement shown in FIG. In addition, since the example described here is addition of three pixels, pixel signals for three lines are transferred by one horizontal transfer performed in synchronization with 1HD. Therefore, in FIG. 4, symbols such as L1HD, L2HD,... Are attached to every three lines as a unit of this horizontal transfer.

  Thereafter, at time t1, as shown in FIG. 5A, the pixel signals (G1 and R1 in this example) are transferred from the vertical transfer path 22 to the horizontal transfer path 23 at a rate of one pixel per three pixels in the horizontal direction. Is done. At time t2, as shown in FIG. 5B, horizontal transfer of two pixels is performed in the left direction (see also horizontal transfer pulses of two pixels between time t1 and time t2 in FIG. 3). In the Bayer array, pixels of two colors are alternately arranged on one horizontal line. Therefore, when two pixels are horizontally transferred, the same color is positioned immediately above the pixel signal already stored in the horizontal transfer path 23. Pixel signal.

  At time t3, as shown in FIG. 6A, the pixel signals G1, R1 are transferred from the vertical transfer path 22 to the horizontal transfer path 23, and the pixel signals G1, R1 already stored in the horizontal transfer path 23 The pixel signals 2G1 and 2R1 are added up. Here, the numbers added in front of R, G, or B indicating the color information indicate the number of added pixels. At time t4, as shown in FIG. 6B, horizontal transfer of two pixels is performed in the left direction.

  At time t5, as shown in FIG. 7, the pixel signals G1 and R1 are transferred from the vertical transfer path 22 to the horizontal transfer path 23, and are added to the pixel signals 2G1 and 2R1 already stored in the horizontal transfer path 23. Pixel signals 3G1 and 3R1 are obtained. Thereby, the addition of the three pixels in the first line of the pixel array shown in FIG. 4 is completed.

  Next, at time t6, as shown in FIG. 8A, the pixel signals B2 and G2 are transferred from the vertical transfer path 22 to the horizontal transfer path 23. At time t7, as shown in FIG. 8B, horizontal transfer of two pixels is performed in the left direction.

  At time t8, as shown in FIG. 9A, the pixel signals B2 and G2 are transferred from the vertical transfer path 22 to the horizontal transfer path 23, and the pixel signals B2 and G2 already stored in the horizontal transfer path 23 Addition results in pixel signals 2B2 and 2G2. At time t9, as shown in FIG. 9B, horizontal transfer of two pixels is performed in the left direction.

  At time t10, as shown in FIG. 10, the pixel signals B2 and G2 are transferred from the vertical transfer path 22 to the horizontal transfer path 23, and added to the pixel signals 2B2 and 2G2 already stored in the horizontal transfer path 23. Pixel signals 3B2 and 3G2 are obtained. Thereby, the addition of the three pixels on the second line of the pixel array shown in FIG. 4 is completed.

  Subsequently, at time t11, as shown in FIG. 11A, the pixel signals G3 and R3 are transferred from the vertical transfer path 22 to the horizontal transfer path 23. At time t12, as shown in FIG. 11B, horizontal transfer of two pixels is performed in the left direction.

  At time t13, as shown in FIG. 12A, the pixel signals G3 and R3 are transferred from the vertical transfer path 22 to the horizontal transfer path 23, and the pixel signals G3 and R3 already stored in the horizontal transfer path 23 The pixel signals 2G3 and 2R3 are added up. At time t14, as shown in FIG. 12B, horizontal transfer of two pixels is performed in the left direction.

  At time t15, as shown in FIG. 13, the pixel signals G3 and R3 are transferred from the vertical transfer path 22 to the horizontal transfer path 23, and added to the pixel signals 2G3 and 2R3 already stored in the horizontal transfer path 23. Pixel signals 3G3 and 3R3 are obtained. Thereby, the addition of three pixels on the third line of the pixel array shown in FIG. 4 is completed.

  At this time, all of the three-pixel addition signal (a signal obtained by adding three pixels to the pixel signal indicated by the symbol L1HD in FIG. 4) has been stored in the horizontal transfer path 23. Thereafter, the imaging drive unit 3 continuously applies horizontal transfer pulses to the image sensor 2 to transfer the 3-pixel addition signal in the horizontal transfer path 23 toward the imaging signal processing unit 4.

  Therefore, at the time t16 when the horizontal transfer is finished, as shown in FIG. 14, the pixel signal in the horizontal transfer path 23 is cleared, and in the next horizontal synchronization period, this is indicated by the symbol L2HD in FIG. The operation of reading out the pixel signal by adding three pixels is performed in the same manner as described above.

  When the above-described pixel addition reading process is performed, it can be seen that vertical transfer of one pixel of the pixel signal may be performed in a relatively short time or may take a relatively long time. That is, in the case of three-pixel addition reading as shown in FIGS. 3 to 14, one pixel is waited for horizontal transfer of two pixels (that is, wait for a time corresponding to times T1 and T2 in FIG. 3). When the vertical transfer for one pixel is performed and when the vertical transfer for one pixel is performed after waiting for the horizontal transfer of all the pixels in the horizontal transfer path 23 (that is, waiting for the time corresponding to the time T3 in FIG. 3). Are repeated in the order of the former twice and the latter once.

  Then, as shown in FIG. 15, when a high-luminance subject image 29 exists in the subject image formed by the lens 1, it is transferred in the vicinity of the high-luminance subject image 29 for a time T1 (= T2). A normal smear is generated in the pixel signal that has been waiting for only T3, and a high-intensity smear is generated in the pixel signal that has been waiting for the time T3. The ratio of the smear level between the normal smear and the high-intensity smear is considered to be the ratio between the standby times T1 and T3 as described above.

  At this time, as shown in FIG. 15, normal smears 25 and high-intensity smears 26 are alternately generated as horizontal stripes in the vertical lines where smear is generated, as shown in FIG. Will do. Here, FIG. 15 is a diagram showing a state in which a stripe pattern is generated in the smear when pixel addition reading is performed when a high-luminance subject exists.

  Even if such smear is uniformly corrected by a smear correction signal obtained by averaging over a plurality of lines in the OB area 2ob, as in general processing, the smear is corrected as shown in FIG. The striped pattern will still remain. FIG. 16 is a diagram showing a state when the smear shown in FIG. 15 is processed by general smear correction.

  Therefore, in the present embodiment, as shown in FIG. 17, the smear stripe pattern is greatly reduced by performing different levels of smear correction between the normal smear and the high brightness smear as described in FIG. The image can be obtained. FIG. 17 is a view showing a state when the smear shown in FIG. 15 is processed by the smear correction of this embodiment.

  Next, FIG. 18 is a flowchart showing smear correction processing in the imaging apparatus.

  When this process is started, the smear detection unit 9 sequentially detects the signal value of the pixel signal in the OB area 2ob of the image reconstructed by the image reconstruction unit 6 in the horizontal line direction, and is a pixel signal including smear. It is compared with a predetermined threshold for determining whether or not. The smear detection unit 9 detects that the vertical line is a smear vertical line when the vertical line of the signal value exceeding the threshold value continues for a predetermined number of lines of 2 or more (step S1). Here, the predetermined threshold value is calculated as, for example, OB level + α using, for example, an OB level representing a signal value of the optical black portion. Here, the coefficient α is a parameter that can be arbitrarily set in accordance with the characteristics of the image pickup device 2 or the shooting scene. In addition, it is determined that a smear vertical line is a smear vertical line only when a predetermined number or more of vertical lines having a signal value exceeding the threshold value are continuous. This is for distinguishing between a case where noise is generated due to the occurrence of noise) and a signal value exceeding the threshold value, and a case where smear occurs and the signal value exceeds the threshold value.

  The smear detection unit 9 performs such detection of the smear vertical line on one horizontal line for a plurality of lines (or all lines) in the OB area 2ob, and the pixel signal of the OB area 2ob on the smear vertical line is detected. Based on the signal value and a predetermined number (3 in the example shown in FIGS. 3 to 14) when the pixel signal in the vertical transfer path 22 is divided, the pixel signal including the normal smear and the high luminance smear are included. A vertical pattern formed by the pixel signal is detected (step S2). Specifically, the average of the signal values of the pixels belonging to the smear vertical line is calculated for each horizontal line, and the vertical movement of the calculated average value is collated with the predetermined number when dividing as described above, and FIG. In the example shown in FIG. 14, a pattern of normal smear 2 lines and high brightness smear 1 line repeated every 3 horizontal lines is detected.

  In such processing, the smear detection unit 9 detects the signal value (smear level) of the pixel signal in which smear has occurred (step S3).

Then, the smear calculation unit 10 performs, for example, a simple addition averaging or a recursive addition averaging of the pixel signals of the OB area 2ob of the plurality of lines in which the normal smear is generated (hereinafter referred to as an OB averaging signal). Calculated). Since this OB average signal is a signal including the OB level in addition to the smear level, the smear calculation unit 10 further performs OB averaging on the smear correction signal which is a reference correction amount indicating only the smear level. For example, a value obtained by subtracting (OB level + α) described above from the signal, that is,
Smear correction signal = OB averaged signal− (OB level + α)
(Step S4). Note that this step S4 assumes that the smear level correction is performed before the OB level correction, but in the reverse order, that is, the smear level correction is performed after the OB level correction. In this case, the processing in step S4 may be performed with the OB level = 0.

Further, the smear calculation unit 10 uses the correction coefficient k for high luminance smear as a ratio of the first vertical transfer time T1 and the second vertical transfer time T3.
k = T3 / T1
(Step S5). Since the first vertical transfer time T1 and the second vertical transfer time T3 are determined in advance depending on the type of the image sensor 2 and the configuration of the image pickup drive unit 3, the correction coefficient k is calculated in advance and is not shown in the figure. It may be recorded in the memory at the time of manufacture, and may be read from the non-volatile memory when processing is performed by the smear correction unit 7.

  After that, when the pixel signal input from the image reconstruction unit 6 to the smear correction unit 7 becomes the pixel signal of the effective region 2ef, the smear subtraction unit 11 detects the vertical line to which the pixel signal to be processed belongs in step S1. It is determined for each pixel whether or not it is a smear vertical line subject to smear correction (step S6).

  If it is not a smear vertical line, the smear subtractor 11 outputs the input pixel signal as it is without performing smear correction to the image processor 8 (step S7).

  On the other hand, if it is a smear vertical line, the pattern detected for the OB area 2ob in step S2 is also applied to the effective area 2ef, and the pixel signal to be processed is a signal including normal smear, or It is determined whether the signal includes high brightness smear (step S8).

  If the pixel signal does not include high-intensity smear, the smear subtraction unit 11 subtracts the smear correction signal that is the reference correction amount calculated in step S4 from the pixel signal and outputs the signal to the image processing unit 8 ( Step S9).

  When the pixel signal includes high-intensity smear, the smear subtracting unit 11 subtracts a signal obtained by multiplying the smear correction signal, which is a reference correction amount, from the pixel signal by the correction coefficient k, and the image processing unit 8 (Step S10).

  The processing in step S9 or step S10 described above is performed on all pixel signals including smear components in the effective area 2ef.

  Thus, when the process of step S9 or step S10 is performed on all pixel signals including a smear component, the process is terminated.

  In the above description, the pixel signal in the vertical transfer path is divided into a predetermined number of consecutive pixel signals, and the first pixel signal out of the divided predetermined number of pixel signals is vertical by one pixel. The first vertical transfer time required to move to the horizontal transfer path and the second vertical transfer time required to move the second pixel signal by one pixel in the vertical direction and transfer to the horizontal transfer path As a case where the reading control is performed so that the vertical transfer time differs from the vertical transfer time, pixel addition reading is taken as an example. However, such readout control is not limited to pixel addition readout, and the same applies to, for example, pixel thinning readout. That is, when performing pixel thinning readout, for example, transfer processing from the vertical transfer path 22 to the horizontal transfer path 23 performed at t3, t5, t8, t10, t13, and t15 shown in FIGS. If the processing is changed so as not to transfer to the horizontal transfer path 23 by sweeping away charges from the vertical transfer path 22 or the like, the process of thinning out 2 lines out of 3 lines is realized. For this reason, the above-described technique described by taking pixel addition readout as an example can be similarly applied to pixel thinning readout.

  In the above description, the image pickup apparatus that performs smear correction has been described. However, the present invention is not limited to the image pickup apparatus, and a method of performing smear correction as described above on an image signal including smear read from the image pickup apparatus. It may be a smear correction processing program for realizing the smear correction method, a recording medium for recording the smear correction processing program, or the like.

  According to the first embodiment, even if normal smear and high-luminance smear occur due to non-uniform time required to vertically transfer the pixel signal for one pixel, the pixel signal including normal smear Or a smear correction coefficient corresponding to the ratio between the first vertical transfer time and the second vertical transfer time, depending on whether the pixel signal is a pixel signal including a high-intensity smear or a smear correction coefficient. Since whether the signal multiplied by the correction signal is subtracted is made different, it is possible to effectively reduce the striped smear.

  In addition, when performing pixel addition readout, the readout order of the pixel addition signal is different from raster scan or the like, but smear correction is performed on the image reconstructed in the correct pixel order by the image reconstruction unit 6. Therefore, the algorithm for the image having the correct pixel order can be applied as it is.

  The above-described processing can also be applied to a single-plate color imaging device in which pixel addition of the same color is performed.

  Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. Moreover, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the embodiment. For example, you may delete some components from all the components shown by embodiment. Furthermore, the constituent elements over different embodiments may be appropriately combined. Thus, it goes without saying that various modifications and applications are possible without departing from the spirit of the invention.

DESCRIPTION OF SYMBOLS 1 ... Lens 2 ... Image sensor (a charge transfer type solid-state image sensor, a single plate color image sensor)
2ef ... Effective area 2ob ... OB area 3 ... Imaging drive unit (reading control unit)
DESCRIPTION OF SYMBOLS 4 ... Imaging signal processing part 5 ... Image signal processing part 6 ... Image reconstruction part 7 ... Smear correction part 8 ... Image processing part 9 ... Smear detection part 10 ... Smear calculation part 11 ... Smear subtraction part 12 ... Display apparatus 13 ... AE / AF processing unit 14 ... memory 15 ... recording medium 16 ... operation unit 17 ... control unit 21 ... pixel 22 ... vertical transfer path 23 ... horizontal transfer path 25 ... normal smear 26 ... high brightness smear 29 ... high brightness subject image

Claims (4)

Pixels that perform photoelectric conversion and generate pixel signals are two-dimensionally arranged in the row direction and the column direction, and the arranged pixels are divided into an effective area that receives subject light and an OB area that blocks subject light, A pixel signal is read out from a charge transfer type solid-state imaging device having a vertical transfer path for vertically transferring a pixel signal transferred from the pixel and a horizontal transfer path for horizontally transferring a pixel signal transferred from the vertical transfer path. In an imaging device that performs smear correction,
The vertical transfer and the horizontal transfer are controlled, and the pixel signal in the vertical transfer path is divided into a predetermined number of consecutive pixel signals, and the first of the divided predetermined number of pixel signals The first vertical transfer time required to move the pixel signal in the vertical direction by one pixel and transfer it to the horizontal transfer path, and the horizontal transfer by moving the second pixel signal in the vertical direction by one pixel. A read control unit for controlling the second vertical transfer time required for transfer to the path to be different;
A smear correction signal is calculated based on the pixel signal in the OB area read by the readout control unit, and a pixel signal including normal smear and a high luminance based on the signal value of the pixel signal in the OB area and the predetermined number Whether a vertical pattern formed by a pixel signal including smear is detected, and the smear correction signal is subtracted from the pixel signal of the effective area read by the read control unit according to the detected pattern Or a smear correction unit that performs smear correction by subtracting a smear correction coefficient multiplied by the smear correction coefficient according to the ratio of the first vertical transfer time and the second vertical transfer time. An imaging apparatus comprising: The readout control unit further performs pixel addition in the horizontal transfer path for pixel signals transferred from the vertical transfer path in units of the predetermined number of divided pixel signals, and pixel addition in the horizontal transfer path Is completed, the horizontal transfer control is performed so as to read out all the added pixel signals held in the horizontal transfer path.
The first vertical transfer time is a time for waiting for horizontal transfer to match the pixel position in the horizontal direction of the pixel signal in the vertical transfer path and the pixel signal in the horizontal transfer path to be added. Is equivalent to
The second vertical transfer time corresponds to a time for waiting for the horizontal transfer to be performed in order to read all the added pixel signals held in the horizontal transfer path,
An image reconstruction unit that reconstructs the added pixel signal read from the horizontal transfer path as an image;
The imaging apparatus according to claim 1, wherein the smear correction unit performs smear correction on the image reconstructed by the image reconstruction unit. The charge transfer type solid-state imaging device is a single-plate color imaging device in which a plurality of color filters are arranged for each color with respect to one of the arranged pixels.
The imaging apparatus according to claim 2, wherein the readout control unit controls the pixel addition to be performed between pixel signals of pixels of the same color. Pixels that perform photoelectric conversion and generate pixel signals are two-dimensionally arranged in the row direction and the column direction, and the arranged pixels are divided into an effective area that receives subject light and an OB area that blocks subject light, A charge transfer type solid-state imaging device having a vertical transfer path for vertically transferring a pixel signal transferred from the pixel, and a horizontal transfer path for horizontally transferring a pixel signal transferred from the vertical transfer path;
The vertical transfer and the horizontal transfer are controlled, and the pixel signal in the vertical transfer path is divided into a predetermined number of consecutive pixel signals, and the first of the divided predetermined number of pixel signals The first vertical transfer time required to move the pixel signal in the vertical direction by one pixel and transfer it to the horizontal transfer path, and the horizontal transfer by moving the second pixel signal in the vertical direction by one pixel. A read control unit for controlling the second vertical transfer time required for transfer to the path to be different;
A smear correction method for performing smear correction on a pixel signal read from an imaging device including:
Calculating a smear correction signal based on the pixel signal of the OB area read by the read control unit;
Detecting a vertical pattern formed by a pixel signal including a normal smear and a pixel signal including a high luminance smear based on the signal value of the pixel signal in the OB region and the predetermined number;
In accordance with the detected pattern, the smear correction signal is subtracted from the pixel signal of the effective area read by the read control unit, or the first vertical transfer time and the second vertical transfer Subtracting the smear correction coefficient multiplied by the smear correction coefficient according to the ratio with time,
A smear correction method comprising:

Images