JP2014165676A - Signal processing device and imaging device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make it possible to reliably remove a horizontal noise component which appears when a high-luminance subject image is incident.SOLUTION: A signal processing device comprises correction means that on the basis of the number of pixels in each row outputting a signal value greater than a first determination value among output signals of an effective pixel part of an image sensor having a plurality of pixels and an output signal of a light shielding pixel part of the image sensor, corrects an output signal of the effective pixel part for each row.

Description

  The present invention relates to a signal processing device and an imaging device.

  In a digital camera, a video camera, or the like, a CCD or CMOS type image sensor is generally used as a solid-state image sensor. If there is a very bright subject in the optical image formed on these image sensors, the vertical, vertical, horizontal, or horizontal strip level fluctuations in the column or row direction of the region where the bright subject exists A so-called smear phenomenon may occur.

  In particular, in a CMOS type image sensor, this phenomenon is generally considered to be caused by a wiring layout such as a power supply and GND. For example, when a high-brightness subject forms an image on a specific portion in the screen and a large amount of charge is generated here, the output of the pixel portion, the vertical output line, the column amplifier portion, or the like varies greatly. When such a change occurs, the GND or the power supply in the same row changes due to the internal wiring resistance of the GND or the power supply line.

  For example, a slight potential difference occurs in the internal reference signal that is output in the reset state due to the occurrence of such fluctuation during optical signal transfer. Since such a difference remains as an offset component for the same line that is read at the same time, it is considered that a so-called horizontal smear occurs in which the level fluctuates over the same line.

  FIG. 17 shows a state in which such a lateral smear occurs. In FIG. 17, the total output 1701 of the image sensor includes a vertical OB (optical black) unit 1702 positioned at the top of the screen, which is a light-shielded region, a horizontal OB unit 1703 positioned at the left end of the screen, and an effective unit 1704 for outputting image data. Is provided. When the high-brightness subject 1705 enters the effective portion 1704 in FIG. 17, regions 1706 are generated on the left and right sides of the output that are sunk below the peripheral pixel output. This causes a similar offset error not only in the effective portion 1704 but also in the horizontal OB portion 1703. In this example, we introduced the case where “sinking” horizontal smearing occurs in the signal level as a phenomenon of horizontal smearing. However, depending on the influence of a high-brightness subject, a brighter area appears on the left and right of the high-brightness subject. Smear may occur. Countermeasures are required for any lateral smear.

  In order to avoid such a phenomenon, in Patent Document 1, a smear component is detected based on the output of the vertical OB unit, and correction processing is performed by subtracting the detected amount from the output of each row. Further, in Japanese Patent Application Laid-Open No. 2004-228561, considering that an offset correction is simply an overcorrection, a smear is corrected by performing a gain correction after the offset correction. At this time, the content of the smear correction process is changed according to the signal level of the effective image signal. More specifically, smear overcorrection is avoided by preventing smear correction when the level of the effective image signal reaches a predetermined signal saturation level.

JP 7-67038 A JP 2001-24943 A

  However, not only smear but also random noise components and offset components are superimposed on the OB portion. For this reason, if the outputs of the horizontal OB portions of each row are simply averaged, the average value still has random variations particularly in a region where the random noise is large, such as at high sensitivity. If this is subtracted, noise such as horizontal stripes generated for each row may be emphasized.

  The present invention has been made in view of the above-described problems, and provides a signal processing device and an imaging device capable of reliably removing a lateral noise component that appears when a high-luminance subject image is incident. With the goal.

  The signal processing apparatus according to the present invention includes a pixel number for each row that outputs a signal value larger than a first determination value among output signals of an effective pixel portion of an image sensor having a plurality of pixels, and a light-shielding pixel portion of the image sensor. It is characterized by comprising correction means for correcting the output signal of the effective pixel portion for each row based on the output signal.

  According to the present invention, it is possible to reliably remove a noise component in the lateral direction that appears when a high-luminance subject image is incident.

1 is a diagram illustrating a block configuration of an imaging apparatus according to a first embodiment. 6 is a flowchart for explaining a camera sequence according to the first embodiment. 6 is a flowchart for explaining a camera sequence according to the first embodiment. 6 is a flowchart for explaining a camera sequence according to the first embodiment. The figure which shows the structure of the row direction correction circuit in 1st Embodiment. The figure which shows the condition of horizontal smear generation | occurrence | production in 1st Embodiment. The figure which shows the structure of the row direction correction circuit in 2nd Embodiment. The figure which shows the condition of horizontal smear generation | occurrence | production in 3rd Embodiment. The figure which shows the structure of the row direction correction circuit in 3rd Embodiment. The figure which shows the condition of horizontal smear generation | occurrence | production in 4th Embodiment. The figure which shows the structure of the row direction correction circuit in 4th Embodiment. The figure which shows the block configuration of the imaging device in 5th Embodiment The figure which shows the structure of the row direction correction circuit in 5th Embodiment. The figure which shows the condition of horizontal smear generation | occurrence | production in 5th Embodiment. The figure which shows the structure of the row direction correction circuit in 6th Embodiment. The figure which shows the calculation method in 6th Embodiment. The figure which shows the condition of horizontal smear generation | occurrence | production.

  Embodiments of the present invention will be described below in detail with reference to the accompanying drawings.

(First embodiment)
FIG. 1 is a block diagram showing a configuration of an imaging apparatus which is a first embodiment of a signal processing apparatus of the present invention. An imaging apparatus 100 according to the present embodiment includes a CMOS type imaging device 101 having a plurality of pixels, an AFE (Analog Front End) 102, a DSP (Digital Signal Processor) 103, a timing generation circuit 104, and a CPU 105.

  The CMOS image sensor 101 has a built-in amplifier circuit (not shown) that switches the gain according to the ISO sensitivity. The AFE 102 incorporates an A / D converter that converts an analog signal from the image sensor 101 into a digital signal, and has a function of clamping an offset level from the target dark level.

  The DSP 103 performs various correction processes, development processes, and compression processes on the digital signal output from the AFE 102. The DSP 103 also performs access processing to various memories such as the ROM 106 and RAM 107, processing for writing image data to the recording medium 108, processing for displaying various data on the liquid crystal display unit (LCD) 114, and the like. Furthermore, various correction processes can be performed on the image data on the RAM 107. The DSP 103 includes a row direction correction circuit 1033 (see FIG. 5) that corrects horizontal smear, horizontal stripe noise, and the like generated in the CMOS image sensor 101 with reference to the output of the HOB unit.

  The timing generation circuit 104 supplies a clock signal and a control signal to the image sensor 101, the AFE 102, and the DSP 103 under the control of the CPU 105, and cooperates with the DSP 103 to generate timing signals corresponding to various readout modes of the image sensor 101. Generate.

  The CPU 105 controls the DSP 103 and the timing generation circuit 104, and controls camera functions using each unit (not shown) such as photometry and distance measurement. For example, a power switch 109, a first-stage shutter switch 110 (SW1), a second-stage shutter switch 111 (SW2), a mode dial 112, and an ISO sensitivity setting switch 113 are connected to the CPU 105. The CPU 105 executes processing according to the setting state of these switches and dials.

  The ROM 106 stores a control program for the imaging apparatus, that is, a program executed by the CPU 105, various correction data, and the like. The ROM 106 is generally composed of a flash memory. The RAM 107 is configured to be accessible at a higher speed than the ROM 106. The RAM 107 is used as a work area and temporarily stores image data processed by the DSP 103.

  As the recording medium 108, for example, a memory card or the like for storing captured image data is used. The recording medium 108 is connected to the DSP 103 via a connector (not shown), for example.

  The power switch 109 is operated by the user when starting the imaging apparatus. When the first-stage shutter switch SW1 is turned on, pre-shooting processing such as photometry processing and distance measurement processing is executed. When the second-stage shutter switch SW2 is turned on, a series of imaging operations for driving a mirror and a shutter (not shown) and writing image data captured by the imaging element 101 to the recording medium 108 via the AFE 102 and the DSP 103 Is started.

  The mode dial 112 is used for setting various operation modes of the imaging apparatus. The ISO sensitivity setting switch 113 is used to set the shooting ISO sensitivity of the imaging apparatus. The LCD 114 displays camera information, reproduces and displays captured images, or displays moving image data.

  Next, the outline of the photographing operation of the imaging apparatus shown in FIG. 1 will be described based on the flowchart of FIG.

  When the power switch 109 is turned on (step S201), the CPU 105 determines whether or not electric energy necessary for photographing remains in the battery (step S202). As a result, if the electric energy necessary for photographing does not remain in the battery, the CPU 105 displays a warning message to that effect on the LCD 114 (step S211), returns to step S201, and the power switch 109 is turned on again. Wait for

  If the electric energy necessary for photographing remains in the battery, the CPU 105 checks the recording medium 108 (step S203). This check is performed by determining whether or not a recording medium 108 capable of recording data of a predetermined capacity or more is loaded in the imaging apparatus. If the recording medium 108 capable of recording data of a predetermined capacity or more is not attached to the imaging apparatus, the CPU 105 displays a warning message to that effect on the LCD 114 (step S211) and returns to step S201.

  When a recording medium 108 capable of recording data of a predetermined capacity or more is loaded in the imaging apparatus, the CPU 105 determines whether the shooting mode set by the mode dial 112 is a still image shooting mode or a moving image shooting mode. A determination is made (step S204). Then, the CPU 105 performs still image shooting processing if the still image shooting mode is set (step S205), and performs moving image shooting processing if the moving image shooting mode is set (step S206).

  Next, details of the still image shooting operation in step S205 of FIG. 2 will be described based on the flowchart of FIG.

  In the still image shooting process, the CPU 105 first waits for the shutter switch SW1 to be turned on (step S301). Then, when the shutter switch SW1 is turned on, the CPU 105 uses a photometry control unit and a distance measurement control unit (not shown) to perform photometry processing for determining the aperture value and shutter speed, and distance measurement for focusing the photographing lens on the subject. Processing is performed (step S302).

  Next, the CPU 105 determines whether or not the shutter switch SW2 is turned on (step S303). As a result, if the shutter switch SW2 is not turned on, the CPU 105 determines whether or not the shutter switch SW1 is kept on (step S304). If the on state of the shutter switch SW1 continues, the CPU 105 returns to step S303 and determines whether or not the shutter switch SW2 is turned on. On the other hand, if the on state of the shutter switch SW1 does not continue, the CPU 105 returns to step S301 and waits for the shutter switch SW1 to be turned on again.

  If it is determined in step S303 that the shutter switch SW2 has been turned on, the CPU 105 executes shooting processing (step S305). Details of this photographing process will be described later.

  Next, the CPU 105 causes the DSP 103 to execute development processing of the captured image data (step S306). Then, the CPU 105 causes the DSP 103 to execute a compression process on the image data that has been subjected to the development process, and stores the image data that has been subjected to the compression process in an empty area of the RAM 107 (step S307).

  Next, the CPU 105 causes the DSP 103 to read out the image data stored in the RAM 107 and perform recording processing on the recording medium 108 (step S308). Then, the CPU 105 checks the on / off state of the power switch 109 (step S309).

  If the power switch 109 remains on, the CPU 105 returns to step S301 to prepare for the next shooting. On the other hand, if the power switch 109 is turned off, the process returns to step S201 in FIG. 2 and waits for the power switch to be turned on again.

  Next, details of the photographing process in step S305 in FIG. 3 will be described based on the flowchart in FIG.

  In the shooting process, the CPU 105 sets the amplifier gain in the image sensor 101, the gain in the AFE 102, and the like so that the shooting ISO sensitivity of the imaging device becomes the ISO sensitivity set by the ISO sensitivity setting switch 113 (step S400). ).

  Next, the CPU 105 moves the mirror to the mirror up position (step S402). Next, the CPU 105 drives the aperture to a predetermined aperture value based on the photometric data acquired in the photometric processing of step S302 in FIG. 3 (step S403).

  Next, the CPU 105 erases (clears) the electric charge of the image sensor 101 (step S404). Thereafter, the CPU 105 starts charge accumulation for the image sensor 101 (step S405).

  Subsequently, the CPU 105 opens the shutter (step S406) and starts exposure of the image sensor 101 (step S407). Then, the CPU 105 stands by until the exposure end time specified by the photometric data elapses (step S408), and when the exposure end time elapses, the CPU 105 closes the shutter (step S409).

  Next, the CPU 105 drives the aperture to the full aperture value (step S410). Next, the CPU 105 drives the mirror to the mirror down position (step S411). Thereafter, the CPU 105 stands by until the set charge accumulation time has elapsed (step S412), and when the charge accumulation time has elapsed, the CPU 105 ends the charge accumulation for the image sensor 101 (step S413).

  Subsequently, the CPU 105 reads a signal from the image sensor 101 (step S414). In this case, the AFE 102 having a function of clamping the dark offset level performs a clamping operation using an output from a light-shielded optical black portion (OB portion) (not shown) of the image sensor 101. The clamp operation in the AFE 102 is set so as to be performed with a relatively slow time constant, responds to dark shading with a low frequency, and does not respond to temporary horizontal OB portion fluctuations that appear due to phenomena such as horizontal stripes and lateral smears. To do.

  Next, the CPU 105 performs row direction correction processing (step S415). The row direction correction process is performed using a row direction correction circuit 1033 shown in FIG. The row direction correction circuit 1033 is built in the DSP 103 in FIG.

  Hereinafter, the row direction correction process will be described in detail. First, an image of a horizontal smear image will be described with reference to FIG. FIG. 6 is a detailed description of FIG. 17 described above.

  As shown in FIG. 6, when a high-luminance subject is incident on the image sensor 101, the maximum value of the digital data is determined by the number of A / D conversion bits, no matter how high-luminance the subject is. For example, if it is 12 bits, 4095 LSB is the maximum value, and if it is 14 bits, 16383 LSB is the maximum value. It is up to the system design to set what kind of digital data to obtain when a predetermined optical signal is incident, but there was an optical signal input that exceeded the maximum level of this digital output. However, the output is clipped at a uniform maximum value.

  On the other hand, the lateral smear phenomenon that occurs in a CMOS image sensor does not occur immediately after reaching the maximum value of this digital output. Depending on the conditions, a lateral smear may be generated from an output level before the digital maximum value, or a lateral smear may be generated only when an optical signal input far exceeding the digital maximum value is incident. This part depends on the type of the CMOS type image pickup element, and even if it is the same image pickup element, there is also a part that changes depending on the gain setting of an analog circuit inside the image pickup element.

  In FIG. 6, the A / D conversion result inside the high brightness subject incident area 602 is the maximum value. The inner region 601 is a region (incidence position) in which an object with higher brightness is incident in the region 602, and the lateral smear phenomenon appears as a region 603 having an offset error due to the influence of the region 601. It is shown that.

  On the other hand, a horizontal stripe component 604 as shown in FIG. The horizontal stripe component 604 is temporarily generated due to the influence of disturbance noise or noise generated in the power source of the CMOS image sensor or the GND wiring.

  Next, the operation of the row correction circuit 1033 in FIG. 5 will be described. In FIG. 5, a digital data stream is input to the input terminal 501. The HOB unit error detection unit 502 selects pixel data that is an output signal of a horizontal OB (HOB) unit that is a light-shielded pixel unit in the data stream input to the input terminal 501, and outputs the output signal of the HOB unit in the corresponding row. The amount of error between the target level is calculated.

  There are many possible methods for calculating the error for each row by the HOB unit error detection unit 502. Here, a method of calculating a simple average for the output of the HOB unit for each row is used. In this case, there is a limit to the number of pixels that can be used to calculate the average value in the HOB part. It is assumed that it is difficult to calculate the average value of. In such a case, since the calculated error amount includes a large random noise component, if the subtraction process is performed directly from the input image, the random noise component is superimposed for each row, resulting in randomness for each row. This increases the horizontal stripe component.

  In order to reduce this, by multiplying the calculated error amount by a constant gain (<1.0) (subtraction result) (subtraction result), while reducing the influence of such random noise, As a result, a correction effect can be obtained. Although depending on the number of pixels in the HOB portion and the random noise level of each pixel, a gain suitable for this horizontal stripe correction is selected to be about 0.4 to 0.7. Here, a gain suitable for horizontal stripe correction is set in the correction coefficient 1 (503), and the correction coefficient 1 is selected by the selector 507 as long as a horizontal smear detection signal described later does not become H.

  Then, the error amount of the HOB unit is multiplied by the correction coefficient 1 selected by the selector 507 by the multiplier 505, the multiplication result calculated for each row is latched by the F / F 506, and the subtractor is output from the output of the corresponding row. At 515, a subtraction process is performed. With this correction processing operation, it is possible to perform correction processing on the horizontal stripe component 604 as shown in FIG. 6 without overcorrection using the HOB portion output.

  On the other hand, the effective part selection unit 508 detects pixel data that is an output signal of the effective pixel part of the image sensor included in the data stream, and transfers only the effective pixel output of the detection target row to the subsequent detection part. First, the comparator 510 compares the effective pixel output with the first determination value set as the determination value 1 (509) (first comparison). As the determination value 1, a maximum value of A / D conversion corresponding to the number of change bits of the AFE 102 or a value close to the maximum value is set as the first determination value. In the comparator 510, the comparison signal (comparison result) becomes H when the pixel output is larger than the first determination value. Further, the counting unit 511 receives the comparison signal output from the comparator 510, and counts the number of pixels that output a signal value larger than the first determination value for each row based on the received comparison signal.

  The output of digital data can be discriminated up to the maximum value of the A / D conversion result, and it cannot be discriminated from the digital data itself whether or not the incident light quantity exceeding the maximum value is incident. However, even if there are effective pixels that output a signal value that is larger than the first determination value whose A / D conversion result is set to the determination value 1, if there are very few effective pixels, a lateral smear is generated. It can be expected that the light signal of the level is not incident. On the other hand, when the number of effective pixels that output a signal value larger than the first determination value whose A / D conversion result is set to the determination value 1 is very large, an optical signal that generates only a lateral smear is incident. Judging that there is a high possibility that

  In the comparator 513, the count result by the counting unit 511 is compared with the second determination value set to the determination value 2 (512) (second comparison). For example, N pixels (where N is a natural number) are set as the second determination value in the determination value 2 (512). The comparator 513 makes the comparison signal H when the count result is larger than the second determination value. In the horizontal smear generation region of FIG. 6, the number of effective pixels that output a signal value larger than the first determination value exceeds the second determination value (that is, N pixels), so horizontal smear may have occurred. The comparator 513 is inverted. The F / F 514 latches the result, and this becomes a lateral smear detection signal. In the case of H, the correction coefficient 2 (504) is selected by the selector 507.

  When the lateral smear detection signal becomes H, most of the error generated in the HOB portion is due to lateral smear, and the gain set for the correction coefficient 2 is a value close to 1.0. Then, the error amount of the HOB portion is multiplied by the correction coefficient 2 selected by the selector 507 by the multiplier 505, the multiplication result calculated for each row is latched by the F / F 506, and the subtracter 515 is output from the output of the corresponding row. Subtract with.

  With this correction processing, it is possible to appropriately perform correction processing on the horizontal smear component 603 as shown in FIG. 6 using the HOB portion output. When the correction coefficient approaches 1.0, the influence of random noise is assumed as described above. However, if the amount of horizontal smear is larger than the amount of variation in the average value due to random noise, even if the result is a slight increase in horizontal stripes, the overall image quality will be much improved. . Further, the counting unit 511, the F / Fs 506 and 514 are reset for each row, and the next row is determined.

  For example, about 31/32 of the A / D conversion maximum value (about 15872 LSB if the A / D conversion is 14 bits) is set as the first determination value in the determination value 1, A numerical value about 1/8 of the number of directional pixels is set as the second determination value. These numerical values can be obtained by confirming in advance the state of occurrence of lateral smear of the CMOS image sensor to be used by changing the luminance of the incident light source, the area of the incident light, or the like. Switching is necessary at least when switching to a photographing sensitivity with a different gain in the image sensor.

  Further, when the drive timing of the image sensor is switched, for example, when the drive timing of the CMOS image sensor is switched between the still image mode and the moving image mode, there is a difference in the influence amount of the power supply GND fluctuation in the image sensor. Therefore, it is necessary to switch each determination value. Thus, it is desirable to be able to switch according to various shooting conditions.

  In the present embodiment, only one effective part selection unit 508 is prepared. However, in an image sensor having a location dependency such that the amount of horizontal smear generated differs depending on which part of the screen the high-intensity part incident on the image sensor is selected, a different region in the screen is selected. In addition, the detection units after the effective part selection unit 508 may be arranged for each place in the screen. An appropriate first determination value and second determination value are set for each location in the screen, and the selector 507 is switched when the lateral smear detection signal becomes H in any region. It may be. Further, the correction coefficient 2 may be switched according to the region to be determined.

  In the present embodiment, all the pixel outputs of the image sensor are handled in the same way. However, a determination value 1 (509), a comparator 510, a counting unit 511, a determination value 2 (512), and a comparator 513 are prepared according to pixel outputs of different colors (by color) of the image sensor. The configuration may be such that the selector 507 is switched when even one comparator 513 corresponding to the color output changes. Further, the selector 506 may be switched when any one of all the comparators 513 changes. In that case, the first determination value and the second determination value can be set independently for each color.

  In this embodiment, the target row is not clarified, but in order to perform correction processing on the corresponding row after determining the signal of the effective pixel area in the data stream, the data stream is temporarily stored. Buffer memory is required. By providing the buffer memory, it is possible to subtract the error correction amount calculated after the error detection in the HOB portion of the corresponding row and the counting processing in the effective pixel region from the output of the first pixel of the corresponding row. . In the block diagram of the present embodiment, since the buffer memory is not provided, the circuit scale is very small. However, the error amount calculated by counting is applied to the correction for the next row after the detection row.

  Actually, the lateral smear phenomenon does not suddenly occur from a specific line, but often occurs so that the error gradually increases over multiple lines, so correction that directly reflects the detection result in the detection line is not performed. It doesn't matter.

  As described above, according to the present embodiment, it is possible to implement appropriate correction processing for noise generated in the row direction such as horizontal smear and horizontal stripes with a circuit configuration having a relatively small circuit scale.

(Second Embodiment)
FIG. 7 is a block diagram of the row direction correction circuit 1033 in the second embodiment. The basic idea is the same as in the first embodiment, but the effective pixel that outputs a signal value larger than the first determination value set to the determination value 1 (509) among the pixel outputs of the effective pixel portion is selected. Means for determining the output of the counting unit 511 for counting is a determination circuit 518.

  The determination circuit 518 sets the count result of the counting unit 511 to the second determination value set to the same determination value 2 (512) as in the first embodiment and the newly set determination value 3 (517). Comparison with the third determination value is performed (third comparison). The second determination value set to the determination value 2 (512) is the same numerical value as that of the first embodiment, but the determination value 3 (517) is higher than the value set to the determination value 2 (512). It is assumed that a small third determination value is set.

The determination circuit 518 outputs “0” when the output of the counting unit 511 is less than the third determination value set in (1) determination value 3 (517).
(2) If the value is equal to or greater than the third determination value set to the determination value 3 (517) and less than the determination value 2 (512), “2” is output.
(3) When the value is equal to or greater than the second determination value set in determination value 2 (512), “1” is output.

Accordingly, the selector 507 selects the correction coefficient 1 when the value is less than the third determination value set to (1) determination value 3 (517).
(2) The correction coefficient 3 is selected when the value is equal to or greater than the third determination value set to the determination value 3 (517) and less than the second determination value set to the determination value 2 (512).
(3) When the determination value is equal to or greater than the second determination value set to determination value 2 (512), correction coefficient 2 is selected.

  An intermediate value as a correction coefficient in the intermediate case where the counting result is not less than the third determination value set in determination value 3 (517) and less than the second determination value set in determination value 2 (512) Is used. As a result, even when the correction coefficient 1 (503) suitable for horizontal stripes and the correction coefficient 2 (504) suitable for horizontal smear are greatly different, the correction process changes rapidly by providing the intermediate correction coefficient 3. This reduces the possibility of affecting the image quality.

  As the value of the correction coefficient 3, a value between the correction coefficient 1 and the correction coefficient 2 is entered. In this case, an average value of these coefficients may be simply entered, or a circuit that automatically sets the average value of these coefficients when the correction coefficient 1 and the correction coefficient 2 are set as the correction coefficient 3. It may be a configuration.

  In particular, when there is no memory for buffering the data stream as described in the first embodiment, there is an effect of absorbing this influence even when the detection result of the corresponding row is used in the correction processing for the next row. . Therefore, even without having a memory with a large circuit scale, an appropriate correction can be made without adversely affecting the image quality by adding a small circuit scale.

(Third embodiment)
An image assumed in this embodiment is shown in FIG. As shown in FIG. 8, it is assumed that a plurality of vertically long slit-like subjects 801 are incident on the high-luminance subject in addition to the regions 601 and 602 as described above with reference to FIG. Further, it is assumed that the width of each slit light is approximately N / 8 pixels, and the total sum for each row is N pixels substantially the same as the width of the region 602.

  In this case, the total sum of the high-intensity subjects for each row is certainly N pixels, but horizontal smear is unlikely to occur. A row direction correction circuit capable of discriminating between the subject area 801 and the subject area 601 in such a case will be described.

  FIG. 9 is a diagram showing a configuration of the row direction correction circuit 1033 in the present embodiment. The row direction correction circuit shown in FIG. 9 has the same configuration as the row direction correction circuit of the first embodiment shown in FIG. A continuous counting unit 520 is prepared instead of the counting unit 511 of the first embodiment.

  Hereinafter, the operation of the continuous counting unit 520 will be described. When the input valid part data is equal to or greater than the first determination value set to the determination value 1 and the comparator 510 is inverted, the continuous counting unit 520 increments the internal counter. If the data of the valid part is continuously equal to or greater than the first judgment value set to the judgment value 1, the continuous counting unit 520 continues to increment the internal counter during that time, but the data of the valid part is again the judgment value 1. If the comparator 510 is inverted because it is less than the first determination value set to, the count result so far is stored and the counter is reset. If there is a count result that has already been saved, the count result is saved only if the current count result is larger than the previous count result. This makes it possible to detect the number of pixels in the section where the high-intensity subject is most continuous in the corresponding row, that is, to consider the continuity of the high-intensity subject.

  If the maximum number of consecutive pixels finally obtained is compared with the determination value 2 (512) and is equal to or greater than the second determination value set to the determination value 2, a lateral smear may have occurred. The comparator 513 is inverted. The F / F 514 latches the result, and this becomes a lateral smear detection signal. When it is H, the selector 507 selects the correction coefficient 2 (504). For each row, the continuous counting unit 520 resets the internal counter and the stored maximum count value.

  By performing such an operation, when a high-luminance subject as shown in FIG. 8 is incident, the maximum number of continuous pixels in the horizontal direction of the high-luminance subject is not large for the subject 801. The value 2 is not exceeded and the result is not determined to be a lateral smear.

  However, for high-luminance subjects such as the subjects 601 and 602, the maximum number of consecutive pixels in the horizontal direction exceeds the determination value 2, so that it is determined that the result is horizontal smear and appropriate correction can be performed.

(Fourth embodiment)
Up to now, the clamping operation in the AFE 102 is set so as to operate with a relatively slow time constant, and responds to dark shading or the like with a relatively low frequency, while on the other hand, a temporary horizontal OB that appears due to a phenomenon such as a horizontal stripe or a lateral smear. It was set not to respond to the fluctuation of the part. However, if the area of the high-brightness subject becomes large and lateral smear occurs over a larger number of rows in the vertical direction, errors caused by the lateral smear are gradually drawn even if the time constant of the clamping operation in the AFE is set late. . This is shown in FIG.

  As shown in FIG. 10A, when the area of the high-luminance subject, particularly the vertical area, is small, the clamping operation of the horizontal OB in the AFE is not particularly affected. As shown in the figure, the effect is visible only in the lateral smear generation region.

  However, as shown in FIG. 10B, when the area of the high-luminance object, particularly the vertical area, increases, the area where the lateral smear occurs due to the horizontal OB clamping operation in the AFE clamps the error amount. It begins to draw and gradually pulls in. When the lateral smear generation region is passed, the pulling operation is performed again with a slow time constant in order to recover from the lateral smear error amount that has already been pulled. For this reason, when the area of the high-luminance subject is large, pattern noise which has not been conventionally produced is caused by the malfunction of the AFE OB clamp below the lateral smear generation region. Hereinafter, a row direction correction circuit corresponding to such a phenomenon will be described.

  FIG. 11 is a diagram showing a configuration of the row direction correction circuit 1033 in the present embodiment. The row direction correction circuit in FIG. 11 has the same configuration except that the operation of the determination circuit 518 is different from the configuration described in the second embodiment in FIG. 7 and that the content of the determination value 3 is different.

  The determination circuit 521 counts the number of pixels having a value equal to or larger than the first determination value set to the determination value 1 in the effective region by the counting unit 511 and the second value set to the determination value 2 (512). Compare the judgment values. If the count result is less than the second determination value set to the determination value 2, the output is set to “0” and the correction coefficient 1 is selected by the selector 507. On the other hand, if the counting result is equal to or greater than the second determination value set to the determination value 2, the output is set to “1”, the correction coefficient 2 is selected by the selector 507, and the number of internal rows Increment the counter. Further, in the next row, when the counting result is equal to or larger than the second determination value set to the determination value 2, the output is set to “1”, the correction coefficient 2 is selected by the selector 507, and the internal row Increment the number counter. As described above, when the counting result of each row is equal to or larger than the second determination value set to the determination value 2, the number of consecutive rows is counted.

  Next, when it is determined that the counting result is less than the second determination value set to the determination value 2, the number of consecutive rows up to that time exceeds the third determination value set to the determination value 3. In this case, the output is set to “2”, and the correction coefficient 3 is selected by the selector 507. Internally, the line number counter is decremented.

  After this, if the counting result continues to be less than the second determination value set to the determination value 2, the output is set to “2” until the decrement result becomes zero. When the internal counter reaches 0, its output is set to “0” and the correction coefficient 1 is selected.

  Even if the counting result is equal to or larger than the second determination value set to the determination value 2, the number of consecutive rows until the next time the counting result becomes less than the second determination value set to the determination value 2 When the coefficient result does not exceed the third determination value set to the determination value 3, the output of the determination circuit is “0” as usual, and the correction coefficient 1 is selected. Further, when the count result becomes less than the second determination value set to the determination value 2 and the count result becomes equal to or more than the second determination value set to the determination value 2 during the decrement, the internal counter Resets.

  By performing such processing, a correction coefficient suitable for correcting the lateral smear is selected in the area where the lateral smear is generated, and the lateral smear does not occur and a clamp error due to AFE may have occurred. In a region where the error is high, a correction amount capable of absorbing the error can be selected. In addition, since it is possible to select a correction coefficient suitable for horizontal stripes that are likely to occur randomly in other regions, image quality is degraded even when horizontal smear occurs over a wide area. Correction is possible without any problem.

  In the present embodiment, in order to reduce the circuit scale, a period for correcting the clamp error by the same number of rows as the number of rows in which the lateral smear is continuously generated is provided. However, the present invention is not limited to this method, and the number of rows for correcting the clamping error may be set by multiplying the number of rows in which the lateral smear is continuous by a proportional coefficient.

(Fifth embodiment)
In the previous embodiment, the method of correcting the clamping error in AFE has been described. However, when it is determined that a lateral smear has occurred, the clamping operation itself in AFE is temporarily stopped to temporarily stop the clamping. Malfunctions can be prevented. Assuming that the clamping operation cannot be performed during this period AFE, if the lateral smear amount is larger than the clamping error amount during the period in which the lateral smear occurs, the lateral smear is corrected even if the clamping operation is stopped. Is more desirable.

  FIG. 12 shows a block diagram of the imaging apparatus in the present embodiment. The block configuration in FIG. 12 is the same as that of the embodiments described so far, and only the control signal from the DSP 103 to the AFE 102 is increased. This control signal is output from the row direction correction circuit 1033 shown in FIG. Next, the row direction correction circuit 1033 will be described.

The row direction correction circuit 1033 shown in FIG. 13 is the same as the row direction correction circuit in the first embodiment, but can output the output of the F / F 514 from the terminal 522 to the outside. This output signal is supplied to the AFE 102, and the horizontal OB clamping operation is stopped in the AFE 102 during a period when this control signal is H. This state will be described with reference to FIG. 14. As shown in FIG. 14, when it is determined that there is a high possibility that a lateral smear has occurred in the region where a high-luminance subject has entered, The result is latched, and this becomes a lateral smear detection signal. When it is H, the correction coefficient 502 is selected by the selector 507. At the same time, this signal is connected to the AFE 102 as an OB clamp stop signal to stop the HOB clamp operation in the AFE.

  As a result, even if the area where the lateral smear is generated is large, the clamping operation in the AFE is stopped, so that an error caused by the lateral smear is not drawn, and as a result, the lower part of the region where the lateral smear is generated Thus, there is no fluctuation caused by the clamping operation again. Therefore, if the clamping operation itself of the AFE 102 can be stopped, the lateral smear can be corrected with high quality more easily without using the circuit configuration shown in the fourth embodiment.

  In this embodiment, the clamp operation in the AFE 102 is controlled. However, in order to improve such controllability, the AFE does not perform the HOB clamp itself, and the digital processing is completely performed inside the DSP. You may make it the structure which clamps. In this case, since the transmission of the control signal from the row direction correction circuit 1033 to the clamp circuit is closed inside the DSP, an equivalent result can be obtained without affecting the circuit configuration or wiring of the entire imaging apparatus. It becomes possible.

(Sixth embodiment)
In the present embodiment, a method for further improving accuracy in order to predict whether or not lateral smear has occurred will be described.

  FIG. 15 is a diagram showing a configuration of the row direction correction circuit in the present embodiment. In FIG. 15, the pixel output equal to or higher than the first determination value set to the determination value 1 by the effective unit is transferred to the calculation unit 523, unlike the configuration in the row direction correction circuit so far. In the calculation unit 523, only a pixel output equal to or higher than the first determination value set to the determination value 1 is input, and a behavior higher than the maximum value of the A / D conversion result of the CMOS image sensor output is predicted from the image data. Thus, the lateral smear generation region is predicted with high accuracy.

  This calculation method will be described with reference to FIG. Assume that a high-luminance subject as shown in FIG. 16 is incident. An area 1601 indicates an area whose output level is equal to the determination value 1. An area 1602 indicates an area that takes the maximum value of the A / D conversion result. An area 1603 is an area that is indistinguishable from the area 1602 as digital data but has reached a signal level that actually generates a lateral smear inside the CMOS image sensor. Therefore, the calculation unit 523 aims to accurately calculate the number of pixels in the region 1602.

  The output of each pixel in the horizontal direction in the nth row shown in FIG. 16 is shown at the bottom. The output of each pixel input to the calculation unit 523 is a value indicated by a solid line. The calculation unit 523 predicts and calculates the number of pixels that are expected to reach the horizontal smear generation level from the output gradient between the determination value 1 and the maximum value of the A / D conversion.

  For example, by performing linear approximation based on the horizontal coordinate of determination value 1 and the horizontal coordinate of the edge portion of the A / D maximum value, the horizontal coordinate that crosses the known horizontal smear occurrence level is obtained. Can do. By performing this in both the left and right directions of the high-luminance subject, it is possible to calculate the number of pixels in the horizontal direction exceeding the horizontal smear generation level (the range of the double arrows in FIG. 16).

  When the number of pixels exceeding the assumed horizontal smear generation level exceeds the determination value 2, the comparator 513 is inverted to generate a horizontal smear generation signal. This is latched by the F / F 514 and the correction coefficient 2 is selected by the selector 507.

  On the other hand, in the m-th row, it can be determined that there is no pixel exceeding the horizontal smear generation level even if the calculation is performed from the gradient of the pixel data input to the arithmetic circuit. In this case, since the number of pixels exceeding the assumed horizontal smear generation level does not exceed the determination value 2, the comparator 513 is not inverted and a horizontal smear generation signal is not generated.

  There are various methods for calculating the number of pixels that are considered to exceed the known horizontal smear generation level from this image data, and any method may be used, but a row in which horizontal smear occurs is calculated with higher accuracy. It becomes possible.

  Further, the lateral smear generation level can be obtained by confirming in advance the lateral smear occurrence state of the CMOS image sensor to be used by changing the luminance of the incident light source, the area of the incident light, or the like. For example, as a horizontal smear generation level, a value equivalent to twice the A / D conversion maximum value can be selected, and a value of about 1/8 of the number of pixels in the horizontal direction can be selected as the number of pixels of the determination value 2. Of course, these values change (switchable) depending on the ISO sensitivity at the time of shooting.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

Claims (17)

  The effective pixel based on the number of pixels for each row that outputs a signal value larger than the first determination value among the output signals of the effective pixel portion of the imaging device having a plurality of pixels and the output signal of the light-shielding pixel portion of the imaging device. A signal processing apparatus comprising correction means for correcting the output signal of each section for each row. First comparison means for comparing an output signal of an effective pixel portion of an imaging device having a plurality of pixels with the first determination value;
Counting means for counting, for each row, the number of pixels that output a signal value larger than the first determination value based on a comparison result by the first comparison means;
2. The signal according to claim 1, wherein the correction unit corrects the output signal of the effective pixel unit for each row based on the output signal of the light-shielding pixel unit of the image sensor and the counting result of the counting unit. Processing equipment.   Second comparison means for comparing the counting result by the counting means with a second determination value is provided, and the correction means corrects the output signal of the effective pixel unit based on the comparison result by the second comparison means. The signal processing apparatus according to claim 2, wherein:   Third comparison means for comparing a counting result by the counting means with a second determination value and a third determination value, and the correction means, based on the comparison result by the third comparison means, the effective pixel The signal processing apparatus according to claim 2, wherein an output signal of the unit is corrected.   The correction means is a switching means for switching the gain based on the means, a multiplication means for multiplying the output signal of the light-shielding pixel unit for each row and the gain switched by the switching means, and the calculation calculated for each row. Subtracting means for subtracting the multiplication result of the multiplication means from the output signal of the effective pixel portion of each row, and performing correction by subtracting the subtraction result from the output signal of the effective pixel portion of each row. The signal processing device according to any one of claims 1 to 4.   The imaging apparatus according to claim 2, wherein the counting unit considers continuity in a horizontal direction of pixels that output a signal value larger than the first determination value.   The imaging apparatus according to claim 2, wherein the counting unit considers continuity in a vertical direction of pixels that output a signal value larger than the first determination value.   The signal processing apparatus according to claim 1, wherein the first determination value is switchable according to ISO sensitivity at the time of shooting.   The signal processing apparatus according to claim 1, wherein the first determination value can be switched according to an output for each color of the subject.   The signal processing apparatus according to claim 1, wherein the first determination value can be switched according to an incident position of a high-luminance subject.   The signal processing apparatus according to claim 2, wherein the second determination value can be switched according to ISO sensitivity at the time of shooting.   The signal processing apparatus according to claim 3, wherein the second determination value is switchable according to an output for each color of the subject.   The signal processing apparatus according to claim 3, wherein the second determination value can be switched according to an incident position of a high-luminance subject.   The signal processing apparatus according to claim 5, wherein the gain is switchable according to ISO sensitivity at the time of shooting.   55. The signal processing apparatus according to claim 54, wherein the gain can be switched according to an output for each color of a subject.   The signal processing apparatus according to claim 5, wherein the gain can be switched according to an incident position of a high-luminance subject.   An imaging device comprising: an imaging device having a plurality of pixels; and the signal processing device according to claim 1.

Images