JP5144286B2 - Imaging apparatus, control method thereof, and program - Google Patents

Description

  The present invention relates to a technique for correcting fixed pattern noise of an image sensor.

  In the field of recent two-dimensional image sensors, demands for a large number of pixels and high-speed imaging are further expanded, and high-speed signal processing is required. For this reason, it is a problem to increase the speed of a circuit for processing an analog signal output from a pixel.

  Therefore, in recent years, signals from pixels are read simultaneously for each column, and subsequent horizontal transfer is not consolidated into one, and a plurality of amplifiers are provided as output stages. Supports imaging.

  However, even though these parallel processing circuits are designed in the same way in layout design, their characteristics vary due to variations in the manufacturing process. This variation in characteristics appears as streaky fixed pattern noise in the output image.

Conventionally, a method for detecting and correcting fixed pattern noise from a video signal in a light-shielding region of an image sensor as in Patent Document 1 is known.
Japanese Patent Laying-Open No. 2005-167918

  However, when the fixed pattern noise is detected and corrected from the video signal in the light shielding area, there is a difference in the fixed pattern noise between the light shielding area and the effective pixel area depending on the difference between the states of the light shielding area and the effective pixel area. There was a problem of reduction.

  For example, if the light-shielding area is arranged in the periphery of the effective pixel area, the fixed pattern noise between the light-shielding area and the effective pixel area may be different due to the difference between the arrangement positions of the light-shielding area and the effective pixel area. is there. As a main cause of the noise difference due to such an arrangement position difference, electrical shading or the like can be considered. That is, the correction effect is reduced by the difference between the arrangement positions of the light shielding area and the effective pixel area.

  In addition, when the noise amount of the effective pixel region depends on the input light amount, the fixed pattern noise detected from the light shielding region that is not affected by the incident light amount may not be able to cope with the noise amount that changes depending on the incident light amount. That is, the correction effect is reduced due to the difference in light amount that the effective pixel region changes in output signal depending on the input light amount, but the light shielding region does not change.

  Therefore, the present invention has been made in view of the above-described problems, and an object of the present invention is to always be able to correct fixed pattern noise with high accuracy.

In order to solve the above-described problems and achieve the object, an imaging apparatus according to the present invention includes an imaging device having a light-shielded pixel region that is optically shielded and an effective pixel region that is not shielded from light, and the light-shielded pixel. First detection means for detecting first fixed pattern noise generated in the image sensor from projection data in the vertical direction of the video signal output from the area, and the first detection means detected by the first detection means. Based on the fixed pattern noise, a first correction value for correcting the first fixed pattern noise to be removed from the video signal output from the effective pixel region is calculated, and the imaging element is fixed. First correction means for subtracting the first correction value from the video signal output from the effective pixel area in the exposure state, and the video signal of the effective pixel area corrected by the first correction means of The straight direction of the projection data, a second detecting means for detecting a second fixed pattern noise occurring in the imaging element, based on the detected second fixed pattern noise by the second detecting means, A second correction value for correcting so as to remove the second fixed pattern noise from the video signal of the effective pixel region corrected by the first correction unit is calculated, and the first correction unit Second correction means for multiplying the corrected video signal of the effective pixel area by the second correction value.

According to another aspect of the present invention, there is provided a method for controlling an image pickup apparatus, the method comprising: controlling an image pickup apparatus including an image pickup element having a light-shielded pixel area that is optically shielded and an effective pixel area that is not shielded from light A first detection step for detecting a first fixed pattern noise generated in the image sensor from projection data in the vertical direction of the video signal output from the pixel region, and the first detection step detected in the first detection step. Based on the fixed pattern noise of 1, a first correction value for correcting the first fixed pattern noise to be removed from the video signal output from the effective pixel region is calculated, and the image sensor is fixed. A first correction step of subtracting the first correction value from the video signal output from the effective pixel region when the exposure state is set, and an image of the effective pixel region corrected in the first correction step Trust From the vertical projection data, a second detection step of detecting a second fixed pattern noise occurring in said image pickup element, based on the second said detected by the detecting step the second fixed pattern noise Calculating a second correction value for correcting so as to remove the second fixed pattern noise from the video signal of the effective pixel region corrected in the first correction step; and the first correction step. And a second correction step of multiplying the video signal of the effective pixel area corrected in step 2 by the second correction value.

  According to the present invention, it is possible to always correct fixed pattern noise with high accuracy.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments of the invention will be described in detail with reference to the accompanying drawings.

(First embodiment)
The first embodiment will be described with reference to FIGS. 1 to 3.

  FIG. 1 is a block diagram illustrating a configuration of an imaging apparatus according to a first embodiment of the present invention.

  In FIG. 1, reference numeral 1 denotes an exposure control unit including an aperture, a shutter, and the like that controls the exposure state of the image pickup device, and 2 denotes an image pickup device having a light-shielded region that is optically shielded and an exposed effective pixel region. Also, 3 is an analog front end (AFE) that amplifies the output signal of the image sensor and performs A / D conversion, and 4 is a fixed pattern noise from the signal of the light shielding region 2a of the image sensor 2 in the video signal output from the AFE 3. It is the 1st detection part to detect. Reference numeral 5 denotes a memory for storing the fixed pattern noise data detected by the first detection unit 4, and 6 denotes a subtraction of the fixed pattern noise data stored in the memory 5 from the signal of the effective pixel area 2 b of the image sensor 2. It is the 1st correction part which corrects. Reference numeral 7 denotes a second detection unit that detects fixed pattern noise from the signal in the effective pixel area of the image sensor in the output signal of the first correction unit 6, and 8 denotes a fixed pattern detected by the second detection unit 7. This is a memory for storing noise data. Reference numeral 9 denotes a second correction unit that corrects the fixed pattern noise data stored in the memory 8 by subtracting it from the signal output from the first correction unit 6. Reference numeral 10 denotes a camera signal processing circuit that performs signal processing on the video signal output from the second correction unit 9, and reference numeral 11 denotes a control unit that controls the exposure control unit 1 and the second detection unit 7.

  FIG. 2 is a schematic diagram showing a light shielding area (light shielding pixel area) and an effective pixel area of the image sensor. 2a represents a light shielding region, and 2b represents an effective pixel region. Fixed pattern noise generated due to variations in characteristics of each column in the image sensor appears as vertical streak noise as shown in FIG. The fixed pattern noise includes an offset noise 2c that does not depend on the input light amount and a gain noise 2d that is substantially proportional to the input light amount.

  FIG. 3 is a schematic diagram showing the amount of fixed pattern noise. The operation of this embodiment will be described with reference to FIG.

  First, the control unit 11 controls the exposure control unit 1 to put the image sensor 2 in a light-shielding state. The video signal output from the image sensor 2 is input to the first detection unit 4 and the first correction unit 6 via the AFE 3. At this time, because of the light shielding state, offset noise 2c (first fixed pattern noise) appears, and the fixed pattern noise amount in the effective pixel region 2b is represented by a graph 3a in FIG. In this graph, the horizontal axis represents the horizontal coordinate of the pixel area of the image sensor 2, and the vertical axis represents the average of the signal level of each horizontal pixel in the vertical direction.

  As shown in FIG. 3B, the first detection unit 4 generates (calculates) projection data 3b in the vertical direction of the video signal in the light shielding region 2a of the image sensor. Specifically, the average value of the video signals in each column may be used, or a cyclic filter may be configured using the memory 5. The projection data 3b is recorded as a correction value in the memory 5 and input to the first correction unit 6.

  Since the first detection unit 4 uses the light shielding region 2a as a detection target, it is possible to perform a detection operation regardless of the exposure state of the effective pixel region 2b. Therefore, it is possible to detect fixed pattern noise continuously even during actual shooting. The first correction unit 6 corrects (removes) the fixed pattern noise obtained from the light shielding region 2a by subtracting the correction value (first correction value) 3b from the video signal 3a of the effective pixel region 2b. . The corrected video signal is indicated by 3c in FIG. This video signal 3c still contains a correction remaining (second fixed pattern noise) due to the difference in arrangement position between the light shielding area 2a and the effective pixel area 2b.

  As shown in FIG. 3D, the second detection unit 7 generates projection data 3d in the vertical direction of the video signal of the effective pixel region 2b from the output signal of the first correction unit 6. This may also be the average value of the video signals in each column, or a cyclic filter may be configured using the memory 8. The projection data 3d is recorded in the memory 8 as a correction value used in the second correction unit 9.

  The projection data 3d is obtained from a video signal obtained by the exposure control unit 1 while the image sensor 2 is shielded from light. For this reason, it is desirable to obtain the image before the photographing in which the effective pixel region 2b is exposed. Therefore, after storing the correction value in the memory 8 in the light shielding state, the control unit 11 stops updating the detection operation of the second detection unit 7. The second correction unit 9 corrects the remaining correction of fixed pattern noise by subtracting the correction value (second correction value) 3d from the video signal 3c output from the first correction unit 6. As shown in FIG. 3E, the correction result is a state 3e in which the fixed pattern noise is sufficiently reduced.

  In this way, after the fixed pattern noise correction based on the detection in the light shielding area, the fixed pattern noise correction based on the detection of the light shielded state in the effective area is performed. Noise can be corrected with high accuracy.

(Second Embodiment)
Next, a second embodiment will be described with reference to FIGS. 4 and 5.

  The configuration of the imaging apparatus of the second embodiment shown in FIG. 4 is substantially the same as the configuration of the first embodiment shown in FIG. Description is omitted.

  In FIG. 4, reference numeral 8 denotes a memory for storing fixed pattern noise data detected by the second detection unit 7 and a correction value input to the second correction unit 9, and 9 denotes a first correction value stored in the memory 8. This is a second correction unit that multiplies and corrects the output signal from the correction unit 6. Reference numeral 12 denotes a calculation unit that reads fixed pattern noise data from the memory 8, calculates a correction value, and outputs the correction value to the memory 8.

  FIG. 2 is a schematic diagram showing a light shielding area and an effective pixel area of the image sensor. 2a represents a light shielding region, and 2b represents an effective pixel region. Fixed pattern noise generated due to variations in characteristics of each column in the image sensor appears as vertical streak noise as shown in FIG. The fixed pattern noise includes an offset noise 2c that does not depend on the input light amount and a gain noise 2d that is substantially proportional to the input light amount.

  FIG. 5 is a schematic diagram showing the noise amount and the correction value of the fixed pattern noise. The operation of this embodiment will be described with reference to FIG.

  First, the control unit 11 controls the exposure control unit 1 to bring the image sensor 2 into an exposure state. The video signal output from the image sensor 2 is input to the first detection unit 4 and the first correction unit 6 via the AFE 3. At this time, because of the exposure state, offset noise 2c (first fixed pattern noise) and gain noise 2d (second fixed pattern noise) appear, and the amount of fixed pattern noise in the effective pixel region is as shown in FIG. It is represented by a graph indicated by 5a in 5 (a). In this graph, the horizontal axis represents the horizontal coordinate of the pixel region of the image sensor 2, and the vertical axis represents the average of the signal level of each horizontal pixel in the vertical direction, and the optical black level at the time of light shielding is set to zero.

  As shown in FIG. 5B, the first detection unit 4 generates projection data 5b in the vertical direction of the video signal in the light shielding region 2a of the image sensor. Specifically, the average value of the video signals in each column may be used, or a cyclic filter may be configured using the memory 5. The projection data 5 b is recorded in the memory 5 as a correction value (first correction value) and input to the first correction unit 6.

  Since the first detection unit 4 uses the light shielding region 2a as a detection target, it is possible to perform a detection operation regardless of the exposure state of the effective pixel region 2b. Therefore, it is possible to detect fixed pattern noise continuously even during actual shooting. The first correction unit 6 corrects the fixed pattern noise obtained from the light shielding region 2a by subtracting the correction value 5b from the video signal 5a of the effective pixel region 2b. The corrected video signal is indicated by 5c in FIG. 5 (c). In this video signal 5c, offset noise 2c is corrected, but gain noise 2d caused by the difference in input light quantity remains.

  As shown in FIG. 5D, the second detection unit 7 generates projection data 5d in the vertical direction of the video signal in the effective pixel region 2b from the output signal of the first correction unit 6. This may also be the average value of the video signals in each column, or a cyclic filter may be configured using the memory 8. The projection data 5d is recorded in the memory 8.

  In order for the second detection unit 7 to detect the gain noise of the first correction unit 6, a video signal obtained by the exposure control unit 1 with the image sensor 2 in a fixed exposure state is required. For this reason, it is desirable to detect gain noise before shooting and recording operations are performed. Therefore, after storing the correction value in the memory 8 in a fixed exposure state, the control unit 11 stops updating the detection operation of the second detection unit 7.

  Next, the calculation unit 12 reads the projection data 5d from the memory 8, and calculates a correction value (second correction value) 5e shown in FIG. The correction value 5e is obtained by, for example, obtaining the ratio of the signal level of each column to the average value of the projection data and reciprocal thereof. In the projection data 5d shown in FIG. 5D, when the average value is A and the signal level of the gain noise 2d is αA, the correction value is 1 / α with respect to the position of the noise 2d, and the others are 1 become.

  The second correction unit 9 corrects the fixed pattern noise by multiplying the video signal 5c output from the first correction unit 6 and the correction value 5e. As shown in FIG. 5F, the correction result is a state 5f in which the fixed pattern noise is sufficiently reduced.

  In this way, after the fixed pattern noise correction by the detection in the light shielding area, the fixed pattern noise is corrected by the detection of the state exposed to a certain amount in the effective area. The fixed pattern noise can be corrected with high accuracy.

(Third embodiment)
Next, a third embodiment will be described with reference to FIGS.

  The configuration of the imaging apparatus of the third embodiment shown in FIG. 6 is substantially the same as the configuration of the first embodiment shown in FIG. Is omitted.

  In FIG. 6, reference numeral 8 denotes a memory for storing fixed pattern noise data detected by the second detection unit 7 and a correction value input to the second correction unit 9. Reference numeral 9 denotes a second correction unit that corrects a correction value stored in the memory 8 by subtracting and multiplying the signal output from the first correction unit 6. Reference numeral 12 denotes a calculation unit that reads fixed pattern noise data from the memory 8, calculates a correction value, and outputs the correction value to the memory 8.

  FIG. 2 is a schematic diagram showing a light shielding area and an effective pixel area of the image sensor. 2a represents a light shielding region, and 2b represents an effective pixel region. Fixed pattern noise generated due to variations in characteristics of each column in the image sensor appears as vertical streak noise as shown in FIG. The fixed pattern noise includes an offset noise 2c that does not depend on the input light amount and a gain noise 2d that is substantially proportional to the input light amount.

  FIG. 7 is a schematic diagram showing the noise amount and the correction value of the fixed pattern noise. The operation of this embodiment will be described with reference to FIG.

  First, the control unit 11 controls the exposure control unit 1 to put the image sensor 2 in a light-shielding state. The video signal output from the image sensor 2 is input to the first detection unit 4 and the first correction unit 6 via the AFE 3. At this time, because of the light shielding state, offset noise 2c appears, and the fixed pattern noise amount in the effective pixel region 2b is represented by a graph 7a in FIG. In this graph, the horizontal axis represents the horizontal coordinate of the pixel area of the image sensor 2, and the vertical axis represents the average of the signal level of each horizontal pixel in the vertical direction.

  As illustrated in FIG. 7B, the first detection unit 4 generates projection data 7 b in the vertical direction of the video signal in the light shielding region 2 a of the image sensor 2. Specifically, the average value of the video signals in each column may be used, or a cyclic filter may be configured using the memory 5. The projection data 7b is recorded in the memory 5 as a correction value and input to the first correction unit 6.

  Since the first detection unit 4 uses the light shielding region 2a as a detection target, it is possible to perform a detection operation regardless of the exposure state of the effective pixel region 2b. Therefore, it is possible to detect fixed pattern noise continuously even during actual shooting. The first correction unit 6 corrects the fixed pattern noise obtained from the light shielding region 2a by subtracting the correction value 7b from the video signal 7a of the effective pixel region 2b. The corrected video signal is indicated by 7c in FIG. This video signal 7c still contains a correction remaining due to a difference in arrangement position between the light shielding area 2a and the effective pixel area 2b.

  As shown in FIG. 7D, the second detection unit 7 generates projection data 7d in the vertical direction of the video signal in the effective pixel region 2b from the output signal of the first correction unit 6. This may also be the average value of the video signals in each column, or a cyclic filter may be configured using the memory 8. The projection data 7d is recorded in the memory 8 as a correction value of the second correction unit 9.

  The projection data 7d is obtained from a video signal obtained in a state where the image sensor 2 is shielded from light by the exposure control unit 1. After the correction value is stored in the memory 8 in the light-shielded state, the control unit 11 stops updating the detection operation of the second detection unit 7. The second correction unit 9 corrects the fixed pattern noise by subtracting the correction value 7d from the video signal 7c output from the first correction unit 6. As shown in FIG. 7E, the correction result is a state 7e in which the fixed pattern noise is sufficiently reduced.

  Next, the control unit 11 controls the exposure control unit 1 to place the image sensor 2 in the exposure state and restart the detection operation of the second detection unit 7. At this time, the output video signal of the first correction unit 6 is shown by 7f in FIG. The offset noise 2c is corrected, but the gain noise 2d caused by the difference in input light quantity remains.

  Next, as shown in FIG. 7G, the second detection unit 7 generates projection data 7g in the vertical direction of the video signal of the effective pixel region 2b from the output signal of the first correction unit 6. This may also be the average value of the video signals in each column, or a cyclic filter may be configured using the memory 8. The projection data 7g is recorded in the memory 8.

  In order to detect the gain noise of the signal output from the first correction unit 6 by the second detection unit 7, a video signal obtained by the exposure control unit 1 with the image sensor 2 in a constant exposure state is obtained. Necessary. Therefore, after storing the correction value in the memory 8 in a constant exposure state, the control unit 11 stops updating the detection operation of the second detection unit 7.

  Next, the calculation unit 12 reads the projection data 7g from the memory 8, and calculates the correction value 7h shown in FIG. For the correction value 7h, for example, the ratio of the signal level of each column to the average value of the projection data 7g is obtained and the reciprocal thereof. In 7g shown in FIG. 7 (g), when the average value is A and the signal level of the gain noise 2d is αA, the correction value is 1 / α with respect to the position of the noise 2d, and the others are 1. .

  The second correction unit 9 corrects the fixed pattern noise by multiplying the video signal 7g output from the first correction unit 6 and the correction value 7h. The correction result is a state 7i in which the fixed pattern noise is sufficiently reduced as shown in FIG.

  As described above, after the fixed pattern noise is corrected by the detection in the light shielding area, the fixed pattern noise is corrected by detecting the light shielding state and the state exposed to a certain amount in the effective area. Thereby, it is possible to accurately correct the fixed pattern noise caused by the difference in arrangement position and the difference in input light amount.

(Other embodiments)
The object of each embodiment is also achieved by the following method. That is, a storage medium (or recording medium) in which a program code of software that realizes the functions of the above-described embodiments is recorded is supplied to the system or apparatus. Then, the computer (or CPU or MPU) of the system or apparatus reads and executes the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention. Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but the present invention includes the following cases. That is, based on the instruction of the program code, an operating system (OS) running on the computer performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

  Furthermore, the following cases are also included in the present invention. That is, the program code read from the storage medium is written into a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer. Thereafter, based on the instruction of the program code, the CPU or the like provided in the function expansion card or function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

  When the present invention is applied to the above storage medium, the storage medium stores program codes corresponding to the procedure described above.

It is a block diagram which shows the structure of the imaging device of 1st Embodiment. It is a schematic diagram showing the light shielding area | region and effective pixel area | region of an image pick-up element. It is a schematic diagram showing the noise amount and correction value of the fixed pattern noise of the first embodiment. It is a block diagram which shows the structure of the imaging device of 2nd Embodiment. It is a schematic diagram showing the noise amount and correction value of the fixed pattern noise of 2nd Embodiment. It is a block diagram which shows the structure of the imaging device of 3rd Embodiment. It is a schematic diagram showing the noise amount and correction value of the fixed pattern noise of 3rd Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Exposure control part 2 Image sensor 3 AFE
4 first detection unit 5 memory 6 first correction unit 7 second detection unit 8 memory 9 second correction unit 10 camera signal processing circuit 11 control unit 12 calculation unit

Claims (3)

An image sensor having a light-shielded pixel region that is optically shielded and an effective pixel region that is not shielded;
First detection means for detecting first fixed pattern noise generated in the image sensor from projection data in the vertical direction of the video signal output from the light-shielding pixel region;
A first correction for removing the first fixed pattern noise from the video signal output from the effective pixel area based on the first fixed pattern noise detected by the first detection means. First correction means for subtracting the first correction value from the video signal output from the effective pixel area when the image sensor is in a constant exposure state,
Second detection means for detecting second fixed pattern noise generated in the image sensor from projection data in a vertical direction of the video signal of the effective pixel region corrected by the first correction means;
Based on the second fixed pattern noise detected by the second detection means, the second fixed pattern noise is removed from the video signal of the effective pixel area corrected by the first correction means. A second correction unit that calculates a second correction value for correction to the image signal and multiplies the video signal of the effective pixel region corrected by the first correction unit by the second correction value;
An imaging apparatus comprising: A method for controlling an imaging apparatus including an imaging device having a light-shielded pixel region that is optically shielded and an effective pixel region that is not shielded.
A first detection step of detecting a first fixed pattern noise generated in the imaging device from projection data in a vertical direction of the video signal output from the light-shielding pixel region;
A first correction for removing the first fixed pattern noise from the video signal output from the effective pixel area based on the first fixed pattern noise detected in the first detection step. A first correction step of subtracting the first correction value from the video signal output from the effective pixel area when the image sensor is in a constant exposure state,
A second detection step of detecting a second fixed pattern noise generated in the imaging device from projection data in a vertical direction of the video signal of the effective pixel region corrected in the first correction step;
Based on the second fixed pattern noise detected in the second detection step, the second fixed pattern noise is removed from the video signal in the effective pixel region corrected in the first correction step. A second correction step of calculating a second correction value for correction to the video signal of the effective pixel region corrected in the first correction step and the second correction value.
An image pickup apparatus control method comprising: A program for causing a computer to execute the control method according to claim 2 .

Images