JP2007053464A - Dark shading elimination device and imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a dark shading elimination device the memory capacity of which required for eliminating a dark shading component can be reduced and to provide an imaging apparatus. <P>SOLUTION: The elimination device includes: a storage means 11 for storing coefficients "a" to "d" of a function for denoting the characteristic of the dark shading component included in an output of an imaging device 22; calculation means 12, 13 for calculating the dark shading component in response to a photographing condition according to the function above, on the basis of the condition of photographing using the imaging device and the coefficients stored in the storage means; and a subtraction means 14 for subtracting a result of the calculation by the calculation means from an output obtained by exposing the imaging device under the photographing condition. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a dark shading removal apparatus and an imaging apparatus that remove dark shading components included in the output of an imaging element.

In order to remove dark shading components contained in the output of the image sensor, it is proposed to store shading correction data for at least one line and subtract the shading correction data from the output during exposure (for example, patents). Reference 1).
JP 2003-244513 A

However, the above-described apparatus has a large memory capacity necessary for storing shading correction data, and there is a limit to the reduction thereof.
An object of the present invention is to provide a dark shading removal apparatus and an imaging apparatus capable of reducing the memory capacity necessary for removing dark shading components.

  The dark shading removal apparatus according to the present invention stores a coefficient of a function representing a characteristic of a dark shading component included in an output of an image sensor, a condition for photographing using the image sensor, and the memory means. And calculating means for calculating the dark shading component according to the shooting condition according to the function based on the coefficient, and the output under the shooting condition obtained when the imaging element is exposed. And subtracting means for subtracting the calculation result by the calculating means.

In addition, it is preferable that the calculation unit calculates the dark shading component immediately after the exposure of the imaging element is completed and immediately before the output of the exposure is read out.
In addition, it is preferable that detection means for detecting a temperature at the time of photographing is provided, and the calculation means calculates the dark shading component in consideration of the temperature detected by the detection means among the photographing conditions.

The function preferably includes at least one of a horizontal position and a vertical position of the image sensor as a variable.
An image pickup apparatus according to the present invention includes an image pickup element that picks up a subject image, and the above-described dark shading removal apparatus that is disposed at a subsequent stage of the image pickup element.

  According to the present invention, it is possible to reduce the memory capacity necessary for removing the dark shading component.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
(First embodiment)
As shown in FIG. 1, the dark shading removal apparatus 10 of this embodiment is incorporated in an imaging apparatus 20 such as a digital camera. In addition to the dark shading removal device 10, the imaging device 20 is provided with a photographing lens 21, an imaging element 22, an analog front end (AFE) 23, and the like. The dark shading removal apparatus 10 is disposed at the subsequent stage of the AFE 23.

  Light from the subject enters the image sensor 22 through the taking lens 21. A subject image is formed on the imaging surface of the imaging element 22. The image sensor 22 is, for example, a CCD sensor or a CMOS sensor, and captures a subject image formed on the imaging surface. The output of the image sensor 22 is guided to the subtraction unit 14 of the dark shading removal apparatus 10 of the present embodiment as digital data after passing through the AFE 23. The output at the time of exposure of the image sensor 22 is obtained by superimposing a dark shading component on a signal component corresponding to a subject image.

  In the dark shading removal apparatus 10 of the present embodiment, the output of the image sensor 22 is guided to the plus input (+) of the subtracting unit 14, and the minus input (−) of the subtracting unit 14 has an appropriate level according to the shooting conditions. The dark shading component is derived. In order to calculate the dark shading component, the dark shading removal apparatus 10 of the present embodiment is provided with a memory 11, a correction unit 12, and a calculation unit 13.

  Here, the characteristic of the dark shading component included in the output of the image sensor 22 (hereinafter referred to as “DS characteristic”) is a horizontal direction or a vertical direction on the imaging surface of the image sensor 22 as shown in FIGS. Alternatively, the shape is such that the level decreases monotonously along the oblique direction. Further, the DS characteristic shows a substantially constant level in a direction (vertical direction in FIG. 2A) orthogonal to the decreasing direction (horizontal direction in FIG. 2A).

  Thus, the DS characteristic has a unidirectional property. Further, the level decreasing direction varies depending on the structure of the image sensor 22. Members that cause dark shading components are the image sensor 22 itself and peripheral circuits such as a driver of the image sensor 22. For example, in the case of a CCD sensor, a dark shading component is generated mainly by a horizontal driver or an output amplifier in the sensor as a heat source. In the case of a CMOS sensor, a dark shading component is generated mainly in the propagation path inside the image sensor 22.

Furthermore, even if the element structures are the same, the DS characteristics (size and inclination) are different due to individual differences of the imaging elements 22. Also, even with one image sensor 22, the DS characteristics change according to the shooting conditions. For example, since the gain of the AFE 23 at the subsequent stage of the image sensor 22 is adjusted according to the ISO sensitivity at the time of shooting, the DS characteristic changes according to the gain.
Therefore, in this embodiment, the removal of the dark shading component will be described by taking as an example a case where the DS characteristic of the image sensor 22 changes according to the ISO sensitivity at the time of shooting. The directionality of the dark shading component is assumed to decrease in the horizontal direction as shown in FIG. In this case, the DS characteristic in the vertical direction shows a substantially constant level. The directionality of the dark shading component is specified when the image sensor 22 is developed.

  Further, in order to grasp the basic form of the DS characteristic, in the manufacturing process of the image sensor 22, dark photography is performed, and actual data (one screen) of the dark shading component is acquired based on the output of the image sensor 22. Then, in consideration of the directionality of the dark shading component, the data are integrated and averaged along a direction indicating a substantially constant level (here, the vertical direction). As a result, the basic form (see FIG. 2A) of the DS characteristic unique to each image sensor 22 can be obtained as data for one line.

In the manufacturing process of the image sensor 22, data of the basic form of the DS characteristic (see FIG. 2A) is approximated by a polynomial of the following expression (1) and used as a function representing the DS characteristic. In Equation (1), Z is the level of the dark shading component, X is a variable corresponding to the horizontal position of the image sensor 22, and a to d are coefficients.
Z = a + bX + cX ² + dX ³ (1)
In order to obtain the coefficients a to d corresponding to the data of the basic form of the DS characteristic (see FIG. 2A), each value of data (level Z of the dark shading component) at each position X in the horizontal direction is expressed by the equation (1). ) And calculate using the least squares method. The coefficients a to d obtained by this calculation are, for example, a = e, b = −f, c = f / 2, and d = −f / 6 using arbitrary constants e and f. These coefficients a to d are stored in the memory 11 (FIG. 1) of the dark shading removal apparatus 10 of the present embodiment.

In the present embodiment, the image sensor 22 is exposed in accordance with the release operation of the image capturing apparatus 20 (FIG. 2), and then the output (exposure data) of the image sensor 22 is read. The exposure data read from the image sensor 22 is guided to the plus input (+) of the subtractor 14 through the AFE 23 as described above.
Furthermore, in the present embodiment, the dark shading removal apparatus 10 performs the following processing between immediately after the release and immediately before reading the exposure data (preferably immediately after the exposure is completed and immediately before the exposure data is read). Then, an appropriate level of dark shading component is calculated in accordance with the conditions of photographing using the image sensor 22 (here, ISO sensitivity), and is led to the minus input (−) of the subtractor 14.

  That is, the dark shading removal apparatus 10 reads the coefficients a to d from the memory 11 and inputs them to the correction unit 12 at a predetermined timing after the release. The correction unit 12 corrects the coefficients a to d according to the ISO sensitivity at the time of shooting. Specifically, the coefficient G corresponding to the ISO sensitivity is multiplied by each of the coefficients a to d (a ← a · G, b ← b · G, c ← c · G, d ← d · G), and the correction is performed. I do. Then, the corrected coefficients a to d are set in the calculation unit 13.

For this reason, the calculation unit 13 performs calculation processing according to the function of the following equation (2), calculates the level Z of the dark shading component corresponding to each position X in the horizontal direction, and inputs the minus input ( Output to-). The coefficient G in Expression (2) corresponds to a value corresponding to the ISO sensitivity at the time of shooting, and the coefficients a to d correspond to values read from the memory 11.
Z = G (a + bX + cX ² + dX ³ ) (2)
This level Z is an appropriate level according to the ISO sensitivity at the time of photographing, and is obtained by multiplying the level of the basic form of the DS characteristic (see FIG. 2A) by G times. FIG. 4A illustrates the level Z calculated from the function of Expression (2). In FIG. 4A, the basic shape of FIG. 2A is indicated by a dotted line for comparison. The coefficient G is not limited to the case of G> 1 illustrated in FIG.

  The correction unit 12, the calculation unit 13, and the subtraction unit 14 of the dark shading removal apparatus 10 are configured by a signal processing LSI. Then, the calculation by the function of Expression (2) is performed for each position X in the horizontal direction in synchronization with the reading of the exposure data from the image sensor 22 described above. For this reason, the subtracting section 14 has a level at an arbitrary position X of the exposure data (FIG. 4B) and a level Z calculated according to the position X (calculation result of the function of the expression (2)). Are input simultaneously.

  The subtracting unit 14 subtracts the level Z of the dark shading component guided to the minus input (−) from the level of the exposure data guided to the plus input (+). Such subtraction processing is performed for each position X in the horizontal direction. As a result, the dark shading component included in the exposure data is removed, and final imaging data including signal components corresponding to the subject image shown in FIG. 4C can be obtained.

As described above, in the dark shading removal apparatus 10 of the first embodiment, only the coefficients a to d of the function (expression (1)) representing the DS characteristics can be stored as the memory 11 necessary for removing the dark shading component. A small capacity memory can be used. Therefore, the capacity of the memory 11 can be reduced, and the size of the component, the cost reduction, and the power saving can be realized.
The coefficients a to d stored in the memory 11 may be fixed values obtained at the time of manufacturing the image sensor 22 or may be updated to optimum values according to the secular change of the image sensor 22.
(Modification of the first embodiment)
In the first embodiment described above, the example in which the expression (1) is used as a function representing the DS characteristic and the coefficients a to d are stored in the memory 11 has been described. However, the present invention is not limited to this. The following equation (3) may be used as a function representing the DS characteristics, and the coefficients A and B may be stored in the memory 11. As the coefficients A and B, for example, arbitrary constants e and f are used, and A = e and B = f.

Z = A−BX + (B / 2) X ² − (B / 6) X ³ (3)
In this case, the correction unit 12 multiplies the coefficients A and B read from the memory 11 by the coefficient G corresponding to the ISO sensitivity (A ← A · G, B ← B · G), and performs the correction. The calculation unit 13 performs calculation processing according to a function (equation (4)) using the corrected coefficients A and B.
Z = G {A−BX + (B / 2) X ² − (B / 6) X ³ } (4)
Further, in the first embodiment described above, the case where the DS characteristic of the image sensor 22 changes according to the ISO sensitivity at the time of shooting has been described as an example, but the present invention is not limited to this. When the DS characteristic changes in accordance with the temperature at the time of shooting, it is preferable to calculate a dark shading component of an appropriate level taking into account the temperature at the time of shooting and to guide it to the minus input (−) of the subtracting unit 14. When the DS characteristic changes according to both ISO sensitivity and temperature at the time of shooting, it is preferable to calculate a dark shading component of an appropriate level according to the combination of ISO sensitivity and temperature. Furthermore, the present invention can also be applied when combining other photographing conditions as necessary.
(Second Embodiment)
Here, the above equation (3) is used as a function representing the DS characteristic of the image sensor 22, and the coefficients A and B are expressed by the function of the temperature T at the time of photographing (equations (5) and (6)). The case where the function coefficients α _A , β _A , γ _A , α _B , β _B , γ _B are stored in the memory 31 (FIG. 5) will be described.

A = α _A + β _A T + γ _A T ² (5)
B = α _B + β _B T + γ _B T ² (6)
In order to obtain these coefficients α _A , β _A , γ _A , α _B , β _B , γ _B , in the manufacturing process of the image sensor 22, dark imaging is repeated while changing the temperature T, and the basic form of the DS characteristics Is acquired (FIG. 6A). Thereafter, the DS characteristic data at each temperature T is approximated by the above equation (3) to obtain the coefficients A and B. The temperature dependence of the coefficients A and B is as shown in FIGS. 6B and 6C, for example.

Then, the data of FIG. 6B is approximated by the above equation (5) to obtain the coefficients α _A , β _A , γ _A, and the data of FIG. 6C is approximated by the above equation (6). Thus, the coefficients α _B , β _B , γ _B are obtained, and the obtained coefficients α _A , β _A , γ _A , α _B , β _B , γ _B are stored in the memory 31 (FIG. 5).
A dark shading removal apparatus 30 according to the second embodiment includes a memory 31 shown in FIG. 5 instead of the memory 11, the correction unit 12, and the calculation unit 33 (FIG. 1) of the dark shading removal apparatus 10 according to the first embodiment. A correction unit 32 and a calculation unit 33 are provided, and a temperature sensor 34 is provided to detect a temperature T at the time of photographing. Here, it is assumed that the arrangement of the temperature sensor 34 is in the vicinity of a member (for example, a horizontal driver in the case of a CCD sensor) that causes dark shading components. In this case, the temperature detected by the temperature sensor 34 is substantially equal to the temperature T at the time of shooting.

  In the dark shading removal apparatus 30 of the second embodiment, shooting using the image sensor 22 is performed immediately after the release until immediately before the exposure data is read (preferably immediately after the exposure is completed and immediately before the exposure data is read). The dark shading component of an appropriate level corresponding to the above condition (here ISO sensitivity and temperature T) is calculated and led to the minus input (−) of the subtracting unit 14.

Specifically, the coefficients α _A , β _A , γ _A , α _B , β _B , γ _B are read from the memory 31 and input to the correction unit 32 at a predetermined timing. The correction unit 32 corrects the coefficients α _A , β _A , γ _A , α _B , β _B , and γ _B by multiplying by a coefficient G corresponding to the ISO sensitivity at the time of shooting (α _A ← α _A · G, β _A ← β _A · G, γ _A ← γ _A · G, α _B ← α _B · G, β _B ← β _B · G, γ _B ← γ _B · G). Then, the corrected coefficients α _A , β _A , γ _A , α _B , β _B , γ _B are set in the calculation unit 33.

For this reason, the calculation unit 33 first calculates the coefficients A and B by substituting the temperature T detected by the temperature sensor 34 into the following equations (7) and (8). The coefficient G in the equations (7) and (8) is a value corresponding to the ISO sensitivity at the time of photographing, and the coefficients α _A , β _A , γ _A , α _B , β _B , and γ _B are values read from the memory 31. Equivalent to.
A = G (α _A + β _A T + γ _A T ² ) (7)
B = G (α _B + β _B T + γ _B T ² ) (8)
Thereafter, the obtained coefficients A and B are substituted into the above equation (3), and an arithmetic process is performed according to this function to calculate the level Z of the dark shading component corresponding to each position X in the horizontal direction. 14 to the negative input (−). This level Z is an appropriate level according to the ISO sensitivity and temperature T at the time of shooting.

  Also in the dark shading removal apparatus 30 of the second embodiment, the subtracting unit 14 determines the level at an arbitrary position X of the exposure data (see FIG. 4B) and the level Z (equation) calculated corresponding to the position X. (3), (7), and (8) calculation results) are subtracted for each horizontal position X. As a result, the dark shading component included in the exposure data is removed, and final imaging data (see FIG. 4C) including a signal component corresponding to the subject image can be obtained.

As described above, in the dark shading removal apparatus 30 according to the second embodiment, the coefficient α of the function (expressions (3), (5), (6)) representing the DS characteristics is used as the memory 31 necessary for removing the dark shading component. _{a, β a, γ a,} α B, β B, γ B only can be used memory capable of storing a small capacity. Accordingly, the capacity of the memory 31 can be reduced, and the size of the component, the cost reduction, and the power saving can be realized.

The coefficients α _A , β _A , γ _A , α _B , β _B , and γ _B stored in the memory 31 may be fixed values obtained at the time of manufacturing the image sensor 22, or according to the secular change of the image sensor 22. It may be possible to update to an optimum value.
(Modification of the second embodiment)
In the second embodiment described above, the temperature sensor 34 is arranged in the vicinity of a member (for example, a horizontal driver in the case of a CCD sensor) that causes the generation of dark shading components. However, such arrangement may be difficult. . In order to accurately remove the dark shading component, it is preferable to grasp the temperature in the vicinity of the member that causes the occurrence of the dark shading component as the temperature T at the time of photographing regardless of the arrangement of the temperature sensor 34.

For this reason, the temperature detected by the temperature sensor 34 is converted into the temperature of the member that causes the occurrence of the dark shading component, and this is substituted into the above equations (7) and (8) as the temperature T at the time of photographing. Is preferred.
Therefore, as shown in FIG. 7, the relationship (T ′ (t)) between the energization time t for the image sensor 22 and the detected temperature T ′ by the temperature sensor 34 is obtained in advance, and similarly, the energization time t and the dark A relationship (T (t)) with the temperature T of the member that causes the generation of shading components was also obtained in advance, and a temperature conversion equation (Equation (9)) was created.

T (t) = T ′ (t) −K1 · t + (M2−M1) + K2 · t (9)
In Equation (9), K1 and M1 represent the characteristics of the detected temperature T ′ of the temperature sensor 34, and K2 and M2 represent the characteristics of the temperature T of the member. The farther the temperature sensor 34 is from the member, the smaller the value of the inclination K1. In addition, the values of the slopes K1 and K2 are considered to show a constant value regardless of the change in the ambient temperature of the imaging device. Further, it is considered that the temperature difference (M2−M1) at the energization time t = 0 also shows a constant value regardless of the change in the ambient temperature of the imaging device.

  The energization time t corresponds to, for example, an elapsed time from the timing of the release operation, and is determined when the exposure to the image sensor 22 is completed. However, the reference for the energization time t is not limited to the timing of the release operation. Depending on the structure of the image pickup element 22, for example, the timing at which the image pickup apparatus is started may be a reference for the energization time t regardless of the release operation. Even in this case, the energization time t is determined when the exposure on the image sensor 22 is completed.

In this modification, when the energization time t is determined (that is, immediately after the exposure to the image sensor 22 is completed and immediately before the exposure data is read), the energization time t and the temperature T ′ detected by the temperature sensor 34 are set. Substituting into the above conversion equation (9), the temperature T at the time of photographing is obtained. Thereafter, the temperature T is substituted into the above equations (7) and (8) to obtain the coefficients A and B. Therefore, the dark shading component can be accurately removed even when the temperature sensor 34 is arranged at a location away from the member that causes the generation of the dark shading component (for example, a horizontal driver in the case of a CCD sensor).
(Overall modification)
In the embodiment described above, the case where the level of the dark shading component of the image sensor 22 decreases along the horizontal direction (FIG. 2A) has been described as an example, but the present invention is not limited to this. In addition, the present invention is applied to the case where the level of the dark shading component decreases along the vertical direction (FIG. 2B) or decreases along the oblique direction (FIG. 2C). The effect of can be obtained.

In the case of the vertical direction, a variable Y with respect to the position of the imaging element 22 in the vertical direction may be used instead of the variable X in the above formula (1). In the case of the oblique direction, a two-dimensional equation (10) using variables X and Y may be used instead of the above equation (1).
Z = a _X + b _X X + c _X X ² + d _X X ³ + a _Y + b _Y Y + c _Y Y ² + d _Y Y ³ (10)
In this case, the coefficients a _{X to} d _X and the coefficients a _{Y to} d _Y may be stored in the memory 11 or the memory 31. The other formulas (2) to (4) may be considered similarly.

It is a block diagram which shows the structure of the dark shading removal apparatus 10 of 1st Embodiment. It is a figure explaining the characteristic (DS characteristic) of a dark shading component. FIG. 6 is a diagram for explaining the timing of exposure / reading of the image sensor 22 and reading / correcting / setting of coefficients in the dark shading removal apparatus 10. It is a figure explaining the gain process in the correction | amendment part 12, and the subtraction process in the subtraction part. It is a block diagram which shows the structure of the dark shading removal apparatus 30 of 2nd Embodiment. It is a figure explaining the temperature dependence (a) of DS characteristic, and the temperature dependence (b), (c) of the coefficients A and B of Formula (3). It is a figure explaining the temperature characteristic used for preparation of temperature conversion formula (9).

Explanation of symbols

10,30 dark shading removal device; 11,31 memory;
12, 32 correction unit; 13,33 calculation unit; 14 subtraction unit; 22 image sensor 34 temperature sensor

Claims (5)

Storage means for storing a coefficient of a function representing a characteristic of a dark shading component included in the output of the image sensor;
A calculation unit that calculates the dark shading component according to the shooting condition according to the function based on a shooting condition using the imaging element and the coefficient stored in the storage unit;
A dark shading removal apparatus comprising: a subtracting unit that subtracts a calculation result by the calculating unit from the output under the photographing condition obtained when the image sensor is exposed. In the dark shading removal apparatus of Claim 1,
The dark shading removal apparatus characterized in that the calculation means calculates the dark shading component immediately after the exposure of the image sensor is completed and immediately before the output of the exposure is read. In the dark shading removal apparatus of Claim 1,
Equipped with detection means to detect the temperature during shooting,
The dark shading removal apparatus, wherein the calculation means calculates the dark shading component in consideration of the temperature detected by the detection means among the photographing conditions. In the dark shading removal apparatus of Claim 1,
The dark shading removing apparatus, wherein the function includes at least one of a horizontal position and a vertical position of the image sensor as a variable. An image sensor for capturing a subject image;
An imaging apparatus comprising: the dark shading removal apparatus according to any one of claims 1 to 4, which is disposed at a subsequent stage of the imaging element.

Images