JP2003169255A - Shading correction method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a shading correction method that applies optimum shading correction to a consumer camera system or the like at a limited cost (the number of gates). <P>SOLUTION: The method comprises capturing in advance data providing a uniform white reference at a sampling point on a horizontal (H) axis and a vertical (V) axis passing through a center point (center) of an effective imaging plane to multiply a correction coefficient with H and V coordinates on a correction approximate line (a curved line or a straight line) at each center axis thereby calculating the product as a shading correction coefficient on optional coordinates (H, V) within the effective imaging plane. Further, the method comprises dividing the effective pixel area of the imaging plane into mesh areas in the unit of prescribed pixels, integrating and averaging pixel values in each mesh area to calculate a correction value for each mesh area so as to correct both shading of a lens system and shading of uneven sensor sensitivity on the basis of the correction value for each mesh area. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a shading correction method for correcting shading of image information read by an image sensor.

[0002]

2. Description of the Related Art Hitherto, as for shading correction of a camera system, it has been proposed to use an expensive and high performance shading correction circuit for information equipment (see, for example, Japanese Patent No. 2805100). Shading correction is essential in the field of facsimiles and digital copying machines using linear sensors, and many correction methods have been proposed (for example, Japanese Patent No. 2580).
No. 118, etc.).

[0003]

However, in the field of consumer CCD solid-state image pickup devices and digital cameras using CMOS sensors, cost constraints are large, and therefore some correction accuracy can be obtained at some cost. A guaranteed shading correction method is preferable, but no suitable shading correction method has been proposed.

Therefore, an object of the present invention is to provide a shading correction method capable of performing optimum shading correction at a limited cost (number of gates) in a consumer camera system or the like.

[0005]

In order to achieve the above object, the present invention preliminarily captures uniform white reference data at sampling points on each of the horizontal and vertical center axes of the effective image pickup surface to obtain the white reference data. A correction approximate line in each of the center axes is calculated using the reference data, a correction coefficient for the horizontal coordinate and a correction coefficient for the vertical coordinate are multiplied based on the correction approximate line, and this multiplication value is used in the effective image pickup plane. Is calculated as a shading correction coefficient on arbitrary coordinates of.

According to the shading correction method of the present invention, the correction approximate line for each center axis is calculated using the white reference data previously captured at the sampling points on the horizontal and vertical center axes of the effective image pickup surface, and the correction approximate line is calculated. The correction coefficient for the horizontal coordinate is multiplied by the correction coefficient for the vertical coordinate based on to calculate the shading correction coefficient on the arbitrary coordinate. Therefore, the actual shading correction can be approximated with good accuracy by a relatively simple calculation method. can do.

[0007]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of a shading correction method according to the present invention will be described below. The embodiments described below are preferred specific examples of the present invention, and various technically preferable limitations are given. However, the scope of the present invention is not limited to the present invention in the following description. Unless otherwise stated, the present invention is not limited to these embodiments.

According to the first embodiment of the present invention, data which is a uniform white reference is sampled on a horizontal (H) axis and a vertical (V) axis passing through a center point (center) of an effective image pickup surface. By capturing in advance at points, the correction coefficient is multiplied by the H coordinate and V coordinate of the correction approximate line (curve or straight line) on each center axis, and this value is placed on the arbitrary coordinates (H, V) within the effective imaging plane. Is calculated as the shading correction coefficient.

Further, according to the second embodiment of the present invention, the effective pixel area of the image pickup surface is divided into mesh areas of a predetermined pixel unit, and the pixel values are integrated and averaged in each mesh area. Calculate the correction value for each mesh area,
Both of the shading of the lens system and the shading of the sensor sensitivity unevenness can be corrected based on the correction value for each mesh area. Further, the above-described correction approximate lines (curves or straight lines) in the H and V directions are switched for each mesh area, and the correction coefficient of any coordinate in the mesh area is calculated by the above-described shading correction coefficient calculation method. I tried to calculate based on it.

Specific examples for realizing such an embodiment will be described below with reference to the drawings. 1 to 9 are
It is a figure for demonstrating the 1st example which implements the 1st embodiment of the present invention, Drawing 1 is an explanatory view showing the shading coefficient calculation method in this example, and Drawing 2 is an explanation explaining calculation accuracy. FIG. 3, FIG. 3 is an explanatory diagram showing a CIF used in this example, FIG. 4 is an explanatory diagram mathematically demonstrating the validity of the calculation formula used in this example, and FIG. 5 is an explanation showing a corrected curved surface of lens shading in this example. FIG. 6 is a block diagram showing a data path of the shading correction circuit according to the present example, FIG.
Is a block diagram showing a control block of the shading correction circuit according to this example, and FIGS. 8 and 9 are flowcharts showing the operation of the shading correction circuit according to this example.

In the first embodiment, an example will be described in which the shading due to the lens is dominant as compared with the shading due to the uneven sensitivity of the image pickup device. In this case, a reference white image is captured at the camera adjustment stage, and
Sampling values on both the H and V center axes of the image pickup surface shown in (4) are loaded into the register. In this case, the sampling points on both axes may be fixed, or the number and intervals of the sampling points may be variably set by setting a register. Further, in this case, in order to facilitate the subsequent design of the H / V approximate linear block, the interval between sampling points can be selected to be Xn + 1-Xn = 2 ^m (m is a natural number).

For example, in the case of CIF having 362 × 293 effective pixels, sampling points are selected at intervals as shown in FIG. 3 such that 19 sampling points in the H direction and 17 sampling points in the V direction are obtained. be able to.

In this example, since the change in the shading correction curved surface due to the lens becomes larger toward the end of the lens and there is a deviation from the center of the optical axis of the lens, the interval between the sampling points becomes closer to the end and center of the lens. Is dense. As shown in FIG. 1, in this embodiment,
Assuming that the shading correction coefficient of the lens at the coordinates (H, V) of an arbitrary effective image pickup surface is P (H, V), this is calculated by the following relational expression. P (H, V) = CH (H) * CV (V) (Equation 1) Here, CH (H) is a correction coefficient for the coordinate H of the correction approximate straight line on the sensor axis in the H direction, and CV ( V)
Is a correction coefficient for the coordinate V of the correction approximate straight line on the center axis in the V direction.

FIG. 4 is a mathematical proof of the validity of the equation (1). In the geometrical arrangement as shown in FIG. 4, the amount of light on the image pickup element surface P (x, y) is calculated by (1) cosine quadruple rule and (2) biaxial multiplication and compared. As a result, it can be seen that when the exit Hitomi distance is sufficiently larger than 1/2 of the diagonal distance of the image sensor, the cosine quadruple rule = biaxial multiplication holds good approximation. Further, FIG. 2 is a graph in which the accuracy of the calculation formula (Formula 1) is examined. Here, the effective pixel surface is 5.3.
When using an image sensor of mm × 4.0 mm, the exit Hitomi distance from the lens to the imaging surface is 3.32 mm
(FIG. 2 (B)) and 3.32 mm × 2 (FIG. 2 (C)). In the figure, the curve described as the cosine fourth law is calculated on the assumption that the light amount ratio from the center of the image pickup to the diagonal positions of the four corners of the image pickup surface purely follows the cosine fourth law. In addition, the curve described as “double-axis multiplication correction” is based on the assumption that the correction curve of the center of both axes of the imaging surface follows the cosine fourth side, and from that value, the correction coefficient of both axes is multiplied by using (Equation 1). Is calculated as a diagonal correction coefficient.

As can be seen from FIG. 2, the exit Hitomi distance is the distance from the center of the imaging surface to the diagonal (3.3 in this case).
2 mm or more, there is almost no difference between the two.
When the exit Hitomi distance and 1/2 of the diagonal distance are equal to each other, the maximum error is about 20% at the four corners of the imaging surface. At present, in the case of microlenses mainly used in the market, in many cases, the relationship of (exit Hitomi distance)> (1/2 of imaging diagonal distance) of the mount is satisfied, so this approximation is valid from FIG. It is a thing. FIG. 5 is a corrected curved surface diagram of lens shading drawn by actually taking in white-based sampling data by the present shading circuit and calculating a correction coefficient calculated based on the sampling data by (Equation 1).

Next, a circuit configuration for realizing the above arithmetic processing will be described. FIG. 6 shows a shading correction circuit data path block according to this embodiment, and FIG. 7 shows a control block diagram thereof. In this embodiment, in order to reduce the number of gates of the circuit, the processing from the white reference data up to the correction coefficient calculation is performed by the control software according to the flowcharts of FIGS. 8 and 9. That is, after the white reference data for one screen is loaded onto the shading RAM, an external PC (personal computer) not shown reads the white reference data in the H / V direction from the shading RAM (steps S1 and S).
2).

At this time, the external PC retrieves the maximum value data (data of the center axis intersection) from all the sampling data and calculates (maximum value data) / (white sampling value data) to obtain the correction coefficient. Is calculated (step S3). Then, the data is written back to the shading RAM as U1.9 data and the value is stored in the E2PROM (steps S4 to S7).
Further, in this correction coefficient calculation process, as shown in FIG. 9, in order to obtain the ideal lens shading characteristics, the sampled points are fitted by the quadratic least squares method to obtain noise and flicker at the time of sampling. The influence of the above is eliminated (steps S11 to S14).
Of course, the process of taking the reciprocal of the white sampling data may be performed by using a divider, and the reciprocal of the integer value is RO.
The configuration may be such that the table is read from the table stored in M.

FIG. 6 shows a shading correction circuit data path block according to this embodiment.
(Low-pass filter) 10, shading RAM 1
2, a shifter 14, a system controller 16 and a correction coefficient data path. First, the operation in the H direction will be described. The correction coefficient values Yn + 1 and Yn necessary for calculating the correction coefficient of the point H to be interpolated are read from the shading RAM 12 to a register. next,
The correction amount Y of the current H count value is calculated by the following calculation formula as the formula of the linear approximation line. Y = Yn + Kn * (X-Xn) (Equation 2) where Kn = (Yn + 1-Yn) * 2- ^m . In addition,
X indicates the present count value of the H counter, and Xn, Xn +
1 indicates the sampling address of the H counter, Yn,
Yn + 1 indicates the correction coefficient in the H direction corresponding to the sampling points of Xn and Xn + 1.

The operation in the V direction is basically the same as the operation in the H direction. In this case, X indicates the current value of the V counter, Xn and Xn + 1 indicate the sampling address of the V counter, and Yn and Yn + 1 indicate the correction coefficient in the V direction corresponding to the V sampling point. . As described above, since the interval between sampling points is selected to be 2 ^m when obtaining the slope of the Kn straight line, this division can be performed by a shift operation. In addition, the configuration of this embodiment is V
Regarding the correction amount in the axial direction, the current correction amount of the V counter in the V direction is calculated during the H blanking period, and the value is latched and held for 1H period. ,
The correction coefficient data path is commonly used.

In the correction coefficient data path, first, (Equation 2)
Based on U, the difference amount of (Yn + 1-Yn) in S1.7 and U
Signed multiplication of the (X-Xn) difference amount of 7.0 is performed, and as a result, the output of S8.7 is obtained. Next, the shifter divides by 2 ^m (in this case, the shift amount is 4/5/6 /
7 bits) to obtain the output of S1.7. Finally, the interpolation value Y at the X point is obtained in the U1.7 format by adding Yn of the base point U1.7. The correction coefficients in the H direction and the V direction are multiplied, and finally become a correction coefficient in the format of U2.7, and finally they are multiplied by the input data U8.0 to obtain corrected output data U8.0.

FIG. 7 shows a shading correction circuit control block, which is a counter block (H / V).
21 to 24, difference blocks (H / V) 25 and 26, shift registers (H / V) 27 and 28, RAM control blocks 29 and 30, an address generator 31, a data path system control signal block 32, and the like. . As described above, in the first embodiment of the present invention, it is possible to accurately calculate the shading correction coefficient of each pixel by a relatively simple correction calculation method.

FIG. 10 and FIG. 11 are views for explaining a second embodiment for realizing the second embodiment of the present invention,
FIG. 10 is a block diagram showing a circuit configuration for performing shading correction in this embodiment, and FIG. 11 is an explanatory diagram showing a correction method by mesh division used in this embodiment. In the second embodiment, a method capable of correcting both the shading of the lens system and the shading of the sensor system (uneven sensitivity of the image sensor) will be described. In the second embodiment, an expensive frame memory is required if the sensitivity unevenness of the sensor is corrected pixel by pixel. Therefore, as an example, the number of pixels is 1280 ×
960 effective pixels are divided, 64 × 64 is set as one mesh area unit, and the lens and sensitivity unevenness can be corrected in the mesh area unit. In this case, the above-mentioned effective pixels are divided into 20 × 15 mesh areas. The correction processing procedure will be described below.

First, as in the first embodiment, a reference white image is picked up at the camera adjustment stage, and the sampling data is taken into the RAM 12. In this case, the difference from the first embodiment is that, as shown in FIG. 11, a value obtained by integrating and averaging data for one screen for each mesh is used as correction data as a sampling value of the mesh. .
In this case, as shown in FIG. 10, 20 meshes for one horizontal line are integrated by the integrator 51 and the temporary register (Reg) 52 and averaged, and then the address generator 31 is added to the data for one horizontal line. Then, an address is added to write to the RAM 12 via the divider 54 and the multiplexer 55.

Thereafter, similarly, the integrator 51 and the temporary Reg 52 are sequentially reset, and the integration of each mesh for the next one horizontal line is performed. As a result, finally RAM
Data of 300 meshes for one screen is written in 12. Next, by the CPU, this RA
Read all data from M12, search for the maximum value data from it, and (maximum value data) /
(White average data of each mesh) is calculated and the result is R
Write back to AM12. At this point, the RAM 12 has been written with the correction coefficient data for each mesh.

Next, how to make an approximate straight line using the correction data for each mesh will be described with reference to FIG. Note that FIG. 11 shows a 3 × 3 9-mesh image pickup surface for simplicity. Further, the circle in the center of each mesh in the figure represents the correction coefficient of the position of the center of gravity in that mesh. At the time of correction, as shown by the hatched area in FIG. 11, for the area newly surrounded by the center of gravity of the four meshes, the correction coefficient P at an arbitrary coordinate within this mesh is calculated from the following (Equation 3). (H + x, v + y) is interpolated from the correction coefficients of the barycentric positions of these four points. P (h + x, v) = P (h, v) + (P (h + m, v) -P (h, v)) * x / m P (h + x, v + m) = P (h, v + m) + (P ( h + m, v + m) -P (h, v + m)) * x / m P (h + x, v + y) = P (h + x, v) + (P (h + x, v + m) -P (h + x, v)) * y / m …… (Formula 3)

Here, P (h, v), P (h + m,
v), P (h, v + m), and P (h + m, v + m)
Is a correction coefficient for the center of gravity of the four corners of one mesh for correction, and m is the width of one mesh. Further, P (h + x, v) and P (h + x, v + m) are correction coefficients P (h
+ X, v + y) is a correction coefficient at an intermediate coordinate for deriving. As described above, in accordance with the scanning in the H and V directions, the correction coefficients at the four required barycentric positions for the points to be interpolated are sequentially read from the shading RAM, so that the correction coefficients can be calculated at all effective pixel points. I have to. In addition, FIG.
The data path block diagram of FIG. 3 also shows a data path for calculating the correction coefficient for realizing the first-order approximation straight line of (Equation 3). In the second embodiment, by applying the correction calculation method of the first embodiment in a limited mesh area,
With respect to the accuracy of correction, it is possible to guarantee much higher accuracy than the accuracy of the first embodiment. This accuracy can be increased by further dividing the mesh. In addition, since the sensitivity unevenness of the sensor system can be corrected to some extent, the optimum correction calculation can be performed.

[0027]

As described above, according to the shading correction method of the present invention, the white reference data previously acquired at the sampling points on the horizontal and vertical center axes of the effective image pickup plane are used to calculate the correction approximate line at each center axis. Based on this correction approximate line, the correction coefficient for the horizontal coordinate and the correction coefficient for the vertical coordinate are multiplied to calculate the shading correction coefficient on the arbitrary coordinate. Shading correction can be approximated with good accuracy. In addition, since the correction formula is calculated by actually taking in the values of the sampling points on both center axes of the effective pixels of the image pickup surface, it is possible to deal with a certain amount of deviation between the center of the image pickup surface and the optical axis center due to mounting accuracy. In addition to the shading correction of the lens system, the above calculation method can be effectively applied to the shading correction of the sensitivity unevenness of the sensor system. Therefore, optimal shading correction can be realized in a system such as a consumer-use camera system that must perform shading correction with a certain degree of accuracy at a limited cost and a limited number of gates.

[Brief description of drawings]

FIG. 1 is an explanatory diagram showing a shading correction method according to a first embodiment of the present invention.

FIG. 2 is an explanatory diagram for explaining calculation accuracy of the shading correction method according to the first embodiment.

FIG. 3 is an explanatory diagram showing a CIF used in the shading correction method of the first embodiment.

FIG. 4 is an explanatory diagram showing a mathematical proof of the validity of the calculation formula used in the shading correction method of the first embodiment.

FIG. 5 is an explanatory diagram showing a correction curved surface of lens shading in the first embodiment.

FIG. 6 is a block diagram showing a data path of a shading correction circuit in the first embodiment.

FIG. 7 is a block diagram showing a control block of a shading correction circuit in the first embodiment.

FIG. 8 is a flowchart showing the operation of the shading correction circuit in the first embodiment.

FIG. 9 is a flowchart showing the operation of the shading correction circuit in the first embodiment.

FIG. 10 is a block diagram showing a configuration of a shading correction circuit according to a second embodiment of the present invention.

FIG. 11 is an explanatory diagram showing a correction method by mesh division used in the second embodiment.

[Explanation of symbols]

10 ... LPF, 12 ... Shading RAM, 14
...... Shifter, 16 …… System controller, 21-2
4 ... Counter block (H / V), 25, 26 ... Difference block (H / V), 27, 28 ... Shift register (H / V), 29, 30 ... RAM control block, 31
...... Address generator, 32 ...... Data path control signal block, 51 ...... Integrator, 52 ...... Temporary register, 54 ...... Divider, 55 ...... Multiplexer.

Continued front page    F term (reference) 5B047 AA01 AB02 BB04 BC11 CB09                       DA04 DC01 EA01                 5C021 PA67 PA76 PA78 XA03 XA67                 5C022 AB51 AC42 AC54                 5C024 AX01 CX35 EX42 HX21 HX30                       HX31                 5C077 LL01 MP01 PP08 PP44 PP58                       PQ03 PQ12 PQ22 TT09

Claims (6)

[Claims] 1. A uniform white reference data is pre-acquired at sampling points on each of the horizontal and vertical center axes of the effective image pickup surface, and the white reference data is used to make a correction approximation in each center axis. A line is calculated, a correction coefficient for the horizontal coordinate and a correction coefficient for the vertical coordinate are multiplied based on the correction approximate line, and the multiplication value is calculated as a shading correction coefficient on an arbitrary coordinate within the effective image pickup plane. Shading correction method characterized by. 2. A correction value for each mesh area is calculated by dividing effective pixels on an imaging surface into mesh areas of a predetermined number of pixels, integrating pixel values in each mesh area, and averaging the mesh values. The shading correction method according to claim 1, wherein both the shading correction of the lens system and the shading correction of the sensor sensitivity unevenness are performed based on the correction value for each area. 3. A shading correction coefficient on arbitrary coordinates in the mesh area for correction, which is surrounded by four centroid points integrated for each mesh, is calculated from the correction coefficients at the centroid positions of these four points. The shading correction method according to claim 2, wherein 4. A correction coefficient in the horizontal axis direction is calculated in the horizontal synchronization period, and a correction coefficient in the vertical axis direction is calculated in the blanking period of the horizontal synchronization, so that one correction approximation line block makes the correction line perpendicular to the horizontal axis direction. The shading correction method according to claim 1, wherein the calculation in the axial direction is selectively performed. 5. The shading correction method according to claim 1, wherein the sampling point is a fixed point on each center axis. 6. The sampling point is a point that can be arbitrarily set on each center axis.
Shading correction method described.