JP6651916B2 - Image processing method - Google Patents

Description

  The present invention relates to an image processing method for performing an appropriate correction process on an image signal.

  2. Description of the Related Art In an image sensor such as a CMOS or a CCD, a subject image transmitted through a photographing lens and formed on a light receiving surface is photoelectrically converted into image data and output. In the image data obtained as a result, false colors and color unevenness occur due to various effects.

  In particular, there are a plurality of possible causes of color unevenness, but the main factor is an incident angle of a light beam incident on each pixel of the image sensor. That is, as the incident angle of the incident light beam becomes larger and the light beam becomes obliquely incident, the deterioration of the image quality due to color unevenness becomes larger.

  In order to solve such problems, various techniques relating to correction of color unevenness have been conventionally proposed.

  For example, in the invention disclosed in Patent Literature 1, the imaging device stores a correction term for correcting unevenness in signal values of captured image data in association with a shooting condition, and the first storage unit includes: A second storage unit that interpolates and calculates a correction term of the captured image data using a correction term that is close to the imaging condition when the captured image data was acquired among the stored correction terms, and a captured image calculated by the first calculation unit Correction means for correcting the unevenness of the signal value of the captured image data using the correction term of the data.

  Further, in the invention disclosed in Patent Document 2, the imaging apparatus includes a correction unit that corrects the image quality of the captured image signal in accordance with the set correction amount. An incident light beam incident on the solid-state imaging device through the condensing lens optical system is converged into a conical shape so that an image is formed on one point of the light receiving surface of the solid-state imaging device by the condensing action of the condensing lens. The correction amount is an image light direction that is an angular width formed by two light beams obtained by intersecting a conical peripheral wall surface of the incident light beam and a plane passing through the center of the light receiving surface of the solid-state imaging device and perpendicular to the light receiving surface. The configuration is defined in accordance with the angle width and the angle of incidence, which is the angle formed by the center line dividing the angle width of the image light direction into two and the normal to the light receiving surface.

JP 2012-244239 A Japanese Patent No. 5597776

  However, the above prior art has the following problems. That is, in the invention disclosed in Patent Document 1, the interpolation calculation of the correction term is performed using the central ray angle and the F value obtained from each ray angle at the maximum image height incidence. In this case, there is no problem if the ray angle monotonically increases with respect to the image height from the center of the image. However, some lenses do not have a monotonically increasing ray angle due to vignetting (vignetting) and the like. In such a lens, there is a possibility that the interpolation of the correction term is inappropriately performed.

  Further, according to the invention disclosed in Patent Document 2, a correction amount is selected based on light incident angle information grouped by image height with respect to a correction amount grouped by image height stored in a storage unit. I was In this case, it is effective for color unevenness generated based on the incident light angle that changes according to the image height (incident light angle), but does not depend on the image height and is not point-symmetric with respect to the center of the image. Interpolation may be inappropriately performed on such asymmetric color unevenness.

  The present invention has been made in view of such a situation, and an object of the present invention is to provide an image processing method capable of appropriately correcting color unevenness at each image position that changes according to a shooting condition (state of an incident light beam). And

In order to achieve the above object, an image processing method embodying the present invention includes a luminous flux information calculating step of calculating a luminous flux center ray angle and a luminous flux area, and a first step based on the calculated luminous flux center ray angle and the luminous flux area, respectively. A correction coefficient calculating step of calculating a correction coefficient and a second correction coefficient; and a correction processing step of performing a correction processing on an image signal using the first and second correction coefficients. expressed by an angle der Ri following conditions forming the upper ray and the lower ray and the normal of the light receiving surface in the transmission and any image height of the imaged imaging light beam on the light receiving surface of the light flux area of the light receiving imaging light flux It is characterized by an area at an arbitrary image height on the surface.
In addition, assuming that the luminous flux center ray angle is θc,
The luminous flux center ray angle θc is obtained by the following equation.
θc = (θlower (h) + θupper (h)) / 2
−θSTDlower (h) + θSTDupper (h)) / 2
here,
θlower (h) is the lower ray angle at the image height h,
θupper (h) is the upper ray angle at the image height h,
θSTDrawer (h) is the lower ray angle of the reference lens at the image height h,
θSTDupper (h) is the upper ray angle of the reference lens at the image height h

  Further, in the image processing method embodying the present invention, the luminous flux information calculating step creates a map of the luminous flux central ray angle corresponding to the light receiving surface based on the ray angle information relating to the ray angle of the imaging light flux. The method includes a creating step, wherein the light beam angle information includes at least an upper light beam angle and a lower light beam angle.

  Further, in the image processing method embodying the present invention, the luminous flux information calculating step includes: a central luminous flux area calculating step of calculating a central luminous flux area at the optical axis center of the light receiving surface of the imaging luminous flux from the ray angle information; A light flux area map creating step of creating a map of the light flux area corresponding to the light receiving surface based on the light intensity drop information and the central light flux area.

Further, in the image processing method embodying the present invention, the light flux information calculating step of calculating the light flux center light ray angle and the light flux area, and the first correction coefficient and the second correction respectively based on the calculated light flux center light ray angle and the light flux area A correction coefficient calculating step of calculating a coefficient, and a correction processing step of performing a correction processing on the image signal using the first and second correction coefficients. The average value of the upper ray angle and the lower ray angle, which is the angle formed between the upper ray and the lower ray and the normal of the light receiving surface at an arbitrary image height of the formed imaging light flux, and the light flux area is the light receiving area of the imaging light flux. The area at an arbitrary image height on the surface, and the luminous flux information calculating step creates a map of the luminous flux central ray angle corresponding to the light receiving surface based on the ray angle information relating to the ray angle of the imaging light flux. Map creation process Wherein the ray angle information at least, has an upper beam angle and lower ray angle, ray angle information further includes a sagittal ray angle is an angle formed between the normal of the sagittal rays and the light receiving surface of the imaging light beam, The luminous flux information calculating step includes a luminous flux area map creating step of calculating a square area inscribed in the imaging luminous flux based on the ray angle information and creating a luminous flux area map corresponding to the light receiving surface.

  Further, in the image processing method embodying the present invention, in the correction coefficient calculating step, the luminous flux center ray angle and the luminous flux center ray angle at a representative point provided in each of a plurality of blocks obtained by dividing the light receiving surface into a predetermined number. A first correction coefficient calculating step of calculating a first correction coefficient at the representative point from the first correction term according to the above, and a second correction term relating to the luminous flux area and the luminous flux area at the representative point. And a second correction coefficient calculating step of calculating a second correction coefficient.

  Further, the image processing method embodying the present invention further includes a correction coefficient interpolation step of calculating first and second correction coefficients for all light receiving surfaces by linear interpolation based on the first and second correction coefficients, In the correction processing step, the correction processing is performed using the first and second correction coefficients calculated in the correction coefficient interpolation step.

  Further, the image processing method embodying the present invention is characterized in that in the correction processing step, the correction processing is performed by uniformly applying the first and second correction coefficients to a block corresponding to the representative point.

  According to the image processing method embodying the present invention, it is possible to provide an image processing method capable of appropriately correcting color unevenness at each image position that changes according to a shooting condition (state of an incident light beam).

FIG. 1 is a block diagram illustrating a main configuration of an imaging device implementing an image processing method according to an embodiment of the present invention. FIG. 3 is a schematic diagram illustrating a shape (area) of an image forming light beam formed on a light receiving surface in a pixel. FIG. 2 is a block diagram illustrating a main configuration for performing color unevenness correction in the imaging device illustrated in FIG. 1. 5 is an example of a data table including light beam angle information and peripheral light intensity drop information stored in a ROM 171. It is a schematic diagram explaining an upper and lower ray angle and a principal ray angle. 5 is an example of a data table including representative point information stored in a ROM 171. FIG. 3 is a schematic diagram illustrating the principle of bilinear interpolation. 5 is an example of a data table including ray angle information stored in a ROM 171. It is a conceptual diagram for demonstrating the formula | equation which calculates | requires the area of the inscribed quadrangle of an imaging light beam.

  Hereinafter, the best mode for carrying out the present invention will be described with reference to the accompanying drawings. The present invention is not limited by the embodiment.

  FIG. 1 is a block diagram showing a main configuration of an imaging apparatus in which an image processing method according to an embodiment of the present invention is mounted.

  Reference numeral 100 denotes a photographing lens, and more specifically, an interchangeable lens that is detachable from the camera body 200. The photographing lens 100 has a zoom lens 110 and a focus lens 120. These are depicted with one lens in the figure, but are not limited to this. Moreover, you may have a lens other than these. The zoom lens 110 and the focus lens 120 are connected to a zoom control unit 140 and a focus control unit 150, respectively, and control such as lens driving and position detection is performed.

  Reference numeral 130 denotes an aperture unit provided in the photographing lens 100, and is connected to the aperture control unit 160. The aperture control unit 160 controls the aperture value (F value) of the aperture unit 130.

  Reference numeral 170 denotes a lens CPU, which determines the control contents of the various control units described above in cooperation with a main CPU 240 described later and issues an instruction. Further, the lens CPU 170 obtains shooting conditions such as a zoom position, a focus position, and an F value of the shooting lens 100 from various control units, and outputs the obtained shooting conditions to the main CPU 240. Further, it outputs the lens ID stored in the memory unit (not shown) to the main CPU 240 as needed.

  Reference numeral 200 denotes a camera body, to which the photographing lens 100 can be attached by a mechanism (not shown). Reference numeral 210 denotes an image sensor, which receives a light beam condensed by the photographing lens 100, performs photoelectric conversion, and outputs an image signal. A CMOS image sensor is used as the image sensor 210 of the present embodiment.

  The light receiving surface of the image sensor 210 is composed of many pixels. These pixels have a vertical color separation type image in which a single pixel can output an image signal composed of each color signal value of RGB by using a difference in depth at which photoelectric conversion is performed depending on the wavelength of incident light. It is a sensor. The image sensor 210 of the present embodiment incorporates a variable gain amplifier for amplifying a color signal value read from a pixel, a gain correction circuit for correcting a gain value, and an A / D converter for digitally converting an analog image signal. I have.

  Reference numeral 220 denotes a color unevenness correction unit that performs image processing related to color unevenness correction on an image signal output from the image sensor 210. The color unevenness correction in the color unevenness correction unit 220 will be described later in detail.

  An image processing unit 230 performs various general image processing on the image signal output from the color unevenness correction unit 220. The processing performed here includes, for example, processing for generating RAW data in a predetermined format from an image signal, white balance processing, and color reproduction processing. The image processing unit 230 also performs development processing on image data in JPEG format or TIFF format.

  Reference numeral 240 denotes a main CPU, which performs comprehensive control of the entire imaging device 300. The main CPU 240 is electrically connected to the lens CPU 170 and controls the photographing lens 100 in cooperation.

  Reference numeral 250 denotes a recording medium I / F, which records or reads RAW data or developed image data with a recording medium (not shown). This recording medium is a removable recording medium such as a semiconductor memory.

  Reference numeral 260 denotes a user I / F, which has operation members such as a release button, a power button, a command dial, and a cross key. When the user operates these operation members, the main CPU 240 issues an instruction to perform a predetermined operation.

  An image display unit 270 displays image data processed by the image processing unit 230, image data read from a recording medium (not shown), and the like.

  When the image pickup device 210 which does not include the above-described variable gain amplifier, gain correction circuit, and A / D converter is employed, these devices may be individually mounted.

  Next, the cause of the color unevenness to be corrected will be considered.

  Conventionally, it has been known that the angle of incident light and the F value affect color unevenness. The larger the angle of the incident light beam, the more the incident light to the pixel becomes oblique, and the color of the image becomes non-uniform due to the image light flux deviating from the pixel or leaking into the adjacent pixel.

  Further, the shape of the imaging light beam changes according to the F-number of the photographing lens 100. That is, the area of the imaging light flux decreases as the F value of the imaging lens 100 increases (darkens), and the area of the imaging light flux increases as the F value decreases (brightens). If the area of the image forming light beam is large, leakage into adjacent pixels is apt to occur and color unevenness occurs. In a range where this characteristic is maintained, the color unevenness depends on the incident angle and the F-number of the image forming light beam.

  However, vignetting may occur in the imaged light beam depending on the photographing lens 100 used. FIG. 2 is a schematic diagram for explaining the shape (area) of the image forming light beam formed on the light receiving surface in the pixel, and shows a state in which vignetting has occurred. When vignetting occurs, the change characteristics of the shape (area) of the imaged light beam are affected, and the imaged light beam has an elliptical shape or a shape close to a missing circle.

  That is, as shown in the figure, the length of the imaging light beam in the sagittal direction still changes according to the F value, but the length in the meridional direction is limited by vignetting. If it is smaller than F2.0 in the case of the figure, it does not change. The length of the imaging light beam in the meridional direction can be calculated from the incident angles of the upper and lower rays of the image light beam. It is not enough to use only the incident angles of the upper and lower rays, and it is necessary to consider the area of the luminous flux.

  The incident light beam angle that affects color unevenness has the same value for each image height. Therefore, it is known that color unevenness depends on the image height, that is, the distance from the center of the image. However, in some cases, the color unevenness has asymmetry with respect to the center of the image. For more accurate evaluation, it is not desirable to perform correction in a form depending only on the image height.

  It can be considered that the asymmetry of the color unevenness is mainly caused by the central asymmetry of the pixel structure forming the image sensor 210. In recent years, the number of pixels and the pixel pitch have become finer in image sensors, and the pixel structure has become highly complicated. As a result, wiring layers and light-shielding layers have become complicated inside the pixels. When the incident angle and the incident direction of the imaging light beam change with respect to such a pixel, the characteristics of vignetting caused by the internal structure of the pixel also change in a complicated manner. This vignetting is variously generated at each image height and each pixel structure, and as a result, it is considered that asymmetric color unevenness occurs at the center.

  In addition to this, manufacturing errors in the pixel structure and the microlens array can also be factors that affect color unevenness.

  For such color unevenness that does not depend on the image height, it is not sufficient to use a correction amount grouped for each image height as in the related art, and the image signal is obtained by dividing the image signal into a predetermined number. It is necessary to determine the correction amount for each of a plurality of blocks.

(First embodiment)
Next, a flow of image processing for performing a correction process for the above-described asymmetric color unevenness will be described. FIG. 3 is a block diagram showing a main configuration for performing image processing related to color unevenness correction in the first embodiment.

  The lens CPU 170 has a ROM 171 and a RAM 172 inside. The ROM 171 stores light beam angle information and peripheral light intensity drop information specific to the photographing lens 100 in advance. Details of such information will be described later. The RAM 172 stores various photographing conditions obtained from the zoom control unit 140 and the like.

  The color unevenness correction unit 220 includes a luminous flux information calculation unit 221, a correction coefficient calculation unit 222, a ROM 223, a correction coefficient interpolation unit 224, and a correction processing unit 225. The ROM 223 stores information on representative points provided in advance in a plurality of blocks obtained by dividing an image signal into a predetermined number. Details of this representative point information will be described later.

  The luminous flux information calculation unit 221 calculates the luminous flux center ray angle and the luminous flux area using the ray angle information and the peripheral light intensity drop information input from the ROM 171, the imaging conditions input from the RAM 172, and the representative point information input from the ROM 223. The calculation is performed for the positions of all the representative points thus performed, and a light flux center ray angle map and a light flux area map are created.

  Next, the correction coefficient calculation unit 222 calculates a correction coefficient at each representative point using the created light flux center ray angle map and light flux area map, and representative point information stored in the ROM 223. The correction coefficient includes a first correction coefficient based on the light beam center ray angle and a second correction coefficient based on the light beam area.

  Next, the correction coefficient interpolation unit 224 calculates the correction coefficient of the entire image signal from the correction coefficient at each representative point obtained by the correction coefficient calculation unit 222 by interpolation calculation. Thereby, correction coefficients corresponding to all image signals are obtained.

  Next, the correction processing unit 225 applies the correction coefficient obtained by the correction coefficient interpolation unit 224 to the signal value of the image signal, thereby completing the image processing related to color unevenness correction. The image signal subjected to the color unevenness correction is sent from the color unevenness correction unit 220 to the image processing unit 230.

  Image processing related to color unevenness correction using the above configuration will be specifically described.

  FIG. 4 is an example of the light beam angle information and the peripheral light amount drop information stored in the ROM 171 and some of the numerical values are omitted. The ROM 171 contains upper and lower ray angles and principal ray angles as ray angle information. As shown in the figure, the light ray angle information and the peripheral light amount drop information are stored in the ROM 171 in the form of a data table corresponding to the image height. The peripheral light amount drop information indicates the ratio of the light amount at each image height when the light amount at image height 0 (image center) is set to 1. These pieces of information are specific to the taking lens 100 and are obtained from design data of the lens optical system.

  FIG. 5 is a schematic diagram illustrating the upper and lower ray angles and the principal ray angle. As shown in the figure, the angle formed by the upper ray of the imaging light flux that has passed through the exit pupil and incident on the image sensor and the normal (dotted line) of the imaging point is the upper ray angle θupper, and the lower ray of the imaging light flux The lower ray angle θ lower is the angle formed by the normal of the imaging point and the lower ray angle θ lower, and the chief ray angle θ chief is the angle formed by the principal ray passing through the center of the exit pupil and the normal of the imaging point. It has been described. In the present embodiment, an angle formed below the normal line of the imaging point is defined as plus, and an angle formed above the normal line is defined as minus.

  Further, since the relationship between the ray angle information, the peripheral light amount drop information and the image height changes according to the imaging conditions, the ROM 171 stores a plurality of different data tables for each imaging condition. The condition under which the data table is stored can be arbitrarily selected, and may be appropriately selected according to the correction accuracy, the capacity of the ROM 171, and the like.

  The luminous flux information calculation unit 221 calculates the luminous flux central ray angle and the luminous flux area using the above-described ray angle information, marginal light amount drop information, and shooting conditions at the time of shooting. Here, the luminous flux center ray angle is an angle obtained as an average value of the upper and lower ray angles at a certain image height, and the luminous flux center ray is indicated by a broken line in FIG.

The luminous flux center ray angle θc is obtained by the following equation 1.
θc = (θlower (h) + θupper (h)) / 2
−θSTDlower (h) + θSTDupper (h)) / 2 (1)
here,
θlower (h) is the lower ray angle at the image height h,
θupper (h) is the upper ray angle at the image height h,
θSTDrawer (h) is the lower ray angle of the reference lens at the image height h,
θSTDupper (h) is the upper ray angle of the reference lens at the image height h.

  Further, in the present embodiment, the light flux area S is obtained by normalizing the light flux area as an area at a predetermined depth in order to simplify the handling of the absorption depth in the pixel which differs depending on the pixel position and the wavelength. In the following description, the term "luminous flux area" basically indicates the normalized luminous flux area.

When the transmittance of the lens is not considered, the peripheral light amount drop corresponds to the luminous flux area ratio between the center of the image and the arbitrary image height. Therefore, the luminous flux area at each image height can be obtained by multiplying the luminous flux area S0 at the center of the image by the corresponding peripheral light intensity drop information. That is, as shown in the following equation 2,
Luminous flux area S at image height h
= Center light flux area S0 × Peripheral light quantity drop information at image height h (2)
Becomes

The central luminous flux area S0 is obtained by the following equation (3).
S0 = π × tan (θ lower (0)) ＾ 2 (3)
Here, θlower (0) is the image height 0, that is, the lower ray angle at the center of the image. If the image height is 0, the lower ray angle and the upper ray angle have the same absolute value, and any value may be used as long as the sign is considered.

  In the example of the data table shown in FIG. 4, each piece of information is stored at an image height of 10%. Therefore, first, the light flux information calculation unit 221 calculates the light flux center ray angle and the light flux area corresponding to the image height in increments of 10%.

  FIG. 6 is an example of the representative point information stored in the ROM 223, and some numerical values are omitted. The representative point information includes the position (i, j) of each representative point and the image height h corresponding thereto. As described above, the representative point is a point set in a plurality of blocks obtained by dividing the image signal into a predetermined number. In the present embodiment, the image signal is divided into 60 × 40 (total 2,400) blocks, and the upper leftmost block is numbered (1,1), and the next lower block is numbered (1,2) in this order. Finally, the block at the bottom right is (60, 40).

  The representative point is the center pixel of each block. Since the size of each block is uniform, the position coordinates of the representative point are uniquely determined.

  When acquiring the above-described representative point information from the ROM 223, the light flux information calculation unit 221 calculates the light flux center ray angle and the light flux area corresponding to the image height of each representative point by interpolation calculation, and calculates the light flux center ray angle map and the light flux area map. create.

  The representative point information further stores first and second correction terms in a form corresponding to each representative point. The first correction term is a correction term related to the light beam center ray angle, and the coefficient between the light beam center ray angle and the image signal obtained by the reference lens is previously approximated by, for example, a polynomial using a cubic function. It is obtained as a1 to d1. That is, the coefficient of the third-order term is a1, the coefficient of the second-order term is b1, the coefficient of the first-order term is c1, and the coefficient of the zero-order term is d1. The second correction term is a correction term relating to the luminous flux area, and is obtained as coefficients a2 to d2 by approximating the relationship between the luminous flux area and the image signal to a three-dimensional polynomial, as in the first correction term. That is, the coefficient of the third-order term is a2, the coefficient of the second-order term is b2, the coefficient of the first-order term is c2, and the coefficient of the zero-order term is d2.

  By calculating these correction terms in each block, the data table of the representative point information shown in this drawing is obtained. These correction terms are to be calculated for each color information. For example, three types corresponding to red (R), green (G), and blue (B), color ratio (R / G), (B / G ), But only one type is shown in the data table in FIG.

  By calculating the relationship between the light beam center ray angle and the light beam area and the coloring (color unevenness characteristics), it is possible to reduce the influence on the color unevenness characteristics caused by different photographing lenses and photographing conditions. Further, by calculating the above for each image position (representative point), it becomes possible to capture the color non-uniformity characteristics caused by the relationship between the incident light beam and the imaging device structure.

The correction coefficient calculation unit 222 calculates a first correction coefficient k1 from the luminous flux center ray angle θc obtained by the luminous flux information calculation unit 221 and the first correction terms a1 to d1 according to the following Expression 4.
k1 = a1 × θc ＾ 3 + b1 × θc ＾ 2 + c1 × θc + d1 (4)
Similarly, the second correction coefficient k2 is calculated from the light flux area S and the second correction terms a2 to d2 according to the following equation 5.
k2 = a2 × S ＾ 3 + b2 × S ＾ 2 + c2 × S + d2 (5)
As a result, a correction coefficient of color unevenness correction at the representative point position of each block is obtained.

  Subsequently, the correction coefficient interpolation unit 224 calculates a correction coefficient corresponding to the entire image signal from each correction coefficient obtained for the representative point by linear interpolation (bilinear interpolation). This bilinear interpolation is an interpolation method for finding an unknown value at one point from four known values, as shown in FIG.

  Consider a case where the values at four points (x1, y1), (x1, y2), (x2, y1), and (x2, y2) are I11, I12, I21, and I22, respectively. At this time, according to the bilinear interpolation, it is known that the value I (x ', y') at one point (x ', y') in the area surrounded by four points can be obtained by Expression 6.

  When the first and second correction coefficients corresponding to the entire image signal are obtained by the above-described bilinear interpolation, the correction processing unit 225 multiplies those correction coefficients by the image signal obtained from the image sensor 210 and applies the multiplied image. Thus, the image processing relating to the color unevenness correction is completed. When the correction term is acquired for each of RGB, the correction process is performed by applying the obtained correction coefficient to the corresponding RGB signal value, and the correction term is, for example, a color ratio R / G. , B / G, the correction process is performed by applying to each corresponding color ratio signal value.

  The image signal on which the correction processing has been completed is sent to the subsequent image processing unit 230 as described above, and is subjected to various general image processing.

  As described above, according to the first embodiment of the image processing method embodying the present invention, it is assumed that the color unevenness that varies depending on the location is a function of the light flux center ray angle and the light flux area, and the correction amount for correcting the color unevenness is the light flux center. It is assumed to be a cubic function of ray angle and luminous flux area. Then, as coefficients of these cubic functions (first and second correction terms), coefficients obtained when the image signal obtained by the reference lens is polynomial approximated by a cubic function of the luminous flux center ray angle and the luminous flux area, respectively. Is used.

  As a result, color unevenness is also captured as a function of the luminous flux area, so that changes that cannot be captured only in the meridional direction (luminous flux center ray angle) can be captured in the luminous flux area taking sagittal into account, and the accuracy of color unevenness correction is improved. Is improved.

  The present applicant initially focused on the change in the sagittal ray angle when capturing the behavior of color change that could not be explained by the ray angle and ray width in the meridional direction alone. No correlation with the color change was obtained in the ratio. As a result of further study, it was found that the product of the meridional beam angle width and the sagittal beam angle width, that is, the area of the luminous flux can be correlated with the color change. This can be interpreted as corresponding to a phenomenon in which the coloring of the image changes according to the F value. By replacing the F value with the luminous flux area for each image height, it becomes possible to handle the coloring change for each image height. Furthermore, the larger the luminous flux area (the smaller the F-number and the brighter the lens), the more easily vignetting due to the pixel internal structure and oblique incidence on adjacent pixels occur. This is useful information.

  Although the central ray angle of the light beam and the area of the light beam depend on the image height, the first and second correction terms are created for each representative point and can be changed independently of the image height. It is possible to appropriately correct even the color unevenness resulting from the dependence phenomenon.

  Further, by using the linear interpolation to calculate the obtained correction coefficients for the entire image signal by interpolation, the change at the block boundary can be reduced while the required number of blocks of the first and second correction terms is suppressed. Correction can be made so as to be smooth.

(Second embodiment)
Next, a second embodiment will be described. In the first embodiment described above, the luminous flux area was calculated using the central luminous flux area S0 at the center of the image and the peripheral light intensity drop information as shown in Expression 2. The marginal light amount drop information is stored in the ROM 171 in the form of a data table corresponding to the image height as shown in FIG. In the second embodiment, the light flux area is calculated by approximating the light flux area by a rectangle inscribed in the imaging light flux. Except for the portion related to the calculation of the luminous flux area, the configuration is the same as that of the first embodiment described above, and the description is omitted.

FIG. 8 shows an example of ray angle information stored in the ROM 171 in the second embodiment, and some numerical values are omitted. Since the upper and lower ray angles and the principal ray angle are the same as those in the first embodiment, the description is omitted. The sagittal ray angle is the angle between the sagittal ray of the imaged light beam and the normal to the light receiving surface, and is obtained from the design data of the lens optical system as with other ray angle information. Since the relationship between the ray angle information and the image height changes depending on the imaging conditions, the ROM 171 stores a plurality of different data tables for each imaging condition.

  FIG. 9 is a conceptual diagram for explaining an equation for calculating the area of the inscribed rectangle of the image forming light beam. As shown in the figure, various light beams are incident on the periphery of the image forming light beam. That is, in the image height (meridional) direction, a lower ray is incident on a side closer to the image center, and an upper ray is incident on a side farther from the image center. Further, a sagittal ray is incident in a sagittal direction perpendicular to the image height direction. Further, a chief ray is incident on one point inside the imaging light beam.

At this time, in order to obtain the area St of the triangle inscribed in the imaged light beam in the figure, considering the image height direction as the base and the sagittal direction as the height, the length of the base is respectively: (tan (θlower (h) −θchief ( h))
−tan (θ upper (h) −θ chief (h)))
Height: tan (θ sagittal (h))
It can be expressed as.
here,
θlower (h) is the lower ray angle at the image height h,
θupper (h) is the upper ray angle at the image height h,
θchief (h) is the chief ray angle at the image height h,
θ sagittal (h) is the sagittal ray angle width at the image height h.

Assuming that the shape of the image forming light beam is substantially line-symmetric with respect to the meridional light beam, the area of the inscribed rectangle of the image forming light beam may be twice the area of the triangle, and the area S of the inscribed rectangle is expressed by the following equation. 7 is required.
S = (tan (θ lower (h) −θ chief (h))
−tan (θ upper (h) −θ chief (h)))
× tan (θ sagittal (h)) (7)

  The luminous flux information calculation unit 221 performs the above-described calculation for all representative points, and creates a luminous flux area map. The subsequent processing flow is the same as in the first embodiment, and a description thereof will be omitted.

  As described above, according to the second embodiment of the image processing method of the present invention, the light flux area is approximated by the area of the inscribed rectangle of the imaging light flux without using the peripheral light intensity drop information. The peripheral light amount drop information includes a light amount decrease due to the transmittance of the photographing lens 100, and an error may increase depending on the photographing lens 100. On the other hand, the second embodiment described above has an advantage that it is not affected by the transmittance of the taking lens 100.

  As a modification, the luminous flux area may be approximated to an ellipse instead of an inscribed rectangle. In that case, the luminous flux area can be obtained by using the formula for obtaining the elliptical area. In addition, it is possible to calculate the luminous flux area by using an appropriate approximation.

  In the image processing method according to the above-described embodiment, the correction coefficients calculated for the representative points are linearly interpolated (bilinear interpolation) by the correction coefficient interpolation unit 224 to obtain the correction coefficients for all image signals. The interpolation method is not limited to this. For example, a correction coefficient for all image signals may be obtained by uniformly applying a correction coefficient for a representative point to a block to which the representative point belongs. In this case, the correction accuracy of the color unevenness is reduced, but the load of calculation can be reduced. Since the correction coefficient interpolation unit 224 is unnecessary, the correction coefficient calculated by the correction coefficient calculation unit 222 is directly sent to the correction processing unit 225.

  As described above, the image processing method according to the present invention provides an image processing method capable of appropriately correcting color unevenness at each image position that changes according to a shooting condition (state of incident light). It becomes possible.

Reference Signs List 100 shooting lens, 110 zoom lens, 120 focus lens, 130 aperture unit, 140 zoom control unit, 150 focus control unit, 160 aperture control unit, 170 lens CPU, 171 ROM, 172 RAM, 200 camera body, 210 image sensor, 220 Color unevenness correction section, 221 light flux information calculation section, 222 correction coefficient calculation section, 223 ROM, 224 correction coefficient interpolation section, 225 correction processing section, 230 image processing section, 240 main CPU, 250 storage medium I / F, 260 user I / F, 270 Image display unit, 300 imaging device

Claims (7)

Light flux information calculation step of calculating the light flux center ray angle and the light flux area,
A correction coefficient calculating step of calculating a first correction coefficient and a second correction coefficient based on the calculated light flux center ray angle and the calculated light flux area, respectively;
A correction processing step of performing correction processing on an image signal using the first and second correction coefficients;
Including
The light beam central ray angle, angle der Ri following condition formed between the normal line of the light receiving surface and the upper ray and the lower ray at an arbitrary image height of the imaging light beam formed on the light receiving surface through the photographic lens Can be represented by
The image processing method according to claim 1, wherein the light flux area is an area of the imaging light flux at an arbitrary image height on the light receiving surface.
In addition, assuming that the light beam center ray angle is θc,
θc is obtained by the following equation.
θc = (θlower (h) + θupper (h)) / 2
−θSTDlower (h) + θSTDupper (h)) / 2
here,
θlower (h) is the lower ray angle at the image height h,
θupper (h) is the upper ray angle at the image height h,
θSTDrawer (h) is the lower ray angle of the reference lens at the image height h,
θSTDupper (h) is the upper ray angle of the reference lens at the image height h . The light flux information calculating step includes:
Based on light beam angle information on the light beam angle of the imaging light beam, including a light beam center ray angle map creating step of creating a map of the light beam center ray angle corresponding to the light receiving surface,
The image processing method according to claim 1, wherein the light ray angle information includes at least the upper light ray angle and the lower light ray angle. The light flux information calculating step includes:
A central light flux area calculating step of calculating a central light flux area at an optical axis center of the light receiving surface of the imaging light flux from the light ray angle information;
The method according to claim 2, further comprising a light flux area map creating step of creating a map of a light flux area corresponding to the light receiving surface based on the peripheral light quantity fall information regarding the peripheral light quantity fall and the center light flux area. Image processing method. Light flux information calculation step of calculating the light flux center ray angle and the light flux area,
A correction coefficient calculating step of calculating a first correction coefficient and a second correction coefficient based on the calculated light flux center ray angle and the calculated light flux area, respectively;
A correction processing step of performing correction processing on an image signal using the first and second correction coefficients;
Including
The light beam center ray angle is an upper ray angle and a lower ray angle, which are angles formed by upper rays and lower rays at an arbitrary image height of an imaging light flux transmitted through the imaging lens and imaged on the light receiving surface and a normal line of the light receiving face. It is the average value with the ray angle,
The light flux area is an area at an arbitrary image height on the light receiving surface of the imaging light flux,
The light flux information calculating step includes:
Based on light beam angle information on the light beam angle of the imaging light beam, a light beam center ray angle map creating step of creating a map of the light beam center ray angle corresponding to the light receiving surface,
The ray angle information has at least the upper ray angle and the lower ray angle,
The ray angle information further includes a sagittal ray angle that is an angle formed by a sagittal ray of the imaging light flux and a normal to the light receiving surface,
The luminous flux information calculation step includes:
An image processing method comprising: calculating an area of a quadrilateral inscribed in the imaging light flux based on the light ray angle information, and generating a light flux area map corresponding to the light receiving surface. The correction coefficient calculation step,
From the first light beam angle and the first correction term relating to the light beam center ray angle at a representative point provided in each of a plurality of blocks obtained by dividing the light receiving surface into a predetermined number, the representative point A first correction coefficient calculating step of calculating the first correction coefficient in
A second correction coefficient calculating step of calculating the second correction coefficient at the representative point from the light flux area and a second correction term relating to the light flux area at the representative point. The image processing method according to claim 1. A correction coefficient interpolation step of calculating the first and second correction coefficients for all the light receiving surfaces by linear interpolation based on the first and second correction coefficients,
6. The image processing method according to claim 5, wherein in the correction processing step, a correction processing is performed using the first and second correction coefficients calculated in the correction coefficient interpolation step. In the correction processing step,
The image processing method according to claim 5, wherein the first and second correction coefficients are uniformly applied to the block corresponding to the representative point to perform a correction process.

Images