JP5848027B2 - Imaging device - Google Patents

Description

The present invention relates to an imaging apparatus that performs image data correction processing.

In a general imaging device, apart from non-uniformity of the amount of light incident on the imaging surface due to the characteristics of an optical system such as a lens, the non-uniformity of the pixels constituting the imaging device and the phase of the microlens corresponding to the pixel Non-uniformity may occur in the light receiving characteristics of the imaging surface due to misalignment, uneven dyeing of the filter, and the like. Luminance shading or color shading caused by this is a problem.

As a method of correcting the shading of the imaging device, image data obtained by photographing a surface with uniform brightness and color in advance is divided into a plurality of blocks, gains are obtained for each block, and the above-mentioned gains are obtained at the time of photographing. A method of correcting image data by using it is known.

Further, there is known a method of selecting an optimum one from a plurality of gains that are provided in accordance with changes in photographing conditions (zoom magnification, aperture diameter, etc.).

Japanese Patent No. 4579570

JP 2008-53931 A

JP 2008-109278 A

 However, if the imaging conditions assumed in the imaging apparatus are diverse and the optimal correction term changes greatly, if the correction terms are prepared for all possibilities, the correction value As a result, the storage capacity for holding the data becomes enormous, and the adjustment process for obtaining the correction term becomes complicated.

 In the present invention, there is provided means for calculating a parameter having a correlation with shading, such as an incident angle of a light beam with respect to an image sensor under an assumed imaging condition, and a parameter such as an incident angle of the light beam under an imaging condition at the time of actual imaging. An object of the present invention is to provide an imaging apparatus that performs an optimal shading correction by performing interpolation calculation of an optimal correction term from the above.

In an imaging apparatus that has a photographic lens optical system, a diaphragm, and an imaging element, and that acquires the captured image data by acquiring an image signal, the first storage unit, the first calculation unit, the correction unit, A second storage unit and a control unit, wherein the first storage unit sets a correction term for correcting the unevenness of the signal value of the captured image data as a parameter suitable for the interpolation calculation corresponding to the imaging condition and the imaging condition . storing correspondingly the parameters suitable for storage in correspondence to a parameter suitable for interpolation calculation is calculated by the control means, said second storage means stores a plurality of parameters suitable for storage corresponding to the imaging conditions, the control means, from a plurality of parameters suitable for storage corresponding to the imaging conditions at the time of actual shooting, calculates the part of the plurality of parameters suitable for interpolation calculation, said first calculation means Of the correction terms stored in the first storage means, including those in which the control means is calculated, corresponding to the photographing conditions when acquiring the captured image data, a plurality of parameters suitable for interpolation calculation, close The correction term of the photographed image data is interpolated using correction terms corresponding to a plurality of parameters suitable for the interpolation calculation, and the correction means uses the correction term of the photographed image data calculated by the first calculation means to capture the image. An imaging apparatus is provided that corrects unevenness in signal values of image data.

In an imaging apparatus that has a photographic lens optical system, a diaphragm, and an imaging device, and that captures captured image data by the imaging device acquiring an image signal, a first storage unit, a second storage unit, a control unit, A first calculating unit, and a correcting unit, wherein the first storage unit sets a correction term for correcting the unevenness of the signal value of the photographed image data to a parameter and a photographing condition suitable for the interpolation calculation corresponding to the photographing condition . Corresponding parameters suitable for storage are stored in correspondence with parameters suitable for the interpolation calculation calculated by the control means , and the second storage means stores the light angle of the photographing condition, the type of the photographing lens, the zoom position, and the in-focus position. , Aperture diameter parameters, presence / absence and type of converter, presence / absence and type of filter, chromatic aberration, curvature of field, distortion, lens shift type image stabilization lens shift Amount, sensor shift method Shake correction sensor shift amount, tilt shift lens tilt shift amount, soft focus lens soft amount, light source type, photographing lens transmittance, photographing lens coating, sensor temperature, exposure time A plurality of parameters corresponding to at least two items and suitable for storage, and the control means includes a light angle of a photographing condition when actually photographing, a type of photographing lens, a zoom position, and a focusing position. , Aperture diameter parameters, presence / absence and type of converter, presence / absence and type of filter, chromatic aberration, curvature of field, distortion, shift amount of lens shift type image stabilization lens, shift amount of sensor shift type image stabilization sensor, tilt shift Lens tilt / shift amount, soft focus lens soft amount, type of light source Transmittance of the photographing lens, the coating of the taking lens, the sensor temperature, of the exposure time, a plurality of parameters suitable for the corresponding stored in at least two or more items, calculates the part of the plurality of parameters suitable for interpolation calculation The first calculation means includes an interpolation calculation corresponding to the shooting condition when the captured image data is acquired, including correction terms calculated by the control means among the correction terms stored in the first storage means. a plurality of parameters, the correction term of the photographed image data using the correction term corresponding to a plurality of parameters suitable for near interpolation calculated interpolation calculation suitable for the correction means, the captured image data to which the first calculation means has calculated An image pickup apparatus is provided that corrects unevenness in signal values of captured image data using the correction term.

The second storage means stores in advance the aperture center ray angle and aperture periphery ray angle corresponding to the imaging condition, and the first storage means associates the correction term with the parameters relating to the beam center ray angle and the aperture diameter. And the control means calculates a light flux center ray angle from parameters relating to the aperture center ray angle and the aperture periphery ray angle at the time of actual photographing, and the aperture diameter, and the first calculation means stores the first memory. Among the correction terms stored in the means, the parameter relating to the aperture diameter corresponding to the imaging condition when the captured image data is acquired, the parameter relating to the aperture diameter close to the beam center ray angle calculated by the control means, and the beam center The imaging apparatus according to claim 1 or 2, wherein the correction term of the photographed image data is calculated by interpolation using a correction term corresponding to the ray angle .

Having a second calculating means;
The second calculation means calculates a correction term for correcting a ratio of signal values of photographed image data from adjustment image data acquired by photographing a reference subject. An imaging apparatus according to claim 3 is provided.

The second calculation means divides adjustment image data acquired by photographing a reference subject into a plurality of blocks, calculates a gain for each block, and sets a correction term that is a set of gains for each block in advance. The first storage means calculates a correction term, which is a set of gains for each block, calculated by the second calculation means, and is a parameter suitable for interpolation calculation corresponding to the imaging condition when the adjusted image data is acquired. The image pickup apparatus according to claim 4, further comprising: a parameter suitable for interpolation calculation calculated by the control unit from a plurality of parameters suitable for storage corresponding to the imaging condition .

Subject to be the reference is to provide an imaging apparatus according to claim 4 or claim 5, wherein the luminance and the color is uniform surface.

The first calculating means calculates a correction term for non-uniformity of the light receiving characteristics of the imaging surface due to the characteristics of the imaging element, and the correcting means is caused by the characteristics of the imaging element calculated by the first calculating means. to provide an imaging apparatus according to any one of claims 1 to 6, characterized in that correction is performed using the correction term for the non-uniformity of the light receiving characteristic of the imaging surface to be.

Replacement of the lens optical system mounting are possible imaging apparatus, according to any one of claims 1 to 7, characterized in that said first memory means is disposed within said imaging device An imaging device is provided.

Replacement of the lens optical system mounting are possible imaging apparatus, according to any one of claims 1 to 7, characterized in that said first memory means is disposed within the lens mounting An imaging device is provided.

According to the present invention, it is possible to improve the image quality by performing the optimum shading correction according to the photographing condition of the image data with the minimum necessary adjustment process and the minimum necessary memory capacity.

1 is a block diagram showing a schematic configuration of an imaging apparatus which is an example of an image processing system according to the present invention. Flow chart of adjustment process in an embodiment of the present invention Flowchart of correction term adjustment process in an embodiment of the present invention Conceptual diagram showing the concept of aperture center ray angle Conceptual diagram showing the upper aperture peripheral ray angle and the lower aperture peripheral ray angle Conceptual diagram showing the light beam center ray angle The figure which shows the correction | amendment term corresponding to the aperture center ray angle, exit pupil, luminous flux center ray angle, and aperture diameter memorize | stored in a correction term memory | storage device. Conceptual diagram showing a method of selecting four correction terms to be read when calculating correction terms used for actually captured image data Conceptual diagram of dividing image data into blocks Conceptual diagram showing a method for calculating a gain to be given to each pixel of photographed image data from a correction term used for actually photographed image data

FIG. 1 is a block diagram showing a schematic configuration of an imaging apparatus which is an example of an image processing system according to the present invention.

In FIG. 1, reference numeral 101 denotes an optical system. For example, an optical system of an interchangeable lens attached to a camera capable of exchanging the optical system or an optical system integrated with a digital camera can be considered.

An aperture 102 is controlled by an aperture control 114.

Reference numeral 101a denotes a zoom lens that moves during zooming, and 101b denotes a focus lens that moves during focusing. The zoom control unit 112 and the focus control unit 113 control a lens 101a that moves during zooming and a lens 101b that moves during focusing, respectively. Yes. However, the zoom lens 101a and the zoom control unit 112 can be used when, for example, a camera with a replaceable optical system is equipped with a single focus lens or when the optical system integrated with the camera is a single focus. Is not included in the configuration.

Reference numeral 111 denotes a CPU, and a zoom control unit 112, a focus control unit 113, and an aperture control unit 114 drive each unit under the control of the CPU 111.

Reference numeral 115 denotes a lens control unit. When the optical system 101 is an optical system of an interchangeable lens attached to a camera capable of exchanging the optical system, the lens control unit 115 has a lens ID of the attached lens, a focal length at the time of photographing, a focusing distance, an aperture. The photographing conditions such as the diameter are read out and sent to the CPU. The lens ID is an ID for identifying an interchangeable lens that can be attached to a camera whose optical system can be exchanged. Which lens among the lenses in which the interchangeable lens attached by the lens ID is registered in the camera in advance is used. It can be referred to in the ray angle storage device 201. Further, when the optical system 101 is an optical system integrated with a digital camera, the CPU 111 reads out shooting conditions such as a focal length, a focusing distance, and an aperture diameter at the time of shooting.

Reference numeral 103 denotes a filter. As such a filter 103, for example, a filter that reflects or absorbs light having a wavelength unnecessary for an image sensor such as infrared rays, a low-pass filter, or the like can be considered. The transmittance of the filter 103 varies depending on the light incident angle.

An imaging element 104 is a CCD that converts light incident through the optical system 101 and the filter 103 into signal charges.

Reference numeral 105 denotes a CDS circuit. Such CDS is an abbreviation for Correlated Double Sampling, and indicates correlated double sampling. In this correlated double sampling, reset noise generated when scanning and transferring inside the CCD is detected, and the difference between the voltage of the feed-through portion of the signal and the voltage of the data portion is detected, and the reset noise is canceled by canceling this difference. It is removed and an accurate signal level is calculated.

A variable electric amplifier 106 amplifies signal charges.

Reference numeral 107 denotes an AD converter, and the signal charge output from the image sensor 104 passes through the CDS circuit 105 and the variable electric amplifier 106, and is converted from analog signals into digital signal image data by the AD converter 107.

Reference numeral 108 denotes an image processing unit. The image processing unit 108 performs various types of image processing on the captured image data. The image processing unit 108 includes a correction term adjustment calculation circuit 108a and a correction circuit 108b.

A correction term adjustment calculation circuit 108 a calculates a correction term from information from a ray angle storage device 201 and a correction term storage device 202 which will be described later, and an imaging condition from the lens control unit 115.

A correction circuit 108b calculates the gain to be given to each pixel of the entire image data from the correction term calculated by the correction term adjustment calculation circuit 108a, and multiplies the calculated gain by the signal value of each pixel. In this way, shading correction is performed.

201 is a ray angle storage device, which is stored in the camera or in each lens in the form of a non-volatile memory, and has a diaphragm center ray angle corresponding to a zoom position, a focus position, etc. The exit pupil position is stored.

Reference numeral 109 denotes a display device. For example, a liquid crystal or an organic EL can be used.

Reference numeral 110 denotes a medium for storing the captured image of the storage device. As the external recording device 110, for example, a recording medium such as an SD memory card can be used.

FIG. 4 is a conceptual diagram showing the concept of the aperture center ray angle.

The stop center ray angle is a ray incident angle of a ray that passes through the stop center and enters the maximum image height portion of the image sensor as shown in FIG.

When the correction term is calculated in the correction term adjustment calculation circuit 108a described above, if the camera can exchange lenses, the interchangeable lens mounted on the camera at the time of shooting and the mounted lens from the ray angle storage device 201 The aperture center ray angle or exit pupil position corresponding to the zoom position, focus position, etc. at the time of shooting of the interchangeable lens being read is read out, and the correction term of the correction term is used using the information on the aperture center ray angle or exit pupil position at the time of shooting. Interpolation calculation is performed, and a correction term used for actually captured image data is calculated. Similarly, in the case of a camera with an integrated optical system, the aperture center ray angle or exit pupil position corresponding to the zoom position and focus position of the optical system at the time of shooting is read from the ray angle storage device 201 and shot. Interpolation calculation of a correction term is performed using information on the aperture center ray angle or exit pupil position at the time, and a correction term used for actually captured image data is calculated.

FIG. 9 is a conceptual diagram in which image data is divided into blocks.

Reference numeral 301 denotes a correction term calculation circuit. The correction term calculation circuit 301 is sufficient to correct the unevenness of the image data of the constant light source acquired in advance by changing the lens or the zoom position, the focus position, and the aperture diameter in the form shown in FIG. It is divided into a number of blocks, the average value of the signal values is calculated for each block, and the gain to be given for each block is set for each block so that the color signal ratio of the obtained signal values is uniform over the entire screen. It is obtained as the reciprocal of the average value of the signal values.

FIG. 7 is a diagram showing a correction term stored in the correction term storage device, corresponding to the aperture center ray angle, the exit pupil, the beam center ray angle and the aperture diameter, and FIG. 7A shows the first embodiment. It is a conceptual diagram of an example.

Reference numeral 202 denotes a correction term storage device. The correction term storage device 202 stores a correction term that is a set of gains for each block calculated by the correction term calculation circuit 301. The correction term, which is a set of gains for each block, is stored in the correction term storage device 202 in the form as shown in FIG. 7A, for example.

For example, when the lens back is mounted on a camera with interchangeable lenses and a constant light source is photographed, the camera reads the aperture center ray angle m at the time of photographing the lens instep from the ray angle storage device 201, and the lens control unit When the aperture diameter of the lens instep is read as n, the in-camera correction term calculation circuit 301 has the aperture central ray angle m, and a correction term that is a set of gains for each block corresponding to the captured image data having the aperture diameter n. Is calculated and stored in the correction term storage device 202.

Next, image data correction processing performed by the imaging apparatus according to the present invention will be described. Such image data correction processing is roughly divided into three steps: an individual adjustment step, a correction term adjustment calculation step, and a correction step. Hereinafter, the three steps will be described in order.

The individual adjustment process in the first embodiment will be described.

The adjustment step is a step of recording an optimal correction term for each individual camera, and FIG. 2 is a flowchart of the adjustment step.

In the adjustment step, first, the lens is exchanged or the zoom position / focus distance is changed. (Step # 1)

In this embodiment, the lens to be replaced in step # 1 is assumed in advance. In other words, since the zoom position, the aperture center ray angle corresponding to the focus position, or the exit pupil position are stored in advance in the camera for each lens ID, the CPU 111 refers to the information to the ray angle storage device 201 and mounts it. It is possible to obtain information such as the aperture center ray angle or the exit pupil position for each zoom position / focusing distance of the lens.

In step # 1 of the present embodiment, for example, among preliminarily assumed lenses, the lens with the smallest aperture center ray angle or the one with the longest distance between the image plane and the exit pupil is attached to the camera in order and information is obtained by photographing. To get.

Next, the aperture diameter of the mounted lens is changed. (Step # 2)

In Step # 2 of the present embodiment, for example, the aperture state is first set to F8.0, data is acquired by photographing in Step # 3, and the gain of each block is calculated in Step # 4. Next, F2.8 is obtained. Next, F2.0, and finally F1.4, the aperture is opened in four stages, and step # 3 and step # 4 are repeated for each aperture diameter, so that four types of aperture patterns can be applied to the mounted lens. Get data at.

After changing the aperture diameter in step # 2, a fixed light source is photographed in step # 3 to obtain image data. In this embodiment, shooting is performed in an infinitely focused state. Then, the correction term calculation device 301 divides the image data into a plurality of blocks sufficient for correcting unevenness in the form as shown in FIG. 9, and calculates an average value of the signal values for each block. The gain to be given for each block is calculated as the ratio of the reciprocal of the average value of the signal value for each block so that the color signal ratio of the obtained signal value is uniform over the entire screen. The correction term as a set is stored in the correction term storage device 202 as a correction term corresponding to the aperture center ray angle and the aperture diameter as shown in FIG.

Further, when the correction term includes correction of the non-uniformity of the amount of light incident on the imaging surface due to the characteristics of an optical system such as a lens, the gain is an average value of signal values for each block, the center of the screen. Calculated as the inverse of the ratio of the signal value to the average value in the block. When only the non-uniformity of the light receiving characteristics of the imaging surface is corrected by such correction term, the gain is the reciprocal of the ratio of the average value of the signal value in each block to the average value of the signal value in the screen center block. Therefore, it is also conceivable to calculate as a value obtained by dividing the reciprocal of the ratio of the amount of light incident on the imaging surface from the optical system used in the adjustment step to the center of the imaging device at the block position.

When the storage is completed, the correction term calculation device 301 next determines whether correction terms have been acquired with a sufficiently large aperture diameter. In this embodiment, sufficiently large aperture diameters are all the four types of aperture patterns that are assumed. (Step # 5)

In this embodiment, as described above, four types of aperture patterns are assumed in the order of F8.0, F2.8, F2.0, and F1.4, and shooting is performed from F8.0 (step # 3), and the gain of each block. Calculation (step # 4) is performed. Therefore, when photographing is performed with a diaphragm F8.0 in a certain lens A (step # 3) and gain calculation for each block is performed (step # 4), the correction term calculation device 301 can only obtain a correction term for the lens A at F8.0. Since it has not been acquired, it is determined that correction terms have not yet been acquired for all four aperture diameters, and it is determined that correction terms have not yet been acquired for a sufficiently large aperture diameter. If correction terms have not been acquired with a sufficiently large aperture diameter, the process returns to step # 2 to change the aperture diameter. For example, when only the correction term at F8.0 is acquired, the aperture diameter is set to F2.8, which is the next largest aperture diameter after F8.0, and correction at F8.0, F2.8. If the term is acquired, the aperture diameter is set to F2.0 having the next largest aperture diameter. If the correction term for F8.0, F2.8, and F2.0 is acquired, the aperture diameter is next set. Is set to a large F1.4.

In this embodiment, the range of the aperture state for obtaining the correction term is F8.0 to F1.4, the number of divisions is set to four, and the data is in the order of F8.0, F2.8, F2.0, and F1.4. In order to obtain the lens, when the correction term at F1.4 is obtained in the lens A, it can be understood that data is obtained with four types of diaphragm patterns. If it is determined that data has been acquired, the process proceeds to step # 6.

In step # 6, the correction term calculation device 301 determines whether correction terms have been acquired with a sufficiently large number of aperture center ray angles. In the first embodiment, the aperture central ray angle is set to three types: small, standard, and large. In this embodiment, a sufficiently large number of aperture center ray angles is all three types of patterns assumed. Then, from a lens with a small aperture center ray angle, a lens with a standard aperture center ray angle, a lens with a large aperture center ray angle, and a lens with a small aperture center ray angle in this order (step # 3), Gain calculation (step # 4) is performed. Therefore, only in the lens A having a small aperture center ray angle, when the aperture is opened in four steps from F8.0 to F2.8, F2.0, and F1.4, and data is acquired with four types of aperture patterns, ( Step # 5) It is determined that the correction term calculation device 301 has acquired only the correction term for the aperture center ray angle of the lens A, and it is determined that the correction term has not been acquired with a sufficiently large aperture center ray angle. .

If correction terms have not been acquired with a sufficiently large number of aperture center ray angles, the process returns to step # 1 to change the lens and change the aperture center ray angle. For example, in this embodiment, the lens A having a small aperture center ray angle is replaced with a lens B having a standard aperture center ray angle. (Step # 1) When Step # 2, Step # 3, and Step # 4 are repeated for the lens B, and data is acquired with all four types of aperture patterns for the lens B as well as the lens A, the step # 5 is performed. To step # 6, the lens B having a standard aperture center ray angle is replaced with a lens C having a large aperture center ray angle. (Step # 1)

On the other hand, when data is acquired with three lenses A, B, and C, in this embodiment, the aperture center ray angle is a standard lens with a small aperture center ray angle, and the aperture center ray angle is standard. Since the three patterns of the lens and the lens having a large aperture center ray angle are set, the adjustment process is completed assuming that correction terms are acquired with a sufficiently large aperture center ray angle.

Next, the correction term adjustment calculation process in the first embodiment will be described.

The correction term adjustment calculation step is a step in which the correction term is adjusted to an optimum shape according to the shooting conditions, and the correction term used for the actually captured image data is calculated. FIG. It is a flowchart of this correction term adjustment process.

After the adjustment process, the image data to be corrected is taken. Then, in step # 10, the zoom control unit 112, the focus control unit 113, and the aperture control unit 114 read the focal length, the focusing distance, and the aperture diameter at the time of shooting, respectively. In step # 11, the lens control unit 115 sets the lens ID. read out.

When the interchangeable lens attached to the camera capable of exchanging the optical system is a lens assumed in advance, the lens ID is acquired from the attached interchangeable lens. On the other hand, when acquisition of the lens ID fails, for example, when the attached interchangeable lens is not a lens assumed in advance, a preset reference lens ID is input. (Step # 12) As the reference lens, for example, in this embodiment, a lens having a standard aperture center ray angle is set in advance.

As described above, the ray angle storage device 201 is arranged in the camera or in each lens in the form of a non-volatile memory, and the aperture center ray angle corresponding to the zoom position and the focus position for each lens assumed in advance, or The exit pupil position is stored. In step # 13, the lens ID at the time of photographing, the zoom position, the aperture center ray angle corresponding to the in-focus position, or the exit pupil position stored in the ray angle storage device 201 is read.

FIG. 8 is a conceptual diagram showing a method for selecting four correction terms to be read when calculating correction terms used for actually captured image data. FIG. 8A is a conceptual diagram of the first embodiment.

Consider a two-dimensional plane as shown in FIG. 8A in which the central ray angle or exit pupil position of the aperture is on the vertical axis and the reciprocal of the aperture diameter at the time of imaging is on the horizontal axis, and stored in the storage means 202 on that plane. A stop center ray angle or exit pupil position corresponding to a plurality of correction terms and the reciprocal of the stop diameter at the time of photographing are plotted, and adjacent points are connected by a broken line, and a block like a broken line is considered. In step # 14, the aperture center ray angle or exit pupil position at the time of imaging and the aperture diameter at the time of imaging are plotted on the plane, and storage means corresponding to the four points constituting the block including that point. The four correction terms stored in 202 are read out.

In step # 15, with respect to the four correction terms, interpolation calculation is performed for each block gain, using the aperture center ray angle or exit pupil position at the time of imaging and the reciprocal of the aperture diameter at the time of imaging as weight parameters, A correction term to be used for actually captured image data is calculated. For example, as shown in FIG. 8A, among the correction terms stored in the storage unit 202, the j-th aperture diameter F (j) arranged in ascending order of the F value and i arranged in ascending order of the numerical value. If the correction term corresponding to the second aperture center ray angle or exit aperture position θ (i) is C (i, j), if the aperture center ray angle or exit aperture position at the time of shooting is θ and the F value is F, The final correction term C is calculated according to Equation 1 below.

(Equation 1)

Next, the correction process in the first embodiment will be described.

The correction process is to adjust the correction term used for the actually captured image data calculated in the correction term adjustment calculation process to an optimum shape for each pixel, and to determine the gain to be given to each pixel of the captured image data. This is a step of calculating and multiplying the value of each pixel. For such correction, there are three patterns depending on the calculated correction term.

First, the first pattern will be described.

In the first pattern, the correction term used for the actually captured image data calculated in Step # 15 of the correction term adjustment calculation process is an optical property such as a lens in addition to the non-uniformity of the light receiving characteristics of the imaging surface. This is a correction term calculated on the assumption that correction is made including non-uniformity of the amount of light incident on the imaging surface due to the characteristics of the system. In the first pattern, the gain to be given to each pixel of the captured image data is calculated from the correction term used for the actually captured image data.

FIG. 10 is a conceptual diagram showing a method for calculating a gain to be given to each pixel of photographed image data from a correction term used for actually photographed image data.

The gain to be given to each pixel of the photographed image data is obtained by performing interpolation calculation of gains corresponding to the four blocks centered on the coordinates of the pixel for which the gain is calculated using the coordinates as weight parameters. For example, as shown in FIG. 10, the I-th block from the left of the image and the J-th block from the top (I, J) are the coordinates of the center (X (I), Y (J)), corresponding to that block. Assuming that the gain amount to be G (I, J) is the pixel (x, y) of the captured image data if the coordinates of the pixel located at the lower right of the center of the block (I, J) are (x, y). The gain g (x, y) to be given to) is calculated according to the following formula 2.

(Equation 2)

In the first pattern, the gain to be given to each pixel of the calculated captured image data is multiplied by the value of each pixel, and the correction process is completed.

Next, the second pattern will be described.

The correction term used for the actually captured image data calculated in step # 15 of the adjustment calculation of the correction term in the second pattern is assumed to correct only the non-uniformity of the light receiving characteristics of the imaging surface. Is the correction term calculated by In the second pattern, a gain to be given to each pixel of the photographed image data is calculated from the correction term, multiplied by the value of each pixel, and the correction process ends. Since this process is the same as the first pattern, the description thereof is omitted.

Next, the third pattern will be described.

The correction term used in the actually captured image data calculated in step # 15 of the adjustment calculation process for the correction term in the third pattern is only the non-uniformity of the light receiving characteristics of the imaging surface, as in the second pattern. This is a correction term calculated assuming correction. In the third pattern, first, a gain to be given to each pixel of the captured image data is calculated from the correction term, and then a correction term for correcting shading due to non-uniformity of the amount of light incident on the imaging surface. Is also converted into a gain form to be given to each pixel of the captured image data. Then, the gain calculated by multiplying both is multiplied by the value of each pixel, and the correction process is completed. Since this process is the same as the first pattern, the description thereof is omitted.

When the correction process for one of the first pattern, the second pattern, and the third pattern is completed, the corrected captured image data is subjected to white balance correction, noise processing, gamma correction, and the like by the image processing unit 108. And output to the display device 109 and the recording device 110.

Next, image data correction in the second embodiment will be described. In the second embodiment, the image data correction is roughly divided into three steps: an individual adjustment step, a correction term adjustment calculation step, and a correction step. The correction step is exactly the same as in the first embodiment. Since this is the process, the description is omitted.

Next, the individual adjustment process in the second embodiment will be described.

As described above, the adjustment step is a step of recording an optimum correction term for each individual camera, and FIG. 2 is a flowchart of the adjustment step.

In the adjustment step, first, the lens is exchanged or the zoom position / focus distance is changed. (Step # 1)

In the second embodiment, as described above, the lens to be replaced in step # 1 is assumed in advance. That is, the zoom center, the aperture center ray angle corresponding to the focus position, or the exit pupil position are stored in advance in the camera for each lens ID, so the camera refers to the information to the ray angle storage device 201 and mounts it. It is possible to obtain information such as the aperture center ray angle or the exit pupil position for each zoom position / focusing distance of the lens.

However, in the second embodiment, a camera in which correction term data corresponding to the light beam center ray angle and the aperture diameter is stored is used instead of the aperture center ray angle.

FIG. 6 is a conceptual diagram showing the luminous flux center ray angle.

As shown in FIG. 6, the light beam central ray angle indicates the central light ray incident angle of the light beam incident on the maximum image height portion of the imaging surface in each aperture state.

Therefore, in step # 1 of the second embodiment, for example, lenses that are assumed in advance are attached to the camera in order from the one with the smallest light beam central ray angle, and information is obtained by photographing.

Next, the aperture diameter of the mounted lens is changed. (Step # 2)

In Step # 2 of the second embodiment, for example, the aperture state is first set to F8.0, data is acquired by photographing in Step # 3, and the gain of each block is calculated in Step # 4. Next, F2. 8. Next, open the aperture in 4 steps, F2.0, and finally F1.4, and repeat step # 3 and step # 4 for each aperture diameter to obtain data for the four types of aperture patterns. Have acquired.

FIG. 7 is a diagram showing a correction term stored in the correction term storage device, corresponding to the aperture center ray angle, exit pupil, beam center ray angle and aperture diameter, and FIG. 7B shows the second embodiment. FIG.

Next, in step # 3, a constant light source is photographed to obtain image data. Then, the correction term calculation device 301 divides the image data into a plurality of blocks sufficient to correct the unevenness in the form as shown in FIG. 9, and calculates an average value of the signal values for each block. The gain to be given for each block is obtained as a ratio of the reciprocal of the average value of the signal value for each block so that the color signal ratio of the obtained signal value is uniform over the entire screen. As shown in FIG. 7B, the correction term storage device 202 stores them in a form corresponding to the light beam center ray angle and the aperture diameter, respectively. (Step # 4)

Further, when the correction term includes correction of the non-uniformity of the amount of light incident on the imaging surface due to the characteristics of an optical system such as a lens, the gain is an average value of signal values for each block, the center of the screen. Calculated as the inverse of the ratio of the signal value to the average value in the block. When only the non-uniformity of the light receiving characteristics of the imaging surface is corrected by such correction term, the gain is the reciprocal of the ratio of the average value of the signal value in each block to the average value of the signal value in the screen center block. Therefore, it is also conceivable to calculate as a value obtained by dividing the reciprocal of the ratio of the amount of light incident on the imaging surface from the optical system used in the adjustment step to the center of the imaging device at the block position.

When the storage is completed, the correction term calculation device 301 next determines whether or not a correction term that is a set of gains of each block has been acquired with a sufficiently large aperture diameter. (Step # 5)

In the second embodiment, as described above, shooting is performed from F8.0 (step # 3) with four types of aperture patterns in the order of F8.0, F2.8, F2.0, and F1.4, and gain calculation for each block is performed. (Step # 4) is performed. Therefore, when photographing is performed with a diaphragm F8.0 in a certain lens A (step # 3) and gain calculation for each block is performed (step # 4), the correction term calculation device 301 can only obtain a correction term for the lens A at F8.0. It is determined that it has not been acquired, and it is determined that correction terms have not yet been acquired with a sufficiently large aperture. In the second embodiment, sufficiently large aperture diameters are all the four types of aperture patterns assumed. If correction terms have not been acquired with a sufficiently large aperture diameter, the process returns to step # 2 to change the aperture diameter. For example, when only the correction term at F8.0 is acquired, the aperture diameter is set to F2.8, which is the next largest aperture diameter after F8.0, and correction at F8.0, F2.8. If the term is acquired, the aperture diameter is set to F2.0 having the next largest aperture diameter. If the correction term for F8.0, F2.8, and F2.0 is acquired, the aperture diameter is next set. F1.4 having a large value is set.

In the second embodiment, the range of the aperture state for obtaining the correction term is set to F8.0 to F1.4, the number of divisions is set to four, and the data is in the order of F8.0, F2.8, F2.0, and F1.4. In order to obtain, when the correction term at F1.4 is obtained in the lens A, it can be understood that data is obtained with four types of aperture patterns, and therefore data is obtained with all the aperture patterns in the lens A. If the correction term is acquired with a sufficiently large aperture diameter, the process proceeds to step # 6.

In step # 6, the correction term calculation device 301 determines whether or not the correction terms have been acquired with a sufficiently large number of light beam central ray angles. In the second embodiment, the light beam center ray angle is set to three types: small, standard, and large. In the second embodiment, a sufficiently large number of light beam central ray angles are all three types of patterns assumed. Then, from an interchangeable lens having a small luminous flux central ray angle, a lens having a standard luminous flux central ray angle, and a lens having a large luminous flux central ray angle, in order from the lens having the smallest luminous flux central ray angle (step # 3), each block Gain calculation (step # 4). Therefore, only in the lens A having a small light beam central ray angle, when the aperture is opened in four steps, F8.0, F2.8, F2.0, and F1.4, and correction terms are acquired with four types of aperture patterns, (Step # 5) It is determined that the correction term calculation device 301 has acquired only the correction term for the light beam center ray angle of the lens A, and it is determined that the correction term has not been acquired with a sufficiently large number of light beam center ray angles. To do.

If the correction term has not been acquired with a sufficiently large number of light beam center ray angles, the process returns to step # 1 to change the lens and change the light beam center ray angle. For example, in the above example, the lens A having a small luminous flux center ray angle is replaced with a standard lens B having a luminous flux central ray angle. (Step # 1) When Step # 2, Step # 3, and Step # 4 are repeated for the lens B, and correction terms are acquired for all four types of aperture patterns for the lens B as well as the lens A, the step # 1 is performed. From step 5, the process proceeds to step # 6, where the lens B having a standard beam center ray angle is replaced with a lens C having a large beam center ray angle. (Step # 1)

On the other hand, when data is acquired with the lens A, the lens B, and the lens C, in the second embodiment, a lens with a small beam center ray angle, a lens with a standard beam center ray angle, and a beam center ray angle are used. Since the lens with a large and the beam center ray angle are set to three patterns, the correction term calculation device 301 acquires the correction terms with a sufficiently large beam center ray angle, assuming that the correction terms are obtained for all three patterns. The adjustment process is terminated.

Next, the correction term adjustment calculation process in the second embodiment will be described.

The correction term adjustment calculation step is a step of adjusting the correction term to an optimum shape according to the shooting conditions and calculating the correction term used for the actually shot image data. FIG. It is a flowchart of this correction term adjustment process.

After the adjustment process, the image data to be corrected is taken. Then, in step # 10, the zoom control unit 112, the focus control unit 113, and the aperture control unit 114 read the focal length, the focusing distance, and the aperture diameter at the time of shooting, respectively, and in step # 11, the lens control unit 115 reads the lens ID. .

When the interchangeable lens attached to the camera capable of exchanging the optical system is an interchangeable lens assumed in advance, the lens ID is obtained from the light angle storage device 201 in the camera. On the other hand, when acquisition of the lens ID has failed, such as when the attached interchangeable lens is not an assumed interchangeable lens, a preset reference lens ID is input. (Step # 12) For example, in the second embodiment, a lens having a standard aperture center ray angle is set as a reference lens.

FIG. 5 is a conceptual diagram showing the upper aperture peripheral ray angle and the lower aperture peripheral ray angle.

The ray angle storage device 201 according to the second embodiment is stored in the camera or in each lens in the form of a non-volatile memory, and has a diaphragm center ray angle corresponding to a zoom position and a focus position for each lens assumed in advance. The maximum value and the minimum value of the light incident angle incident on the maximum image height portion of the imaging surface when the aperture is opened (hereinafter referred to as the aperture peripheral ray angle) are stored. The aperture peripheral ray angle includes an upper aperture peripheral ray angle and a lower aperture peripheral ray angle as shown in FIG. The upper aperture peripheral ray angle is the ray incidence angle of the uppermost ray incident on the maximum image height portion of the image sensor when the aperture is opened. In step # 13, the aperture center ray angle and the aperture peripheral ray angle corresponding to the lens ID, zoom position, and focus position at the time of shooting stored in the ray angle storage device 201 are read.

In step # 13-2, the CPU 111 calculates the center of the light flux from the read aperture central ray angle θupup, upper aperture peripheral beam angle θup, lower aperture peripheral beam angle θdw, and aperture F value F ′ at the time of imaging. The ray angle θcen is calculated according to the following formula 3. In addition, the necessity of step # 13-2 will be described below.
(Equation 3)

In the adjustment process of the second embodiment, the correction term storage device 202 uses the data of each lens not as a correction term corresponding to the aperture center ray angle and the F value, but as a correction term corresponding to the beam center ray angle and the aperture value. I remember it. However, the ray angle storage device 201 according to the second embodiment stores the aperture center ray angle corresponding to the zoom position and the focus position for each lens assumed in advance in the camera or in each lens.

That is, the data of each lens obtained in the adjustment step is a correction term corresponding to the light beam center ray angle and the aperture value, while the information included in the image data taken after the adjustment step includes the aperture center ray angle and the aperture peripheral ray. Since they are corners, the two cannot be matched as they are, which causes a problem.

Therefore, the light flux center ray angle is obtained from the aperture center ray angle at the time of photographing, the aperture peripheral ray angle, and the F value of the iris at the time of photographing, which is information read from the ray angle storage device 201, which is information included in the photographed image data. Since it is necessary to calculate and form the two in correspondence, it is necessary to calculate the luminous flux center ray angle in step # 13.

FIG. 8 is a conceptual diagram showing a method for selecting four correction terms to be read when calculating correction terms used for actually captured image data, and FIG. 8B is a conceptual diagram of the second embodiment.

When the alignment of the aperture center ray angle and the beam center ray angle is completed, consider a two-dimensional plane as shown in FIG. 8B, where the beam center ray angle is the vertical axis and the reciprocal of the aperture diameter at the time of photographing is the horizontal axis. The light beam center ray angle corresponding to a plurality of correction terms stored in the storage unit 202 on the plane and the reciprocal of the aperture diameter at the time of photographing are plotted, and adjacent points are connected by a broken line, and a block like a broken line think of. In step # 14, the light beam center ray angle at the time of photographing and the aperture diameter at the time of photographing are plotted on the plane, and stored in the storage means 202 corresponding to the four points constituting the block including that point. Read four correction terms.

In step # 15, interpolation calculation is performed for each gain of the block using the light flux center ray angle at the time of photographing and the reciprocal of the aperture diameter at the time of photographing for the above four correction terms, and image data actually photographed. The correction term used for is calculated. For example, as shown in FIG. 8B, among the correction terms stored in the storage unit 202, the j-th aperture diameter F (j) arranged in ascending order of the F value and the i-th arranged in ascending order of the numerical value. If the correction term corresponding to the light beam center ray angle θ (i) is C (i, j), if the light beam center ray angle at the time of photographing is θ and the F value is F, the final correction term C is It is calculated according to the number 4 of
(Equation 4)

Next, the correction process in the second embodiment is as described above, and the correction process in the first example and the second example is exactly the same process, and thus the description thereof is omitted.

In the first embodiment, a digital camera capable of exchanging lenses is assumed, and the lens is exchanged with a lens having a small aperture center ray angle, a lens having a standard aperture center ray angle, and a lens having a large aperture center ray angle. For example, in the case of a camera with an integrated optical system, step # 1 can be performed by changing the zoom position and the focusing distance. Further, in the case of a digital camera with interchangeable lenses, the same applies to the case where the aperture central ray angle is changed by changing the zoom position and the focusing distance in one lens, and step # 1 is performed.

In the second embodiment, a digital camera capable of exchanging lenses is assumed, and a step is performed by exchanging the lens with a lens having a small beam center ray angle, a lens having a standard beam ray angle, and a lens having a large beam ray angle. For example, in the case of a camera with an integrated optical system, step # 1 is performed by changing the zoom position and the focusing distance. Further, in the case of a digital camera with interchangeable lenses, the same applies to the case where the light beam center ray angle is changed by changing the zoom position and the focusing distance in one lens and the step # 1 is performed.

In the first and second embodiments, the range of the aperture state for obtaining the correction term is set to F8.0 to F1.4 and the number of divisions is set to four. However, depending on the degree of change in shading accompanying the change in aperture. The range and number of divisions should be changed.

In the first embodiment, the number of divisions of the aperture center ray angle at which the correction term is obtained is set to 3. However, the number of divisions should be changed according to the degree of shading change accompanying the change of the aperture center ray angle. It is.

In the second embodiment, the number of divisions of the beam center ray angle at which the correction term is obtained is set to three. However, the number of divisions should be changed according to the degree of shading change accompanying the change of the beam center ray angle. It is.

In the first embodiment, the correction term for correcting the signal value unevenness is stored in the correction term storage device 301 in correspondence with the aperture center ray angle and the aperture diameter. In the second embodiment, the signal value unevenness is corrected. Are stored in the correction term storage device 301 in correspondence with the light beam center ray angle and the aperture diameter. The correction terms for correcting the signal value unevenness include the ray angle and the type of the photographing lens. , Zoom position, focus position, aperture diameter parameters, presence / absence and type of converter, presence / absence and type of filter, chromatic aberration, curvature of field, distortion, lens shift type image stabilization lens shift amount, sensor shift type image stabilization sensor Shift amount, tilt shift lens tilt shift amount, soft focus lens soft amount, light source type, photographing lens transmittance, photographing lens coating Grayed, temperature sensors, among exposure time, to correspond to at least two or more items may be configured to store the correction term storage 301.

101a Zoom lens 101b Focus lens 102 Diaphragm 103 Filter 104 Image sensor 105 CDS
106 Variable Electric Amplifier 107 AD Converter 108 Image Processing Unit 108a Correction Term Adjustment Calculation Circuit 108b Correction Circuit 109 Display Device 110 Storage Device 111 CPU
112 Zoom control unit 113 Focus control unit 114 Aperture control unit 115 Lens control unit 201 Ray angle storage device 202 Correction term storage device 301 Correction term calculation circuit

Claims (9)

It has a taking lens optical system, a diaphragm, and an image sensor,
In the imaging device that acquires captured image data by the image sensor acquiring an image signal,
A first storage means, a first calculation means, a correction means, a second storage means, and a control means;
The first storage means calculates a correction term for correcting the unevenness of the signal value of the photographed image data from the parameter suitable for the interpolation calculation corresponding to the photographing condition and the parameter suitable for the storage corresponding to the photographing condition. And store them in correspondence with parameters suitable for interpolation calculation .
The second storage means stores a plurality of parameters suitable for storage corresponding to shooting conditions,
The control means calculates a part of the plurality of parameters suitable for the interpolation calculation from the plurality of parameters suitable for the storage corresponding to the photographing conditions at the time of actual photographing,
The first calculation means includes an interpolation calculation corresponding to the shooting conditions when the captured image data is acquired, including correction terms calculated by the control means among the correction terms stored in the first storage means. a plurality of parameters suitable, the correction term of the photographed image data by interpolation calculation using the correction term corresponding to a plurality of parameters suitable for near interpolation calculation,
The image pickup apparatus, wherein the correction unit corrects the unevenness of the signal value of the photographic image data using the correction term of the photographic image data calculated by the first calculation unit. It has a taking lens optical system, a diaphragm, and an image sensor,
In the imaging device that acquires captured image data by the image sensor acquiring an image signal,
A first storage means, a first calculation means, a correction means, a second storage means, and a control means;
The first storage means calculates a correction term for correcting the unevenness of the signal value of the photographed image data from the parameter suitable for the interpolation calculation corresponding to the photographing condition and the parameter suitable for the storage corresponding to the photographing condition. And store them in correspondence with parameters suitable for interpolation calculation .
The second storage means includes a light angle of a photographing condition, a photographing lens type, a zoom position, a focusing position, a parameter relating to an aperture diameter, presence or absence of a converter, presence or absence of a filter, chromatic aberration, field curvature, distortion aberration. , Lens shift type camera shake correction lens shift amount, sensor shift method camera shake correction sensor shift amount, tilt shift lens tilt shift amount, soft focus lens soft amount, light source type, taking lens transmittance, taking lens A plurality of parameters suitable for storage corresponding to at least two items of coating, sensor temperature, and exposure time are stored,
The control means includes a light angle of a photographing condition when actually photographing, a photographing lens type, a zoom position, a focus position, a parameter relating to an aperture diameter, presence / absence of a converter, presence / absence of a filter, chromatic aberration, image plane Curvature, distortion, lens shift method image stabilization lens shift amount, sensor shift method image stabilization sensor shift amount, tilt / shift lens tilt / shift amount, soft focus lens soft amount, light source type, transmission through the taking lens Calculate some of the parameters suitable for the interpolation calculation from the parameters suitable for storage corresponding to at least two items of rate, photographing lens coating, sensor temperature, exposure time,
The first calculation means includes an interpolation calculation corresponding to the shooting conditions when the captured image data is acquired, including correction terms calculated by the control means among the correction terms stored in the first storage means. a plurality of parameters suitable, the correction term of the photographed image data by interpolation calculation using the correction term corresponding to a plurality of parameters suitable for near interpolation calculation,
The image pickup apparatus, wherein the correction unit corrects the unevenness of the signal value of the photographic image data using the correction term of the photographic image data calculated by the first calculation unit. The second storage means stores in advance an aperture center ray angle and an aperture periphery ray angle corresponding to the imaging condition,
The first storage means stores the correction term corresponding to the light beam center ray angle and the parameter relating to the aperture diameter,
The control means calculates the light flux center ray angle from the aperture center ray angle and the aperture periphery ray angle at the time of actual photographing, and parameters relating to the iris diameter,
The first calculation means includes a parameter relating to a diaphragm diameter corresponding to the photographing condition when the photographed image data is acquired, and a light flux center calculated by the control means, among the correction terms stored in the first storage means. The imaging apparatus according to claim 1, wherein the correction term of the captured image data is calculated by interpolation using a parameter relating to the aperture diameter close to the light beam angle and a correction term corresponding to the light beam center light beam angle . Having a second calculating means;
The second calculation means, from the adjustment image data acquired by shooting the reference subject,
4. The imaging apparatus according to claim 1, wherein a correction term for correcting a ratio of signal values of photographed image data is calculated. The second calculation means divides adjustment image data acquired by photographing a reference subject into a plurality of blocks, calculates a gain for each block, and sets a correction term that is a set of gains for each block in advance. Calculate
The first storage means includes a parameter suitable for interpolation calculation corresponding to the imaging condition when the adjustment image data is acquired, and a correction term that is a set of gains for each block calculated by the second calculation means , and 5. The image pickup apparatus according to claim 4, wherein a plurality of parameters suitable for storage corresponding to imaging conditions are stored in association with parameters suitable for interpolation calculation calculated by the control means . The imaging apparatus according to claim 4, wherein the reference subject is a surface having uniform luminance and color. The first calculation means calculates a correction term for the non-uniformity of the light receiving characteristics of the imaging surface due to the characteristics of the imaging device;
2. The correction unit according to claim 1, wherein the correction unit performs correction by using a correction term for non-uniformity of the light receiving characteristics of the imaging surface caused by the characteristics of the imaging device calculated by the first calculation unit. The imaging device according to claim 6. It is an imaging device that can replace the lens optical system to be installed.
The imaging apparatus according to claim 1, wherein the first storage unit is disposed in the imaging apparatus. It is an imaging device that can replace the lens optical system to be installed.
The imaging apparatus according to claim 1, wherein the first storage unit is disposed in a lens to be attached.

Images