JP2002199410A - Imaging device, compensating processing method for deteriorated image and storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an inexpensive imaging device that can attain image quality of an image, that is sufficient and superior on practical level or others by compensating deterioration in the image due to imaging optical system is corrected, even if employing a comparatively inexpensive imaging optical system having no high optical performance, and to provide a deteriorated image correction processing method and a storage medium. SOLUTION: A digital still camera having a data processing section 20 that converts an output signal of an imaging element 12 conducting photoelectric conversion to an object image by the imaging optical system 11 into digital image data and having an output image memory 30 that can record image data during processing by the data processing section 20 or through the processing by the data processing section 20, is provided with a deteriorated image correction processing section 23, that applies processing to correct deteriorated image quality due to the imaging optical system 12 to the image data, before applying luminance/color separation processing and color conversion processing of the image data, on the basis of characteristic data of the image pickup optical system 11 and an output signal from the imaging element 12.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup apparatus, an image deterioration correction processing method, and a storage medium applied to a digital image pickup apparatus such as a digital still camera, and particularly to a digital image pickup apparatus which does not have high performance of an image pickup optical system. Also in this case, the present invention relates to an image pickup apparatus, an image deterioration correction processing method, and a storage medium that are suitable for realizing shooting and output of a high-quality image in which image deterioration due to the image pickup optical system is corrected.

[0002]

2. Description of the Related Art Conventionally, a digital camera had a small number of pixels at the beginning of its sale, was expressed in monochrome, and was used only for special purposes. However, the rapid cost reduction of electronic components such as CCD sensors has made color expression easy, and the spread of personal computers (PCs) has expanded the use of image data processing. It is becoming cheaper and more popular.

[0003] Recently, high quality digital cameras having more than 3 million pixels have been commercialized one after another.
Also, memories for storing (recording) images have become mainstream, shifting from those built in the digital camera body to those using removable storage media having a larger capacity. In the latter digital camera that uses a removable storage medium, processing such as taking recorded image data into a personal computer becomes very simple.

As described above, in addition to an increase in the number of recordable pixels, an improvement in the performance of a photographic optical system and an improvement in image processing technology of photographed image data have also contributed significantly to the improvement of the image quality of a digital camera. I have.

However, the use of a high-precision and high-performance imaging optical system can realize a digital camera capable of outputting a high-quality image, but causes an increase in the cost of the digital camera. Conversely, by using a relatively low-priced imaging optical system, it is possible to reduce the cost of a digital camera.However, a low-priced imaging optical system is inevitably inferior in performance, for example, off-axis chromatic aberration, Distortion aberration, a decrease in the amount of peripheral light, and the like occur, and the image quality deteriorates, such as distortion and a density gradient occurring in the output image.

From such a viewpoint, for example, Japanese Patent Laid-Open No.
JP-A-00-69343 discloses that in a digital image pickup apparatus having an image pickup lens, a data processing unit that processes an output signal from an image pickup device to generate digital image data has a lens characteristic of the photographing lens and a position of a photographed frame in an image There has been proposed a digital imaging device having a function of correcting image quality deterioration caused by a photographing lens using the above.
Further, in an embodiment of a digital imaging apparatus disclosed in Japanese Patent Application Laid-Open No. 2000-69343, a digital imaging device obtains digital image data of a subject using an imaging element using a lens having relatively large aberration, performs various image processing, and performs a photographing lens operation. 2 shows a process flow of correcting image quality deterioration caused by the above and obtaining final output image data.

[0007]

However, the above-mentioned prior art has the following problems. That is,
Unless the order of various types of image processing and the characteristics of image deterioration factors caused by the imaging optical system are taken into account, the above correction processing becomes meaningless. For example, image data of each color system detected by the image sensor is subjected to various kinds of image processing such as low-pass filter processing and edge enhancement.
It is converted to a color difference signal.

The luminance Y and the color difference C of a certain pixel are calculated by using the respective color system luminance data R, G, and B of the corresponding pixel, Y = 0.59R + 0.3G + 0.11B... CR = RY. It is generally obtained by a conversion formula expressed as = BY…. Also in the conversion process of the color system, it is common to use the corresponding pixel data of another color system in order to calculate the pixel data of a certain color system.

Further, in general-purpose or relatively inexpensive digital imaging devices, a product configuration using a single imaging device such as a single-chip CCD or CMOS is generally used. In this case, a color separation filter is arranged in each pixel of the image sensor in an arbitrary periodic array to perform color separation.

An example of an arrangement of primary color separation filters on the light receiving surface of the image sensor will be described for convenience with reference to FIGS. An image formed on the surface of the image sensor is detected by the image sensor as an image signal reflecting the spectral characteristics of the filters of each color system. At this time, one pixel has data of only one color system of the corresponding color separation filter. For this reason, since a certain color system image signal has discrete values as shown in FIG. 9, some interpolation processing is generally performed on each color system data, and finally one pixel is finally obtained. Processing is performed so as to have image signals of all color systems. It is also conceivable that this interpolation processing is performed after the conversion processing of the color system or the conversion into the luminance signal and the color difference signal.

Therefore, an image picked up by an image pickup device using an image pickup optical system having an image deterioration factor, such as off-axis chromatic aberration, in which an image forming position differs in each color system on the light receiving surface of the image pickup system is converted into a color system. And the conversion process to the luminance signal and the chrominance signal, for example, in the area where the image height is large, the luminance and the chrominance are calculated based on the data whose imaging position is shifted for each color system due to off-axis chromatic aberration. Is calculated and processed before the final image is created, so that there is a problem that the image degradation becomes extremely noticeable.

SUMMARY OF THE INVENTION The present invention has been made in view of the above points, and even if a relatively inexpensive imaging optical system having a low optical performance is used, image deterioration due to the imaging optical system is corrected, and a practically sufficient image is obtained. It is an object of the present invention to provide an inexpensive imaging device, an image deterioration correction processing method, and a storage medium that can obtain good image quality and the like.

[0013]

According to a first aspect of the present invention, there is provided a data processing means for converting an output signal of an image sensor, which photoelectrically converts a subject image by an imaging optical system, into digital image data. A recording unit capable of recording image data being processed by the data processing unit or digital image data processed by the data processing unit, wherein the characteristic data of the imaging optical system and the An image deterioration correction processing means for performing processing for correcting image quality deterioration due to the imaging optical system on the image data before performing luminance / color separation processing and color conversion processing of the image data based on the output signal. And

In order to achieve the above object, the invention according to claim 2 is characterized in that the recording means is a recording means built in the imaging apparatus main body or a recording means detachable from the imaging apparatus main body.

In order to achieve the above object, a third aspect of the present invention is to determine whether or not to execute the image deterioration correction processing by the image deterioration correction processing means according to a characteristic value of characteristic data of the imaging optical system. Is characterized by having an image deterioration correction processing judging means for judging.

According to a fourth aspect of the present invention, the image quality deterioration corrected by the image deterioration correction processing means is at least one of off-axis chromatic aberration and distortion of the imaging optical system. It is characterized by including one.

In order to achieve the above object, the invention according to claim 5 is characterized in that the imaging optical system is capable of zooming and has a detection mechanism capable of detecting a zoom state of the imaging optical system. It is characterized in that characteristic data in a zoom state of the imaging optical system can be obtained.

In order to achieve the above object, the invention according to claim 6 is characterized in that the imaging optical system is replaceable and has an identification mechanism capable of identifying the type of the imaging optical system. It is characterized in that characteristic data in an imaging optical system can be obtained.

In order to achieve the above object, the invention according to claim 7 is characterized in that the characteristic data of the photographing optical system is stored in a recording means built in the main body of the imaging device or is detachable from the main body of the imaging device. It can be stored in a recording unit, or can be stored in a recording unit built in a main body of the imaging apparatus via an interface from outside.

In order to achieve the above object, according to the present invention, a display means for displaying a photographed image and an image deterioration correction processing means for performing image deterioration correction processing in accordance with a frame display time interval of the picked-up image. Display control means for causing the display means to display a photographed image based on any of the digital image data which has not been subjected to the image deterioration correction processing.

According to a ninth aspect of the present invention, there is provided a display device for displaying a photographed image, wherein the image deterioration correction processing unit does not perform image deterioration correction processing in accordance with an external selection. Display control means for displaying a photographed image based on any of the digital image data subjected to the image deterioration correction processing on the display means.

In order to achieve the above object, the invention according to claim 10 is characterized in that the data used for the image deterioration correction processing by the image deterioration correction processing means includes lens design values, lens aberration data, lens aberration correction parameters and the like. The data is effective in correcting off-axis chromatic aberration and the distortion.

In order to achieve the above object, the invention according to claim 11 is a data processing means for converting an output signal of an image pickup device for photoelectrically converting a subject image by an image pickup optical system into digital image data, and the data processing means. An image deterioration correction processing method applied to an imaging apparatus having a recording unit capable of recording image data being processed or digital image data processed by the data processing unit, wherein the characteristic data of the imaging optical system and the Before performing the luminance / color separation processing and the color conversion processing of the image data based on the output signal of the image sensor, the image data is subjected to processing for correcting the image quality deterioration due to the imaging optical system.

In order to achieve the above object, a twelfth aspect of the present invention is characterized in that the recording means is a recording means built in the imaging device or a recording device detachable from the imaging device.

According to a thirteenth aspect of the present invention, in order to achieve the above object, it is possible to determine whether or not to execute the image deterioration correction processing by the image deterioration correction processing means according to a characteristic value of characteristic data of the imaging optical system. Is determined.

In order to achieve the above object, the invention according to claim 14 is characterized in that the image quality deterioration subjected to the image deterioration correction processing includes at least one of off-axis chromatic aberration and distortion of the imaging optical system. It is characterized by the following.

In order to achieve the above object, the invention according to claim 15 is characterized in that the imaging optical system is capable of zooming,
It is characterized in that characteristic data in the zoom state of the imaging optical system can be acquired by a detection mechanism capable of detecting the zoom state of the imaging optical system.

In order to achieve the above object, the invention according to claim 16 is characterized in that the imaging optical system is replaceable and the characteristic data in the imaging optical system is identified by an identification mechanism capable of identifying the type of the imaging optical system. It can be acquired.

In order to achieve the above object, an invention according to claim 17 is characterized in that the characteristic data of the photographing optical system is stored in a recording means built in the image pickup apparatus or is detachable from the image pickup apparatus. It can be stored in a recording unit, or can be stored in a recording unit built in the imaging apparatus from an external device via an interface.

[0030] In order to achieve the above object, the invention according to claim 18 is characterized in that, according to a frame display time interval of a captured image,
A captured image based on any of the digital image data that has not been subjected to the image deterioration correction processing or has been subjected to the image deterioration correction processing is displayed.

In order to achieve the above object, the invention according to claim 19 is characterized in that the digital image data which has not been subjected to the image deterioration correction processing or has been subjected to the image deterioration correction processing in accordance with an external selection. And displaying a captured image based on the image.

According to a twentieth aspect of the present invention, the data used in the image deterioration correction processing includes the off-axis chromatic aberration such as lens design values, lens aberration data, lens aberration correction parameters, and the distortion. The data is characterized by being effective data for correction.

In order to achieve the above object, an invention according to claim 21 is a data processing means for converting an output signal of an image pickup device for photoelectrically converting a subject image by an image pickup optical system into digital image data; A storage device readable by a computer storing a program for executing an image deterioration correction processing method applied to an imaging apparatus having a recording unit capable of recording image data being processed or digital image data processed by the data processing unit. A medium, wherein the image deterioration correction processing method comprises the steps of: performing a luminance / color separation process and a color conversion process on the image data based on characteristic data of the imaging optical system and an output signal of the image sensor; And an image deterioration correction processing step of performing processing for correcting image quality deterioration due to the imaging optical system.

[0034] In order to achieve the above object, the invention according to claim 22 is characterized in that the recording means is a recording means built in the imaging device or a recording device detachable from the imaging device.

According to a twenty-third aspect of the present invention, in order to achieve the above object, it is possible to determine whether or not to execute the image deterioration correction processing by the image deterioration correction processing means according to a characteristic value of characteristic data of the imaging optical system. And an image deterioration correction processing determining step of determining the

In order to achieve the above object, according to a twenty-fourth aspect of the present invention, the image quality deterioration corrected in the image deterioration correcting step is at least one of an off-axis chromatic aberration and a distortion of the imaging optical system. It is characterized by including one.

According to a twenty-fifth aspect of the present invention, in order to achieve the above object, the imaging optical system can be zoomed,
A detecting step of detecting a zoom state of the imaging optical system, wherein characteristic data in the zoom state of the imaging optical system can be acquired by the detecting step.

In order to achieve the above object, the invention according to claim 26 has an identification step in which the imaging optical system is replaceable and the type of the imaging optical system can be identified,
It is characterized in that characteristic data in the imaging optical system can be acquired by the identification step.

In order to achieve the above object, the invention according to claim 27 is characterized in that the characteristic data of the photographing optical system is stored in a recording means built in the image pickup apparatus or is detachable from the image pickup apparatus. It can be stored in a recording unit, or can be stored in a recording unit built in the imaging apparatus from an external device via an interface.

In order to achieve the above object, the invention according to claim 28 is characterized in that, according to the frame display time interval of a captured image,
The image processing apparatus further includes a display control step of displaying, on a display unit, a captured image based on any of the digital image data that has not been subjected to the image deterioration correction processing or has been subjected to the image deterioration correction processing in the image deterioration correction processing step.

In order to achieve the above object, the invention according to claim 29, wherein the digital image which has not been subjected to the image deterioration correction processing or has been subjected to the image deterioration correction processing in the image deterioration correction processing step according to an external selection. A display control step of displaying a captured image based on any of the data on the display means.

In order to achieve the above object, the invention according to claim 30 is characterized in that the data used for the image deterioration correction processing in the image deterioration correction processing step includes the lens design values, lens aberration data, lens aberration correction parameters and the like. The data is effective in correcting off-axis chromatic aberration and the distortion.

[0043]

Embodiments of the present invention will be described below in detail with reference to the drawings.

[First Embodiment] A first embodiment of the present invention corresponds to the first, second, third and fourth aspects of the present invention.

FIG. 1 is a block diagram showing a configuration example of a digital still camera which is an example of a digital imaging device according to the first embodiment of the present invention. The digital still camera according to the first embodiment of the present invention includes an imaging unit 10 having an imaging optical system 11 and an imaging element 12, an AGC (auto gain controller), an A / D conversion unit 21, and an image data conversion unit 22. A data processing unit 20 having an image deterioration correction processing unit 23, an image processing unit 24, and a compression processing unit 25; an output image memory 30; a data interface unit 31; a user interface unit 33; Is provided.

The above-described configuration will be described in detail. The CPU and the CPU of the built-in memory unit 34 have a function of managing the entire digital still camera and executing processing instructions. In the figure, the dotted arrows indicate the flow of processing, and the solid arrows indicate the flow of image data.
The imaging unit 10 includes a CPU and a CPU of the built-in memory unit 34 when a shutter (not shown) of the digital still camera is pressed and when the image display unit 32 is used as a finder.
Is a part for projecting the subject on the light receiving surface of the image pickup device 12 by the imaging optical system 11 through the instruction of (1). Here, the imaging optical system 11 of the imaging unit 10 has a zoom function, and the zoom state at the time of shooting is CP.
U has a function that can be detected via the CPU of the built-in memory unit 34. The data of the off-axis chromatic aberration and the distortion of the imaging optical system 11 are stored in the CPU and the internal memory of the internal memory 34.

The imaging device 12 of the imaging unit 10 is a single photoelectric conversion device that converts an optical signal based on an image formed on the light receiving surface into an electric signal for each light receiving pixel corresponding to a position. In addition, the image sensor 12 has a function of performing color separation by the R, G, and B filters arranged in a checkered pattern shown in FIG. The exposure time of the image sensor at the time of shooting and the timing of obtaining the subject image are controlled by the CPU and the CPU of the built-in memory unit 34, and the data of the image obtained as the electric signal is controlled by the CPU and the CPU of the built-in memory unit 34 The data is transferred to the data processing unit 20.

The image signal transferred to the data processing section 20 is first transferred to the AGC / A / D conversion section 21. The transferred image signal is amplified and intensity-corrected by AGC, converted to digital data by an A / D converter, and transferred to the image data converter 22. Further, the transferred digital data is converted into digital image (gradation) data by the image data conversion unit 22.

The digital image data converted by the image data conversion unit 22 is transferred to the image processing unit 24 with or without the image correction processing by the image deterioration correction processing unit 23. This processing branch is performed by the image display unit 3
The captured image acquired to use the No. 2 as a finder is not subjected to the image deterioration correction processing, and the captured image taken for storing the image on the recording medium by pressing the shutter is subjected to the image deterioration correction processing. Occurs. When the CPU and the CPU of the built-in memory unit 34 determine that the image deterioration correction processing is not necessary based on the characteristic values of the off-axis chromatic aberration and distortion of the imaging optical system at the time of shooting, the image taken by pressing the shutter is taken. Even an image is transferred to the image processing unit 24 without performing image deterioration correction processing.

The image deterioration correction processing unit 23 includes the image pickup optical system 1
1 is a part for correcting the image deterioration due to 1. The image data transferred to the image deterioration correction processing unit 23 according to the instruction of the CPU and the CPU of the built-in memory unit 34 includes the off-axis chromatic aberration characteristic and distortion of the imaging optical system 11 stored in the built-in memory unit of the CPU and the built-in memory unit 34. Image deterioration due to off-axis chromatic aberration and distortion of the imaging optical system 11 is corrected based on the aberration characteristics. In this case, the data used for the image deterioration correction processing by the image deterioration correction processing unit 23 may be any data that is effective for correcting off-axis chromatic aberration and distortion, such as lens design values, lens aberration data, and lens aberration correction parameters. .

On the other hand, the image processing unit 24 performs an image conversion process based on a plurality of color system data, such as a color system conversion process and luminance color separation. Generally, these image conversion processes are performed on the assumption that the subject image formation position in each color system matches an arbitrary pixel position.

In a general digital still camera, RGB primary color data is converted into luminance Y, color difference CR, and CB data using the above conversion formula. If the off-axis chromatic aberration of the imaging optical system 11 is relatively large, pixel data whose imaging positions do not match in each color system will be used in the peripheral portion of the image for the above-described reason.
Large image degradation occurs. In order to prevent this image deterioration, it is necessary to perform off-axis chromatic aberration correction prior to color system conversion processing and luminance / color separation processing. Therefore, when the CPU and the CPU of the built-in memory unit 34 instruct to perform the image deterioration correction processing, the image data is transferred from the image data conversion unit 22 to the image deterioration correction processing unit 23, and the image deterioration correction processing is executed. Later, it is transferred to the image processing unit 24.

The processing of the image deterioration correction processing section 23 is represented by a block diagram as shown in FIG. That is, as shown in FIG. 2, the image deterioration correction processing unit 23 includes a coordinate conversion unit 23a and an interpolation processing unit 23b. The off-axis chromatic aberration and distortion of the digital image data are determined by the CPU and the built-in memory unit 3.
Using the aberration characteristic data stored in the internal memory unit of No. 4, coordinate conversion is performed on the R, G, and B image data by the coordinate conversion unit 23a, and thereafter, the interpolation processing unit 23b
Is corrected by performing the interpolation processing in. The coordinate transformation is
This is performed by obtaining a new coordinate system so as to cancel the displacement of the subject image due to off-axis chromatic aberration and distortion occurring in each pixel of the image data. Next, the interpolation process is performed to convert the coordinate system, which is no longer a square lattice coordinate, to a square lattice coordinate system again.

When performing coordinate transformation, as shown in FIG.
Center coordinates (Xo, Y) of image data expanded in a square grid
Using the distance between o) and the position of each pixel, the displacement of the pixel due to the aberration of the imaging optical system 11 of the imaging unit 10 is corrected.
Here, the center coordinate is a coordinate that coincides with the optical axis position on the light receiving surface of the imaging element 12 of the imaging unit 10. Off-axis chromatic aberration and distortion are generally obtained as a function of image height and aberration on an image plane, as shown in FIG. This aberration characteristic value is obtained for each pixel on the image plane, and the off-axis chromatic aberration and distortion are summed up to obtain the aberration at each pixel position as aberration characteristic data as shown in FIG. It is stored in the built-in memory section of the CPU and the built-in memory section 34.

The off-axis chromatic aberration has different characteristics for each of RGB, so that the aberration characteristic data exists for each of R, G, and B. The off-axis chromatic aberration is obtained for an arbitrary wavelength as shown in FIG. 5, but R, G, and B are characterized by spectral characteristics having a finite wavelength width. For this reason, to determine the off-axis chromatic aberration of each color system, a method of determining the center-of-gravity wavelength of each color system and determining the amount of off-axis chromatic aberration at that wavelength, or using the wavelength of the maximum value of the spectral intensity, is used. A method of obtaining the amount of chromatic aberration may be used, but is not particularly specified. In addition, although off-axis chromatic aberration has different amounts of aberration in R, G, and B, in order to reduce the amount of calculation, aberration data of only off-axis chromatic aberration of only R and B is obtained based on the off-axis chromatic aberration of G in advance. You may keep it.

Next, the coordinate data of the relative distance normalized by the maximum image height of the light receiving surface of the image pickup device 12 is defined by using the center coordinates (Xo, Yo) of the square grid coordinates in FIG. 3 as the origin (0, 0). Is calculated, and the sum of the R, G, and B aberration characteristic data is calculated to obtain coordinate-transformed coordinate data (FIG. 6). R, G,
The coordinates at (Xn, Yn) on the square lattice coordinates of each B in FIG. 3 are converted as follows.

[0057]

(Equation 1) Ab _RXn , Ab _GXn , Ab _BXn : X component of the sum of off-axis chromatic aberration and distortion in each of the R, G, B color systems, Ab _RXn , Ab _GXn , Ab
_BXn : Y component of the sum of off-axis chromatic aberration and distortion in each of the R, G, B color systems, Xo, Yo: Relative X between image center point and maximum image height position
Coordinate and relative Y coordinate, h: maximum image height on the light receiving element surface.

As a result, the image data transferred from the image data conversion unit 22 can correspond to the coordinate data subjected to the coordinate conversion in FIG. 6 as object image coordinate data when there is no off-axis chromatic aberration and distortion. it can.

The two-dimensional coordinate position data obtained as described above and the gradation data of the image transferred from the image data conversion unit 22 correspond to the two-dimensional coordinate position corrected for aberration and the gradation data. I have. Since coordinate data subjected to coordinate conversion for aberration correction is not in a square lattice coordinate system, the coordinate data is converted into square lattice coordinates. At this time, the aberration-corrected image is no longer a rectangle as shown in FIG.
Determine the maximum area that can be extracted with a rectangle of the desired aspect ratio so that no image is missing in each of the R, G, B color systems, and determine the square grid coordinate system according to the number of pixels of the required specifications, The gradation data of the image corresponding to each coordinate of the square lattice coordinate system is obtained by performing an interpolation process.

Although the method of this interpolation processing is not particularly specified,
Interpolation is performed by a method such as four-point linear interpolation or cubic convolution interpolation. Further, in order to secure a finally required image area, a wider image range may be photographed by the image sensor in advance, assuming that an image is missing due to aberration correction.

The digital still camera of the first embodiment uses a single-chip image sensor having a checkerboard color filter array as shown in FIG. As shown in FIG. 9, the gradation data of the image B includes pixels having gradation data and pixels not having the gradation data in each color system. For this reason, in the above-described interpolation processing, the gradation data of the pixel having no data is also interpolated at the same time so that one pixel has the gradation data of the entire color system.

The image data subjected to the image deterioration correction is transferred to the image processing section 24, where various image processing such as luminance / color separation processing, white balance adjustment, gray balance adjustment, density adjustment, color balance adjustment, and edge enhancement are performed. Will be applied.

The CPU and the CPU of the built-in memory unit 34
The image data which is determined not to perform the image degradation processing is transferred from the image data conversion unit 22 directly to the image processing unit 24. First, as shown in FIG. 9, the image data is subjected to interpolation processing of gradation data of pixels for which no data exists in each color system, and thereafter, luminance color separation processing, white balance adjustment, gray balance adjustment, density Various image processing such as adjustment, color balance adjustment, and edge enhancement are performed.

The image data subjected to the image processing by the image processing unit 24 is transferred to a compression processing unit 25 and compressed by a predetermined image compression processing method. The compression method is JPEG
(Joint Photographic Expert Group), TIFF (Tag Ima
(Ge File Format), JPEG2000, etc., as long as the method can compress image data. The compressed image is transferred to and stored in the output image memory 30 as an output image.

Output image memory 30 for storing output images
Is a recording medium that can be attached to and detached from the digital still camera, and enables direct transfer of output images to other information terminal devices via this recording medium. The output image memory 30 is a rewritable memory, for example, a card-shaped memory such as a compact flash (registered trademark), a smart media, a memory stick, a PC card, an ATA card, an MO disk, a floppy (registered trademark) disk, ZI
Any form such as a general-purpose memory such as P, CD-R, CD-RW, and DVD-ROM may be used. The output image memory 30
Can be transferred to another information terminal device by wire transmission or wireless transmission via the data interface unit 31.

The image display section 32 is composed of, for example, a liquid crystal display, and can display a subject image as a finder. An image displayed as a finder is displayed using image data that has not been subjected to image deterioration correction processing. The image display function of the image display unit 32 can be switched ON / OFF via the user interface unit 33.

As described above, according to the digital still camera according to the first embodiment of the present invention, the imaging unit 10 including the imaging optical system 11 and the imaging device 12, and the imaging device 1
2, a data processing unit 20 that processes the output signal from the data processing unit 2 into digital image data, and a built-in unit that can record image data being processed in the data processing unit 20 or digital image data processed by the data processing unit 20 In the digital still camera provided with the recording medium or the detachable recording medium 30, the luminance / color separation of the image data in the image processing unit 24 is performed based on the characteristic data of the imaging optical system 11 and the output signal captured from the imaging device 12. Before performing the processing and the color conversion processing, the image deterioration correction processing unit 23 performs the processing for correcting the image quality deterioration by the imaging optical system 11 on the image data.

The image signal picked up by the image pickup unit 10 and the zoom state of the image pickup optical system 11 at the time of image pickup are detected, and the CP is detected.
U, the off-axis chromatic aberration and distortion of the imaging optical system are stored by using the aberration data of the sum of off-axis chromatic aberration and distortion of the imaging optical system in the corresponding zoom state stored in the internal memory unit of the built-in memory unit 34. It is possible to correct image deterioration caused by aberration. This allows for off-axis chromatic aberration,
Even when imaging is performed using an inexpensive imaging optical system having large distortion, an image with a practically sufficient image quality can be obtained. That is, it is possible to satisfactorily hold the output image while reducing the cost of the imaging optical system.

The digital still camera described in the first embodiment of the present invention is not limited to the above-described embodiment. For example, the image pickup device 12 may be a single plate type or a three plate type. May be used. In addition, any specification may be adopted as long as the gist of the first embodiment is followed. Further, the image deterioration correction processing unit 23 is not limited to the configuration shown in FIG. 2, and may have a configuration in which a color system interpolation processing unit 23 c is provided in a stage preceding the coordinate conversion unit 23 a as shown in FIG. 10. .

[Second Embodiment] A second embodiment of the present invention is described in claims 1, 2, 3, 4,
This corresponds to the inventions described in 6, 7, and 8.

The digital still camera according to the second embodiment of the present invention comprises an image pickup unit 10 having an image pickup optical system 11, an image pickup device 12, an AG
C, an A / D conversion unit 21, an image data conversion unit 22, an image deterioration correction processing unit 23, an image processing unit 24, a data processing unit 20 including a compression processing unit 25, an output image memory 30, a data interface unit 31, , A user interface unit 33, a CPU, and a built-in memory unit 34 (see FIG. 1). The configuration of each part has been described in detail in the first embodiment, and a description thereof will be omitted.

In the second embodiment of the present invention, an example of a digital still camera in which the imaging optical system is replaceable will be described. The difference between the digital still camera according to the second embodiment of the present invention and the digital still camera according to the first embodiment in function is that the imaging optical system 11 in the imaging unit 10 is replaceable. It is. The replaceable imaging optical system 11 is equipped with a mechanism capable of mechanically or electrically identifying which imaging optical system and an identification mechanism capable of identifying which zoom state. The detection mechanism of the imaging unit 10 via the CPU of the built-in memory unit 34 identifies the currently mounted imaging optical system and the name or identification code of the zoom state,
It is detected and stored in the CPU and the internal memory of the internal memory.

The aberration characteristic data of the exchangeable image pickup optical system 11 is acquired by taking it into a digital still camera from an external aberration characteristic information source. External aberration information sources include aberration information such as recording media such as CDs and floppy disks equipped with aberration characteristic information attached to the product package of the exchange imaging optical system, and websites of companies that sell the relevant exchange imaging optical system. If it can be obtained, it is not specified.

Each zoom state of the corresponding image pickup optical system and each corresponding aberration information, together with the name of the image pickup optical system and zoom state or identification code, are output to the CP via the output image memory 30 detachable from the digital still camera.
U, stored in the built-in memory section of the built-in memory section 34. Alternatively, the corresponding aberration information is stored in the built-in memory section of the CPU and the built-in memory section 34 via the data interface section 31 together with the name or identification code of the imaging optical system.

At this time, the information is transferred by storing the aberration information once fetched from the aberration information source into an information terminal such as a personal computer in the detachable output image memory 30 via the information terminal, or For example, a method of transferring the aberration information to the data interface unit 31 via the CPU or the like can be considered, but any method may be used as long as the aberration information can be finally stored in the CPU and the internal memory unit of the internal memory unit 34.

At the time of imaging, based on the imaging optical system currently mounted on the digital still camera and the name or identification code of the zoom state, the CPU and the imaging optical system stored in the built-in memory of the built-in memory unit 34 by the above method. The aberration correction data of the system is collated, and if they match, the aberration correction data is used for an image deterioration correction process caused by the imaging optical system. At this time, the aberration characteristic data is the same as in the first embodiment, and is the sum of the values of off-axis chromatic aberration and distortion, and the format is R, G, and B color systems, respectively, as shown in FIG. Are the horizontal and vertical aberration values on the light receiving image plane corresponding to each pixel position of the image sensor.

Although the aberration differs for each zoom state, the aberration characteristics of the main zoom states are stored in the CPU and the built-in memory of the built-in memory unit 34, and the aberration characteristics of the other zoom states are stored in the CPU and the built-in memory. Memory part 3
4 can be obtained by interpolation from the aberration characteristics of the zoom state stored in the built-in memory unit.

It is assumed that the aberration is small depending on the image pickup optical system used for image pickup, and that the image deterioration processing caused by the image pickup optical system is unnecessary. In some color systems, aberrations are so small that image deterioration correction processing is unnecessary, while in other color systems, aberrations are large and image deterioration correction processing is required. The CPU of the built-in memory unit 34 includes R, G, B
Based on the value of the aberration characteristic data of each color system, only the color systems for which it is recognized that aberration correction is necessary are determined and stored so as to perform the image deterioration correction processing. The CPU and the CPU of the built-in memory unit 34 execute a branch process of performing or not performing the image deterioration correction process based on the storage.

The CPU and the CPU of the built-in memory unit 34
The determination of the execution of the image deterioration correction process based on the aberration characteristic data of each color system described above is based on the fact that the aberration characteristic of the imaging optical system is C
The PU is executed when the image data is loaded into the built-in memory unit of the built-in memory unit 34. At the same time, a flag indicating that the process is performed is given to aberration data of the color system determined to perform the image deterioration correction process,
A flag indicating that no processing is performed is given to aberration data of the color system determined not to perform the image deterioration correction processing, and the data itself is deleted.

There are two cases in which imaging is performed using the digital still camera according to the second embodiment of the present invention. That is, a case where the captured image is displayed using the image display unit 32 as a finder and a case where the captured image is stored in the output image memory 30 by pressing the shutter button. Basically, in either case, the image deterioration correction processing is performed on the captured image data. When the image is captured as a finder image, the captured image is displayed on the image display unit 32 at a predetermined frame display time interval that can satisfy the function of the finder.

However, if the predetermined frame time interval cannot be achieved due to a heavy load of the image deterioration correction processing, the CPU and the CPU of the built-in memory unit 34 automatically determine the data without performing the image deterioration correction processing. Processing is performed, and the captured image is displayed on the image display unit 32.

As described above, according to the digital still camera according to the second embodiment of the present invention, even in a replaceable imaging optical system such as a single-lens reflex camera, a mechanism for identifying the exchange imaging optical system, By incorporating the aberration characteristics of a mechanism for identifying which zoom state is used from an external aberration information source into the CPU and the built-in memory unit of the built-in memory unit 34, image deterioration due to off-axis chromatic aberration and distortion of the imaging optical system is corrected. It becomes possible. Accordingly, even when imaging is performed using an inexpensive imaging optical system having large off-axis chromatic aberration and distortion, an image having a practically sufficient image quality can be obtained.
That is, it is possible to satisfactorily hold the output image while reducing the cost of the imaging optical system.

[Third Embodiment] A third embodiment of the present invention is described in claims 1, 2, 4, 6,
This corresponds to the inventions described in 7 and 9.

FIG. 11 is a block diagram showing a configuration example of a digital camera according to the third embodiment of the present invention. The digital still camera according to the third embodiment of the present invention includes:
An imaging unit 10 having an imaging optical system 11 and an imaging element 12;
AGC, A / D conversion unit 21, image data conversion unit 22, image deterioration correction processing unit 23, image processing unit 24, compression processing unit 2
5. Data processing unit 2 having image data interpolation processing unit 26
0, an output image memory 30, a data interface unit 31, a user interface unit 33, a CPU,
And a built-in memory unit 34.

In the third embodiment of the present invention, an example of a digital still camera in which the imaging optical system is replaceable will be described. The imaging optical system 11 in the imaging unit 10 is replaceable, and the imaging element 12 uses a single-chip CMOS having a color filter array as shown in FIG. 12 for performing color separation by a complementary color system of Y, M, C, and G. I have. The replaceable imaging optical system 11 is equipped with a mechanism capable of mechanically or electrically recognizing aberration characteristics in the current zoom state. By the detection mechanism of the unit 10, the CPU and the CPU of the built-in memory unit 34 can sequentially recognize the aberration characteristics of the imaging optical system 11 currently mounted on the digital still camera in the current zoom state.

The specifications are such that the aberration characteristic in each zoom state of the imaging optical system 11 is stored in the CPU and the built-in memory unit of the built-in memory unit. Alternatively, the current aberration characteristics of the imaging optical system can be manually input via the user interface unit 33 and stored in the built-in memory unit of the CPU and the built-in memory unit 34.

The aberration characteristics will be described. In an imaging optical system whose shape is symmetric with respect to the central axis, the aberration characteristics are also axially symmetric. Off-axis chromatic aberration and distortion in the imaging optical system can be represented by a power function of the image height with respect to the image plane of the imaging device. In addition, it is possible to express the aberration characteristic in terms of up to the fifth order.

As the aberration characteristics, the sum of the off-axis chromatic aberration and the distortion of each of the color system, Y, M, C, and G is obtained, expressed by a power function of the image height, and the aberration coefficient is picked up to the fifth order term. It is sufficient if the data is stored in the optical system, and the data amount is small. Further, although the aberration differs for each zoom state, the aberration characteristics of the main zoom state are stored in the imaging optical system, and the aberration characteristics of the other zoom states are stored in the CPU,
It can be obtained by interpolation from the zoom state aberration characteristics already stored in the built-in memory unit of the built-in memory unit 34.

There are two cases in which imaging is performed using the digital still camera according to the third embodiment of the present invention. That is, a case where the captured image is displayed using the image display unit 32 as a finder and a case where the captured image is stored in the output image memory 30 by pressing the shutter button. The captured image when the shutter button is pressed is always subjected to the image deterioration correction processing, and the image captured as the finder image is subjected to the image deterioration correction processing by the input through the user interface unit 33 or not. It is possible to choose. When the image deterioration correction processing is performed, the processing becomes slow, so that the above selection can be performed according to the speed and the image quality at which the subject is to be captured as a finder.

The captured image is processed by the AGC, A / D converter 21 and image data converter 22 to obtain
It is converted to digital image data for each of the Y, M, C, and G color systems. The image data at this stage is in a state where there is a pixel having no gradation data as shown in FIG.
The pixel data is processed by the image data interpolation processing unit 26.
Each pixel is converted into data in which gradation data of all color systems exists. Thereafter, the image data is transferred to the image deterioration correction processing unit 23 when the image deterioration correction processing is performed, and is transferred to the image processing unit 24 when the image deterioration correction processing is not performed.

The image deterioration correction processing in the image deterioration correction processing section 23 is performed in the same manner as in the first embodiment and the second embodiment, as shown in FIG. This is performed by the interpolation processing by the interpolation processing unit 23b. In the coordinate conversion processing, first, a characteristic of the sum of the off-axis chromatic aberration and the distortion with respect to the image height is calculated from the aberration coefficient stored in the built-in memory unit of the CPU and the built-in memory unit 34.

Next, two-dimensional coordinate data corresponding to each pixel position of the image sensor 12 as shown in FIG. 3 is created. At this time, the center coordinate corresponding to the optical axis position is determined as the origin, the distance from the center coordinate to each of the other coordinate points is obtained, and the distance is normalized by the maximum image height. Assuming that coordinates (Xn, Yn) on the square grid coordinates in FIG. 3 are converted to coordinates (X ′, Y ′),

[0093]

(Equation 2) Becomes Here, h is the maximum image height on the light receiving element surface.

This two-dimensional coordinate data is created for each of the color systems Y, M, C, and G. Based on the coordinate values, an image height h ′ at each pixel position is obtained, and the value is used to calculate the CP.
U, two-dimensional coordinate position data based on data of the sum of off-axis chromatic aberration and distortion of the imaging optical system stored in the built-in memory unit of the built-in memory unit 34; (H '). Then, the aberration amount is added to the corresponding two-dimensional coordinate position data in the horizontal (x-axis) direction and the vertical (y
After projecting in the (axis) direction, each is added and converted into two-dimensional coordinate position data (X '', Y '').

[0095]

(Equation 3) As a result, the image data transferred from the image data interpolation processing unit 26 has two-dimensional coordinate position data (X ″, X ′) as object image coordinate data when there is no off-axis chromatic aberration and distortion.
Y '').

The two-dimensional coordinate position data obtained above,
The image data transferred from the image data interpolation processing unit 26 corresponds to the two-dimensional coordinate position subjected to the aberration correction and the gradation data. Since the two-dimensional coordinates corrected for aberration are not square grid coordinates, they are converted to square grid coordinates. At this time, since the aberration-corrected image is no longer a rectangle as shown in FIG. 7, the image is corrected to a rectangle having a desired aspect ratio so as to prevent image loss in all of the Y, M, C, and G color systems. The maximum area that can be extracted is determined, a square grid coordinate system is determined according to the number of pixels of the required specifications, and the gradation data of the image of the extraction area is obtained by interpolation processing.
Thus, the image deterioration correction processing by the image deterioration correction processing unit 23 is completed.

The image data which has been subjected to the image deterioration correction processing is transferred to the image processing section 24, and is converted from the Y, M, C, G color system.
Various image processing such as conversion processing to R, G, and B color systems, luminance color separation processing, white balance adjustment, gray balance adjustment, density adjustment, color balance adjustment, and edge enhancement are performed.

In the case of image data captured by pressing the shutter, after image processing is performed by the image processing unit 24, image compression is performed by the compression processing unit 25 and stored in the output image memory 30. As an image captured as a viewfinder display image, an image subjected to image deterioration correction processing by the image deterioration correction processing unit 23 or an image not processed is displayed by selection through the user interface unit 34. It is possible.

As described above, according to the digital still camera according to the third embodiment of the present invention, even in an interchangeable imaging optical system such as a single-lens reflex camera, the zoom state and the aberration coefficient of its aberration characteristic can be obtained. , A CPU for determining aberration coefficients of a plurality of zoom states, and a built-in memory unit 34
Or by inputting the aberration coefficient through the user interface unit 33 to C
By taking in the PU and the built-in memory unit of the built-in memory unit 34, it is possible to correct image deterioration due to off-axis chromatic aberration and distortion of the imaging optical system. Accordingly, even when imaging is performed using an inexpensive imaging optical system having large off-axis chromatic aberration and distortion, an image having a practically sufficient image quality can be obtained. That is, it is possible to satisfactorily hold the output image while reducing the cost of the imaging optical system.

[Other Embodiments] The first embodiment of the present invention described above.
In the third to third embodiments, the case of an imaging device alone such as a digital still camera has been described as an example, but the present invention is not limited to this, and an imaging device such as a digital still camera, a computer, The present invention may be applied to a system in which an information processing device such as a portable information terminal and an image forming device such as a printer are connected.

The present invention may be applied to a system constituted by a plurality of devices or to an apparatus constituted by a single device. A medium such as a storage medium storing program codes of software for realizing the functions of the above-described embodiments is supplied to a system or an apparatus, and a computer (or CPU or MPU) of the system or the apparatus stores the medium in the medium such as a storage medium. Needless to say, this can also be achieved by reading and executing the program code thus executed.

In this case, the program code itself read from a medium such as a storage medium realizes the functions of the above-described embodiment, and the medium such as a storage medium storing the program code constitutes the present invention. Will be. Examples of a medium such as a storage medium for supplying the program code include a floppy disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R,
A magnetic tape, a nonvolatile memory card, a ROM, a download via a network, or the like can be used.

The functions of the above-described embodiments are implemented when the computer executes the readout program codes, and the OS or the like running on the computer is actually executed based on the instructions of the program codes. It goes without saying that a part or all of the above-described processing is performed, and the functions of the above-described embodiments are realized by the processing.

Further, after the program code read from a medium such as a storage medium is written into a memory provided in a function expansion board inserted into a computer or a function expansion unit connected to the computer, the program code is read out. It is needless to say that a CPU or the like provided in the function expansion board or the function expansion unit performs part or all of the actual processing based on the above, and the functions of the above-described embodiments are realized by the processing.

FIG. 15 is an explanatory diagram showing a conceptual example in which a program for executing the image deterioration correction processing method of the present invention and related data are supplied from a storage medium to a device such as a computer. A program for executing the image deterioration correction processing method of the present invention and related data are stored on a floppy disk or a CD-RO.
M is supplied by inserting a storage medium 151 such as M into an insertion slot 153 of a storage medium drive provided in a device 152 such as a computer. Thereafter, the program for executing the image deterioration correction processing method of the present invention and related data are temporarily installed on the hard disk from the storage medium 151 and loaded on the RAM from the hard disk, or directly loaded on the RAM without being installed on the hard disk. , The program and the related data can be executed.

In this case, when the digital still cameras according to the first to third embodiments of the present invention execute a program for executing the image deterioration correction processing method of the present invention, for example, refer to FIG. By supplying the program and related data to the digital still camera through a device such as a computer as described above, or by storing the program and related data in the digital still camera in advance, the program can be executed. Become.

FIG. 14 is an explanatory diagram showing a configuration example of the storage contents of a storage medium storing a program for executing the image deterioration correction processing method of the present invention and related data. The storage medium is
For example, volume information 141, directory information 14,
2, storage contents such as a program execution file 143 and a program-related data file 144. A program for executing the image deterioration correction processing method of the present invention is a program code based on the control procedure of the above-described first to third embodiments of the present invention.

[0108]

As described above, according to the first aspect of the present invention,
In the imaging device described above, before performing the luminance / color separation processing and the color conversion processing of the image data, the image data is subjected to processing for correcting image quality deterioration due to the imaging optical system. In a case where the imaging optical system is a zoomable imaging device as described in claim 5, or in a case where the imaging optical system is replaceable as in claim 6, the zoom state and the imaging optical system to be used are set. The characteristic data used for the image deterioration correction processing is acquired based on the type of the image deterioration correction processing.

In the image pickup apparatus according to the third aspect, the magnitude of image deterioration caused by the detected characteristics of the image pickup optical system is assumed, and if it is determined that no correction is necessary, the image deterioration correction processing is not performed. It has a processing branch function. The processing branch function at this time may be based on an automatic determination function of the imaging apparatus itself, or may be a processing branch via a user interface.

Here, in the aberration characteristic of the image pickup optical system,
When the off-axis chromatic aberration is relatively large, the image of the object obtained by imaging the image of the object projected through the imaging optical system that generates a large amount of off-axis chromatic aberration by the image sensor is located at a position where the image height is large. , The image of the subject in each color system is shifted.
In order to obtain data of pixels of a certain color system, such as luminance color separation and color conversion processing, using image data having different imaging positions for each color system, pixels of another color system are used. When image processing using the data of the above is performed, at the pixel position where the image height is large, the luminance and color difference when the luminance and color separation are performed, and the pixel data displayed in the new color system when the color system conversion is performed Makes no sense.

For this reason, when the off-axis chromatic aberration of the image pickup optical system is relatively large, the data processing means performs luminance color separation,
Before performing processing such as color system conversion processing, it is necessary to perform closed off-axis chromatic aberration correction for each color system. Further, it is preferable that the distortion that can be corrected by the same method as the off-axis chromatic aberration is also corrected at the same time. It is also preferable to perform other aberration corrections as needed.

According to the seventh aspect of the present invention, there is provided a specification for inputting necessary aberration correction data of an image pickup optical system to a built-in recording means via a recording means detachable from the image pickup apparatus, and a user interface of the image pickup apparatus. According to the specification for directly inputting data into the built-in recording means via the data input interface of the imaging device, the specification for inputting data to the built-in recording means via the data input interface of the imaging apparatus enables acquisition of data for aberration correction.
The data for aberration correction may be any data that is effective for aberration correction, such as lens design values, lens aberration data, and lens aberration correction parameters.

[0113] In the image pickup apparatus according to the eighth aspect, the picked-up image is displayed by the display means. The captured image to be displayed can be switched between an image subjected to image degradation correction by the imaging optical system and an image not subjected to image degradation correction in accordance with a desired frame display time interval.

[0114] In the image pickup apparatus according to the ninth aspect, the picked-up image is displayed by the display means. The captured image to be displayed can be switched between an image subjected to image degradation correction by the imaging optical system and an image not subjected to image degradation correction in accordance with an external selection.

That is, according to the image pickup apparatus of the present invention, even when a relatively inexpensive image pickup optical system having a low optical performance is used, image deterioration due to the image pickup optical system is corrected by introducing the present invention. In addition, it is possible to provide an inexpensive imaging device that can obtain practically sufficient good image quality.

Further, according to the imaging apparatus of the present invention, even when the imaging optical system of the imaging apparatus can be zoomed or exchangeable, the zoom state detecting mechanism of the imaging optical system and the exchangeable imaging optical system can be used. System (interchangeable lens) type identification mechanism,
Using the data for correcting the aberration of the imaging optical system stored in the recording unit of the imaging apparatus in advance, it is possible to correct the image deterioration due to the imaging optical system.

According to the imaging apparatus of the present invention, the data for correcting the aberration of the imaging optical system is transmitted to the imaging apparatus via recording means detachable from the imaging apparatus or a data interface mounted on the imaging apparatus. It is possible to record on the built-in recording means.

Further, according to the image pickup apparatus of the present invention, when the display means of the image pickup apparatus is used as a finder, the image deterioration correction is performed according to the processing speed of the image deterioration correction processing or selection through a user interface. The processed image or the image that has not been subjected to the image deterioration processing can be displayed on the display unit.

Further, in the image deterioration correction processing method of the present invention and the storage medium of the present invention, similarly to the above, even if a relatively inexpensive imaging optical system having low optical performance is used, the image by the imaging optical system can be used. Various effects such as being able to provide an inexpensive imaging device capable of obtaining a practically satisfactory image quality in which deterioration has been corrected are obtained.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration example of a digital still camera according to first and second embodiments of the present invention.

FIG. 2 is a block diagram illustrating a configuration example of an image degradation correction processing unit of a digital still camera according to first, second, and third embodiments of the present invention.

FIG. 3 is an explanatory diagram schematically showing a format of image data converted by an image data conversion unit of the digital still camera according to the first and third embodiments of the present invention.

FIG. 4 is a schematic diagram of aberration characteristic data used in coordinate transformation processing of each of the image degradation correction processing units of the color systems R, G, and B of the digital still camera according to the first and second embodiments of the present invention. FIG.

FIG. 5 is an aberration diagram showing a relation between distortion and image height of off-axis chromatic aberration in the digital still camera according to the first embodiment of the present invention.

FIG. 6 is an explanatory diagram schematically showing two-dimensional coordinate data of each of the color systems R, G, and B subjected to coordinate conversion in the digital still camera according to the first embodiment of the present invention.

FIG. 7 is a diagram showing a digital still camera according to the first and third embodiments of the present invention, in which coordinates are transformed for aberration correction.
FIG. 3 is an explanatory diagram schematically showing coordinates of each of the RGB color systems and an outline of a region to be extracted as an image.

FIG. 8 is an explanatory diagram showing an example of an array of primary color filters on a light receiving surface of a single-chip image sensor in the digital still camera according to the first embodiment of the present invention.

FIG. 9 is an explanatory diagram schematically showing coordinates of each color system in which data can be obtained by the color filter of FIG. 7 in the digital still camera according to the first embodiment of the present invention.

FIG. 10 is a block diagram illustrating a configuration example in a case where data interpolation processing of each color system is added to the image deterioration correction processing unit of the digital still camera according to the first embodiment of the present invention.

FIG. 11 is a block diagram illustrating a configuration example of a digital still camera according to a third embodiment of the present invention.

FIG. 12 is an explanatory diagram showing an example of an arrangement of complementary color filters on a light receiving surface of a single-chip image sensor in a digital still camera according to a third embodiment of the present invention.

FIG. 13 is an explanatory diagram schematically showing coordinates of each color system in which data can be obtained by the color filter of FIG. 12 in the digital still camera according to the third embodiment of the present invention.

FIG. 14 is an explanatory diagram showing a configuration example of storage contents of a storage medium storing a program for executing the image deterioration correction processing method of the present invention and related data.

FIG. 15 is an explanatory diagram showing a conceptual example in which a program for executing the image deterioration correction processing method of the present invention and related data are supplied from a storage medium to a device such as a computer.

[Explanation of symbols]

Reference Signs List 11 imaging optical system 12 imaging element 20 data processing unit (data processing unit) 21 AGC, A / D conversion unit (data processing unit) 22 image data conversion unit (data processing unit) 23 image deterioration correction processing unit (image deterioration correction processing) Means) 30 output image memory (recording means) 31 data interface section (interface) 32 image display section (display means) 34 CPU, built-in memory (image deterioration correction processing determination means, display control means)

Continued on the front page F-term (reference) 2H054 AA01 5C022 AA13 AB51 AC01 AC42 AC52 AC54 5C065 AA03 BB48 CC01 CC08 CC09 DD01 DD17 EE05 EE06 EE07 EE12 EE13 FF03 GG13 GG18 GG30 GG32 5C066 AA01 GA01 EC01 KE09 KE19 KM01 KM06 KM11 LA02

Claims (30)

[Claims] 1. A data processing means for converting an output signal of an image sensor for photoelectrically converting a subject image by an imaging optical system into digital image data, and an image data being processed by the data processing means or a process of the data processing means. An image capturing apparatus having a recording unit capable of recording digital image data that has passed through, performing luminance / color separation processing and color conversion processing of image data based on characteristic data of the imaging optical system and an output signal of the imaging element. An image pickup apparatus, comprising: an image deterioration correction processing unit that performs processing for correcting image quality deterioration due to the image pickup optical system on the image data. 2. The imaging apparatus according to claim 1, wherein the recording unit is a recording unit built in the imaging device main body or a recording unit detachable from the imaging device main body. 3. An image deterioration correction processing determining means for determining whether to execute the image deterioration correction processing by the image deterioration correction processing means according to a characteristic value of characteristic data of the imaging optical system. The imaging device according to claim 1, wherein: 4. The image quality deterioration corrected by the image deterioration correction processing means includes at least one of off-axis chromatic aberration and distortion of the imaging optical system. An imaging device according to any one of the preceding claims. 5. The imaging optical system is zoomable and has a detection mechanism capable of detecting a zoom state of the imaging optical system, and the detection mechanism acquires characteristic data in the zoom state of the imaging optical system. The imaging device according to claim 1, wherein the imaging device is capable of performing the following. 6. The imaging optical system is interchangeable and has an identification mechanism capable of identifying the type of the imaging optical system, and the identification mechanism can acquire characteristic data in the imaging optical system. The imaging device according to any one of claims 1, 3, 4, and 5, wherein: 7. The characteristic data of the photographing optical system is stored in a recording unit built in the main body of the imaging device, or can be stored in a recording unit detachable from the main body of the imaging device.
7. The image pickup apparatus according to claim 1, wherein the image pickup apparatus can be stored in a recording unit built in the image pickup apparatus main body from an external device via an interface. 8. A display means for displaying a captured image, and the digital image data which has not been subjected to image deterioration correction processing or has been subjected to image deterioration correction processing by the image deterioration correction processing means according to a frame display time interval of the captured image. The imaging apparatus according to claim 1, further comprising: a display control unit configured to cause the display unit to display a captured image based on any one of the above. 9. A display means for displaying a photographed image, and one of the digital image data which has not been subjected to image deterioration correction processing or has been subjected to image deterioration correction processing by the image deterioration correction processing means in accordance with an external selection. The image pickup apparatus according to claim 1, further comprising: a display control unit configured to display a captured image based on the display on the display unit. 10. Data used in the image deterioration correction processing by the image deterioration correction processing means is data effective in correcting the off-axis chromatic aberration and distortion, such as lens design values, lens aberration data, and lens aberration correction parameters. The imaging device according to any one of claims 1, 3, 4, 8, and 9, wherein 11. A data processing means for converting an output signal of an image pickup device for photoelectrically converting a subject image by an image pickup optical system into digital image data, and an image data being processed by the data processing means or a process of the data processing means. An image deterioration correction processing method applied to an image pickup apparatus having a recording unit capable of recording digital image data that has passed through, wherein a luminance / luminance of image data is calculated based on characteristic data of the imaging optical system and an output signal of the image pickup device. An image deterioration correction method, comprising: performing a process of correcting image quality deterioration by the imaging optical system on the image data before performing a color separation process and a color conversion process. 12. The image deterioration correction processing method according to claim 11, wherein the recording unit is a recording unit built in the imaging device or a recording unit detachable from the imaging device. 13. A method according to claim 11, wherein whether or not to execute said image deterioration correction processing by said image deterioration correction processing means is determined according to a characteristic value of characteristic data of said imaging optical system. The image deterioration correction processing method described in the above. 14. The image quality deterioration that is subjected to the image deterioration correction processing includes at least one of off-axis chromatic aberration and distortion of the imaging optical system.
14. The image deterioration correction processing method according to 1 or 13. 15. The imaging optical system can be zoomed, and characteristic data in the zoom state of the imaging optical system can be acquired by a detection mechanism capable of detecting the zoom state of the imaging optical system. The image degradation correction processing method according to claim 11, 13 or 14, wherein: 16. The imaging optical system is interchangeable, and characteristic data of the imaging optical system can be acquired by an identification mechanism capable of identifying the type of the imaging optical system. Terms 11, 13, 14, 1
5. The image deterioration correction processing method according to any one of 5. 17. The characteristic data of the photographing optical system is stored in a recording unit built in the imaging device, can be stored in a recording unit detachable from the imaging device, or an interface can be externally provided. 17. The image deterioration correction processing method according to claim 11, wherein the image deterioration correction processing method can be stored in a recording unit built in the imaging apparatus. 18. A captured image based on any of the digital image data that has not been subjected to the image deterioration correction processing or has been subjected to the image deterioration correction processing according to a frame display time interval of a captured image. Claim 1
18. The image deterioration correction processing method according to any one of 1 to 17. 19. A captured image based on any of the digital image data that has not been subjected to the image deterioration correction processing or has been subjected to the image deterioration correction processing according to an external selection. An image deterioration correction processing method according to any one of items 11 to 17. 20. Data used in the image deterioration correction processing is data effective in correcting the off-axis chromatic aberration and the distortion, such as lens design values, lens aberration data, and lens aberration correction parameters. Claims 11 and 1
The image deterioration correction method according to any one of 3, 14, 18, and 19. 21. A data processing means for converting an output signal of an image sensor for photoelectrically converting a subject image by an imaging optical system into digital image data, and an image data being processed by the data processing means or a process of the data processing means. A computer-readable storage medium storing a program for executing an image deterioration correction processing method applied to an imaging apparatus having a recording unit capable of recording digital image data that has passed through, wherein the image deterioration correction processing method includes: Processing for correcting image quality deterioration of the image data by the imaging optical system before performing luminance / color separation processing and color conversion processing of the image data based on the characteristic data of the imaging optical system and the output signal of the imaging element A storage medium having an image deterioration correction processing step of performing the following. 22. The storage medium according to claim 21, wherein the recording unit is a recording unit built in the imaging device or a recording unit detachable from the imaging device. 23. An image deterioration correction processing determining step of determining whether or not to execute the image deterioration correction processing by the image deterioration correction processing means according to a characteristic value of characteristic data of the imaging optical system. 23. The storage medium according to claim 21, wherein: 24. The image quality deterioration corrected in the image deterioration correction processing step includes at least one of off-axis chromatic aberration and distortion of the imaging optical system. The storage medium according to the above. 25. The imaging optical system is capable of zooming and has a detection step capable of detecting a zoom state of the imaging optical system, and the detection step acquires characteristic data in the zoom state of the imaging optical system. 25. A method according to claim 21, wherein
The storage medium according to any one of the above. 26. The imaging optical system is interchangeable and has an identification step of identifying a type of the imaging optical system, and the identification step can acquire characteristic data of the imaging optical system. The storage medium according to any one of claims 21, 23, 24, and 25. 27. The characteristic data of the photographing optical system is stored in a recording unit built in the imaging device, or can be stored in a recording unit detachable from the imaging device, or an interface can be externally provided. The storage medium according to any one of claims 21 to 26, wherein the storage medium can be stored in a recording unit built in the imaging device via the storage device. 28. A captured image based on any of the digital image data that has not been subjected to the image deterioration correction processing or has been subjected to the image deterioration correction processing in the image deterioration correction processing step, according to a frame display time interval of the captured image. The storage medium according to any one of claims 21 to 27, further comprising a display control step of causing the means to display. 29. A photographic image based on one of the digital image data, which has not been subjected to the image deterioration correction processing or has been subjected to the image deterioration correction processing in the image deterioration correction processing step, is displayed on a display means in accordance with an external selection. 3. A display control step of causing the display device to perform a display control step.
8. The storage medium according to any one of 7. 30. Data used for the image deterioration correction processing in the image deterioration correction processing step is data effective for correcting the off-axis chromatic aberration and the distortion, such as lens design values, lens aberration data, and lens aberration correction parameters. Claims 21, 23, 24, 28, characterized in that
30. The storage medium according to any one of 29.