JP2001197354A - Digital image pickup device and image restoring method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To appropriately restore the deterioration of an image due to the influence of an optical system in a digital camera. SOLUTION: In the digital camera, information on the optical system such as lens arrangement and a diaphragm value is obtained as deterioration information and photographing is conducted (steps S102 to S104). A deterioration function showing the deterioration characteristic of the image by the optical system is obtained based on deterioration information (step S105). The image obtained by using the deterioration function is restored (step S106). Thus, the deterioration of the image due to the influence of the optical system can be restored.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a technique for restoring deterioration of an image obtained using an array of light receiving elements.

[0002]

2. Description of the Related Art Conventionally, for an image obtained as image data using a light receiving element array represented by a CCD,
Various techniques for restoring image degradation have been proposed. Deterioration of an image means that an actually obtained image has deteriorated with respect to an ideal image to be obtained from an imaging target.For example, an image obtained by using a digital camera is
The image is degraded by aberration depending on the aperture value, the focal length, the focus position, and the like, and further degraded by an optical low-pass filter for preventing spurious resolution.

[0003] For such a deteriorated image, restoration has been conventionally performed to model an image obtained by modeling the deterioration of the image so as to approach an ideal image. For example,
A filter that applies a restoration function to an image or emphasizes an edge of an image (a so-called aperture correction filter), assuming that image degradation is caused by light beams to be incident on each light receiving element spreading while following a Gaussian distribution. Has been performed to restore images.

[0004]

However, the conventional image restoring method does not consider how the image is actually deteriorated. as a result,
It is often difficult to obtain an ideal image by restoration.

Accordingly, the present invention has been made in view of the above problems, and has as its object to appropriately restore an image by referring to the actual deterioration characteristics of the image.

[0006]

The present invention is a digital image pickup apparatus, comprising: an image pickup means for acquiring an image as image data by using a light receiving element array; and an image system for obtaining the image by an optical system of the image pickup means. Restoring means for restoring the image using at least one deterioration function indicating at least one kind of deterioration characteristic.

According to a second aspect of the present invention, there is provided a digital image pickup apparatus, comprising: an image pickup means for obtaining an image as image data by using a light receiving element array; and at least the image data by the optical system of the image pickup means together with the image data. Output means for outputting at least one deterioration function indicating one type of deterioration characteristic to the outside.

According to a third aspect of the present invention, there is provided an image restoring method for restoring image deterioration, wherein a step of preparing an image acquired as image data by using an image pickup means having a light receiving element array is provided. Restoring the image using at least one deterioration function indicating at least one type of deterioration characteristic of the image by the optical system.

[0009]

DESCRIPTION OF THE PREFERRED EMBODIMENTS <1. First Embodiment><1.1 Configuration of Digital Camera> FIGS. 1 to 3 are views showing the appearance of a digital camera 1 according to a first embodiment of the present invention. FIG. 1 is a front view and FIG. Is a rear view, and FIG. 3 is a left side view. 1 and 2 show a state in which the memory card 91 is mounted, and FIG. 3 does not show the memory card 91.

The main structure of the digital camera 1 is the same as that of an ordinary digital camera. As shown in FIG. 1, a lens unit 2 for guiding light from a subject to a CCD as shown in FIG. Flash 1 that emits
1 is disposed, and a finder 12 for capturing a subject is disposed above the lens unit 2.

A shutter button 13 to be pressed at the time of a shooting operation is arranged on the upper surface, and a card slot 14 for mounting a memory card 91 is provided on the left side surface as shown in FIG.

As shown in FIG. 2, on the back of the digital camera 1, a liquid crystal display 15 for displaying an image obtained by photography and an operation screen, a changeover switch 161 for switching between a photography mode and a reproduction mode, A four-way key 162 for receiving a selection input is arranged.

FIG. 4 is a longitudinal sectional view showing the internal structure of the digital camera 1 in the vicinity of the lens unit 2. The lens unit 2 includes a lens system 2 including a plurality of lenses.
An optical low-pass filter 31 and a single-plate color CCD 32 having a two-dimensional light receiving element array are sequentially arranged behind the lens unit 2. That is, in the digital camera 1, the lens system 21,
The diaphragm 22 and the optical low-pass filter 31 constitute an optical system for guiding light from a subject to the CCD 32.

FIG. 5 is a block diagram showing a main configuration relating to the operation of the digital camera 1. In FIG. 5, the shutter button 13, the changeover switch 161, and the four-way key 162 are illustrated as the operation unit 16.

A CPU 41, a ROM 42 and a RAM 43 shown in FIG. 5 are configured to control the entire operation of the digital camera 1, and the CPU 41, the ROM 42 and the RAM 4
Various configurations are connected to the bus line as well as 3. Then, the CPU 41 sets the RO 43 while using the RAM 43 as a work area.
By performing the arithmetic processing according to the program 421 in the M42, the operation of each part of the digital camera 1 and the image processing are performed.

The lens unit 2 includes a lens driving unit 211 for driving the lens system 21 and the stop 22 together with the lens system 21 and the stop 22.
An aperture driving unit 221 is provided, and the lens system 21 and the aperture 22 are appropriately controlled by the CPU 41 according to the output of the distance measurement sensor and the brightness of the subject.

The CCD 32 is connected to an A / D converter 33, and outputs an image of a subject formed through the lens system 21, the aperture 22, and the optical low-pass filter 31 to the A / D converter 33 as an image signal. I do. Image signal is A / D
After being converted into a digital signal (hereinafter referred to as “image data”) by the converter 33, the digital signal is stored in the image memory 34. That is, the image of the subject is obtained as image data by the optical system, the CCD 32 and the A / D converter 33.

The correction section 44 performs various image processing such as white balance correction, gamma correction, noise removal, color correction, and color enhancement on the image data in the image memory 34. The corrected image data is transferred to a VRAM (video RAM) 151, whereby an image is displayed on the display 15. Further, the image data is transferred to the memory card 91 via the card slot 14 as necessary by the operation of the operator.
Will be recorded.

In the digital camera 1, a process for restoring the deterioration of the acquired image data due to the influence of the optical system is performed.
This is realized by the PU 41 performing arithmetic processing according to the program 421 in the ROM 42. In the digital camera 1, image processing (correction and restoration) is substantially performed by processing image data, but in the following description, “image data” to be processed is simply referred to as “image”. That.

<1.2 Image Degradation by Optical System>
An image deterioration in the digital camera 1 will be described. The image deterioration means that the CCD 32 of the digital camera 1
An image acquired via the A / D converter 33 or the like does not become an ideal image. Such image deterioration occurs because light rays emitted from one point on the subject do not converge on the CCD 32 but have a spread distribution. In other words, when an ideal image is obtained, a light beam to be incident on one light receiving element (that is, a pixel) of the CCD 32 spreads and is incident on a surrounding light receiving element, so that image deterioration occurs.

In the digital camera 1, image deterioration is restored by an optical system mainly composed of a lens system 21, an aperture 22, and an optical low-pass filter 31.

FIG. 6 is a diagram for explaining image deterioration due to the lens unit 2. Reference numeral 71 in FIG. 6 indicates the entire image, and is an ideal image (that is, an image that is not degraded by the influence of the optical system, hereinafter, an “ideal image”).
That. Assuming that the area indicated by reference numeral 701 becomes brighter, in an image actually obtained (hereinafter, referred to as an “acquired image”), the focal length and focus position of the lens system 21 (lens extension amount in a zoom lens) And the area 70 according to the aperture value of the aperture 22.
The area 711 wider than 1 becomes brighter. That is,
Ideally, the luminous flux to be incident on the area on the CCD 32 corresponding to the area 701 spreads and is incident on the area corresponding to the area 711.

Also in the peripheral area of the image 71, assuming that the area indicated by the reference numeral 702 is bright in the case of the ideal image, the area which is substantially elliptical as indicated by the reference numeral 712 is bright in the acquired image. Become.

FIGS. 7 to 9 are schematic diagrams for explaining the deterioration of the image due to the optical influence of the lens unit 2 at the light receiving element level of the CCD 32. FIG. FIG. 7 shows a state in which a light flux of intensity 1 is incident only on the light receiving element at the center of the 3 × 3 light receiving element array in a state where there is no influence of the lens unit 2 (that is, a state where an ideal image is obtained). . 8 and 9 show how the state shown in FIG. 7 changes due to the influence of the lens unit 2.

FIG. 8 shows an example of a state in the vicinity of the center of the CCD 32. Light of intensity 1/3 is incident on the central light receiving element, and light of intensity 1/6 is incident on the upper, lower, left and right adjacent light receiving elements. It shows how to do. That is, the figure shows that the light beam to be incident on the central light receiving element spreads and enters the periphery due to the influence of the lens unit 2. FIG. 9 shows a CCD 32
The figure shows an example of the state of the peripheral portion of FIG. 5, in which light having an intensity of 1/4 is incident on the central light receiving element and light is incident from the upper left to the lower right while spreading.

A function of converting the pixel value of each pixel of the ideal image into a pixel value distribution illustrated in FIGS. 8 and 9 (ie, a two-dimensional filter based on a point image distribution) Therefore, it is called a deterioration function (or deterioration filter).

The deterioration function indicating the deterioration characteristic due to the influence of the lens unit 2 is calculated for each position of the light receiving element (that is, for each pixel position) based on the focal length of the lens system 21, the focus position, and the aperture value of the aperture 22. 2) can be obtained in advance. Therefore, in the digital camera 1, as described later, information on the lens arrangement and the aperture value are obtained from the lens unit 2, a deterioration function corresponding to the position of the pixel is obtained, and restoration of the acquired image is realized based on the deterioration function. ing.

In general, the deterioration function of the lens unit 2 is a non-linear function having parameters such as a focal length, a focus position, an aperture value, and two-dimensional coordinates on the CCD 32 (that is, a pixel in an image). Function. Also,
Although the colors of the image are not described in FIGS. 7 to 9 for the sake of convenience, in the case of a color image, a deterioration function corresponding to each of the RGB colors or a deterioration function obtained by summing the deterioration functions of the respective colors is obtained. However, in order to simplify the process, the chromatic aberration may be ignored, and the degradation functions corresponding to the respective RGB colors may be the same.

FIG. 10 is a schematic diagram for explaining the deterioration due to the influence of the optical low-pass filter 31 at the light receiving element level of the CCD 32. The optical low-pass filter 31 is for preventing false resolution by performing band limitation using a birefringent optical material.
As exemplified by 0, the light to be incident on the upper left light receiving element is first split up and down as indicated by an arrow 721, and further split right and left as indicated by an arrow 722.

In a single-chip color CCD, four adjacent CCDs are used.
Of the two light receiving elements, green (G) filters are formed on two diagonal light receiving elements, and red (R) and blue (B) filters are formed on the remaining two light receiving elements. Then, the RGB values of each pixel are obtained by interpolation processing with reference to information obtained from surrounding pixels. However, since a single-chip color CCD has G pixels twice as many as R and B pixels, an image in which the resolution of G is higher than the resolution of R or B can be obtained by directly using the data obtained from the CCD. As a result, a high-frequency component of the subject image that cannot be captured by the light receiving element on which the R and B filters are formed appears as a false resolution.

Therefore, an optical low-pass filter 31 having a characteristic as shown in FIG. 10 is provided on the front surface of the CCD 32. Due to the effect of the optical low-pass filter 31, the high-frequency component of the image obtained from the G light receiving element is degraded. Would.

FIG. 11 is a diagram illustrating the distribution of a light beam to be incident on the central light receiving element by the optical low-pass filter 31 having the characteristics shown in FIG. 10, that is, the characteristic of the deterioration function corresponding to the optical low-pass filter 31 is schematically shown. FIG. As shown in FIG. 11, the optical low-pass filter 31
Divides a light beam to be incident on the central light receiving element into 2 × 2 light receiving elements. Therefore, in the digital camera 1, as described later, a deterioration function corresponding to the optical low-pass filter 31 is prepared in advance, and restoration of an acquired image based on the deterioration function is realized.

In the restoration using the deterioration function for the optical low-pass filter 31, a luminance component is obtained from the RGB values after the interpolation processing, and the luminance component is restored. As another restoration method, the G component may be restored after the G component interpolation processing, and the R and B components may be interpolated using the restored G component.

In the above description, it has been described that the deterioration function is obtained for each pixel. However, as the deterioration function, a deterioration function of a plurality of pixels or a collection of deterioration functions of all pixels (that is, a plurality of deterioration functions) is obtained. (A transformation matrix corresponding to pixel degradation) may be obtained.

<1.3 Image Restoration> Next, three specific examples of restoration of an acquired image using a deterioration function will be described. In the digital camera 1, any image restoration method may be adopted.

FIG. 12 is a flowchart showing the flow of processing in the first image restoration method. The first image restoration method is a method of obtaining a restoration function from a deterioration function, and applying the restoration function to the acquired image to perform restoration.

Considering a deteriorated image in which a deterioration function is applied to each pixel of an ideal image, the deterioration function has a function of changing the pixel values of surrounding pixels based on the pixel value of each pixel. Is an image having a size larger than the ideal image. Here, assuming that the size of the ideal image and the size of the acquired image are the same, it is possible to regard the image obtained by deleting pixels around the deteriorated image as the acquired image. Therefore, when an attempt is made to obtain a restoration function that performs a reverse conversion to the deterioration function, the restoration function cannot be appropriately obtained because information outside the processing target region (that is, the outer periphery) is missing.

Therefore, in the first image restoration method, first,
A virtual pixel is provided outside the processing target area, and the pixel value of the virtual pixel is appropriately determined (step S11). For example, the pixel value of the pixel inside the boundary of the acquired image is determined as the pixel value of the pixel outside the boundary as it is. This allows
A vector Y which is an array of pixel values of the corrected acquired image
And a vector X, which is an array of pixel values of an ideal image, can be assumed to satisfy the relationship of Expression 1.

[0039]

(Equation 1)

Here, the matrix H is a deterioration function (hereinafter referred to as an "image deterioration function") that applies the deterioration function of each pixel to the entire ideal image in which all the pixels are combined.

Thereafter, an inverse matrix H ^-1 of the matrix H, which is an image deterioration function, is obtained as a restoration function for restoring an image (step S12), and a vector X is obtained from Expression 2.

[0042]

(Equation 2)

That is, an image is restored by applying a restoration function to the corrected acquired image (step S13).

FIG. 13 is a flowchart showing the flow of processing in the second image restoration method. Since the degradation function generally has a characteristic of attenuating a specific frequency component in an ideal image, the second image restoration method restores an image by restoring a specific frequency component in an acquired image.

First, the acquired image is divided into blocks having a predetermined number of pixels (step S21), and each block is subjected to a two-dimensional Fourier transform (ie, discrete cosine transform (DCT)), and each block is transformed into a frequency space. (Step S22).

Next, the frequency components attenuated based on the characteristics of the deterioration function are restored (step S23). Specifically, the Fourier-transformed block is divided by the Fourier-transformed degradation function. Thereafter, the block is subjected to an inverse Fourier transform (inverse DCT) (step S24),
A restored image is obtained by combining the restored blocks (step S25).

FIG. 14 is a flowchart showing the flow of processing in the third image restoration method. In the third image restoration method, an image before deterioration is assumed by assuming an image before deterioration (hereinafter, the assumed image is referred to as an “assumed image”) and updating the assumed image using an iterative method. It is.

First, an acquired image is used as a hypothetical image in an initial state (step S31). Next, a deterioration function (more precisely, a matrix H which is an image deterioration function) is applied to the hypothetical image (step S32), and a difference between the obtained image and the obtained image is obtained (step S33). Thus, the hypothetical image is updated (step S35).

Specifically, based on a vector Y as an acquired image and a vector X as a hypothetical image, W is defined as a weight matrix (or a unit matrix), and Obtained as a modified hypothetical image.

[0050]

(Equation 3)

Thereafter, the updating of the hypothetical image is repeated until the difference between the acquired image and the degraded hypothetical image falls within the allowable range (step S34), and the hypothetical image finally obtained is the restored image.

That is, the difference between the vector Y, which is the acquired image, and the vector HX, which is an image obtained by deteriorating the hypothetical image, is calculated as the sum of squares of the pixel values of the pixels (or the sum of the squares of the loads). By solving the linear equation Y = HX by an iterative method, a vector X that minimizes the difference is obtained. As a reference mentioning details of the third image restoration method, for example, "RESTORATION OF A SINGLE SUPER-R"
ESOLUTION IMAGE FROM SEVERAL BLURRED, NOISY AND UN
DER-SAMPLED MEASURED IMAGES "(M. Elad and A. Feuer, I
EEE Trans., On Image Processing, Vol.6 No.12 pp164
6-1658 Dec / 1997). Of course, various other techniques can be used for the details of the iterative method.

By using the third image restoration method,
Although more appropriate image restoration can be performed than the first and second image restoration methods, the image restoration method adopted in the digital camera 1 can be any of the first to third image restoration methods. Often, other methods may be used.

<1.4 Operation of Digital Camera> The configuration of the digital camera 1, the degradation function indicating the degradation of the acquired image, and the restoration of the image using the degradation function have been described above. The operation of the digital camera 1 for restoring the displayed image will be described.

FIG. 15 is a flowchart showing the operation flow of the digital camera 1 at the time of photographing, and FIG. 16 is a block diagram showing the functional configuration related to photographing of the digital camera 1. FIG.
6, the lens control unit 401, the aperture control unit 402,
The deterioration function calculation unit 403, the deterioration function storage unit 404, and the restoration unit 405 execute the arithmetic processing according to the program 421 in the ROM 42 by the CPU 41.
1, functions realized by the ROM 42, the RAM 43, and the like.

When the shutter button 13 is pressed, the CCD
The optical system of the digital camera 1 is controlled to form an image of a subject on the image 32 (step S101). That is, the lens control unit 401 provides a control signal to the lens driving unit 211, and thereby, the arrangement of a plurality of lenses constituting the lens system 21 is controlled. Further, the aperture control unit 402
A control signal is supplied from the camera to the aperture driving unit 221 and the aperture 2
2 is controlled.

On the other hand, from the lens control unit 401 and the aperture control unit 402, information on the lens arrangement and the aperture value are sent to the degradation function calculation unit 403 as degradation information 431 for obtaining the degradation function (step S102). Thereafter, exposure is performed (step S103), and the CCD 32
The image of the subject acquired by the above is stored in the image memory 34 as image data. Subsequent image processing is performed on the image data stored in the image memory 34.

The deterioration function calculator 403 uses the deterioration information 431 given from the lens controller 401 and the aperture controller 402 to determine the deterioration function of each pixel in consideration of the effects of the lens system 21 and the aperture 22 ( Step S1
04). The obtained deterioration function is stored in the deterioration function storage unit 404. Further, a deterioration function relating to the optical low-pass filter 31 is prepared in the deterioration function storage unit 404 in advance.

In step S104, the deterioration function may be once obtained individually for each configuration and characteristic of the lens unit 2, and thereafter, the deterioration function considering the entire optical system may be obtained. For example, a deterioration function for the lens system 21, a deterioration function for the stop 22, and
A deterioration function for the optical low-pass filter 31 may be separately prepared, and a deterioration function for the lens system 21 may be separately obtained as a deterioration function for the focal length and a deterioration function for the focus position.

Further, in order to simplify the calculation processing for obtaining the deterioration function for each pixel, a deterioration function of a representative pixel in the image is obtained, and the deterioration function of the other pixels is set as the deterioration function of the representative pixel. May be obtained by interpolation.

When the deterioration function is obtained, the restoration processing described above is performed on the acquired image by the restoration unit 405 (step S105). As a result, the deterioration of the acquired image due to the influence of the optical system is restored. That is, restoration of an image using the deterioration function related to the optical low-pass filter 31 is performed, and restoration of the image using the deterioration function related to the lens unit 2 is performed.

In the restoration of an image using the deterioration function relating to the optical low-pass filter 31, a luminance component and a color component are obtained from the RGB values of each pixel of the acquired image after the interpolation processing, and the luminance component is restored. The luminance and color components are returned to RGB values.

On the other hand, in the restoration of an image using the deterioration function relating to the lens unit 2, restoration is performed in consideration of chromatic aberration for each of the RGB values of each pixel of the acquired image.
Of course, restoration of the image deterioration by the optical low-pass filter 31 and the lens unit 2 is performed only for the luminance component,
The processing may be simplified.

The restoration by the optical low-pass filter 31 and the deterioration by the lens unit 2 may be performed simultaneously. That is, image restoration may be performed after obtaining the deterioration function of the entire optical system.

The restored image is subjected to various image processing such as white balance correction, gamma correction, noise removal, color correction, and color enhancement by the correction unit 44 (step S106).
The image data after the correction is stored in the image memory 34.
Further, the image data in the image memory 34 is appropriately stored in the memory card 91 via the card slot 14 (Step S107).

As described above, the digital camera 1 restores the image degradation due to the influence of the optical system by using the degradation function indicating the degradation characteristic of the optical system. Can be.

<2. Second Embodiment> FIG. 17 is a view showing a second embodiment of the present invention. In the first embodiment, image restoration is performed by the digital camera 1, but in the second embodiment, image restoration is performed by the computer 5. That is, data can be transferred between the digital camera 1 having no image restoration function and the computer 5 using the memory card 91 or the communication cable 92, and the image acquired by the digital camera 1 is restored by the computer 5. It is supposed to.

Note that the digital camera 1 according to the second embodiment has the first
In the following description, the same components as those in the first embodiment are denoted by the same reference numerals. The data output from the digital camera 1 may be output from any output means such as the card slot 14 and the output terminal.
The description will be made on the assumption that data is transferred from the digital camera 1 to the computer 5 via the digital camera 1.

A program for performing a restoration process via a recording medium 8 such as a magnetic disk, an optical disk, and a magneto-optical disk is installed in the computer 5 in advance, and the CPU in the computer 5 follows the program using the RAM as a work area. By performing the processing, the image restoration processing is executed in the computer 5.

FIG. 18 is a schematic diagram showing the structure of data recorded on the memory card 91. In the digital camera 1, an image is acquired as image data by the same method as that of the ordinary digital camera 1, but at the same time, a deterioration function indicating a deterioration characteristic given to the image by the optical system is obtained (or stored in advance). The image data 911
And the deterioration function 912 are combined to form the memory card 91
Is output to.

FIG. 19 is a flow chart showing the flow of operation when taking a picture with the digital camera 1 according to the second embodiment.
FIG. 20 is a flowchart showing a flow of the operation in the computer 5. FIG. 21 is a block diagram showing a functional configuration of the digital camera 1 and the computer 5 related to the restoration processing. In FIG. 21, the card slot 5 for reading data from the memory card 91 is used for the computer 5.
1, only the fixed disk 52, and a restoring unit 505, which is a function realized by a CPU, a RAM, and the like. Hereinafter, the operations of the digital camera 1 and the computer 5 according to the second embodiment will be described with reference to FIGS.

When photographing is performed by the digital camera 1, similarly to the first embodiment, control of the optical system is performed by the lens control unit 401 and the aperture control unit 402 (FIG. 19: step S111). , Optical system information is degradation information 4
31 (step S112). afterwards,
Exposure to the CCD 32 is performed (step S113),
The captured image is obtained as image data.

The deterioration function calculator 403 obtains a deterioration function based on the deterioration information 431 relating to the lens unit 2 (step S114), and transfers it to the deterioration function storage 404. Further, similarly to the first embodiment, the deterioration function for the optical low-pass filter 31 is stored in the deterioration function storage unit 404 in advance. On the other hand, the acquired image is subjected to correction processing by the correction unit 44, and the image memory 34
(Accurately, correction processing is performed on the image data in the image memory 34) (step S11).
5).

Thereafter, in the digital camera 1, as shown in FIG. 18, the image data corresponding to the corrected image and the deterioration function are stored in the memory card 91 via the card slot 14.
(Step S116).

When the image data and the deterioration function are stored in the memory card 91, the memory card 91 is inserted into the card slot 51 of the computer 5, and the computer 5 reads the image data and the deterioration function onto the fixed disk 52 and performs a restoration process. (Step S121 in FIG. 20). After that, the above-described restoration processing using the deterioration function is performed on the image indicated by the image data by the restoration unit 505 (step S122), and an image in which the deterioration due to the influence of the optical system of the digital camera 1 is restored is obtained. The data is stored in the disk 52 (step S123).

As described above, in the digital camera 1 according to the second embodiment, the deterioration function is output to the outside together with the image data.
A restoration process using the deterioration function is performed. As a result, the digital camera 1 does not need to perform a restoration process, and therefore, as compared with the first embodiment (particularly, when acquiring an image having a large number of pixels), the time from the start of shooting to storage of image data Can be shortened.

<3. Modifications> Although an example of restoring an image acquired using the digital camera 1 has been described as an embodiment of the present invention, the present invention is not limited to the above embodiment, and various modifications are possible. Is possible.

For example, in the above embodiment, the deterioration function of the lens unit 2 and the deterioration function of the optical low-pass filter 31 have been described as the deterioration functions of the optical system. However, other types of deterioration functions can be obtained. Or
(Prepared in advance). Also, 3C
As in the case where only the deterioration function related to the lens unit 2 or only the deterioration function related to the aperture 22 is used in the digital camera 1 of the CD, only a single kind of deterioration function is used and a specific kind of image deterioration caused by the optical system is used. Only the data may be restored.

Further, as described above, the deterioration function does not need to be obtained for all the pixels. Instead, the deterioration function for the representative pixel (that is, the light receiving element) is obtained using the LUT or the like, and the other functions are obtained. The pixel deterioration function may be obtained by interpolation. Further, when the deterioration function is constant for all the pixels as in the case of the deterioration function relating to the optical low-pass filter 31, it is sufficient to prepare one deterioration function in the ROM 42 in advance.

That is, by using at least one type of at least one type of deterioration function, it is possible to appropriately restore a specific type of deterioration due to the influence of the optical system.

In the above embodiment, the calculation of the deterioration function and the restoration of the image have been described as being performed by the CPU, ROM, and RAM in the digital camera 1 and the computer 5, but the lens controller 401 of the digital camera 1 Aperture control unit 402, deterioration function calculation unit 403, and restoration unit 40
5, and the restoring unit 505 of the computer 5 may be constructed by a dedicated electric circuit, or may be constructed only by a part of the dedicated electric circuit.

The program 421 for restoring an image by the digital camera 1 may be installed in the digital camera 1 via a recording medium such as the memory card 91 in advance.

Further, the present invention is not limited to the restoration of an image acquired by the digital camera 1, but
It is also used for restoring an image acquired by another imaging device that acquires an image using the light receiving element array, for example, an electron microscope or a film scanner. Of course, the light receiving element array is not limited to the two-dimensional array, but may be a one-dimensional array like a scanner.

[0084]

According to the first to third aspects of the present invention, it is possible to appropriately restore at least one type of deterioration that causes an image due to the influence of the optical system.

[Brief description of the drawings]

FIG. 1 is a front view of a digital camera according to a first embodiment.

FIG. 2 is a rear view of the digital camera.

FIG. 3 is a side view of the digital camera.

FIG. 4 is a vertical sectional view near a lens unit.

FIG. 5 is a block diagram illustrating a configuration of a digital camera.

FIG. 6 is a diagram for explaining image deterioration due to a lens unit.

FIG. 7 is a diagram for explaining image deterioration due to a lens unit.

FIG. 8 is a diagram for explaining image deterioration due to a lens unit.

FIG. 9 is a diagram for describing image deterioration due to a lens unit.

FIG. 10 is a diagram for explaining image degradation due to an optical low-pass filter.

FIG. 11 is a diagram for explaining image degradation due to an optical low-pass filter.

FIG. 12 is a flowchart showing the flow of processing in the first image restoration method.

FIG. 13 is a flowchart showing the flow of processing in the second image restoration method.

FIG. 14 is a flowchart showing the flow of processing in a third image restoration method.

FIG. 15 is a flowchart showing a flow of operation of the digital camera at the time of shooting.

FIG. 16 is a block diagram illustrating a functional configuration of a digital camera.

FIG. 17 is a diagram illustrating an overall configuration according to a second embodiment.

FIG. 18 is a schematic diagram showing a data structure in a memory card.

FIG. 19 is a flowchart showing a flow of operation of the digital camera at the time of shooting.

FIG. 20 is a flowchart showing the flow of the operation of the computer.

FIG. 21 is a block diagram illustrating a functional configuration of a digital camera and a computer.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 Digital camera 2 Lens unit 14 Card slot 21 Lens system 22 Aperture 31 Optical low-pass filter 32 CCD 33 A / D converter 41 CPU 42 ROM 43 RAM 405 Restoration unit S103, S104, S106, S121, S122
Steps

Continuation of the front page (72) Inventor Yusuke Nakano 2-3-1, Azuchi-cho, Chuo-ku, Osaka-shi, Osaka F-term in Osaka International Building Minolta Co., Ltd. 5C021 PA51 PA66 PA74 PA78 PA79 PA85 PA92 XA03 XA08 XA34 XA66 5C022 AA13 AB51 AC42 AC51 AC69 5C076 AA40 BA06

Claims (3)

[Claims] 1. A digital imaging apparatus, comprising: an imaging unit that obtains an image as image data using a light receiving element array; and at least one of at least one kind of degradation characteristics of the image caused by an optical system of the imaging unit. A digital imaging device comprising: a restoration unit configured to restore the image using a deterioration function. 2. A digital imaging apparatus, comprising: an imaging unit that acquires an image as image data using a light receiving element array; and at least one kind of deterioration characteristic of the image by an optical system of the imaging unit together with the image data. And an output unit for outputting at least one deterioration function indicating the following to an external device. 3. An image restoration method for restoring deterioration of an image, comprising: preparing an image acquired as image data by using an imaging unit having a light receiving element array; Restoring the image using at least one deterioration function showing at least one kind of deterioration characteristic of the image.