JP2010141661A - Image processing device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing device for obtaining a higher-quality image by digital zoom. <P>SOLUTION: The image processing device is used for processing an imaged subject image. The image processing device has: an image trimming means S5 for trimming a portion of the subject image according to a digital zoom scale factor; a scaling means S6 for scaling the image, trimmed by the image trimming means S5, according to the arbitrarily designated size; and a resolution restoring means S8 for executing restoration processing to restore the image resolution. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image processing apparatus for processing a captured subject image.

  As one of the zoom functions of a digital camera, a digital zoom that trims and enlarges a part of a captured subject image is known (see, for example, Patent Document 1). Conventionally, a decrease in the number of pixels by trimming a part of an image causes a decrease in S / N ratio and a decrease in resolution, which causes a significant deterioration in image quality. Another problem is that the optical transfer function (PSF) of the lens differs depending on the size to be trimmed, which may cause variations in resolution.

Japanese Patent Laid-Open No. 2003-283811

  An object of the present invention is to provide an image processing apparatus for obtaining a high-quality image by digital zoom.

  According to one aspect of the present invention, there is provided an image processing apparatus for processing a captured subject image, the image trimming means for trimming a part of the subject image in accordance with the magnification of the digital zoom, and the trimming by the image trimming means. There is provided an image processing apparatus characterized by comprising scaling means for scaling an image that has been arbitrarily specified according to a specified size, and resolution restoration means for performing restoration processing for restoring the resolution of the image.

  According to the present invention, there is an effect that a high-quality image can be obtained by digital zoom.

  Exemplary embodiments of an image processing apparatus according to the present invention will be explained below in detail with reference to the accompanying drawings.

(First embodiment)
FIG. 1 is a block diagram showing the configuration of the image processing apparatus according to the first embodiment of the present invention. The imaging lens 1 captures light from the subject. The IR cut filter 2 removes infrared light from the light captured by the imaging lens 1. The image sensor unit 3 converts light captured by the plurality of light receiving elements into signal charges by photoelectric conversion. The image sensor unit 3 takes in the red, blue, and green pixel values in the order corresponding to the Bayer array, and amplifies the signal with a gain according to the imaging condition. Further, the image sensor unit 3 converts the obtained signal from an analog system to a digital system, and performs various image processing. The recording unit 4 records a signal subjected to image processing in the image sensor unit 3 in a memory or a recording medium. Thus, the image processing apparatus processes and records the captured subject image.

  FIG. 2 is a flowchart for explaining the procedure of image processing by the image sensor unit 3. The present embodiment is characterized in that the resolution restoration processing shown in step S8 is performed after the scaling shown in step S6. In step S1, processing for suppressing low frequency noise is performed. This suppresses occurrence of horizontal stripes during line reading due to the influence of noise. In step S2, white balance (WB) is set in advance using standard light source light. In step S3, luminance unevenness due to a light amount difference between the central portion and the peripheral portion due to the optical system of the imaging lens 1 is corrected (shading correction). In step S4, the missing portion (scratch) of the signal due to the pixel not functioning normally in the image sensor unit 3 is automatically corrected.

  In step S5, the image trimming means trims a part of the subject image according to the digital zoom magnification. In step S6, the image obtained by the trimming in step S5 is scaled by the scaling means. Scaling is performed according to an arbitrarily designated size, for example, downscaling that reduces the size of an image obtained by trimming. At this time, the S / N ratio is expected to be improved by interpolation with adjacent pixels. As a downscaling method, for example, a bicubic method is employed in order to suppress a decrease in resolution, generation of moiré, and the like.

  In step S7, the noise of the image scaled in step S6 is removed by a noise removing unit (noise reduction). In consideration of the fact that human visual characteristics have higher contrast ratio sensitivity at lower frequencies, noise reduction may reduce noise at lower frequencies. Note that when the light receiving sensitivity of the image sensor unit 3 is high and sufficient S / N can be obtained, noise reduction may be omitted.

  In step S8, the image from which noise has been removed in step S7 is subjected to processing for restoring the resolution by the resolution restoring means. To restore the resolution, the PSF of the imaging lens 1 is used. The PSF is written in advance in a resist or RAM, for example, and is read out during resolution restoration processing. The effect of resolution restoration depends on the algorithm used for restoration. Here, for example, the Richardson-Lucy method is used to restore an image close to the original subject image.

  In step S9, a color image is synthesized from the RAW image obtained by the processing up to step S8 (demosaicing). In step S10, WB is automatically adjusted for the image demosaicing in step S9. In step S11, color matrix calculation processing (color reproducibility processing) for obtaining color reproducibility is executed for the image whose WB has been adjusted in step S10. In step S12, the saturation and brightness of the image are corrected by gamma correction. The image sensor unit 3 outputs a signal subjected to the above image processing. Note that the procedure described in the present embodiment for the image processing by the image sensor unit 3 is merely an example, and other processes may be added or changes such as omitting an optional process may be appropriately performed.

  Thus, by performing trimming, scaling, noise reduction, and resolution restoration in digital zoom, noise can be reduced, resolution can be improved, and image quality can be improved. In addition, by restoring the resolution using PSF, it is possible to reduce variations in resolution due to the trimming size. As described above, there is an effect that a high-quality image can be obtained by digital zoom.

  The present embodiment has an advantage that the resolution restoration processing can be performed at a high speed by downscaling before the restoration of the resolution. Also, the graininess can be reduced by smoothing the roughness of the image. Note that scaling is not limited to downscaling, and may be upscaling that enlarges the size of an image obtained by trimming. Also in this case, an effect of improving the image quality can be obtained.

  Considering that PSF is an index indicating the degree of image degradation caused by the optical system, the difference in refractive index between the central part and the peripheral part of the lens means that the image degradation method is clearly different. It has been known. In the present embodiment, a plurality of PSFs may be prepared at the central portion and the peripheral portion of the imaging lens 1. In the restoration process, the PSF corresponding to the position of the imaging lens 1 is read and used, so that the resolution restoration effect can be enhanced.

  The use state of the imaging lens 1 for the resolution restoration target, for example, the angle of view varies depending on the trimming size. For the resolution restoration, it is necessary to use a PSF corresponding to the trimming size. The image sensor unit 3 may hold a plurality of PSFs corresponding to the distance from the center position of the imaging lens 1 and read out PSF data corresponding to the trimming size. Furthermore, only the PSF for a specific size of trimming (for example, 1 time) is held, and PSF corresponding to an arbitrary size of trimming is estimated and used by interpolation processing using the held PSF. good.

(Second Embodiment)
FIG. 3 is a flowchart for explaining the procedure of image processing by the image processing apparatus according to the second embodiment of the present invention. The present embodiment is characterized in that the resolution restoration processing shown in step S22 is performed before the scaling shown in step S24. The procedure from step S1 to step S4 is the same as that in the first embodiment.

  In step S21, the noise of the subject image is removed by the noise removing means (noise reduction). Also in the present embodiment, the noise reduction may be to reduce low frequency noise. Note that when the light receiving sensitivity of the image sensor unit 3 is high and sufficient S / N can be obtained, noise reduction may be omitted. In step S22, processing for restoring the resolution by the resolution restoration means is performed on the image from which noise has been removed in step S21. Also in the present embodiment, the PSF of the imaging lens 1 is used to restore the resolution. Also, for example, the Richardson-Lucy method is used to restore an image close to the original subject image.

  In step S23, a part of the image whose resolution is restored in step S22 is trimmed by the image trimming unit according to the digital zoom magnification. In step S24, the image obtained by the trimming in step S23 is scaled by the scaling means. Scaling is performed according to an arbitrarily designated size, for example, downscaling. Also in the present embodiment, for example, a bicubic method is employed as a downscaling method. The procedure from step S9 to step S12 is the same as that in the first embodiment.

  Also in this embodiment, as in the first embodiment, it is possible to obtain a high-quality image by digital zoom. In this embodiment, downscaling is performed after the resolution is restored, so that there is an advantage that the accuracy of the restoration process can be improved and the image can be further improved in quality. Further, it is sufficient to prepare one PSF for the imaging lens 1, and it is possible to eliminate the need for PSF holding and interpolation processing for each trimming size. Note that scaling is not limited to downscaling, and may be upscaling that enlarges the size of an image obtained by trimming. Also in this case, an effect of improving the image quality can be obtained.

1 is a block diagram showing a configuration of an image processing apparatus according to a first embodiment. The flowchart explaining the procedure of the image process by an image sensor part. 9 is a flowchart for explaining a procedure of image processing by the image processing apparatus according to the second embodiment.

Explanation of symbols

  1 imaging lens, 3 image sensor unit.

Claims (5)

An image processing apparatus for processing a captured subject image,
Image trimming means for trimming a part of the subject image according to a digital zoom magnification;
Scaling means for scaling the image trimmed by the image trimming means according to an arbitrarily designated size;
An image processing apparatus comprising: a resolution restoring unit that performs a restoration process for restoring an image resolution.   The image processing apparatus according to claim 1, wherein the resolution restoration unit performs the restoration process on the image scaled by the scaling unit.   The image processing apparatus according to claim 1, wherein the image trimming unit trims a part of the subject image subjected to the restoration processing by the resolution restoration unit.   The image processing apparatus according to claim 1, wherein the resolution restoration unit performs the restoration process using an optical transfer function of an imaging lens.   The image processing apparatus according to claim 1, further comprising a noise removing unit that removes image noise.

Images