JP2013152583A - Image processing apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain a sharp and smooth enlarged digital still image by reducing a calculation time required for sharpening a digital image.SOLUTION: In a digital image device employing a Total Variation regularization method, an input signal of a digital image is separated into a skeleton component and a texture component, and the skeleton component is processed by a linear expansion filter, a Shock filter, and a Total Variation filter sequentially in this order to sharpen the image.

Description

    The present invention relates to an image processing apparatus that performs image processing of a digital image.

    Various methods for obtaining a high-definition digital image by restoring a high-frequency component that does not exist in the original digital signal component when increasing the number of images in the digital image and enlarging the image have been studied. These methods are called super-resolution techniques.

    According to Non-Patent Documents 1, 2, and 3, the super-resolution image enlargement method using the TV regularization method (Total Variation regularization method), which is one of the super-resolution techniques, sharpens the edge component of the enlarged image. Therefore, the TV regularization method is useful as one of methods for obtaining a high-definition enlarged image.

Japanese Patent Application No. 2010-056771

Takahiro Saito: “Super-resolution oversampling from a single image”, The Journal of the Institute of Image Media Sciences, Vol.62, No.2, pp.181-189, 2008 Yu Sakurai, Akihiro Yoshikawa, Shotaro Suzuki, Tomoro Goto, Satoshi Hirano: "Restoring Super-Resolution Images Combining Regularization with Total Variation and Case Learning Method", The Journal of the Institute of Image Media and Technology, Vol. 61, No. 9, pp. 1363-1366, 2007 Yasutaka Sakuta, Tomoro Goto, Satoshi Hirano, Yuu Sakurai, “Acceleration of Super-Resolution Expansion Using Total Variation Regularization”, FIT (Information Science and Technology Forum), I-023, 2011, September A. Chambolle, “An algorithm for total variation minimization and applications”, J. Mathematical Imaging and Vision, Vol.20, No.1, pp.89-97, 2004 S. J. Osher and L. I. Rudin: “Feature-oriented image enhancement using shock filters”, SIAM Journal on Numerical Analysis, Vol. 27, pp. 910-940, 1990.

    The methods according to Non-Patent Documents 1 and 2 have a great effect for sharpening the edge component of the enlarged image, but have a drawback that the calculation time for sharpening the edge component becomes very long. Compared with the methods according to Non-Patent Literatures 1 and 2, the method according to Non-Patent Literature 3 can shorten the calculation time for sharpening the edge component of the enlarged image, but is slightly inferior in image quality sharpness. In addition, there is a drawback that the calculation time is too long for application to moving images.

    The present invention has been made in view of the above problems, and it is an object of the present invention to shorten a calculation time for sharpening an edge component of an enlarged image and obtain a sharp image not only for a still image but also for a moving image.

    In order to solve the above problem, the present invention uses the Shock filter used in Non-Patent Document 5 instead of the HPF (High Pass Filter) of the TV regularization enlargement unit used in Non-Patent Document 3. A first feature is that the Shock filter is used in cascade connection with a TV regularization filter.

    Specifically, the present invention sharpens the edge component of the image with respect to the skeleton component obtained by the Total Variation regularization component separation means for obtaining the skeleton component of the input image and the Total Variation regularization component separation means. A shock filter is provided.

    Further, the present invention comprises Total Variation regularization component separation means for obtaining a skeleton component of an input image, and Total Variation regularization expansion means for enlarging the skeleton component obtained by the Total Variation regularization component separation means, The Total Variation regularization enlargement means includes an enlargement filter that enlarges an image with respect to the skeleton component obtained by the Total Variation regularization component separation means, and a Shock filter that sharpens an edge component of the image enlarged by the enlargement filter It is characterized by having.

    Further, the present invention comprises Total Variation regularization component separation means for obtaining a skeleton component of an input image, and Total Variation regularization expansion means for enlarging the skeleton component obtained by the Total Variation regularization component separation means, The Total Variation regularization enlargement means includes an enlargement filter that enlarges an image with respect to the skeleton component obtained by the Total Variation regularization component separation means, and a Shock filter that sharpens an edge component of the image enlarged by the enlargement filter And a Total Variation regularization filter that smoothes the image sharpened by the Shock filter.

    According to the present invention, the image is sharpened by the Shock filter. In addition, when the image is sharpened by the TV regularization filter after the image is sharpened by the Shock filter, the edge is compared with the image before being processed by the two filters. A sharp and smooth image can be obtained. In addition, unlike the method of Non-Patent Document 3, since it is not necessary to smooth the image many times with the TV regularization filter, the total calculation time required for the image processing can be shortened.

It is a figure which shows the super-resolution system based on this invention. It is a figure which shows the structure of the TV regularization expansion part of 1st Embodiment of this invention. It is a figure which shows isolation | separation of the input signal component by TV regularization component isolation | separation. It is a flowchart which shows the calculation algorithm of the TV regularization component separation part which concerns on this invention. It is a flowchart which shows the calculation algorithm of the Shock filter which concerns on this invention. It is a figure which shows the structure of the TV regularization expansion part of 2nd Embodiment of this invention.

(First embodiment)
Hereinafter, a first embodiment of the present invention will be described in detail with reference to FIGS.

    In the first embodiment of the present invention, as shown in FIG. 1, the input image f is converted into an edge component and a low frequency component (hereinafter, both components are collectively referred to as “skeleton component”), and a fine vibration component and a noise component (hereinafter referred to as “skeletal component”). TV regularization component separation unit 1, TV regularization enlargement unit 2, linear interpolation enlargement unit 3, enlarged skeleton component U and enlarged texture component V for separating both components into "texture components") It is comprised from the adder 4 for adding.

    As shown in FIG. 2, the TV regularization expansion unit 2 has a configuration in which a linear expansion filter 5, a Shock filter 7, and a TV regularization filter 8 are connected in cascade.

    A situation in which the input signal f is separated into the skeleton component u and the texture component v in the TV regularization component separation unit 1 is schematically shown in FIG.

    The separated skeleton component u, which is the first component, is sharpened and enlarged by the TV regularization enlargement unit 2. On the other hand, the separated texture component v, which is the second component, is enlarged by the linear interpolation enlargement unit 3 using a linear interpolation method such as the Bicubic method. The output from the TV regularization enlargement unit 2 has the edge component sharpened and enlarged, and the enlarged skeleton component U and the output from the linear interpolation enlargement unit 3, the texture component V enlarged by the linear interpolation method, are added by the adder 4. Are added to form an output image.

    The TV regularization component separation is obtained according to the equation (1) by an iterative operation called the Chamblle projection method used in Non-Patent Document 4.

(1)
In equation (1), p _{i, j} is a two-dimensional vector called a dual vector corresponding to the input pixel signal f _{i, j} . i, j represents the position of the pixel.

    Each symbol in Formula (1) is defined as Formula (2) and Formula (3), respectively.

  (2)

(3)
p _{i, j} is calculated from all the pixels i = 1 to K and j = 1 to L according to the equation (1), and then repeatedly calculated from n = 1 to N. K and L represent the number of pixels. The number of iterations N is usually about 10 to 30 times. The texture component v _{i, j} is obtained from the value of p _{i, j} after the end of the repetitive calculation according to the equation (4).

(4)
On the other hand, the skeleton component u is obtained according to the equation (5).

(5)
FIG. 4 shows a flowchart for performing the calculations of the above formulas (1) to (4).

Next, calculation processing in the Shock filter 7 will be described. The pixel value to be repeatedly calculated

It expresses.

Is obtained by the iterative calculation of equation (6).

(6)
Each element of the 2nd term of a formula (6) is defined as a formula (7) and a formula (8), respectively.

       (7)

(8)
Each element in Formula (7) and Formula (8) is defined as Formula (9), Formula (10), Formula (11), and Formula (12), respectively.

                     (9)

                     (10)

         (11)

(12)
The function m in the equations (9) and (10) is defined as the equation (13).

(13)
FIG. 5 shows a flowchart for performing the calculations of the above formulas (6) to (13).

Image after repeating Shock Filter 7 operation several times

The edge of the original image is emphasized, but at the same time, a jagged shape is formed at the edge of the image in a planar direction and a luminance change due to overshoot and undershoot appears in the vicinity of the edge. The jagged shape and luminance change are smoothed by the TV regularization filter 8 at the next stage.

The shock filter 7 repeats the calculation about 5 times, the TV regularization filter 8 performs the calculation about 20 times, and then the TV regularization filter performs the calculation once.
(Second Embodiment)
As shown in FIG. 6, the TV regularization filter 8 may be omitted from the TV regularization expansion unit 2 of FIG. 1, and only the linear expansion filter 5 and the Shock filter 7 may be configured.
(Third embodiment)
In the first and second embodiments, the image enlargement is shown. However, the image enlargement may not be performed, that is, the linear enlargement filter 5 may be omitted. The above configuration increases the sharpness of the image without enlarging the image, and can be used as a means for improving the sharpness without changing the number of pixels of the current television signal example.
(Other embodiments)
The first, second, and third embodiments are not hardware but may be configured only by software, or may be configured by a combination of hardware and software. When configured by software, the configuration shown in FIGS. 1, 2, 4, 5, 6, etc. is realized as an image processing program for realizing each function.

Industrial applicability

    The method of the present invention can be used to restore high-definition images when enlarging and displaying television images, digital camera images, medical images, surveillance camera images, satellite transmission images, and the like.

1 TV regularization component separation unit (TV regularization component separation means)
2 TV regularization expansion part (TV regularization expansion means)
3 Linear interpolation enlargement section
4 Adder
5 Linear magnification filter (magnification filter)
6 TV regularization filter
7 Shock filter

Claims (3)

Total Variation regularization component separation means for obtaining the skeleton component of the input image, and a Shock filter for sharpening the edge component of the input image with respect to the skeleton component obtained by the Total Variation regularization component separation means A featured image processing apparatus. Total Variation regularization component separation means for obtaining a skeleton component of an input image, and Total Variation regularization expansion means for expanding the skeleton component obtained by the Total Variation regularization component separation means, and the Total Variation regularization expansion means Has a magnifying filter for enlarging an image with respect to the skeleton component obtained by the Total Variation regularization component separating means, and a Shock filter for sharpening an edge component of the image magnified by the magnifying filter. An image processing apparatus. Total Variation regularization component separation means for obtaining a skeleton component of an input image, and Total Variation regularization expansion means for expanding the skeleton component obtained by the Total Variation regularization component separation means, and the Total Variation regularization expansion means The skeleton component obtained by the Total Variation regularization component separation means is an enlargement filter for enlarging an image, a Shock filter for sharpening an edge component of an image enlarged by the enlargement filter, and the Shock filter. An image processing apparatus comprising: a Total Variation regularization filter that smoothes a sharpened image.