JP2008500757A - Method and system for enhancing the sharpness of a video signal - Google Patents

Abstract

The present invention relates to a method and system for modifying an input image (101) to produce an output image (102). The above method
The step (103) of dividing the input image (101) into a coarse image (104) and a fine image (105);
-Enhancing (106) the sharpness of the coarse image (104) to produce an intermediate image (107) with enhanced sharpness;
Combining the intermediate image (107) with enhanced sharpness and the fine image (105) to produce an output image (102) (108); The present invention can be used to enhance the sharpness of an image.

Description

  The present invention relates to a method and system for enhancing the sharpness of a video signal.

  The invention can be used, for example, in the field of video processing.

  Improving image quality is an important issue for video products such as television receivers. When the sharpness of the input image that defines the video is improved, the image quality is particularly perceptually improved. There are known solutions for improving the sharpness of video signals.

  The first solution emphasizes the sharpness of the input image with a filter that is applied to all successive images. The input image may also contain noise components (for example, analog white noise, or digital artifacts resulting from block encoding and block decoding operations), so the noise component increases as the overall quality of the resulting output image degrades. To increase.

  The second solution not only enhances the sharpness of the input image by a filter applied to all successive images, but also changes the enhancement level according to the level of the noise component detected in the input image. When the noise component is at a low level, the sharpness of the resulting output image is significantly increased. In contrast, if the noise component is at a high level, the filter is turned off (or the filter level is lowered) to avoid an increase in the noise component, but in this case the sharpness of the resulting output image is Unfortunately, it is close to the sharpness of the input image. Therefore, this second method is limited in that it is not an efficient method when handling an input image having a lot of noise.

  An object of the present invention is to provide a method for enhancing the sharpness of a video signal and is resistant to noise.

For this purpose, the method according to the invention proposes a modification of the input image in order to generate an output image. The above method
-Dividing the input image into a coarse image and a fine image;
-Enhancing the sharpness of the coarse image to produce an intermediate image with enhanced sharpness;
-Combining the intermediate image with enhanced sharpness and the fine image to produce the output image;
have.
This method is based on the assumption that the input image is defined by the sum of the signal component and the noise component, and the assumption that the signal component has a low frequency spectrum and the noise component has a high frequency spectrum. ing. Therefore, the portion of the frequency spectrum that should be emphasized in the input image exists at a low frequency, and the portion of the frequency spectrum that should not be emphasized in the input image exists at a high frequency.

  The dividing step can generate two types of images (a coarse image having a low frequency spectrum and a fine image having a high frequency spectrum). This division step can selectively enhance a coarse image mainly including signal components. Emphasizing the sharpness of an image with a low frequency spectrum gives better results than enhancing the sharpness of an image with a high frequency spectrum.

  Regardless of the noise level of the input image, the noise component of the resulting output image is not enhanced, so this method is robust to noise. After combining, the resulting output image still has a noise component that is close to the noise component of the input image, but the sharpness of the signal component is enhanced, thus improving the overall image quality.

The dividing step is as follows:
-Low-pass filtering the input image to generate the coarse image;
-Subtracting the coarse image from the input image to generate the fine image;
It is preferable to have.

  Using this subtracting step makes it possible to optimally divide the input image without losing data information.

  It is preferable that the low-pass filter processing step can be applied to the first noise signal obtained from the input image.

  The boundary between the low frequency spectrum corresponding to the signal component and the high frequency spectrum corresponding to the noise component is precisely defined with adaptability to the noise signal. Thereby, irrespective of the noise level of the input image, the sharpness of the entire spectrum of the signal component can be enhanced, and an output image with increased sharpness can be obtained.

  Preferably, the enhancing step can be adapted to a second noise signal obtained from the input image.

  Thus, for example, when the noise level is high, the sharpness enhancement is increased, and when the noise level is low, the sharpness enhancement is decreased, thereby adapting to the noise level of the input image when enhancing the sharpness. be able to.

  The low-pass filter processing step and the emphasizing step are preferably adaptable to a noise signal obtained from the input image.

  This makes it possible to enhance the sharpness of the entire frequency spectrum of the signal component regardless of the noise level of the input image, and also enhance the sharpness according to the noise level of the input image. Thus, an output image with improved sharpness can be obtained.

  It is also an object of the present invention to provide a system for enhancing the sharpness of a video signal, which is resistant to noise and has processing means for performing the various steps of the method according to the invention.

  Detailed descriptions and other aspects of the invention are described below.

  Certain aspects of the present invention have been described with reference to the embodiments described below, and are illustrated using the accompanying drawings. In the drawings, the same parts or the same sub-steps are denoted by the same reference numerals.

  FIG. 1 represents a method according to the invention for modifying an input image 101 to generate an output image 102.

  This method includes a step 103 for dividing the input image 101 into a coarse image 104 and a fine image 105. The frequency spectrum of the coarse image 104 corresponds to the low frequency spectrum of the input image 101, and the frequency spectrum of the fine image 105 corresponds to the high frequency spectrum of the input image 101.

  The method also includes a step 106 of enhancing the sharpness of the coarse image 104 to produce an intermediate image 107 with enhanced sharpness.

  The method also includes a step 108 of combining the intermediate image 107 with enhanced sharpness and the fine image 105 to produce the output image 102. This step 108 makes it possible to restore the output image.

  FIG. 2 represents a detailed arrangement based on FIG. 1 of the method according to the invention for modifying the input image 201 to generate the output image 202. The dividing step 203 has a step F of low-pass filtering the input image 201 (by performing a convolution operation) in order to generate a coarse image 204.

In the first example, the step F for performing the low-pass filter process realizes a Gaussian filter applied to the pixels constituting the coarse pixel 204. This linear filter can be defined by the following kernel k1 or k2.

Or step F which performs a low-pass filter process may implement | achieve the following filters.
-Nonlinear FIR filters (for example with kernels that depend on pixel positions in the image, or kernels applied to the edges of a block of pixels)-Order statistics (such as median filters, rank order filters, or morphological filters) filter. This filter sorts (or ranks) a large number of pixel values, and selects the sorted pixel values based on the rank.

  The step 103 for dividing also includes a step SUB for subtracting the coarse image 204 from the input image 201 in order to generate a fine image 205. This subtraction is performed between the pixels of the input image 201 and the pixels of the coarse image 204. At this time, the coordinates of the pixels of the input image 201 and the pixels of the coarse image 204 are the same. This subtraction is repeated for all pixels in both images.

  The step 206 of enhancing the sharpness of the coarse image 204 can be a non-linear process (described below) performed on the pixels of the lines of the coarse image 204 and / or the pixels of the columns of the coarse image 204. Sharpness can be thought of as a level transition between two unchanging or slowly changing data regions. For example, by applying a gradient filter to the coarse pixel 204 and detecting a region having the highest level in the filtered image, a level transition to be enhanced is detected.

  The step of combining includes the step 208 of adding the intermediate image 207 with enhanced sharpness to the fine image 205 in order to generate the output image 202.

  FIG. 3 shows, as an example, a transition signal T1 of level transition in the direction along the line of the coarse image 204. This transition signal has a first flat range at level S1 and a second flat range at level S2. In order to enhance the transition signal T1, a series of consecutive processing operations can be performed.

ここで、

は、畳込み演算を表す。 In order to generate the first intermediate signal y1 having an overshoot at the start of the transition or at the end of the transition, the first processing operation includes the transition signal T1 and the kernel of the differential filter DF as shown in the following equation: Convolutional operations are included.

here,

Represents a convolution operation.

  The DF is, for example, a filter kernel of the form [−0.25 0.5 −0.25].

ここで、αは、例えば１に等しい係数である。 The second processing operation is an operation for adding the transition signal T1 to a product obtained by multiplying the first intermediate signal y1 by a certain number, as shown in the following expression, in order to generate the second intermediate signal y2. Is included.

Here, α is a coefficient equal to 1, for example.

ここで、ｍｅｄ（Ｓ１，Ｓ２，ｙ２）は、信号Ｓ１、Ｓ２、およびｙ２のデータサンプルの間のメジアン演算を表している。 The third processing operation includes processing for suppressing overshoot remaining in the second intermediate signal y2, as shown in the following expression, in order to generate the enhanced transition signal T2.

Here, med (S1, S2, y2) represents the median operation between the data samples of the signals S1, S2, and y2.

  FIG. 4 shows a preferred configuration based on FIG. 2 of the method according to the invention for modifying the input image 201 to produce the output image 202.

  This configuration differs from FIG. 2 in that the step F of low-pass filter processing can be applied to the noise signal S1 obtained from the input image 201. Thereby, regardless of the noise level included in the input image 201, it is possible to generate a rough image 204 corresponding to the signal component. For example, the filter kernel may be changed according to the noise level carried by the signal S1.

  If the noise level σ of the input image 201 is high, the cutoff frequency of the low pass filter must be low. It may be defined that the previously defined first kernel k1 is used for a low noise level σ1 and the previously defined second kernel k2 is used for a high noise level.

関数（ｆ１，ｆ２，．．．ｆ９）は、例えば、基礎実験から得られる。 Alternatively, each coefficient of the kernel k is defined by a function (f1, f2,...) Depending on the noise level σ as follows, and the filter coefficient can be adapted to the noise level σ detected in the input image 201. You may do it.

The function (f1, f2,... F9) is obtained from a basic experiment, for example.

This configuration differs from FIG. 2 in that the step 206 for emphasis can be applied to the noise signal S2 generated from the input image 201. This makes it possible to generate an enhanced intermediate image 207 having an optimized sharpness regardless of the noise level included in the input image 201. For example, the relational expression (2) can be modified as follows when the coefficient α depending on the noise level σ is introduced.

  As the noise level σ of the input image 201 increases, α must increase. For example, the relationship between α and σ can be made linear.

  The filtering step F and the emphasizing step 206 need not be adapted to the noise level σ at the same time, and the adaptation to the noise level σ may relate to only one of these steps.

  The noise level σ of the input image 201 is measured by step DET. This step generates first and second signals S1 and S2 that are proportional to the noise level σ (signals S1 and S2 may be the same). For example, the noise level σ can be obtained from algorithms known to those skilled in the art. For example, such a signal reflects the magnitude of analog noise (eg, when using an algorithm based on a frequency spectrum) and / or (eg, around an 8 × 8 pixel or 16 × 16 pixel block). Reflects the magnitude of the digital noise (when using a blocking impact detector that measures activity).

The method according to the present invention can be implemented in a system that modifies the input image 101 to produce the output image 102. This system comprises processing means for realizing the various steps of the method according to the invention described above. In particular, this system has the following means and steps:
Means 103 for dividing the input image 101 into a coarse image 104 and a fine image 105; The means 103 may include a low-pass filter that receives the input image 101 to generate the coarse image 104, and a subtraction means SUB that subtracts the coarse image 104 from the input image 101 so as to generate the fine image 105. it can. This means 103 corresponds, for example, to code instructions (ie computer programs) stored in a memory and executed by a signal processor. The low-pass filter coefficients are advantageously adapted to the signal S1 reflecting the noise level arising from the input image 101.
Means 106 for enhancing the sharpness of a coarse image in order to generate an intermediate image 107 with enhanced sharpness. This means 106 corresponds, for example, to a code instruction (ie a computer program) for encoding the sharpness enhancement algorithm described with reference to FIG. This code instruction is stored in memory and executed by the signal processor. It is further advantageous that this enhancement is applied to the signal S2 reflecting the noise level obtained from the input image 101.
Combining the fine image 105 with the intermediate image 107 with enhanced sharpness to produce the output image 102; The means 108 corresponds, for example, to code instructions (ie computer programs) stored in the memory and executed by the signal processor.

  This system may be embedded as an electronic card. The system may also be implemented in a video device (eg, television, video transmission device, etc.) that receives the input signal 101 and displays the output image 102 on a display or transmits the output image 102 to a communication channel.

  The invention also relates to a computer program having code instructions for implementing the various steps of the method according to the invention.

  The use of the verb “comprise” or its conjugations does not exclude the presence of elements or steps other than those stated in the claims.

FIG. 3 shows a method according to the invention for modifying an input image 101 to generate an output image 102. Fig. 2 shows in detail the method according to the invention. It is a figure which does not show a mode that the sharpness of an image is emphasized as an example. FIG. 2 shows a preferred method according to the invention.

Claims (12)

A method for modifying an input image to generate an output image, the method comprising:
The method
-Dividing the input image into a coarse image and a fine image;
-Enhancing the sharpness of the coarse image to produce an intermediate image with enhanced sharpness;
Combining the intermediate image with enhanced sharpness and the fine image to produce the output image;
Having a method. The dividing step includes:
-Low-pass filtering the input image to produce the coarse image;
-Subtracting the coarse image from the input image to generate the fine image;
The method of claim 1, comprising:   The method of claim 2, wherein the low pass filtering is adaptable to a first noise signal obtained from the input image.   The method of claim 2, wherein the enhancing step is adaptable to a second noise signal obtained from the input image.   The method of claim 2, wherein the low-pass filtering and the enhancing step are adaptable to a noise signal obtained from the input image.   The method according to claim 3, 4 or 5, further comprising measuring a noise level of the input signal to generate the noise signal.   3. A method according to claim 1 or 2, wherein the step of combining comprises adding an intermediate image with enhanced sharpness to the fine image to produce the output image. A system for modifying an input image so that an output image is generated,
The system
-Means for dividing the input image into coarse and fine images;
Means for enhancing the sharpness of the coarse image so that an intermediate image with enhanced sharpness is generated;
-Means for combining the intermediate image with enhanced sharpness and the fine image so that the output image is generated;
Having a system. The means for dividing is
A low pass filter for low pass filtering the input image so that the coarse image is generated;
Means for subtracting the coarse image from the input image so that the fine image is generated;
9. The system of claim 8, comprising:   The system of claim 9, wherein the low pass filter and / or the enhancing means is adaptable to a noise signal obtained from the input image. A video device that modifies an input image so that an output image is generated,
The video device comprises:
-Means for dividing the input image into coarse and fine images;
Means for enhancing the sharpness of the coarse image so that an intermediate image with enhanced sharpness is generated;
-Means for combining the intermediate image with enhanced sharpness and the fine image so that the output image is generated;
Having video equipment.   A computer program comprising code instructions for performing the various steps of the method according to any one of the preceding claims.

Images