JP2012094960A - Motion adaptive field interpolation device and motion adaptive field interpolation method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To suppress erroneous detection, as a static image, of an area which has to be detected originally as a moving image, without increase in memory capacity and arithmetic quantity.SOLUTION: A motion adaptive field interpolation device, which converts an interlace image to a progressive image with a coefficient expressing a moving amount of an image based on a difference between frames is provided with: a history update part which calculates and updates an inter-frame difference history expressing a history of the difference between the frames of multiple fields; a coefficient calculation part which calculates the coefficient expressing the moving amount of the image using the inter-frame difference history; and an interpolation part which performs field interpolation on the basis of the coefficient.

Description

  The present invention relates to a motion adaptive field interpolation device and a motion adaptive field interpolation method for converting an interlaced image into a progressive image using a coefficient representing the amount of motion of the image based on the interframe difference.

  Field interpolation is known in which an interlaced image is converted into a progressive image by interpolating lines in each field of the interlaced image. For field interpolation, inter-field interpolation is performed using pixels existing in a field adjacent to the field to be interpolated, and intra-field interpolation is performed using pixels existing in the same field as the field to be interpolated. There is. Intra-field interpolation is suitable for moving images, and inter-field interpolation is suitable for still images. Therefore, in order to obtain a high quality progressive image, motion adaptive field interpolation is known in which intra-field interpolation pixels and inter-field interpolation pixels are adaptively weighted and averaged using a coefficient representing the amount of motion of the image. For example, Patent Document 1 discloses that interpolation is performed by averaging weights between inter-field interpolation pixels and intra-field interpolation pixels using a motion coefficient k (0 ≦ k ≦ 1) representing the amount of motion of an image. Hereinafter, a conventional example disclosed in Patent Document 1 will be described with reference to the drawings.

  FIG. 6 is a configuration diagram of the motion adaptive field interpolation circuit disclosed in Patent Document 1. In FIG. In FIG. 6, 201, 203 and 204 are field memories, 202 and 205 are line memories, 206, 207, 212 and 219 are addition circuits, 208 and 209 are multiplication circuits, 210 and 211 are coefficient multiplication circuits, 213, 214, Reference numeral 215 denotes a subtraction circuit, 216, 217, and 218 denote absolute value circuits, and 220 denotes a coefficient calculation circuit.

  FIG. 7 shows the arrangement of lines of four consecutive fields by a time axis t and a vertical axis V of the field. In FIG. 7, the vertical axis represents the vertical position of the line, and the horizontal axis represents time in field units. i (i = 1, 2,...) indicates the number of fields, and white circles and black circles each indicate one line. Further, a black circle indicates a line to be interpolated. A1, A2, B1, C1, C2, D1, and X in FIG. 6 represent lines indicated by the same reference numerals in FIG.

  FIG. 8 is a flowchart showing a conventional processing procedure based on the configuration diagram shown in FIG. The process of FIG. 8 is repeated for each interpolated pixel. When processing for one interpolation pixel is started, first, interframe differences SAD1 and SAD2 are calculated (FIG. 8, step S10). In FIG. 7, if the field of interest for which interpolation processing is performed is i-1, SAD1 is the absolute difference value between the field of interest (i-1) and two fields before the field of interest (i-3). SAD2 is a difference absolute value between the field immediately before the field of interest (i-2) and the field immediately after the field of interest (i). In FIG. 7, SAD1 is the absolute difference between the pixel on line A1 and the pixel on line C1, and the absolute difference between the pixel on line A2 and the pixel on line C2, and SAD2 is the difference between the pixel on line B1 and the pixel on line D1. This is the absolute value of the difference from the pixel.

  Next, the sum SADsum of the inter-frame differences SAD1 and SAD2 is calculated (FIG. 8, step S20). Further, a motion coefficient k = Gain × SADsum is calculated so that 0 ≦ k ≦ 1 (where 1 is 1 or more) (FIG. 8, step S30). Finally, field interpolation is performed by Pout = k × Pmove + (1−k) × Pstil (FIG. 8, step S40). Pmove is a pixel value interpolated from the pixel of line B1 and the pixel of line D1, and Pstil is a pixel value interpolated from the pixel of line C1 and the pixel of line C2. In this way, field interpolation is adaptively performed based on the inter-frame differences of all lines used for interpolation.

Japanese Examined Patent Publication No. 7-12214

  However, according to the conventional motion adaptive field interpolation, an area that should originally be detected as a moving image may be erroneously detected as a still image. Hereinafter, the background of erroneous detection as a still image will be described with reference to the drawings.

  Here, the case where the pixel (target pixel) indicated by a thick frame is interpolated in the image pattern shown in FIG. I (i = 1, 2,...) In the figure indicates the number of fields. With a pixel value of 1.0 as a background, vertical stripe patterns with pixel values of 0 and 0.5 are moved by two pixels per field from the left to the right of the screen. Although FIG. 9 shows the i-8 field to the i + 1 field, the pixel of interest with a thick frame is assumed to be the background of the pixel value 1.0 before the i-8 field. Lines T1, T2, T3,... In the top field indicate lines in which an image exists, and in the meantime, lines that are thinned out without an image. Similarly, lines B1, B2, B3,... In the bottom field indicate lines in which an image exists, and in the meantime, lines are thinned out without an image.

  The image pattern in FIG. 9 is as shown in FIG. 10 according to the flowchart in FIG. In FIG. 10, SAD1 is the absolute difference value between pixels (pixels on line A1 and pixel on line C1) located immediately above the target pixel indicated by the thick frame, and SAD2 is the same location as the target pixel indicated by the thick frame. The absolute value of the difference between the pixels located at (pixels on the line B1 and pixels on the line D1). First, the i-6th field will be described. Since SAD1 is the absolute difference between the i-6 field and the i-8 field, SAD1 = ABS {(i-6)-(i-8)} = 0. (FIG. 8, step S10). Further, since SAD2 is an absolute difference value between the i-5 field and the i-7 field, SAD2 = ABS {(i-5)-(i-7)} = 0 (FIG. 8, step S10). . As a result, the sum SADsum of the inter-frame differences SAD1 and SAD2 becomes SAD1 + SAD2 = 0 (FIG. 8, step S20), and assuming that the coefficient calculation circuit 220 of FIG. 6 is a multiplier and Gain = 8, the motion coefficient k is Gain × SADsum. = 8 × 0 = 0 (FIG. 8, Step S30). As a result, since Pout = k × Pmove + (1−k) × Pstil = 0 × Pmove + (1−0) × Pstil = Pstil = still image processing is performed (FIG. 8, step S40). In this case, since the target pixel is the background of the pixel value 1.0, the still image processing can be said to be a desirable processing. Desirable processing is performed in the same procedure for the subsequent i-5th to i-3th fields. Here, the still image processing is performed for the i-2th field. In this case, it is desirable to perform the moving image processing. When still image processing is performed, the pixel of interest has a pixel value of 0, but should represent a pixel value of 0.5 in order to represent a vertical stripe pattern. In addition, the still image processing is also performed for the i-1th field. In this case, it is desirable to perform the moving image processing. When still image processing is performed, the pixel of interest has a pixel value of 0.5, but should represent a pixel value of 0 to represent a vertical stripe pattern. Finally, desirable processing is performed for the i-th field.

  As described above, according to the conventional motion adaptive field interpolation, an area that should be detected as a moving image in the i-2th field and the i-1th field is erroneously detected as a still image. As a result, as shown in FIG. 11, the pixel value 0.5 in the i-2 field becomes the pixel value 0, and the pixel value 0 in the i-1 field is the pixel value. There was a problem that the image quality deteriorated to an unpleasant level of 0.5. In order to avoid this, the number of fields to be referred to and the calculation amount are increased to improve the detection sensitivity. However, such a method has a problem that the cost increases with an increase in memory capacity and calculation amount. That is, in the image pattern as shown in FIG. 9, it is necessary to refer to the i-5 field in order to correctly detect the image in the i-1 field as a moving image, and the memory for storing the image for 5 fields is provided. I need it. If the number of repeated vertical stripes increases, it is necessary to refer to a past field, and a memory for storing an image for a larger number of fields is required.

  The present invention has been made in order to solve the above-described problems, and an object of the present invention is to suppress erroneous detection of a region that should originally be detected as a moving image as a still image without increasing the memory capacity or the calculation amount. A motion adaptive field interpolation apparatus and a motion adaptive field interpolation method are provided.

  In order to solve the above problems, a motion adaptive field interpolation device according to an embodiment of the present invention is a motion adaptive field interpolation device that converts an interlaced image into a progressive image using a coefficient representing the amount of motion of an image based on an interframe difference. A history update unit that calculates and updates an inter-frame difference history that represents a history of inter-frame differences in a plurality of fields, and a coefficient that calculates a coefficient that represents an amount of motion of an image using the inter-frame difference history A calculation unit and an interpolation unit that performs field interpolation based on the coefficient are provided.

  In the motion adaptive field interpolation device, the difference between frames may use an absolute difference between a field immediately before the field of interest to be field-interpolated and a field immediately after the field of interest.

  In the motion adaptive field interpolation device, the history update unit may calculate a sum or a maximum value of both the inter-frame difference and the inter-frame difference history.

  In the motion adaptive field interpolation device, the history update unit may provide a history coefficient α that determines the degree of history of inter-frame differences, and perform weighted averaging using the α.

  In the motion adaptive field interpolation device, the history update unit includes a field memory that stores an interframe difference history SADhtop for a top field and a field memory that stores an interframe difference history SADhbot for a bottom field. The inter-frame difference history SADhtop is calculated based on the inter-frame difference, the field memory for the top field is updated, and the inter-frame difference history SADhbot is calculated based on the inter-frame difference of the bottom field. The memory may be updated.

  In order to solve the above problems, a motion adaptive field interpolation method according to an embodiment of the present invention is a motion adaptive field interpolation method that converts an interlaced image using a coefficient that represents the amount of motion of an image based on a difference between frames. A step of calculating and updating an inter-frame difference history representing a history of inter-frame differences of a plurality of fields, a step of calculating a coefficient representing an amount of motion of an image using the inter-frame difference history, and the coefficient And a field interpolation step based on the step.

  According to the present invention, since the inter-frame difference history is calculated and updated for each field of interest, it is possible to increase the memory capacity and the calculation amount to erroneously detect a region that should be detected as a moving image as a still image. It is possible to provide a motion-adaptive field interpolation device and a motion-adaptive field interpolation method that can be suppressed without causing a change.

It is a block diagram of the motion adaptive field interpolation circuit in the embodiment of the present invention. It is a flowchart which shows the process sequence in embodiment of this invention. It is a figure which shows the process result in embodiment of this invention. It is a figure which shows the process result in embodiment of this invention. It is a figure which shows the detection result of a prior art and embodiment of this invention. 1 is a configuration diagram of a motion adaptive field interpolation circuit disclosed in Patent Document 1. FIG. It is a figure for demonstrating interframe difference SAD1 and SAD2. It is a flowchart which shows the conventional process sequence. It is a figure which shows an example of the image used as field interpolation object. It is a figure which shows the conventional process result. It is a figure which shows the image after the field interpolation of the prior art and embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a configuration diagram of a motion adaptive field interpolation circuit 10 according to an embodiment of the present invention. The motion adaptive field interpolation circuit 10 is an example of a motion adaptive field interpolation device that converts an interlaced image into a progressive image using a coefficient representing the amount of motion of the image based on the interframe difference. In FIG. 1, 201, 203, and 221 are field memories, 202 is a line memory, 206, 207, 212, 224, and 225 are adder circuits, 208 and 209 are multiplier circuits, 210, 211, 222, and 223 are coefficient multiplier circuits, Reference numeral 214 denotes an absolute value circuit, and 220 denotes a coefficient calculation circuit. B1, C1, C2, D1, and X in FIG. 1 represent lines indicated by the same reference numerals in FIG.

  FIG. 2 is a flowchart showing a processing procedure of the motion adaptive field interpolation circuit 10 in the embodiment of the present invention. Compared to the conventional case (FIG. 8), step S10 in FIG. 8 is changed to step S11 in FIG. 2, and step S20 in FIG. Further, steps S21 to S24 indicated by a broken line frame in FIG. 2 are added. Steps S30 and S40 are the same as in the prior art. Hereinafter, a case where the pixel (target pixel) indicated by a thick frame in FIG.

  First, the input image signal is delayed by the field memories 201 and 203 and the line memory 202, and the interframe difference SAD2 is calculated via the subtractor 213 and the absolute value circuit 214 (FIG. 2, step S11). Next, after reading the inter-frame difference history SADh (i−1) of the previous field from the field memory 221 (FIG. 2, step S21), the adder 225 causes the inter-frame difference SAD2 and the inter-frame difference history SADh (i−1). ) Is calculated and passed to the coefficient calculation circuit 220 (FIG. 2, step S22).

  When the coefficient calculation circuit 220 receives ΣSAD, it calculates a motion coefficient k = Gain × ΣSAD so that 0 ≦ k ≦ 1 (where 1 is 1) (FIG. 2, step S30). The calculation of the motion coefficient k is to detect the motion of the image, thereby detecting (determining) whether it is a moving image or a still image. Adders 206 and 207, multipliers 208, 209, and 212, and coefficient multipliers 210 and 211 are interpolation circuits. This interpolation circuit is represented by Pmove = (C1 + C2) / 2, Pstil = (B1 + D1) / 2, Pout = k × Pmove + (1−k) × Pstil, and the field is represented by the motion coefficient k received from the coefficient calculation circuit 220. Interpolate (FIG. 2, step S40).

  On the other hand, a new inter-frame difference history SADh (i) is calculated by the coefficient multipliers 222 and 223 and the adder 224 (FIG. 2, step 23), and SADh (i−1) on the field memory 221 is replaced by SADh (i). Update (FIG. 2, step S24).

  The coefficient multipliers 222 and 223 and the adder 224 are represented by SADh (i) = (1−α) × SAD2 + α × SADh (i−1). The history coefficient α in the equation is a coefficient for determining the degree of the history of the interframe difference, and can take a value of 0 ≦ α ≦ 1. The closer the history coefficient α is to 1, the more the past frame difference can be used. As described above, the inter-frame difference history SADh (i) can be said to be an accumulated value of the inter-frame difference history.

  The image pattern in FIG. 9 is as shown in FIGS. 3 and 4 according to the flowchart in FIG. First, the i-6th field will be described. SAD2 is ABS {(i-5)-(i-7)} = 0 (FIG. 2, step S11), and the target pixel in the thick frame is before the i-8 field. Since the background of the pixel value is 1.0, the inter-frame difference history SADh (i−1) of the previous field stored in the field memory 221 at this time is 0 (FIG. 2, step S21). As a result, the sum ΣSAD of the inter-frame difference SAD2 and the inter-frame difference history SADh (i−1) becomes SAD2 + SADh (i−1) = 0 + 0 = 0 (FIG. 2, step S22), and the motion coefficient k is Gain × ΣSAD. = 8 × 0 = 0 (FIG. 2, step S30). As a result, since Pout = k × Pmove + (1−k) × Pstil = 0 × Pmove + (1−0) × Pstil = Pstil = still image processing is performed (FIG. 2, step S40). In this case, the point that still image processing is desirable is as described above. On the other hand, the history coefficient α is assumed to be 0.6, and the new inter-frame difference history SADh (i) is (1−α) × SAD2 + α × SADh (i−1) = 0.4 × 0 + 0.6 × 0. = 0 (FIG. 2, step S23). This inter-frame difference history SADh (i) = 0 is used as the inter-frame difference history SADh (i-1) one field before when the step S21 is executed for the next i-5 field.

  Desirable processing is performed in the same procedure for the subsequent i-5th to i-3th fields. Here, according to the conventional motion adaptive type field interpolation, there is a problem that a region that should be detected as a moving image in the i-2th field and the i-1th field is erroneously detected as a still image. Hereinafter, it will be described that the video is correctly detected according to the present invention.

  First, the i-2 field will be described. SAD2 becomes ABS {(i-1)-(i-3)} = 0 (FIG. 2, step S11), and 1 stored in the field memory 221 at this time. The inter-frame difference history SADh (i−1) before the field is 0.312 (FIG. 2, step S21). As a result, the sum ΣSAD of the inter-frame difference SAD2 and the inter-frame difference history SADh (i−1) becomes SAD2 + SADh (i−1) = 0 + 0.312 = 0.322 (FIG. 2, step S22), and the motion coefficient k is Gain × ΣSAD = 8 × 0.312 = 1 (FIG. 2, step S30). As a result, since Pout = k × Pmove + (1−k) × Pstil = 1 × Pmove + (1-1) × Pstil = Pmove, moving image processing is performed (FIG. 2, step S40). On the other hand, the new inter-frame difference history SADh (i) is (1−α) × SAD2 + α × SADh (i−1) = 0.4 × 0 + 0.6 × 0.312 = 0.1872 (FIG. 2, step) S23).

  Next, the i-1th field will be described. SAD2 becomes ABS {(i-0)-(i-2)} = 0 (FIG. 2, step S11), and 1 stored in the field memory 221 at this time point The inter-frame difference history SADh (i−1) before the field is 0.1872 (FIG. 2, step S21). As a result, the sum ΣSAD of the inter-frame difference SAD2 and the inter-frame difference history SADh (i−1) becomes SAD2 + SADh (i−1) = 0 + 0.1872 = 0.1872 (FIG. 2, step S22), and the motion coefficient k is Gain × ΣSAD = 8 × 0.1872 = 1 (FIG. 2, step S30). As a result, since Pout = k × Pmove + (1−k) × Pstil = 1 × Pmove + (1-1) × Pstil = Pmove, moving image processing is performed (FIG. 2, step S40). On the other hand, the new inter-frame difference history SADh (i) is (1−α) × SAD2 + α × SADh (i−1) = 0.4 × 0 + 0.6 × 0.1872 = 0.12332. S23). A desirable process is performed in the same procedure for the subsequent i-th field.

  FIG. 5 shows a summary of the detection results of the prior art and the embodiment of the present invention. As shown in this figure, according to the prior art, in the i-2th field and the i-1th field, an area that should be detected as a moving image is erroneously detected as a still image. On the other hand, according to the present invention, the i-2 field and the i-1 field are also correctly detected as moving images. Although the still image processing is desirable for the i-5th field, since the vertical stripe pattern appears at the pixel position of interest, even if the moving image processing is performed, the image quality is not disturbed.

  FIG. 11 shows an image after field interpolation according to the prior art and the embodiment of the present invention. As shown in this figure, according to the prior art, the pixel value 0.5 in the i-2 field becomes the pixel value 0, and the pixel value 0 in the i-1 field. The pixel value becomes 0.5. On the other hand, according to the present invention, the pixel value 0.5 in the i-2 field becomes the pixel value 0.5, and the pixel value 0 in the i-1 field is the pixel value 0. It has become.

  As described above, according to the motion adaptive field interpolation circuit 10 according to the embodiment of the present invention, the inter-frame difference history is calculated and updated for each field of interest, so that an area that should originally be detected as a moving image Can be suppressed without increasing the memory capacity and the amount of computation. This makes it possible to obtain a high-quality progressive image with an inexpensive and high-speed circuit.

  Here, the sum ΣSAD of the inter-frame difference SAD2 and the inter-frame difference history SADh (i−1) is calculated (FIG. 2, step S22), but the present invention is not limited to this. For example, the same effect can be obtained by calculating the maximum value of SAD2 and SADh (i−1) instead of the sum ΣSAD.

  Further, here, after reading the inter-frame difference history SADh (i−1) of the previous field from the field memory 221 (FIG. 2, step S21), a new inter-frame difference history SADh (i) is calculated to calculate the field memory. Although SADh (i-1) on 221 is to be updated with SADh (i) (FIG. 2, steps S23 → S24), the present invention is not limited to this. For example, a field memory for storing the inter-frame difference history SADhtop for the top field and a field memory for storing the inter-frame difference history SADhbot for the bottom field are provided, and the inter-frame difference history SADhtop based on the top-frame difference is calculated. Alternatively, the field memory for the top field may be updated and the field memory for the bottom field may be updated by calculating the inter-frame difference history SADhbot based on the inter-frame difference of the bottom field. In this way, although a necessary memory capacity increases, a better image quality can be expected.

  The present invention can be implemented not only as a motion adaptive field interpolation circuit 10, but also as a motion adaptive field interpolation method using a characteristic processing unit included in such a motion adaptive field interpolation circuit 10 as a step. It can also be realized as a program that causes a computer to execute these steps. It goes without saying that such a program can be distributed via a recording medium such as a CD-ROM or a transmission medium such as the Internet.

DESCRIPTION OF SYMBOLS 10 ... Motion adaptive type field interpolation circuit 201,203,221 ... Field memory 202 ... Line memory 206,207,212,224,225 ... Adder 208,209 ... Multiplier 210, 211,222,223 ... Coefficient multiplier 214 ... Absolute value circuit 220 ... Coefficient calculation circuit

Claims (6)

  A motion adaptive field interpolation device that converts an interlaced image into a progressive image using a coefficient that represents the amount of motion of the image based on the interframe difference, and calculates an interframe difference history that represents a history of interframe differences in a plurality of fields. A history update unit for updating, a coefficient calculation unit for calculating a coefficient representing an amount of motion of an image using the inter-frame difference history, and an interpolation unit for performing field interpolation based on the coefficient. Motion adaptive field interpolator.   2. The motion adaptive field interpolation device according to claim 1, wherein the inter-frame difference uses an absolute difference value between a field immediately before a field of interest to be field-interpolated and a field immediately after the field of interest.   3. The motion adaptive field interpolation device according to claim 1, wherein the history update unit calculates a sum or a maximum value of both the inter-frame difference and the inter-frame difference history.   4. The motion adaptive field interpolation according to claim 1, wherein the history update unit provides a history coefficient α that determines a degree of a history of inter-frame differences, and performs weighted averaging using the α. 5. apparatus.   The history update unit includes a field memory for storing an interframe difference history SADhtop for a top field and a field memory for storing an interframe difference history SADhbot for a bottom field, and an interframe difference history based on an interframe difference of a top field. The SADhtop is calculated to update the field memory for the top field, and the inter-frame difference history SADhbot based on the difference between the frames in the bottom field is calculated to update the field memory for the bottom field. The motion adaptive field interpolation device according to any one of claims 1 to 4.   A motion adaptive field interpolation method that converts an interlaced image into a progressive image using a coefficient that represents the amount of motion of the image based on the interframe difference, and calculates an interframe difference history that represents a history of interframe differences in a plurality of fields. A motion adaptive field interpolation comprising: an updating step; a step of calculating a coefficient representing an amount of motion of an image using the inter-frame difference history; and a field interpolation based on the coefficient. Method.

Images