KR20190014381A - Image processing method and display device using the same - Google Patents

Abstract

According to the present invention, an image processing method comprises the steps of: classifying blocks included in a first image into one or more clusters based on brightness information and edge direction information of a block; calculating a first motion vector indicating a corresponding block by finding the corresponding block having the minimum difference value in a second image temporally adjacent to the first image for each block; finding a representative motion vector and a credit value for each cluster including at least one block based on the first motion vector; and selecting a final motion vector among candidate motion vectors including the at least one motion vector and the representative motion vector using the representative motion vector and the credit value for each block. Accordingly, the image quality of an interpolation image can be improved and an artifact can be reduced by accurately estimating a motion vector in a boundary part of two objects.

Description

TECHNICAL FIELD [0001] The present invention relates to an image processing method and a display device using the image processing method.

The present invention relates to an image processing method and a display device using the same.

2. Description of the Related Art [0002] There have been various demands for a display device for displaying images as an information society has advanced. In recent years, various displays such as a liquid crystal display (LCD) and an organic light emitting diode (OLED) Flat panel display devices are widely used.

A frame rate up conversion technique for increasing the number of frames to be displayed per unit time is applied to a flat display device while the data processing capability of the display device is improved and the user's demand for a high quality screen is increased have. This technique is a technique for creating an intermediate image or an interpolation image between input images to eliminate blur or judder that occurs when an object (or object) included in the image moves . Particularly, this technique is highly effective for a liquid crystal display device in which a residual image remains.

In order to increase the frame rate, a motion estimation (ME) is performed on a value representing a motion of an object between consecutive images, that is, a motion vector (MV), and a motion compensation (MC) .

In this motion estimation process, it is important to maintain the same motion vector for each block occupied by the same object. To do this, a candidate motion vector is selected for each block and the final motion vector is selected. In the process of selecting the final motion vector, the motion vectors of the neighboring blocks are used and the brightness similarity between blocks is compared. That is, a motion vector of a neighboring block having similar characteristics is set as a candidate motion vector, and a motion estimation process is performed.

In addition, a block that can not fill the interpolation image with data in the motion estimation process, that is, a hole block, may be generated. In order to fill such a hole block, the motion vector of the hole neighboring block is generally utilized.

However, in order to select the candidate motion vector, the motion vectors of the neighboring blocks on the upper, lower, and both sides are utilized, and the neighboring blocks can not always correspond to the same object. When the object is located on a block or the boundary of the object is located in the adjacent block, the motion vector of another object is referred to, and the motion vector is inaccurately estimated. Artifacts are generated in the final interpolated image, do.

It is an object of the present invention to provide an image processing method and an image processing apparatus for accurately estimating a motion vector when an interpolation image is generated to increase the frame rate.

According to an embodiment of the present invention, there is provided a method of processing an image, the method comprising: classifying blocks included in a first image into one or more clusters based on luminance information and edge direction information of the block; Calculating, for each block, a first motion vector indicating a corresponding block by finding a corresponding block with a minimum difference value in a temporally adjacent second image in the first image; Obtaining a representative motion vector and a credit value for each cluster including at least one block based on the primary motion vectors; And selecting, for each block, a final motion vector from the candidate motion vectors including at least the primary motion vector and the representative motion vector, using the representative motion vector and the credit value.

In one embodiment, the luminance information indicates one of the sections divided by a predetermined number of the entire luminance ranges, and the edge direction information can indicate one of four halves of the 180 degrees and a non-direction.

In one embodiment, the classifying step can be classified into clusters corresponding to combinations of the luminance and edge directions of the blocks among the clusters that can be obtained by combining luminance information and edge direction information for each block.

In one embodiment, the sorting step may include applying a Sobel filter to a block to obtain an edge direction within the block, and selecting the direction with the highest frequency as the edge direction of the block.

In one embodiment, for each block, for each block, for each candidate motion vector added to the block, the difference between the corresponding candidate motion vector and the corresponding block corresponding thereto and the representative motion vector of the cluster to which the block belongs And the penalty value obtained based on the credit value, and the candidate motion vector having the smallest sum can be selected as the final motion vector for the block.

In one embodiment, the penalty value may be a value obtained by multiplying the difference between the candidate motion vector and the representative motion vector by a credit value.

In one embodiment, the step of obtaining comprises calculating an average of the primary motion vectors calculated for the blocks included in the cluster as a representative motion vector for the cluster, and calculating a value inversely proportional to the deviation of the primary motion vectors It can be calculated as the credit value for the cluster.

In one embodiment, the candidate motion vector includes the primary motion vector of the block and the representative motion vector of the cluster containing the block, and the global motion vector, the motion vector of the neighboring block, the zero motion vector, And may further include at least one of the modified motion vectors.

In one embodiment, the image processing method may further comprise generating an intermediate image to be inserted between the first image and the second image using the final motion vector.

According to another aspect of the present invention, there is provided an image processing apparatus comprising: a motion estimator for generating a motion vector from a temporally successive first frame and a second frame; And a motion compensator for correcting the image data of the intermediate frame to be inserted between the first frame and the second frame using the generated motion vector, wherein the motion estimator is configured to calculate, based on the luminance information and the edge direction information of the block, A clustering unit for classifying the blocks included in the first image into one or more clusters; A first motion vector selection unit for calculating a first motion vector indicating a corresponding block by finding a corresponding block having a minimum difference value in the second image for each block; A representative motion vector generation unit for obtaining a representative motion vector and a credit value for each cluster including at least one block based on the primary motion vectors; And a final motion vector selection unit for selecting a final motion vector among the candidate motion vectors including at least the primary motion vector and the representative motion vector using the representative motion vector and the credit value for each block .

According to another aspect of the present invention, there is provided a display device including: a display panel including pixels intersecting data lines and gate lines and formed in a matrix; An image processor for interpolating input image data through motion estimation and motion compensation; A timing controller for adjusting the control signals so that the operation speed is doubled when receiving video data output by the video processing unit and having a doubled frame rate; A data driving circuit for converting image data input from the timing controller into data voltages in accordance with the control of the timing controller and outputting the data voltages to data lines; And a gate driving circuit for generating a scan pulse synchronized with the data voltage in accordance with the control of the timing controller and sequentially supplying the scan pulse to the gate lines, and the image processing unit is configured to perform, based on the brightness information and the edge direction information of the block For each block, a first motion vector pointing to the corresponding block is found by finding a corresponding block in which the difference value is the smallest in the second image temporally adjacent to the first image Calculating a representative motion vector and a credit value for each cluster including at least one block based on the primary motion vectors and for each block using at least a primary motion vector and a credit value, A motion estimation for selecting a final motion vector among the candidate motion vectors including the vector and the representative motion vector .; And a motion compensator for correcting the image data of the intermediate frame to be inserted between the first frame and the second frame using the generated final motion vector.

Accordingly, it is possible to accurately estimate the motion vector at the boundary between two objects, to reduce the occurrence of artifacts, and to improve the image quality of the interpolation image.

FIG. 1 shows a motion estimation and motion compensation process for generating an interpolation image by estimating motion from two successive images,
2 is a block diagram illustrating a motion estimation process,
3A and 3B illustrate a process of selecting a final motion vector among motion vectors generated according to motion estimation,
Figs. 4A and 4B show an example in which an artifact occurs at the boundary between two objects in an interpolation image,
5 is a block diagram of a display device to which an image processing method according to an embodiment of the present invention is applied,
6A and 6B illustrate a method of clustering blocks in a frame using the edge direction and luminance of the block,
FIG. 7 is a block diagram showing a configuration of an image processing unit according to the present invention,
8 is a flowchart illustrating an operation of the motion estimation method according to the present invention,
9A and 9B illustrate an interpolation image generated by estimating a motion vector according to the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Reference will now be made in detail to the preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings. Like reference numerals throughout the specification denote substantially identical components. In the following description, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

FIG. 1 shows a motion estimation and motion compensation process for generating an interpolation image by estimating motion from two consecutive images. FIG. 2 is a block diagram showing a motion estimation process, and FIGS. And a final motion vector is selected from among the generated motion vectors.

The motion estimation ME and the motion compensation MC compare two consecutive images (or frames) Frame (n) and Frame (n + 1), as shown in Fig. 1, Estimates the vector and compensates the motion vector using the estimated motion vector to generate an interpolated frame (Frame (n + 0.5)) between the two frames.

In the process of obtaining the motion vector for frame interpolation, the motion values of the same or similar objects located in two frames, i.e., the motion vectors, are obtained by comparing the front / rear frames (Frame (n) and Frame (n + 1) (Grid mapping or block mapping) to a grid of an intermediate frame (Frame (n + 0/5)), and motion vectors are generated in block or grid units in the intermediate frame. Where the block may be a 4x4 or 8x8 pixel group.

Two intermediate frames are generated using the previous frame (Frame (n)) and the next frame (Frame (n + 1)) based on the motion vector generated for the intermediate frame (Frame (n + 0/5) , And the image data of the intermediate frame (Frame (n + 0.5)) is corrected and output so that the difference is small.

The motion estimation process will be described in detail with reference to FIG. 2. When a previous frame (Frame (n), Frame (n + 1)) is input, (Or the current block) of the previous frame for which the motion vector is to be obtained is compared with the blocks shifted in pixel units within a predetermined search range (Search Range) in the subsequent frame, and the absolute value of the luminance difference And calculates the sum of absolute differences (SAD) for all the pixels included in the block (Block SAD Calculation).

The minimum SAD cost selection is selected by comparing the SAD values for each block moved in pixel units within the search range within the next frame. The block of the next frame having the minimum value is the block most similar to the current block of the previous frame and the primary motion vector for the current block can be obtained based on the position value between the current block and the selected block.

(Candidate MV Addition) is added to the first motion vector extracted through the minimum SAD selection process, and a candidate motion vector that can be a final motion vector is added. In general, a primary motion vector, a motion vector of a neighboring block, a global motion vector (zero MV), a zero vector, and the like may be added as candidate motion vectors. The global motion vector may be given with the front / back frame data.

Then, the final motion vector is selected using the validity of the candidate motion vectors and the image feature (Final MV Selection). There are several methods for selecting the final motion vector. The selection probability of each of the candidate motion vectors can be calculated through the block similarity or the reliability. There are two ways to use the characteristics of the block when calculating the selection probability. One is to determine the probability of selection of the motion vector for the neighboring block by the luminance similarity of the current block and the neighboring block, and the other is to use the bilateral filter ) To calculate the probability depending on the distance and the luminance difference.

Through this process, the motion vector indicating the movement of the object in the frame from the previous frame to the next frame can be estimated, which can be referred to as a forward motion vector.

Similarly, for each block in a subsequent frame, motion vectors can be estimated by searching for similar blocks in the previous frame, which can be referred to as a backward motion vector.

A forward motion vector and a backward motion vector may be used to generate a motion vector for each block of the intermediate frame to be inserted between the previous frame and the subsequent frame. That is, a forward motion vector and a backward motion vector that lead to the same block of the middle frame among the forward motion vector and the backward motion vector estimated for each block from the front and back frames are found, and based thereon, motion vectors .

At this time, if the forward motion vector and the backward motion vector found by guiding to an arbitrary block of the intermediate frame are the same size and the same direction, the motion vector for the corresponding block of the intermediate frame is set as the forward motion vector or half of the backward motion vector As shown in FIG.

However, if the forward motion vector and the backward motion vector guiding the forward motion vector to one block of the intermediate frame among the motion vectors corresponding to half of the backward motion vector are different in size or opposite in direction, The motion vector for the block can not be easily determined, and the motion vector must be determined in a predetermined manner.

In addition, if there is no motion vector guiding the forward motion vector to one block of the middle frame among the motion vectors corresponding to half of the backward motion vector, the corresponding block of the intermediate frame becomes a hole and the motion vector Can not be obtained and the estimated motion vector can not be used for neighboring neighboring blocks.

Further, even if two or more forward motion vectors estimated for different blocks in the previous frame lead to one block of the intermediate frame, the motion vector for the corresponding block of the intermediate frame must be determined. In Fig. 3A, a motion vector is estimated for each block of the previous frame. Since two estimated motion vectors correspond to the same block of the intermediate frame, one motion vector is selected and another motion vector is ignored have.

On the other hand, among the motion vectors selected as the candidate motion vectors in the motion estimation process, the motion vectors of the neighboring blocks are very important. If the primary motion vector selected by the minimum SAD is always correct, it may be simple and good. However, there are various variables in the real image, and there is a block in which a hole is generated and there is no motion vector corresponding to the correct answer.

Therefore, it is necessary to utilize the motion vector of the neighboring block to the maximum in consideration of spatial smoothness. However, there is a problem in the method using the neighboring motion vector so far. Peripheral blocks can not always be the same object. It is common practice to use the motion vectors of the top, bottom, and both side blocks, but if the object spans the block or if there is an object boundary in the neighboring block, the motion vector of another object is referred to.

In order to solve this problem, there is a precedent condition for correctly fetching the surrounding motion vector information. That is, the surrounding motion vector information is used only when the luminance of the neighboring block is similar to the current block. By using this condition, it is possible to reduce the error by limiting the use of surrounding motion vector information when the object is not the same object. However, this method may also fail.

In FIG. 4A, artifacts are generated in the red square. However, since the walls have the same motion vector with the same object, the luminance information of the block is different due to the pattern of the wall, so that the neighboring motion vectors can not be obtained. .

In FIG. 4B, an artifact occurs in the red square. Since the plane and the background building are different objects, the motion vector should not affect each other. However, since the luminance information (Y value) And finally estimates an erroneous motion vector.

Such a problem appears as an artifact in the intermediate image, and can cause a problem that the related area is seriously shaken when checking it with a moving image.

In the present invention, the characteristics of a block for which a motion vector is to be determined are defined more precisely to define the same object between blocks, and the motion vector is spatially smoothed using the same.

To this end, the image processing method according to the present invention classifies each block included in an image into a predetermined number of clusters or clusters by using the luminance and edge direction information of the block, and generates a representative motion vector for each cluster and a credit A representative motion vector for the cluster is added to the candidate motion vector for the block, and a representative motion vector and a confidence value for the cluster are used when the final motion vector is selected.

That is, the candidate motion vectors are shared among blocks having a high probability of being the same object, and the final motion vectors are selected in a similar manner, so that the final motion vectors are classified according to the objects, and the same objects are selected to have uniformity.

5 is a block diagram of a display device to which an image processing method according to an embodiment of the present invention is applied. A display device according to an embodiment of the present invention includes a display panel 10, a timing controller 11, a data driving circuit 12, a gate driving circuit 13, and an image processing unit 20.

The display panel 10 includes pixels in which the data lines 14 and the gate lines 15 intersect and are formed in a matrix form. A thin film transistor (TFT) is formed at the intersection of the data lines and the gate lines of the display panel 10. The display panel 10 may include a liquid crystal display (LCD), a field emission display (FED), a plasma display panel (PDP), an inorganic electroluminescent device, A flat panel display device such as an electroluminescence device (EL) including an organic light emitting diode (OLED), and an electrophoresis (EPD) device. When the display panel 10 is implemented as a display panel of a liquid crystal display element, a backlight unit is required. The backlight unit may be implemented as a direct type backlight unit or an edge type backlight unit.

The image processing unit 20 interpolates the inputted image data (RGB (nHz)) of nHz, for example, 30 Hz through motion estimation and motion compensation. The image processing unit 20 outputs the interpolated image data (RGB '(2 nHz)) of 2 nHz, for example, 60 Hz to the timing controller 11. A detailed description of the image processing unit 20 will be given below with reference to FIGS. 6 to 8. FIG.

The timing controller 11 supplies the video data RGB '(2 nHz) output from the video processing unit 20 to the data driving circuit 12. The timing controller 11 inputs timing signals such as a vertical synchronizing signal Vsync, a horizontal synchronizing signal Hsync, a data enable signal DE and a dot clock CLK from a host system (not shown) And generates control signals for controlling the operation timing of the data driving circuit 12 and the gate driving circuit 13. The control signals include a gate timing control signal GCS for controlling the operation timing of the gate driving circuit 13 and a data timing control signal DCS for controlling the operation timing of the data driving circuit 12. [

The timing controller 11 adjusts the control signals so that the operation speed is doubled when the video data whose frame rate is increased, for example, twice, from the image processing section 20 is input to the data driving circuit 12 and the gate driving circuit 12, (13).

The data driving circuit 12 converts the video data RGB '(2 nHz) into data voltages under the control of the timing controller 11 and outputs them to the data lines 14. The gate drive circuit 13 sequentially supplies a scan pulse synchronized with the data voltage to the gate lines 15 under the control of the timing controller 11. [

The gate drive circuit 13 sequentially supplies the scan pulses synchronized with the data voltage to the gate lines 15 of the display panel 10 under the control of the timing controller 11. [ The gate drive circuit 13 may be composed of a plurality of gate drive integrated circuits each including a shift register, a level shifter for converting the output signal of the shift register into a swing width suitable for driving a TFT of a pixel, and an output buffer . Alternatively, the gate driving circuit 13 may be formed directly on the lower substrate of the display panel 10 using a GIP (gate drive IC in panel) method. In the case of the GIP method, the level shifter is mounted on a PCB (Printed Circuit Board), and the shift register can be formed on the lower substrate of the display panel 10. [

6A and 6B illustrate a method of clustering blocks in a frame using the edge direction and luminance of the block.

In the present invention, in order to cluster the blocks having the same characteristics into the same cluster, the luminance information and the edge direction information of the block are used.

Referring to the edge direction of the block, 180 degrees are divided into four directions (horizontal, 45 degrees, vertical, 135 degrees) at intervals of 45 degrees as shown in FIG. 6A, the edge direction of the block is obtained, The edge direction of the block is obtained by applying a Sobel filter to the block, and the direction with the highest frequency can be selected as the edge direction of the block.

With respect to the luminance of the block, the average luminance (Y value) of the pixels in the block is assigned to one of a predetermined number of ranks. For example, eight levels of luminance of 8 bits (0-255) To the corresponding one of the levels. Alternatively, the intermediate luminance range in which the luminance frequency is high in the entire luminance range may be divided into narrow intervals, and the high luminance and low luminance range may be divided into wide intervals.

The number of clusters can be 32 by combining four directions and eight luminance levels. However, since there are cases where the blocks are composed of only pixels having the same luminance in relation to the edge direction and are not smooth with no edge direction, 40 clusters are possible by combining 5 directions and 8 luminance levels. Therefore, each block can be classified into one of 40 clusters.

FIG. 7 is a block diagram illustrating a configuration of an image processing unit according to the present invention, and FIG. 8 is a flowchart illustrating an operation of a motion estimation method according to the present invention.

The image processing unit 20 according to the present invention receives data of two consecutive frames Frame (n) and Frame (n + 1) and a global motion vector (Global MV) The motion estimator 210 and the motion compensator 220 may be configured to generate and output an intermediate frame (Frame (n + 0.5)) to be inserted in the middle of the frame.

The motion estimator 210 estimates motion for two frames Frame (n) and Frame (n + 1) to estimate a motion vector and performs a grid mapping operation for the intermediate frame using the motion vector, The motion vector estimation unit 210 includes a block clustering unit 211, an SAD calculation unit 212, a minimum SAD selection unit 213, a cluster representative motion vector generation unit 214, A motion vector selection unit 215 and an intermediate frame motion vector determination unit 216. [

The motion compensation unit 220 generates two intermediate frames using the previous frame (Frame (n)) and the next frame (Frame (n + 1)) based on the motion vector generated for the intermediate frame, The image data of the intermediate frame (Frame (n + 0.5)) may be corrected and output so that the difference is smaller.

The motion estimation unit 210 receives the two frames (Frame (n), Frame (n + 1)) and the global motion vector, and generates and outputs a motion vector of the intermediate frame.

Here, the global motion vector is a motion vector generated between two frames when the image sensor swings or moves, or when the image sensor is in a pan / tilt start state, and the object moves in the frame to generate a local motion vector Contrast.

The motion estimator 210 estimates motion from the previous frame (Frame (n)) to the subsequent frame (Frame (n + 1)) to obtain a forward motion vector, (N + 0.5) on the basis of the backward motion vector obtained by estimating the motion from the last frame (Frame (n + 1)) to the previous frame (Frame ), Or both the forward motion vector and the backward motion vector can be obtained, and the motion vector for the intermediate frame can be obtained based on the obtained forward motion vector and the backward motion vector. Hereinafter, the forward motion vector will be mainly described.

The block clustering unit 211 receives two front and rear frames (Frame (n) and Frame (n + 1)) and stores the properties of each block of the preceding frame (Frame The luminance information and the edge direction information are calculated, and the blocks can be classified into a predetermined number of clusters based on the calculated luminance information and edge direction information (Block Clustering in FIG. 8).

As shown in Fig. 6, each block can be classified into one of 40 clusters according to a combination of luminance of eight levels and combination of four directions and smoothness.

The SAD calculation unit 212 obtains the SAD for each block of the previous frame. The SAD calculation unit 212 compares the block with the moved block while moving the block in the predetermined search range within the next frame in pixel units, (Block SAD Calculation in FIG. 8). In the block SAD calculation shown in FIG. 8, the absolute value of the absolute value of the difference is calculated for every pixel included in the block.

In order to reduce the calculation, the SAD calculation unit 212 compares the search range in the following frame with the previous block calculated before the relevant block, for example, the block of the next frame related to the left block Block).

The minimum SAD selection unit 213 selects a block for outputting the smallest value among the SADs calculated by moving the block in pixel units in the subsequent frame by the SAD calculation unit 212, Block is determined, and the amount and direction of the movement to the selected minimum SAD block in the corresponding block are calculated to determine a primary motion vector for the corresponding block (Minimum SAD Cost Selection in FIG. 8).

After the minimum SAD selection unit 213 determines the primary motion vectors for all the blocks in the previous frame, the cluster representative motion vector generation unit 214 generates a cluster representative motion vector based on the primary motion vectors of the blocks belonging to the same cluster, For example, the primary motion vector may be averaged to calculate a representative motion vector for the cluster. A maximum of 40 clusters can be achieved by a combination of luminance and edge directions, but a representative motion vector can be obtained only for a cluster to which a block included in the frame belongs.

In addition, the cluster representative motion vector generation unit 214 obtains a credit value (Confidence) for the motion vector included in the corresponding cluster for each cluster, and the credit value of the motion vector for the cluster is the block For example, a standard deviation or a variance of the first-order motion vector calculated for the motion vector, specifically, a value inversely proportional to the standard deviation.

That is, when the standard deviation is small, it means that the primary motion vectors of the block belonging to the cluster are concentrated near the representative motion vector of the cluster. Therefore, for the blocks included in the cluster, it is possible to increase the probability that the primary motion vector determined for the block is determined as the final motion vector for the block based on the credit value.

On the contrary, if the standard deviation is large, it means that the primary motion vectors of the block belonging to the cluster are distributed widely apart from the representative motion vectors of the cluster. Therefore, with respect to the blocks included in the cluster, it is possible to lower the probability that the primary motion vector determined for the block is determined as the final motion vector for the block based on the credit value.

Thereafter, the final motion vector selection unit 215 determines a final motion vector for each block. First, a candidate motion vector for the corresponding block is added (Candidate MV Addition), and one of the candidate motion vectors is used as a final motion vector Final MV Selection.

As the candidate motion vector, a representative motion vector of the cluster to which the corresponding block belongs may be added in addition to the primary motion vector determined in the corresponding block, the motion vectors of the neighboring blocks, the global motion vector, and the zero motion vector. Since the clustering result is also not 100% accurate, it is not immediately selected, but is added as one of several candidate motion vectors of the block.

In addition, a motion vector in which the x component and the y component of the primary motion vector are respectively changed within a predetermined range may be added as the candidate motion vector. For example, when the primary motion vector is (x, y), (x + 1, y + 1) motion vectors similar to the first-order motion vectors such as (x-1, y-1), (x-1, y + 1), Can be a vector.

When selecting the final motion vector among the candidate motion vectors, the final motion vector selection unit 215 calculates SAD and penalty values for each candidate motion vector, and outputs the candidate motion vector having the smallest value as the final motion vector Can be determined as a vector.

The SAD for the candidate motion vector can be calculated by calculating the difference between the corresponding block and the corresponding block indicated by the corresponding candidate motion vector, that is, by obtaining the absolute value of the luminance difference in the corresponding block and the corresponding block on a pixel-by- .

The penalty value for the candidate motion vector can be set using the representative motion vector of the cluster to which the block belongs and the credit value of the corresponding cluster. That is, Penalty = ((motion vector candidate representative motion vector) x (cluster confidence value)), and the confidence value of the cluster is inversely proportional to the standard deviation of the motion vectors for the block belonging to the cluster Value, for example, Confidence = 256 / (standard deviation).

By setting the penalty value as described above, it is possible to increase the probability that the motion vectors similar to the representative motion vector of the cluster are selected as the final motion vectors for the block. For example, if there are motion vectors of similar selection probability among the plurality of candidate motion vectors , A motion vector close to the representative motion vector of the cluster can be selected.

8, it is possible to estimate a forward motion vector for all the blocks of the previous frame, that is, a motion vector indicating the movement of the object from the previous frame to the following frame, and the intermediate frame motion vector determination unit 216 uses this The motion vector for each block of the intermediate frame can be determined.

Similarly, a motion vector indicating the movement of the object in the backward motion vector of all the blocks in the backward frame, that is, the motion of the object in the preceding frame in the backward frame, can be estimated through the process of FIG. 8, and the intermediate frame motion vector determination unit 216 This can be used to determine the motion vector for each block of the intermediate frame.

The intermediate frame motion vector determination unit 216 may use both the forward motion vector and the backward motion vector to determine the motion vector for each block of the intermediate frame.

9A and 9B illustrate an interpolation image generated by estimating a motion vector according to the present invention.

In the intermediate frame of FIG. 4A according to the conventional method, the motion vector is erroneously determined due to the difference in luminance due to the pattern formed on the wall surface, but the pattern of the wall surface is not constant and artefacts are generated. However, We classify the patterns into one or two clusters, determine the clusters to which each block belongs, estimate the correct motion vector using the representative motion vectors and confidence values for the determined clusters, and thus the intermediate image reflects the pattern of the wall well.

Also, in the intermediate frame of FIG. 4B according to the conventional method, since the luminance of the airplane and the background building do not largely differ, they are recognized as the same object and influence each other on the estimation of the motion vector, resulting in artifacts in the intermediate frame. However, in FIG. 9B, since the aircraft and the wall are selected as different clusters and the representative motion vector of each cluster is used, the artifacts are improved at the boundary between the object and the object

In this manner, the blocks are classified into a plurality of clusters based on the characteristics of blocks such as luminance and edges, the blocks having the same characteristics are grouped together, the representative motion vectors of the clusters and the trust values of the clusters are added to the candidate motion vectors, By using it when selecting a vector, it becomes possible to accurately estimate the motion vector, thereby reducing artifacts in the interpolation image.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention. Therefore, the technical scope of the present invention should not be limited to the contents described in the detailed description of the specification, but should be defined by the claims.

10: display panel 11: timing controller
12: Data driving circuit 13: Gate driving circuit
14: Data line 15: Gate line
20: image processor 210: motion estimator
211: block clustering unit 212: SAD calculation unit
213: minimum SAD selection unit 214: cluster representative motion vector generation unit
215: final motion vector selection unit 216: intermediate frame motion vector determination unit
220: Motion compensation unit

Claims (11)

Classifying the blocks included in the first image into one or more clusters based on the luminance information and the edge direction information of the block;
Calculating a first motion vector pointing to the corresponding block by finding a corresponding block with a minimum difference value in a second image temporally adjacent to the first image for each block;
Obtaining a representative motion vector and a credit value for each cluster including at least one block based on the primary motion vectors; And
And for each block, using the representative motion vector and the credit value, selecting a final motion vector from among the candidate motion vectors comprising at least the primary motion vector and the representative motion vector. The method according to claim 1,
Wherein the brightness information indicates one of a section in which the entire luminance range is divided by a predetermined number, and the edge direction information indicates one of a quarter of 180 degrees and a non-direction. 3. The method of claim 2,
Wherein the classifying step classifies each block into clusters corresponding to combinations of brightness and edge directions of the clusters among clusters that can be obtained by combining the brightness information and edge direction information. 3. The method of claim 2,
Wherein the classifying step comprises applying a Sobel filter to the block to obtain an edge direction within the block, and selecting the direction with the highest frequency among the edge directions. The method according to claim 1,
Wherein the selecting step comprises: for each of the candidate motion vectors added to the block, the difference value between the corresponding candidate motion vector and the corresponding block corresponding to the candidate motion vector, and the representative motion vector and the credit value of the cluster to which the block belongs And a candidate motion vector having the smallest sum is selected as a final motion vector for the corresponding block. 6. The method of claim 5,
Wherein the penalty value is a value obtained by multiplying a difference value between the candidate motion vector and the representative motion vector by the credit value. The method according to claim 1,
Calculating the average of the primary motion vectors calculated for the blocks included in the cluster as a representative motion vector for the corresponding cluster and calculating a value inversely proportional to the deviation of the primary motion vectors for the cluster And calculating a credit value. The method according to claim 1,
The candidate motion vector includes a primary motion vector of a corresponding block and a representative motion vector of a cluster including the corresponding block, and includes a global motion vector, a motion vector of a neighboring block, a zero motion vector, and a transformed motion vector of the primary motion vector The image processing method further comprising: The method according to claim 1,
And generating an intermediate image to be inserted between the first image and the second image using the final motion vector. A motion estimator for generating a motion vector from temporally successive first and second frames; And
And a motion compensation unit for correcting image data of an intermediate frame to be inserted between the first frame and the second frame using the generated motion vector,
Wherein the motion estimator comprises:
A clustering unit for classifying blocks included in the first image into one or more clusters based on luminance information and edge direction information of the block;
A first motion vector selection unit for finding a corresponding block having a minimum difference value in the second image for each block and calculating a first motion vector indicating the corresponding block;
A representative motion vector generation unit for obtaining a representative motion vector and a credit value for each cluster including at least one block based on the primary motion vectors; And
And a final motion vector selection unit for selecting, for each block, a final motion vector among the candidate motion vectors including at least the primary motion vector and the representative motion vector using the representative motion vector and the credit value, And the image processing apparatus. A display panel including pixels intersecting the data lines and the gate lines and formed in a matrix form;
An image processor for interpolating input image data through motion estimation and motion compensation;
A timing controller for adjusting the control signals so that the operation speed is doubled when receiving image data output from the image processing unit, the image data having a doubled frame rate;
A data driving circuit for converting image data input from the timing controller into data voltages under control of the timing controller and outputting the data voltages to data lines; And
And a gate driving circuit for generating a scan pulse synchronized with the data voltage according to the control of the timing controller and sequentially supplying a scan pulse to the gate lines,
Wherein the image processing unit comprises:
The block included in the first image is classified into one or more clusters based on the luminance information and the edge direction information of the block, and a corresponding value of a difference corresponding to the difference in the second image temporally adjacent to the first image Calculating a first motion vector indicating the corresponding block by searching for a block, calculating a representative motion vector and a credit value for each cluster including at least one block based on the first motion vectors, A motion estimator for selecting a final motion vector among the candidate motion vectors including at least the primary motion vector and the representative motion vector using the representative motion vector and the credit value; And
And a motion compensator for correcting image data of an intermediate frame to be inserted between the first frame and the second frame using the generated final motion vector.

Images