JP2005268912A - Image processor for frame interpolation and display having the same - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To generate the image data of an interpolation frame having a small amount of noise, based on a motion vector detected at each block of one kind of size. <P>SOLUTION: When a previous frame F(n-1) associated with a motion vector detected for a block Bb of a current frame F(n), and the block of the interpolation frame F(i-1) are set to be blocks Ba and Be, respectively, the value Ve of a pixel (an interpolation pixel) Pe in the block Be is calculated on the following procedure. It is determined whether or not the difference ¾Va-Vb¾ of values Va and Vb of pixels Pa and Pb positionally corresponding to the interpolation pixel Pe in the block Be in the blocks Ba, Bb each is larger than a prescribed threshold ε. In the case of ¾Va-Vb¾≤ε and ¾Va-Vb¾>ε, the value Ve of the interpolation pixel Pe is calculated by Ve=(Va+Vb)/2 and Ve=(Va+Vb+Vc+Vd)/4, respectively. In this case, Vc and Vd are the values of pixels Pc and Pd positionally corresponding to the interpolation pixel Pe in the previous frame F(n-1) and the current frame F(n). <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an image processing apparatus that generates image data of an interpolation frame to be inserted between adjacent frames constituting a moving image, and a display apparatus including such an image processing apparatus.

  In recent years, the progress of active matrix liquid crystal display devices (hereinafter referred to as “TFT-LCD devices”) using thin film transistors (TFTs) has been remarkable, and viewing angles and responsiveness that have been problematic in conventional TFT-LCD devices. Has also been improved and is used as a display device for large television receivers. However, even at the present time when the response characteristics of the liquid crystal are improved, there is a sense of blur in moving image display on a television receiver using a TFT-LCD device. This is due to the following reason.

  A CRT (Cathode-Ray Tube) display device used in conventional television receivers displays an image by sequentially applying an electron beam to a phosphor during one frame, and each pixel is displayed during one frame. This is a display device that emits light only once instantaneously, that is, an impulse-type display device. In such an impulse-type display device, each pixel does not emit light until it emits light in the next frame after it emits light in a certain frame, so that blur due to line-of-sight tracking does not occur.

  On the other hand, the TFT-LCD device displays an image by controlling the transmittance of the liquid crystal panel with voltage instead of self-emission, and in the case of the transmissive type, it is given from a backlight disposed on the back of the liquid crystal panel. The image is displayed by controlling the amount of light passing through the liquid crystal panel. In a TFT-LCD device, data representing a display image of one frame is written for each line in a plurality of matrix pixel forming portions constituting a liquid crystal panel, and in each pixel where data is written, the data is stored in the data. The corresponding voltage is held until the next frame of data is written. Therefore, the TFT-LCD device is a display device that holds a display image for one frame, that is, a hold-type display device. In such a hold-type display device, it is known that blurring of moving images occurs due to line-of-sight tracking. Conventionally, the response speed of the liquid crystal display device is low, and the response as the display device has required one frame period (16.7 ms) or more, so that the so-called image tailing caused by the low response speed is regarded as a problem. It had been. However, the response characteristics of the liquid crystal have improved over the past few years, and response within one frame period is possible. Accordingly, in the TFT-LCD device, blurring in moving image display due to the hold type (blurring due to line-of-sight tracking) Is becoming a problem.

  As a method of improving blur due to line-of-sight tracking in a hold-type display device such as a TFT-LCD device, the display in the hold-type display device is made to be impulse, such as blinking the backlight or displaying black in one frame. (Hereinafter referred to as “impulse method”) is known. Also known is a method (hereinafter referred to as “frame interpolation method”) in which blurring due to line-of-sight tracking is improved by interpolating an image between frames and doubling the frame frequency (hereinafter referred to as “frame interpolation method”). ).

Of the above methods for improving blur due to line-of-sight tracking in a hold-type display device, the impulse method lowers the brightness or contrast of the display screen (display panel), and also causes flicker that occurs in a CRT display device. The hold type display device also has a problem that it becomes noticeable due to the impulse. For this reason, in a TFT-LCD device or the like, it is preferable to employ a frame interpolation method for increasing the frame frequency in order to improve blurring due to line-of-sight tracking without impairing the features of the hold-type display device as much as possible.
Japanese Patent Laid-Open No. 3-26389 JP-A-2-206991 JP-A-1-192273 JP-A-6-153185 Shuichi Ishiguro, Taiichi Kurita, “Examination of video quality of hold light emitting display by 8 × CRT”, IEICE Technical Report, June 1996, EID96-4, p. 19-26

  As a method of interpolating the frames that make up a moving image, conventionally, one frame of image is divided into several blocks, and interpolated image data between frames using motion vectors detected for each block by the block matching method etc. That is, a method of creating image data of an interpolated frame to be inserted between frames by moving the block by a vector amount obtained by halving the motion vector of each block. (For example, refer to Patent Document 1 and Patent Document 2). However, in this method, since the image data of the interpolation frame is created by moving the image in block units according to the detected motion vector, the image is moved to the background image having no motion. An image including noise is generated as an image. Further, in Patent Document 2, data corresponding to a half value of a motion vector for each block in one of two consecutive frames and 1/2 of the motion vector in the other of the two relevant frames are disclosed. An image processing apparatus that generates image data of an interpolation frame by calculating an average value of data corresponding to the value of the image is disclosed. However, in such a configuration, image data of a good interpolation frame is not necessarily generated. In some cases, noise may be included in the generated image data. That is, in a method of detecting a motion vector for each block, such as the block matching method, the detected motion vector may not be appropriate for each pixel, and noise is included in the image data of the interpolation frame. Sometimes.

  As a method for solving such a problem, a motion vector is detected in a plurality of types of blocks, and the block size is adaptively switched according to the local nature of the screen. A method of obtaining a motion vector is also conceivable (see, for example, Patent Document 3 and Patent Document 4). However, with this method, the scale of a circuit for detecting a motion vector becomes large and real-time processing becomes difficult, so it is not practical to use this method for frame interpolation of moving images.

  Therefore, in the present invention, an image processing apparatus that generates image data of an interpolation frame with less noise based on a motion vector detected for each block of one type of size, and a moving picture quality using such an image processing apparatus It is an object of the present invention to provide a display device with improved performance.

In the first invention, a motion vector representing a motion of an image between a first frame and a second frame, which are adjacent frames constituting a moving image, is detected for each block including a predetermined number of pixels, and the first An image processing device that generates moving image data in which the number of frames is increased by generating image data of an interpolation frame to be inserted between a frame and the second frame based on the motion vector,
Based on the first block in the first frame and the second block in the second frame associated with the motion vector as blocks shifted in position according to the motion vector, the first and second Interpolation frame generation means for generating image data of the interpolation frame by calculating a pixel value of a third block which is a block in the interpolation frame to be associated with the block;
An interpolation frame for generating the moving image data having an increased number of frames by inserting the interpolation frame between the first frame and the second frame based on the image data generated by the interpolation frame generation means An insertion means,
The interpolation frame generation means includes
The value of the interpolation pixel, which is a pixel of the third block, is set to the value of the first pixel, which is a pixel corresponding to the position of the interpolation pixel in the first block, and the position of the interpolation pixel in the second block. Based on the value of the second pixel that is a pixel corresponding to
If the difference between the value of the first pixel and the value of the second pixel is greater than a predetermined threshold value, a third pixel corresponding to the interpolation pixel in the first and second frames. In addition, the value of the interpolation pixel is calculated based on the value of the fourth pixel.

According to a second invention, in the first invention,
The interpolation frame generation means includes
When the difference between the value of the first pixel and the value of the second pixel is equal to or less than the threshold value, an average value of the value of the first pixel and the value of the second pixel is used as the value of the interpolation pixel. Calculate
When the difference between the value of the first pixel and the value of the second pixel is larger than the threshold value, the value of the first pixel, the value of the second pixel, the value of the third pixel, and the value of the fourth pixel An average value of the pixel value is calculated as the value of the interpolated pixel.

According to a third invention, in the first invention,
The interpolation frame generation means includes
When the difference between the value of the first pixel and the value of the second pixel is equal to or less than the threshold value, an average value of the value of the first pixel and the value of the second pixel is used as the value of the interpolation pixel. Calculate
When the difference between the value of the first pixel and the value of the second pixel is larger than the threshold value, an average value of the value of the third pixel and the value of the fourth pixel is used as the value of the interpolation pixel. It is characterized by calculating.

According to a fourth invention, in any one of the first to third inventions,
The interpolation frame generation means generates the image data of the interpolation frame by using, as the third block, a block whose position is shifted from the first or second block by a vector amount ½ of the motion vector. To do.

The fifth invention is a display device,
An image processing apparatus according to any one of the first to fourth inventions,
The moving image is displayed based on moving image data generated by the image processing apparatus.

A sixth invention is a display device according to the fifth invention,
This is an active matrix liquid crystal display device.

According to a seventh aspect of the present invention, a motion vector representing a motion of an image between a first frame and a second frame, which are adjacent frames constituting a moving image, is detected for each block composed of a predetermined number of pixels. An image processing method for generating image data of an interpolation frame to be inserted between a frame and the second frame based on the motion vector,
Based on the first block in the first frame and the second block in the second frame associated with the motion vector as blocks shifted in position according to the motion vector, the first and second An interpolation frame generation step of calculating a pixel value of a third block that is a block in the interpolation frame to be associated with a block;
The value of the interpolation pixel that is the pixel of the third block is the value of the first pixel that is a position corresponding to the interpolation pixel in the first block and the value of the interpolation pixel in the second block. Is calculated based on the value of the second pixel that is a pixel corresponding to
If the difference between the value of the first pixel and the value of the second pixel is greater than a predetermined threshold value, a third pixel corresponding to the interpolation pixel in the first and second frames. The value of the interpolation pixel is calculated based on the value of the fourth pixel.

An eighth invention is an image processing method according to the seventh invention,
The interpolation frame generation step includes:
Determining whether the difference between the value of the first pixel and the value of the second pixel is greater than the threshold;
Calculating an average value of the value of the first pixel and the value of the second pixel as the value of the interpolation pixel when the difference is equal to or less than the threshold;
When the difference is larger than the threshold value, an average value of the value of the first pixel, the value of the second pixel, the value of the third pixel, and the value of the fourth pixel is calculated as the value of the interpolation pixel. And the step of performing.

  According to the image processing apparatus according to the first to fourth inventions as described above or the image processing method according to the seventh or eighth invention, for each block of one kind size (for each block composed of a predetermined number of pixels). Based on the first block in the first frame and the second block in the second frame that are associated with the detected motion vector, the value of the interpolation pixel that is the pixel of the interpolation frame to be inserted is calculated as follows: Is done. That is, the difference between the value of the first pixel that corresponds to the interpolation pixel in the first block and the value of the second pixel that corresponds to the interpolation pixel in the second block is greater than a predetermined threshold value. If it is larger, the values of the third and fourth pixels corresponding to the interpolation pixels in the first and second frames are calculated so as to be reflected in the values of the interpolation pixels. Therefore, while suppressing an increase in circuit scale, which becomes a problem when detecting motion vectors for a plurality of types of blocks, there is little noise even if there are blocks containing pixels that do not move like background image pixels. Interpolated frame image data can be generated.

  According to the display device according to the fifth or sixth invention as described above, the pixel of the background image is suppressed while suppressing an increase in the circuit scale that becomes a problem when detecting motion vectors for blocks of a plurality of types of sizes. Thus, even if there is a block including a non-moving pixel, image data of an interpolation frame with less noise is generated, and a moving image is displayed based on the image data whose frame frequency is increased by inserting this interpolation frame. As a result, display with high quality of moving images becomes possible. In particular, in an active matrix liquid crystal display device that is a hold-type display device, the present invention provides an effect that blurring of moving images due to line-of-sight tracking can be improved without causing noticeable flicker.

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.
<1. Overall configuration and operation>
First, the overall configuration and operation of an active matrix liquid crystal display device including an image processing device according to an embodiment of the present invention will be described. FIG. 1A is a block diagram showing the configuration of the liquid crystal display device. This liquid crystal display device includes a data driver 300 as a video signal line driving circuit, a gate driver 400 as a scanning signal line driving circuit, a liquid crystal panel 600 as an active matrix display unit, a data driver 300 and a gate driver 400. And a display control circuit 100 for controlling the display.

  The liquid crystal panel 600 according to the present embodiment includes a plurality of (m) scanning signal lines, a plurality of (n) video signal lines as data lines that intersect with each of the gate lines, A plurality of (m × n) pixel forming portions provided corresponding to the intersections of the gate lines and the data lines. These pixel formation portions are arranged in a matrix to form a pixel array. Each pixel formation portion has a gate terminal connected to a gate line passing through a corresponding intersection and a source terminal connected to a data line passing through the intersection. TFTs that are switching elements connected to each other, a pixel electrode connected to the drain terminal of the TFT, a common electrode that is a counter electrode provided in common to the plurality of pixel formation portions, and a plurality of pixel formations And a liquid crystal layer sandwiched between the pixel electrode and the common electrode. If necessary, an auxiliary capacitor is added in parallel to the capacitor formed by the pixel electrode and the common electrode. A pixel capacitance is constituted by a capacitance formed by the pixel electrode and the common electrode (a capacitance obtained by adding an auxiliary capacitance to the auxiliary capacitance if added).

  The display control circuit 100 receives, from an external signal source or the like, a video signal representing a moving image composed of continuous frames, that is, a digital video signal Dv representing an image to be displayed, and a horizontal synchronization signal corresponding to the digital video signal Dv. A data driver start pulse signal SSP is received as a signal for receiving the HSY and the vertical synchronization signal VSY and displaying an image represented by the digital video signal Dv on the liquid crystal panel 600 based on the signals Dv, HSY, and VSY. Data driver clock signal SCK, digital image signal DA representing the image to be displayed (signal corresponding to video signal Dv), gate driver start pulse signal GSP, and gate driver clock signal GCK are generated and output. To do.

  As shown in FIG. 1B, the display control circuit 100 includes an image processing device 110 and a timing generation circuit 150. The image processing apparatus 110 generates an image signal with an increased frame frequency as a digital image signal DA by generating image data of an interpolation frame to be inserted between frames constituting the moving image represented by the digital video signal Dv. (Details will be described later). The digital image signal DA is output in synchronization with the following start pulse signals SSP and GSP and clock signals SCK and GCK as timing signals for driving the liquid crystal panel 600. The timing generation circuit 150 generates a data driver clock signal SCK as a signal composed of pulses corresponding to each pixel of the image represented by the digital image signal DA, and for each horizontal scanning period in the image display based on the horizontal synchronization signal HSY. A data driver start pulse signal SSP is generated as a signal that is at a high level (H level) for a predetermined period, and is set to an H level for a predetermined period every one frame period (one vertical scanning period) in image display based on the vertical synchronization signal VSY. As a signal, a gate driver start pulse signal GSP is generated, and a gate driver clock signal GCK is generated based on the horizontal synchronization signal HSY.

  Of the signals generated in the display control circuit 100 as described above, the digital image signal DA, the data driver start pulse signal SSP, and the clock signal SCK are input to the data driver 300 and the gate driver start pulse. The signal GSP and the clock signal GCK are input to the gate driver 400. Based on the digital image signal DA, the start pulse signal SSP for the data driver, and the clock signal SCK, the data driver 300 uses a plurality (n) of analog voltages corresponding to pixel values in each horizontal scanning line of the image represented by the digital image signal DA. Data signals are generated every horizontal scanning period, and these data signals are applied to a plurality (n) of data lines, respectively. The gate driver 400 receives the gate driver start pulse signal GSP and the gate driver clock signal GCK from the display control circuit 100, and each frame period (each vertical scanning period) of the digital image signal DA based on these signals GSP and GCK. ), A plurality (m) of gate lines are sequentially selected for each horizontal scanning period, and an active gate signal (voltage for turning on the TFT of the pixel formation portion) is applied to the selected gate lines.

  In the liquid crystal panel 600, n data signals are applied to n data lines and m gate signals are applied to m gate lines by the data driver 300 and the gate driver 400, respectively. . Thereby, the voltage corresponding to the value of the corresponding pixel in the image represented by the digital image signal DA is held in the pixel capacitance of each pixel formation portion in the liquid crystal panel 600 by the n data signals, and is stored in the liquid crystal layer. A voltage corresponding to the potential difference between the pixel electrode and the common electrode is applied according to the digital image signal DA. That is, the voltage held in each pixel capacitor becomes the voltage applied to the corresponding pixel liquid crystal. The liquid crystal panel 600 displays the image represented by the digital image signal DA, that is, the image represented by the digital video signal Dv received from an external signal source or the like, by controlling the light transmittance of the liquid crystal layer by this applied voltage. Since the image data of the interpolation frame is generated by the image processing device 110 in the display control circuit 100, the frame frequency of the image displayed on the liquid crystal panel 600 is the frame frequency of the image represented by the digital video signal Dv from the outside. Has increased.

<2. Configuration and operation of image processing apparatus>
FIG. 2 is a block diagram illustrating a configuration of the image processing apparatus 110 according to the present embodiment, that is, the image processing apparatus 110 that generates the image data of the interpolation frame in the display control circuit 100. The image processing apparatus 110 includes a frame memory 10, a motion detection circuit 12, a double speed circuit 14, and an interpolation image processing circuit 16, and an external digital video signal Dv is received from the frame memory 10 and the motion detection circuit. 12 is input. In the following, it is assumed that the frame frequency of the external digital video signal Dv is 60 [frames / second], but the present invention is not limited to this. The frame memory 10 stores the image data for one frame indicated by the digital video signal Dv, and the stored image data is read out from the frame memory 10 by 60 frames per second, so that one frame period (1 A digital video signal (hereinafter referred to as “delayed video signal”) D60 delayed by a vertical scanning period) is output.

  The motion detection circuit 12 uses the digital video signal Dv and a delayed video signal D60 obtained by delaying the digital video signal Dv by one frame period, and performs image matching between adjacent frames in an image represented by the digital video signal Dv by a block matching method. A motion vector indicating motion is detected. As shown in FIG. 4, in the block matching method, an image of each frame is divided into a plurality of blocks with an area composed of a predetermined number of pixels as one block, and attention is paid to each block in the image of the current frame F (n). With the block as the reference block, the highest degree of similarity within a predetermined range centered on the block corresponding to the position of the reference block in the previous frame F (n−1) (for example, the difference between corresponding pixel values in both blocks ( Search for the block that minimizes the sum of the absolute values. Then, a positional shift between the reference block in the current frame F (n) and the block with the highest similarity obtained in the previous frame F (n−1) is detected as a motion vector Mv. The motion vector Mv detected for each block in this way is output from the motion detection circuit 12 as a motion vector signal M60 for 60 frames per second.

  Thus, the image data output 60 frames per second as the delayed video signal D60 from the frame memory 10 and the motion vector data output 60 frames per second as the motion vector signal M60 from the motion detection circuit 12 are double speed. Input to the circuit 14.

  As illustrated in FIG. 3, the double speed circuit 14 includes a frame memory 50, a motion vector memory 52, and a timing generation circuit 54. The timing generation circuit 54 is an address generation circuit that generates addresses for writing and reading with respect to the frame memory 50 and the motion vector memory 52, and generates addresses at a speed corresponding to 60 frames per second when writing. When reading, an address is generated at a speed corresponding to 120 frames per second, that is, a double speed. As a result, the delayed video signal D60 is written as image data for 60 frames per second in the frame memory 50, and the image data is read out from the frame memory 50 as the video signal D120 at a speed corresponding to 120 frames per second. However, this video signal D120 is intermittently read out from the frame memory 50 every other frame period as shown in FIG. Similarly, a motion vector signal M60 is written as motion vector data for 60 frames per second in the motion vector memory 52, and the motion vector data is transferred from the motion vector memory 52 as a motion vector signal M120 at a speed corresponding to 120 frames per second. Read out.

  The video signal D120 and the motion vector signal M120 thus obtained by the double speed circuit 14 are input to the interpolated image processing circuit 16. In the interpolated image processing circuit 16, image data of an interpolated frame is generated from the video signal D120 based on the motion vector signal D120. In general, for each block in the previous frame or the current frame, the block is moved by a vector amount obtained by multiplying the motion vector of the block by 1/2, so that the previous frame F (n−1) and the current frame F (n) Image data of one interpolation frame to be inserted between them can be generated. Further, by using a vector obtained by multiplying the motion vector of the block by 1/3 and a vector by 2/3 for each block in the previous frame or the current frame, two interpolation frames can be inserted between adjacent frames ( However, in this case, instead of the double speed circuit 14 that doubles the speed of the video signal D60 and the motion vector signal M60, a triple speed circuit that triples these speeds is required. More generally, a motion vector is multiplied by a coefficient corresponding to the number of interpolation frames to be inserted between adjacent frames to obtain a number of vectors corresponding to the number of interpolation frames, and a plurality of interpolation frames are obtained using these vectors. Can be inserted.

  According to the above method for generating the image data of the interpolation frame, there is no problem when all the pixels in one block in the previous frame have moved to the same position in the current frame, but as shown in FIG. When there is a non-moving pixel such as a background image pixel in the block, the non-moving pixel is moved to generate image data of the interpolation frame. In FIG. 5, the symbol Bc indicates the reference block of the current frame, and the symbol Bm indicates the block most similar to the reference block Bc in a predetermined range in the previous frame.

  On the other hand, one of the two consecutive frames corresponds to data corresponding to 1/2 of the motion vector for each block, and the other of the two corresponding frames corresponds to 1/2 of the motion vector. An image processing device that creates image data of an interpolated frame by calculating an average value of the data to be calculated (see Patent Document 2), or a motion vector is detected by a plurality of types of size blocks to match the local characteristics of the screen A method (for example, Patent Document 3) in which a more appropriate motion vector is obtained in units of pixels by adaptively switching the block size has been proposed. However, as described above, the former image processing apparatus cannot always create good data without noise as the image data of the interpolation frame, and the latter method is used to detect a motion vector. The scale of the circuit becomes large and real-time processing becomes difficult.

  Therefore, the interpolated image processing circuit 16 in this embodiment generates image data of the interpolated frame as described below in order to solve such a problem. Now, as shown in FIG. 6, based on the motion vector detected for a certain block Bb of the current frame F (n), it is inserted between the previous frame F (n−1) and the current frame F (n). Consider a process of calculating the value of the pixel Pe constituting the block Be of the power interpolation frame F (i−1). Here, a block in the previous frame F (n−1) whose position is shifted from the block Bb of the current frame F (n) by the motion vector is defined as a block Ba, and a vector amount that is ½ of the motion vector from the block Bb. A block in the interpolation frame F (i−1) whose position is shifted by a distance is defined as a block Be. That is, the motion vector detected for the block Bb in the current frame F (n) corresponds to the block Bb, the block Ba in the previous frame F (n−1), and the block Be in the interpolated frame F (i−1). Shall be attached. In this case, the interpolated image processing circuit 16 calculates the value of the pixel (hereinafter referred to as “interpolated pixel”) in the block Be of the interpolation frame F (i−1) to be generated as follows.

First, a value of a pixel Pa (a pixel having the same relative position in the block) Pa corresponding in position to the interpolation pixel Pe in the block Be of the interpolation frame F (i-1) in the block Ba of the previous frame F (n-1). A difference | Va−Vb | between Va and a value Vb of a pixel (a pixel having the same relative position in the block) Pb corresponding to the interpolation pixel Pe in the current frame F (n) in the block Bb It is determined whether or not the difference | Va−Vb | is greater than a predetermined threshold value ε. As a result, when | Va−Vb | ≦ ε, the average value of the value Va of the pixel Pa and the value Vb of the pixel Pb is set as the value Ve of the interpolation pixel Pe. That is,
Ve = (Va + Vb) / 2 (1)
And On the other hand, if | Va−Vb |> ε, pixels corresponding in position to the interpolation pixel Pe in the interpolation frame F (i−1) in the previous frame F (n−1) (positions in the frame are the same). The value Ve of the interpolation pixel Pe is calculated using the value Vc of the pixel) Pc and the value Vd of the pixel (a pixel having the same position in the frame) Pd that corresponds to the interpolation pixel Pe in the current frame F (n). To do. Specifically, an average value of the value Va of the pixel Pa, the value Vb of the pixel Pb, the value Vc of the pixel Pc, and the value Vd of the pixel Pd is set as the value Ve of the interpolation pixel Pe. That is,
Ve = (Va + Vb + Vc + Vd) / 4 (2)
And

  In this way, when calculating the value Ve of the pixel (interpolation pixel) Pe of the block Be of the interpolation frame to be generated, the previous frame F (n−1) associated with the block Be by the detected motion vector. The difference | Va−Vb | between the values of the pixels Pa corresponding to the interpolation pixel Pe (hereinafter referred to as “in-block position corresponding pixel”) Pa and Pb in the block Ba of the current frame F and the block Bb of the current frame F (n) When this difference | Va−Vb | is larger than a predetermined threshold ε, not only the values of the intra-block position corresponding pixels Pa and Pb but also the previous frame F (n−1) and the current frame F (n). The value Ve of the interpolation pixel Pe is determined also using the values Vc and Vd of the pixels (hereinafter referred to as “in-frame position corresponding pixels”) Pc and Pd corresponding to the interpolation pixel Pe. Accordingly, even when the detected motion vector Mv is not appropriate when viewed in pixel units, such as when there is a non-moving pixel such as a pixel in the background image in the block, the interpolation frame F (i−1) The image data with less noise can be generated as the image data, and the image represented by the image signal DA obtained by the interpolated image processing circuit 16 has no sense of incongruity. Note that the threshold value ε is a value that should be set as appropriate based on simulations and experiments (such as subjective evaluation experiments). For example, the threshold value ε is a threshold value gradation when an image represented by the video signal Dv is displayed in 12 gradations. The corresponding value may be the above ε.

  FIG. 7 is a block diagram illustrating a configuration example of the interpolation image processing circuit 16 that generates the image data of the interpolation frame by calculating the value Ve of the interpolation pixel Pe as described above. FIG. 8 is a block diagram illustrating this configuration. It is a timing chart for demonstrating operation | movement of the interpolation image processing circuit 16 by an example. Hereinafter, the configuration and operation of the interpolation image processing circuit 16 will be described with reference to FIGS. 7 and 8. In the following, one frame period is 1/120 [second].

  As shown in FIG. 7, the interpolated image processing circuit 16 according to this configuration example includes a first frame memory 111, a second frame memory 112, an address generation circuit 114, a comparator 116, a first adder 118, A first coefficient multiplier 120, a second adder 122, a second coefficient multiplier 124, a first changeover switch 126, and a second changeover switch 128 are provided, and the video signal D120 (from the double speed circuit 14 ( (Hereinafter also referred to as “first video signal Dv1”) and motion vector signal M120 are input to first frame memory 111 and address generation circuit 114, respectively.

  The address generation circuit 114 generates addresses for writing and reading with respect to the first and second frame memories 111 and 112, and thereby operates the first and second frame memories 111 and 112 as follows. That is, as shown in FIGS. 8A and 8B, in each of the odd-numbered frame periods T1, T3, T5,..., The first video signal Dv1 (the video signal D120 from the double speed circuit 14) corresponds to one frame. The image data is written to the first frame memory 111 as image data, and the image data for one frame written before that is read from the first frame memory 111 as the second video signal Dv2. At this time, in the second frame memory 112, the second video signal Dv2 read from the first frame memory 111 is written as image data for one frame. The second video signal Dv2 is input to the second changeover switch 128.

  On the other hand, in each of the even-numbered frame periods T2, T4, T6,..., Writing to the first and second frame memories 111 and 112 is not performed, and the value Ve of the pixel (interpolation pixel) Pe of the interpolation frame to be generated. The values Va and Vb of the in-block position corresponding pixels Pa and Pb and the in-frame position corresponding pixels Pc and Pd used for the calculation of the above-described positions Vc and Vd are read out. For example, in the frame period T4, the value Va of the intra-block position corresponding pixel Pa and the value Vc of the intra-frame position corresponding pixel Pc in the frame F (n−1) are read from the second frame memory 112 (FIGS. 6 and 8). (See (b)), the value Vb of the intra-block position corresponding pixel Pb and the value Vd of the intra-frame position corresponding pixel Pd in the frame F (n) are read from the first frame memory 111 (see FIGS. 6 and 8A). ). Now, assuming that the image data stored in the first frame memory 111 in each of the even-numbered frame periods T2, T4, T6,... Is the image data of the current frame, the second frame memory 112 stores the previous frame in this period. In this period, the value Va of the intra-block position corresponding pixel Pa and the value Vc of the intra-frame position corresponding pixel Pc of the previous frame are read from the second frame memory 112 during the first frame. The value Vb of the intra-block position corresponding pixel Pb and the value Vd of the intra-frame position corresponding pixel Pd of the current frame are read from the memory 111.

  The values Va and Vb of the in-block position corresponding pixels of the previous frame and the current frame read out as described above are input to the comparator 116, the first adder 118, and the second adder 122, and the previous frame and the current frame are input. The values Vc and Vd of the in-frame position corresponding pixels of the frame are input to the second adder 122. In each of the even-numbered frame periods T2, T4, T6,..., A third video signal (hereinafter also referred to as “interpolated video signal”) representing image data of an interpolation frame to be inserted between the previous frame and the current frame. ) Dv3 is generated as follows.

  The comparator 116 determines whether or not the difference | Va−Vb | between the value Va of the in-block position corresponding pixel in the previous frame and the value Vb of the in-block position corresponding pixel in the current frame is greater than the threshold value ε. A determination result signal Sc is generated according to the result. That is, if | Va−Vb | ≦ ε, an L level (low level) signal is output as the determination result signal Sc, and if | Va−Vb |> ε, the determination result signal Sc is at the H level (high level). The signal is output. This determination result signal Sc is input to the first changeover switch 126 as a control signal.

  On the other hand, the first adder 118 adds the value Va of the intra-block position corresponding pixel of the previous frame and the value Vb of the intra-block position corresponding pixel of the current frame, and the first coefficient multiplier 120 adds the addition result Va + Vb. A signal indicating the multiplication result, that is, a signal indicating the average value of the two pixel values Va and Vb is output as the first interpolation video signal Dv31 by multiplying by 1/2. In parallel with this, the second adder 122 corresponds to the value Va of the intra-block position corresponding pixel in the previous frame and the value Vc of the intra-frame position corresponding pixel, the value Vb of the intra-block position corresponding pixel of the current frame, and the intra-frame position. The pixel value Vd is added, and the second coefficient multiplier 124 multiplies the addition result Va + Vb + Vc + Vd by 1/4, and a signal indicating the multiplication result, that is, an average value of the four pixel values Va, Vb, Vc, Vd. Is output as the second interpolated video signal Dv32.

  The first and second interpolated video signals Dv31 and Dv32 generated as described above are input to the first changeover switch 126. When the determination result signal Sc from the comparator 116 is at L level, the first changeover switch 126 selects the first interpolation video signal Dv31 and outputs it as the third video signal (interpolation video signal) Dv3, and the determination result signal Sc. When H is at the H level, the second interpolation video signal Dv32 is selected and output as the third video signal (interpolation video signal) Dv3. The interpolated video signal Dv3 is input to the second changeover switch 128.

  The second changeover switch 128 is a switch for switching the signal DA to be output for each frame period. As shown in FIGS. 8B to 8D, in the odd-numbered frame periods T1, T3, T5,. The second video signal Dv2 indicating the image data of the previous frame is selected and output as the digital image signal DA. In the even-numbered frame periods T2, T4, T6,..., The interpolation video signal Dv3 indicating the image data of the interpolation frame is selected. Select and output as a digital image signal DA.

  As can be seen from the above description, in this configuration example, the interpolation frame generating means for generating the image data (interpolated video signal Dv3) of the interpolation frame to be inserted between the adjacent frames of the image represented by the external digital video signal Dv is provided. The first and second frame memories 111 and 112, the address generation circuit 114, the comparator 116, the first and second adders 118 and 122, the first and second coefficient multipliers 120 and 124, and the first changeover switch 126, The second changeover switch 128 implements an interpolation frame insertion means for generating moving image data (digital image signal DA) having a frame frequency doubled by inserting the interpolation frame between two adjacent frames. The

  In this way, image data of an interpolation frame to be inserted between two adjacent frames constituting an image (moving image) represented by the external digital video signal Dv is generated, and a frame is generated using the image data of the interpolation frame. Image data whose frequency is increased (doubled) is output from the interpolated image processing circuit 16 as a digital image signal DA. As described above, the digital image signal DA is sent to the data driver 300, and the data driver 300 drives the data lines in the liquid crystal panel 600 based on the digital image signal DA. On the other hand, the gate driver 400 drives the gate line in the liquid crystal panel 600 based on the start pulse signal GSP and the clock signal GCK which are timing signals corresponding to the digital image signal DA. As a result, an image represented by the digital image signal DA is displayed on the liquid crystal panel 600. That is, in an active matrix liquid crystal display device that is a hold-type display device including the image processing device according to the present embodiment, an image (moving image) represented by an external digital video signal Dv is doubled in frame frequency. Is displayed.

<3. Effect>
As described above, in the present embodiment, the image data of the interpolation frame is generated as the interpolation video signal Dv3 based on the motion vector detected for each fixed-size block, that is, for each block of one type (FIGS. 7 and 8). c)) At this time, the difference | Va−Vb between the value Va of the intra-block position corresponding pixel of the previous frame and the value Vb of the intra-block position corresponding pixel of the current frame with respect to the interpolated pixel Pe which is a pixel in the interpolation frame to be generated Whether or not | is larger than a predetermined threshold ε is determined. As a result, when the difference | Va−Vb | is equal to or less than the threshold value ε, the average value of the values Va and Vb of the intra-block position corresponding pixels in the previous frame and the current frame becomes the value Ve of the interpolation pixel Pe. However, if this difference | Va−Vb | is larger than a predetermined threshold value ε, not only the values Va and Vb of the in-block position corresponding pixels, but also the values Vc, of the in-frame position corresponding pixels in the previous frame and the current frame. The value Ve of the interpolation pixel Pe is determined in consideration of Vd as well. That is, Ve = (Va + Vb + Vc + Vd) / 4.

  Therefore, according to the present embodiment, while suppressing an increase in circuit scale that causes a problem when detecting motion vectors for a plurality of types of blocks, a block including pixels with no motion such as pixels of a background image can be obtained. Even if it exists, the image data of the interpolation frame with little noise can be generated. According to such a configuration, the digital image signal DA indicating the moving image in which the interpolation frame is inserted is generated in real time, and the moving image whose frame frequency is doubled based on the digital image signal DA is the active matrix type. It is displayed on the liquid crystal display device. Thereby, in an active matrix liquid crystal display device which is a hold type display device, blur due to line-of-sight tracking can be improved without impairing its features (for example, without causing noticeable flicker).

<4. Modification>
In the above embodiment, the motion detection circuit 12 for detecting a motion vector is built in the image processing apparatus 110 (FIG. 1B), but the motion detection circuit 12 may be provided outside the image processing apparatus 110. Good. When a motion vector in an image represented by the digital video signal Dv is given from outside the liquid crystal display device including the image processing device 110 (for example, when a motion vector based on the MPEG (Moving Picture Expert Group) standard is given). In this case, it is sufficient to generate image data of an interpolation frame using this, and in this case, means for detecting a motion vector in the liquid crystal display device and the image processing device 110 becomes unnecessary.

  In the above embodiment, the value Ve of the interpolation pixel Pe is calculated by the equation (1) or the equation (2) when generating the image data of the interpolation frame. Instead, the value Ve of the interpolation pixel Pe may be determined by a calculation for obtaining a weighted average. Further, if the difference | Va−Vb | between the in-block position corresponding pixels Pa and Pb is larger than a predetermined threshold ε, the in-block position corresponding pixels Pa and Pb are regarded as non-moving pixels, and the in-frame position. The average value (Vc + Vd) / 2 of the values of the corresponding pixels Pc and Pd may be set as the value Ve of the interpolation pixel Pe.

  Furthermore, in the above embodiment, one interpolation frame is inserted between adjacent frames. However, as described above, two or more interpolation frames are inserted between adjacent frames. However, the present invention is applicable. In this case, for example, a weighted average is calculated for the intra-block position corresponding pixel values Va and Vb and the intra-frame position corresponding pixel values Vc and Vd in accordance with the relative positions (positions on the time axis) at which each interpolation frame is inserted. By doing so, the value Ve of the interpolation pixel Pe may be determined.

  In the above embodiment, the digital image signal DA with an increased frame frequency (display frequency) is generated by the image processing device 110, and the image represented by the image signal DA is displayed on the active matrix liquid crystal display device. The image represented by the digital image signal DA may be displayed on a hold type display device other than the active matrix type liquid crystal display device. In this case, blurring due to line-of-sight tracking can be improved without causing noticeable flicker. The effect of doing is obtained. Further, even when the image represented by the digital image signal DA is displayed on a non-hold type display device, it is possible to increase the frame frequency with a simple configuration and improve the moving image quality.

1 is a block diagram illustrating an overall configuration of an active matrix liquid crystal display device including an image processing device according to an embodiment of the present invention. It is a block diagram which shows the structure of the image processing apparatus which concerns on the said embodiment. It is a block diagram which shows the structure of the double speed circuit in the said embodiment. It is a figure for demonstrating the detection of the motion vector by a block matching method. It is a figure for demonstrating the problem in the detection of the motion vector by a block matching method. It is a figure for demonstrating the production | generation process of the image data of the interpolation frame in the said embodiment. It is a block diagram which shows the example of 1 structure of the interpolation image processing circuit in the said embodiment. It is a timing chart for demonstrating operation | movement of the interpolation image processing circuit in the said embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10,50 ... Frame memory 12 ... Motion detection circuit 14 ... Double speed circuit 16 ... Interpolation image processing circuit 52 ... Motion vector memory 100 ... Display control circuit 110 ... Image processing apparatus 111, 112 ... Frame memory 114 ... Address generation circuit 116 ... Comparison Unit 118, 122 ... Adder 120, 124 ... Coefficient multiplier 126, 128 ... Changeover switch 150 ... Timing generation circuit 300 ... Data driver (video signal line drive circuit)
400: Gate driver (scanning signal line driving circuit)
600 ... Liquid crystal panel Dv ... Digital video signal (from outside) DA ... Digital image signal D60, D120 ... Digital video signal M60, M120 ... Motion vector signal Dv3 ... Third digital video signal (interpolated video signal)
Sc ... Determination result signal Pa, Pb ... In-block position corresponding pixel Pc, Pd ... In-frame position corresponding pixel Pe ... Interpolated pixel Va, Vb ... In-block position corresponding pixel value Vc, Vd ... In-frame position corresponding pixel value

Claims (8)

A motion vector representing a motion of an image between a first frame and a second frame that are adjacent frames constituting a moving image is detected for each block including a predetermined number of pixels, and the first frame and the second frame are detected. An image processing device that generates moving image data in which the number of frames is increased by generating image data of an interpolation frame to be inserted between and based on the motion vector,
Based on the first block in the first frame and the second block in the second frame associated with the motion vector as blocks shifted in position according to the motion vector, the first and second Interpolation frame generation means for generating image data of the interpolation frame by calculating a pixel value of a third block which is a block in the interpolation frame to be associated with the block;
An interpolation frame for generating the moving image data having an increased number of frames by inserting the interpolation frame between the first frame and the second frame based on the image data generated by the interpolation frame generation means An insertion means,
The interpolation frame generation means includes
The value of the interpolation pixel, which is a pixel of the third block, is set to the value of the first pixel, which is a pixel corresponding to the position of the interpolation pixel in the first block, and the position of the interpolation pixel in the second block. Based on the value of the second pixel that is a pixel corresponding to
If the difference between the value of the first pixel and the value of the second pixel is greater than a predetermined threshold value, a third pixel corresponding to the interpolation pixel in the first and second frames. An image processing apparatus that calculates a value of the interpolation pixel based on a value of the fourth pixel. The interpolation frame generation means includes
When the difference between the value of the first pixel and the value of the second pixel is equal to or less than the threshold value, an average value of the value of the first pixel and the value of the second pixel is used as the value of the interpolation pixel. Calculate
When the difference between the value of the first pixel and the value of the second pixel is larger than the threshold value, the value of the first pixel, the value of the second pixel, the value of the third pixel, and the fourth value The image processing apparatus according to claim 1, wherein an average value of pixel values is calculated as the value of the interpolation pixel. The interpolation frame generation means includes
When the difference between the value of the first pixel and the value of the second pixel is equal to or less than the threshold value, an average value of the value of the first pixel and the value of the second pixel is used as the value of the interpolation pixel. Calculate
When the difference between the value of the first pixel and the value of the second pixel is larger than the threshold value, an average value of the value of the third pixel and the value of the fourth pixel is used as the value of the interpolation pixel. The image processing apparatus according to claim 1, wherein the image processing apparatus calculates the image processing apparatus.   The interpolation frame generation means generates the image data of the interpolation frame by using, as the third block, a block whose position is shifted from the first or second block by a vector amount ½ of the motion vector. The image processing device according to any one of claims 1 to 3. An image processing apparatus according to any one of claims 1 to 4, comprising:
A display device that displays a moving image based on moving image data generated by the image processing device.   The display device according to claim 5, which is an active matrix liquid crystal display device. A motion vector representing a motion of an image between a first frame and a second frame that are adjacent frames constituting a moving image is detected for each block including a predetermined number of pixels, and the first frame and the second frame are detected. An image processing method for generating image data of an interpolation frame to be inserted between and based on the motion vector,
Based on the first block in the first frame and the second block in the second frame associated with the motion vector as blocks shifted in position according to the motion vector, the first and second An interpolation frame generation step of calculating a pixel value of a third block that is a block in the interpolation frame to be associated with a block;
The value of the interpolation pixel that is the pixel of the third block is the value of the first pixel that is a position corresponding to the interpolation pixel in the first block and the value of the interpolation pixel in the second block. Is calculated based on the value of the second pixel that is a pixel corresponding to
If the difference between the value of the first pixel and the value of the second pixel is greater than a predetermined threshold value, a third pixel corresponding to the interpolation pixel in the first and second frames. And an interpolated pixel value is calculated based on the value of the fourth pixel. The interpolation frame generation step includes:
Determining whether the difference between the value of the first pixel and the value of the second pixel is greater than the threshold;
Calculating an average value of the value of the first pixel and the value of the second pixel as the value of the interpolation pixel when the difference is equal to or less than the threshold;
When the difference is larger than the threshold value, an average value of the value of the first pixel, the value of the second pixel, the value of the third pixel, and the value of the fourth pixel is calculated as the value of the interpolation pixel. The image processing method according to claim 7, further comprising the step of:

Images