JP2009182865A - Image display device and method, image processor, and image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prevent degradation in image quality arising from the erroneous detection of a motion vector and to eliminate motion blur in a telop portion, in an image display device equipped with a frame rate conversion part. <P>SOLUTION: A motion vector detection section includes a telop information detecting unit for detecting a region where one or more telops exist and the moving speed of the telop, and supplies the information relating to the detected motion vector and the telop to an interpolation frame generating unit. A motion vector discrimination section uses a vector detected result of the motion vector detecting unit to count the number of the motion detection blocks wherein the detected motion vector exceeds a prescribed range, and outputs, to the interpolation frame generating unit, motion amount discrimination information indicating that the motion amount is large when the counted value exceeds a prescribed threshold value. When the motion amount discrimination information indicating a large motion amount is input, the interpolation frame generating unit performs interpolation frame generation with other generating means instead of carrying out the interpolation frame generation processing using a motion compensation. The telop part is subjected to the interpolation frame generation processing using the motion compensation. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image display apparatus and method having a function of converting a frame rate or a field rate, and in particular, an image display apparatus that prevents image quality deterioration of a telop portion, an image display method using the apparatus, an image processing apparatus, and the apparatus. The present invention relates to an image processing method.

  In contrast to a cathode ray tube (CRT) that has been mainly used for the purpose of embodying moving images, an LCD (Liquid Crystal Display) moves a viewer when a moving image is displayed. There is a so-called motion blur defect in which the outline of a part is blurred and perceived. It has been pointed out that this motion blur is caused by the LCD display method itself (see, for example, Patent Document 1 and Non-Patent Document 1).

  In a CRT that performs display by scanning an electron beam to emit light from a phosphor, the light emission of each pixel is substantially in an impulse shape although there is some afterglow of the phosphor. This is called an impulse type display system. On the other hand, in the LCD, the charge stored by applying an electric field to the liquid crystal is held at a relatively high rate until the next electric field is applied. In particular, in the case of the TFT method, a TFT switch is provided for each dot constituting a pixel, and an auxiliary capacitor is usually provided for each pixel, and the ability to hold stored charges is extremely high. For this reason, light emission continues until the pixel is rewritten by applying an electric field based on image information of the next frame or field (hereinafter referred to as a frame). This is called a hold type display method.

  In the hold-type display method as described above, the impulse response of the image display light has a temporal spread, so that the time frequency characteristic is deteriorated, and the spatial frequency characteristic is accordingly lowered, resulting in motion blur. In other words, since the human line of sight smoothly follows a moving object, if the light emission time is long as in the hold type, the movement of the image becomes jerky due to the time integration effect and looks unnatural.

  In order to improve motion blur in the hold-type display method, a technique for converting a frame rate (number of frames) by interpolating an image between frames is known. This technique is called FRC (Frame Rate Converter) and is put into practical use in liquid crystal display devices and the like.

  Conventionally, as a method for converting the frame rate, there are various methods such as simply repeatedly reading out the same frame a plurality of times and frame interpolation by linear interpolation between frames (for example, see Non-Patent Document 2). ). However, in the case of frame interpolation processing by linear interpolation, motion unnaturalness (jerkiness, judder) due to frame rate conversion occurs, and motion blur interference due to the hold type display method described above is sufficiently improved. The image quality was insufficient.

  Therefore, motion compensation processing using motion vectors has been proposed in order to eliminate the influence of the jerkiness and improve the moving image quality. According to this motion compensation processing, since the motion compensation is performed by capturing the moving image itself, it is possible to obtain a very natural moving image without degradation of resolution and without occurrence of jerkiness. Furthermore, since the interpolated image signal is formed by motion compensation, it is possible to sufficiently improve the motion blur interference caused by the hold type display method described above.

  Patent Document 1 described above discloses a technique for improving a decrease in spatial frequency characteristics that causes motion blur by increasing a frame frequency of a display image by generating an interpolation frame adaptively in motion. ing. In this method, at least one interpolated image signal to be interpolated between frames of a display image is formed in a motion adaptive manner from the preceding and following frames, and the formed interpolated image signal is interpolated between frames and sequentially displayed. ing.

  FIG. 19 is a block diagram showing a schematic configuration of an FRC drive display circuit in a conventional liquid crystal display device. In the figure, the FRC drive display circuit interpolates an image signal subjected to motion compensation processing between frames of an input image signal. An FRC unit 100 that converts the number of frames of the input image signal, an active matrix type liquid crystal display panel 103 having a liquid crystal layer and electrodes for applying a scanning signal and a data signal to the liquid crystal layer, and an FRC unit And an electrode driving unit 104 for driving the scanning electrodes and the data electrodes of the liquid crystal display panel 103 based on the image signal whose frame rate is converted by 100.

  The FRC unit 100 includes a motion vector detection unit 101 that detects motion vector information from an input image signal, an interpolation frame generation unit 102 that generates an interpolation frame based on the motion vector information obtained by the motion vector detection unit 101, and Is provided.

  In the above configuration, the motion vector detection unit 101 may obtain the motion vector information using, for example, a block matching method or a gradient method described later, or the motion vector information is included in some form in the input image signal. If this is the case, this may be used. For example, image data compression-encoded using the MPEG method includes motion vector information of a moving image calculated at the time of encoding, and the motion vector information may be acquired.

  FIG. 20 is a diagram for explaining frame rate conversion processing by the conventional FRC drive display circuit shown in FIG. The FRC unit 100 generates an interpolated frame (an image colored in gray in the figure) by motion compensation using the motion vector information output from the motion vector detecting unit 101, and the generated interpolation By sequentially outputting the frame signal together with the input frame signal, processing for converting the frame rate of the input image signal from, for example, 60 frames per second (60 Hz) to 120 frames per second (120 Hz) is performed.

  FIG. 21 is a diagram for explaining interpolation frame generation processing by the motion vector detection unit 101 and the interpolation frame generation unit 102. The motion vector detection unit 101 detects the motion vector 201 from the frame # 1 and the frame # 2 shown in FIG. That is, the motion vector detection unit 101 obtains the motion vector 201 by measuring how much and in which direction the frame # 1 and frame # 2 have moved in 1/60 second. Next, the interpolation frame generation unit 102 allocates the interpolation vector 202 between the frame # 1 and the frame # 2 using the obtained motion vector 201. An interpolation frame 203 is generated by moving the object (in this case, an automobile) from the position of frame # 1 to a position after 1/120 second based on this interpolation vector 202.

  In this way, motion compensation frame interpolation processing is performed using motion vector information, and the display frame frequency is increased to change the display state of the LCD (hold type display method) to the display state of the CRT (impulse type display method). It is possible to improve the image quality degradation due to motion blur that occurs when displaying a moving image.

  Here, in the motion compensation frame interpolation process, detection of a motion vector is indispensable for motion compensation. As this motion vector detection method, for example, the pattern matching described in “Television image motion detection method” disclosed in Patent Document 2, “Asymptotic detection method of image motion vector” described in Patent Document 3, and the like. Or the iterative gradient method described in the “image motion amount detection method” disclosed in Patent Document 4 and the “initial displacement method in motion estimation of moving images” disclosed in Patent Document 5, respectively. ing.

  In particular, the latter motion vector detection method based on the iterative gradient method is smaller and more accurate than the pattern matching method, and can detect a motion vector. That is, the motion vector detection method based on the iterative gradient method has a predetermined predetermined size of each frame of a digitized television signal, for example, m × n pixels including m pixels in the horizontal direction and n lines in the vertical direction. By subdividing into blocks, each block is subjected to repetitive gradient calculation based on the signal gradient in the screen and the physical correspondence of the signal difference value between the corresponding screens The amount of motion is estimated.

  By the way, a moving image has high correlation between frames and has continuity in the time axis direction. In many cases, a pixel or block that moves in a certain frame moves with the same amount of motion in a subsequent frame or a frame preceding it. For example, in the case of a moving image in which the ball rolls from right to left on the screen, the ball area moves with the same amount of movement in any frame. That is, there are many cases where motion vectors have continuity between consecutive frames.

  Therefore, by referring to the motion vector detection result in the previous frame, the motion vector detection in the next frame can be performed more easily or more accurately. In Patent Document 5, as an initial value when estimating the amount of motion, the detected block is selected from among motion vector candidates already detected in a plurality of peripheral blocks including the block corresponding to the detected block. By selecting the optimal one for motion vector detection as an initial displacement vector and starting gradient method calculation from a value close to the true motion vector of the detected block, the number of gradient method calculations is reduced, For example, a method for detecting a true motion vector by two gradient method computations has been proposed.

  In addition, in the “motion vector detection circuit” disclosed in Patent Document 6, in order to further improve the accuracy of motion vector detection, an initial displacement vector of motion between each block of an image signal separated by at least one field or one frame or more. A method for detecting the above has been proposed. Furthermore, in the block matching method, it is conceivable to perform efficient motion vector detection by changing the search order with reference to the motion vector detection result in the previous frame. As described above, when a motion vector is detected, by using the already detected motion vector, for example, real-time processing of frame rate conversion becomes possible.

  By the way, in television programs and movies, subtitles, so-called telops are often included in image signals. Among them, there is a telop in which characters scroll (move) horizontally or vertically on the screen. According to Non-Patent Document 3, the movement speed of a subject included in a general TV program is mainly distributed to 20 deg / sec or less, and in particular, the frequency of 10 deg / sec or less is high. It can be seen that the scroll speed is 13.8 deg / sec on average and 35.9 deg / sec at the maximum, and the telop appearance frequency of 10 to 20 deg / sec is high. That is, a scrolling telop often moves at a faster speed in a television program than a general subject.

Usually, a subject photographed by a camera includes a blur (camera blur) caused by the light accumulation time of the camera when the movement is fast. As described above, motion blur caused by the hold-type display method is less noticeable for an image originally including blur. On the other hand, since the telop is an image synthesized later, even if the movement is fast, camera blur is not included. For this reason, the motion blur caused by the hold-type display method is easily noticeable in the telop, and when the motion blur is reduced by the FRC, the effect appears remarkably.
Japanese Patent No. 3295437 Japanese Unexamined Patent Publication No. 55-162683 Japanese Patent Laid-Open No. 55-162684 JP 60-158786 A Japanese Patent Laid-Open No. 62-206980 Japanese Patent Laid-Open No. 06-217266 Japanese Patent No. 3562826 JP 2000-333134 A Shuichi Ishiguro, Taiichi Kurita, “Examination of video quality of hold light emission display by 8 × CRT”, IEICE Technical Report, The Institute of Electronics, Information and Communication Engineers, EID96-4 (1996-06), p. 19-26 Tatsuro Yamauchi, “Television Conversion”, Television Society Journal, Vol. 45, no. 12, pp. 1534-1543 (1991) Fujine, et. al. , “Real-Life In-Home Viewing Conditions for FPDs and Statistical Characteristics of Broadcast Video”, Digest AM-FPD'06

  By the way, in Patent Document 7, in an apparatus having a frame rate conversion process by motion compensation, a signal process by motion compensation is provided, and a sum of a plurality of motion vector lengths of one frame is a sum of a plurality of motion vector lengths of another frame. If there is a difference between the sum and a given threshold or more, it can be considered that there is an image signal generated from the movie film, and if the change in the sum vector length in the two frames is large, the movie film A description is given of selecting an applied processing mode (an image signal processing mode without motion compensation). However, when a scrolling telop is superimposed on such an image, more specifically, for example, in a movie aired on a television with a frame rate of 60 Hz, the movie body is an image obtained by pulling down a 24 Hz image by 2-3. When a telop that scrolls at 60 Hz is superimposed on it, according to the technique of Patent Document 7, since an image signal processing mode that does not compensate for motion is selected over the entire screen, motion blur in the telop portion is not eliminated. There is.

  Further, in Patent Document 8, in an apparatus equipped with a frame rate conversion process by motion compensation, scene changes (video contents change greatly), screen flushing (luminance changes greatly change), and image motions are determined very quickly in advance. When there is no optimal vector within the specified search range, the motion vector is fixed to 0 vector, and the generation of an interpolated image by detecting an incorrect motion vector is not performed to improve the image quality degradation based on the detection error How to do is described. However, in the same way as described above, when a scrolling telop is superimposed on such a video, according to the technique of Patent Document 8, the motion vector is fixed to 0 vector over the entire screen, that is, an image signal that is not motion-compensated is included. Since it is used as an inset image, there is a problem that motion blur in the telop part is not eliminated.

  In addition, 2-3 pull-down video, 2-2 pull-down video, video including scene change, and video including flashing, that is, there are cases where the correlation between frames is low in a certain two consecutive frames in the video. It is an existing image. As a similar video, there is, for example, a video that is played back slowly.

  In summary, videos that are prone to error in motion vector detection processing, for example, 2-3 pull-down video, 2-2 pull-down video, video including scene changes, video including flashing, video including slow playback, motion If a scrolling telop is superimposed on an image that includes a fast motion that exceeds the detection limit of the vector detection process, selecting an image signal processing mode that does not compensate for motion over the entire screen does not eliminate motion blur in the telop part. There are challenges.

  The present invention has been made in view of the above problems, and in a motion compensated frame rate conversion (FRC) process, a scrolling telop is superimposed on an image that makes motion vector detection difficult to operate normally. An image display apparatus and method, and an image processing apparatus and method capable of preventing image quality degradation caused by motion blur in a telop portion and preventing image quality degradation caused by motion blur in a telop portion For the purpose.

  The first invention of the present application interpolates an image signal subjected to motion compensation processing using a motion vector between frames or fields of the input image signal, thereby reducing the number of frames or fields of the input image signal. An image display device comprising rate conversion means for converting, a detection means for detecting a feature value of one or more telops included in the input image signal, and a motion amount between frames or fields of the input image signal Determining means for determining whether or not is greater than a predetermined value for each frame or field, and when the motion amount between frames or fields of the input image signal is greater than a predetermined value, the motion compensation is performed in the telop area. Interpolate the processed image signal and invalidate the motion compensation processing in other areas, or The by motion compensation process the interpolating an image signal not subjected and converting the number of number of frames or fields the input image signal.

  According to a second aspect of the present application, the amount of motion between frames or fields of the input image signal is determined based on a motion vector calculated by the determination unit performing a predetermined operation on the input image signal. It is characterized by determining whether it is larger than a predetermined value.

  According to a third aspect of the present invention, flag information is assigned to a block in which the calculated motion vector exceeds a predetermined range, and a count value for each frame or field of the flag information is compared with a predetermined threshold value. To determine whether the amount of motion between frames or fields of the input image signal is greater than a predetermined value.

  In a fourth invention of the present application, the determination unit determines whether a motion amount between frames or fields of the input image signal is larger than a predetermined value based on motion vector information included in the input image signal. It is characterized by that.

  According to a fifth aspect of the present invention, the detecting means divides the screen into a plurality of areas, calculates an average deviation of the average vector for each area, and multiplies the average vector by a predetermined coefficient as a threshold value. A region in which the distance between each average vector and the average vector of the entire screen is larger than the threshold is detected as a telop region.

  The sixth invention of the present application is characterized in that the motion compensation processing is invalidated by replacing the motion vector with a zero vector for a region other than the telop to generate an interpolated image signal.

  The seventh invention of the present application is characterized in that, for an area other than the telop, an interpolated image signal is generated by duplicating either the previous or next frame or field as it is.

  The eighth invention of the present application is characterized in that an interpolated image signal is generated by linear interpolation of a preceding or following frame or field for an area other than the telop.

  A ninth invention of the present application is characterized in that the image of the boundary portion between the detected one or more telop regions and the other regions is made continuous.

  A tenth aspect of the present invention is characterized in that a low-pass filter is applied to the interpolated image signal at a boundary portion between the detected one or more telop regions and the other regions.

  An eleventh invention of the present application is characterized in that a low-pass filter is applied to the motion vector at a boundary portion between the detected one or more telop regions and other regions.

  In a twelfth aspect of the present invention, the rate conversion means generates an image signal to be interpolated by weighted addition of the image signal subjected to the motion compensation process and the image signal not subjected to the motion compensation process. As the distance from the boundary between the telop area and the other area moves away from the telop area, the weighted addition ratio of the image signal subjected to the motion compensation is increased, and the boundary between the telop area and the other area is not a telop. The weighted addition ratio of the image signal subjected to the motion compensation is reduced as the distance to the area increases.

  The thirteenth invention of the present application interpolates an image signal that has been subjected to motion compensation processing using a motion vector between frames or fields of the input image signal, thereby reducing the number of frames or fields of the input image signal. An image display method comprising a rate conversion step for conversion, a detection step for detecting a feature amount of one or more telops included in the input image signal, and a motion amount between frames or fields of the input image signal A step of determining for each frame or field whether or not the amount of motion is greater than a predetermined value, the motion compensation is performed in the telop area. Interpolate the processed image signal and invalidate the motion compensation processing in other areas To, or by interpolating an image signal not subjected to the motion compensation processing, and converting the number of clicks or fields of the frame the input image signal.

  According to a fourteenth aspect of the present invention, the number of frames or the number of fields of the input image signal is calculated by interpolating an image signal subjected to motion compensation using a motion vector between frames or fields of the input image signal. An image processing apparatus comprising rate conversion means for converting, a detection means for detecting a feature quantity of one or more telops included in the input image signal, and a motion amount between frames or fields of the input image signal Determining means for determining whether or not is greater than a predetermined value for each frame or field, and when the motion amount between frames or fields of the input image signal is greater than a predetermined value, the motion compensation is performed in the telop area. Interpolate the processed image signal, and invalidate the motion compensation processing in other areas Is the by motion compensation process the interpolating an image signal not subjected and converting the number of number of frames or fields the input image signal.

  According to a fifteenth aspect of the present invention, the number of frames or the number of fields of the input image signal is calculated by interpolating an image signal subjected to motion compensation using a motion vector between frames or fields of the input image signal. An image processing method comprising a rate conversion step for converting, a detection step for detecting a feature amount of one or more telops included in the input image signal, and a motion amount between frames or fields of the input image signal A step of determining for each frame or field whether or not the amount of motion is greater than a predetermined value, the motion compensation is performed in the telop area. Interpolate the processed image signal and invalidate the motion compensation processing in other areas To, or by interpolating an image signal not subjected to the motion compensation processing, and converting the number of clicks or fields of the frame the input image signal.

  According to the present invention, by adopting the above-described configuration, on a video that makes motion vector detection difficult to operate normally, for example, a video that includes fast motion that exceeds the detection limit of motion vector detection processing, When scrolling telop is superimposed, the telop part is subjected to interpolation processing by motion compensation processing to reduce motion blur, and the motion compensation processing is invalidated in other areas, or motion compensation processing By performing the interpolation process using a method other than the above, it is possible to prevent image quality deterioration due to erroneous detection of motion vectors.

  Hereinafter, preferred embodiments of an image display device of the present invention will be described in detail with reference to the accompanying drawings. However, the same parts as those of the above-described conventional example are denoted by the same reference numerals, and the description thereof is omitted. Although the present invention can be applied to any of a field signal, an interpolated field signal, a frame signal, and an interpolated frame signal, both (field and frame) are in a similar relationship with each other, so A signal and an interpolated frame signal will be described as representative examples.

(1) Overall Configuration and Use of Telop Information FIG. 1 is a block diagram showing a schematic configuration of an FRC drive display circuit in a liquid crystal display device of the present invention. Although the configuration is almost the same as that of the conventional liquid crystal display device shown in FIG. 19, the internal configuration of the motion vector detection unit 105 is different.

  FIG. 2 is a functional block diagram showing an example of the motion vector detection unit included in the image display device of the present invention. The motion vector detection unit 105 included in the FRC unit 100 of the liquid crystal display device of the present invention shown in FIG. This is for explaining the internal configuration in detail. The motion vector detection unit 105 of this embodiment includes a frame delay unit 1, an initial displacement vector selection unit 2, a motion vector calculation unit 3, a vector memory 4, and a telop information detection unit 5.

  The motion vector detection unit 105 according to the present embodiment divides an input image signal input for each frame into a plurality of blocks having a predetermined size, for example, m pixels × n lines (m and n are integers). Then, for each divided block, for example, a motion vector representing the direction and magnitude of motion with the corresponding block in the input image signal one frame before delayed by the frame delay unit 1 is obtained. The candidate vector group selected from the motion vectors already detected and stored in the vector memory 4 and the telop information obtained by the telop information detection unit 5 are used together to obtain an optimal motion vector. An initial displacement vector selection unit 2 for selecting an initial displacement vector in the detected block; and Using the information, and a motion vector calculating portion 3 for obtaining the true motion vector in 該被 detection block correctly by, for example, the gradient method calculation twice.

  In particular, the present embodiment is characterized in that the telop information detection unit 5 is provided and the telop information obtained thereby is used for processing in the initial displacement vector selection unit 2 or the motion vector calculation unit 3. In the initial displacement vector selection unit 2, different processing is performed in the region where the telop exists and the other region, or the initial displacement vector is selected in consideration of the moving speed / direction of the telop, or A combination of both is performed. The motion vector calculation unit 3 performs vector calculation in consideration of the moving speed / direction of the telop in the area where the telop exists. By performing such processing, a more accurate detection vector can be obtained particularly in a region where a telop exists.

  In the telop information detection unit 5, as the telop feature amount (telop information) included in the input image signal, for example, telop area information indicating which motion detection block in the screen corresponds to the telop, and the moving speed / The telop vector information indicating the direction is detected. If there are a plurality of telops in the screen, telop area information and telop vector information may be detected for each of them. Details of the telop information detection unit 5 will be described later.

  Here, an example in which the iterative gradient method is used as a calculation method in the motion vector calculation unit 3 will be described, but the present invention is not limited to this iterative gradient method, and a block matching method or the like may be used.

  More specifically, the motion vector detection unit 105 shown in FIG. 2 includes the initial displacement vector selection unit 2, the motion vector calculation unit 3, the vector memory 4, and the telop information detection unit 5 as described above. It consists of The initial displacement vector selection unit 2 and the motion vector calculation unit 3 are respectively supplied with the current frame signal and the previous frame signal delayed by one frame via the frame delay unit 1.

  The initial displacement vector selection unit 2 selects a value most suitable for the motion of the detected block from the detected motion vectors obtained by the motion vector calculation of the previous frame, for example, a motion vector having a value closest to the motion of the detected block. This is a selection circuit that selects an initial displacement vector as a starting point of the gradient method calculation, and selects an appropriate motion vector from the above-described candidate vector group and telop vector. The initial displacement vector selection unit 2 divides the previous frame signal into blocks of m pixels × n lines as described above, for example, and uses the current frame signal and the previous frame as a reference for selecting the initial displacement vector for each of the divided blocks. Use frame signals.

  For example, as shown in FIG. 3, the initial displacement vector selection unit 2 includes a coordinate conversion unit 2a, a subtraction unit 2b, an absolute value accumulation unit 2c, a selection unit 2d, and a telop vector addition determination unit 2e. . In the initial displacement vector selection unit 2, motion vectors of 8 blocks around the block corresponding to the detected block sequentially read from the vector memory 4, that is, candidate vector groups, and one or more output from the telop information detection unit 5 Telop vector and telop area information, the previous frame signal, and the current frame signal are input.

  The telop vector addition determination unit 2e inputs one or more telop vectors and telop area information, and outputs a telop vector in the telop area to the coordinate conversion unit 2a when the block being processed corresponds to a certain telop area. . When the block being processed corresponds to a plurality of telop areas, the telop vectors in each of the plurality of telop areas, that is, the plurality of telop vectors are output to the coordinate conversion unit 2a.

  Each motion vector of each candidate vector group and one or more telop vectors output from the telop vector addition determination unit 2e are candidates for the initial displacement vector. The initial displacement vector candidates are supplied to the respective coordinate conversion units 2a, and the target block of the previous frame signal supplied from the frame delay unit 1 is displaced by the motion vector to perform coordinate conversion to the current frame. The coordinate conversion result is supplied to each subtraction unit 2b.

  In this embodiment, the candidate vector group uses the motion vector of the previous frame detected in the eight blocks around the detected block as the candidate vector group for selecting the initial displacement vector of the detected block. The candidate vector group is not limited to such an example, but may be configured to be determined from already detected motion vectors in other regions.

  Each subtraction unit 2b performs a subtraction process between the previous frame signal coordinate-converted by the coordinate conversion unit 2a and the input current frame signal to calculate a difference for each pixel. The result is supplied to the absolute value accumulation unit 2c. Each absolute value accumulation unit 2c obtains the absolute value of the difference between the pixels, accumulates the absolute difference for the number of pixels of the block, and selects the accumulated result as an evaluation value of the candidate vector. 2d respectively.

  The accumulated result obtained by the above procedure is called DFD (Displaced Field Difference). The DFD is an index indicating the degree of accuracy of a calculated vector (here, a candidate vector). The smaller the DFD value, the better the matching between the block of the previous frame and the coordinate-converted block of the current frame, Indicates that the corresponding candidate vector is more appropriate.

  Next, the selection unit 2d that has received each accumulated result (DFD) for each block compares the accumulated result (DFD) of each block, and is the candidate vector with the smallest accumulated result (DFD), that is, the most appropriate. The candidate vector which is considered to be detected is detected, the candidate vector is selected as an initial displacement vector, and supplied to the motion vector calculation unit 3. At this time, if the detected block corresponds to a telop area using the telop area information from the telop information detection unit 5, processing is performed so as to preferentially select the telop vector.

  More specifically, for example, when the detected block corresponds to a telop area, weighting is performed so as to reduce the accumulation result (DFD) for the telop vector among the output values from the absolute value accumulation unit 2c. For example, it is possible to use a method of reducing the value of the cumulative result (DFD) for the telop vector by multiplying the cumulative result for the telop vector by a coefficient w (0 <w <1).

  In the above-described embodiment, the method of using both the telop area information and the telop vector information detected by the telop information detection unit 5 in the initial displacement vector selection unit 2 has been described, but only one of the information is used. It does not matter as a configuration to be used. For example, an example of a configuration using only telop area information will be described with reference to FIG. In this configuration, since no telop vector is input, the telop vector addition determination unit 2e in FIG. 3 is excluded.

  In this example, the selection unit 2d performs weighting on the output of the absolute value accumulation unit 2c to prioritize the average vector or 0 vector of the entire screen when the block being processed is outside the telop area. In the case of a telop area, such weighting is not performed. By performing such processing, a relatively fast vector is easily selected in the telop area. That is, a vector corresponding to a fast-moving telop is easily selected.

  An example of a configuration using only telop vector information will be described with reference to FIG. In this configuration, no telop area information is input. For this reason, the telop vector addition determination unit 2e in FIG. 3 is also unnecessary, and telop vectors are added to initial displacement vector candidates for all blocks. Whether or not a vector that is the same as or close to the telop vector exists among the initial displacement vector candidates for the block in which the telop is present depends on the vector detection situation in the previous frame and is not certain. Therefore, it is possible to improve the possibility of selecting a more appropriate initial displacement vector by separately providing a telop vector as a candidate.

  Furthermore, it goes without saying that one or more of the above-described methods for using telop area information and telop vector information may be used in any combination.

  The motion vector calculation unit 3 uses the current frame signal and the previous frame signal to detect a motion vector for each block, and uses the initial displacement vector supplied from the initial displacement vector selection unit 2 as a starting point. This is an arithmetic circuit for obtaining a true motion vector from a frame signal to a current frame signal by gradient method calculation. Note that the motion vector calculation method by the gradient method calculation is detailed in each of the above-mentioned patent documents and non-patent documents, so the description thereof is omitted here, but how the initial displacement vector is used in this embodiment. Is described below using an iterative gradient method as an example.

  For example, in the gradient method calculation, the motion displacement V1 estimated from the coordinate position obtained by displacing the previous frame signal with the initial displacement vector V0 (α, β) as the starting point is represented by the following equation (1). ) And (2).

Formula 1

Formula 2

  In equations (1) and (2), Vx is the x-direction component of the difference between the motion vectors V0 and V1, and Vy is the y-direction component of the difference between the motion vectors V0 and V1. Here, Σ represents that a sum is obtained by calculating all coordinates in a block area of m pixels × n lines, for example, 8 pixels × 8 lines. Δx is the gradient in the x direction of the image luminance at the coordinate of interest (difference value with the adjacent pixel in the x direction), Δy is the gradient in the y direction of the image luminance at the coordinate of interest (the difference value with the adjacent pixel in the y direction), DFD (x, y) indicates the inter-frame difference value between the coordinates (x, y) of the previous frame and the coordinates (x + α, y + β) of the current frame, and is the same calculation method as described above. Further, sign (Δx) and sign (Δy) are codes indicating the direction of the gradient represented by any one of +1, −1, and 0, respectively.

  For example, in the case of the two-time iterative gradient method, as shown in FIG. 6, the initial displacement vector is set as V0, the first displacement V1 and the second displacement V2 are obtained, and the motion vector V obtained by adding them to the vector is obtained. Is obtained by the following equation (3).

Formula 3

  According to Expression (3), as shown in FIG. 7, the image existing in the block of coordinates (m1, n1) in the previous frame is moved to the block of the coordinate position of coordinates (m1 + α0, n1 + β0) in the current frame. At this time, the amount of motion is obtained as a vector V. Here, FIG. 7 is a schematic diagram for specifically explaining the motion vector V of the image moved between the previous frame and the current frame one frame before.

  In this way, the motion vector V obtained by the motion vector calculation unit 3 in FIG. 2 is accumulated in the vector memory 4 and is used as a candidate vector for initial displacement vector selection used for motion vector calculation for the next frame and thereafter. Used.

  As described above, by extracting the motion characteristics of the entire screen and applying the initial displacement vector compensated based on the motion characteristics of the entire screen, erroneous detection of the initial displacement vector is prevented, and, for example, a repetitive gradient that is less than twice It is possible to correctly calculate the true motion vector for each block of the current frame signal by the number of operations by the method.

  Here, one or more pieces of telop area information from the telop information detection unit 5 are input to the motion vector calculation unit 3, and special processing may be performed when the detected block corresponds to the telop area. . For example, when the direction of the telop vector is the horizontal direction, the iterative gradient method may be performed using only the x value, and the finally obtained motion vector may be limited to the motion in the horizontal direction. Alternatively, when the direction of the telop vector is the vertical direction, the iterative gradient method may be performed using only the y value, and the finally obtained motion vector may be limited to the vertical motion. This is to make it easier to follow the movement of the telop by performing the calculation according to the direction of the telop vector.

  In the above description, the motion vector calculation method in the motion vector calculation unit 3 employs the iterative gradient method using one or more gradient method operations, but is not limited to this. Or other calculation methods may be used.

  The vector memory 4 is a storage unit including a RAM (Random Access Memory) that accumulates motion vectors for at least one frame that has already been detected for each block, and its input terminal is connected to the output terminal of the motion vector calculation unit 3. Connected and configured to sequentially update and accumulate motion vectors detected by the motion vector calculation unit 3 at addresses corresponding to the positions of the respective blocks divided into 8 pixels × 8 lines, for example. Has been.

  The motion vector detection result for each block is output from the motion vector detection unit 105 by the above procedure.

  Then, the interpolation frame generation unit 102 generates an interpolation image using the motion vector detection result for each block and the telop area information output from the motion vector detection unit 105. At this time, the interpolation image generation using the motion vector may be performed in the entire area of the screen, the interpolation image generation using the motion vector is performed in the telop area, and the motion vector is used in the other areas. For example, the same image as the previous frame or the subsequent frame may be repeatedly output without generating the interpolated image. Alternatively, an interpolation image generation using a motion vector may be performed in the telop region, and a motion compensation process may be substantially invalidated by replacing the motion vector with 0 in other regions. That is, when the telop is not detected, the interpolated image generation process by motion compensation is not performed over the entire screen, and when the telop is detected, the interpolated image generation process by motion compensation is performed only for the telop area. It is also possible to perform an interpolated image generation process that does not use motion compensation for other regions. Alternatively, an interpolated image generation using a motion vector may be performed in the telop area, and an image generated by linear interpolation processing of the previous and subsequent frames may be performed in the other areas. In the linear interpolation process, an interpolation frame is obtained from the image signals of the previous and subsequent frames by linear interpolation using the frame interpolation ratio α.

  As described with reference to FIG. 21, in the interpolation frame generation process, the detected motion vector 201 is assigned to an interpolation vector 202 between frames, and an interpolation frame 203 is generated based on this interpolation vector. In the process of replacing the motion vector with 0, the detected motion vector itself may be replaced with 0, or the interpolation vector assigned to the interpolation frame may be replaced with 0. The same applies to the description of the process of replacing the motion vector with 0.

  As described above, a subject photographed by a normal camera includes blur caused by the light accumulation time of the camera (camera blur) when the movement is fast. If there are many camera blurs that originally exist, even if the motion blur is reduced by FRC, the effect is difficult to understand. On the other hand, since the telop is an image that has been synthesized later, even if the movement is fast, camera blur is not included, and the motion blur improvement effect by FRC is high. Therefore, even if the motion compensation process is performed only in the telop area, a large improvement effect can be obtained visually, and the interpolated image generation process is limited to the telop area, and an interpolated image is generated. Therefore, the processing amount can be reduced. Of course, if the motion compensation process is performed over the entire screen, it is needless to say that a motion blur improvement effect can be obtained over the entire screen.

  In addition, as described above, when interpolated image generation is performed using different processing in the telop area and the other areas, the difference in the interpolated image generation process in the boundary between the telop area and the other area is generated in the image. It may appear and stand out. In order to reduce this, the boundary between the telop area and other areas is conspicuous by performing a filtering process such as applying a low-pass filter to continuously change the strength of the motion compensation process. It is desirable to suppress this.

  For example, the boundary can be made inconspicuous by applying a low-pass filter at the boundary between the telop area and the other area of the interpolated image.

  Alternatively, when interpolation processing is generated by motion compensation in the telop area and the motion compensation process is substantially invalidated by replacing the motion vector with 0 in other areas, the telop area and other areas A method of applying a low-pass filter to the detected motion vector or the allocated interpolation vector in the motion vector detection block in the boundary portion and its vicinity can be considered. By doing so, an interpolation image is generated in the vicinity of the boundary so that the motion vector gradually decreases from the telop region to the other region, and the effect of motion compensation gradually decreases. That is, it can be said that the intensity of the motion compensation process is continuously changed to make the boundary inconspicuous.

  Alternatively, a process of generating an interpolated image by motion compensation and an image obtained by linear interpolation of the preceding and following frames or an image obtained by copying the preceding and following frames as they are and generating a final interpolated image by weighted addition of them. When performing, the weighted addition ratio may be continuously changed at the boundary between the telop area and the other areas. As the distance from the boundary to the telop region increases, the ratio of the interpolated image by motion compensation is increased, and as the distance from the boundary to the region other than the telop increases, the ratio of the interpolated image by motion compensation is decreased. By doing so, an interpolated image is generated from the telop region to the other region so that the contribution of the interpolated image by motion compensation gradually decreases. That is, it can be said that the intensity of the motion compensation process is continuously changed to make the boundary inconspicuous.

(2) Example of telop information detection method As described above, the telop information detection unit 5 detects telop information using the detected motion vector obtained by the motion vector calculation of the previous frame stored in the vector memory 4. . An example of a specific method for realizing the telop information detection unit 5 will be described in detail below.

  Information that can be used for telop detection is only the motion vector of each motion detection block stored in the vector memory 4 and the texture information of each motion detection block. Among these, since the colors of the telop are various, the texture information can be used only as an auxiliary. For this reason, it is necessary to detect the telop area information and the telop vector information from the motion vector information of each motion detection block.

  In addition to the telop, there are various moving objects such as a person and a car in the video. In some cases, the entire screen is moved relatively by panning the camera. For this reason, it is difficult to make a simple determination, for example, that a fast-moving region is a telop region, that is, whether or not a telop is based on the absolute amount of a motion vector.

  For this reason, in this embodiment, the telop area and the telop speed are detected using statistical information such as the difference between the average vector of the entire screen and the motion vector of each motion detection block, and the average deviation of the motion vectors.

  FIG. 8 shows a state in which the image is decomposed into blocks for vector detection. The overall size of the image is a width Wa pixel and a height Ha pixel. When this image is divided into motion detection blocks having a width of Wb pixels and a height of Hb pixels, the number of blocks is m horizontal blocks and n vertical blocks. Normally, the number of pixels in the entire image is divisible by an integer number of blocks. That is, Wa = Wb × m and Ha = Hb × n.

  For example, if the image has a high-definition resolution (Wa = 1920 pixels, Ha = 1080 pixels) and the block size is 8 × 8 pixels, m = 240 and n = 135. Each motion detection block is called B (i, j), and the motion vector detected by each motion detection block is (V_x (i, j), V_y (i, j)).

  Here, since many telops move in the horizontal direction, in this embodiment, telops that move in the horizontal direction are detected. The telop moving in the horizontal direction is located in a horizontally long belt-like area on the screen as shown in FIG. Therefore, as shown in FIG. 10, the screen is divided into horizontally long strip-like regions L (1) to L (n), and it is determined whether or not each region includes a telop.

  The belt-like region L (j) includes the motion detection blocks B (1, j) to B (m, j) in FIG. When the average vector of motion vectors of the motion detection block included in L (j) is (Vave_x (j), Vave_y (j)),

Formula 4

Formula 5

It is.

  Further, when the average vector (overall average vector) of motion vectors of all motion detection blocks is (Vave_x, Vave_y),

Equation 6

Equation 7

It is.

  Now, as shown in FIG. 11, the screen is divided into a region including a telop and a region other than that. The height of the telop area when the screen height is 1 is k. However, it is assumed that the telop area does not exceed half of the screen. That is,

Equation 8

Assume that The height of the area other than the telop is 1-k. When the area other than the telop is divided into two or more by the telop area, the respective heights are added.

  Next, the average of the motion vectors of the motion detection blocks included in the telop area is (Vt_x, Vt_y), and the average of the motion vectors of the motion detection blocks included in the area other than the telop (referred to as the background area in this specification) is ( Vb_x, Vb_y) between the average vector of the telop area, the average vector of the area other than the telop, and the average vector of the entire screen,

Equation 9

Equation 10

This relationship holds.

  Actually, not all blocks in the telop area have the same motion vector as the moving speed of the telop. For example, a block in a portion where a telop character is interrupted has a motion vector other than the movement speed of the telop. However, in the present embodiment, it is assumed for the time being that all blocks in the telop area have the moving speed of the telop, and the following description proceeds. This assumption will be described later.

  In the present embodiment, since the telop moving in the horizontal direction is a detection target, the following description will be made focusing on the value in the horizontal direction for each average vector, that is, the x value of the vector.

  The relationship among the average vector Vt_x of the telop area, the average vector Vb_x of the area other than the telop, and the overall average vector Vave_x in Expression (9) is as shown in FIGS.

  FIG. 12 illustrates the case of Vb_x <Vt_x. Vave_x is located between Vb_x and Vt_x, and the ratio between the distance between Vb_x and Vave_x and the distance between Vave_x and Vt_x is k: 1-k. This relationship holds regardless of the magnitude or the sign of Vave_x, Vb_x, and Vt_x. Also, from the condition of equation (8),

Equation 11

Always holds. Therefore, the distance between Vb_x and Vave_x is always smaller than the distance between Vave_x and Vt_x.

  FIG. 13 illustrates the case of Vt_x <Vb_x. Vave_x is located between Vt_x and Vb_x, and the ratio between the distance between Vt_x and Vave_x and the distance between Vave_x and Vb_x is 1-k: k. This relationship holds regardless of the magnitude or the sign of Vave_x, Vt_x, Vb_x. Moreover, Formula (11) is always formed from the conditions of Formula (8). Therefore, as in the case of FIG. 12, the distance between Vt_x and Vave_x is always smaller than the distance between Vave_x and Vb_x.

  That is, whether Vb_x <Vt_x or Vt_x <Vb_x, the distance between Vb_x and Vave_x is always smaller than the distance between Vave_x and Vt_x. That means

Formula 12

Always holds.

  Therefore, a certain threshold value T is prepared,

Equation 13

It is determined that Using this threshold value T, the distance between Vave_x (j) and Vave_x and the threshold value T are compared with respect to the average vector x value Vave_x (j) of the motion vectors of the motion detection blocks included in each band-like region Lj in FIG. If it is larger than T, it can be determined that the band-like area Lj is likely to belong to the telop area. That is, regarding a certain band-like region Lj,

Equation 14

Is established, the band-like area is determined as a telop area.

  The problem here is how the threshold value T is set. The method will be described below.

  The setting condition of the threshold T is as shown in Expression (13). From this equation (13), it can be seen that in order to determine the threshold value T, information on | Vb_x-Vave_x | and | Vt_x-Vave_x | is required, but Vt_x and Vb_x cannot be directly obtained. This is because it is not known in advance which area is the telop area.

  Here, | Vb_x-Vave_x | and | Vt_x-Vave_x | are the difference between the global average vector and the average vector of the background area, and the difference between the global average vector and the average vector of the telop area, respectively. It is closely related to the variation of vectors. Therefore, the value of the threshold value T is determined by using an average deviation which is one of the data variation scales.

  Assuming that the average deviation of the average vector Vave_x (j) of each strip region Lj is M, M is

Equation 15

Is calculated by Multiply the average deviation M by a constant α,

Equation 16

And an appropriate constant α is obtained as follows.

  Of each strip-like region Lj, the region belonging to the telop region and its x-direction average vector are Lt (l) and Vt_x (l) (l = 1,..., H), the other strip-like regions and their x-direction averages Let the vectors be Lb (m) and Vb_x (m) (m = 1,..., (N−h)). h is the number of band-like areas belonging to the telop area.

  First, consider the case of Vb_x <Vt_x. For simplicity, it is assumed that Vt_x (l)> Vave_x and Vb_x (m) <Vave_x are satisfied for each band-like region. At this time, when the average deviation M is expressed using Vt_x and Vb_x,

Equation 17

It can be expressed. Substituting equation (9) into equation (17) and rearranging,

Equation 18

Is obtained.

  From Equation (13) and Equation (16),

Equation 19

It is.

  On the other hand, from the condition of Vb_x <Vt_x and the equation (9),

Equation 20

Equation 21

Is obtained.

  Substituting Equation (18), Equation (20), and Equation (21) into Equation (19) and rearranging,

Equation 22

Is obtained.

  Although a detailed description is omitted, Expression (22) is obtained as a conditional expression even when Vt_x <Vb_x.

  The value of the constant α can be determined by assuming the height k of the telop area according to the equation (22). If k = 0.2, substitute it into equation (22)

Equation 23

The condition is obtained. If the constant α is set within this range, a telop area of about k = 0.2 can be detected.

  Also, if k = 0.4, it is similarly substituted into the equation (22),

Formula 24

The condition is obtained. That is, it can be seen that the range in which the constant α can be set becomes narrower in order to cope with the case where a wider telop area exists.

  As described above, the settable range of the constant α can be derived by assuming the value of the height k of the telop area. The actual image is analyzed to determine the tendency of the height k of the telop area to determine k, and the constant α may be determined in accordance with the settable range of the constant α determined thereby.

  If the constant α is determined and the average deviation M is obtained from the equation (15), the value of the threshold T can be determined from the equation (16). Here, it should be noted that the average deviation M is calculated from the detected motion vector. That is, the value of the threshold T changes for each frame depending on the state of the detected motion vector, that is, depending on the motion of the object in the video.

  When the value of the threshold T is obtained, a determination process is performed on the x value Vave_x (j) of the average vector of each strip region Lj according to the equation (14) to determine whether the strip region is a telop region. Can do.

  Further, if each band-like area Lj is determined and it is determined which band-like area is the telop area, the average (Vt_x, Vt_y) of the motion vectors of the motion detection blocks included in the telop area can be calculated. it can. This is the telop vector.

  Here, consideration is given to the assumption that all the blocks in the telop area have the telop speed in the above description. Actually, not all blocks in the telop area have the same motion vector as the telop speed. The more blocks that do not contain telop characters in the telop area, for example, when telop characters exist sparsely, the average vector Vt_x of the telop area is closer to the overall average vector Vave_x than the true telop vector. Become. At the same time, the average vector Vb_x in the area other than the telop is also close to the overall average vector Vave_x. That is, both | Vt_x−Vave_x | and | Vb_x−Vave_x | have smaller values.

  Therefore, the telop area cannot be detected unless the value of the threshold T for which the setting range is limited by the equation (13) or the equation (19) is also set small, that is, unless the value of α is set small. . However, if the threshold value T is reduced, the possibility that a non-telop area is erroneously detected increases.

  On the other hand, when there are few blocks that do not include telop characters in the telop area, for example, when there are dense telop characters, it is possible to expect an operation almost as described above.

  As described above, in actual application, it is necessary to determine the constant α in consideration of how much sparse characters are detected and how much false detection is allowed.

  In the above-described embodiment, the telop is detected on the assumption that the telop moves in the horizontal direction. However, when the telop is moved in the vertical direction or the diagonal direction, it can be detected by the same method. In that case, the method of dividing the band-like region may be set according to the direction to be detected. For example, when it is desired to detect a telop that moves in the vertical direction, the band-like region may be set to a vertically long region.

  Next, a telop information detection method when there are a plurality of telops in the screen will be described. For example, as shown in FIG. 14, the screen is divided into two telop detection areas 1 and 2 at the top and bottom, and the telop detection method described above is executed in each area, so that the telop area information A, the telop vector information Va, and the telop Two pieces of telop information, that is, region information B and telop vector information Vb can be detected.

  Alternatively, for example, as shown in FIG. 15, horizontal telop detection processing and vertical telop detection processing are executed in parallel, so that horizontal telop area information C, telop vector information Vc, and vertical telop area information are displayed. Two pieces of telop information including D and telop vector information Vd may be detected. Here, the common area between the telop area C and the telop area D corresponds to the case where the block being processed corresponds to a plurality of telop areas in the above description of the vector detection means.

  According to the present embodiment, one or more pieces of telop information can be detected by the procedure as described above. The processing using the detected telop information is as described above.

  Note that the above-described telop information detection means is merely an example, and even when the telop information is detected using other means, it is applicable to the motion vector detection method and the interpolated image generation method in the present invention. Needless to say, this is possible.

  Here, it should be noted that the motion vector of each motion detection block stored in the vector memory 4 is used as an input to the telop information detection unit 5. The motion vector of each motion detection block stored in the vector memory 4 is a motion vector detection result of the previous frame. That is, the processing of the telop information detection unit 5 is performed using the motion vector detection result of the previous frame. Since the telop is usually present at the same position over a plurality of frames, the use of the motion vector detection result of the previous frame as described above is not a big problem.

  As described above in detail, in the motion vector detection method according to the present embodiment, a telop area and a telop vector are detected as telop feature quantities, and the results are reflected in initial displacement vector selection and motion vector calculation to compensate for motion. By controlling the processing, it is possible to more correctly detect the motion vector of the telop area, and as a result, the image quality of the telop area can be improved.

(3) Processing of telop in video that is likely to cause an error in motion vector detection processing By the way, as described in the above-mentioned “problem to be solved by the invention”, for example, video that has been pulled down 2-3, In a video that is 2-2 pulled down, a video that includes a scene change, a video that includes flashing, and a video that includes slow playback, there are cases where the correlation between frames is low in a certain two consecutive frames in the video. In addition, a correct motion vector cannot be detected in a video including a fast motion that exceeds the detection limit of the motion vector detection process. In such a case, referring to the motion vector detection result of the previous frame stored in the vector memory 4 may cause erroneous vector detection. For this reason, for example, in Patent Document 8, when the correlation between consecutive frames is interrupted due to a scene change or the like, the motion vector is fixed to 0 vector so that an interpolation image is not generated by detecting an erroneous motion vector. Thus, a method for improving image quality degradation based on detection errors is described.

  However, in the case of an input image signal in which a scrolling telop is superimposed on an image in which a scene change exists, if the above processing is performed, motion blur in the telop portion cannot be eliminated. Therefore, for the area detected as the telop area, even if a scene change occurs in the input image signal, a motion compensation process different from that for other areas is performed to obtain an interpolated image in which the telop scrolls smoothly. It is desirable to be able to.

  First, an example of processing in the case where there is a case where the correlation between frames exists in a certain two consecutive frames in a video will be described with reference to FIG.

  FIG. 16 shows an interpolation frame generation unit 107 in which a motion vector discontinuity detection unit 106 is added to the liquid crystal display device shown in FIG. 1 and the output of the motion vector discontinuity detection unit 106 is newly used. It is to be prepared.

  The motion vector discontinuity detection unit 106 receives the motion vector output from the motion vector detection unit 105 as an input. The length of the motion vector for each motion detection block is calculated, and it is added for one frame.

  If the input image is a video that has been pulled down 2-2, two consecutive images are the same video. Also, if the input image is a video that has been pulled down 2-3, the input video is the same video every two or three consecutive videos. Furthermore, in so-called slow playback, video is played back slower than the original speed by outputting the same image from time to time. Therefore, when the two frames used for motion vector detection are the same image, the added value of the lengths of the detected motion vectors is almost 0, and when the two frames used for motion vector detection are different images, If there is a moving subject in the video, the added value of the length of the detected motion vector becomes a certain amount. That is, in the above-described 2-2 pull-down video, 2-3 pull-down video, and slow-played video, there are frames without motion vector continuity.

  On the other hand, in a normal video, the image is different for each frame, and the movement of the subject in the video is continuous. Therefore, if there is a moving subject in the video, the added value of the length of the detected motion vector has a certain size, and the added value does not change greatly between frames. That is, there is continuity of motion vectors in normal video.

  Therefore, the motion vector discontinuity detecting unit 106 subtracts the added value of the length of the motion vector in a certain frame and the added value of the next frame to obtain an absolute value, and whether it is larger or smaller than a predetermined threshold value. Determine. If it is larger, it can be determined that the frame has no continuity of motion vectors. That is, it can be determined that the video is a 2-2 pull-down video, a 2-3 pull-down video, or a slow-played video. When determined in this way, the motion vector discontinuous information is output to the interpolation frame generation unit 107. Also, in the 2-2 pull-down video, the motion vector is detected as non-continuous once every two frames, and in the 2-3 pull-down video, the motion vector is detected as non-continuous once every two or three frames and is played back slowly. In the video, the motion vector is detected once every several frames (depending on the slow playback speed). Therefore, once the motion vector is detected to be discontinuous, a predetermined number of frames are continuously output to the interpolated frame generation unit 107 for a predetermined number of frames, so that the video that has been 2-2 pulled down, It is possible to detect any type of video that has been pulled down 2-3, such as slow-played video.

  If the motion vector discontinuity detecting means described above is used, the motion vector discontinuity is detected when the scene changes suddenly, or the subject is illuminated by flash light. It is also possible to detect a case where there is a frame with suddenly large and different brightness and vector detection cannot be performed correctly and motion vectors are discontinuous.

  In the example of FIG. 16, the case where the pull-down video and the slow playback video are detected using the motion vector detected by the motion vector detecting unit 105 has been described. However, the pull-down video and the slow playback video are detected by other methods. You may do it. For example, when the input image signal is MPEG-encoded, the same detection can be performed using a motion vector included in the MPEG-encoded signal.

  Alternatively, as an example in which a motion vector is not used, for example, an absolute value of a luminance difference value is obtained for each pixel of two consecutive frames from an input image signal, and is added for one frame, and the value is a predetermined threshold value If it is smaller, the two frames are the same image, and if they are larger, they can be discriminated as different images. This is because there are cases where two consecutive frames are the same image in the 2-2 pull-down video, the 2-3 pull-down video, and the slow-played video.

  Further, a method of determining that the input image signal is a pull-down video from other information, instead of determining from the input image signal itself, can be considered. For example, when the video genre of the input image signal is “movie” by EPG (Electronic Program Guide) added to the input image signal, it can be determined that the video is a pull-down video. Alternatively, when the image mode of the television receiver is set to movie, it is possible to determine that the video viewed by the television receiver in this state is a “movie”, that is, a pull-down video. It is. Here, the image adjustment mode is a set of recommended values for various video settings (image quality adjustments) preset in a video device such as a television receiver or adjusted and stored by a user.

  The interpolation frame generation unit 107 receives motion vectors and telop information from the motion vector detection unit 105, and receives motion vector discontinuity information from the motion vector discontinuity detection unit 106. When motion vector discontinuity information is output from the motion vector discontinuity detection unit 106, interpolation image generation processing by motion compensation is not performed in regions other than the telop, and other interpolation image generation processing is performed.

  The other interpolated image generation processes include, for example, a process in which interpolated image generation by motion compensation is substantially invalidated by replacing all motion vectors with 0, interpolated image generation by linear interpolation of previous and subsequent frames, For example, a process of using any one of the frames as it is to obtain an interpolated image. In the telop area, an interpolated image generation process by motion compensation is performed regardless of motion vector discontinuous information. When motion vector discontinuity information is not output from the motion vector discontinuity detecting unit 106, an interpolated image generation process by motion compensation is performed over the entire screen.

  It should be noted that the video that has been pulled down 2-2 and the video that has been pulled down 2-3 are usually reproduced continuously for a certain period of time, and it is unlikely that they will be changed to different types of video during several frames. Therefore, once it is determined by the motion vector discontinuity detection unit 106 that the input image is a discontinuous motion image, there is a high possibility that this determination result will last at least several frames. Therefore, the accuracy of the determination result can be improved by referring to the determination result of the previous frame (past) for this determination process. In addition, once it is determined that the input image is a discontinuous motion image, a predetermined number of frames thereafter may maintain this determination result, or a predetermined threshold for determination has hysteresis. May be allowed.

  In this way, when an interpolated image is generated by motion compensation only for a telop area, the presence or absence of motion compensation processing may clearly appear in the image at the boundary between the telop area and the other area. In order to reduce this, the boundary between the telop area and other areas is conspicuous by performing a filtering process such as applying a low-pass filter to continuously change the strength of the motion compensation process. It is desirable to suppress this. Since this method has already been described, it is omitted here.

  By performing the above-described processing, the motion vector detection is difficult to operate normally. For example, the video includes 2-3 pull-down video, 2-2 pull-down video, slow-play video, and scene change. When a scrolling telop is superimposed on a video or video that includes flashing, the telop part performs motion compensation processing to reduce motion blur, and in other regions motion compensation processing By performing the interpolation process using a method other than the above, it is possible to achieve both of preventing image quality deterioration due to erroneous detection of motion vectors.

  Next, an example of processing for a video including a fast motion that exceeds the detection limit of the motion vector detection processing will be described with reference to FIG.

  FIG. 17 includes an interpolation frame generation unit 107 in which a motion vector determination unit 108 is added to the liquid crystal display device shown in FIG. 1 and the output of the motion vector determination unit 108 is newly used.

  The motion vector determination unit 108 receives the motion vector output from the motion vector detection unit 105 as input, and determines for each frame whether the amount of motion between frames of the input image signal is greater than a predetermined value.

  The motion vector determination unit 108 adds flag information to a motion detection block whose detected motion vector exceeds a predetermined range, and counts the total number of blocks to which flag information is added for each frame. The count value is compared with a predetermined threshold value, and if the count value is greater than the threshold value, it is determined that the amount of motion is large. When the count value is smaller than the threshold value, it is determined that the amount of motion is small. This motion amount determination information is sent to the interpolation frame generation unit 107. Here, when it is determined that the amount of motion is large, the region of the fast-moving subject in the screen is large, and therefore there is a high possibility of including a fast motion that exceeds the detection limit of the motion vector detection process. Conceivable.

  The interpolation frame generation unit 107 receives a motion vector and telop information from the motion vector detection unit 105, and receives motion amount determination information from the motion vector determination unit 108. When motion amount determination information indicating that the amount of motion is large is received from the motion vector determination unit 108, interpolation image generation processing based on motion compensation is not performed in regions other than the telop, and other interpolation image generation processing is performed. .

  The other interpolated image generation processes include, for example, a process in which interpolated image generation by motion compensation is substantially invalidated by replacing all motion vectors with 0, interpolated image generation by linear interpolation of previous and subsequent frames, For example, a process of using any one of the frames as it is to obtain an interpolated image. In the telop area, an interpolated image generation process by motion compensation is performed regardless of the motion amount determination information. When motion amount determination information indicating that the motion amount is small is received from the motion vector determination unit 108, an interpolated image generation process by motion compensation is performed over the entire screen.

  It should be noted that since a general moving image has continuity of motion in the time direction, once the input image is determined to be a fast-moving image by the motion vector determination unit 108, this determination result continues for several frames. There is a high possibility of doing. Therefore, the accuracy of the determination result can be improved by referring to the determination result of the previous frame (past) for this determination process. Further, once it is determined that the input image is a fast moving image, a predetermined number of frames thereafter may continue this determination result, or a predetermined threshold for determination may be provided with hysteresis. good.

  In the example of FIG. 17, the method of determining that the video is fast motion using the motion vector detected by the motion vector detection unit 105 has been described, but the determination may be performed by other methods. For example, when the input image signal is MPEG-encoded, the same detection can be performed using a motion vector included in the MPEG-encoded signal.

  In this way, when an interpolated image is generated by motion compensation only for a telop area, the presence or absence of motion compensation processing may clearly appear in the image at the boundary between the telop area and the other area. In order to reduce this, the boundary between the telop area and other areas is conspicuous by performing a filtering process such as applying a low-pass filter to continuously change the strength of the motion compensation process. It is desirable to suppress this. Since this method has already been described, it is omitted here.

  By performing the processing as described above, when a scrolling telop is superimposed on a video that makes motion vector detection difficult to operate normally within the vector search range, for example, a video with very fast motion, The part is subjected to motion compensation processing to reduce motion blur, and in other areas, interpolation is performed using methods other than motion compensation processing to prevent image quality deterioration due to motion vector misdetection. To achieve both.

  Next, another example of processing for a video including a fast motion exceeding the detection limit of the motion vector detection processing will be described with reference to FIG.

  FIG. 18 includes an interpolation frame generation unit 107 in which a motion vector reliability determination unit 109 is added to the liquid crystal display device shown in FIG. 1 and the output of the motion vector reliability determination unit 109 is newly used. It is.

  The motion vector reliability determination unit 109 receives the motion vector output from the motion vector detection unit 105 and its DFD value as input, and determines whether the detected motion vector is reliable.

  If there is a subject in the input image that has a very fast motion that exceeds a predetermined vector search range in the motion vector detection unit 105, the motion vector detection unit 105 can detect a correct motion vector. Even if some motion vector is detected, it is not a correct vector. For example, when the gradient method is used, some motion vector is calculated by calculation of the gradient method even in the motion detection block of the portion of the subject that moves very fast that exceeds the predetermined vector search range as described above. . However, when the DFD is calculated for the calculated motion vector, it can be seen that the value is large, that is, the calculated motion vector is incorrect.

  Therefore, the motion vector detection unit 105 calculates a DFD for each calculated motion vector in addition to the motion vector, and outputs the value together with the motion vector to the motion vector reliability determination unit 109. The motion vector reliability determination unit 109 counts the number of vectors such that each DFD value exceeds a predetermined threshold over the entire screen for one frame. If this count value is equal to or greater than another predetermined threshold, it is determined that the detected motion vector has low reliability, and motion vector reliability degradation information is output. By such processing, when the input image is a video with very fast motion and an optimal motion vector cannot be detected within the vector search range, it can be detected.

  Even when other means such as block matching or phase correlation method is used as the motion vector detection method, the correctness of the obtained motion vector can be similarly determined by DFD.

  The interpolation frame generation unit 107 receives motion vectors and telop information from the motion vector detection unit 105, and receives motion vector reliability reduction information from the motion vector reliability determination unit 109. The motion vector reliability determination unit 109 outputs motion vector reliability degradation information indicating that the input image is a video with very fast motion, and an optimal motion vector could not be detected within the vector search range. In this case, in the area other than the telop, the interpolated image generation process by motion compensation is not performed, and the other interpolated image generation process is performed.

  The other interpolated image generation processes include, for example, a process in which interpolated image generation by motion compensation is substantially invalidated by replacing all motion vectors with 0, interpolated image generation by linear interpolation of previous and subsequent frames, For example, a process of using any one of the frames as it is to obtain an interpolated image. In the telop area, an interpolated image generation process by motion compensation is performed regardless of the motion vector reliability information. When no motion vector reliability reduction information is output from the motion vector reliability determination unit 109, an interpolation image generation process by motion compensation is performed over the entire screen.

  In addition, since there is continuity of motion in the time direction in a general video image, if the motion vector reliability determination unit 109 determines that the input image is a video with very fast motion, this determination result Is likely to last several frames. Therefore, the accuracy of the determination result can be improved by referring to the determination result of the previous frame (past) for this determination process. In addition, once it is determined that the input image is a very fast moving image, a predetermined number of frames thereafter may continue this determination result, or a predetermined threshold value for determination may be provided with hysteresis. May be.

  In the example of FIG. 18, when the value of DFD is large, the motion vector is not correctly detected and it is determined that the input image is a video with very fast movement. The value of may be large. For example, even in a video including a scene change and a video including a flushing, the motion vector cannot be detected correctly at the moment of the scene change or the flashing, and the DFD value increases. This example can also be detected in such a case.

  In this way, when an interpolated image is generated by motion compensation only for a telop area, the presence or absence of motion compensation processing may clearly appear in the image at the boundary between the telop area and the other area. In order to reduce this, the boundary between the telop area and other areas is conspicuous by performing a filtering process such as applying a low-pass filter to continuously change the strength of the motion compensation process. It is desirable to suppress this. Since this method has already been described, it is omitted here.

  By performing the processing as described above, when a scrolling telop is superimposed on a video that makes motion vector detection difficult to operate normally within the vector search range, for example, a video with very fast motion, The part is subjected to motion compensation processing to reduce motion blur, and in other areas, interpolation is performed using methods other than motion compensation processing to prevent image quality deterioration due to motion vector misdetection. To achieve both.

  In the example of FIGS. 17 and 18, the example in which it is detected whether the input image is a very fast motion image using the information in the motion vector detection has been described. However, the input image is It may also be detected whether the video is fast moving. For example, a phase correlation method is used to determine whether there is a correlation between two consecutive frames of the input image, and when the correlation between the two frames is low, the input image is a very fast moving image. If the positional deviation of the correlation between the two frames is large, it may be determined that the input image is a very fast moving image.

  The image display device of the present invention can be applied not only to a liquid crystal display using a liquid crystal panel as a display panel but also to image display devices having hold type display characteristics such as an organic EL display and an electrophoretic display. . Needless to say, the input image signal is not limited to a television broadcast signal, and may be various image signals such as an image signal reproduced from an external medium.

  In the above description, an example of an embodiment related to the image display apparatus and method of the present invention has been described. From these descriptions, an image display program for executing the image display method as a program by a computer, and the image A program recording medium in which the display program is recorded on a computer-readable recording medium can be easily understood.

It is a block diagram which shows schematic structure of the FRC drive display circuit in the liquid crystal display device which concerns on one Embodiment of this invention. It is a functional block diagram which shows the structural example of the motion vector detection part in the frame rate conversion part with which the image display apparatus which concerns on one Embodiment of this invention is provided. FIG. 3 is a functional block diagram illustrating a configuration example of an initial displacement vector selection unit in FIG. 2. It is a functional block diagram which shows another structural example of the initial displacement vector selection part in FIG. It is a functional block diagram which shows another example of a structure of the initial displacement vector selection part in FIG. It is a vector diagram for demonstrating the calculation method of the motion vector V by the iterative gradient method twice. It is a schematic diagram for specifically explaining a motion vector V of an image moved between the previous frame and the current frame one frame before. It is explanatory drawing which shows a mode that the image was decomposed | disassembled into the some block. It is explanatory drawing which shows the telop which moves to a horizontal direction on a screen. It is explanatory drawing which shows a mode that the screen was divided | segmented into the some strip | belt-shaped area | region. It is explanatory drawing which shows a mode that the screen was decomposed | disassembled into the area | region containing a telop, and the area | region other than that. It is explanatory drawing which shows the relationship with the average vector of the area | region of a telop, the average vector of areas other than a telop, and the average vector of the whole screen. It is explanatory drawing which shows the relationship with the average vector of the area | region of a telop, the average vector of areas other than a telop, and the average vector of the whole screen. It is explanatory drawing which shows an example in the case of detecting two telop information. It is explanatory drawing which shows another example in the case of detecting two telop information. It is explanatory drawing of the example of the method of detecting the discontinuity of a motion vector and controlling an interpolation image generation process using it. It is explanatory drawing of the example of the method of determining whether the length of a motion vector is larger than predetermined value, and controlling an interpolation image generation process using it. It is explanatory drawing of another example of the method of detecting the reliability of a motion vector and controlling an interpolation image generation process using it. It is a block diagram which shows schematic structure of the FRC drive display circuit in the conventional liquid crystal display device. It is a figure for demonstrating the frame rate conversion process by the conventional FRC drive display circuit shown in FIG. It is a figure for demonstrating the interpolation frame production | generation process by a motion vector detection part and an interpolation frame production | generation part.

Explanation of symbols

1 frame delay unit 2 initial displacement vector selection unit 2a coordinate conversion unit 2b subtraction unit 2c absolute value accumulation unit 2d selection unit 2e telop vector addition determination unit 3 motion vector calculation unit 4 vector memory 5 telop information detection unit 100 frame rate conversion (FRC) ) Unit 101 motion vector detection unit 102 interpolation frame generation unit 103 liquid crystal display panel 104 electrode drive unit 105 motion vector detection unit 106 motion vector discontinuity detection unit 107 interpolation frame generation unit 108 motion vector determination unit 109 motion vector reliability Determination unit 201 motion vector 202 interpolation vector 203 interpolation frame

Claims (15)

Rate conversion means for converting the number of frames or the number of fields of the input image signal by interpolating an image signal subjected to motion compensation processing using a motion vector between frames or fields of the input image signal An image display device,
Detecting means for detecting a feature amount of one or more telops included in the input image signal;
Determination means for determining, for each frame or field, whether or not a motion amount between frames or fields of the input image signal is larger than a predetermined value;
When the motion amount between frames or fields of the input image signal is larger than a predetermined value, the image signal subjected to the motion compensation processing is interpolated in the telop region, and the motion compensation processing is invalidated in other regions. Or an image display device that converts the number of frames or the number of fields of the input image signal by interpolating an image signal not subjected to the motion compensation process. The image display device according to claim 1,
The determination means determines whether a motion amount between frames or fields of the input image signal is larger than a predetermined value based on a motion vector calculated by performing a predetermined operation on the input image signal. An image display device. The image display device according to claim 2,
A frame of the input image signal is obtained by adding flag information to a block in which the calculated motion vector exceeds a predetermined range and comparing a count value for each frame or field of the flag information with a predetermined threshold value. An image display apparatus for determining whether or not a motion amount between fields or fields is larger than a predetermined value. The image display device according to claim 1,
The determination means determines whether or not a motion amount between frames or fields of the input image signal is larger than a predetermined value based on motion vector information included in the input image signal. . In the image display device according to any one of claims 1 to 4,
The detection means divides the screen into a plurality of areas, obtains an average deviation of the average vector for each area, and sets a value obtained by multiplying the average deviation by a predetermined coefficient as a threshold, An image display device, wherein a region having a distance from an average vector larger than the threshold is detected as a telop region. The image display device according to any one of claims 1 to 5,
An image display device, wherein an interpolation image signal is generated by invalidating the motion compensation processing by replacing the motion vector with a zero vector for a region other than the telop. The image display device according to any one of claims 1 to 5,
An image display device, wherein an interpolated image signal is generated by duplicating one of the preceding and following frames or fields as it is for an area other than the telop. The image display device according to any one of claims 1 to 5,
An image display apparatus, wherein an interpolated image signal is generated for a region other than the telop by linear interpolation of preceding and following frames or fields. The image display device according to any one of claims 1 to 8,
An image display device characterized by providing continuity to an image of a boundary portion between one or more detected telop areas and other areas. The image display device according to claim 9,
An image display device, wherein a low-pass filter is applied to the interpolated image signal at a boundary portion between the detected one or more telop regions and other regions. The image display device according to claim 9,
An image display device, wherein a low-pass filter is applied to the motion vector at a boundary portion between the detected one or more telop regions and other regions. The image display device according to claim 9,
The rate conversion means generates an image signal to be interpolated by weighted addition of the image signal subjected to the motion compensation process and the image signal not subjected to the motion compensation process,
As the distance from the boundary between the telop area and the other area moves away from the boundary to the telop area, the weighted addition ratio of the image signal subjected to the motion compensation is increased, and the boundary between the telop area and the other area other than the telop is increased. An image display device characterized in that the weighted addition ratio of the image signal subjected to the motion compensation is reduced as the distance from the region increases. A rate conversion step of converting the number of frames or the number of fields of the input image signal by interpolating an image signal subjected to motion compensation processing using a motion vector between frames or fields of the input image signal; An image display method,
A detection step of detecting a feature amount of one or more telops included in the input image signal;
A determination step of determining, for each frame or field, whether or not a motion amount between frames or fields of the input image signal is larger than a predetermined value;
When the motion amount between frames or fields of the input image signal is larger than a predetermined value, the image signal subjected to the motion compensation processing is interpolated in the telop region, and the motion compensation processing is invalidated in other regions. Or an image display method for converting the number of frames or the number of fields of the input image signal by interpolating an image signal not subjected to the motion compensation process. Rate conversion means for converting the number of frames or the number of fields of the input image signal by interpolating an image signal subjected to motion compensation processing using a motion vector between frames or fields of the input image signal An image processing apparatus,
Detecting means for detecting a feature amount of one or more telops included in the input image signal;
Determination means for determining, for each frame or field, whether or not a motion amount between frames or fields of the input image signal is larger than a predetermined value;
When the motion amount between frames or fields of the input image signal is larger than a predetermined value, the image signal subjected to the motion compensation processing is interpolated in the telop region, and the motion compensation processing is invalidated in other regions. Or an image processing apparatus for converting the number of frames or the number of fields of the input image signal by interpolating an image signal not subjected to the motion compensation process. A rate conversion step of converting the number of frames or the number of fields of the input image signal by interpolating an image signal subjected to motion compensation processing using a motion vector between frames or fields of the input image signal; An image processing method comprising:
A detection step of detecting a feature amount of one or more telops included in the input image signal;
A determination step of determining, for each frame or field, whether or not a motion amount between frames or fields of the input image signal is larger than a predetermined value;
When the motion amount between frames or fields of the input image signal is larger than a predetermined value, the image signal subjected to the motion compensation processing is interpolated in the telop region, and the motion compensation processing is invalidated in other regions. Or an image processing method for converting the number of frames or the number of fields of the input image signal by interpolating an image signal not subjected to the motion compensation process.