JP2011082932A - Method and apparatus for detecting telop image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method for detecting telop images which detects a region and motion of a telop overlaid on N-M pull-down processed images with a few reference frames. <P>SOLUTION: The present invention relates to a method of detecting a telop image from a synthetic image wherein the telop image is overlaid on a pull-down image obtained by converting a source image of a predetermined frame rate to an image of a higher frame rate than the predetermined frame rate through N-M pull-down processing. The method includes: an extraction step (S101) for extracting an extraction image that is an image of a phase corresponding to the source image from the synthetic image; a motion vector detection step (S105) for detecting a motion vector of the extraction image; an estimated telop vector detection step for detecting an estimated telop vector that can be estimated as a motion vector of a telop from among detected motion vectors; and a determination step (S106) which utilizes features, that the pull-down image has, to determine whether or not a region of an image having the estimated telop vector is a region of the telop image. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a telop image detection method and a telop image detection apparatus for detecting a telop in an image.

  Conventionally, when a moving image is displayed on a liquid crystal display or the like, the frame rate using motion compensation is obtained by analyzing the motion of the moving image as a motion vector in order to reduce motion blur or increase the smoothness of the motion. Conversion has been done.

  For example, when playing a 24 Hz film material according to the NTSC standard, it is played back to 60 Hz by performing 3-2 pull-down to display three 24 Hz images in succession. Since the moving image that has been pulled down 3-2 is non-uniform in comparison with the original image in which the time interval of frame (image) change is 24 Hz, there is an unnatural motion. Therefore, the motion of the moving image can be smoothly displayed by further converting the frame rate, for example, by converting to 120 Hz and reproducing.

  Normally, when converting the frame rate of a 3-2 pull-down moving image, in order to detect the motion, a frame corresponding to the original image of 24 Hz is extracted from the 3-2 pull-down image, and the motion between these frames is extracted. Is detected. The outline will be described with reference to FIG.

  FIG. 1 is a diagram for explaining a general method for frame-converting a 3-2 pull-down moving image. FIG. 1 shows a state in which a pull-down image 101, which is a 60 Hz moving image obtained by 3-2 pull-down of a 24 Hz original image, is frame-converted into a 120 Hz moving image. Here, the hatched circles in the frame in the figure indicate objects that are focused on the movement in the frame.

  As shown in FIG. 1, an original image equivalent frame 102 which is a frame corresponding to an original image of 24 Hz is extracted from a pull-down image 101 to detect a motion. Specifically, in the frame 102 corresponding to the original image, the first, fourth, and sixth frames 111, 112, and 113 are extracted from the first to seventh frames of the pull-down image 101 to detect motion. .

  Then, frame rate conversion is performed on the extracted frames 111, 112, and 113 using motion compensation, and a frame rate converted image 103 that is a 120 Hz moving image with smooth motion is displayed.

  However, in television broadcasting or the like, there is a case where a telop moving at 60 Hz is overlaid on a 3-2 pull-down image in which a 24 Hz image is 3-2 pulled down and converted to 60 Hz. In that case, if the motion between the extracted frames is obtained by extracting the frames as described above with respect to the image on which the telop is overlaid in this way, the telop that moves with uniform motion at 60 Hz is unevenly distributed to 24 Hz. The motion is detected from the thinned out. That is, no uniform motion is detected. Therefore, when frame rate conversion is performed using the movement, an unnatural movement occurs in the time direction in the telop overlaid on the frame-converted image. This will be described with reference to FIG.

  2 and 3 are diagrams for explaining a problem in the case of frame conversion of a 3-2 pull-down moving image. Here, the hatched circles in the frame in FIG. 2 indicate telops (characters). In FIG. 2, as in FIG. 1, it is assumed that the 60 Hz image 201 is a 3-2 pull-down image and the frame 202 is a frame corresponding to a 24 Hz original image.

  As shown in FIG. 2, in the frame 202, the 1st, 4th, and 6th frames 211, 212, and 213 of the 1st to 7th frames in the image 201 are extracted, and between the extracted frames. Motion is detected. Then, frame rate conversion is performed on the extracted frames 211, 212, and 213 using motion compensation, and an image 203 that has been frame rate converted to 120 Hz is displayed.

  In the image 201 of 60 Hz, for example, the circular motion vectors 221 with the diagonal lines indicating the telops between the first to sixth frames are the same. That is, the telop is moving at a constant speed. On the other hand, in the image 203 of 120 Hz that has undergone frame rate conversion, the movement of the circles with diagonal lines indicating telops is discontinuous. This is because the motion vectors of the circles (objects) with diagonal lines indicating telops are different between the frames 211-212, the frames 212-213, and the frames 211-213 extracted from the image 201. That is, the frame rate converted 120 Hz image 203 shows, for example, different motion vectors 222 and 223 with respect to the motion vector 224 indicating the desired constant speed, and the motion of the telop is not constant but discontinuous. I can see it.

  That is, as shown in FIG. 3A, in a 60 Hz image 201, for example, characters (indicated as “character” in the figure) indicating a telop between the first to seventh frames are overlaid on the 60 Hz image 201. Because it is moving at a constant speed. On the other hand, a circle with an oblique line indicating an object moves 3-2 pull-down. Therefore, as shown in FIG. 3B, when the frame rate conversion is performed on the 60 Hz image 201 as a 3-2 pull-down image to the 120 Hz image 203, a diagonal line indicating the object is applied. Although the circle is converted into the smooth image 203, judder (discontinuous movement) occurs in the characters indicating the telop.

  Patent document 1 is disclosed with respect to such a subject. In Patent Document 1, it is determined whether or not each block is a pull-down video, and the motion compensation method is switched to prevent judder in characters indicating a telop.

JP 2005-318612 A

  However, in Patent Document 1, pull-down detection processing is required for each block, and the circuit scale becomes larger than pull-down detection in units of frames. For example, in Patent Document 1, since motion detection and motion compensation are switched for each block, the current frame and at least two or more past frame images are required, and the mounted memory becomes large.

  Therefore, the present invention has been made in view of such a situation, and by using a pull-down detection method in units of frames, an N-M (N and M are integers different from each other) pull-down image with a small number of reference frames. It is an object of the present invention to provide a telop image detection method and a telop image detection apparatus for detecting a region and motion of an overlaid telop.

  In order to achieve the above object, in the telop detection method according to the present invention, an original image composed of images of a predetermined frame rate is converted into the predetermined frame rate by NM (an integer different from N and M) pull-down processing. A telop image detection method for detecting a telop image from a composite image obtained by overlaying a telop image at a high frame rate on a pull-down image converted to an image having a higher frame rate, wherein the original image is converted from the composite image. An extraction image extraction step for extracting an extraction image that is an image having a phase corresponding to the above, a motion vector detection step for detecting a motion vector of the extraction image, and estimation as a motion vector of a telop from the detected motion vectors An estimated telop vector detection step for detecting an estimated telop vector that can be performed; and Using the features down period image has a region of an image having been detected the estimated telop vector comprises a determination step of determining whether or not an area of the telop image.

  As a result, by utilizing the characteristics of the pull-down image, it is possible to detect the telop region and motion overlaid on the NM pull-down image with a small number of reference frames.

  Accordingly, it is possible to realize a telop image detection method for detecting a telop region and motion overlaid on an NM pull-down image with a small number of reference frames using a pull-down detection method in units of frames.

  At this time, the predetermined frame rate may be 24 Hz, the NM pull-down process may be a 2-3 pull-down process, and a frame rate higher than the predetermined frame rate may be 60 Hz.

  Further, the determination step includes a motion vector conversion step of generating a conversion vector obtained by converting the estimated telop vector according to the phase of the extracted image by using periodicity which is a characteristic of the pull-down image. A region of an image having the estimated telop vector translated using the conversion vector generated in the motion vector conversion step is compared with a telop image region in the extracted image temporally previous to the extracted image. By doing so, it may be determined whether the area of the image having the estimated telop vector is a telop image area.

  Here, the determination step further detects the degree of coincidence based on a pixel difference between the region of the image having the estimated telop vector and the telop image region in the extracted image temporally previous to the extracted image. The degree of coincidence that determines that the area of the image having the estimated telop vector is a telop image area when the degree of coincidence of each pixel detected in the coincidence degree detection step and the coincidence degree detection step is equal to or greater than a predetermined value A determination step. In addition, the determination step includes the estimated telop vector when a difference between the conversion vector and a motion vector of a telop image area in the extracted image immediately before the extracted image is smaller than a predetermined value. The image area may be determined as a telop image area.

  Further, the determination step includes an iterative detection step for detecting a periodic repetition of the estimated telop vector in a time direction, and the repetition according to the phase of the extracted image repeats a specific pattern indicating the feature. And an iterative determination step of determining that an area of the image having the estimated telop vector is a telop image area.

  In order to achieve the above object, the telop detection device according to the present invention is configured such that an original image composed of an image having a predetermined frame rate is obtained by performing NM (an integer different from N and M) pull-down processing. A telop image detection device for detecting a telop image from a composite image in which a telop image is overlaid at a high frame rate on a pull-down image converted into an image having a frame rate higher than the frame rate, An extracted image extracting unit that extracts an extracted image that is an image having a phase corresponding to the original image, a motion vector detecting unit that detects a motion vector of the extracted image, and a motion vector of a telop from the detected motion vectors An estimated telop vector detection unit that detects an estimated telop vector that can be estimated as By utilizing the characteristics that have a region of the image having been detected the estimated telop vector and a judging section that judges whether a telop image area.

  With this configuration, it is possible to realize a telop image detection apparatus that detects a telop region and motion overlaid on an NM pull-down image with a small number of reference frames using a pull-down detection method in units of frames.

  The present invention is not only realized as an apparatus, but also realized as an integrated circuit including processing means included in such an apparatus, or realized as a method using the processing means constituting the apparatus as a step. Can be realized as a program for causing a computer to execute, or as information, data, or a signal indicating the program. These programs, information, data, and signals may be distributed via a recording medium such as a CD-ROM or a communication medium such as the Internet.

  According to the present invention, a telop image detection method and a telop image detection device for detecting a telop area and motion overlaid on an NM pull-down image with a small number of reference frames using a pull-down detection method in units of frames are realized. can do. Thereby, for example, a 60 Hz telop region overlaid in a pull-down image and its motion vector can be easily detected with a small number of reference frames.

It is a figure for demonstrating the general method which carries out frame conversion of the moving image pulled down 3-2. FIG. 3 is a diagram for describing a problem when frame conversion is performed on a moving image that is 3-2 pulled down. FIG. 3 is a diagram for describing a problem when frame conversion is performed on a moving image that is 3-2 pulled down. It is a block diagram which shows the structure of the telop detection apparatus in this Embodiment. It is a figure for demonstrating a pull-down state signal. It is a figure for demonstrating the example in case an effective frame extraction part extracts a frame based on a pull-down state signal. It is a figure for demonstrating the method in which a motion vector detection part detects a motion vector. It is a figure for demonstrating the method by which an estimated telop vector detection part detects an estimated telop vector. It is a figure for demonstrating the method by which an estimated telop vector detection part detects an estimated telop vector. It is a figure for demonstrating the method by which an estimated telop vector detection part detects an estimated telop vector. It is a figure for demonstrating the method in which a telop vector detection part detects an estimated telop vector. It is a figure for demonstrating that the motion vector of a telop has periodicity. It is a figure for demonstrating the telop area determination method of a telop vector conversion part. It is a figure for demonstrating the telop area determination method of a telop vector conversion part. It is a figure for demonstrating the telop area determination method of a telop vector conversion part. It is a figure for demonstrating another telop area | region determination method of a telop vector conversion part. It is a flowchart for demonstrating the flow of a process of a telop detection apparatus. It is a block diagram which shows the structure of the telop detection apparatus in this Embodiment 2. It is a figure for demonstrating the method in which a telop area | region determination part determines a telop area | region using the property of the motion vector of a telop. It is a figure which shows TV receivers, such as a liquid crystal display, in which a telop image detection method and a telop image detection apparatus are used.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(Embodiment 1)
FIG. 4 is a block diagram showing a configuration of the telop detection device according to the first embodiment.

  In the telop detection apparatus 300 shown in FIG. 4, an original image composed of images of a predetermined frame rate is an image having a frame rate higher than the predetermined frame rate by NM (an integer different from N and M) pull-down processing. This is a telop detection device that detects a telop image from a composite image in which a telop image is overlaid at a high frame rate on a pull-down image converted to. The telop detection apparatus 300 includes an effective frame extraction unit 301, a cache memory 302, a motion vector detection unit 303, an estimated telop vector detection unit 304, a telop vector conversion unit 305, and a telop area determination unit 306. Hereinafter, NM will be described as 3-2.

  The effective frame extracting unit 301 corresponds to the extracted image extracting unit according to the present invention, and extracts an extracted image that is an image having a phase corresponding to the original image from the synthesized image. Specifically, when the input image is a pull-down image, the effective frame extraction unit 301 stores the current frame cache 3022 when the pull-down state signal indicates a valid frame based on the value of the pull-down state signal input from the previous process. The stored image data is stored in the past frame cache 3021. Thereafter, the valid frame extraction unit 301 stores an input image that is a valid frame in the current frame cache 3022.

  Here, when the input image is a pull-down image, the pull-down state signal is a signal indicating the state of the pull-down image, that is, the phase of the pull-down image. This will be described with reference to the drawings.

  FIG. 5 is a diagram for explaining the pull-down state signal. FIG. 6 is a diagram for explaining an example when the effective frame extraction unit extracts a frame based on the pull-down state signal.

  As shown in FIG. 5A, when the input image is not a pull-down image but a normal 60 Hz image 401, the pull-down state signal indicates a value of zero. On the other hand, as illustrated in FIG. 5B, when the input image is a pull-down image 402 that has been pulled down 3-2, the pull-down state signal indicates a value of 1 to 5, for example, according to the phase of the pull-down image. When the pull-down state signal indicates a value of 1, 2, 3 or 4, 5, the same frame in the original image of 24 Hz is repeated. However, in general, when pull-down detection is performed, 2-2 pull-down is often detected at the same time. However, in the present invention, description is omitted because 2-2 pull-down is not considered.

  In addition, when the input image is a pull-down image, the effective frame extraction unit 301 extracts a frame at the moment when the frame of the original image changes in the pull-down image.

  Specifically, as shown in FIGS. 6A and 6B, the effective frame extraction unit 301 generates a frame of the original image from the 60 Hz pull-down image 501 and the 3-2 pull-down pull-down image. For example, a frame 502 in which the pull-down state signal indicates a value of 1 or 4 is extracted from the input image. The valid frame extraction unit 301 stores the data of the current frame cache 3022 in the past frame cache 3021 and stores the extracted frame (input image) in the current frame cache 3022.

  Note that the frame extracted by the effective frame extraction unit 301 is not limited to the above example, and may be any combination that extracts a frame at the moment when the frame of the 24 Hz original image changes. For example, the value of the pull-down state signal is one of six values (1, 4), (2, 4), (3, 4), (1, 5), (2, 5), and (3, 5) It is only necessary to extract two frames from the pull-down image indicating the combination.

  The cache memory 302 includes a past frame cache 3021 and a current frame cache 3022, and stores an image area (typically a frame) stored by the valid frame extraction unit 301.

  The past frame cache 3021 is a cache memory in which past frames are stored among images (frames) extracted by the valid frame extraction unit 301 and stored in the cache memory 302. On the other hand, the current frame cache 3022 is a cache memory in which the current frame is stored among images (frames) extracted by the valid frame extraction unit 301 and stored in the cache memory 302.

  The motion vector detection unit 303 corresponds to the motion vector detection unit according to the present invention, and detects a motion vector of the extracted image. Specifically, the motion vector detection unit 303 moves the motion vector from the current frame (current frame) to the past frame (past frame) between the two images (frames) extracted by the effective frame extraction unit 301. Is detected.

  This will be described with reference to FIG. 3 and FIG. FIG. 7 is a diagram for explaining a method by which the motion vector detection unit detects a motion vector.

  When detecting a motion vector, the motion vector detection unit 303 reads an image area necessary for motion vector detection from the cache memory 302 and uses a general motion vector detection method such as a block matching method for each pixel, or The motion vector is detected for each block.

  For example, the motion vector detection unit 303 moves the motion vector when the object 603 in the past frame 602 held in the past frame cache 3021 moves to the object 604 in the current frame 601 held in the current frame cache 3022. Is detected. That is, a vector 607 indicating the block 606 in the past frame 602 that matches the detection block 605 in the current frame 601 is detected as a motion vector. However, the motion vector detection method is not limited to the block matching method.

  The estimated telop vector detection unit 304 corresponds to the estimated telop vector detection unit according to the present invention, and detects an estimated telop vector that can be estimated as a motion vector of the telop from the detected motion vectors. Specifically, the estimated telop vector detection unit 304 detects an estimated telop vector that can be estimated as the motion vector of the telop from the motion vectors detected by the motion vector detection unit 303.

  This will be described with reference to the drawings. 8 to 11 are diagrams for explaining a method in which the estimated telop vector detection unit detects the estimated telop vector.

  In FIG. 8, a horizontally moving telop 701 is shown, and the telop 701 moves with a motion vector 702 indicated by a horizontal arrow. In this case, the estimated telop vector detection unit 304 detects the motion vector 702 of the telop 701.

  As one method, the estimated telop vector detection unit 304 calculates the average value of the motion vectors detected by the motion vector detection unit 303 for each band region 703 having a predetermined width. Then, among the calculated average values of the motion vectors, a motion vector indicating the motion in the horizontal direction is set as an estimated telop vector.

  As another method, the estimated telop vector detection unit 304 creates a motion vector histogram for each band region 703 and detects a motion vector concentrated in the horizontal motion as an estimated telop vector.

  FIG. 9 is a diagram illustrating an example in which a motion vector histogram for each band region is created. Here, the horizontal axis indicates the horizontal component of the vector, and the vertical axis indicates the frequency of the vector. In this case, the estimated telop vector detection unit 304 detects, as an estimated telop vector, a class vector in the histogram shown in FIG. 9 that maximizes the frequency of horizontal movement. Specifically, in class 811, since the frequency is the maximum and the vector value is 8, the estimated telop vector detection unit 304 detects the estimated telop vector as 8.

  Note that the estimated telop vector detection unit 304 may use a weighted average of the values of class vectors around the class having the highest frequency as an estimated telop vector in order to suppress noise. Specifically, as shown in FIG. 9, the weighted average of the third class having the largest class 811 including, for example, the front and rear one class (class 812 and class 813) is calculated as the third class. The value of the obtained motion vector may be determined as the estimated telop vector.

  Further, in order to improve the accuracy of the detected estimated telop vector, it is used for calculation using the feature on the image of the telop when calculating the average value of the motion vector described above or creating a histogram of the motion vector for each band area. The vector may be limited. For example, the features on the telop image include information such as a strong edge on the image and a single color, and the vector used for the calculation may be limited using such information. Specifically, an edge is detected in an image 901 including a telop area 903 as in the telop image example shown in FIG. In that case, a strong edge is detected as a telop area 904 on the edge detection image 902 as shown in the edge detection image 902 in which the edge of FIG. A region 903 is detected with high accuracy.

  Using the above method, the estimated telop vector detection unit 304 detects an estimated telop vector from the motion vectors detected by the motion vector detection unit 303.

  In the above description, it has been described that the direction of the motion vector of the estimated telop is detected as the horizontal direction, but the present invention is not limited to this. It may be detected as being in a vertical direction or an oblique direction. For example, in the telop image 1001 moving in the vertical direction shown in FIG. 11, a vector moving in the vertical direction is detected as the estimated telop vector 1003 of the telop 1002. In that case, the detection may be performed in the same manner as described above for each detection band 1004 having a width in the horizontal direction.

  The method by which the estimated telop vector detection unit 304 detects the estimated telop vector is not limited to these methods. It is only necessary that the estimated telop vector can be detected from the motion vector detected by the motion vector detection unit 303.

  The telop vector conversion unit 305 corresponds to the motion vector conversion unit according to the present invention, and converts the estimated telop vector according to the phase of the extracted image by using the periodicity that is a characteristic of the pull-down image. Generate a transformation vector. Specifically, the telop vector conversion unit 305 estimates the motion vector of the telop in the current frame from the past estimated telop vector using the feature of the motion vector of the telop. Specifically, the telop vector conversion unit 305 multiplies the estimated telop vector detected in the past based on the periodicity that is a characteristic of the motion vector of the telop to multiply the estimated telop of the current frame. A converted telop vector converted into a motion vector is obtained.

  FIG. 12 is a diagram for explaining that the motion vector of the telop has periodicity.

  In FIG. 12, it is assumed that the telop moves uniformly at 60 Hz. When extracting only a frame corresponding to an original image of 24 Hz from the 3-2 pull-down image, the size of the motion vector of the telop periodically changes at a certain rate between the extracted frames. For example, when a frame in which a pull-down state signal indicates a value of 1 or 4 is extracted from a 3-2 pull-down image, the size of the motion vector of the telop is 2: 3: 2: 3 as shown in FIG. Repeat with the ratio That is, the size of the motion vector of the telop has periodicity.

  By utilizing this property, the telop area can be easily detected. That is, the telop vector conversion unit 305 can obtain the motion vector of the telop in the current frame, that is, the converted telop vector, from the past estimated telop vector by using the above property. That is, the telop vector conversion unit 305 multiplies the estimated telop vector detected one frame before by a predetermined magnification according to the pull-down state of the extracted frame, and calculates the motion vector (conversion telop vector) estimated in the current frame. ). Specific examples will be described below.

  For example, when the extracted frame is a frame whose pull-down state signal indicates (1, 4), the size of the motion vector of the telop has a periodicity of repeating 2: 3: 2: 3. Therefore, the conversion telop vector can be calculated by multiplying the estimated telop vector detected one frame before (the frame whose pull-down state signal is 4) by 2/3 in the frame whose pull-down state signal is 1. Similarly, the conversion telop vector can be calculated by multiplying the estimated telop vector detected one frame before (the frame whose pull-down state signal is 1) by 3/2 in the frame where the pull-down state signal is 4.

  For example, when the extracted frame is a frame whose pull-down state signal indicates (2, 4), the size of the motion vector of the telop has a periodicity of repeating 3: 2: 3: 2. Therefore, the conversion telop vector can be calculated by multiplying the estimated telop vector detected one frame before (the frame where the pull-down state signal is 4) by 3/2 in the frame where the pull-down state signal is 2. Similarly, the conversion telop vector can be calculated by multiplying the estimated telop vector detected one frame before (the frame where the pull-down state signal is 2) by 2/3 in the frame where the pull-down state signal is 4.

  For example, when the extracted frame is a frame whose pull-down state signal indicates (3, 4), the size of the motion vector of the telop has a periodicity of repeating 4: 1: 4: 1. Therefore, the conversion telop vector can be calculated by multiplying the estimated telop vector detected one frame before (the frame where the pull-down state signal is 4) four times in the frame where the pull-down state signal is 3. Similarly, the conversion telop vector can be calculated by multiplying the estimated telop vector detected one frame earlier (the frame where the pull-down state signal is 3) by ¼ for a frame whose pull-down state signal is 4.

  For example, when the extracted frame is a frame whose pull-down state signal indicates (1, 5), the size of the motion vector of the telop has a periodicity of repeating 1: 4: 1: 4. Therefore, the conversion telop vector can be calculated by multiplying the estimated telop vector detected one frame before (the frame where the pull-down state signal is 5) by 1/4 in the frame where the pull-down state signal is 1. Similarly, the conversion telop vector can be calculated by multiplying the estimated telop vector detected one frame before (the frame where the pull-down state signal is 1) four times in the frame where the pull-down state signal is 5.

  For example, when the extracted frame is a frame whose pull-down state signal indicates (2, 5), the size of the motion vector of the telop has a periodicity of repeating 2: 3: 2: 3. . Therefore, the conversion telop vector can be calculated by multiplying the estimated telop vector detected one frame before (the frame where the pull-down state signal is 5) by 2/3 in the frame where the pull-down state signal is 2. Similarly, the conversion telop vector can be calculated by multiplying the estimated telop vector detected one frame before (the frame where the pull-down state signal is 2) by 3/2 in the frame where the pull-down state signal is 5.

  For example, when the extracted frame is a frame whose pull-down state signal indicates (3, 5), the size of the motion vector of the telop has a periodicity of repeating 3: 2: 3: 2. . Therefore, the conversion telop vector can be calculated by multiplying the estimated telop vector detected one frame before (the frame whose pull-down state signal is 5) by 3/2 in the frame whose pull-down state signal is 3. Similarly, the conversion telop vector can be calculated by multiplying the estimated telop vector detected 1 frame before (the frame where the pull-down state signal is 3) by 2/3 in the frame where the pull-down state signal is 5.

  In this way, the telop vector conversion unit 305 calculates a conversion telop vector from the estimated telop vector of the previous frame using the nature of the motion vector of the telop.

  The telop area determination unit 306 corresponds to the determination unit according to the present invention, and uses the characteristics of the pull-down image to determine whether the area of the image having the estimated telop vector detected is a telop image area. Determine. Specifically, the telop area determination unit 306 determines a telop area using the converted telop vector converted by the telop vector conversion unit 305. Hereinafter, this telop determination method will be described with an example.

  FIG. 13A, FIG. 13B, and FIG. 14 are diagrams for explaining the telop area determination method of the telop vector conversion unit.

  13A and 13B, when an image of a predetermined area (including characters and telops) in the current frame is translated by the converted telop vector and an image including the telop area in the past frame matches, It is determined that the area image is an image indicating the telop area. Therefore, the degree of coincidence is calculated.

  Specifically, the degree of coincidence between the telop 1203 on the current frame 1201 shown in FIG. 13A and the telop 1204 of the past frame 1202 is determined using the converted telop vector. At that time, as shown in FIG. 13B, an image 1206 obtained by translating the current frame 1201 with the converted telop vector 1205 and an image (telop) of the past frame 1202 are compared to determine the degree of coincidence.

  Here, as a method of calculating the degree of coincidence, for example, a luminance difference in units of pixels may be calculated, and the reciprocal of the absolute value may be used as the degree of coincidence. Further, the maximum value that can be taken by the difference (255 when the luminance is 8 bits) −the absolute difference value may be used as the degree of coincidence. As another coincidence degree detection method, the sum of absolute values of pixel differences may be calculated for each block having a predetermined size, and the coincidence degree may be determined in the same manner as the coincidence degree for each pixel.

  In this way, the telop area determination unit 306 determines the telop area using the converted telop vector converted by the telop vector conversion unit 305. Specifically, the telop area determination unit 306 compares the area estimated as the telop area in the current frame with the telop area in the past frame using the converted telop vector, and the degree of coincidence calculated as described above is predetermined. If the value is larger than the value of, the estimated area is detected as a telop area.

  FIG. 14 is a diagram summarizing FIGS. 13A and 13B.

  In FIG. 14, the first, third, and sixth frames are extracted from the 3-Hz pulled-down 60 Hz image 201 in FIG. 3A, and between characters (telops) included in the first and third frames. A motion vector is detected. In FIG. 14, the degree of coincidence between the region including the character 1208 in the third frame corresponding to the current frame and the region including the character (telop) 1207 in the past frame, which are translated using the converted telop vector, is shown. Comparing. As a result, for example, since the pixel difference is small, the degree of coincidence becomes larger than a predetermined value, and it is shown that the area including the character 1208 in the third frame corresponding to the current frame is determined as the telop area.

  The telop area determination method is not limited to the above method. FIG. 15 is a diagram for explaining another telop area determination method of the telop vector conversion unit.

  As another telop area determination method, for example, the difference between the motion vector for each pixel or block detected by the motion vector detection unit 303 and the converted telop vector converted by the telop vector conversion unit 305 is calculated. If the absolute value of the difference is smaller than the predetermined value, it is determined that the motion is like a telop and the region including the detected motion vector is determined as a telop region.

  Specifically, the motion vector 1303 detected by the motion vector detection unit 303 is compared with the converted telop vector 1305 on the current frame 1300. In FIG. 15, since the difference between the motion vector 1303 of the telop 1301 and the converted telop vector 1305 is small, the telop area determination unit 306 detects an area including the motion vector 1303 as a telop area. On the other hand, since the motion vector 1304 of the object 1302 has a large difference from the converted telop vector 1305, the telop area determination unit 306 does not determine that the area including the motion vector 1304 is a telop area.

  In this way, the telop area determination unit 306 determines the telop area using the converted telop vector converted by the telop vector conversion unit 305.

  Note that when determining the telop area, the telop area may be limited using features on the image of the telop in order to improve detection accuracy. For example, since the edge of an image is strong in a normal telop image, the telop area may be limited using the strength of the edge. In addition, since there are many telop images or two colors including a border, a portion where colors or gradations are concentrated may be detected as a telop image.

  As described above, the telop detection device 300 according to the first embodiment is configured.

  Next, a processing flow of the telop detection apparatus 300 configured as described above will be described.

  FIG. 16 is a flowchart for explaining the flow of processing of the telop detection apparatus.

  First, it is assumed that a 60 Hz input image (hereinafter referred to as a composite image) pulled down 3-2 and a pull down state signal corresponding to the input image are input to the telop detection apparatus 300. Then, the composite image and the pull-down state signal are input to the effective frame extraction unit 301.

  Next, the effective frame extraction unit 301 sequentially extracts at least two images (frames) corresponding to the original image of 24 Hz from the synthesized image based on the value of the pull-down state signal (S101). Then, the effective frame extraction unit 301 sequentially stores the extracted images in the cache memory 302.

  Next, the motion vector detection unit 303 detects the motion vector of the image extracted by the effective frame extraction unit 301 and stored in the cache memory 302 (S103). Specifically, the motion vector detection unit 303 calculates a motion vector from the current frame (current frame) to the past frame (past frame) between the two images (frames) extracted by the effective frame extraction unit 301. To detect.

  Next, the estimated telop vector detection unit 304 estimates a telop motion vector from the extracted motion vector (S105). Specifically, the estimated telop vector detection unit 304 detects an estimated telop vector that can be estimated as the motion vector of the telop from the motion vectors detected by the motion vector detection unit 303.

  Next, the telop vector conversion unit 305 estimates the motion vector of the telop in the current frame using the feature of the motion vector of the telop from the past estimated telop vector. Specifically, the telop vector conversion unit 305 estimates the telop in the current frame by multiplying the estimated telop vector detected in the past by a predetermined magnification based on the periodicity that is a feature of the motion vector of the telop. Convert to motion vector.

  Next, the telop area determination unit 306 determines a telop area using the converted telop vector converted by the telop vector conversion unit 305 (S107).

  As described above, the telop detection apparatus 300 performs processing for detecting a telop area.

  As described above, according to the telop image detection method and telop image detection apparatus of the first embodiment, when motion vector detection is performed using the leading images of 2-3 pulldown images, The telop area is detected by using the feature of the motion vector of the telop that the vector repeats 2: 3: 2: 3. That is, it is possible to detect a region having a small pixel difference in a vector 2/3 times or 3/2 times the motion vector of the telop one frame before as a 60 Hz telop region. As described above, in the telop detection apparatus according to the first embodiment, one past frame can be detected, and the past frame can be detected with a smaller number of frames as compared with the conventional technique.

  As described above, according to the telop image detection method and the telop image detection apparatus of the first embodiment, by using the pull-down image feature, the pull-down detection method in units of frames, for example, with as few as two reference frames. It is possible to detect a telop area and motion overlaid on a 3 pull-down image.

  In the above description, a typical example of a pull-down image obtained by converting an original image composed of a 24 Hz image into a 60 Hz image has been described. It is sufficient that the frame rate of the image constituting the original image is smaller than the frame rate of the image constituting the pull-down image subjected to frame rate conversion. The pull-down method is not limited to 2-3, but may be NM (N is a positive integer different from M).

(Embodiment 2)
In the first embodiment, the method for detecting a telop area using the motion vector feature of the telop has been described. However, the present invention is not limited to this. In the second embodiment, another method for detecting a telop area using the motion vector feature of the telop will be described.

  FIG. 17 is a block diagram illustrating a configuration of the telop detection device according to the second embodiment. A telop detection apparatus 1400 illustrated in FIG. 17 includes an effective frame extraction unit 301, a cache memory 302, a motion vector detection unit 303, an estimated telop vector detection unit 304, and a telop area determination unit 1406. Note that elements similar to those in FIG. 4 are denoted by the same reference numerals, and detailed description thereof is omitted.

  The telop detection apparatus 1400 shown in FIG. 17 differs from the telop detection apparatus 300 according to Embodiment 1 in that the telop vector conversion unit 305 is not provided and the configuration of the telop area determination unit 1406 is different.

  The telop area determination unit 1406 determines the telop area using the property of the motion vector of the telop.

  Here, it is assumed that the telop moves uniformly at 60 Hz. When only a frame corresponding to a 24 Hz original image is extracted from the 3-2 pull-down image, which is a composite image, the size of the motion vector of the telop is cyclic at a certain rate between the extracted frames. To change. The telop area determination unit 1406 determines the telop area using such a property of the motion vector of the telop, that is, the periodicity.

  FIG. 18 is a diagram for explaining a method in which the telop area determination unit determines the telop area using the property of the motion vector of the telop. In FIG. 18, the 1st, 3rd, and 6th frames are extracted from the 3-Hz pulled-down 60 Hz image 201 in FIG. 3A, and between characters (telops) included in each frame (character 1501- 1502 and characters 1502-1503) are detected. Specifically, when a frame whose pull-down state signal indicates 1 or 4 is extracted from the 3-2 pull-down image, the size of the motion vector (MV1, MV2,...) Between the telops (characters) is 3 : 2: 3: 2: 3 .. Has a periodicity that repeats at a ratio. Using this property, the telop area determination unit 1406 can easily detect the telop area.

  That is, if the estimated telop vector detected for each block line by the estimated telop vector detection unit 304 has the above-described periodicity for a predetermined period T or more, the telop area determination unit 1406 has a telop for that line. Judge as the line to be

  For example, it is assumed that the two frames (images) extracted by the effective frame extraction unit 301 are frames indicating the pull-down state signals (1, 4). In that case, when the estimated telop vector detected by the estimated telop vector detection unit 304 has a periodicity that repeats 2: 3: 2: 3..., The telop area determination unit 1406 sets the line as a line where the telop exists. judge.

  Further, for example, it is assumed that the frame extracted by the effective frame extraction unit 301 is a frame indicating the pull-down state signal (2, 4). In that case, when the estimated telop vector detected by the estimated telop vector detection unit 304 has a periodicity that repeats 3: 2: 3: 2,..., The telop area determination unit 1406 sets the line as a line where the telop exists. judge.

  For example, assume that the frame extracted by the effective frame extraction unit 301 is a frame indicating the pull-down state signal (3, 4). In that case, when the estimated telop vector detected by the estimated telop vector detection unit 304 has a periodicity that repeats 4: 1: 4: 1..., The telop area determination unit 1406 sets the line as a line where the telop exists. judge.

  For example, assume that the frame extracted by the effective frame extraction unit 301 is a frame indicating the pull-down state signal (1, 5). In that case, when the estimated telop vector detected by the estimated telop vector detection unit 304 has a periodicity that repeats 1: 4: 1: 4..., The telop area determination unit 1406 sets the line as a line in which the telop exists. judge.

  For example, assume that the frame extracted by the effective frame extraction unit 301 is a frame indicating the pull-down state signal (2, 5). In that case, when the estimated telop vector detected by the estimated telop vector detection unit 304 has a periodicity that repeats 2: 3: 2: 3..., The telop area determination unit 1406 sets the line as a line where the telop exists. judge.

  For example, assume that the frame extracted by the valid frame extraction unit 301 is a frame indicating the pull-down state signal (3, 5). In that case, when the estimated telop vector detected by the estimated telop vector detection unit 304 has a periodicity that repeats 3: 2: 3: 2,..., The telop area determination unit 1406 sets the line as a line where the telop exists. judge.

  In this way, the telop area determination unit 1406 determines the telop area using the property of the motion vector of the telop.

  Note that when determining the telop area, the telop area may be limited using features on the image of the telop in order to improve detection accuracy. For example, since the edge of an image is strong in a normal telop image, the telop area may be limited using the strength of the edge. Also, since there are many telop images or two colors including a border, a portion where colors or gradations are concentrated may be detected as a telop image.

  As described above, the telop detection device 1400 according to the second embodiment is configured.

  Note that the processing flow of the telop detection apparatus 1400 is the same as that of the telop detection apparatus 300 of the first embodiment when the above-described telop area detection method is taken into account, and thus the description thereof is omitted.

  As described above, according to the telop image detection method and telop image detection apparatus of the second embodiment, when performing motion vector detection using the respective leading images of 2-3 pulldown images, The telop area is detected by using the feature of the motion vector of the telop that the vector repeats 2: 3: 2: 3. That is, when the telop motion vector repeats at 2: 3, the area can be detected as a telop area of 60 Hz. As described above, in the telop detection device according to the second embodiment, one past frame can be detected, and the number of past frames can be detected with a smaller number compared to the conventional technique.

  As described above, according to the telop image detection method and telop image detection apparatus of the second embodiment, by using the pull-down image feature, the number of pull-down detection methods in units of frames is reduced to, for example, as few as two. It is possible to detect a telop area and motion overlaid on the 2-3 pull-down image in the reference frame.

  That is, the telop image detection apparatus according to the present invention is configured such that an original image composed of images of a predetermined frame rate is higher than the predetermined frame rate by NM (an integer different from N and M) pull-down processing. A telop detection device for detecting a telop image from a composite image in which a telop image is overlaid at a high frame rate on a pull-down image converted into an image of the image, and having a phase corresponding to the original image from the composite image An extracted image extracting unit that extracts the extracted image, a motion vector detecting unit that detects a motion vector of the extracted image, and an estimated telop vector that can be estimated as a motion vector of the telop from the detected motion vectors An estimated telop vector detection unit to detect and a feature of the pull-down image are used. , Regions of the image with the estimated telop vector and a judging section that judges whether a telop image area.

  According to this configuration, a telop image detection method and a telop image detection device that detect a telop region and motion overlaid on an NM pull-down image with a small number of reference frames using a pull-down detection method in units of frames are realized. can do.

  Thereby, in the telop image detection device, the memory to be mounted can be reduced, so that an increase in circuit scale can be suppressed. Therefore, for example, it can be mounted on a TV receiver such as a liquid crystal display as shown in FIG. In such a TV receiver such as a liquid crystal display, when a moving image pulled down 3-2 is converted into a frame rate and displayed on a liquid crystal display or the like, a telop overlaid at 60 Hz is displayed with natural motion. Is possible.

  The telop image detection method and telop image detection apparatus of the present invention have been described above using the embodiment, but the present invention is not limited to this embodiment. Unless it deviates from the meaning of this invention, the form which carried out the various deformation | transformation which those skilled in the art can think to this embodiment, and the structure constructed | assembled combining the component in different embodiment is also contained in the scope of the present invention. .

  INDUSTRIAL APPLICABILITY The present invention can be used for a telop image detection method and a telop image detection device. In particular, when a 60 Hz moving image that has been pulled down 3-2 is converted into a frame rate and displayed on a liquid crystal display or the like, a telop overlaid at 60 Hz is naturally displayed. The present invention can be used for a telop image detection method and a telop image detection device that can display with a simple motion.

101, 402, 501 Pull-down image 102 Frame corresponding to original image 103 Frame rate converted image 111, 112, 113, 211, 212, 213 Frame 201, 202, 203, 401, 901, 1206 Image 222, 224, 702, 1303, DESCRIPTION OF SYMBOLS 1304 Motion vector 300, 1400 Telop detection apparatus 301 Effective frame extraction part 302 Cache memory 303 Motion vector detection part 304 Estimation telop vector detection part 305 Telop vector conversion part 306, 1406 Telop area determination part 502 Frame 601, 1201, 1300 Current frame 602 1202 Past frame 603, 604, 1302 Object 605 Detection block 606 Block 607 Vector 701, 1002, 1203, 1204, 13 01 Telop 703 Band area 811, 812, 813 Class 902 Edge detection image 903, 904 Telop area 1001 Telop image 1003 Estimated telop vector 1004 Detection band 1205, 1305 Converted telop vector 1208, 1501, 1502, 1503 Character 3021 Past frame cache 3022 Current frame cache

Claims (15)

On the pull-down image in which the original image composed of the image of the predetermined frame rate is converted into an image having a frame rate higher than the predetermined frame rate by NM (an integer different from N and M) pull-down processing, A telop image detection method for detecting a telop image from a composite image in which a telop image is overlaid at a high frame rate,
An extracted image extraction step of extracting an extracted image that is an image of a phase corresponding to the original image from the synthesized image;
A motion vector detection step of detecting a motion vector of the extracted image;
An estimated telop vector detection step of detecting an estimated telop vector that can be estimated as a telop motion vector from the detected motion vectors;
A telop image detection method comprising: determining whether an area of the image having the detected estimated telop vector is a telop image area by using a feature of the pull-down image. The predetermined frame rate is 24 Hz;
The NM pull-down process is a 2-3 pull-down process.
The telop image detection method according to claim 1, wherein a frame rate higher than the predetermined frame rate is 60 Hz. The determination step includes
A motion vector conversion step of generating a conversion vector obtained by converting the estimated telop vector according to the phase of the extracted image by using periodicity which is a characteristic of the pull-down image;
A region of an image having the estimated telop vector translated using the conversion vector generated in the motion vector conversion step is compared with a telop image region in the extracted image temporally previous to the extracted image. The telop image detection method according to claim 1, wherein it is determined whether or not an area of the image having the estimated telop vector is a telop image area. The determination step further includes:
A degree of coincidence detection step of detecting a degree of coincidence based on a pixel difference between an area of the image having the estimated telop vector and a telop image area in the extracted image temporally previous to the extracted image;
A coincidence degree determination step of determining that an area of the image having the estimated telop vector is a telop image area when the coincidence degree of each pixel detected in the coincidence degree detection step is equal to or greater than a predetermined value. Item 4. The telop image detection method according to Item 3. The determination step includes
When the difference between the conversion vector and the motion vector of the telop image area in the extracted image immediately before the extracted image is smaller than a predetermined value, the area of the image having the estimated telop vector is determined as the telop image area. The telop image detection method according to claim 3. The determination step includes
Iterative detection step of detecting periodic repetition of the estimated telop vector in the time direction;
And a repetition determining step of determining that an area of the image having the estimated telop vector is a telop image area when the repetition according to the phase of the extracted image repeats a specific pattern indicating the feature. 3. The telop image detection method according to 1 or 2. On the pull-down image in which the original image composed of the image of the predetermined frame rate is converted into an image having a frame rate higher than the predetermined frame rate by NM (an integer different from N and M) pull-down processing, A telop image detection device for detecting a telop image from a composite image in which a telop image is overlaid at a high frame rate,
An extracted image extraction unit that extracts an extracted image that is an image having a phase corresponding to the original image from the synthesized image;
A motion vector detection unit for detecting a motion vector of the extracted image;
An estimated telop vector detection unit that detects an estimated telop vector that can be estimated as a telop motion vector from the detected motion vectors;
A telop image detection apparatus comprising: a determination unit that determines whether or not a region of an image having the detected estimated telop vector is a telop image region using a characteristic of the pull-down image. The predetermined frame rate is 24 Hz;
The NM pull-down process is a 2-3 pull-down process.
The telop image detection apparatus according to claim 7, wherein a frame rate higher than the predetermined frame rate is 60 Hz. The determination unit
A motion vector conversion unit that generates a conversion vector obtained by converting the estimated telop vector according to the phase of the extracted image by using periodicity that is a characteristic of the pull-down image;
A region of an image having the estimated telop vector translated using the conversion vector generated by the motion vector conversion unit is compared with a telop image region in the extracted image temporally previous to the extracted image. The telop image detection apparatus according to claim 7, wherein the image area having the estimated telop vector is determined to be a telop image area. The determination unit further includes:
A degree of coincidence detection unit that detects a degree of coincidence based on a pixel difference between an area of the image having the estimated telop vector and a telop image area in the extracted image temporally previous to the extracted image;
A coincidence degree determination unit that determines that an area of an image having the estimated telop vector is a telop image area when the degree of coincidence of each pixel detected in the coincidence degree detection step is a predetermined value or more; Item 10. The telop image detection device according to Item 9. The determination unit
When the difference between the conversion vector and the motion vector of the telop image area in the extracted image temporally previous to the extracted image is smaller than a predetermined value, the area of the image having the estimated telop vector is defined as a telop image area. The telop image detection device according to claim 9. The determination unit
An iterative detection unit that detects periodic repetition of the estimated telop vector in the time direction;
An iterative determination unit that determines that an area of an image having the estimated telop vector is a telop image area when the repetition according to the phase of the extracted image repeats a specific pattern indicating the feature. The telop image detection device according to 7 or 8. A television receiver comprising the telop image detection device according to any one of claims 7 to 12. On the pull-down image in which the original image composed of the image of the predetermined frame rate is converted into an image having a frame rate higher than the predetermined frame rate by NM (an integer different from N and M) pull-down processing, An integrated circuit for detecting a telop image from a composite image overlaid with a telop image at a high frame rate,
An extracted image extraction unit that extracts an extracted image that is an image having a phase corresponding to the original image from the synthesized image;
A motion vector detection unit for detecting a motion vector of the extracted image;
An estimated telop vector detection unit that detects an estimated telop vector that can be estimated as a telop motion vector from the detected motion vectors;
An integrated circuit comprising: a determination unit that determines whether a region of an image having the detected estimated telop vector is a telop image region using a feature of the pull-down image; On the pull-down image in which the original image composed of the image of the predetermined frame rate is converted into an image having a frame rate higher than the predetermined frame rate by NM (an integer different from N and M) pull-down processing, A program for detecting a telop image from a composite image in which a telop image is overlaid at a high frame rate,
An extracted image extraction step of extracting an extracted image that is an image of a phase corresponding to the original image from the synthesized image;
A motion vector detection step of detecting a motion vector of the extracted image;
An estimated telop vector detection step of detecting an estimated telop vector that can be estimated as a telop motion vector from the detected motion vectors;
A program for causing a computer to execute a determination step of determining whether or not a region of an image having the detected estimated telop vector is a region of a telop image using characteristics of the pull-down image.

Images