JP5188272B2 - Video processing apparatus and video display apparatus - Google Patents

Description

  The present invention relates to a technique for detecting a motion vector of a video and performing motion correction type frame rate conversion, and more particularly to a technique for frame rate conversion suitable for a case where a character telop moving in a predetermined direction is included in a video.

  In order to improve moving image performance, a technique for generating an interpolation frame from a motion vector of a video and converting the frame rate of an input video signal is conventionally known. In such frame rate conversion, when a motion vector of a video is erroneously detected, a video that does not exist in the original video appears as noise (hereinafter referred to as video failure), which causes image quality deterioration. In particular, character information represented by character telop in a movie or character telop scrolling that flows at the end of a program moves (or does not move) completely different from the background image. Is difficult, and video breakdown is likely to occur. In addition, because it is a place where viewers can easily watch, the recognition rate of bankruptcy is also high.

  As a conventional technique for detecting a region including a character telop in a video in a frame rate conversion technique, for example, a technique described in Patent Document 1 is known.

JP 2003-271970 A

  Since Patent Document 1 obtains the character telop area and the amount of motion by block matching processing that is also applied to a normal video (object), there is a possibility that the character cannot be accurately distinguished from other objects. There is. Patent Document 1 does not consider the case where there is a character telop having a plurality of movements in the video.

  The present invention provides a frame rate conversion technique with higher image quality, that is, reduced video failure even when character telop is included in the video.

  The present invention detects a feature amount related to a character telop in an input video signal to detect the character telop region and motion information, and uses the detected character telop motion information for the character telop region. The present invention is characterized in that an interpolation process different from other parts is performed.

  Here, the character telop area and motion are detected by a first histogram indicating the frequency of the number of lines corresponding to the motion of the character telop in one frame and a second histogram indicating a histogram of character motion information in line units. May be.

  In addition, the input video signal includes a video having a first motion cycle (for example, 24 (25) frames / field per second) and a second motion cycle (for example, 60 (50) per second) shorter than the first motion cycle. In the case of including character information having (frame / field), the processing for frame rate conversion may be stopped.

  According to the present invention, even when a character telop is included in a video, it is possible to perform frame rate conversion processing with higher image quality (reducing video breakdown).

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram showing an example of the configuration of a video display apparatus according to the first embodiment of the present invention.

  Hereinafter, the configuration and operation of the present embodiment will be described by taking as an example the case of converting the frame rate of an image as shown in FIG. 3 having two types of motion character telops in the image. In FIG. 3, it is assumed that the character telops 301 and 302 are moving at 3 pix / frame (3ppf) and 15 pix / frame (15ppf) in the direction of the arrows in the drawing, respectively.

  In FIG. 1, an input signal 101 is first converted into a luminance signal Y and a color difference signal CbCr by an RGB / YCbCr converter 102. The luminance signal Y and the color difference signal CbCr are input to the character telop detection unit 106, input to the memory I / F unit 114, temporarily stored in the image memory 115, and then output as a one-frame delay signal 105. .

  The line memory unit 103 outputs a signal obtained by delaying the input signal 101 (current frame signal 104) and the 1-frame delay signal 105 by 1 line to the video unit motion detection unit 111 and the interpolation frame generation unit 113, respectively.

  Only the luminance signal Y of the current frame signal 104 and the previous frame signal 105 (hereinafter sometimes referred to as the previous frame) is input to the character telop detection unit 106 to detect the character telop portion in the input image and to detect the character telop Are output for selection interpolation enable signals 109 and 110 for performing interpolation processing separately from others, and motion amounts 107 and 108 which are scroll speeds of character telops. In this embodiment, each character telop, for example, the character telops 301 and 302 shown in FIG. 3 is supported in order to perform good interpolation processing for two types of character telops having different motions in one image. Two selection interpolation enable signals 109 and 110 and two motion amounts 107 and 108 are configured to be output. Of course, it may be configured to output two or more. If there is one type of character telop, the selection interpolation enable signal and the motion amount may be output one by one.

  Only the luminance signal Y delayed by one line sentence in the line memory is input to the video part motion detection unit 111, and the motion vector 112 of the whole video including the character part is detected, and the detected motion vector 112 is interpolated. Output to the frame generation unit 113. For example, in the case where an interpolation frame is inserted between two temporally continuous frames, that is, between the current frame and the previous frame, the motion vector 112 first shows a straight line passing through a certain pixel of interest (interpolation pixel) in the interpolation frame. Set more than one. The straight line is set in a predetermined rectangular area provided in each of two frames (current frame and previous frame) before and after the interpolation frame. For each of a plurality of straight lines set in the area, the difference between the pixels of the two frames through which the straight line passes is calculated. Then, a straight line having the smallest difference is set as a motion vector corresponding to the target pixel (interpolation pixel). Here, the details of the motion vector detection method in the video portion motion detection unit 111 are known in, for example, Japanese Patent Application Laid-Open No. 2006-165602 and Japanese Patent Application Laid-Open No. 2007-082030, and the description thereof is omitted. In the present embodiment, a block matching method, a gradient method, a phase correlation method, and the like can be applied as a method for detecting motion of an image in addition to the methods described in these documents.

  The interpolation frame generation unit 113 uses the motion vector 112 from the video part motion detection unit 111 for the video part (other than the text telop part) and uses the motion amount 107 and 108 for the text telop part. An interpolation frame is generated by performing interpolation processing using the motion vector of the telop portion. It is assumed that the region to which the motion amounts 107 and 108 are applied, that is, the character shine telop region, is designated by the selective interpolation enable signals 109 and 110 output from the character telop detection unit 106.

  In the interpolation process, for example, when an interpolation frame is inserted between the current frame and the previous frame that are temporally continuous, the pixels of the two frames indicated by the motion vector passing through a certain pixel of interest in the interpolation frame Is an average value (or a weighted average corresponding to the temporal distance between the interpolated frame and the current frame or the previous frame). The details of such interpolation processing are also described in the above-mentioned document and are well-known, and will be omitted here. The interpolation frame created as described above is inserted into the frame sequence in the input video signal by the operations of the image memory 115 and the memory I / F 114. For example, when the frame rate (frame frequency) of the input video signal is 60 Hz and the frame rate is converted to 120 Hz, which is twice as high, one interpolation frame is inserted every other frame in the input video signal. The details of this processing are also described in the above Japanese Patent Laid-Open Nos. 2006-165602 and 2007-082030, and are omitted here.

  The YCbCr / RGB conversion unit 116 converts the YCbCr signal into RGB. The RGB-converted data is output to the FPD panel 118 configured by, for example, a liquid crystal panel or PDP via the timing controller unit 117.

  2 shows a specific example of the character telop detection unit 106 in FIG. 1. In the high luminance / edge detection unit 203, the current frame signal 104 from the line memory unit 103 and one frame before A signal 105 is input, and a portion having a high brightness and a high edge as a character feature is detected for each of the current frame signal 104 and the previous frame signal 105. Since the character portion generally has a higher luminance than other images, and the luminance difference from other images is large and its change is steep, in this embodiment, the character feature amount from the input video signal As a result, a high-brightness and high-edge signal is extracted so that a character portion can be detected well from an image. The high luminance and high edge signal is a signal having a luminance value equal to or higher than a predetermined threshold. For example, the luminance signal Y has 256 gradations (8-bit digital signal). In some cases, for example, the luminance is 150 gradations or more. The high edge signal is a high frequency component (edge component) extracted by performing high-pass filtering (or first-order or second-order differentiation) on the luminance signal Y and is a signal having a predetermined threshold value or more.

Further, the high brightness / edge detection unit 203 according to the present embodiment detects the rising edge and the falling edge by using, for example, a differentiation circuit, etc., for each of the above edges, and thereby features of the following four types of characters: A signal relating to the quantity shall be output.
High brightness and rising edge of the current frame signal High brightness and falling edge of the current frame signal High brightness of the signal 1 frame before and high brightness and falling edge of the signal 1 frame before the signal In other words, in this embodiment, the current frame A period in which a high-intensity signal having a high brightness and a rising edge of the signal and having a predetermined level or more and a substantially constant level continues to the high-intensity and falling edge of the current frame signal is defined as a character telop area of the current frame. In addition, a period in which a high-intensity signal having a high brightness and a rising edge of the signal before one frame is a predetermined point and continues to the high-intensity and falling edge of the signal one frame before is the character before one frame. The telop area. In this way, the area where the character telop exists in each frame is specified. In order to specify such a character area, the above four signals (that is, high brightness and rising / falling edge of each frame) will be referred to as “character area data” below.

  On the other hand, the motion detection unit 204 detects the scroll speed (that is, the motion amount) of the character telop using the character area data detected by the high luminance / edge detection unit 203 and creates a motion amount histogram for the character telop. To do. Then, from the character telop movement amount histogram information, the character telop appearance pattern is identified and the character telop identification signal 208 is output, and the character telop movement amounts 107 and 108 are detected and output. In this embodiment, as shown in FIG. 3, the motion detection unit 204 outputs, for example, two types of character telop motion amounts 107 (3 ppf) and 108 (15 ppf) having main motions in the video. To do. Details of the configuration for generating the character telop identification signal 208 and the motion amounts 107 and 108 from the character telop motion amount histogram will be described later. Furthermore, the motion detection unit 204 outputs a line mask enable signal 209 to prevent erroneous detection.

  The interpolation enable generation unit 205 uses the motion amounts 107 and 108 detected from the motion detection unit 204 and the line mask enable signal 209 to generate an interpolation enable signal 210 for the character telop portion. Here, it is assumed that the interpolation enable signal 210 includes two types of signals corresponding to the character telops 301 and 302 shown in FIG. The selector unit 206 selects either one of the two interpolation enable signals 210 according to the character pattern recognition signal 208 and outputs the selected interpolation enable signal 109 or 110.

  FIG. 4 shows a specific example of the motion detection unit 204 in FIG. The data shift processing unit 401 in FIG. 4 receives four signals output from the high luminance / edge detection unit 203 in FIG. 2, that is, character area data. The data shift processing unit 401 shifts character area data of the current frame or one frame before by one dot (pixel). In this embodiment, by setting this shift amount to 0 to 30, it is possible to detect up to a motion amount of 30 ppf.

  Then, the data shift processing unit 401 and the matching processing unit 402 detect the motion amount of the character telop from the character region data while shifting the character region data. For example, the data shift processing unit 401 compares (shifts) one pixel at a time in the horizontal direction with reference to the character region data in the current frame, and the matching processing unit 402 compares the character region data with the previous frame. Here, the matching processing unit 402 compares a certain pixel in the character area data in the current frame with a pixel in the same position spatially as the certain pixel in the character area data one frame before. In this embodiment, the shift amount starts from 0 so that a stationary character can be detected. When the difference value obtained as a result of the comparison is equal to or smaller than a predetermined value, both are “matched”, and the number of “matched” pixels is referred to as “matching amount”.

  Therefore, when the character telop moves 10 dots from the previous frame to the current frame in the actual video, when the character area data in the current frame is shifted by 10 pixels, the matching amount with the character area data of the previous frame is the maximum. It becomes. That is, by detecting this matching amount, the amount of movement of the character telop can be specified. In this embodiment, the processing for obtaining the matching amount is performed until the shift amount reaches 30 pixels. In the above example, the character area data in the current frame is shifted by one pixel and compared with the character area data in the previous frame. Conversely, the character area data in the previous frame is shifted by one pixel and the character area data in the current frame. You may compare with the data. As an edge portion detection method, for example, the method described in Japanese Patent Application Laid-Open No. 2007-316293 already filed by the present applicant is known, and details thereof are apparent by referring to the document. Therefore, the description is omitted.

  The line histogram detection unit 400 counts the matching amount calculated by the matching processing unit 402 in units of frames for each shift amount (that is, motion amount) of 0 to 30. For example, in a matching calculation with a motion amount 10 (character region data shift amount 10), if the matching amount of each line is a value as shown in FIG. The line in which the quantity 10 exists is as shown in FIG. 5B, and the count number is 5. That is, information is obtained that there are five lines in the frame where an object moving with a motion amount of 10 (ppf) exists. As a result, a line histogram having the number of lines on the vertical axis and the amount of motion on the horizontal axis can be obtained as described later in FIG. With this line histogram, the number of lines in which a character telop exists in a certain frame and the appearance pattern of the character telop can be specified as will be described later. In FIG. 5, the matching amount is detected even in a region where there is no character telop. This is because the character feature amount (high brightness and high edge portion) may be detected for images other than characters. It is. However, by making the predetermined threshold value to be compared with the matching amount appropriate (300 in this embodiment), it is possible to eliminate or reduce the influence of the detected character feature amount on the video other than the character. . The predetermined threshold may be about 20 to 60% of the number of pixels in the horizontal direction of one line.

  On the other hand, the pixel histogram detection unit 404 obtains a pixel histogram by counting the matching amount calculated by the matching processing unit 402 in units of lines for each shift amount (that is, motion amount) of 0 to 30. For example, in FIG. 6A, the histogram distribution in a predetermined line 6001 including an object moving at a motion amount of 10 (ppf) is as shown in FIG. That is, FIG. 6B shows that a certain line 601 contains the most characters that move at a movement amount of 10 (ppf).

  Here, the reason why there is a count value even in the portion with motion 0 in FIG. 6B is because there is a high possibility that the portion without motion character in FIG. 6A is matched as motion 0. In addition, in the case where the background is a general video, it is expected that a more complicated histogram shape is expected. In this embodiment, the motion is detected by detecting the character feature amount of high brightness and high edge. Since the detection condition is focused on the character telop, high detection accuracy can be maintained. Further, in order to extract only a more dominant motion amount, the histogram information may be smoothed.

  The pattern determination unit 405 uses the line histogram obtained by the line histogram 403 detection unit and the pixel histogram obtained by the pixel histogram detection unit 404 to determine the character telop display state in the input video, that is, the character telop pattern. Identify. An example of the character telop pattern is shown in FIG.

  No. 0 in FIG. 7 shows a case where there is no character telop in the video. In this case, since the character area data is not detected, each motion amount is detected in the line histogram and the pixel histogram relating to the character motion amount. The frequency is zero. In other words, when the frequency of the line histogram detected by the line histogram detection unit 403 is 0 over the entire amount of motion, it can be determined that there is no character telop in the video.

  The upper example of No. 1 is a case where a stationary character exists in Line 1 of a certain video. In this case, because of the above-described matching processing, the matching amount is maximized when the shift amount of the character region data is 0. Therefore, as shown in the drawing, the frequency of the motion amount 0 is maximized in the line histogram. In other words, in the line histogram detected by the line histogram detection unit 403, when the frequency with the motion amount being 0 is the maximum, it can be identified that there is a stationary character in a certain video.

  The example below No. 1 is an example in which there is a character telop that moves 10ppf to the left in Line 1 of a video. In this case, since the matching amount is maximized by the character region data shift amount 10 by the above-described matching processing, the frequency of the motion amount 10 is the largest in the line histogram as shown in the figure. In other words, in the line histogram detected by the line histogram detection unit 403, when the frequency with a motion amount of 10 is the maximum, it can be identified that a character with a motion amount of 10 exists in a certain video.

  The upper example of No. 2 is an example in which there is a character telop that moves at 10 ppf to the left in Line 2 and a character that stops at Line 1 of a video. In this case, since the matching amount between the shift amounts 0 and 10 of the character area data is increased by the above-described matching processing, the frequency of the motion amounts 0 and 10 exceeds a predetermined value in the line histogram as shown in the figure. It becomes a peak. In other words, in the line histogram detected by the line histogram detection unit 403, when the frequency of motion amounts 0 and 10 is larger than a predetermined value, there are still characters and characters of motion amount 10 in a video. Can be identified.

  The example below No. 2 is an example in which there is a character telop that moves at 3 ppf to the left in Line 1 and a character telop that moves at 10 ppf to the left in Line 2. In this case, since the matching amount of the shift amounts 3 and 15 of the character area data is increased by the above-described matching processing, the frequency of the motion amounts 3 and 15 exceeds a predetermined value in the line histogram as shown in the figure. It becomes a peak. In other words, in the line histogram detected by the line histogram detection unit 4003, when the frequencies of the motion amounts 3 and 15 are larger than the predetermined values, it is identified that characters of the motion amounts 3 and 15 exist in a certain video. be able to.

  Thus, in this embodiment, the display state of the character telop, that is, the appearance pattern can be identified by detecting the line histogram. Further, in the line histogram, the frequency exceeds a predetermined value, or the maximum amount of movement can be set as the amount of movement of the character telop.

  Furthermore, in this embodiment, as described above, the pixel histogram detection unit 404 detects the pixel histogram of the motion amount for each of a plurality of lines in one frame. For example, when the number of effective lines of the input video signal is 1080, the pixel histogram of the motion amount is detected for each line sequentially from the first line to the 1080th line of a certain frame. As a result, a line in which a character telop exists in a certain frame can be specified.

  For example, in FIG. 7, it is assumed that Line1 is the 580th line and Line2 is the 950th line. In the example above No. 1 in FIG. 7, the detected pixel histogram of Line 1 is the maximum at motion 0 and matches the corresponding line histogram. In such a case, it can be determined that there is a character telop on the 580th line. Similarly, in the example under No. 1, the detected pixel histogram of Line 1 is the largest at motion 10 and matches the corresponding line histogram, so that there is a character telop on the 580th line. Can be judged.

  In the upper example of No. 2, the pixel histogram of Line 1 has peaks at movements 0 and 10, and matches the corresponding line histogram. Therefore, the static character telop with movement 0 on the 580th line. It can be determined that there is a character telop with a movement of 10 in Line2. Similarly, in the example under No. 2, since the pixel histogram of Line1 has peaks at movements 3 and 15, and matches the corresponding line histogram, the character telop with movement 3 is present on the 580th line. It can be determined that there is a character telop with a movement of 15 in Line2.

  The pattern of FIG. 3 which is an example of a character telop in the present embodiment is the pattern 2 in FIG. 7 from the profile of the line histogram in which two peaks exist. That is, as shown in FIG. 7, the pattern determination unit 405 has a frequency that exceeds a predetermined value (that is, there are two peaks) at the two motion amounts 3 and 15 in the line histogram below No. 2. Then, a character pattern recognition signal 208 (in this case, a value “2” of pattern 2) 208 indicating that the current input video has two types of motion telop information is generated, and the selector 206 and the interpolation enable generation unit 205 in FIG. Output to. Further, in the line histogram below No. 2, since the frequency exceeds a predetermined value in the motion amounts 3 and 15, motion amounts 107 and 108 corresponding to the two types of character telops are generated, respectively, as shown in FIG. The data is output to the interpolation frame generation unit 113, the interpolation enable generation unit 205 and the line mask generation unit 409 in FIG. In the case of this example, values of the motion amount 3 ppf and the motion amount 15 ppf are output as two types of motion amounts 406.

  Further, in the pixel histogram obtained by the pixel histogram detection unit 404, the maximum amount of motion 408 in units of lines is output by detecting the amount of motion with the maximum appearance frequency. At this time, since the timing at which the maximum value in each line is output is delayed by one line, as shown in FIG. 1, the character telop is applied to the data input to the video portion motion detection unit 106 and the interpolation frame generation unit 107. The signal 104 one line before is input to the detection unit 106. When the image of FIG. 3 is input, a value of motion amount 3 for line 1 and a motion amount of 15 for line 2 is output to each line.

  The line mask generation unit 409 determines a line to be masked from the two kinds of dominant motion amounts 107 and 108 per frame and the dominant motion amount 408 detected for each line, and determines the line to be masked. A line mask enable signal 209 is output. In the case of the example in FIG. 3, the line 1 (for example, the movement amount 3 in the 580th line) is masked so that the enable of the movement amount 15 is not generated, and only the enable of the movement amount 3 is generated in the line 1. So that Specifically, the two types of dominant motion amounts 107 and 108 in units of frames are compared with the dominant amount of motion 408 in units of lines, and the values are the same (or even within different ranges). The line mask enable signal 209 is output to the interpolation enable generation unit 205 of FIG. 2 as 0 for the lines and 1 for the different lines.

  Similarly, the line 2 (for example, the movement amount 15 in the 950th line) is masked so that the enable of the movement amount 3 is not generated, and only the enable of the movement amount 15 is generated in the line 2. . Which line of motion corresponds to which line and the line to be masked can be specified by the correspondence between the line histogram and the pixel histogram for each line as described above.

  FIG. 8 shows a specific example of the interpolation enable generation unit 205 in FIG. In FIG. 8, the first enable generation unit 806 receives the above-described character region data output from the high luminance / edge detection unit 203 and the motion amount 107 from the pattern determination unit 405. Then, the character area data of the current frame signal is shifted (delayed) horizontally by the amount of movement 107 with respect to the character area data of the previous frame signal, and the shifted character area data and the character area data of the previous frame signal The logical product of Thereby, the character area data (for example, the character 301 in FIG. 3) moving with the movement amount 107 is identified and specified. The second enable generation unit 8007 receives the character area data output from the high luminance / edge detection unit 203 and the motion amount 108 from the pattern determination unit 405. Similarly, the character area data of the current frame signal is shifted (delayed) in the horizontal direction by the amount of motion 108 minutes, and the logical product of this shifted and the character area data of the signal one frame before is obtained, Character area data (for example, character 302 in FIG. 3) moving with the amount of movement 108 is identified and specified. In the above description, the signal one frame before may be shifted instead of the current frame.

  In the interpolation enable determination unit 808, the character region moving with the movement amounts 107 and 108 from the character pattern recognition signal 208 from the pattern determination unit 405 in FIG. 2 and the line mask enable signal 209 from the line mask generation unit 409 in FIG. For each piece of data, enable / disable is determined. For example, the operation of the interpolation enable determination unit 808 will be described by taking as an example the case where the input video includes character telops having two types of motion in the frame as shown in FIG. 3 (that is, pattern 2). In this case, since the character pattern recognition signal 208 from the pattern determination unit 405 is “pattern 2”, the interpolation enable determination unit 808 uses the line mask enable signal 209 to enable the two regions where the character telop exists. Valid / invalid is judged. The character area data identified and output by the first enable generation unit 806 (for example, the character 301 in FIG. 3) is valid only when the line mask enable signal 209 is 0, and is invalid when the line mask enable signal 209 is 0. Cleared. Similarly, the character area data identified and output by the second enable generation unit 807 is valid only when the line mask enable signal 209 is 0, and is invalid and cleared to 0 when it is 1. By this mask processing, enable can be generated only in the most dominant direction of movement in the line, and it is possible to prevent adverse effects due to erroneous detection.

  The interpolation enable expansion / trace unit 809 performs processing for expanding the enable applied only to the character area data to the entire area including the characters. As a result, when there are two character telops having different motions as shown in FIG. 3, the interpolation enable signal 210 corresponding to the region including each character region is generated. For this extension processing, refer to, for example, the method described in JP-A-2007-316293. This extension process can be omitted.

  When the image of FIG. 3 is input, the interpolation enable for the motion amount 107 (3 ppf) is output as FIG. 9A and the interpolation enable for the motion amount 108 (15 ppf) is output as FIG. It becomes like this. In FIG. 9, the white oval portion is the area where the enable is applied.

  Finally, the selector unit 206 in FIG. 2 switches the output of the enable signal in accordance with the character pattern recognition signal 208, and corresponds to each of the motion amounts together with the two types of motion amounts 107 and 108 from the motion detection unit 204. The interpolation enable signals 109 and 110 are output. Here, if the character pattern recognition signal 208 is pattern 1, only one interpolation enable signal is output, and if it is pattern 2, two are output corresponding to each character area. In the case of pattern 1, only one amount of motion 107 is output, and in the case of pattern 2, two amounts of motion 107 and 108 are output. Further, in the selector unit 206, for example, in FIG. 7, when the pattern 0 has no character, the interpolation enable signal 210 is 0, and if only one type of character is present, such as pattern 1, the interpolation is performed. One of the enable signals 210 performs an operation such that 0 is set.

  Output to the interpolation frame generation unit 107 the motion amounts 107 and 108 of the two types of characters detected by the character telop detection unit 105 and interpolation enable signals 109 and 110 indicating the areas where the characters having the respective motion amounts exist. Thus, the interpolation frame generation unit 107 masks the area indicated by the interpolation enable signal 109/110 in the input video, that is, the character telop area. Accordingly, the interpolation frame generation unit 107 performs the above-described interpolation processing using the motion amounts 107 and 108 for the masked regions (that is, the character telop regions indicated by the interpolation enable signals 109 and 110). On the other hand, for the background video other than the text telop area, interpolation processing is performed using the motion vector from the video motion detection unit 101. Therefore, according to the present embodiment, the character telop and the other video area can be individually (independently) interpolated, and the character area collapses due to the influence of the motion of the background video portion, or vice versa. It becomes possible to prevent this phenomenon.

  As described above, in this embodiment, the feature amount of the character is detected, and the line histogram and the pixel histogram indicating the frequency for each motion amount are created, so that a plurality of motion character telops present in the video are displayed. It becomes possible to detect. In addition, since the enable corresponding to each amount of motion of each character telop can be generated, each motion telop can be identified with high accuracy.

  In this embodiment, the case where two types of motion telops exist in the video has been described. However, by providing a plurality of processing systems for the motion detection unit 204 and the interpolation enable generation unit 205 in FIG. The above character telops can also be supported.

  In this embodiment, a matrix conversion unit is provided at the input / output stage of the video signal, and the detection system for the amount of motion of the character telop is all configured using only the luminance signal Y. However, the present invention is not limited to this. . For example, the motion amount may be detected using the RGB signal as it is, or the detection may be performed using data obtained by combining the RGB signal and the luminance signal Y. By adopting such a configuration, for example, the detection accuracy of the amount of motion of a colored telop other than white is improved. In order to further improve the detection accuracy of the amount of motion of a colored telop other than white, for example, an RGB / HSV converter (not shown) for converting an RGB signal into an HSV signal is separately provided and input It is also possible to generate hue (Hue: H value) and saturation (Saturation: S value) information from the RGB signals and input them to the character telop detection unit 106 to detect the amount of movement of the character telop. In this way, by adding color information, it is possible to improve the detection accuracy of character telops.

  Next, a second embodiment of the present invention will be described with reference to FIGS. In the second embodiment, even when the character telop display state in the input video is the case of pattern 3 in which a plurality of motion telops exist on the same line as shown in FIG. It is characterized by being made identifiable. In the configuration of the first embodiment, since motion detection and enable control are performed in units of lines, in the case of pattern 3, only one of the two motions included in the same line can be handled. On the other hand, in this embodiment, by performing motion detection and enable control in units of pixels, even when character telops with a plurality of motion amounts exist on the same line, they can be separated and processed. Yes.

  In the second embodiment, compared with the first embodiment, (1) In the first embodiment, a signal of one line before the current frame and previous frame data is used to detect the movement of the character telop. However, in the second embodiment, in addition to this, a signal 5 lines before the current frame and previous frame data is also used. (2) High luminance and high edge data as character area data is The difference is that the signal is generated based on the signal five lines before and (3) the configuration of the interpolation enable generation unit shown in FIGS. 2 and 8 is different. A specific example of the interpolation enable generation unit according to the second embodiment is shown in FIG.

  In FIG. 10, the same components as those of the example of the interpolation enable generation unit shown in FIG. The specific example of FIG. 10 is obtained by adding an area determination unit 120 between the first enable generation unit 806, the second enable generation unit 807, and the interpolation enable determination unit 808 in the configuration shown in FIG. A specific example of the determination unit 120 is shown in FIG.

  The circuit shown in FIG. 12 is a character telop that moves with a plurality of motion amounts (here, motion amount 1 and motion amount 2) included in the same line in a predetermined area centered on the position of a certain target pixel. For each, the original movement of the pixel is determined by counting its edges. In FIG. 12, the enable signal 121 with the motion amount 1 output from the first enable generator 806 of FIG. 10 is input to the line memory 122 and delayed by 10 lines, followed by the 10clk delay processing and HST count unit 123. Is delayed by 10 clocks, that is, 10 pixels. On the other hand, the enable signal 124 with the motion amount 2 output from the second enable generation unit 807 of FIG. 10 is input to the line memory 125 and delayed by 10 lines, and subsequently 10 clk delay processing and the HST count unit 126 outputs 10 signals. Delayed by clock, 10 pixels. Here, in the present embodiment, since the data of 5 lines before is input as described above, by performing the above-described 10-line delay and 10clk delay processing, for example, the target pixel 135 as shown in FIG. It is possible to refer to the area of V: ± 5 × H: ± 5.

The 10clk delay processing and HST count units 123 and 126 count the number of edges for the motion amount 1 and the motion amount 2 included in the same line, and output the count numbers 127 and 128 corresponding to the motion amounts, respectively. To do. Here, the counting of edges is performed by counting the number of pixels having edges higher than a predetermined value among the pixels in the area of FIG. Therefore, in the present embodiment, the maximum value of the count number is 121 (11 × 11). Now, the motion count 1 edge count 127 is N_MV1, the motion count 2 edge count 128 is N_MV2, the predetermined threshold is N_TH, the motion amount 1 area mask signal 132 is AM1_mk, and the motion amount 2 area mask. When the signal 133 is AM2_mk, the area mask generation unit 129 performs the following processing.
When N_MV1 ≧ N_MV2 and N_MV1 ≧ N_TH, AM1_mk = 0, AM2_mk = 1
When N_MV2 ≧ N_MV1 and N_MV2 ≧ N_TH, AM1_mk = 1, AM2_mk = 0
Otherwise, AM1_mk = 1, AM2_mk = 1
In this manner, the area mask generation unit 129 generates the area mask signal 133 with the motion amount 2 from the area mask signal 132 with the motion amount 1 and outputs the generated area mask signal 133 to the selector of FIG.

  The above process will be described with reference to FIG. 14 in which the pattern 3 shown in FIG. 11 is enlarged. In FIG. 14A, a character string 141 including “ABC” is stationary (this is a movement amount 1), and a character string 142 including “Aiue Okakiku” is 15 ppf from the right to the left of the screen (this is a movement amount 2). )). A square area 143 in the figure indicates an area of V: ± 5 × H: ± 5 pixels shown in FIG. Area histograms for the target pixel 144 and the target pixel 145 are shown in FIGS. 14B and 14C, respectively. Through the above calculation, the enable of motion amount 2 is masked in the target pixel 144 (AM2_mk = 1), and the enable of motion amount 1 is masked in the target pixel 145 (AM1_mk = 1).

  As a result, in the case of pattern 3, the interpolation enable determination unit 808 sends an enable 146 corresponding to the region including the character string 141 of the motion amount 1 (0ppf) and the motion amount 2 (15ppf) through the EN expansion / trace processing unit 809. The corresponding enable 147 including the character string 142 of) is output.

  As described above, in the present embodiment, even when a plurality of moving characters exist on the same line, the plurality of characters are separated from the background image by performing enable control in units of pixels, and individually interpolated. It becomes possible to process.

  Further, in the interpolation enable determination unit 808 of FIG. 10, when the character pattern is 3, only the area mask signal is used, and in the case of the patterns 1 and 2, the area mask signal and the line mask signal are logically ORed. With respect to the patterns 1 and 2, it is possible to obtain the same effect as in the first embodiment.

  In this embodiment, the area histogram is V: ± 5 × H: ± 5 pixels with the target pixel as the center, but is not limited to this. For example, the area may be 11 × 11 or more, and the detection accuracy is further improved by changing the shape of the area (such as a rectangle instead of a square). Furthermore, the area size may be adaptively controlled based on the number of pixels of the input image (such as SD or FullHD).

  In the first and second embodiments, the description has been made only with the horizontal movement of the character telop. However, for example, a predetermined number of line memories are provided and data for a predetermined line is input to the character telop detection unit. Therefore, it is possible to cope with the movement in the vertical direction. Furthermore, if one frame is divided into, for example, a plurality of predetermined area block units, and the above-described motion amount detection and character area specifying processing is performed in units of the predetermined area units, the movement in the vertical, horizontal, and diagonal directions is performed. It is also possible to respond to.

  FIG. 15 shows an example of the configuration of a frame rate conversion apparatus applied to the video processing apparatus in the third embodiment of the present invention.

  In this embodiment, frame rate conversion processing is improved for video in which objects having different motion cycles are mixed. For example, a telecine signal composed of 24 (25) frames is composed of 60 (50) frames. When the character information to be displayed is displayed, the motion correction frame rate conversion function is turned off. Thereby, it is possible to prevent the character information from rattling due to the frame rate conversion process. In FIG. 15, the same components as those in the circuit shown in FIG.

  The telecine detection unit 152 detects whether or not the input video signal 151 is a telecine signal (movie source), and outputs a telecine discrimination signal. The telecine discrimination method detects the phase of 2-3 pulldown from the difference information between the current frame data and the previous frame data obtained by the memory I / F 114 and the image memory 115, and is generally known. Details are not described here because it is a technology. The character telop detection unit 153 detects the movement of the character telop in the input video by the same method as described in the first embodiment, and outputs the result. The configuration may be the configuration described in the first and second embodiments, or may have a function of simply detecting the presence or absence of a moving character telop. The motion vector detection unit 154 detects the motion vector of the video from the current frame data and the previous frame data by the same method as described in the first embodiment. In addition to this, the motion vector may be detected using a block matching method, a gradient method, a phase correlation method, or the like. Based on the motion vector information from the motion vector detection unit 154 (possibly also from the character telop detection unit 153), the interpolation frame generation unit 155 generates an interpolation frame by the same method as described in the first embodiment. Generate. Here, each of the motion vector detection unit 154, the interpolation frame generation unit 155, and the memory interface unit 156 receives a telecine discrimination signal from the telecine detection unit 152, and the input video is a movie (that is, 2-3, 2). -2 pull-down telecine signal format) or other modes.

  In many movie broadcasts on TV, for example, character information such as movie advertisement information is later superimposed on the video and broadcast. In this case, the video part is a video image of 24 frames per second pulled down 2-3, while the character information is a video image of 60 frames per second. When such a video input is made in the system shown in FIG. 15, the telecine detection unit 152 normally determines that the video input is a movie having a telecine signal format because the main video unit has 24 frames. In that case, since the character information portion is recognized as a movie and the motion correction frame rate conversion is performed, the character information portion originally composed of 60 frames is rattled. The reason for this will be described with reference to FIG.

  In FIG. 16, frames represented by the same alphabet (upper case) indicate the same frame information, and lower case alphabets indicate the contents of the character information. When a 24-frame movie is broadcast on TV, it is converted to 60 frames per second by 2-3 pull-down and sent as shown in FIG. On the other hand, since character information data such as advertisement information is originally 60 frames per second, it is sent as shown in (b). In the interpolation frame generation unit 195, the telecine detection unit 152 determines that the input video is a telecine signal, and when receiving the determination signal, the key frame (a plurality of frames of the same video content) indicated by the arrow in the figure is received. Among them, the position of the earliest frame in time is detected, the motion between the key frames is detected, and an interpolation frame is generated. Here, when “d” is the amount of movement of the character information data in the video, when the character data frame at the same arrow position as the key frame in (a) is viewed, the character that should have moved at the constant speed “d” It can be seen that the data has 2d and 3d motion amounts, and the motion amounts differ from frame to frame. Therefore, since a different motion is detected for each frame and an interpolation frame corresponding to the motion is generated, the character information data portion is rattled.

  Therefore, in this embodiment, when the telecine detection unit 152 determines that the input video is in the format of the telecine signal (that is, a movie), if the character telop detection unit 153 has a motion telop in the video, the motion compensation frame The rate conversion operation is turned off, and the current frame is processed through (that is, output as it is). That is, if there is a motion telop during a telecine signal format movie broadcast, it is displayed as it is without doing anything. This process makes it impossible to improve the moving image quality of the video part, but it is possible to improve the adverse effect of the rattling of the character information data.

  As described above, in the present embodiment, when the input video is a movie and there is a motion telop in the video, the motion correction frame rate conversion processing is turned off to prevent the motion telop from being adversely affected. It becomes possible.

  In the present embodiment, all the processing is performed by hardware. However, the present invention is not limited to this, and necessary information (telecine signal, motion character telop presence / absence information, etc.) is read from an external microcomputer, and the microcomputer performs software. It is good also as a structure to process.

  FIG. 17 is a block diagram illustrating an example of a configuration of a character telop detection unit according to the fourth embodiment of the present invention. In FIG. 17, the same components as those of the image display device according to the first embodiment shown in FIG.

  This embodiment is different from the first embodiment in that low luminance and edge detection means are provided so that low luminance telops such as black characters can be detected. According to the present embodiment, it is possible to improve the moving image quality even for a character telop having a low luminance.

  In general broadcasting, telops of various colors and brightness are superimposed. Many general character telops are brighter than the background, but some telops have low luminance such as black characters, so they can be detected only on the condition that they have high luminance and edges. I can't. Even if the character has low luminance, the edge can be captured if the background has high luminance, but the character portion is not strictly captured.

  Therefore, in this embodiment, the low luminance telop is detected by adding another low luminance / edge detecting unit 171 for low luminance telop in addition to the high luminance / edge detecting unit 203. The configuration of the high luminance / edge detection unit 203 and the low luminance / edge detection unit 171 may be the same. Specifically, in the luminance determination, detection is performed under the condition that the high luminance and edge detection unit 2003 is equal to or higher than a predetermined threshold, whereas the detection is performed under the condition that the low luminance and edge detection unit 23001 is equal to or lower than the predetermined threshold. The area correction unit 172 calculates the logical sum of the area output from the high luminance / edge detection unit 203 and the area output from the low luminance / edge detection unit 171, and the edge of the area that satisfies both the high luminance and the low luminance conditions. A signal is output. The subsequent operation is the same as in the first and second embodiments.

  As described above, in this embodiment, it is possible to detect not only high luminance telops but also low luminance telops, and an effect of improving moving image quality can be obtained.

  Further, in this embodiment, by providing two systems of luminance and edge detection systems, the correspondence to high luminance and low luminance character telops is aimed at, but this is not restrictive, and there are two systems or more. Thus, the brightness of the character telop may be finely discriminated to improve the accuracy of identifying the character telop.

The figure which shows one specific example of the video processing apparatus which concerns on 1st Example of this invention. It is a figure which shows one specific example of the character telop detection part 106 which concerns on 1st Example. The figure which shows an example of the character telop which has two types of movement. The figure which shows the specific example of the motion detection part 204 which concerns on 1st Example. The figure which shows an example of the detection of the line histogram which concerns on 1st Example. The figure which shows an example of the detection of the pixel histogram which concerns on 1st Example. The figure for demonstrating the example of the character display pattern which concerns on 1st Example. The figure which shows one specific example of the interpolation enable production | generation part 205 which concerns on 1st Example. It is an enable output image figure which concerns on 1st Example. The figure which shows one specific example of the interpolation enable production | generation part 205 which concerns on 2nd Example of this invention. The figure for demonstrating another example of a character display pattern. The figure which shows one specific example of the area determination part 120 which concerns on 2nd Example. The figure which shows an example of the count area | region of the area histogram which concerns on 2nd Example. The figure for demonstrating operation | movement of 2nd Example. The figure which shows an example of the video processing apparatus which concerns on 3rd Example of this invention. The figure which shows the motion of the character information in a telecine signal. FIG. 10 is a diagram showing another example of the motion detecting unit 204 according to the third embodiment of the present invention.

Explanation of symbols

101 ... Input signal, 102 ... RGB / YCbCr converter, 103 ... Line memory, 104 ... Data one line before current frame, 105 ... Data one line before one frame, 106 ... Character telop detector, 107,108 ... Character telop , 109, 110: Character telop presence / absence signal, 111: Video part motion detection part, 112 ... Video part motion vector, 113 ... Interpolation frame generation part, 114 ... Memory interface part, 115 ... Image memory, 116 ... YCbCr / RGB conversion unit, 117 ... timing control unit, 118 ... FPD panel.

Claims (10)

Information on the current frame of the input video, and a motion vector detection unit that detects information on the motion vector of the video using temporally past frames with respect to the current frame;
For the input video, a first histogram indicating the number of lines in which an object moving with the amount of motion exists for each amount of motion within one frame is generated, and pixels for each amount of motion within one line are generated. A second histogram indicating a number is generated, and based on the first histogram and the second histogram, the input image does not include a character telop, and the input image includes a stationary character telop The input video includes a moving character telop, the input video includes both a stationary character telop and a moving character telop, and the input video is different. It is determined which state includes a plurality of types of character telops of movement, and a sentence with movement when there is a character telop with movement By determining the line and motion amount of the telop, and the character telop detector for detecting the motion amount of the character telop included in presence and the input image of a character telop included in the input image,
An interpolation frame generation unit that generates an interpolation frame by performing an interpolation process on the input video using the character telop area and motion information detected by the character telop detection unit or a motion vector generated by the motion vector generation unit When,
A video signal generation unit that generates a video having a frame rate different from the frame rate of the input video by combining the interpolation frame generated by the interpolation frame generation unit with a frame sequence of the input video;
The interpolation frame generation unit performs interpolation processing corresponding to the character telop area detected by the character telop detection unit, differently from interpolation processing of a video portion other than the character telop area. Video processing device. The video processing device according to claim 1, wherein the character telop detection unit further includes a detection unit for detecting luminance and an edge of the input video as a feature amount related to the character telop in the input video, A video processing apparatus that detects a portion where the detected luminance is equal to or higher than a predetermined threshold and the detected edge is equal to or higher than the predetermined threshold as the character telop. The video processing apparatus according to claim 2, wherein a plurality of detection units are provided to detect at least two types of characters having different luminances. 4. The video processing device according to claim 2, wherein the detection unit detects luminance and edges as feature amounts of the character telop using RGB values of the input video or luminance information Y and RGB values. 5. An image processing apparatus characterized in that: 3. The video processing apparatus according to claim 2, further comprising HSV conversion means for converting the input video to an HSV color space, wherein the luminance information Y in the input video, the hue H value obtained by the HSV conversion means, and An image processing apparatus that detects the luminance and edge as a feature amount of the character telop from a saturation S value.   Information on the current frame of the input video, and a motion vector detection unit that detects information on the motion vector of the video using temporally past frames with respect to the current frame;
  For the input video, a first histogram indicating the number of lines in which an object moving with the amount of motion exists for each amount of motion within one frame is generated, and pixels for each amount of motion within one line are generated. A second histogram indicating a number is generated, and based on the first histogram and the second histogram, the input video does not include a character telop, and the input video includes a stationary character telop The input video includes a moving character telop, the input video includes both a stationary character telop and a moving character telop, and the input video is different. It is determined which state includes a plurality of types of character telops of movement, and a sentence with movement when there is a character telop with movement By determining the line and motion amount of the telop, and the character telop detector for detecting the motion amount of the character telop included in presence and the input image of a character telop included in the input image,
  An interpolation frame generation unit that generates an interpolation frame by performing an interpolation process on the input video using the character telop area and motion information detected by the character telop detection unit or a motion vector generated by the motion vector generation unit When,
  A video signal generation unit that generates a video having a frame rate different from the frame rate of the input video by combining the interpolation frame generated by the interpolation frame generation unit with a frame sequence of the input video;
  A display unit for displaying the video generated by the video signal generation unit,
  The interpolation frame generation unit performs interpolation processing corresponding to the character telop area detected by the character telop detection unit, differently from interpolation processing of a video portion other than the character telop area. Video display device.   The video display device according to claim 6, wherein the character telop detection unit further includes a detection unit for detecting luminance and an edge of the input video as a feature amount related to the character telop in the input video, A video display device characterized in that a portion where the detected luminance is equal to or greater than a predetermined threshold and the detected edge is equal to or greater than the predetermined threshold is detected as the character telop.   8. The video display device according to claim 7, wherein a plurality of systems of the detection unit are provided to detect at least two types of characters having different luminances.   4. The video display device according to claim 7, wherein the detection unit detects luminance and edges as feature amounts of the character telop using RGB values of the input video or luminance information Y and RGB values. 5. An image display device characterized in that it performs.   8. The video display device according to claim 7, further comprising HSV conversion means for converting the input video into an HSV color space, wherein the luminance information Y in the input video, the hue H value obtained by the HSV conversion means, and An image display device characterized by detecting the luminance and edge as a feature amount of the character telop from a saturation S value.

Images