JP2010169822A - Image output device and method for outputting image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image output device having a circuit that forms and outputs an interpolation image other than an original image, the image output device minimizing image quality deterioration caused by disturbance of a stationary title portion, thereby: achieving a circuit for a higher image quality; eliminating the need to add a special buffer memory to mount the circuit. <P>SOLUTION: When an interpolation image is generated from an interpolation vector based on an original image in an interpolation image forming circuit 6 with respect to a stationary title area detected by a stationary title detection circuit 3, the value of an interpolation vector relative to a pixel determined as the stationary title area is used as 0. This prevents a stationary title from being pervaded and deleted by a surrounding dynamic motion picture portion. Further, if a pixel specified by an interpolation vector is in a stationary title area when an interpolation image is generated by pixels outside the stationary title area, and an interpolation image is generated having an interpolation vector value as 0. This prevents pixels in the stationary title area from being located outside the stationary title area. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image output device and an image output method, and belongs to a technique for displaying an image on a video display device such as a liquid crystal television.

  Conventionally, in television broadcasting, an image of 60 frames is transmitted per second, the television receiver receives the image, and outputs an image of 60 frames per second to express a moving image. In a hold type display such as a liquid crystal television, as described in Non-Patent Document 1, due to human visual characteristics, it is one cause of image quality degradation such as edge blurring.

As one of the techniques for reducing the image quality deterioration, there is a double speed display technique in which the number of images displayed per second is doubled (120 frames per second).
This is because an interpolated image (hereinafter referred to as “interpolated image”) generated based on the original image is generated between the original image and the original image for 60 frames per second received, and every other frame. In this method, the original image and the interpolated image are displayed alternately.

  There are various methods for generating an interpolated image, and as an example, between two original images of an original image at a certain time (frame N) and an original image 1/60 seconds later (frame N + 1). There is a method in which an interpolation image after 1/120 second of the original image (frame N) is generated based on the motion vector.

FIG. 4 shows a configuration example of a display circuit for generating the interpolated image and displaying the original image and the interpolated image alternately. In this configuration example, an image in which the input data of the original image input in some form is converted into the RGB format by the image input unit 1 is used.
This display circuit is converted by an RGB buffer memory 9 for storing an RGB format image, an RGB-Y conversion circuit 2 for converting the same RGB format image into a luminance (Y) signal, and an RGB-Y conversion circuit 2. A Y buffer memory 8 for storing the obtained luminance image, a motion vector detection circuit 4 for detecting a motion vector from the luminance image of the continuous two frames of the original image, and determining an optimum motion vector from the obtained motion vector. An interpolation vector evaluation circuit 5 to be selected, an interpolation image generation circuit 6 that generates an interpolation image from the obtained interpolation vector and the original image, and an image output that alternately outputs the original image and the interpolation image together with a synchronization signal Part 7.

The operation in this configuration example will be described below.
The RGB format image of the input original image data is output as an original image frame or written to the RGB buffer memory 9 for use in generating an interpolated image. On the other hand, the RGB format image of the input original image data is converted into a luminance image by the RGB-Y conversion circuit 2 in order to obtain a motion vector. The converted luminance image is stored in the Y buffer memory 8 and simultaneously transmitted to the motion vector detection circuit 4 together with the luminance image of the previous frame.

  The motion vector detection circuit 4 obtains a motion vector from the obtained luminance image between two frames. There are various methods for obtaining the motion vector, and any method may be used. However, in any method, depending on the contents of the image, the vector detection result may include a large error. For example, an object that existed at the time of the original image (frame N-1) does not exist in the original image (frame N) because it enters the back side of another object, and the motion vector cannot be detected normally. Such as the case.

  Next, the obtained motion vector is transmitted to the interpolation vector evaluation circuit 5. The interpolation vector evaluation circuit 5 mainly determines and selects an optimal motion vector from the obtained motion vector candidates. When a certain pixel (x, y) to be evaluated is evaluated, in some cases, two or more motion vectors may intersect, and conversely, one motion vector may not be indicated. In such a case, select the most probable vector, or assign an interpolated vector based on the average value in consideration of the state of surrounding motion vectors, and finally add an internal vector from the original image. An interpolation vector for generating an inserted image is determined. At this time, if the determination accuracy of the interpolation vector is not good, the generated interpolation image loses continuity with the preceding and succeeding original images, causing image quality degradation.

  Finally, the obtained interpolation vector is transmitted to the interpolation image generation circuit 6. The interpolated image interpolated with the interpolated vector generated based on the original image is output alternately with the original image together with the synchronization signal by the image output unit 7.

In this manner, an image of 60 frames per second is expanded to an image of 120 frames per second at double speed and displayed on a display such as a liquid crystal television. As a method for preventing further image quality degradation, the interpolation image generated between the original image (frame N) and the original image (frame N-1) is increased to 2 frames, and 1/180 seconds after the original image (frame N) It is also possible to display after 2/180 seconds and display 180 frames for 3 seconds for 1 second, or similarly, display 240 frames for 1 second for 4 times.
In this case, it is necessary to adjust the size of the interpolation vector generated from the motion vector according to the time of the image to be interpolated.

  Note that the motion vector detection can be realized as an RGB format image as it is, but the RGB-Y conversion circuit 2 is used in the present configuration example in view of an increase in circuit scale.

On the other hand, there is a technique for detecting subtitles such as characters from an image, and there are methods shown in Patent Documents 1 to 3.
Patent Document 1 discloses a method for extracting a still subtitle area (telop area) from a moving image for the purpose of creating a digest video. In the present invention, a still caption area is determined using a difference in luminance data between two temporally adjacent images.
However, the still subtitle area detected as in the invention of Patent Document 1 can be applied to an image portion that is not originally a static subtitle when, for example, an image with almost no motion is continuous between two frames in an image other than the still subtitle portion. There is a possibility that it is erroneously determined as a still caption area.

  In the method shown in Patent Document 2, there has been a proposal for extracting a telop character from a luminance distribution using only an image in one frame. In the present invention, information between frames that are temporally adjacent to each other is not used at all. Therefore, subtitles that move between frames and subtitles that are stationary are detected.

In the method shown in Patent Document 3, extraction of a portion having uniform luminance within one frame, extraction of a caption boundary portion by high-frequency region detection, and extraction of a portion that does not change for a certain period of time by a number of frame memories, Proposals have been made to extract still caption areas from these extraction results. In the present invention, detection is performed using the following features of still captions (telops).
(1) It is composed of pixels having the same signal level.
(2) Since the display is normally performed for 3 to 4 seconds or more, the pixels constituting the telop hold the same signal level for a predetermined time or more.
(3) Since the telop is conspicuous, there is a large difference between the background image and the signal level.

  That is, the telop area is an area having a relatively high spatial frequency on the image plane. However, according to the invention of Patent Document 3, the extracted still subtitle area remains a rectangular area including the entire telop. Also, since all luminance differences are calculated using the difference between adjacent pixels, for example, still captions can be detected accurately in an image with clear captions, but the caption boundary appears to be blurred. It is not possible to detect a simple image.

JP-A-12-23062 Japanese Patent Laid-Open No. 12-182053 JP-A-10-233994

IEICE technical report, electronic display (EID2001-102)

  According to the above-mentioned background art, high-quality images can be achieved by the double-speed display technology, but the motion vector detection detection error that cannot be prevented, and the interpolated image in which the continuity of the moving image is impaired due to the interpolation processing of the interpolation vector There is a problem that a video that is unsightly for viewers is output. In particular, a disturbance of an image having a sharp outline and a stationary state like a stationary subtitle is easily recognized in a moving image, and has been a factor of image quality degradation.

On the other hand, with regard to caption detection technology, moving captions and still captions cannot be distinguished, and even when still captions can be detected, even still image portions other than captions may be erroneously detected. Moreover, even for an algorithm that accurately detects still captions for sharp images, accurate detection is possible even for images in which the boundary of still captions is blurred due to, for example, scaling processing from the original image. There was a problem that could not be done.
Also, a large amount of memory is required to store continuous data for several frames, or the detected still caption area is only a rough rectangular area.

  The present invention has been made in view of the above circumstances, and in an image output apparatus provided with a circuit that generates and outputs an interpolated image other than the original image, image quality degradation due to disturbance of the still caption portion is minimized. An object of the present invention is to provide an image output apparatus capable of realizing a circuit that suppresses image quality and achieves high image quality, and that can be mounted without adding a special buffer memory, and an image output method executed by the image output apparatus It is what.

  In order to solve the above problems, the first technical means of the present invention provides one or a plurality of interpolated frames (interpolated frames) between two input image frames that are temporally continuous with the image frames to be displayed. In an image output device that generates and outputs an image, an image input unit for inputting an image, an original image storage buffer memory for storing the input image without converting the image format, and luminance of the input image sequentially An image format conversion circuit that generates a luminance image by converting to an information-only image format, a luminance image storage buffer memory for storing the luminance image, and a motion vector between two temporally continuous luminance images A motion vector detection circuit for performing interpolation, and each pixel or a plurality of interpolation frames inserted between two temporally continuous frames using the motion vector An interpolation vector evaluation circuit that selects an interpolation vector of a block composed of elements, a still caption detection circuit that detects a stationary caption image between two input temporally continuous luminance images, An interpolated image generating circuit for generating a desired interpolated frame from the interpolated vector and the still caption detection result based on the original image stored in the image storing buffer memory, and the original image stored in the original image storing buffer memory And an image output unit for outputting the frames and the interpolated frames generated by the interpolated image generation circuit so as to be temporally continuous, and the interpolated image generation circuit is determined to be subtitles by the still subtitle detection circuit The value of the interpolation vector assigned to a pixel or a block including a plurality of pixels is set to 0, and the block including a pixel or a plurality of pixels that has not been determined as a caption by the still caption detection circuit If the interpolation vector is determined as the subtitle for a pixel instruction is obtained by and generates an interpolation frame interpolation vector allocated to the block containing a pixel or a plurality of pixels as 0.

  The second technical means is the same as in the first technical means, in which the still caption detection circuit is used in two frames in which the luminance difference absolute values of two pixels at the point target position centered on the evaluation target pixel are temporally continuous. When the absolute value of the luminance difference is almost the same between two frames when the threshold is exceeded and the evaluation target pixel is determined to be a caption boundary pixel, the number of caption boundary pixels is within a block composed of a plurality of pixels. When the value exceeds the threshold, the pixels in the block are determined as captions.

  According to a third technical means, in the second technical means, the still caption detection circuit determines a pixel determined as a caption boundary pixel existing in a row (line) to which the evaluation target pixel belongs for a pixel determined as a caption boundary pixel. When the number of pixels and the number of pixels determined as subtitle boundary pixels existing in the line immediately below the line both exceed a certain threshold, all the pixels of the line to which the evaluation target pixel belongs are determined to be black belt boundaries, It is characterized by canceling the determination of the pixel determined as the boundary pixel.

  According to a fourth technical means, in the second or third technical means, for each evaluation target pixel of two temporally continuous original image frames, the still caption detection circuit determines that the luminance difference absolute value of the evaluation target pixel is When a certain threshold value is exceeded, the evaluation target pixel is determined as a still pixel, and when the caption boundary pixel exists in the peripheral pixels of the still pixel, the still pixel is determined as a caption, and the block is composed of a plurality of pixels. When the sum of the number of subtitle boundary pixels and the number of subtitle pixels exceeds a certain threshold, the pixel in the block is determined as a subtitle.

  The fifth technical means is that in any one of the second to fourth technical means, when two temporally continuous frames of the input image frames are determined to have almost no difference in the entire screen. One of the frames is discarded, no interpolated frame is generated between the two frames, and no caption detection processing is performed in the still caption detection circuit.

  The sixth technical means provides an image executed by an image output device that generates and outputs one or a plurality of interpolation frames (interpolation frames) between two input image frames that are temporally continuous with the image frames to be displayed. In the output method, an image input step for inputting an image to the image output device, an original image storage step for storing the input image as it is in the buffer memory without converting the image format, and an image for which only the luminance information is sequentially input An image format conversion step for generating a luminance image by converting into a format, a luminance image storage step for storing the luminance image in a buffer memory, and a motion vector detection step for detecting a motion vector between two temporally continuous luminance images And an interpolation frame inserted between the two consecutive frames using the motion vector. An interpolation vector evaluation step for selecting an interpolation vector of a block composed of each pixel or a plurality of pixels, and a stationary subtitle for detecting a stationary subtitle image between two temporally continuous luminance images inputted A detection step, an interpolation image generation step for generating a desired interpolation frame from the interpolation vector and the still caption detection result based on the original image stored in the buffer memory in the original image storage step, and a buffer in the original image storage step An image output step for arranging and outputting the original image frame stored in the memory and the interpolation frame generated in the interpolation image generation step so as to be temporally continuous, and the interpolation image generation step includes still caption detection The value of the interpolation vector assigned to a block or a block including a plurality of pixels determined as subtitles in step is set to 0, and still subtitle detection is performed. If it is determined that the subtitle is determined in the pixel indicated by the interpolation vector among the pixels that are not determined as subtitles in the step or the block including a plurality of pixels, the interpolation vector assigned to the block including the pixels or the plurality of pixels is set to 0. It is characterized by generating an insertion frame.

  According to a seventh technical means, in the sixth technical means, the static subtitle detection step is performed in two frames in which the luminance difference absolute values of two pixels at the point target position centering on the evaluation target pixel are temporally continuous. When both exceed the threshold and the absolute value of the luminance difference between the two frames is almost the same value, the evaluation target pixel is determined to be a caption boundary pixel, and the number of caption boundary pixels is within a block composed of a plurality of pixels. When the threshold value is exceeded, the pixel in the block is determined as a caption.

  According to an eighth technical means, in the seventh technical means, the still caption detection step is a pixel determined as a caption boundary pixel existing in a row (line) to which the evaluation target pixel belongs with respect to a pixel determined as a caption boundary pixel. When the number of pixels and the number of pixels determined as subtitle boundary pixels existing in the line immediately below the line both exceed a certain threshold, all the pixels of the line to which the evaluation target pixel belongs are determined to be black belt boundaries, It is characterized by canceling the determination of the pixel determined as the boundary pixel.

  According to a ninth technical means, in the seventh or eighth technical means, the static subtitle detection step has a luminance difference absolute value of the evaluation target pixel for each of the evaluation target pixels of two original image frames that are temporally continuous. When a certain threshold value is exceeded, the evaluation target pixel is determined to be a still pixel, and when a caption boundary pixel exists in the peripheral pixels of the still pixel, the still pixel is determined to be a caption, and the block is composed of a plurality of pixels. When the sum of the number of subtitle boundary pixels and the number of subtitle pixels exceeds a certain threshold, the pixel in the block is determined as a subtitle.

  In the tenth technical means, in any one of the seventh to ninth technical means, when two temporally continuous frames of the input image frames are determined to have almost no difference in the entire screen. An image output method characterized in that either one of the frames is discarded, no interpolated frame is generated between the two frames, and no caption detection processing is performed in the still caption detection step.

According to the present invention, in a display such as a liquid crystal television capable of double-speed display, it is possible to realize a circuit for improving image quality by minimizing image quality degradation due to disturbance of a still caption portion, and adding a special buffer memory And can be implemented.
The same applies to a display other than the double-speed display when an interpolated image is generated in addition to the original image.

It is explanatory drawing which shows typically the structure which shows one form of the frame rate conversion processing circuit with which the image output device of this invention is provided. It is a figure for demonstrating the detection procedure of the still caption area | region by the still caption detection circuit of FIG. It is explanatory drawing which shows typically the structure which shows one form of the frame rate conversion processing circuit with which the image output device of this invention is provided. It is explanatory drawing which shows typically the structure which shows one form of the frame rate conversion processing circuit with which the conventional image output apparatus is provided. It is explanatory drawing which shows typically the video frame pulled down 3-2. It is explanatory drawing which shows typically the frame structure which produces | generates an interpolation image, and the update timing of a still caption detection with respect to the video frame pulled down 3-2.

In the embodiment of the present invention, a static subtitle detection circuit based on still subtitle detection with high accuracy can be implemented with a buffer memory for two frames in order to prevent disturbance of a still subtitle portion that occurs when an interpolated image is generated by a double speed display technique or the like. A simple circuit.
Pixels determined as still caption areas when generating an interpolated image from an interpolation vector determined based on the original image in the interpolated image generation circuit for the still caption area detected by the caption detection circuit By treating the value of the interpolation vector with respect to 0 as 0, it is possible to prevent the still caption from being eroded by the surrounding moving image portion and being deleted. In addition, when generating an interpolated image with pixels outside the still caption area, if the pixel indicated by the interpolation vector is a still caption area, by generating an interpolated image with the value of the interpolation vector set to 0, It is possible to prevent the pixels in the still caption area from jumping out of the still caption area, thereby preventing image quality deterioration and improving the image quality.
The embodiment of the present invention having the above features will be described more specifically with reference to the drawings.

(Embodiment 1)
Embodiment 1 describes a case where a subtitle-added video is input in a circuit that acquires an input image of 60 frames per second and performs a frame rate conversion process (converts a video of 60 frames per second to a video of 120 frames per second). To do.

(Embodiment 1: Configuration)
FIG. 1 is an explanatory diagram schematically showing a configuration showing an embodiment of a frame rate conversion processing circuit provided in the image output apparatus of the present invention.
In FIG. 1, input image data 11 is input to an image input unit 1, and original image data 17 in RGB format is distributed into two signal systems. The first signal system is input to the RGB-Y conversion circuit 2, converted into a luminance image, and then the luminance image data is stored in the Y buffer memory 8. The Y buffer memory 8 is composed of a memory having a data capacity for two frames, and is connected to the still caption detection circuit 3 and the motion vector detection circuit 4.

The still caption detection circuit 3 detects a still caption area in the image from the luminance image data (frame N) 12 and the luminance image data (frame N-1) 13 of two consecutive frames. The detected caption area information 16 is sent to the interpolated image generation circuit 6.
The motion vector detection circuit 4 detects a motion vector between images from the luminance image data 12 and the luminance image data 13 of two consecutive frames.

  The result detected by the motion vector detection circuit 4 is sent to the interpolation vector evaluation circuit 5 as a motion vector 14 to evaluate whether the result detected by the motion vector detection circuit 4 is valid. A final interpolation vector 15 is obtained by a similar method. The obtained interpolation vector 15 is sent to the interpolated image generation circuit 6 together with the caption area information 16 generated by the still caption detection circuit 3.

  The second signal system is connected to the RGB buffer memory 9 and accumulates original image data 17 in the RGB format in the RGB buffer memory 9. The accumulated original image data 17 is sent to the interpolation image generation circuit 6 as the interpolation image generation original image data 18 at a timing synchronized with the interpolation vector 15 generated by the first signal system and the caption area information 16. It is configured to output.

  The interpolated image generating circuit 6 generates an interpolated image 20 based on the obtained interpolated image generating original image data 18, the interpolated vector 15, and the caption area information 16. The interpolated image generation original image data 18 to be referred to may be frame N-th original image data or frame N-1th original image data, and is interpolated with the frame N-th original image data. If so, the vector direction is reversed.

The interpolated image generation circuit 6 outputs the generated interpolated image 20 to the image output unit 7 alternately with the output original image data 19. The output order is set in the order of the (N-1) th original image data, the interpolated image data, and the Nth original image data.
The image output unit 7 adds a synchronization signal necessary for display display and outputs output image data 21 to the display display device at a predetermined timing.

(Embodiment 1: Operation)
Next, the operation of this embodiment will be described.
The Nth input image data 11 at a certain time is input to the image input unit 1, and the original image data 17 in RGB format is transmitted to the first signal system and the second signal system.
In the first signal system, original image data 17 in RGB format is input to the RGB-Y conversion circuit 2 and a luminance image (frame N) 12 is generated by format conversion processing. The luminance image (frame N) 12 is written into the Y buffer memory 8, and at the same time, a past luminance image (frame N-1) 13 continuous in time is read from the Y buffer memory 8, and these two frames (frames) are read out. The luminance image of N, frame N−1) is transmitted to the still caption detection circuit 3 and the motion vector detection circuit 4.

  The motion vector detection circuit 4 obtains a motion vector 14 from the input luminance image of two frames. Various proposals have been made for a method for obtaining the motion vector 14, and for example, there is a motion vector detection method by block matching. In this method, a plurality of pixels are grouped into blocks, whether or not the two frames of the frame N and the frame N-1 match each other is evaluated in block units, and a motion vector is obtained from the positional relationship with the most matched block. Is the method. The obtained motion vector 14 is transmitted to the interpolation vector evaluation circuit 5.

  The interpolation vector evaluation circuit 5 generates an interpolation vector 15 necessary for generating an interpolation image. The interpolation vector evaluation circuit 5 uses the motion vector indicating the pixel as the interpolation vector of the evaluation target pixel (or the evaluation target block composed of a plurality of pixels) based on the motion vector 14 as an interpolation vector. There may be a case where a plurality of motion vectors indicate the pixel or a single motion vector does not indicate the pixel. In such a case, for example, in an evaluation target pixel (or an evaluation target block) that does not designate the pixel, interpolation is performed with an interpolation vector of peripheral pixels of the evaluation target pixel, or 0 vector is assumed to have no motion. Or assigning a plausible interpolation vector as appropriate. The interpolation vector 15 assigned to the evaluation target pixel is transmitted to the interpolation image generation circuit 6.

On the other hand, the still caption detection circuit 3 detects a still caption area from two temporally continuous luminance images (frame N) 12 and luminance image (frame N-1) 13.
The still caption detection circuit 3 detects a still caption area in the procedure shown in FIG. In step S1, the caption boundary detection unit 101 detects caption boundary pixels. In the caption boundary detection unit 101, the luminance difference absolute value of a pixel between two points that are in a point-symmetrical position with respect to a certain evaluation target pixel (x, y) is used as a past luminance image (frame N-1) 13 and the current luminance value. Each luminance image (frame N) 12 is obtained.

Using the pixel position differences p and q and the frame number N from the pixel to be evaluated, they are expressed as Expression (1) and Expression (2).
Y _N (p, q) = | Y _N (x−p, y−q) −Y _N (x + p, y + q) | (1)
Y _N-1 (p, q) = | Y _N-1 (xp, yq) −Y _N-1 (x + p, y + q) | (2)

In the present embodiment, the absolute value of luminance difference of 3 × 3 taps around the evaluation target pixel (x, y) is obtained. At this time, the luminance difference absolute value Y _N (p, q), Y for each combination of (p, q) = (1,1), (0,1), (−1,1), (1,0). _N-1 (p, q) is obtained.
In the case where a similar value is obtained around the evaluation target pixel (x, y) at the surrounding P × Qtap, [(Q−1) / 2 ≧ q> 0, (P−1) / 2 ≧ p ≧ − (P -1) / 2] and all combinations [q = 0, (P-1) / 2 ≧ p ≧ 0].

Here, it is evaluated whether or not the obtained luminance difference absolute values satisfy the conditions of Expressions (3) to (5). The conditional expression (3) determines the absolute value of the difference between the luminance difference absolute values at the same position in two temporally consecutive frames, determines whether the difference absolute value is smaller than a certain threshold T _diff , Is evaluated. Further, in the conditional expressions of Expression 4 and Expression 5, it is determined whether the respective luminance difference absolute values Y _N (p, q) and Y _N-1 (p, q) are larger than the threshold value T _power , and It is evaluated that the feature is “there is a large difference between the background image and the signal level”.

| Y _N (p, q) −Y _N-1 (p, q) | <T _diff (3)
Y _N (p, q)> T _power (4)
Y _N-1 (p, q)> T _power (5)

  The number of combinations (p, q) satisfying all the conditional expressions (expression (3), expression (4), expression (5)) is counted, and when the number of combinations exceeds a certain threshold value, the pixel (x , Y) is determined to be a stationary subtitle boundary.

  According to this method, a still caption boundary is detected without using any luminance value of the evaluation target pixel (x, y) itself. In addition, the result of the absolute value of the luminance difference between pixels spaced apart according to the number of taps is also considered. In the stage before input image data 11 is input to this circuit, for example, when a video of SD resolution (640 × 480) is scaled to a video of FullHD resolution (1920 × 1080), a still subtitle boundary portion is intermediate There are cases in which interpolation is performed due to luminance and blur is caused in adjacent pixel portions. In such a case, even if an attempt is made to detect a caption boundary between adjacent pixels, the luminance shifts stepwise by the intermediate luminance pixel and cannot be detected. However, according to this method, since it is possible to detect using the absolute value of the luminance difference between pixels several pixels apart, it is possible to detect the caption boundary with high accuracy.

  Further, in the detection of the caption boundary for the pixel (x, y), threshold determination is performed using the result of a combination of a plurality of (p, q). This is based on the feature that there are many combinations of pixels including the feature of the caption boundary pixel in the vicinity of the caption boundary portion. When the number of taps is increased, erroneous detection other than the caption boundary portion is prevented. It is effective for.

In step S2, black band boundary detection is performed on the detected caption boundary pixel 111 by the black band boundary detection unit 102, and necessary filter processing is performed when a black band is detected.
Some video content may be displayed in a letterbox with black bars on the top and bottom. In this case, a boundary line between the black belt and the captured image is generated in the horizontal direction, and depending on the state of the captured image, the feature may be similar to the caption boundary, and the boundary line may be erroneously determined as the caption boundary pixel. .

For the purpose of avoiding misjudgment in this letterbox display, if the number of subtitle boundary pixels detected on the same line is more than a certain threshold compared to the number of subtitle boundary pixels in the upper and lower lines, Filter processing is performed to exclude all subtitle boundary pixels on the line from subtitle boundary pixels.
However, in letterbox display, subtitles may be superimposed on the boundary line between the black belt and the captured video. In this case, a pixel to be detected as a caption boundary pixel may be subjected to the filtering process by black band detection.
Since it is difficult to distinguish whether the pixel is a subtitle boundary pixel or a boundary line between a black belt and a photographic pixel, a relief measure is applied to the pixel in step S4.

In step S <b> 3, still pixel detection is performed by the still pixel detection unit 103. Since the caption pixel is stationary and has a feature of high brightness, a pixel having this feature is extracted around the caption boundary pixel. The luminance difference absolute value Y _N (p, q), Y for two evaluation target pixels (x, y) of the past luminance image (frame N−1) 13 and the current luminance image (frame N) 12 _When the difference absolute value of _N−1 (p, q) is equal to or smaller than a threshold value T _diff , the evaluation target pixel (x, y) is determined as a still pixel.
| Y _N-1 (x, y) −Y _N (x, y) | <T _diff (6)

In step S4, the filtered subtitle boundary pixel 112 obtained in step S2 and the still pixel 113 obtained in step S3 are used to exclude the pixel (x, y) determined to be a still pixel. With reference to the caption boundary pixel detection result of a pixel within a range of ± 1 pixel, if a caption boundary pixel exists, the pixel (x, y) is determined as a caption pixel.
With this processing, even when a pixel that is not originally a caption is detected as a caption boundary pixel due to erroneous detection in step S1, no caption pixel is detected in the vicinity, so that the effect of noise removal is achieved by the processing after step S5. Fulfill. In addition, when the caption boundary pixel and the black belt boundary line are overlapped, it is determined to be the black belt boundary, and the pixel that has been excluded from the caption boundary pixel is detected again as the caption pixel. Improved.

  In step S5, a block composed of a plurality of pixels with m * n pixels as a unit is formed (this is referred to as a detection block), and the filtered caption subtitle pixel 112 obtained in step S2 and obtained in step S4. Based on the subtitle pixel 114, the sum of the numbers of subtitle boundary pixels and subtitle pixels detected in the detection block is used as an index indicating the likelihood of subtitles in the detection block (hereinafter, the value of this index is referred to as subtitle reliability). .

In step S6, the subtitle reliability 115 obtained in step S5 is used, and for the detection block to be evaluated, the subtitle reliability of the detection block to be evaluated and the subtitle reliability of the detection blocks around the evaluation target are determined. When the sum total exceeds a certain threshold value T _block , the detection block to be evaluated is set as a caption block, and all the pixels in the detection block are determined to be captions.

The weighting method for the detection blocks around the evaluation target is not limited to the following, but here, a method of performing weighting using the following caption characteristics is shown.
(Feature 1) A detection block adjacent to a caption block is highly likely to be a caption block.
(Characteristic 2) There is a high possibility that a detection block existing around a caption block has several consecutive blocks in at least one direction.
From the above characteristics, the detection blocks around the evaluation target are weighted by the following procedure.

(A) The subtitle reliability S (x, y) of the evaluation target detection block is weighted (α times).
(B) Weighting (β times) is performed on the sum S (x ± 1, y ± 1) of subtitle reliability of the detection blocks within one block around the detection target detection block.
(C) Weighting (γ) for caption reliability S (x ± t, y) and S (x, y ± 1) of detection blocks of a plurality of (t) blocks at vertical / horizontal positions from the evaluation target detection block Times).

For the detection blocks that overlap in steps (a), (b), and (c), the subtitle reliability that has been used once as a priority in the order of (a), (b), and (c) is not used in the subsequent steps. And As for the values weighted in the procedure (c), the sum is obtained in each of the upper, lower, left and right directions, and the one direction having the largest value is used as the representative value.
The sum of the values obtained in steps (a), (b), and (c) is compared with a threshold value T _diff, and if a certain threshold value is exceeded, the detection block to be evaluated is determined as a caption block, and the pixels in the detection block All are determined to be subtitle areas, and are transmitted to the interpolated image generation circuit 6 as subtitle area information 16.

  Next, the interpolated image generation circuit 6 receives the interpolation vector 15 generated by the interpolation vector evaluation circuit 5 and the caption area information 16 generated by the still caption detection circuit 3 and stores them in the RGB buffer memory 9. An interpolated image is generated based on the original image data 18 for the interpolated image. The original image data 18 for the interpolated image may be the original image data of the frame N or the original image data of the frame N-1, but when the frame N is used as the original image data, It is necessary to invert the value of the interpolation vector (vx, vy) for (x, y) to (−vx, −vy). Hereinafter, in this operation example, the original image data of the frame N-1 is used as the original image data 18 for generating the interpolated image.

  The interpolation image generation circuit 6 refers to the interpolation vector (vx, vy) at the position of the pixel (x, y) to be generated, and indicates the interpolation vector in the interpolation image generation original image data 18. The value of the pixel (x-vx, y-vy) to be assigned is assigned to the pixel value of the pixel (x, y) of the generated interpolation image 20. However, in the pixel (x, y) generation process, when the detection block including the pixel is determined to be a caption area by the caption area information 16 (mask condition 1), or a pixel (indicated by an interpolation vector) When the detection block including x-vx, y-vy) is determined to be a caption area by the caption area information 16 (mask condition 2), the interpolation vector for the pixel (x, y) is 0 (vx = vy). = 0), the pixel (x, y) generation process is performed.

  The pixel (x, y) generated by the mask condition 1 has an effect of preventing the subtitle pixel from being destroyed due to another pixel jumping into the pixel that should originally be in the subtitle area due to erroneous detection of the motion vector. . In addition, the pixel (x, y) generated by the mask condition 2 has an effect of preventing a pixel in the subtitle region from jumping into a captured image and destroying the captured image due to erroneous detection of a motion vector with respect to a pixel that is not originally a subtitle region. There is.

  Finally, the image output unit 7 receives the output original image data 19 from the RGB buffer memory 9, receives the interpolated image 20 from the interpolated image generation circuit 6, and alternately displays these as output image data 21 on the display. Output to the device. The output order at this time is controlled to be in the order of original image data (frame N-1), interpolated image data, and original image data (frame N).

As described above, in the image output apparatus provided with the frame rate conversion circuit that generates the interpolated image frame and improves the image quality, the circuit that realizes the high image quality by preventing the image quality deterioration due to the breakdown of the subtitle portion. Thus, it is possible to realize the caption area detection without increasing a special memory.
Further, according to the present invention, it is equally effective for all devices that convert the frame rate. For example, the same applies to a device that outputs video content shot at 24 frames per second from 60 frames per second to 120 frames per second. It is.

(Embodiment 2)
As Embodiment 2, an image obtained by converting an image of 24 frames per second into an image of 60 frames per second, that is, an input image of 60 frames per second pulled down 3-2 is acquired, and frame rate conversion processing (60 frames per second video is obtained). A case where subtitled video is input in a circuit that converts video into 120 frames per second) will be described.

(Embodiment 2: Configuration)
FIG. 3 is an explanatory diagram schematically showing a configuration showing an embodiment of a frame rate conversion processing circuit included in the image output apparatus of the present invention. The configuration of the present embodiment is the same as the configuration of the first embodiment shown in FIG. 1, but the Y buffer memory 8, the still caption detection circuit 3, the motion vector detection circuit 4, the interpolation vector evaluation circuit 5, and the interpolation image generation. The functions and operations of the circuit 6 and the like are different from those of the first embodiment as described below.

  In FIG. 4, input image data 11 is input to the image input unit 1, and the original image data 17 in the RGB format is distributed into two signal systems. The first signal system is input to the RGB-Y conversion circuit 2, converted into a luminance image, and then the luminance image data is stored in the Y buffer memory 8. The Y buffer memory 8 is composed of a memory having a data capacity of two frames or more, and is connected to the still caption detection circuit 3 and the motion vector detection circuit 4.

The static subtitle detection circuit 3 detects the luminance image data (frame N−1) that is shifted from the luminance image data (frame N) 12 and the activation image data (frame N) by 1/24 seconds in time (frames that are not at the same time). ) A still caption area in the image is detected from the two frames. The detected caption area information 16 is sent to the interpolated image generation circuit 6.
The motion vector detection circuit 4 detects a motion vector between images from the luminance image data 12 and the luminance image data 13.

  The result detected by the motion vector detection circuit 4 is sent to the interpolation vector evaluation circuit 5 as a motion vector 14 to evaluate whether the result detected by the motion vector detection circuit 4 is valid. A final interpolation vector 15 is obtained by a similar method. The obtained interpolation vector 15 is sent to the interpolated image generation circuit 6 together with the caption area information 16 generated by the still caption detection circuit 3.

  The second signal system is connected to the RGB buffer memory 9 and accumulates original image data 17 in the RGB format in the RGB buffer memory 9. The accumulated original image data 17 is sent to the interpolation image generation circuit 6 as the interpolation image generation original image data 18 at a timing synchronized with the interpolation vector 15 generated by the first signal system and the caption area information 16. It is configured to output.

  The interpolated image generating circuit 6 generates an interpolated image 20 based on the obtained interpolated image generating original image data 18, the interpolated vector 15, and the caption area information 16. The interpolated image generation original image data 18 to be referred to may be frame N-th original image data or frame N-1th original image data, and is interpolated with the frame N-th original image data. If so, the vector direction is reversed.

The interpolated image generation circuit 6 outputs the generated interpolated image 20 to the image output unit 7 for four consecutive frames. Next, the output original image data 19 is output to the image output unit 7 for only one frame. The output order is controlled so that the output original image data at a certain time is used as a reference, and the (N−1) th original image data, the four interpolated image data frames, and the Nth original image data are output in this order.
The image output unit 7 adds a synchronization signal necessary for display display and outputs output image data 21 to the display display device at a predetermined timing.

  As described above, in the image output apparatus provided with the frame rate conversion circuit that generates the interpolated image frame and improves the image quality, the circuit that realizes the high image quality by preventing the image quality deterioration due to the breakdown of the subtitle portion. Thus, it is possible to realize the caption area detection without increasing a special memory.

(Embodiment 2: Operation)
Next, the operation of this embodiment will be described.
The Nth input image data 11 at a certain time is input to the image input unit 1, and the original image data 17 in RGB format is transmitted to the first signal system and the second signal system.
In the first signal system, original image data 17 in RGB format is input to the RGB-Y conversion circuit 2 and a luminance image (frame N) 12 is generated by format conversion processing. The luminance image (frame N) 12 is written into the Y buffer memory 8, and at the same time, the past luminance image (frame N-1) 13 shifted by 1/24 seconds is read from the Y buffer memory 8, and these Luminance images of two frames (frame N and frame N−1) are transmitted to the still caption detection circuit 3 and the motion vector detection circuit 4.

  The input image is a so-called 3-2 pull-down video in which frames taken at the same time are alternately followed by 3-frame input, 2-frame input, and 3-frame input. FIG. 5 shows how the video is input. Input frame N-1 is input 2 frames (frame N-1 (1) to frame N-1 (2)), and then input frame N is input 3 frames (frame N (1) to frame N (3)). Further, 2 frames (frame N + 1 (1) to frame N + 1 (2)) are input as input frame N + 1, and thereafter, 3 frame input and 2 frame input are repeated. FIG. 6 shows a state in which an interpolated image is generated and output for an input frame that has been pulled down 3-2 in this way. Four interpolated images are generated between the frame N-1 and the frame N. At this time, the necessary frame image data is two frames of the image data of the frame N-1 and the image data of the frame N, which are the same. Since a plurality of frame data indicating a frame is not required, a capacity of 3 frames or more is mounted in the Y buffer memory 8, and the frame at the time point when the data is written to the Y buffer memory 8 in the input order and read (frame N) Then, control may be performed so that a past image (frame N-1) shifted by 1/24 second is read out, or control is performed so that frames at the same time are not written before writing to the Y buffer memory 8. Or, it overwrites the area on the Y buffer memory 8 where the frame at the same time is written, and discards the frame data at the same time. In the present invention, the control method and the capacity of the Y buffer memory are not limited to the above methods.

  The motion vector detection circuit 4 obtains a motion vector 14 from the input luminance image of two frames. Various proposals have been made for a method for obtaining the motion vector 14, and for example, there is a motion vector detection method by block matching. In this method, a plurality of pixels are grouped into blocks, whether or not the two frames of the frame N and the frame N-1 match each other is evaluated in block units, and a motion vector is obtained from the positional relationship with the most matched block. Is the method. The obtained motion vector 14 is transmitted to the interpolation vector evaluation circuit 5.

  The interpolation vector evaluation circuit 5 generates an interpolation vector 15 necessary for generating an interpolation image. The interpolation vector evaluation circuit 5 uses the motion vector indicating the pixel as the interpolation vector of the evaluation target pixel (or the evaluation target block composed of a plurality of pixels) based on the motion vector 14 as an interpolation vector. There may be a case where a plurality of motion vectors indicate the pixel or a single motion vector does not indicate the pixel. In such a case, for example, in an evaluation target pixel (or an evaluation target block) that does not designate the pixel, interpolation is performed with an interpolation vector of peripheral pixels of the evaluation target pixel, or 0 vector is assumed to have no motion. Or assigning a plausible interpolation vector as appropriate. The interpolation vector 15 assigned to the evaluation target pixel is transmitted to the interpolation image generation circuit 6.

  Further, in the input image pulled down 3-2 as described above, it is necessary to generate four interpolation frames between two frames that are shifted by 1/24 seconds in time. Ideally, these interpolated frames generate frames using motion vectors that are shifted in time by 1/120 seconds. For example, based on the motion vector obtained by the motion vector detection circuit 4, interpolation vector evaluation is performed using vectors having a size of 1/5, 2/5, 3/5, and 4/5 of this vector. It can be realized by doing. Also, the interpolation frames generated at this time between the frame N and the frame N−1 are respectively represented as an interpolation frame N-1 ′ (vector 1/5 times) and an interpolation frame N-1 ″ (vector 2 / 5 times), an interpolation frame N-1 ′ ″ (vector 3/5 times), and an interpolation frame N-1 ″ ″ (vector 4/5 times). However, the method of generating the interpolation vector is not limited to the method of this example.

  On the other hand, the still caption detection circuit 3 detects a still caption area from two luminance images (frame N) 12 and luminance image (frame N-1) 13 that are temporally continuous (not at the same time). The method for detecting a still subtitle area is the same as in (Embodiment 1: Operation).

  Next, the interpolated image generation circuit 6 receives the interpolation vector 15 generated by the interpolation vector evaluation circuit 5 and the caption area information 16 generated by the still caption detection circuit 3 and stores them in the RGB buffer memory 9. An interpolated image is generated based on the original image data 18 for the interpolated image. The original image data 18 for the interpolated image may be the original image data of the frame N or the original image data of the frame N-1, but when the frame N is used as the original image data, It is necessary to invert the value of the interpolation vector (vx, vy) for (x, y) to (−vx, −vy). Hereinafter, in this operation example, the original image data of the frame N-1 is used as the original image data 18 for generating the interpolated image.

  The interpolation image generation circuit 6 refers to the interpolation vector (vx, vy) at the position of the pixel (x, y) to be generated, and indicates the interpolation vector in the interpolation image generation original image data 18. The value of the pixel (x-vx, y-vy) to be assigned is assigned to the pixel value of the pixel (x, y) of the generated interpolation image 20. However, in the pixel (x, y) generation process, when the detection block including the pixel is determined to be a caption area by the caption area information 16 (mask condition 1), or a pixel (indicated by an interpolation vector) When the detection block including x-vx, y-vy) is determined to be a caption area by the caption area information 16 (mask condition 2), the interpolation vector for the pixel (x, y) is 0 (vx = vy). = 0), the pixel (x, y) generation process is performed.

  The pixel (x, y) generated by the mask condition 1 has an effect of preventing the subtitle pixel from being destroyed due to another pixel jumping into the pixel that should originally be in the subtitle area due to erroneous detection of the motion vector. . In addition, the pixel (x, y) generated by the mask condition 2 has an effect of preventing a pixel in the subtitle region from jumping into a captured image and destroying the captured image due to erroneous detection of a motion vector with respect to a pixel that is not originally a subtitle region. There is.

  Also, four frames of interpolated image data are generated between the original image data (frame N-1) and the original image data (frame N). The same information is used for four frames. For this reason, as shown in FIG. 6, control is performed so that still caption area detection is performed only when switching between frame N-1 and frame N and still caption area information is updated.

  Finally, the image output unit 7 receives the output original image data 19 from the RGB buffer memory 9, receives the interpolated image 20 from the interpolated image generation circuit 6, and outputs these as output image data 21 to the display device. Output. As shown in FIG. 6, the output order at this time is as follows: original image data (frame N-1), interpolation image data (interpolation frame N-1 ′), interpolation image data (interpolation frame N-1 ″). ), Interpolated image data (interpolated frame N-1 ′ ″), interpolated image data (interpolated frame N−1 ″ ″), and original image data (frame N).

  As described above, in the image output apparatus provided with the frame rate conversion circuit that generates the interpolated image frame and improves the image quality, the circuit that realizes the high image quality by preventing the image quality deterioration due to the breakdown of the subtitle portion. The present invention can also be applied to a case where a plurality of images at the same time are input between consecutive input images, such as an input image that is 3-2 pulled down.

DESCRIPTION OF SYMBOLS 1 ... Image input part, 2 ... Y conversion circuit, 3 ... Still subtitle detection circuit, 4 ... Motion vector detection circuit, 5 ... Interpolation vector evaluation circuit, 6 ... Interpolation image generation circuit, 7 ... Image output part, 8 ... Y buffer memory, 9 ... RGB buffer memory, 11 ... input image data, 12 ... luminance image, 13 ... luminance image, 14 ... motion vector, 15 ... interpolation vector, 16 ... subtitle area information, 17 ... original image data, 18 ... Original image data for generating an interpolated image, 19 ... Original image data for output, 20 ... Interpolated image, 21 ... Output image data, 101 ... Subtitle boundary detection unit, 102 ... Black belt boundary detection unit, 103 ... Still pixel detection 111: subtitle boundary pixels, 112: subtitle boundary pixels, 113: still pixels, 114: subtitle pixels, 115: subtitle reliability.

Claims (10)

In an image output device that generates and outputs one or more interpolated frames between two input image frames that are temporally continuous with image frames to be displayed,
An image input unit for inputting an image, an original image storage buffer memory for storing the input image without converting the image format, and the input image sequentially converted to an image format of only luminance information An image format conversion circuit for generating an image, a luminance image storage buffer memory for storing a luminance image, a motion vector detection circuit for detecting a motion vector between two temporally continuous luminance images, and a motion vector An interpolation vector evaluation circuit for selecting an interpolation vector of a block composed of each pixel or a plurality of pixels of an interpolation frame inserted between two temporally consecutive frames using A stationary subtitle detection circuit for detecting a stationary subtitle image between two consecutive frames of luminance images, and the original image storage buffer memo An interpolated image generating circuit for generating a desired interpolated frame from the interpolated vector and the still caption detection result based on the original image accumulated in the original image, the original image frame stored in the original image accumulating buffer memory, and the inner image An image output unit that outputs the interpolated frames generated by the inserted image generation circuit so as to be continuous in time, and
The interpolated image generation circuit sets a value of an interpolation vector assigned to a pixel or a block including a plurality of pixels determined to be a caption by the still caption detection circuit to 0, and the still caption detection circuit does not determine a caption When the pixel indicated by the interpolation vector in the block including the pixel or the plurality of pixels is determined to be a caption, the interpolation frame assigned to the block including the pixel or the plurality of pixels is set to 0 to generate the interpolation frame. An image output apparatus.   The image output device according to claim 1, wherein the static subtitle detection circuit has a luminance difference value of two pixels at a point target position centering on the evaluation target pixel exceeding a threshold value in both temporally continuous frames. When the luminance difference value is almost the same between the two frames, the evaluation target pixel is determined to be a caption boundary pixel, and the number of the caption boundary pixels is within a block composed of a plurality of pixels. An image output device that determines a pixel in the block as a subtitle when the number exceeds.   The image output device according to claim 2, wherein the still caption detection circuit includes, for the pixel determined as the caption boundary pixel, the number of pixels determined as the caption boundary pixel existing in a line to which the evaluation target pixel belongs, and When the number of pixels determined as the caption boundary pixels existing in the line immediately below the line both exceeds a certain threshold, all the pixels of the line to which the evaluation target pixel belongs are determined as black belt boundaries, and the caption An image output apparatus that cancels the determination of a pixel determined as a boundary pixel.   4. The image output device according to claim 2, wherein for each evaluation target pixel of two temporally continuous original image frames, the still caption detection circuit exceeds a threshold value with a luminance difference value of the evaluation target pixel. The evaluation target pixel is determined to be a still pixel, and when the caption boundary pixel is present in a peripheral pixel of the still pixel, the still pixel is determined to be a caption, and the block is composed of a plurality of pixels. An image output apparatus, wherein when the sum of the number of subtitle boundary pixels and the number of subtitle pixels exceeds a certain threshold, a pixel in the block is determined as a subtitle.   The image output device according to any one of claims 2 to 4, wherein when two temporally continuous frames of input image frames are determined to have almost no difference in the entire screen, one of the frames is determined. An image output apparatus characterized in that no frame is discarded, no interpolated frame is generated between the two frames, and no caption detection processing is performed in the still caption detection circuit. In an image output method executed by an image output apparatus that generates and outputs one or a plurality of interpolated frames between two input image frames that are temporally continuous with image frames to be displayed,
An image input step of inputting an image to the image output device;
An original image storage step for storing the input image as it is in the buffer memory without converting the image format;
An image format conversion step for sequentially converting the input image into an image format of only luminance information and generating a luminance image;
A luminance image storage step for storing the luminance image in the buffer memory;
A motion vector detection step of detecting a motion vector between two temporally continuous luminance images;
An interpolation vector evaluation step of selecting an interpolation vector of a block composed of each pixel or a plurality of pixels of an interpolation frame inserted between two temporally continuous frames using a motion vector;
A stationary subtitle detection step for detecting a stationary subtitle image between two temporally continuous luminance images input;
An interpolated image generating step for generating a desired interpolated frame from the interpolated vector and the still caption detection result based on the original image accumulated in the buffer memory in the original image accumulating step;
An image output step of arranging and outputting the original image frames stored in the buffer memory in the original image accumulation step and the interpolation frames generated in the interpolation image generation step so as to be temporally continuous, and
In the interpolated image generation step, a value of an interpolation vector assigned to a pixel or a block including a plurality of pixels determined as a caption in the still caption detection step is set to 0, and the caption is not determined as a caption in the still caption detection step. When the pixel indicated by the interpolation vector in the block including the pixel or the plurality of pixels is determined to be a caption, the interpolation frame assigned to the block including the pixel or the plurality of pixels is set to 0 to generate the interpolation frame. An image output method.   7. The image output method according to claim 6, wherein in the still caption detection step, luminance difference values of two pixels at a point target position centering on the evaluation target pixel both exceed a threshold value in two temporally continuous frames. When the luminance difference value is almost the same between the two frames, the evaluation target pixel is determined to be a caption boundary pixel, and the number of the caption boundary pixels is within a block composed of a plurality of pixels. An image output method characterized by determining a pixel in the block as a subtitle when the number exceeds.   8. The image output method according to claim 7, wherein the still caption detection step includes, for the pixels determined as the caption boundary pixels, the number of pixels determined as the caption boundary pixels existing in a line to which the evaluation target pixel belongs, and When the number of pixels determined as the caption boundary pixels existing in the line immediately below the line both exceeds a certain threshold, all the pixels of the line to which the evaluation target pixel belongs are determined as black belt boundaries, and the caption An image output method comprising: canceling the determination of a pixel determined as a boundary pixel.   9. The image output method according to claim 7, wherein the still caption detection step exceeds a certain threshold for a luminance difference value of the evaluation target pixel for each of the evaluation target pixels of two temporally continuous original image frames. The evaluation target pixel is determined to be a still pixel, and when the caption boundary pixel is present in a peripheral pixel of the still pixel, the still pixel is determined to be a caption, and the block is composed of a plurality of pixels. An image output method comprising: determining a pixel in the block as a subtitle when a sum of the number of subtitle boundary pixels and the number of subtitle pixels exceeds a certain threshold.   The image output method according to any one of claims 7 to 9, wherein when two temporally continuous frames of input image frames are determined to have almost no difference on the entire screen, one of them is determined. An image output method characterized by discarding the frame, generating no interpolated frame between the two frames, and not performing the caption detection processing in the still caption detection step.

Images