JPH09322173A - Method and device for extracting time-varying image telop - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide the method and device directly extracting time-varying image data where a telop part of a moving image is coded without decoding processing. SOLUTION: The method consists of a stage where coded time-varying image data are discriminated as to whether a data frame is coded by using inter-frame correlation or coded without using inter-frame correlation, a stage where whether a picture element of the frame coded by using the inter-frame correlation is coded without using motion compensation, a stage where a coded numeral is stored in a 2-dimensional count matrix depending on a location of each picture element, a stage where a value stored in each count matrix is compared with a threshold level, and a stage where a picture element whose value is higher than the threshold level through comparison is decided to be a telop part, and the operation stages as above are conducted to each picture element within a prescribed area and a telop part in the time-varying image is extracted through the overall discrimination as above.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and apparatus for extracting a telop portion from a moving image, and more particularly to a method and apparatus for extracting a telop portion using coded pixels.

[0002]

2. Description of the Related Art There is a method of extracting a telop portion from a moving image as a method of extracting information representing the content of the moving image in order to perform processing such as searching and editing based on the content of the moving image. The telop here includes subtitles, photographs, symbols, patterns, marks, icons, etc. that appear in a moving image by a method such as superimposing, and the telop portion represents a pixel or a group of pixels including the telop. .

As a conventional technique for automatically extracting a telop portion from a moving image, a method using the property that the image area of the telop has a higher brightness than the background portion and an edge is easily extracted (for example, Motegi) Others, "indexing articles based on character recognition in news video", IEICE Technical Research Report IE95-153, 1996, etc.), and a method of utilizing a large brightness difference in the surroundings of telop castles (For example, Nemoto et al., “Searching for Material Video by Recognition of Telop”, 1994 Spring Conference of the Institute of Electronics, Information and Communication Engineers D-427, 1994, etc.).

In the conventional method of Mogi et al., An edge is extracted from an image by first-order differentiation, the edge image is projected in the horizontal and vertical directions, and a rectangular area of a character string is cut out.
The part where the characters are lined up horizontally has a large difference from the part other than the telop, and the telop region can be stably extracted from the peaks and valleys of the histogram.

In the conventional method such as Root, first, a frame in which a telop appears is found by looking at changes in the distribution of luminance and hue between frames. Next, the telop territory is extracted by taking the difference between the frames immediately before and immediately after the telop appears.

As a method of improving the conventional method such as the root method, a method of increasing the accuracy of telop extraction (Kurakake et al., "Study of Key Target Indexing in Video Using Recognition Technology" IEICE Technical Report IE95-150, 199)
6) has been proposed. This is a method of enhancing the accuracy of telop extraction by averaging a plurality of frames in which telops are present, thereby emphasizing the telops and reducing the influence of background variations.

In the telop extraction from the encoded moving image, the image is completely decoded and restored to the original image, and then the above-mentioned extraction work is performed.

[0008]

In order to extract a telop portion using a conventional method for a moving image coded by utilizing the correlation between frames, the coded moving image is framed. Images must be decoded. For this reason, image decoding processing is required in addition to the telop area extraction processing, which increases the processing cost and makes it difficult to quickly extract the telop portion.

Further, when a method of averaging a plurality of frames for a coded moving image is used, all frame images must be decoded and then averaged, resulting in higher processing cost. There's a problem.

According to the present invention, a moving picture telop for extracting telops at high speed with a small processing cost without decoding frame images from moving picture data encoded by utilizing correlation between frames. It is to provide an extraction method and device.

[0011]

The moving picture telop extraction method of the present invention is a method for extracting a telop portion of encoded moving picture data by utilizing the correlation between frames.
The method includes an extraction step of extracting a portion in which pixels coded without correlation between frames using temporal correlation are concentrated temporally and spatially.

Further, in the moving picture telop extraction method of the present invention, as the extraction step, a counting step of counting the frequency of appearance of coded pixels using correlation between frames and not using motion compensation, It has a selection step of selecting a telop portion based on the result of comparison between the frequency and a threshold value.

Further, in the present invention, as the extraction step, a counting step of counting the frequency of appearance of pixels coded by using correlation between frames and not using motion compensation, and a projection histogram of frequency are provided. It also includes a projection stage to be created and a selection stage to select a telop portion based on the result of comparison between the frequency of the projection histogram and the threshold value.

The invention also includes what is further provided after the counting step, with a merging step for merging the frequencies based on spatial characteristics of the frequencies.

Further, the present invention also includes one further having a storage step of storing the telop portion extracted from the moving image.

Further, the present invention also includes one having a decoding step capable of decoding only the telop portion extracted from the moving image.

Further, according to the present invention, in an apparatus for extracting a terror portion of moving image data coded by utilizing the correlation between frames, the coding is performed without using the correlation between frames and without using motion compensation. An extraction unit is provided for extracting a portion where the converted pixels are temporally and spatially concentrated.

[0018]

Next, embodiments of the present invention will be described with reference to the drawings.

In the following embodiments, the input data of the present invention is assumed to be moving image data encoded by a method based on MPEG as moving image data encoded by utilizing frame correlation. However, the moving image data is not limited to this, and other than this, the moving image data encoded by the encoding method that can change the encoding method on a pixel-by-pixel basis using the correlation between frames is used. Can also be applied.

Further, as a method of expressing pixels, H.264 is used. 26
1 and the representation by the macro block adopted by MPEG are assumed. However, the pixel representation method is not limited to this, and may be applied to other pixel representations.

In MPEG, the frame of the moving image is
There are (1) a frame in which coding is performed exclusively using correlation within a frame without using correlation between frames, and (2) a frame in which coding is performed using correlation between frames.

In MPEG, the type of coding can be changed in units of macroblocks. There is a distinction in the type of coding between (a) use of correlation between frames and (b) use of motion compensation. There are four possible combinations of (a) and (b), but since there is no combination that does not use correlation between frames and that uses motion compensation, there are actually three types.

Next, the moving image data coded by utilizing the correlation between frames, which is the basis of the operation of the present invention, is coded by using the correlation between frames and without using motion compensation. The basis of the property that pixels are temporally and spatially concentrated in the telop part is described.

First, a method and a feature amount that are generally used for determining the pixel coding will be described. H. 261 and M
According to an encoding method that uses correlation between frames, such as PEG, for the purpose of optimizing encoding efficiency, (1) distribution of pixel values within a frame (hereinafter abbreviated as var)
And (2) variance of pixel value differences between frames when motion compensation is used (interframe motion compensation error variance) (hereinafter abbreviated as vmc), and (3) between frames when motion compensation is not used. The three variances of the pixel value difference (error variance between frames) (hereinafter abbreviated as v0) in
By selecting the case where the variance is as small as possible, the most efficient pixel encoding type is determined. The method will be described with reference to FIG.

The step number (stage number) in the following flow chart is written with S instead of the step. First, var and vmc are calculated (S21), and var and vm are calculated.
c is compared (S22), and if vra is small, coding is performed without using correlation between frames (S23). Otherwise, coding is performed using the correlation between frames. Further, when the correlation between frames is used, v0 is calculated (S
24), vmc is compared with v0 (S25), if v0 is small, it is encoded without using motion compensation (S26), and if not, it is encoded using motion compensation (S27).
Further, in the figure, α in S25 is a bias value larger than 1, and v
When mc and v0 are almost equal, encoding without motion compensation is prioritized. The reason is that if the variances are almost equal, the code amount relating to the pixel value does not change, and therefore it is considered advantageous to omit the code for expressing motion compensation because the total code amount decreases.

Next, how the pixels in the telop portion and the pixels in the other portions tend to be coded will be described based on the pixel characteristics of the moving image.

Since the pixels in the telop part have the property that they do not change for a certain period of time and do not move to the same position, vmc
Tends to be smaller. In addition, since the contrast of the telop portion is higher than that of the pixels of other portions, va
r tends to be large. As a result, vmc is replaced by v
Since ar becomes large, encoding using the correlation between frames tends to be adopted for the pixels of the telop portion.

In the telop portion, since there are no pixels that are strongly edged and resemble the surroundings, the motion vector of the pixel between frames is 0 or very small. This gives vmc and v0
Are similar to each other, so that there is a tendency that the encoding in which the motion compensation is not used is adopted for the pixels in the telop portion.

As for moving parts other than the telop, vmc is obviously smaller than v0, so that the coding using motion compensation tends to be adopted.

Pixels that are stationary in a portion other than the telop, such as a stationary background, actually tend to have v0 that is not 0 due to screen flicker and noise. But,
Pixels other than the telop part do not have a strong edge compared to the telop part, and because there are pixels similar to the surroundings,
Although it has not actually moved, it is determined that the pixel has moved to the positions of similar pixels in the surroundings, and the motion compensation tends to make vmc smaller than v0. As a result, for pixels that are stationary in parts other than the telop, there is a tendency that encoding using motion compensation is adopted.

Similarly, in the telop part, v0 tends not to be 0 due to screen flicker or noise.
Edges are stronger than flicker and noise on the screen, and pixels similar to the surroundings do not exist, so that motion is not detected and vmc tends to be the same as v0 and not become small. Therefore, even if you consider flicker and noise on the screen,
Coding without using motion compensation tends to be adopted for pixels in the telop portion. Due to the above properties of the telop portion and the other pixels, the pixels of the telop portion tend to be coded using the correlation between frames and without motion compensation, and the pixels other than the telop are It was found that there is a tendency to be encoded by the method of. As a result, in the moving image data encoded by using the correlation between frames, the pixels encoded by using the correlation between frames and without motion compensation are temporally and spatially concentrated in the telop portion. Can be said.

Twelve embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a flow chart showing a first embodiment of the moving picture telop extraction method of the present invention. First, M
The moving image data encoded by PEG is input in frame units (S1), and then it is determined whether the frame is encoded by using the correlation between frames (S2). A frame coded without using correlation between frames is not specifically processed in this embodiment, but may be used in other embodiments. Frames coded using the correlation between frames are output in macroblock units in the next counting step (S3). In the counting step S3,
Regarding the macroblock coding method, it is determined whether or not the macroblock has been coded using correlation between frames and without using motion compensation. Based on the determination result, the number of macroblocks is counted, and the coefficient is stored in a two-dimensional counting matrix corresponding to the position of the macroblock. The coefficient is repeated within the given counting interval (S4). When the counting within the counting section is completed, the process proceeds to the next selection step (S5). In the selection step (S5), the value stored in the counting matrix is compared with the threshold value to select the telop portion. This is set as the processing in one counting section, and the processing in the next counting section is performed if necessary (S6). Although omitted in the figure, the counting matrix is initialized prior to the processing.

Three embodiments will be described below for the counting step of the first embodiment.

FIG. 3 is a flow chart showing the second embodiment, which is an example of the counting step of the first embodiment.
In the counting step, a macro block is input from a frame (S31), a coded macro block is discriminated using correlation between frames and without motion compensation (S32), and based on the position of the macro block. Then, 1 is added to the value of the counting matrix (S33). This is repeated for all macroblocks in the frame (S3
4). When the processing of all macroblocks is completed, the processing of the counting stage is ended (S35).

If the counting matrix is M and the macroblock is B, then M (i, j) ← M (i, j) +1; B (i, j) is interframe correlation coding without motion compensation. Macroblock M (i, j); Represented as others.

In this embodiment, since the position where a macroblock coded without using motion compensation and using the correlation between frames appears at least once is extracted as a telop, it is judged as a telop by human eyes. Pixels that are not selected may be erroneously extracted, but there is an advantage that telop extraction omission can be reduced.

FIG. 4 is a flow chart showing a third embodiment, which is an embodiment in which the counting step of the first embodiment is changed. In the counting step, a macroblock is input from a frame (S41), and a coded macrobook is discriminated using correlation between frames and without motion compensation (S42). For macroblocks coded with correlation between frames and without motion compensation:
1 is added to the value of the counting matrix (S43), and 1 is subtracted from the value of the counting matrix for macroblocks that are not (S44). This is repeated for all macroblocks in the frame (S45). When the processing of all macroblocks is completed, the processing of the counting stage is ended (S46).

Similarly, this processing is performed by M (i, j) ← M (i, j) +1; B (i, j) is an interframe correlation coding macroblock M (i, j) which does not use motion compensation. -1; Expressed as other than that.

In this embodiment, the case where the portion which is not the telop is accidentally determined to be the telop can be canceled depending on the case where it is not determined.
There is an advantage that the possibility of erroneous extraction can be reduced in this embodiment.

FIG. 5 is a flow chart showing a fourth embodiment, which is a further different embodiment of the counting step of the first embodiment. In the counting step, a macroblock is input from a frame (S51), and a coded macroblock is discriminated using correlation between frames and without motion compensation (S52). For a macroblock coded using correlation between frames and not using motion compensation, 1 is added to the value of the counting matrix (S5
3) For other macroblocks, the value of the counting matrix is set to 0 (S54). This is repeated for all macroblocks in the frame (S5
5). When the processing of all macroblocks is completed, the processing of the counting stage is ended (S56).

Similarly, this processing is expressed as M (i, j) ← M (i, j) +1; B (i, j) is an interframe correlation coding macroblock 0 without motion compensation; To be done.

In this embodiment as well, it is possible to cancel the case where the part which is not the telop is accidentally judged to be the telop depending on the case where it is not judged, so that the possibility of erroneous extraction in the second embodiment is reduced. The advantage is that

In the following, four different embodiments will be described for the selection stage of the first embodiment.

FIG. 6 is a flow chart showing a fifth embodiment, which is different from the first embodiment in the selection step. In the selection stage, the counting matrix M and the threshold T
h, the number of frames n used for counting is input (S6
1). A value obtained by dividing the value M of the counting matrix by n is compared with a threshold value (S62), and a portion larger than the threshold value is output as a telop area. When the two-dimensional area matrix representing the existence of the telop area is R, the value of R at the position where the telop exists is set to 1 (S63), and the other part is set to 0.
(S64). This is repeated for all M elements (S65), and the finally obtained R is output as a region matrix (S66).

At this stage, It is expressed as

The threshold value may be a constant value given in advance or may be changed. For example, the threshold value may be adaptively changed for each counting section so that the number of telop portions extracted in the counting section does not exceed a certain number.

FIG. 7 is a flow chart showing a sixth embodiment, which is an embodiment in which the selection step of the first embodiment is different. In this embodiment, there are a projection step of creating a one-dimensional histogram in which the counting matrix that is the output of the counting step is projected in the vertical or horizontal direction, and a selection step of selecting a section in which the frequency of the histogram is larger than a threshold value as a telop area. Have. In FIG. 7, the counting matrix M and the threshold T
Input h and the number n of frames used for counting (S7
1). Next, M is projected in the horizontal direction, and the projection histogram H
Is created (S72). Then, a section [a, b) in which the value of H is divided by n is larger than the threshold Th is obtained (S7).
3), the value of the area matrix R is set to 1 for this section (S74). This is repeated until all H elements are compared with the threshold value (S75), and finally obtained R
Is output as a region matrix (S76).

Although FIG. 7 shows an example of lateral projection,
Embodiments using vertical projection may be similarly created.

FIG. 8 is a flow chart showing a seventh embodiment, which is different from the first embodiment in the selection step. In this embodiment, the portion selected by the sixth embodiment is further projected in the projection direction of the histogram, which is not used in the sixth embodiment, and the section in which the histogram value is larger than the threshold value is selected as the telop area. To do. In FIG. 8, the counting matrix M, the thresholds Th1 and Th2, and the number of frames n used for counting are input (S81). Next, M is projected in the horizontal direction to create a projection histogram H1 (S82). Next, the value obtained by dividing H1 by n is the threshold Th.
A section [a, b) larger than 1 is obtained (S83), and M is further projected in the vertical direction for the section to create a projection histogram H2 (S84). Next, a section [c, d) in which the value obtained by dividing H2 by n and the section size (ba) is larger than the threshold Th2 is obtained (S85). Then, the value of the area matrix R is set to 1 for the sections [a, b) and [c, d) (S86). This is all H2 and H
Repeat until elements of 1 are compared (S87, S8
8) Then, the finally obtained R is output as a region matrix (S89).

Further, an embodiment in which the order of the projection directions in FIG. 8 is reversed may be similarly created. In the seventh embodiment, it is possible to extract the telop with higher accuracy than in the sixth embodiment.

FIG. 9 is an eighth embodiment and is a flow chart showing an example of a selecting step for further narrowing down the results of the selecting steps of the fifth to seventh embodiments.

The input at this stage is the region matrix R
And a threshold value indicating the upper limit and the lower limit of the feature amount of various areas (S91). First, the connected region R ′ of the region matrix is obtained (S92). The connected region is a region in which the values of the region matrix are 1 and the portions in contact with each other are collected. As for the direction of contact, four neighborhood connections are assumed to be in contact with the four directions of up, down, left, and right, and four directions of upper right, upper left, lower right, lower left
There are 8-neighbor connections with added directions. The process of obtaining the connected region is sometimes called labeling.

For each connected region that is an element of R '(S93), the width and height, the area, and the position of the center of gravity are obtained,
Each value is further selected using a threshold value (S9
4). For the selected connected region, the portion occupied by it is added to R ″ (S95).
The element of is repeated (S96), and the finally obtained R ″ is output as a region matrix (S97).

As the value used for selection, various values other than the value used here may be used. For example, values such as the width-height ratio and the perimeter of the area may be used.

In this embodiment, if a threshold value created according to a typical usage of telops is given, an area which is erroneously extracted as a telop can be removed and the accuracy of telop extraction can be improved. The advantage is that

FIG. 10 is a flow chart showing an example of the moving picture telop extraction method according to the ninth embodiment. In this embodiment, a merging step (S105) for merging the values of the counting matrix is inserted between the counting step and the selecting step of the first embodiment. Counting stage (S103)
The second to eighth embodiments can be used in the selection step (S106).

In the merging step, a Gaussian filter is applied to the counting matrix to smooth the values of the counting matrix, making it difficult to select a portion having a low frequency, and merging a portion having a high frequency of proximity.

This stage is expressed as M ← G ^* M. However, G is a Gaussian filter G (i, j) = (1 / 2πσ ^ 2) exp (-(i ^ 2
+ J ^ 2) / 2σ ^ 2). It is.

The merging method is not limited to the Gaussian filter, and other smoothing method or area merging method may be applied.

This embodiment is advantageous in that the disparate telop areas can be spatially put together and the problem of excessive division of areas can be solved.

FIG. 11 is a flow chart showing an example of the moving picture telop extraction method according to the tenth embodiment. In this embodiment, the accumulation step (S116) of accumulating the information on the telop portion is added to the first embodiment, but it may be similarly applied to the ninth embodiment. Also, the second to eighth embodiments can be used in the counting step (S113) and the selecting step (S115).

The information to be accumulated is one of the information about each telop area such as the index of the frame used for extraction, the counting matrix, the area matrix, the projection histogram, the width, height, area of each telop area, and the position of the center of gravity. One or a combination of two or more.

The accumulated information relating to these telop portions can be used for purposes such as retrieval, editing, and decoding of moving images using telop portions.

Various periods may be set for the storage information storage period. For example, a storage period equivalent to that of a moving image can be set for the purpose of searching or editing. Further, when the main purpose is to decode only the telop portion from the moving image and there is no need for long-term storage, the minimum period necessary for decoding can be set.

Further, when telops occupying the same area are repeatedly extracted, it is possible not to accumulate the information regarding the overlapping telops in order to reduce the amount of accumulated information.

FIG. 12 is an eleventh embodiment and is a flow chart showing an embodiment of the moving picture telop extraction method. This embodiment is an embodiment in which a decoding step (S125) capable of decoding only the telop portion is added to the first embodiment, but it may be similarly applied to the ninth and tenth embodiments. In addition, the counting step (S123) and the selection step (S12)
The second to eighth embodiments can be used in 4).

In this embodiment, the telop portion is extracted by setting the interval between two frames in which the intraframe predictive coding is used as the counting section (S122). In order to decode the telop part, among the macroblocks that make up the intra-frame predictive coding frame that is the delimiter after the section,
Only the macroblock corresponding to the telop part selected in the selection step is decoded.

Although the telop part is decoded using the intraframe predictive coding frame which is the delimiter after the section, the intraframe predictive coding frame which is the delimiter before the section is also used. Alternatively, an interframe predictive coding frame may be used.

FIG. 13 is a block diagram of an embodiment of the moving picture telop extracting apparatus of the present invention. The input encoded moving image data is moving image data encoded by utilizing the correlation between frames, and here MPEG is assumed. The encoded moving image data may be data stored in a storage medium such as a video tape or a magnetic disk, as well as data transmitted by wireless or wired broadcasting, a LAN line or a telephone line.

First, the frame discriminating unit 1 discriminates between a frame (P) in which interframe predictive coding is performed and a frame (I) in which intraframe predictive coding is performed. The P frame is sent to the counting section 2 and the counting matrix 3 is created. Details of the counting unit will be described later. The counting matrix can be switched by a switch as to whether or not to pass through the merging unit 4. At the switch position b, the merging unit 4 merges the values using the Gaussian filter according to the method of the ninth embodiment.
The selection unit 5 creates the area matrix 6 based on the result of comparison with the threshold value. The selection unit will be described later in detail. The decoding unit 7 decodes the macroblock at the position where the area matrix is 1 in the intraframe predictive coding frame output from the frame determination unit 1,
Output as a telop image. In addition, the storage unit 7 stores the information about the region output from the selection unit 5, the region matrix 6, the image data decoded by the decoding unit 7, and the like.

Next, details of the counting section will be described with reference to FIG. Macro blocks are sequentially input to the counting unit. First, the position decoding unit 11 decodes only the code indicating the position of the macro block, and becomes the position information of the counter. next,
The coding discrimination unit 12 discriminates the type of coding and discriminates whether the coding is performed using the correlation between frames and without using motion compensation (indicated as "noMC coded" in the figure). To be done. The determination result is used as an increase / decrease or reset signal for the counter 13. The methods of the second, third, and fourth embodiments can be switched by the switch positions a, b, and c, respectively. Finally, the counter values are output as a counting matrix.

Next, using FIG. 15, the selection unit 5 (FIG. 13)
Will be described in detail. The counting matrix is input to the selection unit 5. At the switch position a, the value of the counting matrix is compared with the threshold value according to the method of the fifth embodiment, and the area synthesizing unit 28 creates the area matrix.
At the switch position b, the method of the sixth and seventh embodiments is followed. In this case, the horizontal projection unit 22 projects in the horizontal direction to create a projection histogram 23. The comparison unit 4 compares the value of the projection histogram with the threshold value. The position c of the switch is sent to the area synthesizing section 8 immediately after the result of the comparison according to the method of the sixth embodiment, and an area matrix is created. At the switch position d, the vertical projection unit 5 further projects in the vertical direction according to the method of the seventh embodiment to create a projection histogram 26. The comparison unit 7 compares the threshold value with the threshold value, and the result is sent to the region synthesis unit 28 to create a region matrix. At the switch position e, the area matrix created by the area composition unit 28 is output as it is. At the switch position f, the connection area 30 is created by the connection area creating unit 29 according to the eighth embodiment. Each connected region is compared with a threshold value by the comparison unit 31, and based on the result, the region combination unit 32 newly creates a region matrix and outputs it. Information regarding the area is also output and sent to the storage unit 8 in FIG.

The present invention is not limited to the above-described embodiments, but can be variously modified and implemented within the scope of the invention. For example, various sections can be set such that the counting section is a scene between cut points or a fixed time interval.

Also, after decoding the telop portion, a conventional telop extraction method may be used to improve the accuracy of the telop extraction.

Further, the subtitle in the telop may be recognized after the telop portion is decoded.

Further, the original image data decoded for each telop part may be stored as the information on each telop part accumulated in the accumulating stage.

When the telop represents a character, the result of character recognition of the telop may be stored as the information on each telop portion accumulated in the accumulation step.

[0079]

As described above, the present invention is a method for extracting a telop portion of coded moving image data by utilizing the correlation between frames, which uses the correlation of framing and motion compensation. It is possible to extract the telop portion from the encoded moving image data without decoding the frame image by the extraction step of extracting the portion where the encoded pixels are temporally and spatially concentrated without using Therefore, there is an effect that the telop portion can be extracted at high speed with a small processing cost for the encoded moving image.

Since the telop portion can be extracted at high speed and the information on the telop portion can be accumulated at a low processing cost without decoding the frame image from the encoded moving image data, the content of the moving image can be stored. There is also an effect that it is possible to easily perform processing such as search and edit based on it.Since the telop portion can be extracted at high speed without decoding the frame image from the encoded moving image data, it is encoded. There is also an effect that image data obtained by decoding only the telop portion from the moving image data can be created at high speed.

[Brief description of drawings]

FIG. 1 is a flowchart of a first embodiment of a moving image telop extraction method of the present invention.

FIG. 2 is an explanatory diagram showing a procedure for determining the type of encoding.

FIG. 3 is a flowchart of a second embodiment in which the counting step of the moving image telop extraction method of the present invention is different.

FIG. 4 is a flowchart of a third embodiment in which the counting step of the moving image telop extraction method of the present invention is different.

FIG. 5 is a flowchart of a fourth embodiment in which the counting step of the moving image telop extraction method of the present invention is different.

FIG. 6 is a flowchart of a fifth embodiment in which the selection step of the moving image telop extraction method of the present invention is different.

FIG. 7 is a flowchart of a sixth embodiment in which the selection step of the moving image telop extraction method of the present invention is different.

FIG. 8 is a flowchart of a seventh embodiment in which the moving image telop extraction method of the present invention has different selection steps.

FIG. 9 is a flowchart of an eighth embodiment in which the selection step of the moving image telop extraction method of the present invention is different.

FIG. 10 is a flowchart of a ninth embodiment of the moving image telop extraction method of the present invention.

FIG. 11 is a flowchart of a tenth embodiment of the moving image telop extraction method of the present invention.

FIG. 12 is a flowchart of an eleventh embodiment of the moving image telop extraction method of the present invention.

FIG. 13 is a block diagram of an embodiment of a moving image telop extraction device of the present invention.

14 is a detailed block diagram of the counting unit 2 shown in FIG.

15 is a detailed block diagram of a selection unit 5 shown in FIG.

[Explanation of Codes] 1 Frame Discriminating Section 2 Counting Section 3 Counting Matrix 4 Gaussian Filter 5 Selecting Section 6 Area Matrix 7 Decoding Section 8 Accumulating Section 11 Position Decoding Section 12 Encoding Discriminating Section 13 Counter 21 Comparing Section 22 Horizontal Projecting Section 23 Projection histogram 24 Comparison unit 25 Vertical projection unit 26 Projection histogram 27 Comparison unit 28 Region synthesis unit 29 Connected region creation unit 30 Connected region 31 Comparison unit 32 Region synthesis unit

 ──────────────────────────────────────────────────の Continued on the front page (72) Inventor Hiroshi Hamada 3-19-2 Nishi-Shinjuku, Shinjuku-ku, Tokyo Nippon Telegraph and Telephone Corporation

Claims (16)

[Claims] 1. A moving image for extracting a telop portion, which means an inserted character such as subtitles, photographs, symbols, and icons on the screen, of moving image data encoded by utilizing the correlation between frames which means a unit screen. An image telop extraction method, characterized by comprising an extraction step of extracting a portion where pixels coded without using motion compensation and using correlation between frames are concentrated temporally and spatially. Image telop extraction method. 2. A counting step of counting a frequency of appearance of a pixel coded by using correlation between frames and not using motion compensation in the extracting step, and a frequency of the pixel appearance and a threshold value. The moving image telop extraction method according to claim 1, further comprising a selection step of selecting a telop portion based on a comparison result. 3. A projection step of creating a frequency histogram of the pixel appearance, and a selection step of selecting a telop portion based on a result of comparison between the frequency indicated by the projection histogram and a threshold value.
The moving image telop extraction method described. 4. The method according to claim 2, further comprising, after the counting step, a merging step of merging frequencies based on spatial characteristics of the frequencies.
The moving image telop extraction method described in. 5. The moving image telop extraction method according to claim 2, further comprising a storage step of storing a telop portion extracted from the moving image. 6. The moving image telop extraction method according to claim 2, further comprising a decoding step capable of decoding only a telop portion extracted from the moving image. 7. The moving image according to claim 2, wherein the counting step assigns 1 only to a pixel coded with correlation between frames and without motion compensation, and then selects. Telop extraction method. 8. The counting step assigns 1 to pixels coded with correlation between frames and without motion compensation, and -1 to other pixels, and then ,
The moving picture telop extraction method according to claim 2, wherein the moving picture telop is selected. 9. Using correlation between the frames, and
3. A pixel which is encoded without using motion compensation is provided with 1 and 0 is provided with respect to the other pixels, and then selected.
The moving image telop extraction method described. 10. The moving image telop extraction according to claim 2, wherein the selecting step compares a numerical value obtained by dividing a total value of numerical values given to the same pixel by the number of times of measurement with a threshold value and selects a pixel larger than the threshold value. Method. 11. The moving image telop extraction method according to claim 3, wherein a portion larger than a threshold value is selected for each direction and section of the histogram and set to 1, and the whole range is compared and selected. 12. Using correlation between the frames, and
Gaussian filter is used to smooth the sum of the numerical values assigned to each pixel coded without motion compensation, making it difficult to select infrequent parts, and merging steps that merge frequently adjacent parts. 5. The moving picture telop extraction method according to claim 4. 13. A moving image for extracting a telop portion which means an inserted character such as a subtitle, a photograph, a symbol, an icon on the screen of moving image data encoded by utilizing the correlation between frames which means a unit screen. An image telop extraction apparatus, characterized by having an extraction unit for extracting a portion in which pixels coded without using motion compensation using correlation between frames are temporally and spatially concentrated. Image telop extractor. 14. A frame discrimination unit for discriminating between a frame subjected to inter-frame coding and a frame subjected to intra-frame coding, and encoded moving image data of the frame subjected to inter-frame coding. A counting unit that projects a count matrix by projecting onto a predetermined plane, a switch that selectively connects a Gaussian filter, a count matrix that compares the count matrix with a threshold value, and an area matrix that displays pixel portions larger than the threshold value on the predetermined plane. A comparison unit to be created, a decoding unit for decoding the encoded moving image using the area matrix and outputting a telop image, and a storage unit for storing the area matrix and the image data decoded by the decoding unit The moving picture telop extraction device according to claim 13. 15. A position decoding unit, wherein the counting unit decodes only a code indicating a position of a macroblock which is an input unit pixel portion to obtain position information, and uses a correlation between frames, and performs motion compensation. And a coding determination unit that determines whether or not the pixel is a coded pixel, and 1 is assigned to the pixel data that is coded using the correlation between frames and without motion compensation. 15. The moving picture telop extraction device according to claim 14, further comprising: a counter that outputs nothing to other pixels, adds -1, or resets them selectively to output a counting matrix. 16. The selection unit, when the count matrix is input, compares the count matrix with a threshold value and outputs the result, and a horizontal projection to generate a horizontal projection histogram. A horizontal projection unit, a second comparison unit that compares the horizontal projection histogram with a threshold value, a vertical projection unit that performs vertical projection to create a vertical projection histogram, and a vertical histogram And a threshold value, a third comparing unit, a region synthesizing unit that records a pixel larger than the output threshold value from each comparing unit to create a region matrix, a connected region creating unit that creates a connected region, and the connected region 15. The moving image telop extraction device according to claim 14, further comprising a fourth comparing unit that compares pixel data for each region with a threshold value, and a second region combining unit that creates a region matrix from the comparison result.