JP3109682B2 - Detection method of character area in video - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for detecting a character area in a moving image from a moving image.

[0002]

2. Description of the Related Art With the recent rapid development in the technology of combining a computer and a moving image, the conventional moving image technology (accumulation,
In addition to access, processing, and searching), the possibility of technology for interactively handling moving images has been studied. In particular,
There is an increasing need for a technique for automatically defining characteristics of a moving image in moving image search and processing and indexing the characteristic to the moving image. Recently, as a method of extracting the physical meaning as indexing into a moving image, a method of detecting a video change point from a change in a luminance histogram between frames has been reported. "Otsuji, Totomura, Oba: Moving images using luminance information Browsing, Information Processing Society of Japan, Image Engineering Study Group, 1991. " Also, a method of detecting a scene change from a change in difference between frames or between fields has been reported. "Yanaka, Endo, Kokura, Ishibashi: Video signal browsing apparatus, JP-A-64-68084, pp5.
61-566 ". The above report relates to a scene change point in the specific physical meaning of a moving image, and no report is made on the physical meaning of a scene. The unique physical meaning of a moving image includes the physical meaning in a scene, and a method of specifying an input device operation when inputting a moving image has been reported as one of methods for specifying the physical meaning. "Akutsu, Oba: Extracting physical features from video,
IPSJ National Convention, 1991 ". However, the above report relates to a general scene or a part of a specific physical meaning of a moving image, and no report is made on the physical meaning of a frame or a field.

The physical meaning of a moving image includes the physical meaning of a frame or a field, and how to define the physical meaning of a frame or a field depends on how to improve the handling technology of a moving image. It is thought to hold the key. However, there is a problem that a report on the definition of the physical meaning of a frame or a field has not been made yet.

[0004]

As described above, in examining a more interactive moving image handling technique associated with a rapid combination of a computer and a moving image, the physical meaning of a frame or field as a useful index is considered. There is a need to specify.

[0005] Today, with the development of video cameras and related peripheral devices, it has become possible to easily superimpose characters on moving images, not only at broadcasting stations but also at home level.

[0006] At both the broadcasting station and the home level, when superimposing characters on a moving image, it can be understood that the action represents some intention of the inserting side. The viewer also receives a great deal of other information from the image by text. Based on the above, it is recognized that the validity of being able to detect a character area in a moving image is a physical meaning of a frame or a field for studying a moving image handling technique.

The present invention has been made in order to solve the above problems.
It is an object of the present invention to provide a method for detecting a character region in a moving image, which detects a character region included in the moving image by using a spatio-temporal physical characteristic of the moving image.

[0008]

SUMMARY OF THE INVENTION A method for detecting a character area in a moving image according to the present invention is included in a moving image by using spatio-temporal physical characteristics relating to two-dimensional position, wavelength, and time, which are variables of the moving image. This is to detect the regularity of the spatiotemporal physical quantity change of the character area and detect the character area included in the moving image. As a spatio-temporal physical feature, it has an area composed of dots having the same light intensity and single color, an edge stands in this area, and the area has the same light intensity, single color, and the edge is invariable for a certain period of time. Some features, such as the fact that a region exists in a plurality of frames for a certain period of time, are used alone or in combination of a plurality of combinations in consideration of the order.

[0009]

According to the present invention, since the timing of insertion of character information in a moving image can be determined mechanically, in addition to the moving image technology (accumulation, access, processing, search), a technology for interactively handling moving images can be used. Useful indices can be automatically detected. The details will be described below with reference to the drawings.

[0010]

DESCRIPTION OF THE PREFERRED EMBODIMENTS A character area superimposed on a moving image has the following spatio-temporal physical feature regularity. (Spatial Feature) (A-1) The character area is composed of dots (pixels) of the same light intensity and single color. When a character area is superimposed on a gradation image (general moving image or still image), it is inserted with a certain gradation difference from the necessity of being distinguished from the background. Therefore, (A-2) an edge stands in the outline of the character area. (Temporal characteristics) The superimposed character area is
For human understanding, (A-3) The dots (pixels) that make up the character area are:
Light intensity and color do not change for a certain period of time. (A-4) The same character area exists in a plurality of frames within a certain time.

In the detection of a character area in a moving image, the spatio-temporal physical features are used alone or in combination with the spatio-temporal physical features (A-1) to (A-4). The method of use depends on the type of moving image to be handled, for example, a binary image, a gradation image, and the like, and a method of superimposing a character area. The method of superimposing is classified into a method of inserting and deleting a character area, a method of causing a character area to exist in a moving image, and the like. When a character area is inserted or erased, the configuration bits (pixels) corresponding to the superimposed or superimposed character area are as follows:
The light intensity and color are temporally changed based on the characteristics, (A-1) and (A-3). They are shown in FIG. 1-1
Is a case where luminance, chromaticity, hue, etc. change rapidly with time, and a character suddenly appears or disappears when observing a moving image. 1-2 is temporal luminance, chromaticity,
This is a case where the hue or the like changes gradually, and a case where characters gradually appear or disappear when observing a moving image. This is called the character's fade-in (gradual appearance) and fade-out (gradual disappearance).

FIG. 2 shows how a character area is present in a moving image. In FIG. 2, the ON state indicates that a character area is present in the image. For example, 2-1 is a case where a character area exists from a certain point of time of a scene or cut (a in FIG. 2) to a certain point of time (b in FIG. 2). This shows a case where a character area exists from the beginning of a scene or cut (a in FIG. 2) and the character area disappears at a certain point in the scene or cut (b in FIG. 2). Hereinafter, 2-3 is a case where a character area is inserted in the middle of a scene or cut (a in FIG. 2), and a character area is present in the image until the end of the scene or cut. This shows a case where a character area is inserted from the beginning to the end.

As an embodiment of the present invention, for a general moving image, such as a television image or a video image, the character area is determined based on the spatio-temporal physical characteristics based on the respective superimposing methods shown in FIGS. A method for detecting a character region in a moving image using the method will be described below.

The method of using the spatiotemporal physical features for detecting a character area in a moving image is as follows.
Algorithm of the method using spatio-temporal physical features in the detection of character areas in moving images
Considers the following in consideration of its processing efficiency.

First, temporal physical features are used, and then spatial physical features are used.

Considering the above, it is expected that the time efficiency of the character area detection processing in the moving image will be improved. Detecting a character area in a moving image means that the following information is obtained from the moving image. (B-1) Detection of points a and b in FIG. (B-2) Detect the position of the existing character area in the frame or field. (B-3) Detection of size of existing character area.

First, a method of detecting the points a and b of the information B-1 will be described. The spatiotemporal physical characteristics used are A-1 and A-3 described above. Point a, b from A-1 and A-3
The point can be interpreted as a point where the trend intensity and the monochrome dot (pixel) constituting the character area change with time. As a result, the detection of the points a and b can be performed from the difference between the frames. This can be considered from the fact that the luminance, chromaticity, hue, and the like between the frames corresponding to the light intensity and color of the character area inserted or lost at the points a and b change with time.

Next, a description will be given of a method of detecting the position in the frame or field of the character area where the information B-2 exists, and a method of detecting the size of the existing character area. The spatiotemporal physical features used are A-1 and A-2 above. The position of the character area from A-1 and A-2 is a place where dots (pixels) of the same light intensity and single color are concentrated in a frame or a field, and edges are also concentrated. Can be interpreted as a place where From this,
The position of the character area can be detected from the same light intensity, the density distribution of a single-color dot (pixel) in the frame or field, or the concentration of the edge frame or the density in the field. The size of the character area can be detected from the fact that the density distribution in the frame or the field has a certain width corresponding to the size of the character area.

FIG. 3 shows a specific example of an embodiment of the present invention for detecting a character area in a moving image using each of the above methods. In the figure, a and b indicate insertion or vanishing points (frame and time), and "Iroha" indicates a character area superimposed by 1-2 in FIG. 1 and 2-1 in FIG. First, histograms of differences such as luminance, hue, and chromaticity between frames are obtained from a moving image, and peaks of the histograms (A, A in FIG. 3) are obtained.
A and b are detected from the presence or absence of B), and the light intensity and color information of the character area can be obtained at the same time (procedure φ1 in FIG. 3).
Next, a binary image and an edge image are created using the acquired light intensity, color information, and the moving image. This method (procedure φ in FIG. 3)
2) includes median filtering, level slicing, thinning, edge filtering, and the like. next,
Each density distribution is calculated from the binary image and the edge image. This method (procedures φ3 and φ4 in FIG. 3) includes calculation of block division density, texture analysis, use of coefficient change of block DCT, and the like. From these density distributions, the variance, standard deviation, center of gravity, moment axis, and the like are calculated (procedure φ5 in FIG. 3), and the position and size of the character area can be detected therefrom.

Using the detected size and the variance, standard deviation, center of gravity, moment axis, etc. of the calculated density, the character type,
Character font can be determined. This is because the character type and the character font have unique variance of density, standard deviation, center of gravity, moment axis, and the like. The specific example of the embodiment of the character detection in the moving image according to the present invention has been described above.

[0021]

As described above, according to the present invention, the spatio-temporal physical quantity change of a character region included in a moving image is performed by using spatio-temporal physical characteristics relating to two-dimensional position, wavelength, and time, which are variables of the moving image. The character region included in the moving image can be detected by detecting the regularity of the character region included in the moving image. As a result, the insertion of the character region information in the moving image can be performed. Timing and other information can be determined mechanically and at high speed, and in addition to video technology (accumulation, access, processing, search), useful indexes for technologies that handle video interactively can be automatically detected. There are advantages.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating types of insertion and deletion of a character area related to superimposition for explaining the principle of the present invention.

FIG. 2 is a timing chart showing the type of existence of a character area in an image relating to superimposition in FIG. 1;

FIG. 3 is a diagram showing a character area search procedure according to a specific example of the present invention.

[Explanation of symbols]

ON The state where the character area exists in the image OFF The state where the character area does not exist in the image a Frame and time when the character area is inserted b Frame and time when the character area is lost A Luminance and hue caused by inserting the character area , Chromaticity, etc., the frequency peak B. Luminance, hue,
Frequency peak of change amount such as chromaticity

Continuing from the front page (72) Inventor Seita Otsuji 1-1-6 Uchisaiwaicho, Chiyoda-ku, Tokyo Nippon Telegraph and Telephone Corporation (72) Inventor Hiroyuki Kimiyama 1-1-6 Uchisaiwaicho, Chiyoda-ku, Tokyo Nippon Telegraph and Telephone Stock In-company (56) References JP-A-62-190991 (JP, A) JP-A-3-282779 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) G06K 9/20- 9/82 G06T 7/00-7/60 JICST file (JOIS)

Claims (2)

(57) [Claims] A first step of finding a regularity of a change in a spatiotemporal physical quantity of a character region included in the moving image by using spatiotemporal physical characteristics relating to two-dimensional position, wavelength, and time, which are variables of the moving image; A method for detecting a character region in a moving image, comprising detecting a character region included in the moving image. 2. A spatio-temporal physical feature is to have an area composed of dots having the same light intensity and a single color, that an edge is formed in this area, and that the area has the same light intensity and the same color for a certain period of time. The edge is invariable, the region is present in a plurality of frames for a certain period of time, and the character region included in the moving image is used using a plurality of combinations considering characteristics such as alone or in the order thereof. 2. The method according to claim 1, wherein the detection is performed.