JP2002014973A - Video retrieving system and method, and recording medium with video retrieving program recorded thereon - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce errors in the judgment of similarity caused by a gap between with meaning contents which the feature of a picture and voice has. SOLUTION: A character information recognition part 2 recognizes character information such as telop characters displayed in video with respect to video stored in a memory and outputs the character codes of plural character recognition result candidates with respect to one character displayed on a screen. A recognition result similarity calculation part 4 takes out a character code string candidate recognized from each video frame extending to two video frames from plural recognition results stored in a recognition result storing part 3 and calculates similarity between the character recognition results with respect to all of the pairs. A result outputting part 5 outputs the character code information and similarity of the character recognizing results.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video search device that determines the semantic similarity of video content or image content and searches for video based on the determination result.

[0002]

2. Description of the Related Art Techniques for judging the semantic similarity of video contents and image contents and displaying a video based on the judgment result have been studied in the past, and those proposed so far have been proposed. There is a technology described in Documents [1], [2] and [3]. Each of the technologies described in the literatures [1] and [2] defines the similarity of an image based on signal information such as color, shape, motion, and sound in the image, and the characteristics of the image signal have the meaning of the image. If the content is reflected, it is possible to effectively determine a similar image. Also,
In document [3], similarity between texts is obtained by using text information related to video content such as closed captions, drama scenarios, news manuscript articles, etc., which are present separately from the video, and the similarity between the texts is calculated. Is associated with similar images.

List of References [1] Eiji Kasuya, Masato Murai, Hideyoshi Tominaga, "Video Scene Retrieval Method Based on Extraction of Significant Information from Compressed Video", Proc. Of IEICE General Conference, D-12-74, p. .27
3, (1998-03) [2] Hidehiko Iwasa, Hidenori Yamamoto, Naokazu Yokotani, Haruo Takemura, "Similar Image Search Considering Compositional Similarity Between Images", Proceedings of the Annual Conference of the Institute of Image Electronics Engineers of Japan, pp .47-48, (199
0-06) [3] Shingo Miyauchi, Noboru Babaguchi, Tadahiro Kitahashi, "Event Detection of Video Media by Extracting Features from Language Streams", Proceedings of IEICE General Conference, D-12
-78, p.248, (2000-03) [4] Shoji Kurakake, Hidego Kuwano, Hiroyuki Arai, Kenji Ogura, "Study of Character Recognition for Video Search", NTTR & D, v
ol. 47, pp. 11-16, (1998-01)

[0004]

However, the techniques described in the above-mentioned documents [1], [2], and [3] have a problem that similar images cannot be determined in some cases.

[0005] In the techniques of Documents [1] and [2], the characteristics of the video signal do not always match the semantic contents of the video.
The determination of similarity may fail. For example, when judging the similarity of the use of color information between an image showing a red car and an image showing a yellow car, both have the same meaning of "the car is shown". Are different, it is determined that they are not similar in this case. Also,
Even when there are a plurality of videos having the same semantic content, if the characteristics of the video signals are completely different, it is difficult to find a semantic similarity between them by a method using the signal characteristics. For example, an image showing the “red sunset sky” and an image showing a “city with huge buildings” that are completely different in image signal characteristics such as color, shape, movement, sound, etc. Are common in the sense of "a scene in a dream seen by a certain person", it is difficult to determine that these are similar only by the characteristics of the video signal.

[0006] In addition, in the technique of Reference [3], it is necessary to prepare text information different from video in advance by some method, and it cannot be applied when only video exists.
In general, text information related to the contents of a video is owned only by a specific video user such as a video production company. Therefore, it is difficult to apply the technique of Ref. [3] to any video.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a video search device, a video search method, and a recording medium on which a video search program is recorded, which reduces errors in similarity judgment caused by gaps in the meaning and content of images and audio features. .

[0008]

In order to achieve the above object, a first video search device of the present invention comprises: a video input storage unit for inputting video data and storing the video data in a memory; A character information recognizing unit that extracts subtitles displayed in the video frame for each video frame from input video data and recognizes a character code string candidate of the subtitle, and a recognition unit that stores the character code string candidate. A result storage unit, a recognition result similarity calculating unit that calculates a similarity based on a degree of matching of characters constituting a character string candidate in a combination of all two video frames of the video frame, and the character code string It has a result output unit for outputting candidates and similarities.

A second video search device according to the present invention comprises: a video input storage unit for inputting video data and storing the video data in a memory; and a video data input from the video input storage unit. A character information recognition unit that extracts a subtitle displayed in a frame and recognizes a character code string candidate of the subtitle, a recognition result storage unit that stores the character code string candidate, and all two images of the video frame. A recognition result similarity calculation unit that calculates a similarity based on a degree of matching of characters forming a character string candidate in a combination of frames, and classifies the similarity information based on a predetermined rule; A similarity classifying unit that outputs similarity information that is identification information of each group; a similarity information storage unit that stores the similarity information output by the similarity classifying unit in a memory; Searches based on pre-given request sex information, has a search result output unit for outputting.

According to a third aspect of the present invention, there is provided a video search device for inputting video data and storing the video data in a memory, and for the video data input from the video input storage unit, for each video frame. A character information recognition unit that extracts a subtitle displayed in a frame and recognizes a character code string candidate of the subtitle, a recognition result storage unit that stores the character code string candidate, and all two images of the video frame. A recognition result similarity calculation unit that calculates a similarity based on the degree of matching of characters constituting a character string candidate in a combination of frames; and a classification unit that classifies the similarity information based on a predetermined rule. A similarity classification unit that outputs similarity information that is identification information of each group; a similar character recognition result storage unit that stores the similarity information output by the similarity classification unit in a memory; To the input video data, information that enables the video to be reproduced from an arbitrary point specified using a predetermined method is added, and a video storage unit that stores the video, and the similarity information A search result output unit that searches for similarity information of the video stored in the storage unit according to a request given in advance, requests video information corresponding to the search result to the video storage unit, and outputs the video information together with the search result.

The present invention automatically recognizes character information such as telop characters displayed in a video, and utilizes a similarity of a character code as a result of the recognition to accurately correspond to a video having a similar meaning and content. In addition, even if text information related to the content of the video does not exist in advance, it is possible to associate a similar video.

In the video, various titles, such as news titles, person names, and place names in a news video, recipe information of a cooking program, etc. Is displayed. The character code information obtained as a result of recognizing these character information using the recognition process is information at a level closer to the semantic content of the video than the signal characteristics of the video such as color, shape, motion, sound, etc. It can be said that it is more effective as an index for evaluating the similarity of the semantic contents of the video than the feature. For example,
2 raised in the above “Problems to be solved by the invention”
For one example, in the first example, "car"
When characters such as are displayed, it is impossible with color information, but since the present invention is applied, the character code "car" obtained by the character recognition process is common, so it is determined that both are similar. Becomes possible. Also in the following example, if the character information “Scene in a dream seen by a certain person” is displayed in both images, use the commonality of character codes obtained as a result of the character recognition According to the present invention, it is possible to determine the similarity between videos.

Further, in the present invention, the text information obtained by automatically recognizing character information such as telop characters displayed in the video is used for calculating the similarity of the video. It is possible to obtain text information related to the content. For this reason, even if there is no text information such as closed captions in advance,
It is possible to determine the similar video using the text information of the character recognition result.

According to the present invention, as text information obtained by the character recognition processing, a plurality of character codes obtained as a result of the character recognition processing are used for one character displayed in the video, so that one character is displayed. As compared with the case where only one character recognition result character code is used, there is a high possibility that a correct character mode is included in the character recognition result, so that it is possible to define a highly accurate similarity.

[0015] Further, in broadcast video and the like, since many highly topical videos are broadcast in news programs and the like, the present invention utilizes the similarity of the recognition result of character information such as telop characters to broadcast video. By associating the similar images in each other and outputting them for each similar image, the image user can know the topicality of the information through the output contents of the image.

[0016]

Next, embodiments of the present invention will be described with reference to the drawings.

Referring to FIG. 1, a similar image retrieval and display apparatus according to a first embodiment of the present invention includes a video input storage unit 1, a character information recognition unit 2, a recognition result storage unit 3, a recognition result similarity calculation unit 4, It comprises a result output unit 5.

The video input storage unit 1 receives video data to be processed and stores it in a memory. For example, a video signal from a video reproducing apparatus such as a video is stored in a memory on a computer using a video capturing board mounted on a personal computer.

The character information recognizing unit 2 applies character information such as telop characters displayed in the video to the video stored in the memory on the computer by the video input storage unit 1 using a predetermined method. And outputs character codes of a plurality of character recognition result candidates for one character displayed on the screen (individual still images constituting video data). As the predetermined method, for example, reference [4]
You can use software on a computer that implements the method described in.

The recognition result storage unit 3 is a hard disk of a computer or the like, and stores character code information of the character recognition result output from the character information recognition unit 2.

The recognition result similarity calculation unit 4 extracts character code string candidates recognized from each video frame over two video frames from a plurality of character recognition results stored in the recognition result storage unit 3 and For the two combinations,
Calculate the similarity between character recognition results. For example: • Character recognition result A: Tomorrow opening high school baseball • Character recognition result B: High school baseball, if there is a stop due to rain, the same character code that is included in both character recognition results A and B is “High school baseball”. There are character codes. A and B are codes for distinguishing video frames, respectively. On the other hand, if the similarity between the two is defined as “the number of common character codes”, the similarity is 4.
This example shows a case where one character code exists for one character as a character recognition result, for convenience of description of an example of calculating the similarity. However, there is actually an error in the character recognition processing, and a correct character code is not always obtained as in the above-described example. Therefore, the character information recognition unit 2 uses a plurality of character codes for one character on the screen as character recognition result candidates. Using the above example, a case where two character codes are obtained for one character will be described. Character recognition result A: Tomorrow closing, high school baseball (first candidate character code string): Atashikusa-Aman Mori Hayashi (Character code string of the second candidate) ・ Character recognition result B: Takabayashi baseball, rain and insect stoppage (Character code string of the first candidate): Amakumachi Toshi-Cloud and Chuho (Character of the second candidate) Code sequence). In this case, if only the character code string of the first candidate exists, the concept of the similarity indicates that:
The similarity between the two is “3” from the three character codes “high”, “field” and “ball”. When the similarity is calculated using the second candidate, four characters “high school baseball” are obtained. Since they are the same, the similarity is 4. By obtaining a plurality of character recognition results for one character displayed on the screen in this manner, the accuracy of the similarity is improved.

The result output unit 5 outputs character code information and similarity of the character recognition result.

FIG. 2 is a flow chart for explaining the processing of the recognition result similarity calculation unit 4, showing the specific processing steps for calculating the similarity between two character recognition results A and B. An example is shown. In this example, the similarity of the recognition result is defined by the number of character codes common to both, and the character recognition result A and the character recognition result B are assumed to be composed of K and L character codes, respectively. In step 41, the character recognition result A is read. In step 42, the character recognition result B is read. In step 43, the counter variable I is initialized to 1. In step 44, the counter variable J is initialized to 1. In step 45, the similarity S is set to 0
Initialize to In step 46, it is determined whether or not the counter variable I is equal to or smaller than K. If the counter variable is equal to or smaller than K, the process proceeds to step 47; In step 47, it is determined whether or not the counter variable J is equal to or less than L. If it is equal to or less than L, the process proceeds to step 48, and if it is greater than L, the process proceeds to step 51. In step 48, it is determined whether the I-th character code in the character recognition result A and the J-th character code in the character recognition result B are the same. If they are the same, the process proceeds to step 49; . In step 49, the similarity variable S is increased by one. In step 50, the counter variable J is increased by one. Step 51
Then, the counter variable I is increased by one. In step 52, the counter variable J is set to 1.

Referring to FIG. 3, a similar image retrieval and display apparatus according to a second embodiment of the present invention comprises a video input storage unit 1, a character information recognition unit 2, a recognition result storage unit 3, and a recognition result similarity calculation unit 4 And a similarity classifying unit 6, a similarity information storage unit 7, and a search result output unit 8.

This embodiment is similar to the first embodiment,
This shows an example of an apparatus configuration in which, after acquiring similarities of a plurality of pieces of character information displayed in a video, the similarities are classified, and a result is displayed based on the classification result. The components from the video input storage unit 1 to the recognition result similarity calculation unit 4 are the same as in the first embodiment. The similarity classifying unit 6 classifies the similarity information of the character recognition result obtained by the recognition result similarity calculating unit 4 based on a predetermined rule. For example, when the character “high school baseball combination announcement” is present as the character recognition result C, the character recognition results A and B, B and C, and
Each combination of A and C has the same character code, and in the above-described concept of similarity, each has a similarity of 4. In this case, for example, if all combinations having the same character code and a similarity of 4 or more are regarded as one similar group, A, B, and C are all determined to be similar images.

The similarity information storage unit 7 stores the similarity information of the character recognition result obtained by the similarity classifying unit 6 on a hard disk of a computer or the like. The similarity information includes, for example, an identification number of each character recognition result for each group of similar character recognition results. In the above example, the identification numbers A and B of the character recognition result A and the character recognition result B can be used as the similarity information.

The video output unit 8 retrieves and outputs similarity information of the video stored in the similarity information storage unit 7 based on a request given in advance.

In the configuration shown in FIG. 3, the similarity information calculated for the first video data already exists in the similarity information storage unit, and in this state, new video data is input and new video data is input. The present invention may be applied to a case where the similarity is calculated not only by the information of the obtained character recognition results but also by calculating the similarity again including the similarity information of the character of the first existing video data.

FIG. 4 is a block diagram of a similar image search and display device according to a third embodiment of the present invention. The image search device according to the second embodiment has an image storage unit 9.

The video storage unit 9 adds information to a video input to a memory such as a personal computer so that the video can be reproduced from an arbitrary point specified using a predetermined method, The video is stored in a storage device such as a hard disk of a computer. As the predetermined method, for example, a method of storing the frame information of each video and the storage position of the video in the video storage device corresponding to the frame information can be used.

From the search result output unit 8 to the similarity information storage unit 7
The search condition is output to the search result output unit 8 from the similarity information storage unit 7. The result at this time is not video but only textual information. Therefore, in order for the search result output unit 8 to output a video in which the character information output from the similarity information storage unit 7 is actually displayed, based on the result output from the similarity information storage unit 7,
The search result output unit 8 requests the video storage unit 9 for video information corresponding to the output result, and the video storage unit 9 answers the request.

FIG. 5 is a block diagram showing an example of the character information recognition unit 2, the recognition result storage unit 3, and the recognition result similarity calculation unit 4 in FIGS. 1, 3, and 4.

The character recognition unit 2 comprises a character recognition calculation unit 11, a character code output unit 12, and a reliability output unit 13.

The character recognition calculation unit 11 performs a character recognition process on character information displayed in the video by using a predetermined character recognition method for the input video, and obtains a character code of the recognition result. A numerical value (reliability) representing the reliability of the information and the character code information as the recognition result is obtained. As a character recognition method, for example, software on a computer that realizes the method introduced in Reference [4] can be used. In addition, the reliability of the character code of the recognition result is, for example, a value in the range of 0 to 1, and the higher the numerical value is, the higher the reliability is. About baseball, "A: 0.78"
75 "opening: 0.89""curtain:0.70""high: 0.6
5 "School: 0.91""Field:0.87""Sphere:0.95"
And a numerical value corresponding to each character code. In this case, it can be said that "sphere" is the character recognition result with the highest reliability. The reliability is obtained by a character recognition method described in reference [4] or the like. According to the method described in reference [4], a character area extracted from a video is used as an input pattern, and a character area in dictionary data for character recognition prepared in advance is used as a dictionary pattern, and the similarity between the input pattern and the dictionary pattern is determined. The character is recognized by calculating. The reliability is a value obtained when calculating the similarity between the input pattern and the dictionary pattern, and takes a value proportional to the similarity.

The character code output unit 12 outputs character code information of the recognition result obtained by the character recognition calculation unit 11.
The reliability output unit 13 outputs a numerical value of the reliability of each character code obtained by the character recognition calculation unit 11.

The recognition result storage unit 3 stores the character code storage unit 2
1 and a reliability storage unit 22. The character code storage unit 21 stores the character code of the character recognition result. The reliability storage unit 22 stores the reliability of the character code as a result of the character recognition.

The recognition result similarity determination unit 4 includes a character code similarity calculation unit 31 and a similarity reliability calculation unit 32.

The character code similarity calculation unit 31 takes out two character recognition results from the plurality of character recognition results stored in the character code storage unit 21 and compares the character recognition results with each other for all two combinations. Calculate the similarity of character codes. This processing unit is the same as the recognition result similarity calculation unit 4.-Character recognition result A: Tomorrow opening high school baseball-Character recognition result B: High school baseball, when there is a stop due to rain, both A and B are included As the same character code, there are four character codes “high school baseball”. On the other hand, when the similarity is defined as the number of the same character codes, the similarity of the character recognition results of A and B is calculated as 4. The similarity reliability calculation unit 32 stores the character code storage unit 21
, Two character recognition results are extracted from the plurality of character recognition results stored in the storage unit, and for each combination, the reliability of the similarity between the two is calculated from the reliability of the character code. A processing example of this processing unit will be described using the above example. The above example is
It is assumed that the reliability value of each character code of the character recognition result A and the character recognition result B is obtained, for example, as a numerical value in parentheses to the right of each character code below.

Character recognition result A: a (0.78) (0.
75) Opening (0.89) Curtain (0.70) High (0.65) School (0.91) Field (0.87) Ball (0.9
5) Character recognition result B: High (0.88) school (0.78) field (0.
67) Ball (0.96), (0.55) Rain (0.77), (0.81) Medium (0.72) Stop (0.92) Character code common to the reliability of similarity between the two , The sum of the reliability values of the character code strings corresponding to “high school baseball” in each character recognition result is 3.38 for the character recognition result A, and The result B is 3.29, the two are added together, and the average value of the reliability per character is calculated to be 0.83. The similarity reliability of both is calculated as 0.83.

FIG. 6 shows steps added to the flowchart of FIG. 2 when the recognition result similarity calculator 4 is composed of a character code similarity calculator 31 and a similarity reliability calculator 32. It is.

In this example, a case is shown in which the similarity reliability of the character recognition result is calculated as a value based on the sum of the reliability of common character codes. In step 61, the similarity reliability E
Is initialized to 0. In step 62, the sum of the reliability value T1 of the character code I and the reliability value T2 of the character code J is added to the similarity reliability E.

FIG. 7 is a block diagram showing the search result output unit 8 of FIG.

The video storage / display unit 82 retrieves a video corresponding to the character recognition result determined to be similar from the similarity information stored in the similarity information storage unit 7 from the video storage unit 9, and stores the video on the computer. It is arranged in the buffer 81.
Using the above example, an image displaying the characters "Tomorrow Opening High School Baseball" and an image displaying the characters "High School Baseball, Canceled by Rain" are taken out from the hard disk of the computer and Temporarily placed in memory, etc.

The video storage / display unit 82 further organizes and displays the similar video combinations stored in the video buffer 81 on the display screen of the computer. FIG. 8 shows an example of the display screen of the computer when this processing is performed. As shown in FIG. 8, for each similar video, for example, the video content is broadcast by arranging from the top to the bottom of the screen in the same group in which the number of character recognition results is large, together with similar character code information and number information. In the case of video, a video user can efficiently grasp highly topical information in a broadcast video. FIG. 8 shows an example in which the title “Top 5 Best Videos” is displayed at the top of the screen, but this has the effect of clearly explaining the display contents to the user. FIG. 8 shows an example in which character information for each similar group is displayed in the upper part of the screen in the order of the number of cases, and video belonging to each similar group is displayed together with broadcast station, date, and time information at the lower part of the screen.

The similar image display unit 82 performs not only the image similarity classification result but also the image display using only the value of the similarity calculation result before the classification. For example, one image is designated, and an image whose similarity value to the designated image is larger than a predetermined value is displayed. When an image displayed as “High school baseball, tomorrow opening” is selected, an image having similarity, such as an image displaying characters “Stopped by rain, High school baseball”, is displayed.

FIG. 9 is a block diagram in the case where the video search device of the first embodiment shown in FIG. 1 is implemented on a computer such as a personal computer. The input device 91 is for inputting video data. The storage device 92 stores the input video data and character information recognition result. The storage device 93 is a hard disk. The output device 94 is a disc display for outputting a video frame. Recording medium 95
Is a floppy disk, a CD-ROM, a magneto-optical disk, or the like, on which a video search program consisting of processes from the video input storage unit 1 to the result output unit 5 (excluding the recognition result storage unit 3) is recorded. The data processing device 96 includes a CPU and various interfaces, reads a video search program from the recording medium 95, and executes the program.

The video search apparatuses of the second and third embodiments can be similarly implemented on a computer.

[0048]

As described above, according to the present invention,
By classifying character codes such as telop characters in the video by character recognition processing and character information as a result of recognition by character recognition processing, and by grouping videos with similar semantic contents, the color of the video It is possible to determine the similarity of the meaning and content, which is impossible in the related art using characteristics of signal information such as shape, motion, sound, and the like. This allows
For example, in the case of a broadcast video or the like, it is possible to know information with high topicality in the broadcast video by referring to the grouped video information. In addition, a computer equipped with this function is used as a server, and a video user is provided with a function of referring to video information of each similar group on the server from a client terminal through a computer network, and a user who uses this function is provided with the function. It is also possible to construct a billing system using a computer network for collecting usage fees.

[Brief description of the drawings]

FIG. 1 is a block diagram of a similar image search and display device according to a first embodiment of the present invention.

FIG. 2 is a flowchart showing processing of a recognition result similarity calculation unit 4 in FIG.

FIG. 3 is a block diagram of a similar image search and display device according to a second embodiment of the present invention.

FIG. 4 is a block diagram of a similar image search and display device according to a third embodiment of the present invention.

FIG. 5 is a block diagram of a character information recognition unit 2, a recognition result storage unit 3, and a recognition result similarity calculation unit 4.

FIG. 6 is a flowchart showing processing of a recognition result similarity determination unit 5;

FIG. 7 is a block diagram of a video output unit 8;

FIG. 8 is a diagram showing a display example of a similar image display unit 82.

FIG. 9 is a block diagram when the video search device of FIG. 1 is implemented on a computer.

[Explanation of symbols]

 Reference Signs List 1 video input storage unit 2 character information recognition unit 3 recognition result storage unit 4 recognition result similarity calculation unit 5 result output unit 6 similarity classification unit 7 similarity information storage unit 8 search result output unit 9 video storage unit 11 character recognition calculation Unit 12 character code output unit 13 reliability output unit 21 character code storage unit 22 reliability storage unit 31 character code similarity calculation unit 32 similarity reliability calculation unit 41 to 52, 61, 62 step 81 video buffer 82 video storage Display unit 91 Input device 92, 93 Storage device 94 Output device 95 Recording medium 96 Data processing device

──────────────────────────────────────────────────の Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) G06T 1/00 200 G06T 1/00 200E H04N 5/76 H04N 5/76 B (72) Inventor Haruhiko Kojima Tokyo 2-3-1, Otemachi, Chiyoda-ku, Tokyo Nippon Telegraph and Telephone Corporation F-term (reference) 5B050 AA08 AA09 BA10 BA15 EA04 GA08 5B064 AA10 CA08 5B075 ND08 ND12 NK10 NR12 PQ02 PQ29 PQ36 PQ62 PQ74 PR06 QM08 50405

Claims (15)

[Claims] A video input storage unit for receiving video data and storing the video data in a memory; and extracting video data input from the video input storage unit and extracting subtitles displayed in the video frame for each video frame. A character information recognizing unit that recognizes a character code string candidate of the subtitle; a recognition result storage unit that stores the character code string candidate; and a character string candidate in a combination of all two video frames of the video frame. A video search device comprising: a recognition result similarity calculating unit that calculates a similarity based on a degree at which constituent characters match; and a result output unit that outputs the character code string candidate and the similarity. 2. A video input storage unit for inputting video data and storing the video data in a memory, and extracting subtitles displayed in the video frame for each video frame from the video data input from the video input storage unit. A character information recognition unit that recognizes a character code string candidate of the subtitle; a recognition result storage unit in which the character code string is stored; and a character string candidate in a combination of all two video frames of the video frame. A recognition result similarity calculation unit that calculates a similarity based on the degree to which constituent characters match, and classifying the similarity information based on a predetermined rule;
A similarity classifying unit that outputs similarity information that is identification information of the classified groups; a similarity information storage unit that stores the similarity information output by the similarity classifying unit; Search based on the request given in advance,
A video search device having a search result output unit for outputting. 3. A video input storage unit for inputting video data and storing the video data in a memory; and extracting video data input from the video input storage unit and extracting subtitles displayed in the video frame for each video frame. A character information recognizing unit that recognizes a character code string candidate of the subtitle; a recognition result storage unit that stores the character code string candidate; and a character string candidate in a combination of all two video frames of the video frame. A recognition result similarity calculation unit that calculates a similarity based on the degree to which constituent characters match, and classifying the similarity information based on a predetermined rule;
A similarity classifier that outputs similarity information that is identification information of each of the classified groups; a similarity information storage unit that stores the similarity information output by the similarity classifier; and input video data To the video storage unit that stores information that allows video to be reproduced from an arbitrary point specified using a predetermined method, and stores the video, and is stored in the similarity information storage unit. A video search device having a search result output unit that searches for similarity information of the video in response to a request given in advance, requests video information corresponding to the search result to the video storage unit, and outputs the video information together with the search result. 4. The character information recognizing unit performs a character recognition process on character information displayed in a video on an input video frame using a predetermined character recognition method, and obtains a recognition result. Character code information, a character recognition calculation unit that calculates a reliability that is a numerical value representing the reliability as a recognition result of the character code information, and character code information of the recognition result obtained by the character recognition calculation unit. A character code output unit, and a reliability output unit that outputs the reliability of each character code obtained by the character recognition calculation unit, wherein the recognition result storage unit stores a character code of a character recognition result. A character code storage unit, and a reliability storage unit that stores the reliability of the character code of the character recognition result. The recognition result similarity calculation unit includes a plurality of characters stored in the character code storage unit. From recognition results A character code similarity calculation unit that extracts character recognition results one by one and calculates the similarity of character codes for each combination; and two character recognition units from a plurality of character recognition results stored in the character code storage unit. 4. A similarity calculating unit for extracting a result, and for each combination, calculating a reliability of the similarity between the character codes from the reliability of the character codes stored in the reliability storing unit. An apparatus according to any one of the preceding claims. 5. The search result output unit outputs, from the image storage unit, an image corresponding to a character recognition result determined to be similar based on an image buffer and similarity information stored in the similarity information storage unit. 5. The apparatus according to claim 3, further comprising: a video storage / display unit that takes out the video, stores the video in the video buffer, and organizes and displays a combination of similar videos stored in the video buffer on a display screen. . 6. A video input storage step of inputting video data and storing the video data in a memory, and extracting subtitles displayed in the video frame for each video frame from the video data input from the video input storage step. A character information recognition step of recognizing a character code string candidate of the caption; a recognition result storing step of storing the character code string candidate in a memory; and a character string candidate in a combination of all two video frames of the video frame. And a result output unit that outputs the character code string candidate and the similarity. 7. A video input storage step of inputting video data and storing the video data in a memory, and extracting subtitles displayed in the video frame for each video frame from the video data input from the video input storage step. A character information recognition step of recognizing a character code string candidate of the caption; a recognition result storing step of storing the character code string candidate in a memory; and a character string candidate in a combination of all two video frames of the video frame. A recognition result similarity calculating step of calculating a similarity based on the degree of matching of the characters constituting, and classifying the similarity information based on a predetermined rule,
A similarity classifying step of outputting similarity information which is identification information of each of the classified groups; a similarity information storing step of storing the similarity information output in the similarity classifying step in a memory; and the similarity information. Is searched based on a request given in advance,
A video search method having a search result output step of outputting. 8. A video input storage step of inputting video data and storing the video data in a memory, and extracting subtitles displayed in the video frame for each video frame from the video data input from the video input storage step. A character information recognition step of recognizing a character code string candidate of the caption; a recognition result storing step of storing the character code string candidate in a memory; and a character string candidate in a combination of all two video frames of the video frame. A recognition result similarity calculating step of calculating a similarity based on the degree of matching of the characters constituting, and classifying the similarity information based on a predetermined rule,
A similarity classifying step of outputting similarity information that is identification information of each of the classified groups; a similarity information storing step of storing the similarity information output in the similarity classifying step in a memory; Attach information enabling reproduction of a video from an arbitrary point specified using a predetermined method to the data, and store the video in the video storage step of storing the video and the similarity information storage step Searching for similarity information of the obtained video by a previously given request, requesting video information corresponding to the search result to the video storage unit step,
A video search method including a search result output step of outputting the search result together with the search result. 9. The character information recognizing step performs a character recognition process on character information displayed in a video on an input video frame using a predetermined character recognition method, Character code information and a character recognition calculating step of obtaining a reliability which is a numerical value representing reliability as a recognition result of the character code information; and a character for outputting character code information of the recognition result obtained in the character recognition calculating step A code output step, and a reliability output step of outputting the reliability of each character code obtained in the character recognition calculation step, wherein the recognition result storing step stores the character code of the character recognition result in a memory. Character code storing step, and a reliability storing step of storing the reliability of the character code of the character recognition result in a memory, wherein the recognition result similarity Calculating a character code similarity calculating step of extracting two character recognition results from a plurality of character recognition results stored in the character code storing step and calculating similarity of character codes for each combination; Two character recognition results are extracted from the plurality of character recognition results stored in the character code storage step,
9. A similarity reliability calculating step of calculating a similarity reliability of the two from the character code reliability stored in the reliability storage unit step for each combination. The method of claim 1. 10. The search result output step retrieves, from the video storage step, a video corresponding to a character recognition result determined to be similar from the similarity information stored in the memory in the similarity information storage step, from the video storage step. 8. The method according to claim 7, further comprising the steps of: storing in a display buffer; and organizing and displaying on a display screen for each combination of similar images stored in the buffer. 11. A video input storage procedure for inputting video data and storing the video data in a memory, and extracting subtitles displayed in the video frame for each video frame from the video data input in the video input storage procedure. A character information recognition procedure for recognizing a character code string candidate of the caption; a recognition result storing procedure for storing the character code string candidate in a memory; and a character string candidate in a combination of all two video frames of the video frame. And a video search program for causing a computer to execute a result output step of outputting the character code string candidate and the similarity based on a degree of similarity calculated based on the degree of matching of the characters constituting The recording medium on which it was recorded. 12. A video input storage procedure for inputting video data and storing the video data in a memory, and extracting subtitles displayed in the video frame for each video frame from the video data input in the video input storage procedure. A character information recognition procedure for recognizing a character code string candidate of the caption; a recognition result storing procedure for storing the character code string candidate in a memory; and a character string candidate in a combination of all two video frames of the video frame. A recognition result similarity calculation procedure for calculating a similarity based on the degree of matching of the characters constituting the, and classifying the similarity information based on a predetermined rule,
A similarity classifying step of outputting similarity information that is identification information of each of the classified groups; a similarity information storing procedure of storing the similarity information output in the similarity classifying procedure in a memory; and the similarity information. Is searched based on a request given in advance,
A recording medium storing a video search program for causing a computer to execute a search result output procedure to be output. 13. A video input storage procedure for inputting video data and storing the video data in a memory, and extracting subtitles displayed in the video frame for each video frame from video data input from the video input storage procedure. A character information recognition procedure for recognizing a character code string candidate of the caption; a recognition result storing procedure for storing the character code string candidate in a memory; and a character string candidate in a combination of all two video frames of the video frame. A recognition result similarity calculation procedure for calculating a similarity based on the degree of matching of the characters constituting the, and classifying the similarity information based on a predetermined rule,
A similarity classification procedure for outputting similarity information that is identification information of each of the classified groups; a similarity information storage procedure for storing the similarity information output in the similarity classification procedure in a memory; The data is provided with information enabling reproduction of the video from an arbitrary point specified using a predetermined method, and stored in the video storage procedure for storing the video and the similarity information storage procedure. Searching for similarity information of the searched video according to a previously given request, requesting the video storage procedure for video information corresponding to the search result, and causing the computer to execute a search result output procedure for outputting the search result together with the search result. Recording medium on which the program of the above is recorded. 14. The character information recognizing step performs a character recognition process on character information displayed in a video by using a predetermined character recognition method on an input video frame, and Character code information and a character recognition calculation procedure for obtaining a reliability which is a numerical value representing the reliability as a recognition result possessed by the character code information, and a character for outputting the character code information of the recognition result obtained in the character recognition calculation procedure A code output procedure, and a reliability output procedure for outputting the reliability of each character code obtained in the character recognition calculation procedure, wherein the recognition result storing procedure stores the character code of the character recognition result in a memory. Character code storing procedure, and a reliability storing procedure of storing the reliability of the character code of the character recognition result in a memory, wherein the recognition result similarity calculating procedure includes the character code storing procedure. A character code similarity calculation procedure for extracting character recognition results two by two from the stored plurality of character recognition results and calculating similarity of character codes for each combination, and a plurality of character code similarity calculation procedures stored in the character code storage procedure. Similarity reliability calculation for extracting two character recognition results from the character recognition result and calculating the reliability of the similarity between the character combinations for each combination from the reliability of the character code stored in the reliability storage procedure. 14. The method according to any one of claims 11 to 13, comprising:
Recording medium according to the item. 15. The search result output step retrieves, from the video accumulation procedure, a video corresponding to a character recognition result determined to be similar from the similarity information stored in the memory in the similarity information storage procedure. Having a procedure of storing in the display buffer and a procedure of organizing and displaying on the display screen for each combination of similar images stored in the procedure.
The recording medium according to claim 12.