JP2007293602A - System and method for retrieving image and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To objectively retrieve classification of images and to shorten a time for selecting a plurality of images according to a common keyword. <P>SOLUTION: An image distribution server registration device 14 transmits correspondence data information as a result of an image text relating means 12 associating image data and image associated data with each other to an image distribution server 30. The image distribution server 30 is provided with an image data storage area 31, an image associated data storage area 32, a correspondence data storage area 33, a mining means 35 performing weighting during retrieval, and a classification dictionary 34 and the like for performing weighting. A terminal 40 transmits information such as a keyword to the image distribution server 30. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a technique for performing an ambiguous video search in conjunction with text.

  Conventionally, when searching through a network of video data, information related to the video is converted into text, indexed, and included in the video / video file, thereby distributing the video / video that matches the input keyword as a result. It was.

  In this case, there arises a problem that a result cannot be obtained unless it matches the input keyword, and that a video can be reproduced only from the beginning.

Here, when the search target is image data, the image data and information related to the image data are generated by object extraction, feature extraction, word estimation, phrase estimation, and synonym estimation, and the image data is searched by keyword. A technique for improving a search system (accuracy for searching an image intended by a user) has been proposed (for example, see Patent Document 1).
JP 2004-362314 A

  However, the conventional example described above has the following problems.

  In the conventional method, it is difficult to realize the search accuracy (search accuracy based on objective classification, not subjective classification) based on correspondence with video-related data in video (moving image) data search. there were.

  In addition, there is a problem that the result that the user wants to acquire cannot be obtained even when the outline data related to the video data is correlated and the outline data is searched by a keyword or the like.

  The present invention has been made in view of the circumstances described above, and an object of the present invention is to objectively search for a video classification and reduce the time for selecting a plurality of videos using a general keyword. It is in.

  In order to solve the above-described problems, the present invention provides a video search system for specifying a video from text data acquired from audio accompanying the video.

The present invention also provides a video text corresponding means for associating video data and video accompanying data;
A video distribution server registration device for registering corresponding data information as a result of the video text corresponding means in a video distribution server;
Video data storage area, video associated data storage area, corresponding data storage area, mining means for weighting at the time of search, classification dictionary for weighting, search result generating means for generating results by mining means as search results, and video distribution apparatus A video distribution server comprising:
Provided is a video search system in which terminal devices that send information such as keywords to the video distribution server are connected via a network.

  The present invention also provides a video search method for identifying a video from text data acquired from audio accompanying the video.

The present invention also provides a video text corresponding means for associating video data and video accompanying data;
A video distribution server registration device for registering corresponding data information as a result of the video text corresponding means in a video distribution server;
Video data storage area, video associated data storage area, corresponding data storage area, mining means for weighting at the time of search, classification dictionary for weighting, search result generating means for generating results by mining means as search results, and video distribution apparatus A video distribution server comprising:
A video search method using a terminal device that sends information such as keywords to the video distribution server is provided.

  ADVANTAGE OF THE INVENTION According to this invention, the classification | category of an image | video can be searched objectively, and the time which selects a some image | video with a general keyword can be shortened.

  Hereinafter, a first embodiment of the present invention will be described in detail with reference to the drawings.

  Referring to FIG. 1, the video search system according to the present embodiment includes video data 10 and video accompanying data 11 (all audio in video data is changed to text data, or video data is converted to text data). , A title, etc.) are associated with the specified time in the video data by the video text corresponding means 12 so that the video is associated with the video accompanying data and the corresponding data 13 is generated.

  The video distribution server registration device 14 is connected to the video distribution server 30 via the network for the video data 10, the video associated data 11, and the corresponding data 13. The video distribution server 30 includes a video data storage area 31 and a video associated data storage area 32. , A corresponding data storage area 33, a classification dictionary 34, a mining means 35, a search result generating means 36 for converting a search result into a reproduction video position, and a video distribution device 37.

  Furthermore, the terminal 40 is connected to the video distribution server 30 by an input device 41 and a display / playback device 42.

  Next, the video fuzzy search operation in conjunction with the text in this embodiment will be described.

  The video data 10 is created by text data such as a video title and outline, or a text editor or voice recognition that converts all voices in the video to text data, and is used as video accompanying data 11.

  The video data 10 and the video associated data 11 are input and passed to the video text corresponding means 12, and the video text corresponding means 12 takes the correspondence between the separation time of the video data 10 and the video associated data 11, and the correspondence between the reproduction time and the text. Data 13 is created.

  This may be realized, for example, by a technique as disclosed in JP 2004-152063 A. That is, the time code generation means outputs a word time code, which is information on the start time and end time of each word in the voice information, to the voice recognition means. The voice recognition means adds a word time code to the voice recognition text information and outputs it to the mapping means together with the video information. Therefore, in this embodiment, the time code generation means is a time code indicating the start time and end time of each divided portion when the speech recognition text information is divided into a plurality of predetermined divided portions (for example, a plurality of paragraphs). A word time code is generated as information for generating.

  The text input means receives text media from the operator. When the text media is a digital medium, the text input means 7 outputs the text information recorded on the text media to the mapping means. When the text medium is an analog medium, the text input means is configured to include, for example, an OCR (optical character reading) device. Then, digital conversion is performed on the text information recorded on the text medium, and the digitized text information is output to the mapping means.

  The mapping means divides the text information into divided parts at appropriate intervals. Here, for example, paragraphs that are chunks of sentences are detected by detecting line breaks, indents, and the like, and the text information is delimited using the line breaks and the like as delimiter positions. Further, the mapping means compares the text information with the speech recognition text information, delimits the speech recognition text information based on the delimiter position in the text information, and generates paragraph speech recognition text information. Furthermore, correspondence information that is information indicating a one-to-one correspondence between each paragraph (delimited text information) and each paragraph speech recognition text information in the text information is generated. Then, the correspondence information is output to the structuring means together with the paragraph voice recognition text information and the text information.

  The structuring means calculates each time code that is the start time and end time of each paragraph from the word time code added to each word in the paragraph speech recognition text information. Further, the structuring unit generates structured information that is information indicating a one-to-one correspondence between each paragraph of the text information and each time code based on the time code and the correspondence information.

  Further, the structuring unit 11 generates a text media file for storing text information and a time code file for storing time code based on the structured information, and a video media file for storing video information.

  For example, text information in each paragraph is stored in the text media file in the order of paragraphs, and a time code corresponding to each paragraph is stored in the time code file in the same order as the paragraph order in the text information. A plurality of text media files each corresponding to each paragraph and a time code file may be generated.

  Next, the structuring unit outputs the video media file, the text media file, and the time code file to the data storage unit. The video media storage means in the data storage means stores the video media file, the text media storage means in the data storage means stores the text media file, and the time code storage means in the data storage means stores the time code file. . In this example, the storage means for storing the text information and the video information corresponds to the video media storage means and the text media storage means, and the time code storage means for storing the time code of each divided portion is the time code storage means. Equivalent to.

  The video information may be supplied from the video input unit to the data storage unit via the voice recognition unit, the mapping unit, and the structuring unit, but is directly supplied from the video input unit to the data storage unit. You may do it.

  Further, the structuring means may generate, as management information, the start address and end address of each paragraph of the text information in the text media file, and the start address and end address of the time code of each paragraph in the time code file. . At this time, the multimedia content apparatus includes a management file storage unit that stores management information in the data storage unit. The structuring unit outputs the management information to the management file storage unit, and the management file storage unit stores the management information.

  Further, the structuring unit may generate a structured text media including the time code obtained by combining the text information and the time code.

  Further, there is a method in which the structuring means generates an XML file with a structural description in the MPEG7 (moving picture expert group 7) format in the XML (Extensible Markup Language) language. In the case of generating an XML file, the multimedia content apparatus does not include a text media storage unit and a time code storage unit in the data storage unit, but includes an XML file storage unit. Then, the structuring unit outputs the XML file to the XML file storage unit, and the XML file storage unit stores the XML file. In this example, the storage means for storing the video information corresponds to the video media storage means, and the XML file storage means for storing the XML language describing the time code, text information, and information indicating the storage position of the video information is stored. This corresponds to the XML file storage means.

  When a user requests desired video information and text information, the user inputs a word / phrase as a keyword to the synchronous data utilization means. Then, the input / output means outputs the phrase input by the user to the search control means. The search control unit searches the text media storage unit or the XML file storage unit for a paragraph of text information including the word and outputs the corresponding text information paragraph to the input / output unit. Further, when the user selects a paragraph of certain text information, the input / output means requests the search control means to output video information synchronized with the text information selected by the user.

  The search control means extracts the video information from the video media storage means, extracts the time code of the paragraph of the text information selected by the user from the time code storage means, the text media storage means, or the XML file storage means, and sends it to the synchronization means. Output. The synchronization means outputs the video information to the input / output means with the start time indicated by the time code as the start time of the video information output and the end time indicated by the time code as the final time of the video information output. The synchronizing means processes the text information based on the time code and outputs it to the input / output means. As processing at this time, for example, there is a method of scrolling text information.

  The synchronous data generating means and the synchronous data using means can be realized by a computer system. However, the input / output means corresponds to input / output means such as a keyboard and a display unit of a user-side microcomputer. When the synchronous data generating means and the synchronous data using means (excluding the input / output means) are realized by a computer system, voice recognition means, time code generating means, mapping means, structuring means, search control means, and synchronizing means Is realized by software. The data storage means is realized by a storage medium such as a magnetic disk in the computer system.

  Specifically, the software implemented in the computer system compares the speech recognition text information and the text information with the processing for generating speech recognition text information by performing speech recognition processing based on the speech information corresponding to the text information. The processing for dividing the speech recognition text information into divided parts based on the position where the text information is separated, the process for generating the time code indicating the start time and the end time of each divided part, and the generated time code A process for associating and structuring text information and video information for each predetermined divided portion, a process for storing text information, video information, and time code, and a search condition input by the user The process for identifying the divided part in the text information that matches and the time code corresponding to the identified divided part are extracted. It includes a process of, identifies the image information corresponding to the extracted time code, the identified image information, a program for executing the process of providing the user with the text data matching the search keyword.

  Now, the video distribution server registration device 14 sends the video data 10, the video associated data 11, and the corresponding data 13 to the video distribution server 30 via the network, respectively, a video data storage area 31, a video associated data storage area 32, and a corresponding data storage area 33. Register with.

  The mining means 35 uses the data sent from the input device 41 of the terminal 40 as an input, searches the video accompanying data 11 with the classification dictionary 34 as an aid, scores the appearance frequency and the relationship between the data, and searches the search result. The result is passed to the result generation means 36.

  Based on the result of the mining means 35, the search result generating means 36 determines the video and the playback start position from the corresponding data, and sends it to the display / playback apparatus 42 of the terminal 40.

  In the terminal 40, video playback is started by selecting a result displayed on the display playback device 42.

  Next, a specific example of this embodiment will be described with reference to FIGS.

  First, regarding the self-introduction video data 10 for about 5 minutes, all the statements are converted into text, and video accompanying data 11 is created. The video data 10 and the video accompanying data 11 are passed to the video text corresponding unit 12, and the video text corresponding unit 12 generates corresponding data 13 as shown in FIG.

  Next, the video distribution server registration device 14 includes the video data storage area 31, the video associated data storage area 32, and the corresponding data storage area 33 of the video distribution server 30 in which the video data 10, the video associated data 11, and the corresponding data 13 are designated. Register each.

  Next, the keyword “advantage” is input by the input device 41 of the terminal 40.

  The input data is sent to the video distribution server 30 and passed to the mining means 35.

  The mining means 35 receives the keyword “merits” and obtains the keywords “personality” and “disadvantages” from the classification dictionary 34.

  The mining means 35 performs a full-text search on the video-associated data 11 in the video-associated data area 32 by using the received “advantages” and keywords of “personality” and “disadvantages” from the classification dictionary 34, and a keyword appears for each paragraph. Weighting is performed according to the number of times, and a score is obtained for each paragraph.

  The mining means 35 passes this result to the search result generating means 36.

  The search result generating means 36 generates the result shown in FIG. 3 by collating with the corresponding data 13 in the corresponding data storage area 33 in the descending order of the score for each paragraph, and the result is displayed on the display reproduction device 41 of the terminal 40. return.

  When the user selects the result displayed on the display / playback device 41 of the terminal 40, the user selects the result from the specific playback position of the video data 10 in the video data storage area 32. Start playback and distribute the video.

  According to the present embodiment, the video classification can be objectively searched by the mining means and the classification dictionary, and the time for selecting a plurality of videos can be shortened by using a general keyword.

  Further, by outputting the result of the mining engine as information on the relationship with other videos, it is possible to provide a new video to the user.

  Next, a second embodiment of the present invention will be described.

  In a state where a plurality of videos are stored in the video distribution server 30, the keyword is handed over to the mining means 35 not from the keyword input from the input device 41 of the terminal 40 but from the text word of a general Web page. You can also.

  For example, in the FAQ (Frequently Asked Questions and Answers) page, when it is desired to carry out explanation by video rather than explanation of only words, a word as a keyword is selected and passed to the mining means 35, so that related video data The list can be returned to the user through the same route as shown in the example, and answers can be obtained by video.

  Each of the above-described embodiments is a preferred embodiment of the present invention, and various modifications can be made without departing from the scope of the present invention. For example, the processing for realizing the function of each device may be performed by causing each device to read and execute a program for realizing the functions of the video distribution server registration device 14, the video distribution server 30, and the terminal 40. Further, the program is transmitted to another computer system by a transmission wave via a computer-readable recording medium such as a CD-ROM or a magneto-optical disk, or via a transmission medium such as the Internet or a telephone line. Also good.

  In each of the above-described embodiments, the configuration in which the video distribution server registration device 14, the video distribution server 30, and the terminal 40 are realized as one computer system has been described. However, the configuration and function in which each device is connected separately. Of course, the present invention can also be applied to a configuration in which a plurality of devices are added for each.

  INDUSTRIAL APPLICABILITY The present invention can be applied to an operation of finding a necessary video from a large number of videos without searching by a simple keyword in a business that owns a large number of video contents or a business that conducts video education. In addition, since a video such as education can be reproduced from a necessary place, a learning effect can be expected.

It is a block diagram which shows the structure of the video search system in the 1st Embodiment of this invention. It is a figure which shows an example of the corresponding data which the video text corresponding | compatible means in the 1st Embodiment of this invention produces | generates. It is a figure which shows an example of the data produced | generated by the search result production | generation means in the 1st Embodiment of this invention collating with corresponding data.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Video data 11 Video accompanying data 12 Video text corresponding means 13 Corresponding data 14 Video distribution server registration apparatus 30 Video distribution server 31 Video data storage area 32 Video accompanying data storage area 33 Corresponding data storage area 34 Classification dictionary 35 Mining means 36 Search result Generation means 37 Video distribution device 40 Terminal 41 Input device 42 Display reproduction device

Claims (9)

  A video search system characterized in that a video is specified from text data acquired from audio accompanying the video.   2. The video search system according to claim 1, wherein the video data is weighted at the time of search by a keyword.   3. The video search system according to claim 1, wherein a playback start point is specified in the search result. Video text correspondence means for associating video data and video accompanying data;
A video distribution server registration device for registering corresponding data information as a result of the video text corresponding means in a video distribution server;
Video data storage area, video associated data storage area, corresponding data storage area, mining means for weighting at the time of search, classification dictionary for weighting, search result generating means for generating results by mining means as search results, and video distribution apparatus A video distribution server comprising:
A video search system, wherein terminal devices that send information such as keywords to the video distribution server are connected to each other via a network.   A video search method characterized in that a video is specified from text data acquired from audio accompanying the video.   6. The video search method according to claim 5, wherein the video data is weighted when searching by a keyword.   7. The video search method according to claim 5, wherein a playback start point is specified in the search result. Video text correspondence means for associating video data and video accompanying data;
A video distribution server registration device for registering corresponding data information as a result of the video text corresponding means in a video distribution server;
Video data storage area, video associated data storage area, corresponding data storage area, mining means for weighting at the time of search, classification dictionary for weighting, search result generating means for generating results by mining means as search results, and video distribution apparatus A video distribution server comprising:
A video search method using a terminal device that sends information such as a keyword to the video distribution server.   A program for causing a computer to realize the function according to any one of claims 1 to 4.

Images