JP2011049707A - Moving image playback device, moving image playback method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a moving image playback device, a moving image playback method, and a program which can retrieve a moving image file based on a speech content of a speaker in the moving image, or retrieve a scene in the moving image file from among a large amount of the moving image files. <P>SOLUTION: The moving image playback device 1 is provided with a voice recognition means 2 which performs voice recognition of the moving image file; a text file creation means 3 which creates a text file from output of the voice recognition means 2; a keyword detection means 5 which detects a keyword pre-stored in a memory means 4 from the text file; a moving image order decision means 6 which decides a priority order of the playback moving image referring keyword weight attached to the keyword; a moving image playback means 7, which carries out playback of the moving image selected by user's selection referring the moving image order; and a weight updating means 8 which updates a keyword weight, based on the moving image selected by the user and a mode occurred by the keyword. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a moving image reproducing apparatus, a moving image reproducing method, and a program, and more particularly, to a moving image reproducing device, a moving image reproducing method, and a program for recognizing a moving image file and reproducing a moving image in a priority order desired by a user.

  In recent years, a large number of moving image files have been uploaded to moving image sharing websites such as YouTube (registered trademark). Metadata is added to the uploaded video file by a video producer (video poster) or a user (video viewer). Metadata refers to additional data such as title information, category information, and explanatory text.

  When searching for a target moving image file, the user searches the reproduction data that he / she desires to reproduce by category search, keyword search, or the like using the metadata. However, since the metadata is only limited information, it is difficult to find the target moving image file by searching using the metadata. Further, it has been more difficult to find a specific scene (scene or section) in the moving image file.

  Patent Document 1 describes a moving image search system that performs a keyword search for a large number of moving images registered on a moving image site such as the Internet. In this moving image search system, a moving image file in a moving image file database is recognized by speech to generate text data (text), and this text is registered as a new search word to prevent the search keyword from becoming obsolete. .

  Japanese Patent Application Laid-Open No. 2004-228561 describes an operation playback device that extracts or displays a keyword included in a selected moving image file and causes a user to select the keyword to immediately play back a desired scene.

  Patent Document 3 describes a technique in which moving image content is divided into scenes by image recognition and a representative still image for each scene is stored along a story. In this technique, a representative still image of each scene is reproduced, and a user views it to find a desired scene.

JP 2008-134799 A JP 2008-148077 A JP 2002-335473 A Japanese Patent Laid-Open No. 11-25271

  The technique described in Patent Document 1 merely prevents the search keyword from becoming obsolete. In addition, the techniques described in Patent Documents 1 and 2 merely present keywords and representative still images to the user, and the user searches for a desired scene. Based on the utterance contents of the speakers in the video, the video file Cannot be searched. Therefore, it is difficult to search for a target moving image file from a large number of moving image files and to find one scene in the moving image file while narrowing down the conditions.

  Patent Document 4 describes a technique for searching for similar images by image recognition. However, even if one scene of a video file is searched using this technology, it is merely a search using image recognition, and a search based on the utterance content of a speaker in the video as in Patent Documents 1 to 3. I can't.

  The present invention provides a video playback device, a video playback method, and a program capable of searching a video file from a large number of video files based on the utterance content of a speaker in the video and searching for a scene in the video file. With the goal.

In order to achieve the above object, the present invention provides a voice recognition means for voice recognition of a moving image file,
Text file generation means for generating a text file from the output of the voice recognition means;
Keyword detecting means for detecting a keyword stored in advance in the storage device from the text file;
A video ranking determining means for determining the priority of the video to be played with reference to the keyword weight attached to the keyword,
Movie playback means for playing back the movie selected by the user's selection referring to the movie ranking;
There is provided a moving image reproducing apparatus including weight updating means for updating the keyword weight based on a moving image selected by a user and an aspect in which the keyword is generated.

The present invention also includes a speech recognition step for recognizing a moving image file,
A text file generation step for generating a text file from the output of the speech recognition step;
A keyword detection step of detecting a keyword stored in advance in the storage device from the text file;
A video ranking determination step that determines the priority of the videos to be played by referring to the keyword weights attached to the keywords;
A video playback step of playing back the video selected by the user's selection referring to the video ranking;
There is provided a moving image reproduction method including a weight update step of updating the keyword weight based on a moving image selected by a user and an aspect in which the keyword is generated.

Furthermore, the present invention is a program for a moving image reproducing apparatus that includes a computer and reproduces a moving image, and the computer includes:
Voice recognition processing that recognizes video files,
A text file generation process for generating a text file from the output of the speech recognition process;
A keyword detection process for detecting a keyword stored in advance in the storage device from the text file;
A video ranking determination process that determines the priority of videos to be played with reference to the keyword weights assigned to the keywords,
A video playback process for playing back the video selected by the user's selection referring to the video ranking;
There is provided a program for executing a weight update process for updating the keyword weight based on a moving image selected by a user and an aspect in which the keyword is generated.

  In the moving image playback apparatus, the moving image playback method, and the program of the present invention, a desired moving image file can be searched from a large number of moving image files based on the utterance content of a speaker in the moving image, or one scene in the moving image file can be searched.

The block diagram which shows the minimum structure of the moving image reproduction apparatus of this invention. 1 is an overall view showing a moving image playback apparatus according to an embodiment of the present invention. The block diagram which shows the structure of the moving image reproducing apparatus shown in FIG. The sequence diagram which shows operation | movement of the moving image reproducing apparatus shown in FIG. The flowchart which shows the procedure which determines a moving image order and a scene order. The figure which illustrates keyword weight. (A) And (b) is a figure which illustrates keyword information. The figure which illustrates the search result list containing animation ranking and scene ranking. The figure which shows the search result list screen on a web browser. The figure which illustrates the keyword weight updated. The figure which illustrates the keyword weight of another keyword.

  FIG. 1 is a block diagram showing the minimum configuration of the moving picture reproducing apparatus of the present invention. The moving picture reproducing apparatus 1 of the present invention has, as its minimum configuration, a voice recognition means 2, a text file generating means 3, a storage device 4, a keyword detecting means 5, a moving picture rank determining means 6, and a moving picture reproducing means 7. And weight updating means 8. The voice recognition means 2 recognizes a moving image file by voice. The text file generation unit 3 generates a text file from the output of the voice recognition unit 2. The keyword detection means 5 detects a keyword stored in advance in the storage device 4 from the text file. The moving picture order determining means 6 determines the priority order of moving pictures to be reproduced with reference to the keyword weights attached to the keywords. The moving image reproducing means 7 reproduces the moving image selected by the user's selection referring to the moving image ranking. The weight updating unit 8 updates the keyword weight based on the moving image selected by the user and the mode in which the keyword is generated.

  In the moving image reproduction device 1, keywords included in the text file are detected from a text file generated by voice recognition of the moving image file, and keyword weights stored in advance in the storage device 4 are acquired. Referring to the acquired keyword weight, the priority order of the moving image to be reproduced is determined. The moving image selected by the user with reference to the determined moving image order is reproduced. Next, the keyword weight is updated based on the moving image selected by the user and the mode in which the keyword is generated in the moving image. The updated keyword weight is used when determining the priority order of the moving image to be reproduced next time.

  In other words, if the user inputs a keyword related to the target video file, the priority order of the video to be played that includes this keyword is once determined, and the user selects the video by referring to this video ranking. The keyword weight that is one of the parameters for determining the ranking is updated. Therefore, from the next time, the video ranking is determined using keyword weights that are dynamically changed by feeding back the user's selection, so the more repeatedly used, the target video is searched from a large number of video files. Increase the accuracy. In addition, if a plurality of keywords reflecting the utterance content of the speaker in the video are input as keywords related to the target video file, one scene (section, scene) in the target video file can be searched.

  In addition, the moving image playback method and program of the present invention have a configuration corresponding to the minimum configuration of the moving image playback device 1, and similarly to the above, based on the utterance contents of the speakers in the video from a large number of video files. You can search for a movie file or a scene in a movie file.

  Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to FIGS. FIG. 2 is an overall view showing the moving image playback apparatus according to the embodiment of the present invention. For example, the video playback device 10 displays a video that the user 12 of the user terminal 11 aims or meets the purpose (hereinafter, target video) and one scene (hereinafter, section or scene) in the target video. It is a device for searching and playing.

  The video playback device 10 includes a Web server 20, a video search server 30, and a file server 40, and these servers 20, 30, and 40 are connected via a local network (LAN) 50. In addition, a plurality of user terminals 11 and 13 are connected to the moving image playback apparatus 10. The user terminals 11 and 13 are connected to the Web server 20 via the Internet 60 and can access various Web sites. In the following, the website is a video sharing website such as YouTube (registered trademark), the user 12 of the user terminal 11 is a user who searches for a target video, and the user 14 of the user terminal 13 is a video sharing Assume that a video poster submits a video to a website.

  FIG. 3 is a block diagram illustrating a configuration of the moving image playback apparatus 10. The user terminal 13 of the user 14 who is a video poster has a video posting unit 15. The user terminal 11 of the user 12 who is a user has a keyword setting unit 16 and a moving image selection unit 17. Each of these means 15, 16, and 17 can access the Web server 20. The video posting unit 15 is a unit for accessing a video sharing site on the Web and posting a video file in accordance with the operation of the user 14. The keyword setting means 16 is a means for inputting a keyword that seems to be related to the target moving image in accordance with the operation of the user 12. The moving image selection means 17 is means for selecting a moving image that matches (or does not match) the purpose from the moving image file search result list displayed on the website screen in accordance with the operation of the user 12.

  The Web server 20 includes a moving image posting screen display unit 21, a keyword input screen display unit 22, and a moving image search result display unit 23. The moving image search server 30 includes a voice recognition unit 31, a moving image file searching unit 32, a keyword information temporary search result database 33, a moving image file temporary search result output database 34, and a keyword weight change unit 35. Further, the file server 40 includes a moving image file database 41, a language model database 42, a recognized text database 43, and a keyword information database 44.

  The moving image posting screen display means 21 stores the moving image file posted by the moving image posting means 15 and the metadata added to the moving image file in the moving image file database 41. The metadata is additional data such as title information, category information, and explanatory text regarding the moving image file. The moving image file is uploaded because it is stored in the moving image file database 41.

  The voice recognition means 31 acquires the uploaded moving picture file from the moving picture file database 41 and performs voice recognition with reference to the language model (dictionary data) in the language model database 42. The voice recognition unit 31 converts the text into a text by voice recognition and generates a recognized text (text file). At this time, the speech recognition means 31 not only performs text conversion, but also associates each word (description) included in the recognized text with the utterance start time from the beginning when the moving image file of each word is reproduced. Record.

  If the metadata added to the moving image file includes a word that is not registered in the language model of the language model database 42, the speech recognition unit 31 additionally registers this word in the language model. By registering additional words, the registered words included in the language model automatically increase. Examples of the word include a word included in a title or an explanatory text added to the video file by the video poster, a keyword designated by the user, and the like. The voice recognition unit 31 stores the recognized text in the recognized text database 43. The recognized text stored in the recognized text database 43 is indexed. An index refers to an index indicating which data is stored in the database in order to improve the data search speed.

  Next, a case where the user 12 who is a user accesses the video sharing site using the user terminal 11 will be described. When the user 12 accesses the video sharing site using the user terminal 11, the keyword input screen display unit 22 in the Web server 20 displays the keyword input screen on the user terminal 11. Next, when a keyword related to a scene of a target video (video to be searched) is input using the keyword setting unit 16 on the keyword input screen, the keyword input screen display unit 22 displays the input keyword as a video. Delivered to the file search means 32.

  The moving image file search means 32 searches the keyword information database 44 for the input keyword. The keyword information database 44 stores keyword weighting information (keyword weights) attached to the keywords, and the moving image file search means 32 acquires the keyword weights of the input keywords from the keyword information database 44.

  In addition, the moving image file search unit 32 searches the recognized text database 43 for the input keyword, and acquires a moving image file including the same keyword in the recognized text from the moving image file database 41. The moving image file search means 32 calculates, for example, the moving image ranking and the scene ranking to be reproduced from the acquired recognition text and keyword weight of the moving image file, and the moving image file search result including these calculation results is stored in the moving image file temporary search result database. 34. At the same time, the moving image file search means 32 acquires the elapsed time (seconds) from the beginning when the input keyword is uttered in the moving image file, for example, from the moving image file search result, and the keyword information temporary search result as keyword information Store in the database 33. Here, since the speech recognition means 31 records each word included in the recognized text and the utterance start time from the beginning when the moving image file of each word is played back, the elapsed time from the beginning is recorded. Can be acquired. The moving image file search means 32 associates the description in the recognized text with the elapsed time from the beginning when the moving image file is played back, and the elapsed time for dividing the recognized text into a plurality of sections (scenes) according to the moving time of the moving image file. It also functions as a calculation means. Note that the keyword information includes, for example, a scene ID, a keyword name, a moving image file name including the keyword, an utterance start time, a keyword weight, and the like.

  The moving image search result display means 23 acquires a moving image file search result from the moving image file temporary search result database 34 and keyword information from the keyword information temporary search result database 33. Then, the moving image search result display means 23 displays the information acquired from these databases 33 and 34 as a moving image file search result list on the Web site screen displayed on the user terminal 11.

  The moving image selection means 17 of the user terminal 11 selects a moving image that matches (or does not match) the purpose from the moving image file search result list displayed on the website screen according to the operation of the user 12. The moving picture search result display means 23 sends the selection result by the moving picture selection means 17 to the keyword weighting change means 35.

  The keyword weighting changing unit 35 acquires keyword information of keywords included in the moving image file that matches (or does not match) the purpose selected by the moving image selecting unit 17 from the keyword information temporary search result database 33. The keyword weight changing unit 35 updates the keyword weight of the acquired keyword based on the selection result by the moving image selecting unit 17, and stores the updated keyword weight in the keyword information database 44. The updated keyword weight stored in the keyword information database 44 is used when the user 12 searches again using the user terminal 11. That is, when a keyword related to one scene of the target moving image is input again from the keyword setting means 16, the updated keyword weight is set when the moving image file searching means 32 determines the moving image rank and the scene order to be reproduced. Will refer to it.

  Hereinafter, with reference to FIGS. 3 to 10, the operation of the moving image playback apparatus 10 will be specifically described. FIG. 4 is a sequence diagram showing the operation of the moving image playback apparatus 10. The moving image posting means 15 accesses a moving image sharing site on the Web and posts a moving image file to the Web server 20 in accordance with the operation of the user 14 who is a moving image poster (step S11). The moving image posting screen display means 21 of the Web server 20 stores the posted moving image file (moving image data) and the metadata added to the moving image file in the moving image file database 41 of the file server 40 (step S12).

  Next, when the moving image file is stored in the moving image file database 41, the voice recognition unit 31 of the moving image search server 30 automatically acquires the moving image file from the moving image file database 41 (step S13). Subsequently, the voice recognition means 31 acquires a language model from the language model database 42 (step S14), performs voice recognition, converts the voice in the moving image file into text, and creates a recognized text (step S15). In step S15, when converting to text, the number of seconds from the beginning of each word is described in association with each word in the recognized text.

  Subsequently, the speech recognition means 31 additionally registers words that are not registered in the language model in the language model database 42 (step S16). Next, the speech recognition means 31 stores the recognized text in the recognized text database 43 of the file server 40 (step S17). The stored recognized text is automatically indexed in the recognized text database 43 on the file server 40 (step S18).

  A case will be described in which the user 12 as a user searches for a target moving image after the processes in steps S11 to S18 are performed. First, when an access to the video sharing site occurs from the user terminal 11 according to the operation of the user 12 (step S19), the keyword input screen display means 22 of the Web server 20 displays a search keyword input screen on the user terminal 11 ( Step S20). When a keyword that seems to be related to the target video is input from the keyword setting unit 16 on the search keyword input screen (step S21), the keyword input screen display unit 22 converts the input keyword into a video file of the video search server 30. It transmits to the search means 32 (step S22).

  Next, the moving image file search means 32 accesses, for example, the moving image file database 41 of the file server 40, and searches for the moving image file in which the keyword input in step S21 is included in the metadata added to the moving image file (step S21). S23). Next, the moving image file search unit 32 accesses the recognized text database 43 and searches for a moving image file in which the input keyword is included in the recognized text (step S24).

  Subsequently, the moving image file search means 32 acquires a moving image file name list (search result moving image list) including the input keyword from the recognized text database 43 from the search results of steps S23 and S24 (step S25). Next, the moving image file search means 32 acquires the metadata (title, category information, etc.) of the moving image file included in the search result moving image list acquired in step S25 from the moving image file database 41 (step S26). Next, the moving image file search means 32 calculates the display order of the moving image files included in the search result moving image list (step S30). The display order in step S30 includes the priority order (rank) of moving images to be reproduced and the priority order (rank) of scenes included in the moving images. Hereinafter, with reference to FIG. 5, the process of step S30 by the moving image file search means 32 will be described.

  First, the moving image file search means 32 acquires the keyword name and the number of keywords included in the recognized text for each moving image file included in the search result moving image list from the recognized text database 43 (step S31). Next, the keyword weight of the keyword included in the recognized text is acquired from the keyword information database 44 (step S32).

  Here, keywords and keyword weights will be described. Keywords are classified into positive keywords and negative keywords. This distinction depends on whether or not the user 12 has selected a moving image file included in the moving image file search result moving image list 70 shown in FIG. . Note that the selection of the moving image file may be made by checking the “target” area 71 shown in FIG. 9 and not checking if the “non-target” area 72 is checked. For example, even if the video file has the highest video ranking, if the user 12 does not select a video file as a video that does not meet the purpose (a video that does not include the target scene), the keyword included in the video file is negative. Defined as a keyword. On the other hand, even if the moving image rank is low, if the user selects a moving image file as a moving image suitable for the purpose (moving image including the target scene), the keyword included in the moving image file is defined as a plus keyword.

  The keyword weight of the plus keyword is indicated by a ratio between the number of occurrences of the plus keyword in all moving image files selected by the user 12 and the number of all moving image files selected by the user 12. That is, the keyword weight of the plus keyword refers to the average number of occurrences of the plus keyword included in one moving image selected by the user 12.

  The keyword weight of the minus keyword is indicated by a ratio between the number of occurrences of the minus keyword in all moving image files not selected by the user 12 and the number of all moving image files not selected by the user 12. That is, the keyword weight of the minus keyword refers to the average number of occurrences of the minus keyword included in one moving image that is not selected by the user 12.

  FIG. 6 shows specific examples of plus keywords, minus keywords, and keyword weights stored in the keyword information database 44. Here, when the user 12 wants to hear an opinion about the American economy at the meeting of the Prime Minister A, the keyword setting means 16 is used as search keywords “A”, “Prime Minister”, “America”, “Economy”. Assume the case of input. When a moving image including these input keywords is displayed in the moving image file search result moving image list 70 shown in FIG. 9, the user 12 selects a moving image that suits or does not fit the purpose. As an example, a video such as a conference is selected by the user 12 as a video suitable for the purpose. For this reason, the keywords included in the moving image such as the conference are all set as positive keywords 44a as shown in FIG. On the other hand, a moving image such as news is not a meeting of Prime Minister A, but is mainly a remark of a caster, for example, and is therefore selected by the user 12 as a moving image that does not meet the purpose. For this reason, all the keywords included in the moving images such as news are set to the minus keywords 44b.

  The keyword weights of the plus keywords are “0.30”, “0.42”, “3.17”, “2.50” for “A”, “Prime Minister”, “America”, and “Economy”, respectively. It has become. The reason why the keyword weights of “A” and “Prime Minister” are smaller than the keyword weights of “America” and “Economy” is because Prime Minister A talks in videos such as conferences. This is because the average number of occurrences of positive keywords such as “A” and “Prime Minister” is small.

  On the other hand, the keyword weights of the negative keywords are “2.50”, “3.26”, “0.60”, “0.0” for “A”, “Prime Minister”, “America”, and “Economy”, respectively. 24 ". Note that the keyword weights of “A” and “Prime Minister” are larger than the keyword weights of “America” and “Economy”. This is because, in a video such as news, the caster speaks many negative keywords such as “A” and “Prime Minister”, so the average number of occurrences of these negative keywords is large.

  Here, attention is focused on the keyword “A”. As an example in which the keyword weight of the plus keyword “A” is “0.30”, the number of moving images selected by the user 12 is 10, and the number of occurrences of “A” is 3 times in the 10 moving images. And the like. Further, as an example in which the keyword weight of the minus keyword “A” is “2.50”, the number of moving images not selected by the user 12 is 12, and the number of occurrences of “A” in these 12 moving images. Is 30 times. The number of moving images and the number of occurrences used when calculating these keyword weights are held in the keyword information database 44, for example.

  Returning again to FIG. The moving image file search means 32 calculates the difference (keyword weight value difference) between the keyword weight of the plus keyword and the keyword weight of the minus keyword from the keyword weight acquired in step S32 (step S33). From the keyword weights shown in FIG. 6, the difference between the weight values of the keyword “A” is “−2.20”, the difference between the weight values of the keyword “Prime” is “−2.84”, and the weight value of the keyword “America”. Is “2.57”, and the difference between the weight values of the keyword “economy” is “2.26”.

  Next, the moving image file search means 32 calculates a time interval between keywords in the moving image file (step S34). As described above, the time interval between the keywords is determined by associating each keyword in the recognized text with the elapsed time from the beginning at the time of playback of the moving image file as the elapsed time calculating means. Can be calculated by dividing it into a plurality of sections (scenes) according to the playback time of the moving image file. That is, as shown in FIG. 7A, the moving image file detecting unit 32 generates keyword information including a scene ID 33a, a keyword name 33b, a moving image file name 33c including the keyword, and an utterance start time 33d. The keyword information is stored in the keyword information temporary search result database 33. The keyword information includes keyword weights acquired from the keyword information database 44 by the moving image file search means 32. In the following, attention is focused on the moving image with the moving image file name “moving image 1”.

  When the keyword information of “Movie 1” is shown in time series as shown in FIG. 7B, the time interval from the keyword “Prime Minister” to the keyword “USA” is 45 from the start of movie playback to the end of movie playback. Second, the time interval from the keyword “America” to the keyword “Economy” is 2 seconds, and the time interval from the keyword “Economy” to the keyword “America” is 3 seconds.

  Subsequently, the moving image file search means 32, for example, for “moving image 1”, “the number of keywords included in the recognized text” acquired in step S31, “the difference in keyword weight values” acquired in step S33, and The moving image ranking is determined based on the three elements of “time interval between keywords” acquired in step S34 (step S35).

As a calculation formula used at Step S35, the following formula (1) is mentioned, for example.
{(Difference in weight value of keyword “A”) × (Number of keywords of keyword “A”) + (Difference in weight value of keyword “Prime”) × (Number of keywords of keyword “Prime”) + ...} + {( Number of time intervals between keywords within 30 seconds) / (Average number of seconds between keywords within 30 seconds)}
Formula (1)

  Here, referring to FIG. 7B, the number of keywords “A” in “Movie 1” is 0, the number of keywords “Prime” is 1, and the number of keywords “America” is 2. The number of keywords “Economy” is one. Similarly, referring to FIG. 7B, the number of keywords whose time interval between keywords is within 30 seconds is 3, and the average number of seconds between these three keywords is (2 + 3) /2=2.5 seconds. .

Therefore, when these numerical values are substituted into the equation (1),
{(-2.20 × 0) + (− 2.84 × 1) + (2.57 × 2) + (2.26 × 1)} + (3 / 2.5) = 5.76
Thus, the value “5.76” for determining the moving image ranking is obtained. By performing such calculation for other moving image files, the moving image ranking of the moving image files can be determined. That is, the moving image file search means 32 sets the moving image rank higher as the difference between the keyword weight values is larger, the keyword is generated more frequently, and the keyword is generated more times per unit time.

  Next, the moving image file search means 32 determines the rank for each section (scene) divided by each keyword in the moving image file based on the “time interval between keywords” and “the keyword weight of a plus keyword”. (Step S36). Here, the case where the order of the four scenes ID “001” to “004” in “Movie 1” is determined will be described with reference to FIG.

As a calculation formula used at Step S36, the following formula (2) is mentioned, for example.
{(Keyword weight of plus keyword) / (Total number of seconds between adjacent keywords)}
Formula (2)

  However, when there is one adjacent keyword, a value obtained by doubling the number of seconds between the adjacent keywords is set as the total value. Therefore, referring to FIG. 7B, the total value of the number of seconds between adjacent keywords is as follows: scene ID: 001 is “45 × 2 = 90 seconds”, scene ID: 002 is “45 + 2 = 47 seconds”, scene ID: 003 is “2 + 3 = 5 seconds” and scene ID: 004 is “3 × 2 = 6 seconds”.

  Therefore, by substituting these total values and the keyword weights of the plus keywords shown in FIG. 6 into Expression (2), the scene ID: 001 is “0.42 / 90 = 0.007” and the scene ID: 002 is “3.17 / 47 = 0.67”, scene ID: 003 “2.50 / 5 = 0.50”, and scene ID: 004 “3.17 / 6 = 0.53” are obtained. The larger the value obtained, the higher the scene ranking. Therefore, the scene order in “Movie 1” is Scene ID: 004> Scene ID: 003> Scene ID: 002> Scene ID: 001. That is, the moving image file search unit 32 determines that the scene is more important as the keyword weight of the plus keyword of the scene is larger and the keyword generation interval is shorter, and ranks the scene higher.

  Subsequently, the moving image file search means 32 stores the moving image file ranking determined in step S35 and the scene order in the moving image file determined in step S36, together with information such as the moving image file name, in a moving image file temporary search result database. 34 (step S37). In step S37, as shown in FIG. 8, in the moving image file temporary search result database 34, a moving image rank 34a, a scene rank 34b, a moving image file name 34c, a scene ID 34d, a title, a category, a moving image size, etc. Is stored.

  Returning again to FIG. The moving image file search means 32 stores the keyword information stored in the keyword information temporary search result database 33 with the contents shown in FIG. 7A and the search result list stored in the moving image file temporary search result database 34 shown in FIG. It transmits to the moving image search result display means 23 of the Web server 20 (step S40).

  Next, the video search result display means 23 displays a search result list screen (video file search result list) 70 shown in FIG. 9 on the Web screen based on the keyword information acquired in step S40 and the search result list. (Step S41). In the moving image file search result list 70, as shown in the figure, moving image ranking, scene ranking, moving image title, category, moving image file name, and search word (input keyword) are displayed. Further, in the moving image file search result list 70, as shown in the figure, a “target” area 71 and a “non-target” for the user 12 to select a moving image that matches the purpose or a moving image that does not meet the purpose. ”Area 72, moving image playback screen 73, video time axis 74, and scene position 75 in the moving image are displayed. If the scene position 75 is clicked, the scene can be cued in the moving image reproduction.

  Subsequently, when the “target” area 71 or the “non-target” area 72 is checked in accordance with the operation of the user 12 from the moving image file search result list 70 displayed in step S 30, the moving picture selection means 17 of the user terminal 11. Selects a moving image that matches or does not match the purpose (step S42), and transmits the selection result to the moving image search result display means 23. The moving image search result display unit 23 transmits the selection result to the keyword weight change unit 35 of the moving image search server 30 (step S43).

  Based on the selection result, the keyword weighting changing unit 35 applies the keyword (that is, the plus keyword) included in the moving image file selected as the moving image suitable for the purpose and the moving image file selected as the moving image not suited for the purpose. The keywords included (ie, minus keywords) and their keyword weights are acquired from the keyword information temporary search result database 33 (step S44).

  Next, the keyword weight changing means 35 changes (updates) the keyword weight of the acquired plus keyword and the keyword weight of the minus keyword (step S45). Hereinafter, with reference to FIG. 10, the process of updating the keyword weight in step S45 will be described. FIG. 10 shows the keyword weight of the plus keyword and the keyword weight of the minus keyword after the update. Here, attention is focused on the keyword “A” as an example.

The keyword weighting changing means 35 accesses the keyword information temporary search result database 33. The keyword weight change means 35 is used to calculate the keyword weight “0.3” of the plus keyword “A” before update held in the keyword information database 44 in the file server 40 and this value. The number of moving images “10” selected by the user 12 and the number of occurrences “3” of “A” in the ten moving images are acquired. As an example, it is assumed that five moving images are newly selected as moving images that match the purpose of the selection result in step S42, and the number of occurrences of “A” in the five moving images is one. In this case, the keyword weight changing means 35 sets the new keyword weight of the plus keyword “A”.
{(3 + 1) / (10 + 5)} ≈0.27.

  That is, as shown in FIG. 10, the keyword weight changing means 35 changes the keyword weight 44c of the plus keyword “A” from “0.3” to “0.27” shown in FIG. 6 according to the selection result of step S42. Update to

Further, the keyword weighting changing means 35 accesses the keyword information temporary search result database 33, and stores the keyword weight “2.50” of the negative keyword “A” before update held in the keyword information database 44 and this value. The number of moving images “12” selected as being unsuitable for the user 12 and the number of occurrences “30” of “A” in these 12 moving images used for calculation are acquired. . As an example, it is assumed that three moving images are newly selected as moving images that do not match the purpose of the selection result in step S42, and the number of occurrences of “A” in these three moving images is eight. In this case, the keyword weight changing unit 35 sets a new keyword weight for the minus keyword “A”.
{(30 + 8) / (12 + 3)} ≈2.53.

  That is, as shown in FIG. 10, the keyword weight changing means 35 changes the keyword weight 44d of the minus keyword “A” from “2.5” to “2.53” shown in FIG. 6 according to the selection result in step S42. Update to By applying the above calculation to other keywords, it is possible to calculate the keyword weight of the updated plus keyword and the keyword weight of the minus keyword, which are illustrated in FIG.

  Next, the keyword weight changing means 35 gives the updated keyword weight to the keyword already stored in the keyword information database 44 (step S46). As shown in FIG. 10, keyword weights are stored in the keyword information database 44 for each set of keywords searched simultaneously (for example, one set including “A”, “Prime Minister”, “America”, and “Economy”). In step S46, keywords that are not stored in the keyword information database 44 are newly registered in the keyword information database 44 together with the calculated keyword weights.

  The user 12 inputs a keyword using the keyword setting unit 16 of the user terminal 11 until the target video is searched, and further selects the target video from the video file search result list 70 using the video selection unit 17. To do. The keyword weight change unit 35 updates the keyword weight according to the selection of the moving image by the operation of the user 12. Then, the moving image file search means 32 refers to the updated keyword weight when determining the moving image ranking and the scene ranking. In other words, in the video playback device 10, when the user 12 searches again with the same search keyword, the updated keyword weights are applied, so that the ranking of the video and scene that suits the purpose is displayed higher, You can search for video files that fit your needs.

  In the present embodiment, there is a keyword weight for each keyword, and this keyword weight is dynamically updated to reflect the user's operation. The displayed accuracy is increased. In addition, by focusing on the time interval between keywords, the order of scenes in a moving image file can be determined. Furthermore, since the keyword is associated with the reproduction position (elapsed time) from the beginning when the moving image file is reproduced, the target scene where the keyword is uttered in the moving image file can be found immediately. In addition, it is possible to perform a highly accurate search by combining the search from the utterance content by the voice recognition of the moving image file and the search by the metadata of the moving image file. Therefore, in this embodiment, it is possible to efficiently search a scene in which a speaker in a moving image actually speaks a keyword from a large amount of moving image files on the Web or the like. Further, by storing (learning) the input keywords, it is possible to narrow down search efficiently.

  In the above-described embodiment, the case where the user 12 wants to hear an opinion about the US economy at a meeting of Prime Minister A as a moving image including a target scene is illustrated, but the present invention is not limited to this. As an example, as shown in FIG. 11, when the user 12 wants to hear the impression of the goal in an interview with a soccer player B, he / she entered “soccer” “B” “interview” “goal” as a search keyword. To do.

  When a moving image including these search keywords is displayed in the moving image file search result moving image list 70 shown in FIG. 9, the user 12 selects a moving image that suits or does not match the purpose. For example, the video of the interview video is selected by the user 12 as a video suitable for the purpose. For this reason, all the keywords included in the video of the interview video are the positive keywords 44e. The keyword weights of the plus keywords are “0.24”, “0.81”, “1.05”, and “2.76” for “soccer”, “B”, “interview”, and “goal”, respectively. It has become. In the interview video, the keyword weight is small because the positive keywords such as “soccer” and “B” have a smaller average number of occurrences than other positive keywords.

  On the other hand, since the video of the news video and the football play video is not the interview of the player B but mainly the remark of the caster, for example, the video is selected by the user 12 as an unsuitable video. For this reason, all the keywords included in the video of the news video and the soccer play video are set to the minus keyword 44f. The keyword weights of the negative keywords are “2.20”, “2.89”, “1.55”, and “1.10” for “soccer”, “B”, “interview”, and “goal”, respectively. It has become. In news videos and soccer play videos, negative keywords such as “soccer” and “B” have a larger average number of occurrences than other negative keywords, and thus the keyword weight is large. The positive keyword 44e and the negative keyword 44f are stored in the keyword information database 44. Even in such a case, according to the configuration of the above-described embodiment, a scene in which a speaker in a moving image actually speaks a keyword can be searched with high accuracy.

  Moreover, although the said embodiment demonstrated the example which searches the moving image containing the target scene based on the audio | voice data of the moving image file uploaded on Web, it is not limited to this. As an example, a target scene can be searched from a moving image file stored in a hard disk recorder or the like.

  As described above, the present invention has been described based on the preferred embodiment. However, the moving image reproducing apparatus, the moving image reproducing method, and the program of the present invention are not limited to the configuration of the above embodiment, and the configuration of the above embodiment. To which various modifications and changes are made within the scope of the present invention.

DESCRIPTION OF SYMBOLS 1,10: Moving image reproducing device 2, 31: Voice recognition means 3: Text file generating means 4: Storage device 5: Keyword detecting means 6: Moving image ranking determining means 7: Moving image reproducing means 8: Weight updating means 11, 13: User Terminal 12: User (user)
14: User (video contributor)
15: Movie posting unit 16: Keyword setting unit 17: Movie selection unit 20: Web server 21: Movie posting screen display unit 22: Keyword input screen display unit 23: Movie search result display unit 30: Movie search server 32: Movie file search Means 33: Keyword information temporary search result database 34: Movie file temporary search result database 35: Keyword weight change means 40: File server 41: Movie file database 42: Language model database 43: Recognition text database 44: Keyword information database 50: LAN
60: Internet 70: Search result list screen

Claims (9)

A voice recognition means for voice recognition of a video file;
Text file generation means for generating a text file from the output of the voice recognition means;
Keyword detecting means for detecting a keyword stored in advance in the storage device from the text file;
A video ranking determining means for determining the priority of the video to be played with reference to the keyword weight attached to the keyword,
Movie playback means for playing back the movie selected by the user's selection referring to the movie ranking;
A moving image playback apparatus comprising: a moving image selected by a user; and weight updating means for updating the keyword weight based on a mode in which the keyword is generated.   The moving image reproduction apparatus according to claim 1, further comprising an elapsed time calculation unit that associates the description in the text file with the elapsed time from the beginning when the moving image file is reproduced.   The elapsed time calculating means divides the text file into a plurality of sections according to the playback time of the moving picture file, the moving picture ranking determining means determines the priority order of the sections to be played back, and the moving picture playing means The moving image playback device according to claim 2, wherein a section to be played back is selected from the sections.   4. The moving image reproducing apparatus according to claim 2, wherein the keyword generation mode includes at least one of the number of occurrences of the keyword per unit time, the number of occurrences of the keyword in the moving image file, and the keyword occurrence interval. The weight updating means defines a keyword included in the video selected by the user as a positive keyword, and the ratio of the number of selected videos and the number of videos to be selected increases as the number of occurrences of the positive keyword increases. The smaller the, the higher the keyword weight of the plus keyword,
A keyword included in a video that is not selected by the user is defined as a negative keyword. The more the negative keyword is generated, the smaller the ratio between the number of videos that are not selected and the number of videos to be selected. The moving image reproducing device according to claim 1, wherein the moving image reproducing device is modified so that the keyword weight of the minus keyword increases.   The moving image ranking determining means reproduces the larger the difference between the keyword weight of the plus keyword and the keyword weight of the minus keyword, the greater the number of occurrences of the keyword, and the greater the number of occurrences of the keyword per unit time. The moving image reproducing apparatus according to claim 5, wherein the priority order of moving images to be given is higher.   6. The moving image ranking determining means sets the priority of the section to be played back higher as the keyword weight of the plus keyword in the section to be played back of the moving picture is larger and the keyword occurrence interval is shorter. 6. The moving image playback device according to 6. A voice recognition step for recognizing a video file,
A text file generation step for generating a text file from the output of the speech recognition step;
A keyword detection step of detecting a keyword stored in advance in the storage device from the text file;
A video ranking determination step that determines the priority of the videos to be played by referring to the keyword weights attached to the keywords;
A video playback step of playing back the video selected by the user's selection referring to the video ranking;
A video reproduction method comprising: a weight update step of updating the keyword weight based on a video selected by a user and an aspect in which the keyword is generated. A program for a video playback device that includes a computer and plays back a video, the computer comprising:
Voice recognition processing that recognizes video files,
A text file generation process for generating a text file from the output of the speech recognition process;
A keyword detection process for detecting a keyword stored in advance in the storage device from the text file;
A video ranking determination process that determines the priority of videos to be played with reference to the keyword weights assigned to the keywords,
A video playback process for playing back the video selected by the user's selection referring to the video ranking;
A program for executing weight update processing for updating the keyword weight based on a moving image selected by a user and an aspect in which the keyword is generated.

Images