JP2008148121A - Motion picture summary automatic generation apparatus and method, and computer program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a motion picture summary automatic generation apparatus for automatically generating a motion picture summary. <P>SOLUTION: An outline generating device 10 comprises: a scene extraction unit 51 which divides motion picture data into scenes and extracts scene motion picture data for each of the divided scenes; an audio recognition unit 53 which recognizes audio data contained in the scene motion picture data and generates text data from the audio data; an important word extraction unit 53 which extracts an important word that becomes a keyword, from the text data; a representative image extraction unit 54 which extracts representative image data that become an image representing the scenes from the scene motion picture data; and a scene merging unit 55 which merges the important word and the representative image data for each scene and creates motion picture summary data as a plot. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a moving picture summary automatic creation apparatus, method, and computer program. More particularly, the present invention relates to an apparatus for automatically creating a summary of a moving image, a method for automatically creating a summary of a moving image, and a computer program thereof.

  Conventionally, a moving image stored on a medium such as a DVD is divided into scenes into one unit, one of which is a chapter, and for each chapter, the first image of the chapter or the scene specified by the media creator Some have created a digest that displays an image. A method that improves user convenience, such as enabling the cueing of a moving image of the corresponding chapter, is used by recording the created digest in addition to the moving image on a DVD containing the moving image. It has been.

  As an automatic video summarization method, an automatic video summarization method for outputting a video summary image by detecting a topic conversion word in a video with captions is disclosed (for example, Patent Document 1). According to this Patent Document 1, a topic conversion word that is a word for switching topics such as “next” and “by the way” is detected, thereby outputting a topic conversion signal and an image at the time when the topic conversion signal is output. Or, an image after a certain time is used as an image for summarization. In this method, a summary of a video is generated with fewer images by collecting images at every turn of video with caption information.

In another document, a broadcast radio wave is received, video information and audio information of a representative part included in the signal are extracted separately, and then the video information and audio information are synthesized and summarized. A creating method is disclosed (for example, Patent Document 2).
Japanese Patent Application Laid-Open No. 11-333161 JP 2002-149672 A

  However, Patent Document 1 relates to video with subtitles. For example, there are many non-captioned videos, such as video broadcast on television and the Internet, and Japanese movies. The method described in Patent Document 1 cannot be used for such a video without captions that is widely broadcast in Japan. In addition, an image at the time when a topic conversion word appears or an image after a certain period of time is used for summarization. However, Patent Document 1 is premised on a news program with subtitles and the like. Even if this method is used, a representative image cannot always be displayed.

  Further, Patent Document 2 extracts video information and audio information separately, but the audio information is obtained by extracting a part of the audio as it is. Therefore, when a plurality of voices, for example, noises are input, the voices containing the noises are extracted as they are, and it is difficult to hear them. As a result, the voices that are not important are included.

  In view of the above-mentioned problems, the inventor used speech recognition as a method for creating a summary even when there is no subtitles or a video in which a plurality of sounds are input. We considered using voice data as text and extracting important words. In addition, the present inventors have found out that a representative image is found from a moving image based on the same idea as extracting a key word from text data.

  Therefore, the present invention extracts important words from text data obtained by converting speech data into text using speech recognition, and extracts extracted important words and representatives extracted based on the same idea as extracting important words from text data. An object of the present invention is to provide an apparatus for automatically creating a moving picture summary that integrates images and automatically creates a moving picture summary.

  For the above purpose, the following are specifically provided.

(1) A moving picture summary automatic creation device that automatically creates a moving picture summary for each scene constituted by a group of the image data from moving picture data constituted by a plurality of image data,
A scene extracting unit that divides the moving image data into the scenes and extracts scene moving image data for each of the divided scenes;
A voice recognition unit that recognizes voice data included in the scene video data and generates text data from the voice data;
An important word extraction unit for extracting an important word as a keyword from the text data;
A representative image extraction unit for extracting representative image data to be an image representing the scene from the scene video data;
A moving picture summary automatic creation apparatus comprising: a scene integration unit that creates the moving picture summary data by integrating the important word and the representative image data for each scene.

  According to the configuration of (1), the moving image summary automatic creating apparatus includes the scene extraction unit. A scene is composed of a group of image data and indicates a part of moving image data. In addition, a speech recognition unit, an important word extraction unit, and a representative image extraction unit are provided. The important words generated from the audio data and the representative image data are extracted separately. The scene integration unit integrates the extracted important words and the representative image data for each scene to create moving image summary data.

  Therefore, by extracting a scene from moving image data, extracting an important word and a representative image for each scene, and integrating them, an outline that is a summary of a moving image can be automatically created.

  Using voice data as a method for extracting an important word has an advantage that an abundant amount of information included in the voice data can be utilized. On the other hand, there is a problem that the recognition rate of voice recognition is low at present. If the recognition rate of voice recognition is low, an accurate document cannot be generated, so it is difficult to use it as it is. However, weak points with low recognition rates can be absorbed by extracting key words as keywords from the generated text data.

(2) The moving image data further includes character data together with the image data,
The video summary automatic creation device according to (1), wherein the important word extraction unit extracts an important word from the character data and the text data generated by the voice recognition unit.

  According to the configuration of (2), when the moving image data further includes character data together with the image data, the important word extraction unit included in the moving image summary automatic creation device includes the character data and the text data generated by the voice recognition unit. And extract important words.

  Therefore, for example, when the audio data is not Japanese, such as a Western movie with subtitles, it is possible to extract important words for each scene by using character data that is subtitles. Can be used. Further, even in a video including Japanese audio data with subtitles, it is possible to more accurately extract important words by using subtitles and text data converted from the audio data.

  (3) The representative image extraction unit extracts an object from the image data constituting the scene moving image data, and displays the entire display area of the image data, the display area of the object, the number of image data including the object, and the The apparatus for automatically creating a moving picture summary according to (1) or (2), wherein the moving picture summary is derived from the number of image data constituting the scene moving picture data using a result calculated by a predetermined calculation formula.

  According to the configuration of (3), the representative image extraction unit provided in the video summary automatic creation device extracts an object from the image data constituting the scene video data, and displays the entire display area of the image data, the display area of the object, the object Can be derived from the number of image data including the image data and the number of image data constituting the scene moving image data, using a result calculated by a predetermined calculation formula.

  The entire display area of the image data, the display area of the object, the number of image data including the object, and the number of image data constituting the scene moving image data are easily obtainable information. Therefore, a representative image can be extracted simply by substituting it into a certain calculation formula using specific information regarding an image that can be easily acquired.

  (4) The moving picture summary automatic creation device according to (1) to (3), wherein the important word extraction unit extracts the important words from the text data using a TF * IDF method.

  According to the configuration of (4), the important word extraction unit provided in the video summary automatic creation device can extract the important words from the text data using the TF * IDF method. Therefore, words that appear biased at a specific place can be extracted as important words.

  (5) The moving image summary according to (1) to (4), wherein the scene integration unit integrates the moving image summary data generated for each scene moving image data so that the moving image data is displayed first. Automatic creation device.

  According to the configuration of (5), the scene integration unit included in the video summary automatic creation device can further integrate the video summary data created for each scene video data so that the video summary data is displayed first. . Therefore, by creating moving image summary data that is a summary of a moving image and displaying it at the beginning of the moving image data, the user can see the summary as a digest version first.

(6) A moving picture summary automatic creation method for automatically creating a moving picture summary for each scene constituted by a set of image data from moving picture data constituted by a plurality of image data,
Dividing the video data into the scenes, and extracting scene video data for each of the divided scenes;
Recognizing audio data included in the scene video data and generating text data from the audio data;
Extracting an important word as a keyword from the text data;
Extracting representative image data to be an image representing the scene from the scene video data;
A method of automatically creating a moving picture summary including the step of creating the moving picture summary data by integrating the important word and the representative image data for each scene.

  According to the structure of (6), the method which has the same effect | action and effect as said (1) can be provided.

(7) The moving image data further includes character data together with the image data,
The method for automatically creating a moving picture summary according to (6), wherein the step of extracting the important word extracts the important word from the character data and the text data generated by the step of generating the text data.

  According to the structure of (7), the method which has the same effect | action and effect as said (2) can be provided.

  (8) The step of extracting the representative image data includes extracting an object from the image data constituting the scene moving image data, and displaying the entire display area of the image data, the display area of the object, and the number of image data including the object And a method for automatically creating a moving picture summary according to (6) or (7), which is derived from the number of image data constituting the scene moving picture data using a result calculated by a predetermined calculation formula.

  According to the structure of (8), the method which has the same effect | action and effect as said (3) can be provided.

  (9) The moving picture summary automatic creating method according to (6) to (8), wherein the step of extracting the important word extracts the important word from the text data by using a TF * IDF method.

  According to the structure of (9), the method which has the same effect | action and effect as said (4) can be provided.

  (10) The step of creating the moving image summary data integrates the moving image summary data created for each scene moving image data so as to be further displayed at the beginning of the moving image data. Automatic video summary creation method described.

  According to the structure of (10), the method which has the effect | action and effect similar to said (5) can be provided.

(11) A computer program for automatically creating a summary of a moving image for each scene formed of a group of the image data from moving image data including a plurality of image data,
Dividing the video data into the scenes, and extracting scene video data for each of the divided scenes;
Recognizing audio data included in the scene video data and generating text data from the audio data;
Extracting an important word as a keyword from the text data;
Extracting representative image data to be an image representative of the scene from the scene video data;
A computer program for causing a computer to execute the step of integrating the important word and the representative image data for each scene to create moving image summary data.

  According to the configuration of (11), it is possible to provide a computer program having the same operation and effect as the above (1).

  According to the present invention, an important word is extracted from text data obtained by converting speech data into text using speech recognition, the extracted important word, and a representative extracted based on the same idea as extracting the important word from the text data. It is possible to provide a moving picture summary automatic creation device or the like that automatically creates a moving picture summary by integrating images.

  Hereinafter, the best mode for carrying out the present invention will be described with reference to the drawings. This is merely an example, and the technical scope of the present invention is not limited to this.

[System overall configuration]
FIG. 1 is a block diagram showing the overall configuration of a computer system 1 and the functional configuration of a synopsis creation device 10 according to an example of a preferred embodiment of the present invention.

  The synopsis creation device 10 of the present invention is a device that automatically creates a summary of a moving image and is connected to a user terminal 20 via a communication line 30. The user terminal 20 may be a mobile terminal such as a mobile phone or a PDA in addition to the PC. The communication line 30 indicates the Internet, for example.

  In this embodiment, the synopsis creation device 10 is realized by hardware such as a server. Here, the physical configuration of the server is not limited to this. The synopsis creation device 10 is not limited in the number of hardware, and may be configured by one or more as necessary. Further, the hardware of the synopsis creation device 10 may be configured to include a Web server, a DB server, and an application server as necessary, and may be configured with one server or different servers. Good.

  The synopsis creation apparatus 10 includes a control unit 50. The control unit 50 divides at least moving image data into scenes, and extracts a scene moving image data for each divided scene. A speech recognition unit 52 that recognizes included speech data and generates text data from the recognized speech data, a keyword extraction unit 53 that extracts a keyword as a keyword from the generated text data, and a scene from scene video data A representative image extraction unit 54 that extracts representative image data to be an image to be processed, and a scene integration unit 55 that creates moving image summary data in which the extracted key words and the extracted representative image data are integrated for each scene. Prepare.

  The synopsis creation device 10 further includes a storage unit 60. The storage unit 60 includes at least a moving image DB 62, a representative image extraction DB 64, and a synopsis DB 66.

  The moving image summary data (summary) created by the synopsis creation device 10 is downloaded to the user terminal 20 via the communication line 30. Thus, the user of the user terminal 20 connected to the communication line 30 can see the outline. Further, by storing the moving image summary data (summary) in a medium such as a DVD and distributing it to the user, the distributed medium can be reproduced on the user terminal 20 or the like.

[Conceptual diagram]
FIG. 2 shows a conceptual diagram until a synopsis is created by the synopsis creation apparatus 10 according to an example of a preferred embodiment of the present invention.

  The moving image data is composed of a plurality of image data. This image data is basically 24 frames (24 image data) per second for a movie and 30 frames (30 image data) per second for a television. The number of frames (number of image data) per second of a moving image is represented by FPS (Frame Per Second), and the larger the number (the larger the FPS value), the smoother the moving image, and the correspondingly the file size increases. Therefore, a movie can be expressed as 24 FPS, and a television can be expressed as 30 FPS.

  Here, one moving image data, which is a set of a plurality of image data, is divided for each certain scene. A number is assigned to each scene. The scene includes audio data and image data. The audio data is, for example, a speech or narration related to video, and text data is generated from the audio data. After that, important words are extracted from the text data. Also, representative image data is extracted from the image data.

  Text 1 is generated from speech 1 that is audio data of scene 1, and important word 1 is extracted. Further, the representative image 1 is extracted from the image data of the scene 1. Similarly, the same processing is performed for scene 2 and scene 3.

  Integrate the created key words and representative images to create a synopsis. In that case, the synopsis for each scene may be displayed in parallel, or the synopsis may be displayed so that the scene changes for each scene.

  As described above, it is possible to create a synopsis by extracting and integrating representative data of each scene from the audio data and the image data.

[Scene separation example]
FIG. 3 shows an example in which a scene is cut by the synopsis creation device 10 according to an example of a preferred embodiment of the present invention.

  The right side of FIG. 3 shows image data that is part of the moving image data. 3 (a) and 3 (b) show how a rabbit is jumping over the field, while FIGS. 3 (c) and 3 (d) show how a person has left the house. Yes. In this case, it is switched to a completely different image. Therefore, it is determined that the transition between the scenes (b) and (c) in FIG. 3 is a scene change, and as shown on the left in FIG. ”, (C), and (d) are“ scene 2 ”.

[Processing flow]
FIG. 4 shows the main flow of the synopsis creation process which is the main process according to an example of the preferred embodiment of the present invention.

  First, in step S <b> 1, the control unit 50 takes out moving image data stored in the moving image DB 62. Thereafter, the control unit 50 moves the process to step S2.

  In step S2, the control unit 50 (scene extraction unit 51) divides the moving image data extracted in step S1 into scenes. The scene division process will be described later with reference to FIG. Thereafter, the control unit 50 moves the process to step S3.

  In step S3, the control unit 50 assigns numbers to the scenes divided in step S2 in order. Thereafter, the control unit 50 moves the process to step S4.

  In step S4, the control unit 50 extracts one scene (scene i) from the divided scenes. Thereafter, the control unit 50 moves the process to step S5.

  In step S5, the control unit 50 (voice recognition unit 52) generates text data by voice recognition from the extracted voice data of the scene i. The voice recognition process will be described later with reference to FIG. Thereafter, the control unit 50 moves the process to step S6.

  In step S6, the control unit 50 (important word extracting unit 53) extracts an important word from the generated text data. The important word extraction process will be described later with reference to FIG. Thereafter, the control unit 50 moves the process to step S7.

  In step S7, the control unit 50 (representative image extraction unit 54) extracts representative image data, which is a scene representing the moving image data, from the plurality of image data constituting the moving image data of the scene i extracted in step S4. . The representative image extraction process will be described later with reference to FIG. Thereafter, the control unit 50 moves the process to step S8.

  In step S8, the control unit 50 determines whether or not processing has been performed for all scenes i. When the process is performed for all scenes i (when YES is determined in the process of step S8), the control unit 50 moves the process to step S9. On the other hand, when all the scenes i have not been processed yet (when NO is determined in the process of step S8), the control unit 50 moves the process to step S5, and for the scene i that has not been processed. Continue processing.

  In step S <b> 9, the control unit 50 (scene integration unit 55) connects the important word and the representative image data for the scene i to create integrated data. Thereafter, the control unit 50 moves the process to step S10.

  In step S10, the control unit 50 (scene integration unit 55) connects the generated integrated data in order, generates synopsis data, and stores the synopsis data in the synopsis DB 66. Thereafter, the control unit 50 ends this process.

  Next, the scene division process will be described with reference to FIG.

  First, in step S21, the control unit 50 (scene extraction unit 51) advances the moving image data of the scene i. Specifically, the image data constituting the moving image data is sent in order. Thereafter, the control unit 50 (scene extraction unit 51) moves the process to step S22.

  In step S22, the control unit 50 (scene extraction unit 51) determines whether or not a completely different image has been switched. Specifically, this corresponds to a case where there is no overlap between partial image data, which occupies a part of the image data, between the current image data and the previous image data. . When the image is switched to a completely different image (when YES is determined in the process of step S22), the control unit 50 (scene extraction unit 51) moves the process to step S23. On the other hand, when the image has not been switched to a completely different image (when NO is determined in the process of step S22), the control unit 50 (scene extraction unit 51) moves the process to step S26.

  In step S23, the control unit 50 (scene extraction unit 51) divides the moving image data before and after the place where the image is completely different. Thereby, it can be set as another scene in the place of a scene change. Thereafter, the control unit 50 (scene extraction unit 51) moves the process to step S24.

  In step S24, the control unit 50 (scene extraction unit 51) stores the divided first half of the moving image data in the WK of the storage unit 60. WK refers to a temporary area in the storage unit 60. Thereafter, the control unit 50 (scene extraction unit 51) moves the process to step S25.

  In step S25, the control unit 50 (scene extraction unit 51) advances the moving image data for the divided latter half moving image data. Thereafter, the control unit 50 (scene extraction unit 51) moves the process to step S22. Thereafter, the process is repeated until the moving image data ends.

  On the other hand, in step S26, the control unit 50 (scene extraction unit 51) determines whether or not the video of the moving image data has ended. When the video of the moving image data has ended (when YES is determined in the process of step S26), the control unit 50 (scene extraction unit 51) ends this process and returns to the main process. On the other hand, when the video of the moving image data has not ended (when NO is determined in the process of step S26), the control unit 50 (scene extraction unit 51) moves the process to step S22, and the moving image data ends. Repeat the process until

  Next, the speech recognition process will be described with reference to FIG.

  First, in step S51, the control unit 50 (voice recognition unit 52) extracts voice data from the moving image data of the scene i. Thereafter, the control unit 50 (voice recognition unit 52) moves the process to step S52.

  In step S52, the control unit 50 (voice recognition unit 52) inputs the voice data extracted in step S51, and extracts text data that matches the voice data. Specifically, text data corresponding to the input voice data is extracted by using a conversion table in which the voice data and the text data are associated with the input of the voice data. Thereafter, the control unit 50 (voice recognition unit 52) moves the process to step S53.

  In step S53, the control unit 50 (voice recognition unit 52) generates a document in which the text data extracted in step S52 is combined. Thereafter, the control unit 50 (voice recognition unit 52) ends this processing and returns to the main processing.

  Next, the keyword extraction process will be described with reference to FIG.

  First, in step S61, the control unit 50 (important word extraction unit 53) performs morphological analysis of text data. Specifically, the text data is divided into parts of speech such as nouns. Thereafter, the control unit 50 (important word extraction unit 53) moves the process to step S62.

  In step S62, the control unit 50 (important word extraction unit 53) puts the result data analyzed in step S61 into the WK of the storage unit 60. Thereafter, the control unit 50 (important word extraction unit 53) moves the process to step S63.

  In step S63, the control unit 50 (important word extraction unit 53) extracts one piece of data from the result data put in WK and calculates a TF * IDF value. Thereafter, the control unit 50 (important word extracting unit 53) moves the process to step S64.

  The TF * IDF value is a value calculated by the TF-IDF method, and is calculated by an algorithm using two indexes, an index called TF (Term Frequency) and an index called IDF (Inverse Document Frequency). Value. This value can be scored for each keyword. The higher the TF * IDF value, the more important the keyword.

  In step S64, the control unit 50 (important word extraction unit 53) determines whether or not the result data stored in the WK of the storage unit 60 is empty. If all the result data stored in WK are processed and are empty (when YES is determined in step S64), control unit 50 (important word extraction unit 53) moves the process to step S65. On the other hand. If all the result data stored in the WK has not been processed and is not empty (NO is determined in the process of step S64), the control unit 50 (important word extracting unit 53) performs the process in step S63. Move to.

  In step S65, the control unit 50 (important word extraction unit 53) performs ranking based on the TF * IDF value calculated in step S63. Thereafter, the control unit 50 (important word extraction unit 53) moves the process to step S66.

  In step S66, the control unit 50 (important word extracting unit 53) extracts the most significant data as an important word. Thereafter, the control unit 50 (important word extraction unit 53) ends this processing and returns to the main processing.

  Finally, representative image extraction processing will be described with reference to FIG.

  First, in step S71, the control unit 50 (representative image extraction unit 54) extracts one image data from the moving image data of the scene i. Thereafter, the control unit 50 (representative image extraction unit 54) moves the process to step S72.

  In step S72, the control unit 50 (representative image extraction unit 54) further extracts partial image data from the extracted image data, and stores the partial image data in the representative image extraction DB 64 for each partial image data. Thereafter, the control unit 50 (representative image extraction unit 54) shifts the processing to step S73.

  In step S73, the control unit 50 (representative image extraction unit 54) calculates an area ratio for each partial image data extracted in step S72 and stored in the representative image extraction DB 64. The area ratio is obtained by dividing the area of the entire image data from the calculated area of the partial image data. Thereafter, the control unit 50 (representative image extraction unit 54) shifts the processing to step S74.

  In step S74, the control unit 50 (representative image extraction unit 54) determines whether or not the processing for calculating the area ratio has been performed for all the partial image data extracted in step S72. When the control unit 50 (representative image extraction unit 54) performs the area ratio calculation processing for all partial image data (when YES is determined in the process of step S74), the control unit 50 (representative image). The extraction unit 54) moves the process to step S75. On the other hand, when the control unit 50 (representative image extraction unit 54) has not performed the area ratio calculation process for all the partial image data (when NO is determined in the process of step S74), the control unit 50 The (representative image extraction unit 54) moves the process to step S73.

  In step S75, when the same partial image data exists in a plurality of continuous frames (image data), the control unit 50 (representative image extraction unit 54) counts the number of frames (number of image data). Thereafter, the control unit 50 (representative image extraction unit 54) shifts the processing to step S76.

  In step S76, the control unit 50 (representative image extraction unit 54) determines whether or not processing has been performed for all image data of the scene i. If all image data has been processed (YES is determined in step S76), control unit 50 (representative image extraction unit 54) moves the process to step S77. On the other hand, when the processing has not been performed for all the image data (when NO is determined in the process of step S76), the control unit 50 (representative image extraction unit 54) moves the process to step S71 and performs the remaining processing. The process from step S71 to step S75 is performed on the image data.

  In step S77, the control unit 50 (representative image extraction unit 54) calculates an OS * iIF value for the partial image data stored in the representative image extraction DB 64. Based on the calculated OS * iIF value, ranking is performed in descending order. Thereafter, the control unit 50 (representative image extraction unit 54) shifts the processing to step S78.

The OS * iIF value is a value calculated by the following calculation formula.

  Here, OS * iIF is obtained by multiplying OS (Object Space) and iIF (inverse image frequency). OS indicates an area ratio that is a ratio of the partial image data to the entire image data area, and N indicates the number of frames (number of image data) of all image data included in the moving image data of the scene i. IF indicates the number of frames (number of image data) in which the target partial image data appears. Partial image data with a large area ratio displayed in a short time has a large OS * iIF value. Therefore, the representative image data can be extracted by scoring each partial image data included in the moving image data by using this calculation formula in which the OS * iIF value has a high score.

  In step S78, the control unit 50 (representative image extraction unit 54) acquires the higher-order data of the OS * iIF values ranked in step S77, and extracts this as representative image data. Thereafter, the control unit 50 (representative image extraction unit 54) ends this processing and returns to the main processing.

  As described above, the subroutine called from the main process of FIG. 4 has been described with reference to FIGS. 5 to 8. However, the subroutine is not limited to these processes, and other known methods may be used.

  For example, regarding the scene division processing, the division processing is performed in response to switching to a completely different image. However, the present invention is not limited to this. For example, during the period of a moving image captured by continuously moving the camera, Estimate the presence / absence of the body, determine the scene where the photographer moved the camera to track a specific subject and the scene where the camera moved to move the line of sight to another subject. It is good also as performing a division | segmentation process regarding what moved the camera in order to move a line of sight.

[Hardware Configuration of Synopsis Creation Device 10]
FIG. 9 is a diagram showing a hardware configuration of the synopsis creation device 10 according to an example of the preferred embodiment of the present invention. The synopsis creation apparatus 10 includes a central processing unit (CPU) 110 (a central processing unit) that includes a scene extraction unit 51, a voice recognition unit 52, an important word extraction unit 53, a representative image extraction unit 54, and a scene integration unit 55. In a multiprocessor configuration, a plurality of CPUs such as the CPU 120 may be added), a bus line 105, a communication I / F 140, a main memory 150, a BIOS (Basic Input Output System) 160, a USB port 190, an I / O controller 170, and Input means such as a keyboard and mouse 180 and a display device 122 are provided.

  The I / O controller 170 can be connected to storage means constituting the storage unit 60 such as a tape drive 172, a hard disk 174, an optical disk drive 176, and a semiconductor memory 178.

  The BIOS 160 stores a boot program executed by the CPU 110 when the synopsis creation device 10 is started, a program depending on the hardware of the synopsis creation device 10, and the like.

  The hard disk 174 stores various programs for functioning as the synopsis creation device 10 and programs for executing the functions of the present invention, and can configure various databases as necessary.

  As the optical disk drive 176, for example, a DVD-ROM drive, a CD-ROM drive, a DVD-RAM drive, or a CD-RAM drive can be used. In this case, the optical disk 177 corresponding to each drive is used. A program or data may be read from the optical disk 177 by the optical disk drive 176 and provided to the main memory 150 or the hard disk 174 via the I / O controller 170. Similarly, the tape medium 171 corresponding to the tape drive 172 can be used mainly for backup.

  The synopsis created by the synopsis creation device 10 and stored in the synopsis DB 66 can be written to the optical disc 177 via the optical disc drive 176.

  The program provided to the synopsis creation device 10 is provided by being stored in a recording medium such as the hard disk 174, the optical disk 177, or a memory card. This program may be read from the recording medium via the I / O controller 170 or downloaded via the communication I / F 140 to be installed and executed in the synopsis creation apparatus 10.

  The above program may be stored in an internal or external storage medium. Here, as a storage medium constituting the storage unit 60 including the moving image DB 62, the representative image extraction DB 64, and the synopsis DB 66, in addition to the hard disk 174, the optical disk 177, or the memory card, a magneto-optical recording medium such as an MD, a tape medium 171 can be used. Further, a storage device such as a hard disk 174 or an optical disk library provided in a server system connected to a communication line such as a dedicated communication line or the Internet is used as a recording medium, and a program is provided to the synopsis creation apparatus 10 via the communication line. May be.

  Here, the display device 122 displays a screen for accepting data input to the administrator of the synopsis creation device 10 or displays a calculation processing result screen by the synopsis creation device 10. CRT) and display devices such as a liquid crystal display (LCD).

  Here, the input means accepts input by the administrator of the synopsis creation device 10 and may be configured by a keyboard and a mouse 180 or the like.

  The communication I / F 140 is a network adapter for enabling the synopsis creation apparatus 10 to be connected to a terminal via a dedicated network or a public network. The communication I / F 140 may include a modem, a cable modem, and an Ethernet (registered trademark) adapter.

  In the above example, the synopsis creation device 10 has been mainly described. However, the functions described above can also be realized by installing a program in a computer and causing the computer to operate as the synopsis creation device 10. Therefore, the functions realized by the synopsis creation apparatus 10 described as an embodiment in the present invention are executed by executing the above-described method by the computer, or by introducing the above-described program into the computer and executing it. Is also feasible.

  As mentioned above, although embodiment of this invention was described, this invention is not restricted to embodiment mentioned above. The effects described in the embodiments of the present invention are only the most preferable effects resulting from the present invention, and the effects of the present invention are limited to those described in the embodiments of the present invention. is not.

1 is a diagram illustrating an overall configuration of a computer system and a functional configuration of a synopsis creation apparatus according to an example of a preferred embodiment of the present invention. It is a conceptual diagram until an outline is produced by the outline production apparatus which concerns on an example of suitable embodiment of this invention. It is a figure which shows the example which isolates a scene with the synopsis production apparatus which concerns on an example of suitable embodiment of this invention. It is a figure which shows the main flow about the synopsis creation process which concerns on an example of suitable embodiment of this invention. It is a figure which shows the flow about the scene division | segmentation process which concerns on an example of suitable embodiment of this invention. It is a figure which shows the flow about the speech recognition process which concerns on an example of suitable embodiment of this invention. It is a figure which shows the flow about the important word extraction process which concerns on an example of suitable embodiment of this invention. It is a figure which shows the flow about the representative image extraction process which concerns on an example of suitable embodiment of this invention. It is a figure which shows the hardware constitutions of the synopsis preparation apparatus which concerns on an example of suitable embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Computer system 10 Synopsis production apparatus 20 User terminal 30 Communication line 50 Control part 51 Scene extraction part 52 Speech recognition part 53 Important word extraction part 54 Representative image extraction part 55 Scene integration part 60 Storage part 62 Movie DB
64 Representative image extraction DB
66 Synopsis DB

Claims (11)

A moving picture summary automatic creation device that automatically creates a moving picture summary for each scene constituted by a group of the image data from moving picture data constituted by a plurality of image data,
A scene extracting unit that divides the moving image data into the scenes and extracts scene moving image data for each of the divided scenes;
A voice recognition unit that recognizes voice data included in the scene video data and generates text data from the voice data;
An important word extraction unit for extracting an important word as a keyword from the text data;
A representative image extraction unit for extracting representative image data to be an image representing the scene from the scene video data;
A moving picture summary automatic creation apparatus comprising: a scene integration unit that creates the moving picture summary data by integrating the important word and the representative image data for each scene. The moving image data further includes character data together with the image data,
The video summary automatic creation device according to claim 1, wherein the important word extraction unit extracts an important word from the character data and the text data generated by the voice recognition unit.   The representative image extraction unit extracts an object from image data constituting the scene moving image data, and displays the entire display area of the image data, the display area of the object, the number of image data including the object, and the scene moving image data. 3. The moving picture summary automatic creating apparatus according to claim 1, wherein the moving picture summary automatic creating apparatus derives from the number of image data constituting the image using a result calculated by a predetermined calculation formula.   4. The moving picture summary automatic creating apparatus according to claim 1, wherein the important word extraction unit extracts the important words from the text data using a TF * IDF method.   5. The moving image summary automatic creation device according to claim 1, wherein the scene integration unit further integrates the moving image summary data generated for each scene moving image data so that the moving image data is displayed at the beginning of the moving image data. A video summary automatic creation method for automatically creating a video summary for each scene constituted by a group of image data from video data constituted by a plurality of image data,
Dividing the video data into the scenes, and extracting scene video data for each of the divided scenes;
Recognizing audio data included in the scene video data and generating text data from the audio data;
Extracting an important word as a keyword from the text data;
Extracting representative image data to be an image representing the scene from the scene video data;
A method of automatically creating a moving picture summary including the step of creating the moving picture summary data by integrating the important word and the representative image data for each scene. The moving image data further includes character data together with the image data,
7. The moving picture summary automatic creation method according to claim 6, wherein the step of extracting the important word extracts the important word from the character data and the text data generated by the step of generating the text data.   The step of extracting the representative image data includes extracting an object from the image data constituting the scene video data, and displaying the entire display area of the image data, the display area of the object, the number of image data including the object, and the 8. The moving image summary automatic creation method according to claim 6 or 7, wherein the method is derived from the number of image data constituting the scene moving image data, using a result calculated by a predetermined calculation formula.   9. The moving image summary automatic creating method according to claim 6, wherein the step of extracting the important word extracts the important word from the text data by using a TF * IDF method.   10. The video summary automatic according to claim 6, wherein the step of creating the video summary data further integrates the video summary data created for each scene video data so as to be displayed at the beginning of the video data. How to make. A computer program that automatically creates a summary of a moving picture from a moving picture data constituted by a plurality of image data, for each scene constituted by a group of the image data,
Dividing the video data into the scenes, and extracting scene video data for each of the divided scenes;
Recognizing audio data included in the scene video data and generating text data from the audio data;
Extracting an important word as a keyword from the text data;
Extracting representative image data to be an image representative of the scene from the scene video data;
A computer program for causing a computer to execute the step of integrating the important word and the representative image data for each scene to create moving image summary data.

Images