JP2007243359A - Device and program for extracting video-image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a video-image extraction device that is compatible with video-images in various fields and is capable of extracting a video image corresponding to information showing a part of video-image contents without utilizing knowledge or metadata in manual extraction. <P>SOLUTION: The video-image extraction device 5 is provided with a rarity data storage means (a morpheme probability information storage means) 56 which stores rarity showing an appearance probability of each morpheme included in voice text data of a plurality of other video images; a similarity calculation means 55 which calculates similarity between each sentence in an introductory text and a CC sentence on the basis of the rarity and an appearance frequency of a morpheme included in each sentence of the introductory text, in a plurality of the CC sentences constituting the voice text data (CC) of the inputted video images; a candidate section detection means (a section detection means) 57 for selecting the CC sentence on the basis of the similarity so as to detect a video-image section corresponding to the CC sentence; and a section video-image extractor (a section video-image extraction means) 58a for extracting a video image of the section from the inputted video images. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a video extraction apparatus and a video extraction program for extracting a part corresponding to this information from a video based on information indicating a part of the content of the video.

  Conventionally, for a video in a certain field, a part of the video is extracted to generate a summary video or a spot video for introducing a program. In these techniques, when extracting a part of a video, knowledge in the case of manually extracting from a video in the field is used. For example, in a baseball video, before the slow video, there is a high possibility that it is an important play video such as a scoring scene, and when a character super is displayed at this position, it is when a score is entered Such visual knowledge is used. Furthermore, in sports images, audio information is used that a section with a large cheer is an important scene.

In addition, a technique for generating a summary video from a video is disclosed (see Patent Document 1). Here, the importance level of the event is calculated based on how many keywords related to the user's preference are included in the event sequence listing the events that occurred in the video as events. The video of the section corresponding to the high event is extracted to be a summary video.
JP-A-2004-289513 (paragraph numbers 0022 to 0074)

  However, when generating summary video using the knowledge of manual extraction, in order to deal with videos in various fields, we have knowledge to generate summary video for videos in various fields. Need to be. Moreover, in most cases, they are extracted sensuously by hand, and the rules for extraction are not clear. In addition, it is not clear how to determine what knowledge is important, and when there are multiple extraction methods for a video in a certain field, it is not possible to determine which extraction method has priority. . Further, the technique described in Patent Document 1 has a problem that it cannot be applied when there is no metadata corresponding to a video.

  The present invention has been made to solve the above-described problems of the prior art, and corresponds to videos in various fields, and does not use knowledge and metadata when manually extracted, and part of the contents of the video. An object of the present invention is to provide a video extraction apparatus and a video extraction program capable of extracting a video corresponding to this information from information indicating the above.

  In order to solve the above-described problem, the video extraction device according to claim 1, a video, audio text data that is audio text data corresponding to the video, and extracted video content information indicating a content of a part of the video, And extracting a part of the video corresponding to the extracted video content information, comprising: a morpheme probability information storage means, a similarity calculation means, a section detection means, and a section video extraction means. It was set as the structure provided.

  According to such a configuration, the video extracting device stores, in the morpheme probability information storage unit, a plurality of other voice text data of each morpheme included in the other voice text data which is voice text data corresponding to a plurality of other videos. Appearance probability information indicating the appearance probability in, and morpheme probability information in which the morpheme is associated are stored.

  Further, the video extraction device uses the similarity calculation means to generate appearance probability information corresponding to each morpheme included in the extracted video content information and voice classification data obtained by dividing the voice text data into a plurality of sections in the morpheme probability information. Based on the frequency at which the morpheme appears, a similarity indicating the degree of similarity between the extracted video content information and the audio classification data is calculated. Here, morphemes with a high appearance probability appear in audio corresponding to various videos, and thus do not indicate video characteristics. On the other hand, a morpheme with a low appearance probability has a specific video or a specific section. Since it appears only in the audio corresponding to the video, there is a high possibility that it shows the characteristics of the video. For this reason, the more the extracted video content information and the audio classification data contain more morphemes with a low appearance probability, the more similar the characteristics of the video content indicated by each of the extracted video content information and the audio classification data. Thus, the video extraction device can detect each piece of audio segment data and extracted video content based on the appearance probability information of each morpheme included in the extracted video content information and the frequency with which the morpheme appears in each audio segment data. Information similarity can be calculated.

  Further, the video extraction device selects the audio segment data corresponding to the extracted video content information based on the similarity by the segment detection unit, detects the segment of the video corresponding to the audio segment data, and the segment video extraction unit Thus, the video of the detected section is extracted. As a result, the video extraction device can extract the video having the content indicated by the extracted video content information from the input video.

  According to a second aspect of the present invention, there is provided the video extracting device according to the first aspect, further comprising a cut dividing unit that divides the video extracted by the section video extracting unit into cuts.

  As a result, the video extraction device can convert the video that has been continuously shot into the cut video when the video is largely switched in the middle of the video of the zone corresponding to the audio classification data extracted by the zone video extraction means. Can be divided.

  The video extraction program according to claim 3 is composed of a video, audio text data that is audio text data corresponding to the video, and a plurality of morphemes, and indicates the content of a part of the video Information, and stored in the morpheme probability information storage device, each morpheme included in other speech text data which is speech text data corresponding to a plurality of other images, in the plurality of other speech text data Based on the appearance probability information indicating the appearance probability and the morpheme probability information corresponding to the morpheme, the computer is used to extract a part of the video corresponding to the extracted video content information. And function as section video extraction means.

  According to such a configuration, the video extraction program divides the appearance probability information corresponding to each morpheme included in the extracted video content information and the voice text data into a plurality of sections in the morpheme probability information by the similarity calculation means. Based on the frequency of appearance of the morpheme in each of the audio classification data, a similarity indicating the degree of similarity between the extracted video content information and the audio classification data is calculated. Further, the video extraction program selects the audio segment data corresponding to the extracted video content information based on the similarity by the segment detection unit, detects the segment of the video corresponding to the audio segment data, and the segment video extraction unit Thus, the video of the detected section is extracted. As a result, the video having the content indicated by the extracted video content information can be extracted from the input video.

  The video extraction apparatus and video extraction program according to the present invention have the following excellent effects. According to the first and third aspects of the present invention, the extracted video content information can be extracted from the input video regardless of the field of the video and without using knowledge and metadata when manually extracting. It is possible to extract a video having the content indicated by. This makes it possible to easily generate summary videos, introduction videos, etc. from videos in various fields, and output video candidates by simply inputting a natural sentence or a list of morphemes indicating the contents of the video to be extracted. This reduces the amount of work for creators who produce summary videos and introductory videos.

  According to the second aspect of the present invention, since the video extracted by the section video extraction means is a section video corresponding to the audio classification data, the video break does not coincide with the video break and the video is cut halfway. In some cases, the images of different scenes are mixed and extracted, but by dividing this video into cuts, it is possible to extract only the continuously shot portions.

Embodiments of the present invention will be described below with reference to the drawings.
[Configuration of video extraction device]
First, with reference to FIG. 1, the structure of the spot image | video production | generation apparatus 1 provided with the image | video extraction device 5 in this invention is demonstrated. FIG. 1 is a schematic diagram schematically showing a configuration of a spot video generation device including a video extraction device according to the present invention.

  The spot video generation device 1 generates a spot video composed of a part of the input video corresponding to the introduction text input from the outside. Here, the spot video generation device 1 includes a morpheme scarcity calculation device 3 and a video extraction device 5.

  The morpheme rarity calculation device 3 calculates the rarity (appearance probability information) of each morpheme included in the broadcast video CC (other audio text data) that is audio text data (closed caption; CC) corresponding to a plurality of videos. To do. The rarity indicates how much a certain morpheme is included in the broadcast video CC of a specific program in a plurality of broadcast video CCs. A certain morpheme is included in the broadcast video CC of a specific program. The higher the value, the higher the value. Here, the target video is a video of a past broadcast program. However, for example, the video may be input from a network such as the Internet, or may be input via a broadcast wave or the like. Good. Further, here, the audio text data is the CC added to the video, but it may be the result of voice recognition of the audio corresponding to the video. Here, the morpheme rarity calculation device 3 includes a broadcast video CC storage unit 30, a morpheme analysis unit 31, a rarity calculation unit 32, and a morpheme frequency storage unit 33.

  The broadcast video CC storage means 30 stores a plurality of broadcast videos CC and is a general storage means such as a hard disk. The broadcast video CC stored here is used by referring to the broadcast video CC by the morphological analysis means 31.

  Here, the configuration of the CC will be described with reference to FIG. FIG. 2 is an explanatory diagram showing an example of CC. In FIG. 2, for convenience of explanation, a line number is assigned to the left side of each line of CC. As shown in FIG. 2, each line of CC is composed of time code information D1 and utterance content information (hereinafter referred to as CC sentence D2). The CC sentence D2 is audio text data corresponding to the video. Here, when one sentence in CC is within a predetermined number of characters, the sentence becomes one CC sentence D2, and when one sentence exceeds a predetermined number of characters, it is divided within the predetermined number of characters. , Are CC statements D2, D2,... [For example, lines (4) to (5)]. The time code information D1 is information indicating the start point of the video corresponding to the CC sentence D2.

  Returning to FIG. 1, the description will be continued. The morpheme analysis unit 31 performs morpheme analysis on each CC sentence of the broadcast video CC stored in the broadcast video CC storage unit 30. Here, the morphological analysis unit 31 selects one broadcast video CC from the plurality of broadcast videos CC stored in the broadcast video CC storage unit 30. Then, the morpheme analyzing unit 31 sequentially performs morpheme analysis on the CC sentences in the selected broadcast video CC. When morphological analysis is completed for all CC sentences of one broadcast video CC, the morphological analysis means 31 sequentially selects the broadcast video CC and performs morphological analysis on each CC sentence. The morpheme information analyzed here (morpheme, part of speech, program name of the broadcast video CC including the morpheme, etc.) is output to the rarity calculation means 32. The morpheme analyzing means 31 can be realized by a general morpheme analysis system.

  The scarcity calculator 32 calculates the scarcity of each morpheme analyzed by the morpheme analyzer 31. The rarity calculated here is stored in the rarity data storage means 56 of the video extraction device 5.

  Hereinafter, a method of calculating the scarcity of a morpheme by the scarcity calculator 32 will be described. Here, the broadcast video CC is composed of a plurality of CC sentences, and each CC sentence is composed of a set of morphemes. The scarcity calculation means 32 receives the morpheme of each CC sentence of each broadcast video CC. Then, the rarity calculation means 32 obtains the frequency with which each morpheme appears in the CC sentence, and stores each morpheme, the part of speech, the frequency, and the program name in association with each other in the morpheme frequency storage means 33. Then, when the analysis of the frequency of each morpheme for each CC sentence of all broadcast images CC is completed, entropy is calculated for each morpheme. This entropy indicates how biased morphemes appear in all broadcast video CCs stored in the broadcast video CC storage unit 30.

The rarity calculation means 32 calculates the probability P (t _j , g _i ) included in the broadcast video CCg _i with a certain morpheme t _j by the following equation (1). Here, the broadcast video CCg _i is represented by a set of a plurality of CC statements, G represents a set of broadcast videos CC stored in the broadcast video CC storage means 30, and tf (t, l) is a CC statement. This represents the frequency of the morpheme t appearing in l.

Then, the rarity calculator 32 calculates the entropy H (t _j ) of the morpheme t _j by the following equation (2). It can be said that the smaller the value of this entropy H (t _j ) is, the morpheme that appears biased toward a specific program.

Here, the rarity degree calculation means 32 gives a large value to the morphemes that appear in a biased manner, so that the rarity degree S (t _j ) in which the increase and decrease of the entropy H (t _j ) are reversed. Is calculated by the following equation (3). Here, | G | indicates the total number of broadcast video CCs stored in the broadcast video CC storage unit 30.

The rarity calculation means 32 obtains the rarity S (t _j ) of each morpheme, and associates each morpheme with the part of speech and the rarity S (t _j ) to store the rarity data storage means of the video extraction device 5. 56. The rarity S (t _j ) is low in morphemes widely used in all programs in past broadcast programs, and is used only in specific programs, or almost used in past broadcast programs. It is high for morphemes that are not. The calculation of the morpheme rarity by the morpheme rarity calculator 3 may be performed only once in advance and stored in the rarity data storage means 56 of the video extractor 5.

  The morpheme frequency storage means 33 stores the morpheme included in a plurality of broadcast images CC, the part of speech, the frequency in the CC sentence of the morpheme, and the program name in association with each other, and is a general storage means such as a hard disk. It is. The frequency data stored here is referred to and used when the rarity level calculation means 32 calculates the rarity level.

  The video extraction device 5 inputs a video, CC (voice text data) which is audio text data corresponding to the video, and an introduction sentence of the video, and generates a spot video which is a video corresponding to the introduction sentence. Is to be generated. Here, the audio text data is CC added to the video, but it may be the result of simply recognizing the audio corresponding to the video. Here, the video extraction device 5 includes an introduction sentence input unit 50, a CC input unit 51, a video input unit 52, a morpheme analysis unit 53, a morpheme analysis unit 54, a similarity calculation unit 55, and a rarity data. The storage means 56, the candidate area detection means 57, the area image division means 58, and the spot image output means 59 are provided.

  The introductory text input means 50 inputs an introductory text indicating a part of the content of the video input from the video input means 52 from the outside. The introductory sentence input here is output to the morpheme analyzing means 53. Here, the introductory text input means 50 inputs the program introductory text described in the electronic program guide or the like. Since the program introduction sentence is usually composed of a plurality of sentences, the case where the introduction sentence is composed of a plurality of sentences (extracted video content information) will be described here.

  The CC input means 51 inputs CC, which is audio text data corresponding to the video input from the video input means 52, from the outside. Here, the audio text data is the CC added to the video, but it may be the result of voice recognition of the audio corresponding to the video. The CC input here is output to the morphological analysis means 54.

  The video input means 52 is for inputting video from outside. Here, the video input means 52 inputs the video of the broadcast program from the outside. However, the input video may be, for example, a video input from a network such as the Internet, or a video input via a broadcast wave or the like. The video input here is output to the segment video dividing means 58.

  The morpheme analysis unit 53 performs morpheme analysis on each sentence of the introduction sentence input from the introduction sentence input unit 50. Here, the morpheme analyzing means 53 sequentially performs morpheme analysis on the sentences included in the introduction sentence. The analyzed morpheme information (morpheme, part of speech, etc.) is output to the similarity calculation means 55. The morpheme analyzing means 53 can be realized by a general morpheme analysis system.

  The morpheme analysis unit 54 performs morpheme analysis on each CC sentence (voice classification data) of the CC input from the CC input unit 51. Here, the morpheme analyzing means 54 sequentially performs morpheme analysis on the CC sentences included in the CC. The analyzed morpheme information (morpheme, part of speech, etc.) is output to the similarity calculation means 55. The morpheme analyzing means 54 can be realized by a general morpheme analysis system.

  The similarity calculation means 55 calculates the similarity between each sentence of the introduction sentence input from the introduction sentence input means 50 and each CC sentence of the CC input from the CC input means 51. The similarity calculated here is output to the candidate section detecting means 57.

Hereinafter, a method of calculating the similarity by the similarity calculator 55 will be described. The similarity calculation means 55 can calculate the similarity Sim (q, d _i ) between a certain sentence q in the introduction sentence and the CC sentence d _i in the CC by the following equation (4).

Here, Q is a set of morphemes included in the sentence q in the introduction sentence, D is a set of CC sentences d _i included in the CC, and | D | is a total number of CC sentences d _i included in the CC. Here, the similarity calculation means 55 reads the rarity S (t) corresponding to each morpheme t included in the sentence q in the introduction sentence input from the morpheme analysis means 53 from the rarity data storage means 56. The frequency tf (t, d _i ) at which each morpheme t input from the morpheme analyzing means 54 appears in the CC sentence d _i is obtained, and the similarity Sim (q, d _i ) is obtained by the above equation (4). calculate.

  Here, with reference to FIG. 3, the relationship between each morpheme and rarity will be described using a specific example. FIG. 3 is an explanatory diagram showing a specific example of the scarcity of morphemes contained in the introduction text of a specific program. Here, the morpheme, the part of speech of the morpheme, and the rarity are shown in the same column in descending order of rarity. As shown in FIG. 3, the rarity level of morphemes (for example, “mine damage” and “mineral poison”) that are only slightly included in the plurality of broadcast images CC is set high, and the morphemes included in most of the plurality of broadcast images CC ( For example, the rarity of “te” and “.”) Is set low. The rarity level of morphemes (eg, “search”, “recovery”) included in a part of the plurality of broadcast images CC is set to an intermediate value.

  In this way, introductory sentences and CCs contain not only morphemes that show video features but also many morphemes that do not show video features such as particles, auxiliary verbs, and punctuation marks. It is not possible to evaluate whether the contents of the introductory sentence and the contents indicated by the CC sentence are similar. However, by considering the importance of each morpheme based on the degree of rarity, it is possible to evaluate whether the contents of the introductory sentence and the CC sentence are similar.

  That is, a morpheme having a high degree of rarity is a morpheme that is biased to be included in the CC of a specific program, and is likely to indicate the characteristics of the video. On the other hand, a morpheme with low rarity is highly likely to be a morpheme that is not related to the content of the video and contained in general sentences, such as particles, auxiliary verbs, and punctuation marks. For this reason, if there are many rare morphemes contained in the introductory sentence in a certain CC sentence in the CC, the morpheme indicating the characteristics of the video is shared with this sentence. It is included in many CC sentences and the contents are similar. Therefore, the similarity calculation means 55 uses the rarity degree to calculate the similarity between the sentence in the introduction sentence and the CC sentence including many morphemes with a high degree of rarity among the morphemes included in the sentence in the introduction sentence. By calculating a high value, it is possible to calculate a high degree of similarity between a CC sentence that commonly includes a morpheme indicating a video feature and a sentence in the introduction sentence.

  In Expression (4), the similarity is normalized by the sum of the rarity of morphemes in the introductory sentence, but without doing this, for example, the longer the sentence, as in Expression (5) below, The degree of similarity may be calculated by a method that increases. This is effective when it is considered that it is more important for the sentence in the introduction sentence to contain a lot of important morphemes in the long sentence than in the case where the short sentence has one morpheme having high importance.

The similarity calculation means 55 considers each sentence q and CC sentence d _i of the introduction sentence as a vector of morphemes t included in the introduction sentence and the CC sentence, and uses the distance to calculate the following expression The similarity Sim (q, d _i ) may be calculated by (6). R represents a set of morphemes included in the introduction sentence and the CC sentence.

Furthermore, the similarity calculation unit 55 may calculate the similarity Sim (q, d _i ) by using the cosine instead of the distance by the following equation (7).

  The rarity data storage means (morpheme probability information storage means, morpheme probability information storage device) 56 includes morphemes included in a plurality of broadcast images CC and the morphemes calculated by the rarity degree calculation means 32 of the morpheme rarity degree calculation device 3. Is a general storage means such as a hard disk, which stores data (morpheme probability information) in which the degree of rarity and the part of speech are associated with each other. The scarcity data stored here is referred to and used by the similarity calculation means 55.

  The candidate section detection means (section detection means) 57 selects a CC sentence based on the similarity calculated by the similarity calculation means 55, and detects a video section corresponding to the CC sentence. Here, the candidate section detection unit 57 detects the section of the video corresponding to the CC sentence with the high similarity calculated by the similarity calculation unit 55 as the section of the video constituting the spot video. Information on the section detected here is output to the section video dividing means 58. Here, the candidate section detecting means 57 selects a plurality of CC sentences having a high degree of similarity for each sentence included in the introduction sentence, and selects a plurality of videos corresponding to the CC sentence as video images constituting the spot video. As a candidate, the plurality of video sections are detected. However, the candidate section detection means 57 may select one CC sentence having the highest similarity for each sentence included in the introduction sentence, and detect a section of the video corresponding to the CC sentence.

  Here, the candidate section detection means 57 can determine the start time of the section of the video to be extracted based on the time code information added to the selected CC sentence. Further, the candidate section detecting means 57 can obtain the end time of the section by calculating the time of the section from the number of characters of the CC sentence based on the speech speed, for example. At this time, the candidate section detection means 57, for example, a CC sentence in which one sentence exceeds a predetermined number of characters in CC and is divided within a predetermined number of characters, as in the CC sentence D2 on the (4) line in FIG. Based on the time code information D1 of D2, the number of characters of the CC sentence D2, and the time code information D1 of the next line, audio corresponding to the number of characters is output between the times indicated by the two time code information D1. As a result, the speech speed can be calculated.

  The section video dividing unit 58 extracts a video of a section detected by the candidate section detection unit 57 from the video input from the video input unit 52 and generates a spot video from the video. Here, the section video dividing unit 58 includes a section video extracting unit 58a, a cut dividing unit 58b, a dividing unit 58c, and a video selecting unit 58d.

  The section video extraction unit (section video extraction means) 58 a extracts the video of the section detected by the candidate section detection means 57 from the video input from the video input means 52. The extracted video is output to the cut dividing unit 58b.

  The cut division unit (cut division unit) 58b divides the video extracted by the section video extraction unit 58a into cuts. Here, the term “cut” refers to a video section continuously photographed with one camera. Since the video is largely switched at the break of the video section, the cut dividing unit 58b takes the color difference between the frame images before and after the video extracted by the section video extraction unit 58a, for example, and the difference value is large. Sometimes, the video can be divided to divide the extracted video into cuts. The video of the cut divided here is output to the dividing unit 58c. Note that the cut dividing unit 58b may take, for example, a difference in frequency characteristics between frame images, and may divide the video when the difference is large, or may divide the frame image into a plurality of small regions, Block matching that examines where the small area has moved in the next frame image may be performed, and the video may be divided when the number of small areas for which the movement destination cannot be specified is greater than a predetermined value.

  The dividing unit 58c further divides an image whose cut length is greater than or equal to a threshold among the cut images input from the cut dividing unit 58b based on the motion of the image. The divided video is output to the video selection unit 58d. Note that the threshold value may be a preset value (for example, the average length of cuts used in spot images created so far), or the length of a spot image input from input means (not shown). It may be a value calculated on the basis of the information on the number and the number of sentences included in the introduction sentence.

  Here, an example of a method in which the dividing unit 58c further divides the cut will be described with reference to FIG. FIG. 4 is an explanatory diagram for explaining an example of a method in which the dividing unit further divides the cut. First, the dividing unit 58c determines whether the cut length of the cut video input from the cut dividing unit 58b is equal to or greater than a threshold value. If it is equal to or greater than the threshold, the movement of the camera (not shown) that captured the video is determined based on this video. Here, the dividing unit 58c determines the camera motion by analyzing the motion vector by block matching.

  As shown in FIG. 4A, when the cut video is composed of a section Z1 captured with the camera stationary and a section Z2 captured with the camera moving, the dividing unit 58c. Is divided just before the camera starts moving, that is, divided just before the boundary between the section Z1 and the section Z2 (predetermined time). In this way, by deleting a part of the section Z1, it is possible to remove the part where the camera is stationary and the same video continues for a long time.

  Further, as shown in FIG. 4B, when the cut video is composed of a section Z2 shot with the camera moving and a section Z1 shot with the camera stationary, the division is performed. The unit 58c is divided slightly before the camera stops, that is, divided slightly before the boundary between the section Z2 and the section Z1 (predetermined time).

  Here, the viewer sees a video that changes from a still camera to another cut as soon as the camera starts moving, or a video that changes from a camera moving to a different cut immediately after the camera moves. In this way, it becomes a natural image as viewed from the viewer by dividing it just before the camera starts moving from a stationary state or just before moving from a moving state. Can be divided as follows.

  Furthermore, as shown in FIGS. 4C, 4D, and 4E, when the cut image was shot in a stationary state after the camera moved a little, or after the camera moved a little still When the image is taken in a state, when the image is taken with the camera kept stationary, or when the image is taken with the camera kept moving, the dividing unit 58c divides the image by a predetermined length. Further, the dividing unit 58c removes the section when it is divided by a predetermined length and the last remaining section is shorter than the predetermined length. When the length of the video obtained by dividing by these methods is equal to or greater than the threshold, the dividing unit 58c further divides by the above method.

  The video selection unit 58d selects a video constituting the spot video from the video input from the dividing unit 58c. The images selected here are joined together and output to the spot image output means 59 as a spot image.

  Here, the video selection unit 58d preferentially selects videos with the highest similarity so that videos with the same direction of motion vectors do not continue and videos with the same color tone do not continue. Then, the selected video is connected to generate a spot video.

  The spot video output unit 59 outputs the spot video input from the video selection unit 58d of the section video division unit 58 to the outside.

  The introductory sentence input from the introductory sentence input means 50 of the video extracting apparatus 5 of the present invention may be composed of one sentence, or may be a list of a plurality of morphemes (extracted video content information). Good. Further, the extracted video content information described in the claims only needs to indicate a part of the content of the video, and may be, for example, a summary sentence instead of an introduction sentence. Furthermore, here, the video extraction device 5 extracts the corresponding video for each sentence of the introductory text. However, it is also possible to extract the corresponding video for the entire introductory text. At this time, the extracted video content information described in the claims corresponds to an introduction sentence. The video extraction device 5 may extract a video corresponding to each of the character strings obtained by dividing the introduction sentence based on, for example, the number of characters and the time length. At this time, the extracted video content information described in the claims corresponds to this character string.

  Here, the scarcity is calculated by the morpheme scarcity calculation device 3 and stored in the scarcity data storage means 56 of the video extraction device 5. For example, the morpheme scarcity calculation device 3 calculates the entropy of the morpheme. Then, it may be stored in the rarity data storage means 56. At this time, the appearance probability information described in the claims corresponds to entropy, and the video extraction device 5 uses the similarity calculation means 55 to increase or decrease the entropy so as to give a large value to the morphemes that appear unevenly. Conversely, the similarity may be calculated.

  Furthermore, the morpheme rarity calculation device 3 and the video extraction device 5 can also realize each means as each function program in a computer, and combine each function program as a morpheme rarity calculation program and a video extraction program. It is also possible to operate.

[Operation of spot image generator]
Next, the operation of the spot video generation device 1 will be described with reference to FIGS. FIG. 5 is a flowchart showing an operation in which the morpheme scarcity calculating device calculates the scarcity of the morpheme stored in the scarcity data storage means of the video extracting device of the present invention. FIG. 6 is a flowchart showing an operation in which the video extraction apparatus of the present invention calculates the similarity between each sentence of the introductory sentence and the CC sentence, and generates a spot video corresponding to the introductory sentence.

(Operation of morpheme rarity calculator)
First, referring to FIG. 5 (refer to FIG. 1 as appropriate), the operation of the morpheme scarcity calculation device 3 (rareness calculation operation) will be described.

  The morpheme scarcity calculation device 3 reads one broadcast video CC among the plurality of broadcast videos CC stored in the broadcast video CC storage unit 30 by the morpheme analysis unit 31 (step S31). Subsequently, the morpheme scarcity calculation device 3 uses the morpheme analysis unit 31 to perform morpheme analysis of one CC sentence of the broadcast video CC read in step S31 (step S32).

  Further, the morpheme rarity calculation device 3 obtains the frequency of appearance of each morpheme included in the CC sentence based on the morpheme analysis result in step S32 by the rarity calculation unit 32, and the morpheme, the part of speech, the frequency, The program name is associated with and stored in the morpheme frequency storage means 33 (step S33). Then, the morpheme scarcity calculation device 3 determines whether or not the morpheme analysis unit 31 has finished all the CC sentences read in step S31 (step S34). And when not complete | finished (it is No at step S34), it returns to step S32 and the operation | movement after the operation | movement after the morpheme scarcity calculation apparatus 3 performs the morphological analysis of the next CC sentence by the morpheme analysis means 31 is performed. Do.

  When the processing is completed (Yes in step S34), the morpheme scarcity calculation device 3 determines whether or not all broadcast images CC have been read in step S31 by the morpheme analysis unit 31 (step S35). If all the broadcast images CC have not been read (No in step S35), the process returns to step S31, and the morpheme scarcity calculating device 3 reads the next broadcast image CC by the morpheme analyzing means 31. Perform the operation.

  When all broadcast videos CC have been read (Yes in step S35), the morpheme scarcity calculating device 3 is based on the frequency stored in the morpheme frequency memory 33 in step S33 by the scarcity calculator 32. The degree of rarity of each morpheme is calculated, the morpheme, the part of speech, and the degree of rarity are associated with each other and stored in the rarity degree data storage means 56 of the video extraction device 5 (step S36), and the operation is terminated.

  Through the above operation, the morpheme scarcity calculating device 3 calculates the scarcity of morphemes included in the plurality of broadcast videos CC stored in the broadcast video CC storage unit 30, and the rarity data storage unit 56 of the video extraction device 5. To remember.

(Operation of video extractor)
Next, the operation (video extraction operation) of the video extraction device 5 will be described with reference to FIG.

  The video extraction device 5 inputs the introduction sentence by the introduction sentence input means 50, the CC by the CC input means 51, and the picture from the outside by the video input means 52 (step S51). Subsequently, the video extraction device 5 performs morpheme analysis of one sentence of the introduction sentence input in step S51 by the morpheme analysis unit 53 (step S52).

  Further, the video extraction device 5 performs morphological analysis of one CC sentence of the CC input in step S51 by the morphological analysis unit 54 (step S53). Then, the video extraction device 5 uses the similarity calculation unit 55 to calculate the similarity between the introductory sentence analyzed in step S52 and the CC sentence analyzed in step S53 (step S54).

  Then, the video extracting device 5 determines whether or not all the CC sentences input in step S51 have been completed by the morphological analyzer 54 (step S55). If it is not completed (No in step S55), the process returns to step S53, and the video extraction device 5 performs the operation after the operation of performing the morphological analysis of the next CC sentence by the morphological analysis means 54.

  When the processing is completed (Yes in step S55), the video extraction device 5 selects a CC sentence with a high similarity calculated in step S54 by the candidate section detection unit 57 (step S56), and the CC sentence The video section corresponding to is detected (step S57). Then, the video extraction device 5 determines whether the morphological analysis unit 53 has finished all the introductory sentences input in step S51 (step S58). If it has not ended (No in step S58), the process returns to step S52, and the video extraction device 5 performs the morphological analysis of the next sentence after the introduction sentence by the morphological analysis means 53. Do.

  If the processing has been completed (Yes in step S58), the video extraction device 5 has received the video of the section detected in step S57 by the section video extraction unit 58a of the section video dividing means 58 in step S51. Extract from the video (step S59). Then, the video extracting device 5 divides the video extracted in step S59 into cuts by the cut dividing unit 58b of the section video dividing unit 58, and the dividing unit 58c converts the video whose cut length is greater than or equal to the threshold to the motion of the video. Are further divided to adjust the video section (step S60).

  Subsequently, the video extracting device 5 selects and joins the video from the video whose section is adjusted in step S60 by the video selection unit 58d of the section video dividing unit 58 based on the motion, color tone, and similarity of the video. A spot video is generated (step S61). Further, the video extracting device 5 outputs the spot video generated in step S61 by the spot video output means 59 (step S62).

  With the above operation, the video extraction device 5 uses the morpheme scarcity calculation device 3 to calculate the similarity between each sentence of the introduction sentence input from the outside and the CC sentence based on the scarcity of each morpheme. Can be calculated. Then, a spot video can be generated by extracting a video of a section corresponding to a CC sentence having a high similarity.

[Application example of video extractor]
The video extraction device 5 has a video display means (not shown), which outputs information about the section, introduction text, similarity, etc. to the display means (not shown) together with the spot video. It is good also as displaying on. Further, the video extraction device 5 may include spot video editing means (not shown), and the spot video editing means may edit the spot video based on an operator command input from command input means (not shown). Good.

  Here, an application example of the video extraction device 5 will be described with reference to FIG. FIG. 7 is a schematic diagram schematically showing an example of a spot video editing screen generated by the video extraction device. When the operator inputs a video, a CC of the video, and an introduction sentence to the video extraction device 5, the video extraction device 5 extracts a video corresponding to each sentence of the introduction sentence. Then, as shown in FIG. 7, the video extraction device 5 is selected and edited from the spot video Va edited by the operator and the candidate video by the operator on the display screen by video display means (not shown). Corresponding to the video Vb as the material, the information Vc of the video section constituting the spot video, the sentences Vd1, Vd2, Vd3,... And the sentences Vd1, Vd2, Vd3,. .. Are displayed in the descending order. Then, the spot video editing means (not shown) adjusts the length of the cut to the spot video on the timeline or adds other candidate images Ve to the operator so that the operator can make an introductory sentence. Corresponding spot images can be produced easily.

  Furthermore, the video extraction device 5 may be applied to video search such as the Internet. For example, when applied to Internet video search, the video extraction device 5 is connected to the Internet, and the text data input from the introductory text input means 50 and the CC of the Internet video input from the CC input means 51. Based on the above, a video showing the contents of the text data is selected from the Internet video inputted from the video input means 52. Here, as shown in FIG. 8 (a), a plurality of morphemes t1 and t2 are input as text data from the introductory text input means 50 by the operator instead of the introductory text. Based on the above, the similarity between the morphemes t1 and t2 input from the introductory sentence input means 50 and the CC sentence of the CC input from the CC input means 51 is calculated, and the video in the section with high similarity is selected. To do.

  Then, as shown in FIG. 8 (b), the video display means displays video Ve, Ve,... In a section with high similarity on the display screen, and similarity information Vf, Vf of the video Ve, Ve,. ,... Are displayed as search results. FIG. 8 is a schematic diagram schematically showing an example of a display screen when the video extracting device of the invention is applied to Internet video search. FIG. 8A is a morpheme indicating the content of the video to be searched. The schematic diagram which showed the example of the input screen to input, (b) is the schematic diagram which showed the example of the screen which displays the searched image | video.

It is the schematic diagram which showed typically the structure of the spot image | video production | generation apparatus provided with the image | video extraction apparatus in this invention. It is explanatory drawing which showed the example of CC used for a spot image generation apparatus provided with the image | video extraction apparatus in this invention. It is explanatory drawing which shows the specific example of the scarcity of the morpheme contained in the introduction sentence of the specific program calculated by the spot image | video production | generation apparatus provided with the image | video extraction apparatus in this invention. It is explanatory drawing for demonstrating the example of the method in which the division part of the video extraction device in this invention further divides | segments a cut. 5 is a flowchart showing an operation of calculating a morpheme rarity stored in a rarity data storage unit of a video extraction device of a video extraction device by a morpheme rarity calculation device of a spot video generation device including the video extraction device according to the present invention. . It is the flowchart which showed the operation | movement which the image extraction apparatus of this invention calculates the similarity degree of each sentence of an introduction sentence, and CC sentence, and produces | generates the spot image | video corresponding to an introduction sentence. It is the schematic diagram which showed typically the example of the edit screen of the spot image | video produced | generated by the image | video extraction apparatus of this invention. The schematic diagram which showed the example of the display screen at the time of applying the image | video extraction apparatus of this invention to the search of the image | video of the internet, (a) is an input screen which inputs the morpheme which shows the content of the image | video to search The schematic diagram which showed the example, (b) is the schematic diagram which showed the example of the screen which displays the searched image | video.

Explanation of symbols

5 Video extraction device 55 Similarity calculation means 56 Rareness data storage means (morpheme probability information storage means)
57 Candidate section detection means (section detection means)
58a Section video extraction unit (section video extraction means)
58b Cut division part (cut division means)

Claims (3)

Corresponding to the extracted video content information by inputting a video, audio text data that is audio text data corresponding to the video, and extracted video content information that is composed of a plurality of morphemes and indicates the contents of a part of the video A video extraction device for extracting a part of the video,
Appearance probability information indicating the appearance probability of each morpheme included in other speech text data, which is speech text data corresponding to a plurality of other videos, and the morpheme are associated with each other. Morpheme probability information storage means for storing morpheme probability information;
In the morpheme probability information stored in the morpheme probability information storage means, the appearance probability information corresponding to each morpheme included in the extracted video content information, and speech segment data obtained by dividing the speech text data into a plurality of segments A similarity calculation means for calculating a similarity indicating the degree of similarity between the extracted video content information and the audio classification data, based on the frequency of appearance of the morpheme in each;
Section detection means for selecting the audio segment data corresponding to the extracted video content information based on the similarity calculated by the similarity calculation means and detecting the segment of the video corresponding to the audio segment data;
Section video extraction means for extracting the video of the section detected by the section detection means,
A video extraction apparatus comprising:   The video extracting apparatus according to claim 1, further comprising cut dividing means for dividing the video extracted by the section video extracting means into cuts. A video, audio text data that is audio text data corresponding to the video, and extracted video content information that is composed of a plurality of morphemes and indicates a part of the video are input and stored in a morpheme probability information storage device The appearance probability information indicating the appearance probability of each of the morphemes included in the other voice text data that is the voice text data corresponding to the plurality of other videos, and the morpheme In order to extract a portion of the video corresponding to the extracted video content information based on the matched morpheme probability information,
In the morpheme probability information stored in the morpheme probability information storage device, the appearance probability information corresponding to each morpheme included in the extracted video content information, and speech classification data obtained by dividing the speech text data into a plurality of sections Similarity calculation means for calculating a similarity indicating the degree of similarity between the extracted video content information and the audio classification data based on the frequency of appearance of the morpheme in each;
Section detection means for selecting the audio segment data corresponding to the extracted video content information based on the similarity calculated by the similarity calculation means and detecting the segment of the video corresponding to the audio segment data;
A video extraction program which functions as section video extraction means for extracting a video of a section detected by the section detection means.

Images