JP2008301426A - Featured value generating device, summary video detecting device, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To attain a summary video detection of high detection accuracy which is applicable to a variety of videos. <P>SOLUTION: A feature vector generating device 3 characteristically comprises a program video feature vector generator 30 and a stored video feature vector generator 32, which extract one or a plurality of index words from at least one of character data or an audio signal corresponding to a video, and generates a featured value associated with the video, based on the number of times of appearance of each index word in the video. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a feature quantity generation device, a summary video detection device, and a program.

  In recent years, services that accumulate television broadcast video as program video for each program so that anyone can view it at any time are gaining popularity. In such a service, the amount of the program video to be accumulated becomes enormous, so a summary video is prepared for each program video so that the user can select the program video that he / she wants to see efficiently. It is hoped that.

  Although it is conceivable to create a new summary video when the program video is stored, many programs actually create a spot video or a notification video before broadcasting. Video can be acquired.

  However, the summary video is rarely prepared for each program at the accumulation stage, and it is usually necessary to find the summary video from the video broadcast on television. Therefore, there is a demand for a technique for detecting a portion suitable for a summary video (hereinafter referred to as a summary portion) from a television broadcast video (hereinafter referred to as a stored video).

In this regard, Non-Patent Documents 1 and 2 disclose techniques that can be used as such techniques.
According to the technique disclosed in Non-Patent Document 1, a feature vector based on a color histogram is obtained for each section of accumulated video, and a similar feature vector is obtained for a program video. The part can be detected.

Further, according to the technique disclosed in Non-Patent Document 2, the flash emission pattern of the camera flash is obtained for each section of the stored video, and the same light emission pattern is obtained for the program video, and the similarity of the light emission patterns is obtained. The summary part can be detected.
Kunio Kanno et al., “Quick Search Method for Audio and Video Signals Based on Histogram Pruning”, IEEE TRANSACTIONS ON MULTIMEDIA, Vol.5 No.3, 2003 September, p.348-357 Masao Enomoto et al., “Discovery of the same flash scene video from large-capacity broadcast video archives”, IEICE Transactions (D), Vol. J89-D, No. 12, December 2006, p.2699- 2709

  However, the technique disclosed in Non-Patent Document 1 has a problem that detection accuracy does not increase so much because a summary portion is found based only on color similarity. In addition, the technique disclosed in Non-Patent Document 2 has a problem that it can be applied only to video (such as on-site video inserted into news) in which the flash of the camera is being shot.

  Accordingly, one of the objects of the present invention is to provide a feature amount generation device, a summary video detection device, and a program for realizing summary video detection with high detection accuracy applicable to a wide range of videos.

In order to solve the above problems, a feature value generating apparatus according to the present invention includes an extraction unit that extracts one or a plurality of index words from at least one of character data or audio signals corresponding to a video, and each index word And a feature quantity generating means for generating a feature quantity related to the video based on the number of appearances in the video.
Since the feature quantity reflects the semantic content of the video, the summary image detection performed using the feature quantity is applicable to a wide range of videos and has high detection accuracy. The character data may include closed captions included in the video and character data of a portion related to the video in the electronic program guide. In this way, character data can be acquired from a closed caption or an electronic program guide.

Further, in each of the feature value generation devices, the feature value generation means generates a feature value related to the video based on the rarity of each index word determined based on the number of appearances in the stored video. It's good.
According to this, the accuracy of the summary video detection based on the feature amount can be further increased.

In addition, the feature value generation device according to one aspect of the present invention includes an extraction unit that extracts one or a plurality of index words from at least one of character data or audio signals corresponding to a section video for each section of the video. , A section video feature generating unit that generates a feature amount related to the section video based on the number of appearances in the section video for each index word, and a feature related to each section video generated by the section video feature generating unit And a continuous segment video feature quantity generating unit that generates a feature quantity related to a continuous segment video composed of a plurality of adjacent segment videos based on the quantity.
According to this, it is possible to create a feature amount for each section of the video, and to reduce the processing load of feature amount calculation.

A summary video detection apparatus according to the present invention includes a stored video feature quantity acquisition unit that acquires a feature quantity generated by the feature quantity generation apparatus according to claim 1 for each section of a stored video, and a program video. The program video feature quantity acquisition means for acquiring the feature quantity generated by the feature quantity generation device according to claim 1, the feature quantity related to the section for each section of the stored video, and the program video Similarity calculation means for calculating the similarity between the feature quantity, and summary video detection for detecting the summary video of the program video from each section of the stored video based on the calculation result of the similarity calculation means Means.
According to this, summary video detection with high detection accuracy applicable to a wide range of videos is realized.

According to another aspect of the present invention, there is provided a summary video detection apparatus that generates features generated by the feature value generation device according to claim 3 for each section of accumulated video and each of consecutive sections including a plurality of adjacent sections. Accumulated video feature quantity acquisition means for acquiring a quantity, program video feature quantity acquisition means for acquiring a feature quantity generated by the feature quantity generation device according to claim 1 for the program video, and the section of the stored video And for each of the continuous sections, based on the calculation result of the similarity calculation means for calculating the similarity between the feature quantity related to the section or the continuous section and the feature quantity related to the program video, the calculation result of the similarity calculation means, And summary video detection means for detecting a summary video of the program video from each section and each continuous section of the stored video.
In this way, summary video detection with high detection accuracy applicable to a wide range of videos is realized. In addition, video sections of various lengths can be handled as summary video candidates.

  In addition, the program according to the present invention includes an extracting unit that extracts one or a plurality of index words from at least one of character data or audio signals corresponding to a video, and the number of appearances in the video for each index word. Is a program for causing a computer to function as a feature amount generating means for generating a feature amount related to the video based on the image.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a diagram showing a system configuration of a summary video detection system 1 according to the present embodiment. As shown in the figure, the summary video detection system 1 includes a video database 2, a feature vector generation device 3, a feature vector database 4, and a summary video detection device 5.

  FIG. 1 also shows functional blocks of each device. As shown in the figure, the video database 2 is functionally configured to include an accumulated video storage unit 20, and the accumulated video storage unit 20 includes a program video storage unit 21. The feature vector generation device 3 is functionally configured to include a program video feature vector generation unit 30, a rarity information calculation unit 31, and an accumulated video feature vector generation unit 32. The feature vector database 4 is functionally configured to include a program video feature vector storage unit 40 and an accumulated video feature vector storage unit 41. In addition, the summary video detection device 5 functionally includes a program designation receiving unit 50, a program video feature vector acquisition unit 51, a similarity calculation unit 52, an accumulated video feature vector acquisition unit 53, a similarity storage unit 54, and a summary video detection unit. 55 is comprised. Hereinafter, each of these parts will be described in detail.

  First, the video database 2 will be described. The stored video storage unit 20 is a predetermined video that has been broadcast on television in the past (including a program video that has been edited and broadcast as a program, as well as a spot video, a notification video, etc., hereinafter referred to as a stored video). It is stored in a video format (for example, MPEG). The stored video storage unit 20 also stores character data (closed caption (CC), electronic program guide (EPG) for each program, etc.) and audio signals corresponding to the video in synchronization with the video.

  Next, the feature vector generation device 3 and the feature vector database 4 will be described. Hereinafter, generation of a feature vector related to a program video will be described first. The program video feature vector generation unit 30 generates a feature vector for each program video stored in the stored video storage unit 20. This will be specifically described below.

  FIG. 2 is a diagram illustrating an internal configuration of the program video feature vector generation unit 30. As shown in the figure, the program video feature vector generation unit 30 includes an extraction unit 300 and a feature vector generation unit 301.

The extraction unit 300 extracts one or a plurality of index words from at least one of character data or audio signals added to the program video (extraction means).
The processing of the extraction unit 300 will be described with reference to the drawing showing its internal configuration. FIG. 3 is a diagram illustrating an internal configuration of the extraction unit 300. As shown in the figure, the extraction unit 300 includes a stream separation unit 3000, an index word extraction unit 3002, a speech recognition unit 3003, an index word extraction unit 3004, and an integration unit 3005.

  The stream separation unit 3000 separates added character data (closed captions and electronic program guide information) and audio signals from the video stored in the accumulated video storage unit 20. Then, the character data is output to the index word extraction unit 3002, and the speech signal is output to the speech recognition unit 3003.

  The voice recognition unit 3003 performs a predetermined voice recognition process on the voice signal input from the stream separation unit 3000, thereby converting the voice signal into character data. The speech recognition unit 3003 outputs the acquired character data to the index word extraction unit 3004.

  The index word extraction unit 3002 and the index word extraction unit 3004 extract the index word from each character data by analyzing each character data input from the stream separation unit 3000 and the speech recognition unit 3003, respectively. It is preferable to use morphological analysis for the analysis of character data. In this case, a combination of morpheme and part of speech is used as an index word. That is, even if the morphemes are the same, different index words are obtained if the parts of speech are different. Further, the index word extraction unit 3002 and the index word extraction unit 3004 may extract all of the index words constituting the character data, or may extract only nouns, for example.

  The integration unit 3005 integrates the extraction results of the index word extraction unit 3002 and the index word extraction unit 3004 and outputs one extraction result. Specifically, when one of the index word extraction unit 3002 and the index word extraction unit 3004 fails to obtain an extraction result (such as when there is no character data or audio signal corresponding to the program video), it is obtained by the other. Output the extracted results. Further, when both the index word extraction unit 3002 and the index word extraction unit 3004 obtain the extraction result, the extraction result of the index word extraction unit 3002 with high reliability (not subjected to the speech recognition process) is output. However, in the closed caption, information displayed as a character superimposition on the screen may be omitted. When there is the above omission in the closed caption, the closed caption includes a symbol representing the omission. The integration unit 3005 detects the omitted part by detecting this symbol. Then, the detected omitted part is replaced with the extraction result of the index word extraction unit 3004.

  Returning to FIG. 2, the feature vector generation unit 301 uses the extraction result output from the integration unit 3005 to acquire the number of appearances in the program video for each index word. Then, a feature vector related to the program video is generated based on the obtained number of appearances for each index word (feature amount generating means). At this time, the feature vector generation unit 301 generates a feature vector based on the rarity of each index word determined based on the number of appearances in the stored video.

Specifically, the feature vector generation unit 301 generates a feature amount that is an element for each index word of the feature vector by the following equation (1). Here, tf (t _k , Pi) is the number of appearances of the index word t _k in the program Pi, and S (t _k ) is rarity information indicating the rarity of the index word t _k (described later). Further, v _k ^Pi is a feature amount for the index word t _k of the program Pi.

The feature vector generation unit 301 calculates Equation (1) for all index words included in the extraction result output from the integration unit 3005. And thus used, according to equation (2), generates a feature vector V _Pi is a vector composed of the feature quantity for each index term t _k. Here, the value D is the number of index words stored in the program video feature vector storage unit 40 described later. For an index word that is not included in the extraction result output by the integration unit 3005, the feature vector generation unit 301 substitutes zero for v _k ^Pi when generating the feature vector V _Pi .

The feature vector generation unit 301 stores the feature vector V _Pi in the program video feature vector storage unit 40.
FIG. 4 is a diagram showing a specific example of the contents stored in the program video feature vector storage unit 40. As shown in the figure, the program video feature vector storage unit 40 stores the number of appearances for each television program and the feature amount for each index word. The index words stored in the program video feature vector storage unit 40 are all index words previously extracted from the stored video (including the program video).

When the feature vector generation unit 301 stores the feature vector V _Pi in the program video feature vector storage unit 40, for the index word that has already been stored, the number of occurrences tf (t _k acquired in the index word row is stored. , Pi) and the calculated feature value v _k ^Pi are stored. On the other hand, for an index word that has not yet been stored, a row of the index word is added, and the obtained number of appearances tf (t _k , Pi) and the calculated feature value v _k ^Pi are stored in the added row. For the other rows, zero is stored for both the number of appearances and the feature amount.

Here, the rarity information S (t _k ) will be described. The rarity information calculation unit 31 calculates rarity information S (t _k ) for each index word t _k based on the stored contents of the program video feature vector storage unit 40. Specifically, the scarcity information S (t _k ) is calculated using the formula (3) or the formula (4). Note that pf (t _k ) is the number of program videos in which the number of occurrences of the index word t _k is 1 or more, and the rarity information calculation unit 31 is based on the stored contents of the program video feature vector storage unit 40. t _k ) is calculated. N is the total number of past program videos.

Expression (3) is an IDF (Inverse Document Frequency) value, and Expression (4) is a value based on entropy. By using each of these formulas, the value of the scarcity information S (t _k ) of an index word that appears only in a specific program video is an index word that does not (index word that appears in various program videos). Becomes higher than the value of the scarcity information S (t _k ).

  Next, generation of feature vectors relating to stored video (including program video) will be described. The stored video feature vector generation unit 32 generates a feature vector for the stored video stored in the stored video storage unit 20. This will be specifically described below.

  FIG. 5 is a diagram illustrating an internal configuration of the stored video feature vector generation unit 32. As shown in the figure, the accumulated video feature vector generation unit 32 includes an extraction unit 302 and a feature vector generation unit 303.

  The processing of the extraction unit 302 is substantially the same as the processing of the extraction unit 300 included in the program video feature vector generation unit 30, but for each section of the stored video, of the character data or audio signal added to the section video It differs from the extraction unit 300 in that one or more index words are extracted from at least one. This section is preferably a section having a predetermined time length, and is particularly preferably a section having the greatest common divisor of the time length of commercial video or spot video.

The feature vector generation unit 303 includes a section video feature vector generation unit 3030 and a continuous section video feature vector generation unit 3031.
The section video feature vector generation unit 3030 uses the extraction result output from the extraction unit 302 to acquire the number of appearances in the section video for each index word. Then, a feature vector related to the section video is generated based on the obtained number of appearances for each index word (section video feature value generation means). A specific feature vector generation method is almost the same as that of the feature vector generation unit 301.

However, the feature vector generation unit 301 generates a feature vector for each program video, but the section video feature vector generation unit 3030 generates a feature vector for each section video. For this reason, it is preferable to use different symbols representing the feature vectors, which are defined below. First, a section video starting at time Tx and ending at time Ty is expressed as Tx to Ty. As a result, equation (1) is rewritten as the following equation (5). _Here, ^{v k Tx~Ty} is a feature amount for index terms _{t k} interval program Tx～Ty.

また、区間映像Ｔｘ〜Ｔｙに関する特徴ベクトルは、式（６）のＶ_{Ｔｘ〜Ｔｙ}で表される。

In addition, the feature vectors related to the section videos Tx to _Ty are expressed by _VTx to _Ty in Expression (6).

  The continuous segment video feature vector generation unit 3031 generates a feature vector related to a continuous segment video composed of a plurality of adjacent segment videos based on the feature vector related to each segment video generated by the segment video feature vector generation unit 3030 (continuous). Section video feature vector generation means). Specifically, the continuous segment video feature vector generation unit 3031 generates a feature vector related to a continuous segment video by adding, for each element, a feature vector related to a continuous segment video composed of a plurality of adjacent segment videos.

FIG. 6 is a diagram illustrating an example of a feature vector related to a continuous section video. In the example shown in the figure, first, feature vectors V _T1 to _T2 , V _{T2 to T3} , V _T3 to _T4 , and V _T4 to _T5 are generated for each section video. Next, feature vectors V _{T1 to T3} = V _{T1 to T2} + V _{T2 to T3} and V _{T3 to T5} = V _{T3 to T4} + V _T4 to _T5 are generated every two adjacent sections. Further, feature vectors V _{T1 to T5} = V _{T1 to T3} + V _{T3 to T5} are also generated for every four adjacent sections. Such generation is repeated thereafter. In this example, as a result, the feature vectors are generated hierarchically, and there is no overlap in the same hierarchy.

FIG. 7 is a diagram illustrating another example of the feature vector related to the continuous section video. The example of the figure is an example in the case where duplication within the same hierarchy is recognized in the example of FIG. That is, in this example, when generating feature vectors for every two adjacent sections, feature vectors V _{T1 to T3} = V _{T1 to T2} + V _{T2 to T3} , V _{T3 to T5} = V _{T3 to T4} + V _{T4 to T5} , V _{T2 to T4} = V _{T2 to T3} + V _{T3 to T4} are also generated. V _T2 to _T4 and V _{T1 to T3} , V _T2 to _T4 and V _{T3 to T5} have overlapping sections, respectively. In addition, for the upper layers, feature vectors are generated in a similar manner with recognition of duplication.

  Note that the hierarchical depth in the case of calculating feature vectors hierarchically in this way is the time length of the video to be detected as the summary video, with the section length of the highest hierarchy determined based on the section length of the lowest hierarchy. To be determined. Taking the example shown in FIG. 6 as a specific example, if the section length of the lowest layer is about 10 seconds and the time length of the video to be detected as the summary video is about several minutes, 5-6 It becomes a hierarchy.

  The section video feature vector generation unit 3030 and the continuous section video feature vector generation unit 3031 cause the accumulated video feature vector storage unit 41 to store the feature vectors generated as described above.

  FIG. 8 is a diagram illustrating a specific example of the stored contents of the accumulated video feature vector storage unit 41. As shown in the figure, the accumulated video feature vector storage unit 41 stores the feature quantity for each section for each index word. The index words stored in the stored video feature vector storage unit 41 are all index words extracted from the stored video (including program video) in the past.

When the section video feature vector generation unit 3030 and the continuous section video feature vector generation unit 3031 store the generated feature vector in the accumulated video feature vector storage unit 41, the index word row of the index word already stored is stored. Then, the calculated feature values v _k ^Tx to ^Ty are stored. On the other hand, for an index word that has not yet been stored, a row of the index word is added, and feature quantities v _k ^Tx to ^Ty are stored in the added row. Zero is stored for the feature values of the other rows.

  Next, referring back to FIG. 1, the summary video detection device 5 will be described. First, the summary video detection device 5 includes display means such as a display and input means such as a keyboard / mouse, and the program designation receiving unit 50 uses these to store each of the program video feature vector storage units 40 stored therein. The designation by one user of the program video is accepted.

  The program video feature vector acquisition unit 51 acquires the feature vector generated by the feature vector generation device 3 and stored in the program video feature vector storage unit 40 for the program video specified by the user (program video feature vector acquisition means) ). Then, the acquired feature vector is output to the similarity calculation unit 52.

  The similarity calculation unit 52 that has received a feature vector input from the program video feature vector acquisition unit 51 uses a stored video feature vector acquisition unit 53 (stored video feature vector acquisition means), and uses a feature vector for each section of the stored video. A feature vector generated by the generation device 3 and stored in the accumulated video feature vector storage unit 41 is acquired. The section to be acquired at this time should be a section of several weeks before the designated program video is broadcasted in consideration of the fact that the program video spot video and announcement video start to be broadcast several weeks before the program broadcast. Is preferred.

  The similarity calculation unit 52 uses each feature vector acquired by the stored video feature vector acquisition unit 53 and uses a feature vector related to the section and a feature vector related to the program video for each section (including continuous sections) of the stored video. The similarity is calculated (similarity calculation means).

The similarity calculation unit 52 preferably calculates the similarity based on the equation (7). Here, sim (V _Pi , V _{Tx to Ty} ) is the similarity between the feature vector V _pi related to the program video Pi and the feature vectors V _{Tx to} Ty related to the section video Tx to Ty. I (t _k ) is a weighting coefficient that represents the importance of the index word t _k in the program video Pi. For example, I (t _k ) is set to a larger value for index words appearing as titles, subtitles, and performers of the program video Pi in the electronic program guide. In addition, I (t _k ) is set to a larger value for an index word having a higher number of appearances in the program video Pi and having a higher rarity. Note that these conditions for determining the value of I (t _k ) are appropriately determined, but appropriate conditions may be determined by machine learning from past data.

According to Equation (7), the similarity is represented by the cosine value of the angle formed by the feature vectors weighted by the weighting coefficient I (t _k ), so the absolute value of the feature vector affects the similarity. do not do. The longer the video is, the larger the absolute value of the feature vector is. However, by using Equation (7), it is possible to calculate a similarity that is not affected by the length of the video.
The similarity calculation unit 52 stores the calculated similarity for each section video in the similarity storage unit 54.

  Here, the order of feature vector acquisition by the stored video feature vector acquisition unit 53 will be described. In the first method, the accumulated video feature vector acquisition unit 53 first determines a starting point, and sequentially increases the section length from the starting point to acquire a feature vector related to the section video. When the section length reaches a predetermined maximum length, the maximum length is added to the start point to obtain a new start point. The subsequent processing is the same.

  In the second method, the accumulated video feature vector acquisition unit 53 first determines a start point, and acquires a feature vector related to a section video having a predetermined section length from the start point. Next, a new start point is acquired by adding a predetermined time length (predetermined time length> predetermined section length) to the start point. The subsequent processing is the same.

  When the second method is employed, it is preferable that the similarity calculation unit 52 performs similarity calculation every time the accumulated video feature vector acquisition unit 53 acquires a feature vector. And, when the similarity of a certain section length from a certain starting point is less than or equal to a predetermined value (when almost or not similar at all), or when the similarity is greater than or equal to a predetermined value (when very similar) It is preferable that the value added to the start point is not the predetermined time length but the predetermined section length.

FIG. 9 is a diagram illustrating an example of feature vectors acquired by the accumulated video feature vector acquisition unit 53 when the second method is employed. In the example of the figure, the predetermined time length is one section and the predetermined section length is four sections. In this figure, the similarity _between the feature vectors V _{T3 to T7} is equal to or less than a predetermined value. In this case, the stored video feature vector acquisition unit 53 performs V _T4 to _T8 , V _{T5 to T9} , and V _{T6 to T10,} and skip the acquisition of _{V T7~T11,} has acquired the _{V T8~T12} to the next _{V T3~T7.} By doing so, it is possible to reduce the time required for similarity calculation and speed up the processing.

  The summary video detection unit 55 detects the summary video of the program video from each section of the stored video and each continuous section based on the calculation result of the similarity calculation unit 52 stored in the similarity storage unit 54. (Summary video detection means). Specifically, a section video with the highest similarity may be detected as a summary video, or several section videos may be detected as a summary video in descending order of similarity.

  The summary video detection unit 55 uses the display means of the summary video detection device 5 to present the detected summary video to the user. When presenting a plurality of section videos, it is preferable to display them in order of similarity.

Finally, each process of the summary video detection device 5 described above will be described in more detail again with reference to the processing flow.
FIG. 10 is a flowchart showing the processing flow of the summary video detection device 5. As shown in the figure, the summary video detecting device 5 first acquires a feature vector V _pi related to the program video Pi (step S1). Next, the search range of the stored video is set (step S2), and the section length maximum value T _LMAX , the section length increment T _I , the start point T _S in the stored video, and the video section length _TL are initialized (step S3). -Step S6).

Next, the digest video detector 5 determines whether the image interval length _{T L} is equal to or less than the interval length maximum value _{T LMAX} (step S7), and if less, the image interval length _{T LMAX} to _{T S} Add and return to step S6 (step S8).

On the other hand, if it is determined in step S7 that the video segment length T _L is equal to or _smaller than the maximum segment length value T _LMAX , the summary video detection device 5 next performs the video segment length T _L from the start point T _S. It is determined whether or not the section is included in the search range initially set in step S2 (step S8). If the result is out of the search range, the process proceeds to step S14. If the search range, obtains a feature vector _{V Tx～Ty} relates to a video interval length _{T L} min interval (a Tx~Ty.) From the starting point _{T S.} Then, the similarity between the feature vector V _pi and the feature vectors V _{Tx to Ty} is calculated (step S12) and stored in the similarity storage unit 54 (step S13). Next, the digest video detector 5 adds _{T I} to _{T L,} the process returns to S7.

  In step S <b> 14, the summary video detection device 5 detects the summary video based on the similarity stored in the similarity storage unit 54. And it sorts and shows to a user in order with high similarity.

As described above, according to the summary video detection system 1, the feature vector reflects the semantic content of the video, so that the summary image detection performed by the summary video detection device 5 can be applied to a wide range of videos. The detection accuracy is high.
In addition, the feature vector generation device 3 can acquire character data from a closed caption or an electronic program guide as character data on which the feature vector is generated.

Also, the feature vector generation device 3 can create a feature vector for each video section. Further, since the feature vectors are generated for each section of the video, the feature vectors can be calculated hierarchically, so that the processing load for calculating the feature vectors can be reduced.
Furthermore, the summary video detection system 1 can handle video sections of various lengths as summary video candidates.

  Although the embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and the present invention can of course be implemented in various modes without departing from the scope of the present invention. It is.

  For example, in the above embodiment, the feature vector related to the stored video is calculated and stored in advance, but when the program video is designated by the program designation receiving unit 50, the feature vector related to the stored video is calculated. Also good.

Further, in the calculation of the feature vector for each continuous section, in the above embodiment, the feature vector for the lower layer section is obtained by addition. However, by subtracting the lower-order feature vector from the higher-order feature vector, It may be asking. For example, the feature vectors V _{T1 to T9} of _{T1 to T9} can be obtained as V _{T1 to T10} −V _{T9 to T10} .

Further, a program for realizing the functions of the feature vector generation device 3 and the summary video detection device 5 is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into a computer system and executed. As a result, the above-described processes may be performed.
Here, the “computer system” may include an OS and hardware such as peripheral devices. Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used.
The “computer-readable recording medium” means a flexible disk, a magneto-optical disk, a ROM, a writable nonvolatile memory such as a flash memory, a portable medium such as a CD-ROM, a hard disk built in a computer system, etc. This is a storage device.
Furthermore, the “computer-readable recording medium” includes a volatile memory (for example, DRAM (DRAM) in a computer system that becomes a server or a client when a program is transmitted through a network such as the Internet or a communication line such as a telephone line. Dynamic Random Access Memory)), etc., which hold programs for a certain period of time.
The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.
Further, the program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve each function mentioned above in combination with the program already recorded on the computer system, what is called a difference file (difference program) may be sufficient.

It is a figure which shows the system configuration | structure of the summary image | video detection system concerning embodiment of this invention. It is a figure which shows the internal structure of the program image | video feature vector production | generation part concerning embodiment of this invention. It is a figure which shows the internal structure of the extraction part concerning embodiment of this invention. It is a figure which shows the specific example of the memory content of the program image | video feature vector memory | storage part concerning embodiment of this invention. It is a figure which shows the internal structure of the accumulation | storage image | video feature vector production | generation part concerning embodiment of this invention. It is a figure which shows the example of the feature vector regarding the continuous area image | video concerning embodiment of this invention. It is a figure which shows the example of the feature vector regarding the continuous area image | video concerning embodiment of this invention. It is a figure which shows the specific example of the memory content of the accumulation image | video feature vector memory | storage part concerning embodiment of this invention. It is a figure which shows the example of the feature vector acquired by the stored image | video feature vector acquisition part concerning embodiment of this invention. It is a flowchart which shows the processing flow of the summary image | video detection apparatus concerning embodiment of this invention.

Explanation of symbols

1 summary video detection system,
2 video database,
3 feature vector generator,
4 Feature vector database,
5 summary video detection device,
20 Accumulated video storage unit,
30 Program video feature vector generator,
31 Rareness information calculator,
32. stored image feature vector generation unit,
40 Program video feature vector,
41 stored image feature vector storage unit,
50 Program designation reception part,
51 Program video feature vector acquisition unit,
52 similarity calculation unit,
53. Accumulated video feature vector acquisition unit,
54 similarity storage unit,
55 Summary video detector,
55 similarity storage unit,
300,302 extraction unit,
301, 303 feature vector generator,
3000 stream separator,
3002 Index word extraction unit,
3003 voice recognition unit,
3004 Index word extraction unit,
3005 Integration Department,
3030 section image feature vector generation unit,
3031 A continuous segment video feature vector generation unit.

Claims (6)

Extraction means for extracting one or a plurality of index words from at least one of character data or audio signals corresponding to video;
Feature quantity generating means for generating a feature quantity related to the video based on the number of appearances in the video for each index word;
The feature-value production | generation apparatus characterized by including. In the feature-value production | generation apparatus of Claim 1,
The feature amount generation means generates a feature amount related to the video based on the rarity of each index word determined based on the number of appearances in the stored video;
The feature-value production | generation apparatus characterized by this. Extraction means for extracting one or a plurality of index words from at least one of character data or audio signal corresponding to the section video for each section of the video;
Section video feature value generation means for generating a feature value related to the section video based on the number of appearances in the section video for each index word;
A continuous section video feature quantity generating means for generating a feature quantity regarding a continuous section video composed of a plurality of adjacent section videos based on a feature quantity regarding each section video generated by the section video feature quantity generation means;
The feature-value production | generation apparatus characterized by including. For each section of the stored video, stored video feature quantity acquisition means for acquiring the feature quantity generated by the feature quantity generation device according to claim 1;
A program video feature amount acquisition means for acquiring a feature amount generated by the feature amount generation apparatus according to claim 1 for a program video;
For each section of the stored video, similarity calculation means for calculating the similarity between the feature quantity related to the section and the feature quantity related to the program video;
Summary video detection means for detecting a summary video of the program video from the sections of the stored video based on the calculation result of the similarity calculation means;
A summary video detection apparatus comprising: An accumulated video feature amount acquisition means for acquiring a feature amount generated by the feature amount generation device according to claim 3 for each of the sections of the stored video and each continuous section composed of a plurality of adjacent sections;
A program video feature amount acquisition means for acquiring a feature amount generated by the feature amount generation apparatus according to claim 1 for a program video;
For each of the sections and the continuous sections of the stored video, a similarity calculation unit that calculates a similarity between the feature quantity related to the section or the continuous section and the feature quantity related to the program video;
Summary video detection means for detecting a summary video of the program video from the sections and the continuous sections of the stored video based on the calculation result of the similarity calculation means;
A summary video detection apparatus comprising: Extracting means for extracting one or a plurality of index words from at least one of character data or audio signals corresponding to the video, and a feature quantity related to the video based on the number of appearances in the video for each index word Feature quantity generating means to generate,
As a program to make the computer function.

Images