JP5898117B2 - Video summarization apparatus, video summarization method, and video summarization program - Google Patents

Description

  The present invention relates to a video summarization device, a video summarization method, and a video summarization program for generating a summary video from a video.

  Video media that can be accessed by individuals via broadcast waves and the Internet are already enormous, and the scale has been increasing. For example, it has been reported that a certain video sharing site has a total video length of 72 hours uploaded per minute. Video is a media with a time axis, and in general, its contents cannot be understood without viewing. Therefore, the act of grasping the outline from the video of a scale that is difficult or impossible to watch or finding the desired information is obviously accompanied with great effort. Based on this problem awareness, many inventions of video summarization techniques have been made so far in order to compress video in a short time, and to make it possible to grasp an outline and discover information in a short time.

  Here, the video summarization technique refers to a technique for selecting a video section to be included in the summary from a group of video sections obtained from one or more videos. As typical video summarization methods, the techniques described in Patent Document 1 and Non-Patent Document 1 can be cited. In Patent Document 1, the weight of various features extracted from the video section is adjusted by the viewer, and the video section in which the feature exceeds a certain threshold is extracted to meet the viewer's preference. A technique for generating a summary video is disclosed. Also, Non-Patent Document 1 discloses a technique for clustering news video sections that are similar in terms of both the appearance and meaning of the video section, focusing on the fact that the same topic is likely to be handled in news that is close to the broadcast date and time. Has been.

JP 2012-44390 A

W.-T. Chu, C.-C. Huang and W.-F. Cheng: News Story Clustering from Both What and How Aspects: Using Bag of Word Model and Affinity Propagation, in Proc.AIEM-Pro, pp.7 -12, 2011

By the way, in the recent situation where there is a video of a scale that is difficult to view, it is desirable that the video summarization technique satisfies the following items (1) to (3).
(1) There is no overlap in the selected video section.
(2) The size of the data necessary for the summary video processing that needs to be always stored in a storage device such as a hard disk is small.
(3) A summary video can be generated at high speed from data stored in a storage device.

  First, regarding (1), since the summary video needs to include as much information as possible in a short time, it is desirable that no overlap occurs between the selected video sections. In addition, it is desirable that the overlap does not occur not only in individual videos but also in video sections obtained from all videos to be summarized.

  Next, regarding (2), in order to generate a summary video from an enormous scale image, it is generally necessary to analyze an enormous amount of data. In particular, data that needs to be always stored in a storage device such as a hard disk affects the capacity of the storage device necessary for data storage and also affects the operation cost. In order to realize the summary process at a low cost, it is desirable that the data capacity that needs to be always stored in the storage device is small.

  Finally, with regard to (3), the summarization process based on the data stored in the storage device is performed again with the addition of video data, or is performed on demand, for example, to reflect personal preferences and interests. It is assumed that Therefore, it is desirable that the processing for outputting the summary video from the data stored in the storage device is particularly fast.

  However, in Patent Document 1, the selection of video sections included in the summary is managed only by the threshold value of the feature, so there is a possibility that the extracted video sections are only similar, and the viewpoint of (1) is There is a problem that it is not considered at all.

  On the other hand, in Non-Patent Document 1, a method of collecting similar video sections by clustering is taken, and the problem of overlapping videos is solved.

  However, in order to perform clustering used in Non-Patent Document 1, it is necessary to store the similarity between all video sections in a storage device. In general, it is said that a 1-hour video is composed of about 600 scenes with different appearances. However, if we consider summarizing a 100-hour video, the number of scenes is approximately 60000, and the similarity between all video segments. The number of elements is about 4 billion, and there is a problem that a huge capacity storage device is required to store data. In addition, various clustering methods such as Affinity Propagation used in Non-Patent Document 1 have a calculation cost proportional to the linear order of the data size or higher. In particular, when the number of videos to be summarized is enormous, there is a problem that the calculation cost for generating the summary video from the data stored in the storage device is enormous.

  The present invention has been made in view of such circumstances, and provides a video summarization apparatus, a video summarization method, and a video summarization program capable of generating a summary video while suppressing an increase in hardware resources and calculation cost. With the goal.

  The present invention is a video summarization device for generating a summary video from a video, and based on the similarity between the video segment constituting the video and another video segment, the video segment is defined as one node and a similarity between the nodes. A graph constructing unit that represents the video by a graph having a degree as an edge, a query extracting unit that extracts a query node that is a starting point of graph division from a group of nodes constituting the graph, and the query node is started As a point, a graph division processing unit that divides the graph by generating a subgraph composed of nodes in the vicinity of the query node, and a cluster generation unit that generates a cluster in which those having a high degree of overlap of the subgraphs are combined, A representative node extracting unit for extracting a representative node from the nodes constituting the cluster, and a video section corresponding to the representative node. And use, characterized in that it comprises a video summary output unit for outputting the video summary generation to.

  The present invention is characterized in that the graph division processing unit uses the subgraph as a subgraph that satisfies a local evaluation index representing the clusteriness.

  The present invention is characterized in that the representative node extracting unit extracts the node having many similar elements in the cluster as the representative node.

  In the present invention, the summary video output unit ranks according to the size of the cluster in which the representative node is included, and sequentially combines video sections corresponding to the nodes with high ranking until the specified summary video length is reached. Thus, the summary video is generated.

  The present invention is a video summarization method performed by a video summarization device for generating a summary video from a video, and based on the similarity between the video segment constituting the video and another video segment, the video segment is a node, A graph construction step of expressing the video by a graph having the similarity between the nodes as an edge, a query extraction step of extracting a query node as a starting point of graph division from a group of nodes constituting the graph, Starting with the query node as a starting point, generating a subgraph composed of nodes in the vicinity of the query node, and generating a cluster that combines the graphs with a high degree of redundancy of the subgraphs, and a graph division processing step for dividing the graph A cluster generation step, and a representative node extraction for extracting a representative node from the nodes constituting the cluster. A method, characterized by having an abstract image output step of outputting the video summary generated by using the image section corresponding to the representative node.

  The present invention is a video summarization program for causing a computer to function as the video summarization apparatus.

  According to the present invention, it is possible to generate a summary video at a high speed while suppressing an increase in hardware resources and calculation cost.

It is a block diagram which shows the structure of one Embodiment of this invention. It is a flowchart which shows the processing operation of the video summarizing apparatus 1 shown in FIG. It is a figure which shows the example of construction | assembly of the graph in the case of accompanying the division | segmentation of each image | video. It is a figure which shows the construction example of the graph when not dividing | segmenting each image | video. It is a figure which shows an example of a graph division | segmentation process. It is a figure which shows an example of cluster production | generation.

  Hereinafter, an image summarizing apparatus according to an embodiment of the present invention will be described with reference to the drawings. In the following description, an image composed of one or more images is referred to as an image group. FIG. 1 is a block diagram showing the configuration of the embodiment. In this figure, reference numeral 1 denotes a video summarizing device constituted by a computer device. Reference numeral 11 denotes a video input unit for inputting a video group. Reference numeral 12 denotes a storage unit that stores a video group input by the video input unit. The stored data may include context information called metadata in addition to the video itself. As metadata, for example, there are video titles and summary sentences, speech content obtained by voice recognition and closed captioning, and time data.

  Reference numeral 13 denotes a graph construction unit that constructs a graph based on the video group stored in the storage unit 12. Reference numeral 14 denotes a query extraction unit that extracts a query node used in the graph partitioning process from the neighborhood undirected graph obtained by the graph construction unit 13. Reference numeral 15 denotes a graph division processing unit that performs graph division processing using each query obtained in the query extraction unit 14 as a starting point. Reference numeral 16 denotes a cluster generation unit that generates a cluster based on the graph division result. Reference numeral 17 denotes a representative node extraction unit that extracts a representative node from each cluster obtained by the cluster generation unit 16. Reference numeral 18 denotes a summary video output unit that outputs a summary video by combining video sections corresponding to the representative nodes obtained by the representative node extraction unit 17.

  Next, the processing operation of the video summarizing apparatus 1 shown in FIG. 1 will be described with reference to FIG. FIG. 2 is a flowchart showing the processing operation of the video summarizing apparatus 1 shown in FIG. First, the video input unit 11 inputs a video group from the outside and stores it in the storage unit 12 (step S1). When the video group is stored in the storage unit 12, the graph construction unit 13 constructs a graph expressing the video section as a node and the similarity between the nodes as an edge from the video stored in the storage unit 12 (step S2). ). The video section corresponding to each node defined here is a candidate video section output as a summary video. The video section may be defined as one video, or may be defined by dividing each video when the video length of each video is long, for example. In the following, as one example, a processing operation for constructing a graph by dividing each video into sections and considering the similarity of the obtained video sections will be described.

  First, the graph construction unit 13 divides each video stored in the storage unit 12 into video sections. As a dividing method, it may be divided at regular intervals. Reference 1 “Y. Tonomura, A. Akutsu, Y. Taniguchi and G. Suzuki: Structured Video Computing, IEEE Multimedia, pp. 34-43, 1994. The information may be divided at cut points that are points at which the appearance switches discontinuously. When metadata with time information such as closed caption is also input by the video input unit 11, the video may be divided according to the time.

  Next, a feature amount is extracted from each obtained video section. The features of the video section are (i) a moving image feature obtained by analyzing the moving image, (ii) an audio feature obtained by analyzing the audio, and (iii) metadata added by the video input unit. In this case, it is composed of at least one of metadata features, and these feature amounts are represented by a histogram or a vector. Any feature can be used.

In the case of moving image features, for example, a color histogram obtained by counting the values of each axis in the L ^* a ^* b ^* color space, or Reference 2 “A. Oliva and A. Torralba: Building the Gist of a Scene: The Role of Global Image Features in Recognition, Progress in Brain Research, 155, pp. 23-36, 2006. ”can be used vectors obtained using GIST descriptors representing landscape features.

  In the case of voice features, for example, Mel-Frequency Cepstral Coefficients (MFCC) representing features related to the prosody of voice can be used. Further, in the case of metadata features, for example, one video section is regarded as a document, metadata attached to the video section is regarded as a word, a TF-IDF value for each word is calculated, and a document vector having the value as an element is used. Can do.

  Note that the features extracted here are not necessary after the graph is obtained, and need not necessarily be stored in the storage unit 12.

  Next, using the obtained video segment group and the characteristics of each video segment, a neighborhood undirected graph is constructed that expresses one video segment as a node and the similarity between nodes as an edge. FIG. 3 is a diagram illustrating an example of construction of a graph in the case where each video is divided. As shown in FIG. 3, the video A (first video) is converted into video sections a1, a2, a3,. . . And video B (second video) is divided into video segments b1, b2, b3,. . . Divide into The same applies to the third video and the fourth video. Then, a neighborhood undirected graph is constructed in which each video section is expressed as a node (indicated by a circle in FIG. 3), and a similarity between nodes is expressed as an edge (indicated by a straight line in FIG. 3). In FIG. 3, a1 to a3 and b1 to b3 are video section candidates included in the summary.

  The graph can be constructed in an arbitrary form, for example, a k-neighbor graph in which edges are extended only to k neighboring nodes as viewed from each node, or edges are provided only to nodes existing within a distance ε as viewed from each node. A stretched ε-graph may be constructed. k is a parameter that takes a positive integer, and ε is a parameter that takes a positive real number. Here, the similarity or distance between nodes is calculated between the features obtained by the video segment feature extraction unit. Any similarity scale or distance scale can be used. For example, a cosine similarity or Jaccard coefficient can be used for the similarity scale, and a known scale such as Euclidean distance or chi-square distance can be used for the distance scale.

  The simplest method for constructing a neighborhood graph is to calculate a similarity or distance between all nodes and select a neighborhood node that meets the condition for each node. However, the calculation order of this process is high in cost because it is proportional to the square of the number of data. Since there are many known techniques for constructing a neighborhood graph at high speed, these may be used. For example, Reference 3 “W. Dong, M. Charikar and K. Li: Efficient K-Nearest Neighbor Graph Construction for Generic Similarity Measures, in Proc. WWW, pp. 577-586, 2011”, Reference 4 “W. Liu, J. He and SF Chang: Large Graph Construction for Scalable Semi-Supervised Learning, in Proc. ICML, 2010, Reference 5 `` J. Chen, H. Fang and Y. Saad: Fast Approximate kNN Graph Construction for High Techniques such as “Dimensional Data via Recursive Lanczos Bisection, Journal of Machine Learning Research, 10, pp. 1989-2012, 2009” can be applied.

  The graph construction unit 13 stores the information of the graph constructed here in the storage unit 12. Since the graph structure itself includes only node and edge information, the data size stored in the storage unit 12 can be reduced. An arbitrary database can be used for storing the graph structure. For example, if a graph database having a data structure that does not require an index reference for access to an adjacent node is used, high-speed access to data is possible.

  In the above description, the processing operation of the graph construction unit 13 when video division is involved has been described, but the processing operation when video division is not involved may be performed as shown in FIG. FIG. 4 is a diagram illustrating a construction example of a graph in a case where each video is not divided. As shown in FIG. 4, a neighborhood undirected graph expressing each of the images 1 to N as a node (indicated by a circle in FIG. 4) and a similarity between the nodes as an edge (indicated by a straight line in FIG. 4). Build up. In FIG. 4, 1 to N are video segment candidates included in the summary. When the processing operation shown in FIG. 3 is compared with the processing operation shown in FIG. 4, the processing operation shown in FIG. 4 is different in that there is no processing for dividing the video section. That is, the processing shown in FIG. 4 is the same as the case where there is only one video section in one video in the processing operation shown in FIG.

  Next, the query extraction unit 14 extracts query nodes used in the graph partitioning process from the neighborhood undirected graph obtained by the graph construction unit 13 (step S3). Any method can be used to extract query nodes. For example, it is possible to use (i) a method of extracting a node having a high degree (the number of edges attached to a node) as a query, and (ii) a method of extracting an arbitrary number of nodes as a query at random.

  In addition, when the user's interests and preferences are known, a query may be extracted based on the information. For example, (iii) When video sections in which the user is interested are given, a method using them as a query, (iv) When metadata is given to a node, the metadata in which the user is interested A method of extracting a given video section as a query can be used.

  Next, the graph division processing unit 15 performs graph division processing using each query obtained in the query extraction unit 14 as a starting point (step S4). FIG. 5 is a diagram illustrating an example of the graph division process. As shown in FIG. 5, graph division is performed while tracing an edge in the vicinity of the start point with the query node as the start point. The graph partitioning process here refers to a process of outputting the most “cluster-like” subgraph among subgraphs composed of one or more node groups near the query node. There are two types of evaluation indexes that represent “cluster-likeness”: a global evaluation index based on information of the entire graph and a local evaluation index based only on local information of the graph. Graph segmentation processing is performed using the local evaluation index.

  By using a local evaluation index, the graph division processing starting from each query does not depend on the data size of the entire graph, and thus high-speed processing is possible. Further, the graph partitioning process is completely independent for each query, and can be easily parallelized. This is also a factor contributing to the speeding up of the process. Furthermore, since the evaluation index strongly reflects the locality of the graph, the obtained subgraph also reflects the locality strongly. By strongly reflecting the locality, a subgraph with a clear similarity between elements is generated, and as a result, an effective video section is included in the summary video by grasping the outline of the video group.

Any local evaluation index for clusteriness can be used. For example, an index called density shown in equation (1) or an index called conductance shown in equation (2) may be used.

  In equations (1) and (2), vol (S) is the order sum of nodes included in the subgraph S, δ (S) is the number of edges connecting the subgraph S and the outside, and | S | is the number of elements in the subgraph. Represents. Density indicates that the larger the value, the smaller the value, and the smaller the value, the more likely the cluster.

  With respect to these two indexes, it is more preferable to adopt the conductance as the index because the result using the conductance as the evaluation index can be output better than the density.

  Several graph partitioning algorithms using conductance as an index have been proposed, and any of these known techniques may be used here. For example, Reference 6 “DA Spielman and SH Ten: A Local Clustering Algorithm for Massive Graphs and its Application to Nearly-Linear Time Graph Partitioning, CoRR, abs / 0809.3232, 2008” is obtained from the order of each node constituting the graph. A method is disclosed in which a transition probability from a query node to a neighboring node is calculated based on a transition probability matrix to be output, and a subgraph composed of a node group having a high transition probability is output with a minimum conductance.

  Reference 7 “R. Andersen and F. Chung: Detecting Sharp Drops in PageRank and a Simplified Local Partitioning Algorithm, in Proc. TAMC, pp. 1-12, 2008” is a page rank matrix instead of a transition probability matrix. Is used. A method is disclosed in which a subgraph composed of a group of nodes having high importance for a query node is output with a minimum conductance.

  Reference 8 “R. Andersen and Y. Peres: Finding Sparse Cuts Locally Using Evolving Sets, in Proc. STOC, pp.235-244, 2009” simulates the state transition probabilities near the starting point by sampling. A method of outputting a subgraph that minimizes conductance without using the transition probability matrix of the entire graph is disclosed.

  Among the graph division processes disclosed in Reference Documents 6 to 8, the process of Reference Document 8 is the fastest. On the other hand, the technique disclosed in Reference 8 has a feature that subgraphs obtained for each trial do not always match because a value sampled stochastically is used.

  In any of the techniques disclosed in Reference Documents 6 to 8, the obtained subgraph is updated so that the size (the number of nodes included in the subgraph) is increased by iterative processing. As a method of ending the iterative process, for example, an arbitrary method such as setting a conductance threshold value or setting an upper limit of the number of repetition processes can be used.

  Since the graph division processing in the graph division processing unit 15 is completely independent for each query, there is a possibility that the obtained subgraph groups partially or completely overlap. The cluster generation unit 16 combines the subgraphs for those with high subgraph overlap, and generates a new subgraph. Through this process, the cluster generation unit 16 generates a cluster group from the subgraph group obtained by the graph division processing unit 15 (step S5). FIG. 6 is a diagram illustrating an example of cluster generation. As shown in FIG. 6, in the subgraph 1 and the subgraph 2, the overlapping portion is large (there are many overlapping nodes), so the subgraph 1 and the subgraph 2 are combined to form a cluster.

The degree of overlap of subgraphs can be obtained by an arbitrary method. For example, for the subgraphs S ₁ and S ₂ , when the evaluation formula (3) is satisfied, a new subgraph may be generated as a union of both. In equation (3), ρ is a real parameter.

  Next, the representative node extraction unit 17 extracts a representative node from each cluster obtained by the cluster generation unit 16 (step S6). Here, a representative node is extracted that contains the most similar elements in the cluster. Each element in the cluster is basically composed of similar elements. Among them, a node having many similar elements can be said to be a node that plays a central role in the cluster. By extracting a node that can be said to be the cluster center as a representative, the summary video obtained as a result of the subsequent processing contributes to including a video section that is more effective for grasping the video group.

  Any method can be used to extract a node including the most similar elements in the cluster. For example, using the method of extracting the node with the highest degree in the cluster as a representative node, or using the similarity between nodes calculated by the graph construction unit, the node with the maximum sum of similarities in the cluster is extracted as the representative node What is necessary is just to use the method to do.

  Next, the summary video output unit 18 combines the video sections corresponding to the representative nodes obtained by the representative node extraction unit 17 to output a summary video (step S7). Here, for example, when the video length of the summary video to be output has been specified in advance, the video image corresponding to the representative node cannot be included in the summary video. Therefore, video segments extracted from clusters having more similar nodes are preferentially included in the summary video. For this purpose, the number of elements in the cluster is counted and the cluster group is ranked. By giving priority to video segments with a large number of elements in the cluster, video segments representing the video group are preferentially included in the summary video, and the resulting summary video is an effective video for grasping the outline of the video group. Contributes to including intervals.

  An arbitrary method can be used for the order of the video sections. The simplest method is to arrange the ranking results of clusters calculated for video segment selection. Alternatively, when metadata indicating the context of the video such as broadcast date and time is given in advance, the video sections may be rearranged based on the information.

  As described above, in order to realize the summarization processing satisfying all of (1) to (3) described above, a large-scale video group is represented by a graph having video segments as nodes and feature similarity between nodes as edges. The graph structure is stored in the storage unit. And compared with the existing method which needs to hold | maintain all the combinations of the similarity between elements as data, in this invention, the whole image of data can be comprised only with the information of the edge which connects between a node and a neighboring node. Therefore, the problem of the data size necessary for the summarization process (2) can be solved.

  Also, the summary video is output by clustering the graphs stored in the storage unit and extracting representative nodes from the obtained clusters. As a result, the redundancy problem (1) can be solved. Further, the clustering processing used in the present invention performs clustering based on local graph division processing based on an evaluation index based on only local information of a graph. Compared with the prior art in which the evaluation index for performing the clustering process depends on the data size, in the present invention, the calculation cost of the clustering process does not depend on the number of data, and the parallelism of the clustering process increases. The processing speed problem can be solved.

  In particular, the video group is represented by a graph in which the video section is a node and the feature similarity between the nodes is an edge, thereby suppressing the size of data that should always be stored in the storage unit during the summary video generation process. it can. Further, since the processing for each query in the graph division processing unit does not depend on the size of the graph, that is, the number of data, the processing can be executed at high speed. In addition, since the graph partitioning process is completely independent for each query, it can be easily parallelized, and the processing speed can be increased.

  In addition, since the locality of the graph is strongly reflected in the clusters obtained from the graph division processing unit and the cluster generation unit, it is possible to generate a cluster composed of elements whose similarity is clear more intuitively. This has the effect of increasing the accuracy of the generated summary video, and it is possible to output a high-quality summary video with little overlap of video sections.

  In addition, by ranking the clusters according to the number of elements included in the cluster, and by setting the video section corresponding to the node with the highest similarity in the cluster from the upper cluster as the video section to be included in the summary video, there is no overlap of video sections. It is possible to output a high-quality summary video with little quality.

  Note that a video summarization process is performed by recording a program for realizing the function of the processing unit in FIG. 1 on a computer-readable recording medium, causing the computer system to read and execute the program recorded on the recording medium. You may go. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer system” includes a WWW system having a homepage providing environment (or display environment). The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Further, the “computer-readable recording medium” refers to a volatile memory (RAM) in a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In addition, those holding programs for a certain period of time are also included.

  The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, what is called a difference file (difference program) may be sufficient.

  As mentioned above, although embodiment of this invention has been described with reference to drawings, the said embodiment is only the illustration of this invention, and it is clear that this invention is not limited to the said embodiment. is there. Accordingly, additions, omissions, substitutions, and other changes of the components may be made without departing from the technical idea and scope of the present invention.

  It can be applied to applications where it is indispensable to generate summary video while suppressing increase of hardware resources and calculation cost.

  DESCRIPTION OF SYMBOLS 1 ... Image | video summarization apparatus, 11 ... Image | video input part, 12 ... Memory | storage part, 13 ... Graph construction part, 14 ... Query extraction part, 15 ... Graph division | segmentation process part, 16. ..Cluster generation unit, 17 ... representative node extraction unit, 18 ... summary video output unit

Claims (6)

A video summary device for generating a summary video from a video,
Based on the similarity between the video section constituting the video and another video section, a graph construction unit that represents the video by a graph having the video section as one node and the similarity between the nodes as an edge;
A query extraction unit that extracts a query node that is a starting point of graph division from a group of nodes constituting the graph;
A graph division processing unit that divides the graph by generating a subgraph composed of nodes near the query node, starting from the query node;
A cluster generation unit for generating a cluster in which the subgraphs having a high degree of overlap are combined;
A representative node extracting unit for extracting a representative node from the nodes constituting the cluster;
A summary video output unit that generates and outputs the summary video using a video section corresponding to the representative node ;
The representative node extraction unit includes:
The video summarization apparatus, wherein the node having many similar elements in the cluster is extracted as the representative node . A video summary device for generating a summary video from a video,
Based on the similarity between the video section constituting the video and another video section, a graph construction unit that represents the video by a graph having the video section as one node and the similarity between the nodes as an edge;
A query extraction unit that extracts a query node that is a starting point of graph division from a group of nodes constituting the graph;
A graph division processing unit that divides the graph by generating a subgraph composed of nodes near the query node, starting from the query node;
A cluster generation unit for generating a cluster in which the subgraphs having a high degree of overlap are combined;
A representative node extracting unit for extracting a representative node from the nodes constituting the cluster;
A summary video output unit that generates and outputs the summary video using a video section corresponding to the representative node ;
The summary video output unit includes:
The summary video is generated by ranking by the size of the cluster in which the representative node is included, and sequentially combining video sections corresponding to the nodes with a high ranking until the specified summary video length is reached. A video summary device. The graph division processing unit
Among the sub-graph, the video summarizing apparatus for an according to satisfy the local evaluation index representing the cluster likelihood to claim 1 or 2, characterized in that said subgraph. A video summarization method performed by a video summarization device that generates a summary video from a video,
Based on the similarity between the video section constituting the video and another video section, a graph construction step for expressing the video by a graph having the video section as one node and the similarity between the nodes as an edge;
A query extraction step of extracting a query node that is a starting point of graph division from a group of nodes constituting the graph;
A graph division processing step of dividing the graph by generating a subgraph composed of nodes near the query node, starting from the query node;
A cluster generation step of generating a cluster in which the subgraphs having a high degree of overlap are combined;
A representative node extracting step of extracting a representative node from the nodes constituting the cluster;
Using an image section corresponding to the representative node possess an abstract image output step for generating and outputting the video summary,
In the representative node extraction step,
The video summarizing method, wherein the node having many similar elements in the cluster is extracted as the representative node . A video summarization method performed by a video summarization device that generates a summary video from a video,
Based on the similarity between the video section constituting the video and another video section, a graph construction step for expressing the video by a graph having the video section as one node and the similarity between the nodes as an edge;
A query extraction step of extracting a query node that is a starting point of graph division from a group of nodes constituting the graph;
A graph division processing step of dividing the graph by generating a subgraph composed of nodes near the query node, starting from the query node;
A cluster generation step of generating a cluster in which the subgraphs having a high degree of overlap are combined;
A representative node extracting step of extracting a representative node from the nodes constituting the cluster;
Using an image section corresponding to the representative node possess an abstract image output step for generating and outputting the video summary,
In the summary video output step,
The summary video is generated by ranking by the size of the cluster in which the representative node is included, and sequentially combining video sections corresponding to the nodes with a high ranking until the specified summary video length is reached. Video summarization method. A video summarization program for causing a computer to function as the video summarization device according to any one of claims 1 to 3 .

Images