JP4949307B2 - Moving image scene dividing apparatus and moving image scene dividing method - Google Patents

Description

  The present invention relates to a moving image scene dividing device and a moving image scene dividing method.

  In recent years, video sharing services have attracted attention. In this moving image sharing service, a general user posts (uploads) moving image data shot using a digital camera, a video camera, or a camera built in a mobile phone to the moving image sharing server via the Internet. Thus, the moving image data can be disclosed to other users. The contributor of the moving image data shares the moving image data with the category information indicating the genre related to the moving image data, the title and comment text indicating the content, and the keyword that can be freely assigned by the contributor called “tag” together with the moving image data. Register with the server. On the other hand, the viewer can freely view various registered moving image data at any time. At this time, the viewer performs text search for the category information, title, comment, and tag registered by the contributor, uses a ranking list such as the number of views and popularity, or is extracted from moving image data 1 Search the moving image data you want to view by looking at the still images.

  However, since there is a large amount of moving image data registered in the moving image sharing server, it is not easy to efficiently search for desired moving image data. The viewer must guess the content based on a small amount of text information attached to each moving image data and one still image, and generally does not actually reproduce the moving image data. And, it cannot be confirmed whether or not it is moving image data having a desired content. In general, the playback time of moving image data posted to a moving image sharing server varies from several seconds to 10 minutes or more. For example, for moving image data having a playback time of 1 minute, the content can be changed in several seconds. When it becomes possible to understand, it is thought that the search efficiency will be greatly improved.

  Therefore, there is a high need for a technique for summarizing moving image data so that the contents of moving image data can be understood. Selecting one image that most symbolically represents the content of the original moving image data as one still image presented by an existing moving image sharing service is one of moving image summarization techniques. However, it is very difficult to select an optimal one of moving image data having 15 or 30 still images per second. In a general moving image sharing service, a frame for moving image data is used. In most cases, a still image at a certain position is selected for all moving image data, such as a certain number from the beginning (for example, the 30th). For this reason, in order to understand the content of moving image data, it is desirable to include a plurality of still images in the summary content of moving image data. However, when extracting a plurality of still images to be used for summary content from moving image data, how to determine the extraction range is a problem.

  Here, in most cases, moving image data posted by a general user to a moving image sharing server is not particularly processed as it is captured by a camera. The moving image data that has not been particularly processed since such shooting is hereinafter referred to as “consumer-generated moving image”. A consumer-generated moving image has a feature that editing such as clear scene switching is not performed. On the other hand, a video such as a TV program shot and produced by a professional using broadcasting equipment is referred to as a “professional generated moving image”. These are beautifully edited, with no camera shake, and carefully edited to include subtitles and cuts.

  In the prior art 1 described in Patent Document 1, a video with subtitles, which is one of professionally generated moving images, is targeted as moving image data, and a portion where subtitles are displayed corresponds to that portion as a semantically important scene. A summary content is generated in which still images are cut out and a list of the cut out still images can be displayed.

In the prior art 2 described in Non-Patent Document 1, only a color arrangement feature amount of a still image is used for consumer-generated moving images, without considering the temporal order of the still images constituting the moving image data. Then, still images are clustered, a group of still images in each classified cluster is extracted as one shot, and the extracted shot group is used as image data for similarity determination of the original moving image data. Similarity is determined.
Japanese Patent Laid-Open No. 7-192003 Keiichiro Hoashi, Toshiaki Kamiko, Kazunori Matsumoto, Fumiaki Sugaya, “Proposal of CGM video content retrieval method using frame clustering”, The Institute of Electronics, Information and Communication Engineers, IEICE Technical Report, Vol.107, No .281, pp. 87-92, October 18, 2007

However, the above-described conventional technique 1 targets a professional-generated moving image that is a video with captions, and cannot be applied to a consumer-generated moving image in which no caption is inserted.
In the related art 2, the similarity between moving image data can be determined for a consumer-generated moving image, but it is not considered to be applied to summary content. In addition, since each cluster classified by the conventional technique 2 is clustered based only on the color arrangement feature amount without considering time information, the original moving image data is temporally included in each cluster. Discrete still images are mixed. For this reason, each cluster is insufficient to be regarded as one scene that matches the content of the original moving image data. Further, it is difficult to determine which still image should be extracted from each cluster for generating summary content.

  The present invention has been made in consideration of such circumstances, and its purpose is to perform clustering based on video feature amounts such as color arrangement feature amounts extracted from still images and clustering based on time series of still images. A video scene segmentation device capable of performing still image clustering according to the content of the original video data even if it is used together even with consumer-generated video that has not been clearly edited such as scene switching It is to provide a moving image scene dividing method.

  In order to solve the above-described problem, a moving image scene dividing apparatus according to the present invention includes a still image extraction unit that extracts a still image from video data, and a still image feature amount extraction unit that extracts a video feature amount from the still image. A feature for classifying the still image into one of a plurality of upper clusters based on the scene feature number determining means for determining the number of scene divisions of the video data and the video feature amount of the still image extracted from the video data. Quantity clustering means, time series information clustering means for dividing the upper cluster into lower clusters according to the time series of still images extracted from the video data, and the lower order according to the time series of still images extracted from the video data Cluster integrating means for integrating clusters and generating clusters corresponding to the number of scene divisions.

  In the moving image scene segmentation device according to the present invention, the feature amount clustering means classifies the still image into a cluster based on the video feature amount of the still image extracted from the video data; And a means for generating a representative vector representing a feature, a means for calculating a similarity between clusters based on the representative vector, and a means for integrating clusters having a high similarity.

  In the moving image scene dividing device according to the present invention, the time-series information clustering means includes, in the same lower cluster, still images that are adjacent to each other in the time-series order with respect to the still image group in the upper cluster. Features.

  In the moving image scene segmentation device according to the present invention, the cluster integration unit generates a representative vector representing the characteristics of the lower cluster, a unit for calculating the similarity between the lower clusters based on the representative vector, And means for integrating lower clusters having a high degree of similarity.

  In the moving image scene segmentation device according to the present invention, the cluster integration unit, when there is a lower cluster sandwiched between two lower clusters originally belonging to the same upper cluster, It is characterized by being integrated into.

  In the moving image scene segmentation device according to the present invention, the cluster integration unit is configured such that, for the total number of all the still images in the three adjacent lower clusters, the upper cluster that is different among the three adjacent lower clusters. The three subordinate clusters are integrated only when the ratio of the number of still images in the subordinate clusters belonging to is less than or equal to a certain value.

  In the moving image scene dividing device according to the present invention, the similarity is obtained based on a distance between the representative vectors, and it is determined that the closer the distance is, the higher the similarity is.

  In the moving image scene dividing apparatus according to the present invention, the number of scene divisions is defined as a ratio to the total number of still images in the video data.

  The moving image scene dividing apparatus according to the present invention is characterized in that means for changing the number of scene divisions according to the video feature amount of a still image is provided when the lower clusters are integrated.

  In the moving image scene dividing apparatus according to the present invention, a change amount of the video feature amount between time-series continuous still images is obtained, and the number of scene divisions is increased when the change amount is larger than a certain amount. And

  In the moving image scene dividing device according to the present invention, the amount of change in the video feature amount between time-series continuous still images is obtained, and the number of scene divisions is reduced when the amount of change is smaller than a certain amount. And

  In the moving image scene segmentation device according to the present invention, the video feature amount is a color arrangement feature amount.

  A moving image scene dividing method according to the present invention is a moving image scene dividing method for dividing video data into a plurality of scenes, a still image extracting step for extracting a still image from the video data, and a video feature amount from the still image. A still image feature extraction step to extract, a scene division number determination step to determine the number of scene divisions of the video data, and a plurality of still images based on video feature values of still images extracted from the video data. A feature amount clustering step for classifying the upper cluster into any one of the upper clusters, a time series information clustering step for dividing the upper cluster into lower clusters according to the time series of the still images extracted from the video data, and extracted from the video data The lower clusters are integrated according to the time series of the still images, and clusters corresponding to the number of scene divisions are generated. Characterized in that it comprises a cluster integration step.

  According to the present invention, a combination of clustering based on video feature quantities such as color arrangement feature quantities extracted from still images and clustering based on time series of still images enables clear scene switching and other editing. Even if it is not a consumer-generated moving image, the effect that still image clustering suitable for the content of the original moving image data can be performed is obtained.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram showing a schematic configuration of a moving image scene dividing apparatus 1 according to an embodiment of the present invention. In FIG. 1, a video input unit 3 reads video data (hereinafter referred to as original video data) that is a target of a video scene division process from a video database (video DB) 2. The video analysis unit 4 extracts and analyzes video feature amounts representing video features from the original moving image data. The moving image dividing unit 5 divides the original moving image data into scenes based on the analysis result by the video analyzing unit 4.

  The moving image DB 2 may be provided in the moving image scene dividing device 1 or may be provided in an external storage device. For example, the moving image scene dividing device 1 may be configured to download video data to be subjected to moving image scene dividing processing from an external moving image database server via a communication line.

  Next, the operation of the moving image scene dividing device 1 according to this embodiment will be described.

  In this embodiment, particularly when a consumer-generated moving image is a target of moving image scene division processing, a more remarkable effect is obtained. Consumer-generated video is video data that has not been processed in particular since the time of shooting. Like professional-generated video such as TV programs, scene switching such as combining multiple video images and captions (character strings) ) And logos are not inserted at all. That is, the consumer-generated moving image corresponds to one scene used in video analysis processing or the like when generating the professional-generated moving image. In addition, consumer-generated moving images are not high-performance video cameras, but are often taken by shooting devices such as mobile phone cameras and digital cameras, resulting in low resolution, poor image quality, and many camera shakes. There is. In addition, when a scene is switched or a caption is inserted as in a professionally generated moving image such as a TV program, a method of determining each scene or a portion where a caption is displayed can be used. In consumer-generated moving images, there is little additional information as such a base point.

In the present embodiment, it is assumed that summary content of consumer-generated moving images is generated based on the following criteria.
A still image extracted from the original moving image data (hereinafter referred to as a summary still image) is used.
・ Summary content viewing time is short and data size is small. In other words, the number of summary still images should be small.
• Extract summary still images from locations that are temporally discrete. That is, the summary content uses only a still image extracted from a scene obtained by dividing the original moving image data, not a continuous still image extracted from a certain section of the original moving image data.
The number of summary still images or the number of scene divisions is changed according to the contents of the original moving image data.
・ In order to make it possible to understand the contents of the original moving image data to some extent just by viewing the summary content, the summary still image includes a portion that clearly represents the content of the original moving image data and a noteworthy portion. To do.

  In the present embodiment, the above criteria are taken into account when performing the moving image scene division process on the consumer generated moving image. Each of the divided scenes is a range for extracting a summary still image, and a summary still image is extracted from each scene.

  In this embodiment, still image groups are classified based on the frame clustering process disclosed in Non-Patent Document 1. In the frame clustering process, the temporal order of the candidate still image groups is excluded, and the still image groups are classified using only the color arrangement feature amount. Even in the case of extracting a summary still image, it is effective to extract a still image more suitable within the range after the extraction range is determined to some extent when the still image is extracted discretely. Therefore, also in the present embodiment, by using the color arrangement feature amount, it is possible to perform rough range division on a consumer-generated moving image that has not been originally edited and range-divided.

  Hereinafter, the moving image scene dividing process performed by the moving image scene dividing device 1 according to the present embodiment will be described in detail. FIG. 2 is a flowchart showing a procedure of moving image scene division processing according to the present embodiment.

  In FIG. 2, in step S <b> 1, the video input unit 3 reads out video data (original moving image data) to be subjected to moving image scene division processing from the moving image DB 2.

  In step S2, the video analysis unit 4 determines the number of scene divisions. In this embodiment, it is assumed that “the number of scene divisions = the number of still images for summarization”, and the number of frames for summation β is selected with respect to the total number of frames α of the original moving image data (where β ≧ 1). .

Here, when the frame rate (frame / second: fps) of the original moving image data is θ (for example, 30 fps), the reproduction time of the original moving image data is α / θ [seconds]. Further, when each summary still image is displayed at ω [seconds] (for example, 2 seconds) during summary content playback, the playback time of β summary still images is β × ω [seconds]. Become. At this time, the value of β is determined so that the relationship of the following equation is established between “α × θ” and “β × ω”.
(Β × ω) / (α / θ) = 1 / R
However, R is a constant.
For example, if R = 6, θ = 30, and ω = 2, the relationship between α and β is expressed by the following equation.
β / α = 1/360
From this relational expression, the summary still image number β (scene division number) is calculated with respect to the total number of frames α of the original moving image data.

  In step S3, the video analysis unit 4 extracts candidate still images from the original moving image data. The candidate still image is a target of feature amount extraction processing described later. In extracting the feature amount of the original moving image data, if all the still images included in the original moving image data are used, the calculation amount becomes very large. Therefore, in the present embodiment, candidate still images are extracted from the original moving image data at regular intervals. For example, candidate still images are extracted at intervals of 10 frames.

  In step S <b> 4, the video analysis unit 4 extracts a color arrangement feature amount for the candidate still image. In the color arrangement feature amount extraction process, the color arrangement feature amount of the still image is calculated for each candidate still image. The color arrangement feature amount represents a spatial distribution of colors in the still image.

  In steps S5 and S6, the moving image dividing unit 5 performs clustering (classification) of the candidate still images based on the color arrangement feature amounts of the candidate still images obtained in step S4 and the time series of the candidate still images. By this clustering, the number of clusters is finally set to the scene division number β obtained in step S2. The clustering process will be described later.

  In step S7, the moving image dividing unit 5 outputs the result of classification of candidate still images in steps S5 and S6.

  Next, the clustering process according to steps S5 and S6 will be described.

  First, the candidate still images are classified based on the color arrangement feature amount of the candidate still image obtained in step S4. The number of clusters here is the number of scene divisions β obtained in step S2. FIG. 3 shows a procedure of feature amount clustering processing based on the color arrangement feature amount. This feature clustering process corresponds to the frame clustering process disclosed in Non-Patent Document 1.

  In FIG. 3, in steps S11 to S14, a cluster of each color arrangement feature value and a cluster representative vector representing the feature of the cluster are generated for the color arrangement feature values of all candidate still images. For each cluster, only one corresponding color arrangement feature amount becomes the belonging data. Therefore, the cluster representative vector at this time represents one corresponding color arrangement feature amount.

Next, in step S15, the inter-cluster distance is calculated for all combinations of clusters. A distance d (C _i , C _j ) between the clusters C _i and C _j is expressed by the following equation.
d (C _i , C _j ) = E (C _i ∪C _j ) −E (C _i ) −E (C _j )
However, E (C _i) is the sum of the square of the distance between the cluster representative vectors all belonging data and cluster C _i Cluster C _i.

Next, in step S16, the two clusters having the smallest distance d (C _i , C _j ) between the clusters are integrated into one cluster. In step S17, a cluster representative vector of the integrated cluster is generated. In step S18, it is determined whether or not the clusters are consolidated into one. Steps S15 to S18 are repeated until the clusters are integrated into one.

  When clusters are aggregated into one, clusters are extracted in step S19. Here, clusters are extracted by the number of scene divisions β obtained in step S2 (that is, the number of clusters = β). As the clusters to be extracted, a distance threshold value at the time of cluster integration is set, and β clusters are selected from the clusters integrated at a distance equal to or less than the threshold value. As a result, β clusters are obtained.

  FIG. 4 shows an example (# 1, # 2, # 3) of summary clusters (# 1, # 2, # 3) obtained as a result of the feature amount clustering process. The numbers assigned to the data belonging to each summary cluster in FIG. 4 are the identification numbers of the candidate still images. The identification numbers are assigned in order of time series of candidate still images. In the example of FIG. 4, thirty candidate still images (identification numbers 1 to 30) are classified into three summary clusters # 1, # 2, and # 3 by the feature amount clustering process. The cluster resulting from the feature amount clustering process is hereinafter referred to as “upper cluster”.

  Next, the upper cluster is reclassified based on the time series of the candidate still images (time series information clustering process). In the upper cluster, still images that are temporally discrete in the original moving image data are mixed. Therefore, all candidate still images are rearranged in time series with respect to the upper cluster. At this time, if still image groups belonging to the same upper cluster are dispersed, they are reassigned as separate clusters. Specifically, for still image groups in each upper cluster, still images adjacent in time series are classified into the same cluster (hereinafter referred to as “lower cluster”), and a lower cluster is generated. . Then, all the lower clusters are rearranged in chronological order.

  The result of performing the time series information clustering process on the example of FIG. 4 is shown in FIG. In FIG. 5, the group of still images in the summary cluster # 1 in FIG. 4 is reclassified into six lower clusters 101, 103, 105, 109, 111, and 113. In addition, the still image group in the summary cluster # 2 is reclassified into five lower clusters 102, 106, 108, 110, and 115. In addition, the still image group in the summary cluster # 3 is reclassified into four lower clusters 104, 107, 112, and 114. As a result, a total of 15 subordinate clusters 101 to 115 are generated. Then, the lower clusters 101 to 115 are arranged in order of time series of candidate still images.

  Next, lower clusters are integrated (lower cluster integration processing). As a result of the time-series information clustering process, the number of lower clusters is larger than the scene division number β. Therefore, lower clusters are integrated until the number of clusters reaches the number of scene divisions β. In this lower cluster integration process, the similarity is calculated between adjacent lower clusters, and the lower clusters are integrated based on the similarity relationship. Specifically, for all the lower clusters, the lower clusters having high similarity are integrated in order, and the integration is repeated until β clusters are finally formed.

  For example, the feature amount of a candidate still image belonging to a certain lower cluster and the feature amount of the lower cluster are obtained. The feature amount of the candidate still image is a feature amount vector obtained by vectorizing the color arrangement feature amount of the candidate still image. The feature quantity of the lower cluster is a representative vector of the feature quantity vectors of the still images belonging to the lower cluster. As the representative vector, for example, a vector obtained by averaging feature quantity vectors of all still images belonging to the lower cluster can be used. Next, the similarity between the representative vectors of adjacent lower clusters is calculated for all adjacent lower cluster sets. The similarity is calculated by, for example, a cos function for two representative vectors. Next, the lower clusters corresponding to the two representative vectors having the smallest similarity are integrated. This is repeated until the number of clusters reaches β.

  As a specific example of the lower cluster integration processing, in addition to the above-described method using the representative vector of the lower cluster, for example, information on the upper cluster to which three or more adjacent lower clusters originally belonged, and the lower cluster And a method of integrating them based on information about still images belonging to. In this method, when there is a lower cluster sandwiched between two lower clusters originally belonging to the same upper cluster, these three lower clusters are integrated into the same cluster. For example, in FIG. 5, since there is a lower cluster 102 sandwiched between two lower clusters 101 and 103 that originally belonged to the same upper cluster (summary cluster # 1), these three lower clusters 101, 102 and 103 are integrated into one cluster. However, with respect to the total number of all candidate still images in the three adjacent lower clusters, the ratio of the number of still images in the lower cluster belonging to a different upper cluster among the three adjacent lower clusters is a constant value. Applicable only when:

  In the lower cluster integration process, the scene division number β may be changed. The scene division number β is determined according to the number of frames (reproduction time) of the summary content in step S2 of FIG. 2, but when integrating the lower clusters, the β value is set according to the feature amount of the candidate still image. Provide a process to change. In this scene division number changing process, the amount of change in color arrangement feature amount between candidate still images that are continuous in time series is obtained, and the value of β is increased when the amount of change is larger than a certain amount. On the other hand, if the amount of change is smaller than a certain amount, the value of β is decreased. As a result, for example, the number of still images for summarization is relatively large in places where the change is rapid, such as a scene where the camera or the subject moves greatly, while the number of still images for summarization is relatively small in places where there is almost no change such as a scene in a landscape Therefore, the contents of the original moving image data can be effectively incorporated into the summary content. As a result, it is possible to contribute to deepening the understanding of the contents of the original moving image data when the user views the summary content. Note that the scene division number changing process may be applied to all candidate still images, or may be applied only to candidate still images included in some lower clusters.

  The above is the detailed description of the clustering processing according to steps S5 and S6 of FIG. The β clusters obtained as a result of the above-described lower cluster integration process correspond to scenes (β) obtained by dividing the original moving image data.

Next, some examples will be given regarding a method of extracting a still image used for summary content from each scene divided by the moving image scene division processing according to the present embodiment.
[Embodiment 1 of Still Image Selection Method]
A still image at the center position in the scene is extracted as a summary still image.
[Embodiment 2 of still image selection method]
In the scene, a still image showing a person's face is extracted as a summary still image. At this time, when a plurality of still images are candidates, a still image that is most likely to be a face is extracted.
[Third embodiment of still image selection method]
A still image that is representative of a group of still images in the scene is extracted as a summary still image. For example, an average vector is obtained for the feature vectors of all still images in the scene, and a still image having a feature vector closest to the average vector is extracted.

  As described above, according to the present embodiment, a combination of clustering based on the color arrangement feature amount extracted from the still image and clustering based on the time series of the still image enables editing such as clear scene switching. Even if it is not a consumer-generated moving image, an effect that still image clustering suitable for the content of the original moving image data can be performed is obtained.

Further, according to the present embodiment, the following effects can be obtained.
(1) It is possible to divide a consumer-generated moving image into semantically valuable scenes.
(2) The summary content for the consumer-generated moving image can be generated by extracting a still image included in the divided scene. As a result, in a moving image sharing system in which a large amount of consumer-generated moving images are accumulated, it is possible to understand the contents of the consumer-generated moving images only by playing the summary content without playing the consumer-generated moving images themselves, This can contribute to the realization of efficient retrieval of consumer-generated moving images.
(3) Since a summary content with a small data size using a small number of still images can be generated for the original moving image data, the summary content can be stored even in a communication device with a low communication speed such as a cellular phone. You can download and watch at high speed.

  In the above-described embodiment, the color arrangement feature quantity is used as the video feature quantity. However, other video feature quantities such as motion information and audio information may be used.

  Further, the moving image scene dividing apparatus 1 according to the present embodiment may be realized by dedicated hardware, or may be configured by a computer system such as a personal computer, and the moving image scene dividing device shown in FIG. You may implement | achieve the function by running the program for implement | achieving the function of each part of the apparatus 1. FIG.

In addition, an input device, a display device, and the like (none of which are shown) are connected to the moving image scene dividing device 1 as peripheral devices. Here, the input device refers to an input device such as a keyboard and a mouse. The display device refers to a CRT (Cathode Ray Tube), a liquid crystal display device or the like.
The peripheral device may be directly connected to the moving image scene dividing device 1 or may be connected via a communication line.

  As mentioned above, although embodiment of this invention was explained in full detail with reference to drawings, the specific structure is not restricted to this embodiment, The design change etc. of the range which does not deviate from the summary of this invention are included.

It is a block diagram which shows schematic structure of the moving image scene division | segmentation apparatus 1 which concerns on one Embodiment of this invention. It is a flowchart which shows the procedure of the moving image scene division | segmentation process which concerns on the embodiment. It is a flowchart which shows the procedure of the feature-value clustering process which concerns on the embodiment. It is an example of the result of the feature-value clustering process which concerns on the embodiment. It is an example of the result of the time series information clustering process concerning the embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Moving image scene division | segmentation apparatus, 2 ... Moving image database, 3 ... Video input part, 4 ... Video analysis part, 5 ... Moving image division part

Claims (13)

Still image extraction means for extracting still images from video data;
A still image feature amount extracting means for extracting a video feature amount from a still image;
Scene division number determining means for determining the number of scene divisions of the video data;
Feature quantity clustering means for classifying the still image into any one of a plurality of upper clusters based on the video feature quantity of the still image extracted from the video data;
Time series information clustering means for dividing the upper cluster into lower clusters according to the time series of still images extracted from the video data;
Cluster integration means for integrating the lower clusters according to the time series of still images extracted from the video data, and integrating the lower clusters until the number of clusters reaches the number of scene divisions ;
A moving image scene dividing apparatus comprising: The feature amount clustering means includes:
Means for generating a cluster of each video feature amount, targeting the video feature amount of all still images extracted from the video data;
Means for generating a cluster representative vector representing video features belonging to a cluster;
Means for calculating the similarity between clusters based on the cluster representative vector,
It integrates two clusters with the highest similarity between clusters and generates a cluster representative vector of the cluster as a result of the integration. Selecting the number of scene divisions as an upper cluster from, and classifying still images corresponding to video features belonging to the upper cluster into the upper cluster,
The moving image scene dividing apparatus according to claim 1, wherein:   3. The time series information clustering unit includes still images adjacent in a time series order in a same lower cluster with respect to a group of still images in the upper cluster. Video scene segmentation device. The cluster integration means includes
Means for generating a representative vector representing the characteristics of the lower cluster;
Means for calculating a similarity between lower clusters based on the representative vector;
Means for integrating lower clusters having high similarity according to the time series of still images extracted from the video data ;
The moving image scene dividing device according to any one of claims 1 to 3, wherein the moving image scene dividing device is provided.   The cluster integration means integrates the three lower clusters into the same cluster when there is a lower cluster sandwiched between two lower clusters originally belonging to the same upper cluster. The moving image scene dividing device according to claim 4.   The cluster integration means calculates the number of still images in lower clusters belonging to different upper clusters among the three adjacent lower clusters, with respect to the total number of all still images in the three adjacent lower clusters. 6. The moving image scene dividing apparatus according to claim 5, wherein the integration of the three subordinate clusters is performed only when the ratio is equal to or less than a predetermined value.   5. The moving image scene dividing apparatus according to claim 2, wherein the similarity is obtained based on a distance between the representative vectors, and the similarity is determined to be higher as the distance is closer.   The moving image scene dividing apparatus according to claim 1, wherein the number of scene divisions is defined by a ratio to a total number of still images in the video data.   The moving image according to any one of claims 1 to 7, further comprising means for changing a number of scene divisions according to a video feature amount of a still image when the lower clusters are integrated. Scene division device.   10. The moving image scene division according to claim 9, wherein a change amount of a video feature amount between still images that are continuous in time series is obtained, and the number of scene divisions is increased when the change amount is larger than a predetermined amount. apparatus.   11. The change amount of the video feature amount between time-sequential still images is obtained, and when the change amount is smaller than a certain amount, the number of scene divisions is reduced. Video scene segmentation device.   12. The moving image scene dividing device according to claim 1, wherein the video feature amount is a color arrangement feature amount. A moving image scene dividing method for dividing video data into a plurality of scenes,
A still image extraction step for extracting a still image from video data;
A still image feature extraction step for extracting a video feature from a still image;
A scene division number determining step for determining the number of scene divisions of the video data;
A feature amount clustering step of classifying the still image into one of a plurality of upper clusters based on the video feature amount of the still image extracted from the video data;
A time series information clustering step for dividing the upper cluster into lower clusters according to the time series of still images extracted from the video data;
A cluster integration step of integrating the lower clusters according to a time series of still images extracted from the video data, and integrating the lower clusters until the number of clusters reaches the number of scene divisions ;
A moving image scene dividing method.

Images