JP2008153920A - Motion picture list displaying apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a motion picture list displaying apparatus capable of easily understanding difference between motion pictures to be displayed as a list by a user by summarizing and reproducing the motion pictures so as not to be similar mutually when a plurality of motion pictures are displayed after being arranged. <P>SOLUTION: The motion picture list displaying apparatus 100 selects a motion picture from motion pictures which are accumulated in an accumulating unit 101 by a motion picture selecting unit 102 based on a condition designated by a condition inputting unit 103. The selected motion picture is detected its feature by a feature detecting unit 109, then correlation calculation of a motion picture group is performed by a correlation calculating unit 104 using this feature. Then summary of the motion picture added with the calculated correlation is produced by a summary producing unit 105, and a list of the produced summary is displayed. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a moving image list display device that displays a list of a plurality of moving images.

  In recent years, with the speeding up of networks and the diversification of channels, the amount of moving image content that can be viewed by individuals has increased dramatically. In addition, with the increase in capacity of recording media such as DVDs (Digital Versatile Discs) and hard disk drives (HDDs), a large amount of moving images are recorded on these recording media and the program can be viewed without being restricted by the broadcast time. The style became common.

  In this way, when the amount of moving images handled by an individual becomes enormous, a technique for quickly and easily confirming the contents of the moving images is required in order to search for moving images that the user wants to view. As means for solving this problem, there are Patent Documents 1 and 2.

Japanese Patent Application Laid-Open No. 2004-228561 is a method of creating a summary of a moving image and confirming it. Further, in Patent Document 2, in addition to selecting a plurality of thumbnail still images from moving image data and creating a summary of moving images obtained by frame-by-frame advancement, the summary can be played back as a list at the same time. Makes it possible to quickly check the contents of multiple moving images.
JP 2003-101939 A JP-A-11-284948

  However, for example, when a summary of each of a plurality of moving images classified into the same genre is created, the contents may be similar. When these similar summaries are displayed as a list as shown in Patent Document 2, the user cannot fully understand the difference between the moving images, and as a result, it is not possible to select which moving image may be selected. There is a problem.

  For example, when summarizing each of a plurality of dramas of the same title, the moving image summary creation technique disclosed in Patent Document 1 selects partial moving images using parameters optimized for each genre, and the time length is reduced. Create a shortened video summary. When applying the summary conditions prepared for each genre, the genre of the moving image group to be summarized is the same for “drama”, so the summary conditions to be applied are also the same.

  Here, consider a case where these dramas are composed of patterns that are stylized every time. If a part of a moving image that includes the stylized pattern is selected in one of the story summaries, a part that includes the same pattern is selected in the other story summaries summarized under the same summarization condition. The possibility increases.

  In general, this stylized pattern is often similar in terms of video or audio. Therefore, even if these video summaries are arranged in a list, all of them have the same content. It is difficult to understand the difference between the moving images. As a result, it is difficult for the user to determine which moving image should be viewed.

  The present invention has been made in consideration of the above circumstances, and by taking into account the correlation of each scene of the displayed moving images, a unique moving image summary that makes it easy for the user to compare them is provided. An object of the present invention is to provide a moving image list display device that creates and displays a list.

In order to solve the above-described problems, the moving image list display device of the present invention has the following configuration.
The moving image list display device of the present invention includes a storage unit that stores a moving image including a plurality of information including video and audio, and a selection unit that selects a plurality of moving images to be displayed in a list according to a predetermined condition from the storage unit. Based on information obtained from the moving image itself or information attached to the moving image, a feature detecting unit that detects feature information of the entire moving image or each scene, and the feature information detected by the feature detecting unit A correlation calculation unit that calculates a correlation between scenes for each scene of a plurality of moving images selected by the selection unit; a correlation between the scenes calculated by the correlation calculation unit; and a detection by the feature detection unit A summary creation unit that creates a summary of each moving image selected by the selection unit based on the feature information that is selected, and a list display unit that displays a summary of the summary created by the summary creation unit Is.

  Alternatively, the moving image list display device according to the present invention includes a storage unit that stores a plurality of pieces of information including video and audio, and feature information of the moving image, and a list display according to a predetermined condition from the storage unit. A correlation unit that calculates a correlation between scenes for each scene of the plurality of moving images selected by the selection unit using the feature information stored in the storage unit; A summary for creating a summary of each moving image selected by the selection unit based on the calculation unit, the correlation between the scenes calculated by the correlation calculation unit, and the feature information stored by the storage unit A creation unit; and a list display unit that displays a list of summaries created by the summary creation unit.

Here, the predetermined condition may be information such as genre designation, keyword designation, or a selection operation performed by a user from a list of moving images, and information obtained from the moving image itself or information attached to the moving image may be, for example, , Video information, audio information, subtitle information, and information (eg, EPG, tag information, etc.) attached to a moving image.
The correlation calculation unit obtains a correlation between scenes using an agglomeration type clustering method, a branch type clustering method represented by k-means, or the like.

The summary creation unit creates a summary of a moving image by any of the following.
(1) Based on the correlation between the scenes calculated by the correlation calculation unit, a time length for reproducing the scene is determined according to the number of similar scenes, and each scene included in the moving image is determined A summary of each moving picture is created by playing back in time length.
The time length may be such that as the number of similar scenes in the scene increases, the playback time is shortened, and further, based on the feature information obtained from the feature detection unit, the time length of the scene. The length may be adjusted.
In addition, in the case of a scene that clearly shows a difference from other moving images, the playback time of the scene may not be changed.
Furthermore, the time length of the scene may be adjusted depending on whether the moving image including the similar scene is the same moving image or a different moving image.

(2) Based on the correlation between the scenes calculated by the correlation calculation unit, a summary of each moving image is created by collecting scenes where the number of similar scenes is less than a predetermined number.
(3) A target playback time is determined for each moving image, and a scene that clearly shows a difference from other moving images can be played at the target playback time based on the correlation between the scenes calculated by the correlation calculation unit. Collect as much as possible and create a summary of each video.

(4) An operation input unit that receives designation of a specific moving image from a list of moving image groups, and the summary creation unit includes a moving image (hereinafter referred to as an attention moving image) specified by the operation input unit. For the summary of other moving images, a scene similar to the scene included in the specified moving image summary is created. Create without. As a result, even when a plurality of moving images are displayed in a list, it is difficult for a user to select a moving image of a moving image because it is difficult to select a scene in the summary that is determined to have many similar images. It becomes easy to confirm the difference, and it is easy to find a desired moving image.

  Further, in the above moving image list display device, the accumulation unit detects and accumulates characteristic information in advance for all accumulated moving images, and the correlation calculation unit and the summary creation unit include: The feature information of the moving image selected by the moving image selection unit may be directly read from the storage unit.

  In addition, a program for causing a computer to function as each unit of the moving image list display device having the above-described configuration is created, or the program is recorded on a computer-readable recording medium, and the program is executed by the computer. The above-mentioned problem can be solved also by executing.

In the present invention, a plurality of moving images are displayed as a list, and when at least one of the displayed moving images is reproduced as a moving image summary, a combination of moving images displayed as a list of the generated summaries It changes according to.
Further, correlation between scenes is used as one of the criteria for creating a summary of moving images, and scenes that are judged to have many similarities among the displayed moving images are made difficult to be selected in the summary.
Thereby, even when a plurality of moving images are displayed in a list, the user can easily confirm the difference between the moving images, and can easily find a desired moving image.

DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments of a moving image list display device of the invention will be described with reference to the drawings.
<Embodiment 1>
FIG. 1 is a block diagram illustrating a functional configuration of the moving image list display device according to the first embodiment.
In FIG. 1, a moving image list display device 100 includes a storage unit 101, a moving image selection unit 102, a condition input unit 103, a correlation calculation unit 104, a summary creation unit 105, a screen synthesis unit 106, a display unit 107, and an operation input unit 108. , Including a feature detection unit 109.

Details of each part of the moving image list display device according to the first embodiment will be described below.
The accumulation unit 101 is formed of a storage medium such as a DVD or an HDD, and accumulates a plurality of moving images targeted in the first embodiment. This moving image is recorded by receiving a broadcast wave, acquired through a network such as the Internet, acquired through a recording medium such as a DVD or various memory cards, and the like. WMV, MPEG-1, MPEG- 2. It may be compressed by MPEG-4, H.264 / AVC or the like. In addition to these moving images, moving image additional information (for example, subtitle information, EPG (Electronic Program Guide), and moving images) used for calculating correlation of each scene and creating a moving image summary. Tag information etc.) is accumulated.

  The condition input unit 103 inputs a condition for selecting a moving image to be displayed as a list from a plurality of moving images stored in the storage unit 101. As a condition for selecting the moving image, for example, a search using a genre designation, a keyword, or the like can be given. Of course, it is also possible to select all moving images without specifying any of these conditions.

  The moving image selection unit 102 selects a moving image that matches the condition input by the user through the condition input unit 103 from the plurality of moving images stored in the storage unit 101.

  The feature detection unit 109 acquires various information from the moving image about the moving image selected by the moving image selection unit 102, detects the entire moving image or the feature of each scene, and the moving image obtained here The feature is transmitted to the correlation calculation unit 104 and the summary creation unit 105, and is used when calculating the correlation of each scene and creating a summary of a moving image.

Here, the scene in the present invention refers to a frame section having a semantic unit in a moving image that is a time series of a plurality of frames, and some standard, for example, metadata or a moving image previously described by hand. It is assumed that the image has already been divided by an automatic division method using image information.
As this automatic division method, a known method (for example, Shin Yamada, Toshikazu Fujioka, Katsuhiro Kanamori, Koji Matsushima: Examination of a scene change detection method focusing on a common color for each partial region: Television Society Technical Report: Vol 17, No 55 ) Shall be used.

The feature detection unit 109 detects features using information obtained from the moving image itself or information attached to the moving image.
Examples of information obtained from the moving image itself include video information included in the moving image, audio information or caption information, and information attached to the moving image includes EPG, tag information attached to the moving image, and the like. .
Examples of features obtained from the moving image information include average luminance information obtained from video information, color histogram information in various color spaces, edge direction / density, edge information, motion vector information, audio level obtained from audio information, Important keywords such as frequency distribution, utterance position, background music (BGM) information, telop information obtained from text information, title obtained from EPG and tag information, genre information, performers, recording date and time, and the like.

The correlation calculation unit 104 calculates the correlation between the entire moving images to be displayed obtained from the moving image selection unit 102 or each scene included in each to obtain the correlation between the moving images.
The scene correlation uses a plurality of feature information obtained by the feature detection unit 109 singly or in combination, and the feature information of each scene is converted into a feature vector of an arbitrary dimension. Here, the feature vector is a vector of an arbitrary dimension obtained from feature information obtained from each scene.

Next, an example of correlation calculation for each scene using an agglomerative clustering technique will be described.
As shown in FIG. 2, it is assumed that there is a moving image 1 divided from scene A to scene F and a moving image 2 divided from scene G to scene L. Here, it is assumed that an m-dimensional feature vector V_i is obtained from the feature of a moving image of an arbitrary scene i. At this time, in the m-dimensional space, based on the vector V_i, the n scenes from the scene 1 to the scene n are classified into n clusters (C_1 to C_n) each including only one scene, and this is set as an initial state.

From this initial state, the Euclidean distance d (V_i, V_j) between the feature vectors V_i and V_j of the two scenes is acquired. This distance d (V_i, V_j) indicates the high correlation between the scene i and the scene j, and a small value indicates that the correlation between the scene i and the scene j is high.
Further, using this information d (V_i, V_j), the distance of each cluster is obtained. The distance D (C_i, C_j) between the clusters C_i and C_j is obtained by the following equation (1).

  Here, n_i and n_j are the numbers of scenes included in the clusters C_i and C_j, respectively. Here, for the sake of explanation, the distance between all scenes included in the cluster C_i and all scenes included in the cluster C_j is obtained, and the average is the distance D (C_i, C_j) between the clusters. The distance between clusters may be calculated using. For example, the shortest of the distances d (V_i ′, V_j ′) between any scene i ′ included in the cluster C_i and any scene j ′ included in the cluster C_j is the distance between the clusters. A distance method or a maximum distance method in which the maximum distance among the above-described distances is used may be used.

  Next, the distance D (C_i, C_j) between the clusters is calculated among all the clusters, and the clusters C_k and C_l that satisfy the following formula (2) with the smallest distance are merged as the same cluster.

  By repeating this merging process for (n-1) clusters after merging, the clusters are gathered for each step. However, when D_min (C_k, C_l) exceeds a preset threshold Th, the merging process is terminated. That is, when all the clusters are separated by the threshold Th or more, the clusters are regarded as scene clusters that are not similar to each other.

  FIG. 3 is a diagram illustrating an example of a result of classification using the above-described clustering method from m-dimensional feature vectors obtained from the scenes A to L illustrated in FIG. Here, for the sake of explanation, the plot is simplified to two dimensions. As a result of the classification, each scene is classified into each cluster surrounded by a dotted line.

  In the present invention, in the plurality of clusters obtained in this way, scenes classified into the same cluster are judged to have high correlation with each other, and conversely, scenes classified into different clusters have low correlation. Hereinafter, a cluster in which one or more scenes are classified is called a scene cluster.

  In the above description, an agglomeration type clustering method is used to calculate the correlation of scenes. However, a branch type clustering method represented by k-means may be used, and scenes using other techniques may be used. You may obtain | require the correlation between.

  The summary creation unit 105 creates a summary of each moving image selected by the moving image selection unit 102 using the correlation of each scene shown in the correlation calculation unit 104 and the feature information obtained from the feature detection unit 109. . Hereinafter, an example of a technique for creating a summary of a moving image will be described with reference to FIGS.

Consider a case in which four moving images, moving image 1, moving image 2, moving image 3, and moving image 4 are displayed as a list. Each of these moving images is divided into scenes as shown in FIG. 4, and all the scenes are classified into scene clusters A to O using the above-described method.
As a result of the classification, scene clusters A, B, C, and D each including a set of a plurality of similar scenes are created, and the scenes included in the other scene clusters E to O do not have similar scenes. Assume that only one scene is classified for one cluster.

  Here, as the number of similar scenes increases, that is, as the number of scenes classified into the same scene cluster increases, it is desired to set a coefficient that lowers the importance of those scenes. When this coefficient is the importance ω_i, the importance ω_i is calculated by the following equation (3). Here, a and b are arbitrary constants preset in the apparatus, and n_i is the number of scenes classified into the scene cluster i.

The importance ω_i is used to determine the playback time length of each scene or the presence / absence of scene playback.
For example, when the importance ω_i of each scene cluster is obtained from the result of classifying the moving image 4 from the moving image 1 shown in FIG. 4, the following relationship is established.

      ω_A <ω_B <ω_C = ω_D <ω_E = ω_F = ... = ω_O

In the above description, all scenes are treated as equivalent and ω_i is obtained. However, the importance may be determined by considering whether or not the scenes are included in the same moving image.
For example, in the method of calculating importance ω_i described above, the importance ω_i decreases even in the cluster i in which only scenes obtained from the same moving image are collected.

  However, from the viewpoint of confirming a difference between moving images, a cluster i in which only scenes obtained from the same moving image are collected is a cluster in which scenes that clearly show differences from other moving images are collected. I can say that. Therefore, when many scenes obtained from the same moving image are included in a scene group classified into a certain cluster, the importance ω_i is set so that the importance of the scene classified into the cluster becomes high. It sets with the following formula | equation (4). However, a and b are preset constants, and m_i is the number of scenes in which scenes included in the cluster i are not counted repeatedly.

  In FIG. 3, based on the importance calculation method shown above, the clusters 301 and 302 in which two scenes are classified have the same importance, but based on the importance calculation method shown this time. For example, 302 is more important than 301. Here, 302 is a cluster including many scenes of the moving image 2 and is considered to be a cluster suitable for showing a difference from the moving image 1.

Next, a summary of a moving image is created using the obtained importance ω_i.
Here, as one example of creating a summary of moving images, an example is shown in which importance is used for calculating the playback time of each scene after summarization.
Using the obtained importance ω_i, the reproduction time length Tx_i after summarization is obtained by the following equation (5). However, tx_i represents the reproduction time length of the scene x classified into the scene cluster i.

As shown in FIG. 5, the scene x is summarized in accordance with the reproduction time length Tx_i after the summary.
As a method of summarizing a scene in accordance with a specified time, it may be realized by partial reproduction or high-speed reproduction using a known summarization technique (for example, Japanese Patent No. 3640615). Other techniques may be used.

  FIG. 6 summarizes the moving image 1 to the moving image 4 in FIG. 4 in this way. The summarized moving images 1 to 4 are obtained by summarizing the reproduction times of the respective scenes of the moving images 1 to 4 using the above-described method. In the example of FIG. 6, the more similar scenes in each moving image, the shorter the summary, and conversely, a summary is created around a unique scene that is not similar to other scenes.

  A summary creation method is also conceivable in which the length of the summary is determined in advance and a summary of the moving image is created in accordance with the length. In the summary creation example shown in FIG. 6, since the importance level is applied to the moving image as it is to determine the playback time length of each scene, the playback time lengths of the summaries are not uniform. By normalizing the reproduction time length of the summary with the target reproduction time length, it is possible to make all the summary reproduction times constant.

  Also, in FIG. 7, the scenes constituting each moving image are rearranged in descending order according to the importance ω_i obtained earlier, that is, rearranged so that unique scenes are preferentially reproduced, and an arbitrary amount of time from the beginning. Is reproduced as a summary. Now, assuming that the time t is set as the target reproduction time and a scene that can be reproduced within the time t is reproduced from the beginning, a summary of a moving image as shown in FIG. 8A is created. Further, as shown in FIG. 8B, a set of scenes that can be reproduced within the target reproduction time t obtained in FIG. 8A is displayed along the time axis, that is, in the order of appearance in the moving image before summarization. According to the above, it may be rearranged again.

  As another summarization technique using importance ω_i, it is also possible to use a format in which reproduction is performed only when importance ω_i satisfies a preset condition. FIG. 9 is a diagram illustrating an example of a case where only scenes that have no other similar scenes are reproduced, that is, only scenes satisfying ω_i = a + b are reproduced.

  In addition, as shown in FIG. 10, any one scene may be selected and reproduced from scenes having high importance ω_i, or a plurality of representative still images may be extracted from each scene. The number of representative still images to be extracted may be proportional to the importance ω_i.

  Of course, the importance ω_i is not derived from one index, and a plurality of indices may be combined. By using the feature information detected by the feature detection unit 109 in addition to the correlation information of each scene obtained by the correlation calculation unit 104, a more flexible moving image summary can be created.

  FIG. 11 is a diagram illustrating an example in which a scene having a high audio level 1101 and not included in another moving image is an important scene. That is, the importance ω_i in the scene i is obtained by the equation (6). Here, a, b, and c are arbitrary constants set in advance, and s_i is an average sound level in the scene i.

If the scene having the highest importance ω_i is selected as the summary of the moving image, the scene F is selected as the summary of the moving image.
In addition to this, the feature information that can be used to determine the importance includes scene switching frequency. For example, in sports programs such as hits and throwing scenes in baseball broadcasts, important passes and goal scenes in soccer broadcasts, etc. Switching occurs frequently. For this reason, when the scene is divided at a position that causes a scene change such as the camera switching described above, the degree of importance is high when the interval between scenes is short, and the degree of importance is low when it is long. The importance ω_i is set in consideration of such an index. That is, the importance ω_i is obtained by the following equation (7). Here, it is assumed that l_i is a reproduction time length in the scene i, and a, b, and c are coefficients set in advance.

  In addition, it is determined whether the scene contains many sections that are determined to be commercial sections based on the type of monaural or stereo audio information, and the ratio of non-commercial (main part) in each scene is used as one of the indicators. By using it, the commercial part may be excluded from the summary of the moving image, or the intensity of the motion of the scene is obtained from the motion vector information obtained from the video information, and this is one of the indices for calculating the importance ω_i It may be used as

  Also, frames that match preset feature information, such as a face of a specific person, are detected using a method such as pattern matching, and the ratio of the frames included in the scene is considered as one of the indices. In order to increase the importance of a scene that includes many areas with high edge strength, such as a scene in which embedded telop appears in the video, the edge strength of each frame included in the scene is used as an index. It's okay.

  In the example of creating importance, when the moving image to be handled is compressed in some form, the importance is calculated using information that can be acquired from the decoded video information. The importance may be calculated using. For example, when the importance is calculated using the previous motion vector, if the video information is compressed by an inter-frame predictive encoding method such as MPEG (Moving Picture Experts Group), the video information before decoding The motion vector information obtained from the above may be used.

  In this way, by taking into account the correlation between moving images and creating a summary of moving images so that the unique scenes of each moving image are emphasized, even if the user views these summaries side by side, the difference in content can be reduced. It becomes easy to grasp.

  The operation input unit 108 includes an input device such as a remote controller that receives user operations, and issues instructions to the screen composition unit 106 through selection of moving images displayed in a list.

The screen synthesizing unit 106 reproduces all the moving images selected by the moving image selection unit 102 and all the moving images summarized by the summary video creating unit 105 and displays them on the display unit 107 as a list.
However, instead of playing all of them at the same time, only a moving image selected by the user by moving the cursor is summarized and reproduced, and other moving images not selected may be displayed as a representative still image.

  As shown in FIG. 12 (A), the moving images to be displayed in a list may have a format in which only the summary of moving images is displayed as a list, or as shown in FIG. Information (for example, title, recording date and time) may be displayed simultaneously.

  Further, only the details of the moving image on which the user moves the cursor may be displayed. For example, as shown in FIG. 12C, when the user moves the cursor to the moving image 3 (1201), an enlarged summary of the selected moving image 3 and character information 1203 indicating the details of the moving image are displayed. May be displayed. Here, at the position of the enlarged display 1202, the moving image 3 before being summarized may be reproduced, or a summary that does not take the correlation of moving images into consideration may be reproduced. .

  However, the summary that does not take into account the correlation of the moving image in the present invention does not use the correlation obtained from the correlation calculation unit 104, and uses only the feature information obtained from the feature detection unit 109 to summarize the generated moving image. That is, it is created by a known method (for example, JP-A-2003-101939).

  Further, as described above, it is not necessary to display all the moving images selected by the moving image selection unit 102 on the screen at a time, and the scroll bar 1204 can be used for scrolling, as shown in FIG. 12 (E), the moving image may be divided into a plurality of pages and displayed in advance, and the page may be switched by the user selecting the page turning return button 1205 or the next button 1206. .

  The display unit 107 includes a liquid crystal display or the like, and displays a result input by the user through the condition input unit 103 and the operation input unit 108, a list of a plurality of summarized moving images combined by the screen combining unit 106, and the like.

<Modification of Embodiment 1>
In the first embodiment (FIG. 1), the feature information is sequentially calculated from the moving images selected by the moving image selection unit 102. However, the moving images handled by the feature detection unit 109 are already recorded. Therefore, the feature information may be detected and accumulated for all moving images in advance.

  FIG. 13 is a block diagram illustrating a functional configuration of a moving image list display device 200 according to a modification of the first embodiment. In FIG. 13, the moving image list display device 200 includes a storage unit 201, a moving image selection unit 102, and a condition input. Unit 103, correlation calculation unit 104, summary creation unit 105, screen composition unit 106, display unit 107, and operation input unit 108. 13, the same functions as those in FIG. 1 are denoted by the same reference numerals, and the description thereof is omitted.

  In the case of this modification, feature information is previously detected and stored in the storage unit 201 for all moving images, and the correlation calculation unit 104 and the summary creation unit 105 are selected by the moving image selection unit 102. The feature information of the moving image is directly read from the storage unit 201.

  Further, the feature information recorded in the storage unit 201 may be detected by a device other than the feature detection unit 109 of the present apparatus, and the feature information of a moving image detected in advance is acquired through a broadcast wave, a network, or various recording media. The data may be recorded in the storage unit 201.

<Embodiment 2>
In the first embodiment, it may be possible to select a moving image by further narrowing down the conditions after displaying a list. The second embodiment is an example in the case where the user further confirms the contents from the moving image group displayed in the list in the first embodiment.

FIG. 14 is a block diagram illustrating a functional configuration of the moving image list display device 300 according to the second embodiment. In the drawing, the moving image list display device 300 includes a storage unit 101, a moving image selection unit 102, and a condition input unit. 103, a correlation calculation unit 104, a summary creation unit 105, a screen synthesis unit 106, a display unit 107, an operation input unit 308, and a feature detection unit 109. 14, the same functions as those in FIG. 1 are denoted by the same reference numerals, and the description thereof is omitted.
The operation input unit 308 accepts an instruction from the user for confirming the contents of the list of moving images displayed in a part, and instructs the screen composition unit 106.
Note that, as in the modification of the first embodiment described above, the feature information may be detected and stored in advance in the storage unit 101 for all moving images. The configuration in this case is configured using FIG. 13 instead of FIG.

The operation in the case of the second embodiment will be described with reference to FIGS.
Now, as shown by 1501 in FIG. 15A, the user requests to display only the summary of the moving image 3, the moving image 4, the moving image 5, and the moving image 8 from the displayed moving images. It shall be.
FIG. 15B is a diagram after selecting and selecting from these moving images with the cursor. At this time, information that the moving image 3, the moving image 4, the moving image 5, and the moving image 8 are selected and extracted is transmitted from the operation input unit 308 to the condition input unit 103 as a moving image selection condition. After being selected by the moving image selection unit 102, the correlation between the scenes is calculated again, and the moving images corresponding to the moving image 3, the moving image 4, the moving image 5, and the moving image 8 that take into account only the correlation of these four moving images are used. A new summary is created.

The method for selecting a moving image is not limited to the method in which the user directly selects and instructs as shown in FIG. 15A, but as shown in FIG. 15C, the moving image can be selected in advance. It is also possible to select a moving image that falls within a range 1502 having a certain length.
Further, as a method of selecting an arbitrary moving image from FIG. 15A, not only selecting by a range, but also judging from EPG information accompanying the moving image as shown in FIG. 15D. By limiting with genre information 1503, it is good also as selecting some moving images, and it is good also considering search results, such as a keyword search, as selection conditions, as shown in FIG.15 (E).

<Embodiment 3>
In the third exemplary embodiment, the moving image currently focused by the user is set as a summary of moving images that do not take correlation into account, and the summaries of moving images displayed in the vicinity thereof are not similar based on the summary. To.
FIG. 16 is a block diagram illustrating a functional configuration of the moving image list display device 400 according to the third embodiment. In the drawing, the moving image list display device 400 includes a storage unit 101, a moving image selection unit 102, and a condition input unit. 103, a correlation calculation unit 104, a summary creation unit 405, a screen synthesis unit 106, a display unit 107, an operation input unit 408, and a feature detection unit 109. In FIG. 16, the same functions as those in FIG. 1 are denoted by the same reference numerals, and the description thereof is omitted.
Note that, as in the modification of the first embodiment described above, the feature information may be detected and stored in advance in the storage unit 101 for all moving images. The configuration in this case is configured using FIG. 13 instead of FIG.

The summary creation unit 405 takes into account the sum of each moving image obtained by the moving image selection unit 102 in consideration of whether or not the moving image is currently focused on by the user in addition to the correlation obtained by the correlation calculation unit 104. Create
The operation input unit 408 receives an instruction from the user as to whether or not the moving image is currently focused, and passes the instructed moving image to the summary creation unit 405.
The operation of the third embodiment will be described below with reference to FIGS.

  As described in the first embodiment (FIG. 1), first, the user inputs selection conditions for moving images to be displayed in a list from the condition input unit 103, and the moving image selection unit 102 starts from the storage unit 101 based on the conditions. By selecting a moving image, using the feature information of the selected moving image obtained by the feature detection unit 109, a correlation is calculated by the correlation calculation unit 104, and a summary of the moving image considering the correlation is calculated by the summary creation unit 405. create.

  However, at this time, there is no output from the operation input unit 408, and the summary creation unit 405 operates in the same manner as the summary creation unit 105 described in the first embodiment. Thereafter, the summary of the moving images to be displayed in a list is combined by the screen combining unit 106 and the list is displayed on the display unit 107.

  When this list display is performed, it is assumed that the user has selected a moving image currently focused on by the user through the operation input unit 408. At this time, the operation input unit 408 not only reflects the user operation input on the display unit 107 via the screen composition unit 106 but also transmits it to the summary creation unit 405.

  In a summary creation unit 405, if there is a moving image that the user is paying attention to, a summary that does not take into account the correlation with other moving images is created for the moving image. That is, in the summary creation unit 405, when a moving image noticed by the user is designated from the operation input unit 408 by moving the cursor to the moving image displayed in a list (FIG. 17) or the like, the moving image is performed according to the following procedure. Create a summary of

  First, a summary is created without using the correlation between the moving images calculated by the correlation calculation unit 104 for the moving image that the user is paying attention to. The summary without using the correlation of the moving images is created using voice features as shown in FIG. 18, for example. FIG. 18 shows a summary creation method using an audio level, which is one of the characteristics obtained from audio of a moving image. Mainly in sports programs, the cheering of the audience increases in important scenes. That is, it can be determined that a scene with a high audio level has a high importance.

Therefore, a summary of a moving image is created by extracting a scene including an audio level exceeding the threshold 1802 as an important scene with respect to the audio level 1801 of the audio included in each scene. However, the threshold value 1802 is an arbitrary value set in advance. Here, corresponding scenes B, C, and E are selected, and summary 1 is created.
Of course, in order to create a summary from video features without taking into account the correlation of moving images, other methods may be used. For example, the technique of a known document (for example, Japanese Patent Laid-Open No. 11-284948) may be used. It may be used.
Here, the summary of the created moving image is referred to as the summary of the moving image of interest.

Next, a summary of moving images other than the moving image focused by the user is performed.
Now, the correlation calculation unit 104 calculates the correlation between the scenes A to F of the moving image 1 shown in FIG. 18 and the scenes G to L of the moving image 2 shown in FIG. Here, it is assumed that clustering as described above is performed, and FIG. 20 is a diagram in which the results are plotted in two dimensions.

  At this time, it is desired to summarize a moving image that is not focused on by the user (hereinafter, referred to as a non-focused moving image) so as not to be similar to the focused moving image summary that is a summary of the moving image focused on by the user. If the moving image that the user is paying attention to is the moving image 1 and the summary of the moving image of interest is the summary 1, it is similar to the scenes B, C, and E included in the summary 1. The importance is reduced for scenes, that is, scenes classified into the same cluster as each of scene B, scene C, and scene E.

  FIG. 21 shows an example in which scenes that are not classified into the same cluster as the summary 1 that is the summary video of the moving image of interest are extracted as the summary video. As shown in FIG. 20, scenes G, Scene I, and Scene L classified into the same clusters 2001, 2002, and 2003 as Scene B, Scene C, and Scene E, which are scenes included in Summary 1, are summarized. The summary 2 is created with the scene H, the scene J, and the scene K which are omitted from the summary 1 and are not included in the summary 1.

  Of course, as described in the summary creation unit 105, the summary of the moving image is not created based only on the correlation between the moving images, but is created taking into account various feature information obtained from the feature detection unit 109. It may be a thing.

  As shown in the above configuration, by creating other similar summaries based on the summaries created for the moving images that the user notices among the list of moving images, While confirming an important scene, it is possible to confirm the difference between the surrounding moving images and the target moving image.

  Note that the present invention is not limited to the above-described embodiment, and various modifications and corrections can be made without departing from the scope of the present invention. For example, the function of each part of the moving image list display device can be realized as a computer program, and the computer program can be installed in the moving image list display device and executed. Further, the computer program can be recorded on a detachable recording medium, or downloaded via a network or broadcast wave, so that the transfer can be performed easily and easily.

3 is a block diagram illustrating a functional configuration of a moving image list display device according to Embodiment 1. FIG. It is a figure which shows the mode of the moving image divided | segmented into the scene. It is the figure which plotted the mode after clustering the scene shown in FIG. 2 in two dimensions. It is a figure which shows a mode that each scene shown in FIG. 2 was classified into each cluster. It is a figure which shows a mode that the moving image 1 shown in FIG. 2 was summarized based on the set target reproduction | regeneration time. It is a figure which shows a mode that the moving image 4 shown in FIG. 2 was summarized based on the set target reproduction | regeneration time. It is a figure which shows a mode that the scene of each of the moving image 1 to the moving image 4 shown in FIG. 2 was rearranged in order of importance based on the obtained importance. FIG. 3 is a diagram showing a state in which a scene to be reproduced is selected from each scene of moving image 1 to moving image 4 shown in FIG. 2 based on a target reproduction time t. It is a figure which shows a mode that the moving image 1 shown in FIG. 2 was summarized based on the set threshold value. It is a figure which shows a mode that it summarized by selecting arbitrary 1 scenes from the moving image 1 shown by FIG. It is a figure which shows a mode that it summarizes based on the similarity and audio | voice level of each scene shown by FIG. It is a figure which shows an example of the list display method of a moving image. It is a block diagram which shows the function structure of the moving image list display apparatus which concerns on the modification of this embodiment. 6 is a block diagram illustrating a functional configuration of a moving image list display device according to a second embodiment. FIG. It is the figure which showed a mode that the arbitrary moving images were selected from the moving image displayed by the list. It is a block diagram which shows the function structure of the moving image list display apparatus which concerns on Embodiment 3. FIG. It is the figure which showed a mode that the user selected one arbitrary moving images. It is the figure which showed a mode that the moving image 1 was summarized using the audio | voice level. It is the figure which showed the mode of the moving image 2 by which the scene division | segmentation was carried out. It is the figure which plotted the clustering result of each scene of the moving image 1 and the moving image 2 shown by FIG. 18, FIG. 19 in two dimensions. It is a figure which shows a mode that the moving image 2 shown by FIG. 19 was summarized.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100, 200, 300, 400 ... Moving image list display apparatus, 101, 201 ... Accumulation part, 102 ... Moving image selection part, 103 ... Condition input part, 104 ... Correlation calculation part, 105, 405 ... Summary preparation part, 106 ... Screen composition unit 107 ... Display unit 108 308 408 Operation input unit 109 Feature detection unit 301 Scene cluster 1, 302 Scene cluster 2, 1101, 1801 Voice level graph 1802 Threshold 1201 ... selected moving image, 1202 ... enlarged moving image thumbnail of the selected moving image, 1203 ... character information indicating details of the moving image, 1204 ... scroll bar, 1205 ... "return" button to return to the previous page, 1206 ... “Next” button for proceeding to the next page, 1501... Selected moving image, 1502. 1503 ... selected category information, 2001 ... scene cluster 1,2002 ... scene cluster 2,2003 ... scene cluster 3.

Claims (15)

  A storage unit that stores a moving image including a plurality of information including video and audio, a selection unit that selects a plurality of moving images to be displayed in a list according to a predetermined condition from the storage unit, and information or a moving image obtained from the moving image itself Based on information attached to the image, a feature detection unit that detects feature information for the entire moving image or each scene, and a plurality of features selected by the selection unit using the feature information detected by the feature detection unit For each scene of the moving image, based on the correlation calculation unit that calculates the correlation between scenes, the correlation between the scenes calculated by the correlation calculation unit, and the feature information detected by the feature detection unit, A moving image list display device comprising: a summary creation unit that creates a summary of each moving image selected by the selection unit; and a list display unit that displays a summary of the summary created by the summary creation unit .   The moving image list display device according to claim 1, wherein the predetermined condition is given by genre designation or keyword designation.   The moving image list display device according to claim 1, further comprising an operation input unit that receives a specific moving image selected from a list of moving image groups, wherein the predetermined condition is input by the operation input unit. A moving image list display device, which is given by a user's selection operation.   2. The moving image list display device according to claim 1, wherein information obtained from the moving image itself or information attached to the moving image is one or more of video information, audio information, caption information, EPG, and tag information. A moving image list display device characterized by being a combination.   The moving image list display device according to claim 1, wherein the correlation calculation unit calculates a correlation of the scenes using a clustering method.   2. The moving image list display device according to claim 1, wherein the summary generation unit reproduces the scene according to the number of similar scenes based on the correlation between the scenes calculated by the correlation calculation unit. And a summary of each moving image is created by reproducing each scene included in the moving image with the determined time length.   7. The moving image list display device according to claim 6, wherein the time length of the scene is shortened as the number of similar scenes increases.   The moving image list display device according to claim 7, wherein the summary creation unit further adjusts the time length of the scene based on the feature information obtained from the feature detection unit. Moving image list display device.   7. The moving image list display device according to claim 6, wherein the summary creating unit is a scene that clearly shows a difference from other moving images based on the feature information obtained from the feature detecting unit. A moving picture list display device characterized by not changing the playback time of the scene.   The moving image list display device according to claim 6, wherein the summary creation unit further adjusts the time length of the scene depending on whether the moving images included in the similar scene are the same moving image or different moving images. A moving image list display device.   2. The moving image list display device according to claim 1, wherein the summary creation unit collects scenes in which a number of similar scenes is smaller than a predetermined number based on the correlation between the scenes calculated by the correlation calculation unit. A moving image list display device for creating a summary of images.   2. The moving image list display device according to claim 1, wherein the summary creating unit determines a target reproduction time for each moving image, and based on the correlation between the scenes calculated by the correlation calculating unit, A moving image list display device that collects scenes that clearly show differences from the above in a reproducible manner for the target reproduction time and creates a summary of each moving image.   13. The moving image list display device according to claim 1, further comprising an operation input unit that receives designation of a specific moving image from a list of moving image groups, wherein the summary creation unit includes the operation For the moving image specified by the input unit, a summary is created independently of the correlation between the scenes calculated by the correlation calculation unit, and for the summary of other moving images, the summary of the specified moving image is included. A moving image list display device, wherein a moving image list display device is created by excluding a scene similar to an included scene.   14. The moving image list display device according to claim 1, wherein the accumulation unit detects and accumulates characteristic information in advance for a moving image, and the correlation calculation unit and the summary creation unit A moving image list display device, wherein the feature information of the moving image selected by the moving image selection unit is read from the storage unit.   A moving image composed of a plurality of pieces of information including video and audio; a storage unit that stores the feature information of the moving image; a selection unit that selects a plurality of moving images to be displayed in a list according to a predetermined condition from the storage unit; Using the feature information stored in the storage unit, for each scene of a plurality of moving images selected by the selection unit, a correlation calculation unit that calculates the correlation between scenes, and the correlation calculation unit Based on the correlation between the scenes and the feature information accumulated in the accumulation unit, a summary creation unit that creates a summary of each moving image selected by the selection unit, and created by the summary creation unit A moving image list display device comprising: a list display unit for displaying a summary list.

Images