JP3323842B2 - Video summarizing method and apparatus - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a computer system capable of processing moving image data, and more particularly to a method and an apparatus for summarizing moving image data in an image processing apparatus.

[0002]

2. Description of the Related Art Terrestrial digital broadcasting is about to start in 2003 following BS digital television broadcasting in 2000. At that time, a viewing method different from that of current analog broadcasting is conceivable.

[0003] In the conventional analog broadcasting, a user can watch a live in real time or record a video on a video deck set at a specific time, and can only find a recording start time.

[0004] The video which is the object of the present invention is an image which changes with time, and is synonymous with moving image data, video data and the like.

However, by the time digital broadcasting becomes widespread, a low-cost, large-capacity hard disk drive (HDD) is a set-top box (Set-Top), which is a receiver.
Box, STB) to record long-term video according to the user's preference. However, since the total viewing time of the user does not change significantly at any time, it is necessary to search for video efficiently, summarize it, and present it. Becomes even more important.

Here, a set-top box (ST)
B) is a general name of a device that provides an additional function by connecting to a television, such as a control device of a cable TV (CATV) or a terminal device of a communication karaoke. being called.

[0007] For example, when watching a sports video from the middle, a video expression such as summarizing and presenting the progress up to that point and catching up with a live video has been considered.

However, according to the conventional summarization method, the summarization result is determined by the broadcasting station that is the distributor of the content, and it is difficult to generate a summarization according to personal preference.

[0009] Here, the summarization is to generate a video compressed on the time axis from a target video. A sequence that is not interesting to the user is deleted from a long-time target video, and only a sequence that is interesting is left. It is a thing.

[0010] Video summarization requires a high-performance personal computer (PC) because of the need for content analysis. However, in digital broadcasting, which will start broadcasting in the future, data broadcasting will be distributed between video gaps. Therefore, a plurality of data prepared by the distributor can be transmitted by data broadcasting, and the user can freely combine the data to generate a summary suitable for each user.

Also, summarization using data processed by the STB having an image processing function is possible.

[0012] As a conventional video summarizing technique, there is an example in which a story board in which head images of scene changes in which pixel values of continuous frames find a large difference in a frame are arranged.

However, the story board itself cannot be a summary because it does not convey the contents of the scene.

[0014] A scene transition graph (1) in which a similarity between images of a story board is obtained and a series of scenes are aggregated by a relation graph [1]
"M. Yeung, BL Yeo, etc.," Extracting story units
from long programs for video browsing and navigat
ion, "IEEE ICMCS, pp. 296-305, 1996." can convey the structure of video, but is not a summary.

In the summarizing technique, a video skimming method (Vi
deo Skimming) [2] "M. Smith and T. Kanade," Vid
eo skimming and characterizationthrough the combin
ation of image and language understanding techniqu
es, "IEEE CVPR, pp. 775-781, 1997." shows effectiveness using not only video but also audio and closed captions, but there is no learning function to reflect personal preferences .

In the United States, STB has begun to spread, and models having a hard disk drive (HDD) capable of recording for a long time have been announced. MbTV, Tivo, etc. are leading manufacturers and personal preference The Bayes method is used as an algorithm for accumulating video images (see [3] “MbTV, Preference Determination Engin”).
e ").

However, these methods are methods of accumulating in units of video titles, and do not summarize the contents as in the present invention described later.

According to the present invention, a plurality of data sent by a distributor or data processed by a user's STB can be used by a user to weight data that emphasizes a scene desired to be viewed according to personal preference. .

As types of data representing the contents of video, there are semantic data which directly represents the meaning of a scene and numerical data (numerical data) which represents the strength of a signal in the video.

The semantic data expresses the content of each scene included in the video by words, and the numerical data expresses the content of each scene included in the video by numerical values.

Here, a scene is generally a frame or a sequence of frames to which semantic interpretation can be given.

For the user, the relationship between the input and the result is clear for semantic data, but it is difficult to understand how it works in the case of numerical data.

Since the semantic data needs to interpret a video scene, a fine-grained annotation is required for summarizing based only on the semantic data.

Normally, since semantic data is manually input, it takes time and money to annotate a video with fine-grained and detailed annotations. To associate a plurality of parameters of the semantic data,
An accurate definition statement is required.

Since not all images are accompanied by fine-grained semantic data, it is necessary to summarize only the numerical data when there is no semantic data.

What kind of numerical data is effective is
Weights can be assigned to scenes that have been adopted in the past.

According to the present invention, the parameter potential of the numerical data can be changed depending on whether each scene is adopted or skipped, and the video importance of each user can be set. An approximate summary video can be obtained.

According to the present invention, it is possible to extract only a video including a scene that the user likes from a long video, and to enjoy an interesting video in a short viewing time.

The data attached to the video has a smaller data capacity than the video and can be summarized by an inexpensive STB so that it can be transmitted by data broadcasting.

If the data attached to the video is not enough, the video needs to be processed by the STB. However, if the processed data is used, summarization can be performed with the same configuration.

[0031]

One of the problems to be solved by the present invention is that the importance for summarizing a long video into a short digest desired by a user can be changed for each user. Is to provide functionality.

[0032]

SUMMARY OF THE INVENTION The present invention is an apparatus for creating a summary from a video, means for extracting numerical data from a target video, and determining the importance of each scene based on the extracted numerical data. Means for creating a summary from a portion where the importance is equal to or greater than a predetermined threshold.

[0033]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS As an embodiment of the present invention, a summary of a live broadcast of soccer will be introduced below.

At present, since a broadcasting station does not add information such as semantic data attached to a video, image processing for summarization is performed by the user himself.

During the absence of the user, S implementing the present invention
The TB accumulates (records) a soccer video and executes predefined image processing (tracking, motion classification, camera motion detection, player position estimation), and extracts image features (parameters).

At the same time, the parameter potential and the two-dimensional potential
The video importance of the broadcast is determined using the graph.

If an uninteresting summary appears while watching, the video is fast-forwarded and the scene is skipped.

When the user finishes viewing, the STB votes and adds the parameters corresponding to the scene whose summary has been accepted to the parameter potential and the two-dimensional parameter graph in order to learn the viewing behavior of the user on that day. , Parameters corresponding to the scenes for which the summarization was skipped are defined as parameter potentials and two-dimensional potentials.
Subtract from the graph.

If the broadcasting station is transmitting semantic data such as semantic data accompanying the video,
Before generating the summary, the user can input a desired keyword (player name, type of play, etc.) and add it to the video importance, but this is not essential.

First, when the user does not have a parameter potential for determining the importance of the individual, the video importance sent by the video distributor is used as a sample,
It is necessary for the user to input a keyword of a scene that the user wants to see individually to generate a summary, or to generate a sample of a personal summary using fast-forwarding of a video.

In the sample, the importance is given as a one-dimensional graph for each time, and its value is a real number from 0 to 1, and an annotation describing a scene is attached to a time zone having a high importance ( (Fig. 1).

Next, the user selects the desired annotation from the semantic data, if any, to change the sample video importance, and view the digest video automatically obtained from the changed importance. The user selects whether to adopt while skipping or skipping (when adopting, there is no need to do anything, and only when skipping, fast forward is selected).

When the user finishes watching the video, the weight of the numerical data corresponding to the latest video importance is calculated.

Assuming that the value of numerical data (Xi) included in the adopted image is ki, Xi is the average of ki,
Vote the value of the normal distribution with variance σ∧2 to the parameter potential.

Next, when the value of the numerical data (Xi) is hi in another time interval, the voting value is similarly added, and the parameter potential for the numerical data (Xi) is as shown in FIG.

The number of additions S and X in the digest video
A threshold Th at which the sum of the parameter potentials at i becomes 0 is obtained, and the parameter potentials are normalized to 0 to 1.

The parameter / potential is similarly calculated for the other numerical data (Xn).

When the number of additions S is small, the value of the parameter potential varies, so the threshold value Th has a large value. However, when S is large and the relevance of the selected scene and parameter increases, the parameter potential becomes sharp. At this time, the threshold value Th becomes smaller.

As will be described later, when determining the degree of importance, the sum of the difference between the potential value and the threshold value is obtained, so that the sharpened potential value is emphasized.

When a plurality of numerical data are used, a two-dimensional potential graph is generated in order to associate the plurality of numerical data with each other.

Numerical data (Xi) in the same frame
When the values of and (Xj) are ki and pj, respectively, the average is k
Add a two-dimensional normal distribution with variance σ∧2 at i and pj.

The threshold Th2 is determined so that the sum of the potentials becomes 0, and the two-dimensional potential graph is normalized to 0-1. n (n-1) for n kinds of numerical data
/ 2 two-dimensional potential graph can be generated.

On the other hand, when the user skips the video,
The parameter potential is changed by means of reinforcement learning so that the video is not actively adopted thereafter.

Here, the reinforcement learning is generally a method of proceeding the learning by rewarding the success determined by the external system.

Assuming that the value of numerical data (Xi) included in the skipped video is ki, Xi is an average of ki, σ is a variance, and p (ki) / 2 times the potential p (ki) of ki. Reduce the normal distribution of.

The potential graph is changed by similarly subtracting the other numerical data (Xn) and the two-dimensional potential graph based on a plurality of numerical data.

Next, the importance is determined using the parameter potential and the two-dimensional potential graph.

The sum of (potential value-Th) of the numerical data of the image in each frame and (potential value-Th2) of the two-dimensional potential graph is obtained, and the importance is determined by the sigmoid function.

T = c / S in the sigmoid function (c:
As a constant), the certainty of importance is improved by the number of additions.

As described above, the user having the parameter potential can set the individual video importance from the potential graph without using the importance of the sample from the next time. By selecting “adopt” or “skip”, the potential graph is changed, and the preference of the user is strongly reflected.

The user can change the degree of importance obtained from the potential graph using the semantic data.

When the user inputs a keyword that matches the annotation of the semantic data, the importance of the corresponding time section is maximized, and a summary based on the importance is performed.

The input situation at this time is also reflected in the potential graph.

The above is the outline of the processing flow of the present invention.
This will be described in more detail below with reference to the drawings.

FIG. 3 is a flowchart showing the overall flow of the present invention.

In step 110, the process of the present invention is started.

In step 120, the importance is calculated.

At step 130, screen display and user selection processing are performed.

At step 140, it is checked whether or not to update the parameter potential. As a result of the check, if the parameter potential is to be updated, the process proceeds to step 170, and the parameter potential is not updated. In this case, the process proceeds to step 150.

In step 170, the parameter potential is updated using the reinforcement learning method.

Updating the parameter potential
This is done at the end of this content or when the user wants to update.

In step 180, a parameter / potential is selected.

When the parameter potential of the user does not exist, the default value (for example, determined by the content distributor) is used, and when the parameter potential of the user exists, it is used.

At step 150, it is checked whether all frames have been completed.

For example, it is possible to end when all of one content has been viewed.

At step 160, the next frame is taken out, and the routine goes to step 120.

At step 190, the process ends.

FIG. 4 is a flowchart illustrating the importance calculation process (step 120) of FIG. 3 in more detail.

At step 210, the importance calculation process is started.

In step 220, the value of p (Xi) -Th_i is calculated for the numerical data Xi (i = 1, 2,... N) in the current frame.

At step 230, a set of numerical data (Xi, Xj) (1 <= i, j <
= N, all combinations)
Calculate the value of (Xi, Xj) -Th2_ij.

In step 240, step 220
And the sum (sum_p) of all the values obtained in 230.

[0083]

(Equation 1)

In step 250, importance is obtained by a sigmoid function.

Here, the jigmoid function is given by the following equation.

[0086]

(Equation 2)

FIG. 5 is a flowchart illustrating the image display and the user selection / voting process (step 130) of FIG. 3 in more detail.

At step 310, image display and user selection / voting processing (step 130) are started.

In step 320, it is determined whether or not to summarize. If the summary is to be performed, the process proceeds to step 330; otherwise, the process proceeds to step 340.

If it is determined in step 330 that the degree of importance is less than the predetermined threshold value Th_Imp, this scene is not considered important, and the process skips to step 370.

On the other hand, if the degree of importance is equal to or greater than the predetermined threshold value Th_Imp, this scene is not skipped and step 3
Proceed to 40.

At step 340, the screen of the current frame is displayed.

At step 350, the user who is looking at the current screen wants to see this scene (Accept)
A selection (determination) is made as to whether the user does not want to see (Reject) or neither (Neutral).

As a result of this judgment, the user wants to see (Accept)
Or, if you do not want to see (Reject), step 3
Proceed to 60, and if neither (neutral), proceed to step 370.

At step 360, the user wants to see the numerical data Xi (i = 1, 2,... N) of the current frame (Ac
Vote and store according to the result of “cept” or “Reject”.

FIG. 6 is a flowchart illustrating the parameter / potential update process (step 170) by reinforcement learning in FIG. 3 in more detail.

In step 410, a parameter / potential update process by reinforcement learning is started.

In step 420, voting data (numeric data Xi (i = 1, 2,... N)) is extracted.

In step 430, the extracted voting data is desired to be viewed (Accept) or not desired (R
eject), and if it is desired to see (Accept), the process proceeds to step 440, and the user does not want to see (Reject).
In the case of t), the process proceeds to step 450.

At step 440, if the value of Xi is ki
Then, the value of the normal distribution with the mean ki and the variance σ ＾ 2 is added to the parameter potential of Xi, and i = 1 to n
Repeat the same process until.

At the same time, if the values of (Xi, Xj) are (ki, pj), respectively, the mean (ki, pj) and the variance σ
The two-dimensional normal distribution of ＾ 2 is added to the potential graph of (Xi, Xj), and all (i, j) (i = 1 to n, j
= 1 to m), the same processing is repeated.

In step 450, the value of Xi is set to ki
Then, a normal distribution of p (ki) / 2 is subtracted from the parameter potential of Xi with respect to the potential p (ki) of ki, and the same processing is repeated until i = 1 to n.

At the same time, if the values of (Xi, Xj) are (ki, pj), respectively, the average (ki, pj) and the variance σ
And the potential p (ki, pj) of (ki, pj)
, The two-dimensional normal distribution of p (ki, pj) / 2 is subtracted from the potential graph of (Xi, Xj), and the same process is repeated for all (i, j) pairs.

In step 460, the sum of all the obtained parameter potentials is 0.
The threshold value Th1_i is obtained such that

Similarly, the threshold value Th2_ij is obtained for the two-dimensional potential graph and normalized to 0-1.

In step 470, the parameter / potential update processing by reinforcement learning ends.

FIG. 7 shows a PVR (persponal Video Recorder) as an example of hardware suitable for the present invention.
Are shown in blocks.

Block 510 is a video storage unit.
A large-capacity storage device such as a hard disk device or a video tape recorder capable of storing video data for a long time is used.

A block 520 is software to which the present invention is applied.

Block 530 is a video buffer for temporarily storing the video summarized based on importance, and a storage device such as a hard disk device is used.

The video data 505 is input to and stored in the video storage unit 510 of the PVR.

The video storage section 510 sequentially stores the input video data 505.

The video data stored in video storage section 510
The data is processed by the software 520 to which the present invention is applied, and a summary is created according to the importance.

The summarized video data is temporarily stored in video buffer 530 and output as video output 535 when desired by the user.

On the other hand, the video data stored in the video storage
The data 515 can be output as a video output without being summarized.

FIGS. 8 to 10 show examples in which the present invention described above is applied to actual soccer images.

As the user interface of the present invention, a window 800 as shown in FIG. 8 can be considered.

The window 800 displays the image display area 85
0, a learning area 810, a summary area 820, and the like.

“Start” button 804
By clicking, the target video is displayed in the video display area 8
The display is started at 50, and the displayed image is stopped by clicking a “Stop” button 806, and the displayed image is paused by clicking a “Pause” button 808.

First, the learning method performs the following processing.

While looking at the screen, the user clicks an “Accept” button 812 if the image is an image that the user wants to emphasize, and clicks a “Reject” button 814 if the image is not an important image.

By clicking the “Update” (update) button 816, the parameter potential is updated, and the user's preference is reflected.

Although the user cannot see the parameter / potential itself, by seeing the numerical value of the importance 818 on each screen, it is possible to confirm whether or not the user's preferences are appropriately reflected.

Next, the summarization method performs the following processing.

When the radio button is set to “On” 822
Thus, the importance is calculated from the current frame to the last frame, and the target video is summarized based on the calculation result of the importance.

The display area 830 shows the result of summarization, with white portions representing scenes to be skipped and black portions representing scenes to be summarized.

That is, when displaying the summary, only the scene in the black portion is displayed.

Further, even during the summary display, the user can select “Accept
By clicking the “t” button 812 or the “Reject” button 814 as appropriate, the preference can be further reflected.

In the summary area 820, a threshold 826 and a summary rate (%) 828 can be displayed.

The threshold 826 indicates the lowest importance of a scene to be adopted as a summary.

The summarization rate (%) is a ratio of the display time of the target video to the display time of the summary video expressed as a percentage.

FIG. 9 shows an example of numerical data used in the application example of soccer shown in FIG.

As numerical data representing a soccer game as shown in FIG. 8, camera motion (camera direction), area division result, average color information, parameter correlation of these numerical data, and the like are used.

As the camera motion, two types of values, that is, pan (right and left direction (angle)) and tilt (up and down direction (angle)) are used.

FIG. 9 shows parameter potentials 910 for panning (left and right directions) of camera motion.

The horizontal axis 912 represents the direction (angle) of the pan. The left side is the angle -90 degrees and the right side is the angle +90 degrees.

The vertical axis 914 represents a numerical value corresponding to each pan angle.

In the parameter potential 910, the one-dot broken line 920 indicates the parameter potential before voting, the broken line 918 indicates voting data by the user, and the solid line 916 indicates the parameter potential after voting.

As the area division result, three values of the area (S) of the maximum area and the color (rg, by) of the maximum area are used.

FIG. 9 shows a parameter potential 930 for the area (S) of the maximum area as a result of the area division.

The horizontal axis 932 indicates the area (S) of the maximum area.
Where the area is 0 on the left side and the area increases as going to the right side.

The vertical axis 934 shows a numerical value corresponding to the area (S) of each maximum area.

In the parameter potential 930, the one-dot broken line 940 indicates the parameter potential before voting, the broken line 938 indicates voting data by the user, and the solid line 936 indicates the parameter potential after voting.

As average color information, R (red), G (green),
Three values of B (blue) are used.

FIG. 9 shows the parameter potential 950 for the average color information G (green).

The horizontal axis 952 represents the average color information G (green), where the left has an area of 0 and G increases toward the right.

The vertical axis 954 represents a numerical value corresponding to each average color information G (green).

In the parameter potential 950, the one-dot broken line 960 represents the parameter potential before voting, the broken line 958 represents voting data by the user, and the solid line 956 represents the parameter potential after voting.

Further, 28 kinds of correlation values of the above eight parameters are used.

Therefore, in the example of soccer shown in FIG. 8, a total of 36 kinds of numerical data are used for determining the importance.

However, the type of numerical data used to calculate the degree of importance differs depending on the video to be processed, and the accuracy of summarization also changes by changing the type of numerical data.

As the numerical data of camera motion (camera direction), area division result, and average color information, it is possible to use original data or feature data obtained by performing principal component analysis on the data.

However, if the original data is used as it is, since the correlation between the data is ignored, the contributing data and the non-contributing data are treated on the same scale. For the highly correlated data, double voting is performed. There is a problem that the same effect may be obtained.

Therefore, a principal component analysis is performed on these original data, and voting is performed by replacing the original data with a feature amount that reduces the correlation between the data.

Here, the principal component analysis is to represent n-dimensional data with a small number of indexes independent of each other.

In the principal component analysis, an n-order parameter group is represented by m-th (m <n) feature quantities arranged in the order of the smallest correlation.

Each data shown in FIG. 9 represents a state of voting when the original data is used.

FIG. 10 shows the degree of importance of each scene of the video obtained from the numerical data described with reference to FIG.

In FIG. 10, the horizontal axis 1002 indicates time, and the vertical axis 1004 indicates importance.

A threshold value of importance suitable for creating a desired summary is indicated by reference numeral 1006.

That is, the portion of the target soccer video having the importance not less than the threshold is included in the summary (scenes A, B, C, and D), and the portion having the importance less than the threshold (shaded portion). Will not be included in the summary and will be cut.

As described above, in the case of the example shown in FIG. 10, a summary whose time is reduced to about half is created.

[0163]

According to the configuration of the present invention, it is possible to create a video summary according to the user's preference.

Further, it is possible to reflect the user's preference in the summary while watching the video.

The following is a summary of another embodiment.

(1) A method of creating an abstract from a video, comprising the steps of extracting numerical data from a target video, determining the importance of each scene based on the extracted numerical data, Creating a summary from portions whose importance is greater than or equal to a predetermined threshold. (2) A method of creating an abstract from a video, comprising the steps of: extracting numerical data from a target video; and, if the target video includes semantic data, adding numeric data to the semantic data and the numerical data. Determining the importance of each scene based on the numerical data, and determining the importance of each scene based on the numerical data if the target video does not include semantic data; Creating a digest from the portions above the threshold. (3) A method of creating a summary from a video, wherein a plurality of types of numerical data are extracted from a target video, and importance of each scene is determined based on the extracted plurality of types of numerical data. And summarizing from portions where the importance is greater than or equal to a predetermined threshold. (4) A method of creating an abstract from a video, comprising the steps of: extracting numerical data from the target video; receiving data representing user preference while displaying the target video; A method comprising: determining importance of each scene based on data and data representing the user's preference; and creating a summary from portions where the importance is greater than or equal to a predetermined threshold. (5) A method of creating an abstract from a video, comprising the steps of: extracting numerical data from a target video; determining the importance of each scene based on the extracted numerical data; A method comprising: creating a summary from a portion equal to or greater than a predetermined threshold; displaying the created summary; and changing the importance by user input while displaying the summary. (6) An apparatus for creating a summary from a video, a means for extracting numerical data from a target video, a means for determining the importance of each scene based on the extracted numerical data, Means for creating a digest from portions that are greater than or equal to a predetermined threshold. (7) An apparatus for creating an abstract from a video, comprising: means for extracting numerical data from a target video; and, when the target video includes semantic data, the means for extracting the semantic data and the numerical data. Means for determining the importance of each scene based on the numerical data, and determining the importance of each scene based on the numerical data when the target video does not include semantic data; Means for creating a summary from the portions above the threshold. (8) a device for creating a digest from a video, a means for extracting a plurality of types of numerical data from a target video;
An apparatus comprising: means for determining importance of each scene based on the extracted plurality of types of numerical data; and means for creating a summary from a portion where the importance is equal to or greater than a predetermined threshold. (9) An apparatus for creating a summary from a video, a unit for extracting numerical data from a target video, a unit for receiving data indicating user's preference while displaying the target video, and the extracted numerical value Apparatus comprising: means for determining importance of each scene based on data and data representing the user's preference; and means for creating a summary from portions where the importance is equal to or greater than a predetermined threshold. (10) An apparatus for creating a digest from a video, means for extracting numerical data from a target video, means for determining the importance of each scene based on the extracted numerical data, Means for creating a summary from a portion equal to or greater than a predetermined threshold, and means for displaying the created summary,
Means for changing the importance by user input while the summary is being displayed. (11) A system for creating a summary from a video, the system including a processor, a memory, a storage device for storing an input video, and a video summary device, wherein the video summary device includes a target A means for extracting numerical data from the video, a means for determining importance of each scene based on the extracted numerical data, and a means for creating a summary from a portion where the importance is equal to or greater than a predetermined threshold. system. (12) A recording medium for storing a program for creating an abstract from a video, wherein the program extracts numerical data from a target video, and determines importance of each scene based on the extracted numerical data. A recording medium comprising: a step of determining; and a step of creating an abstract from a portion where the importance is equal to or greater than a predetermined threshold.

[Brief description of the drawings]

FIG. 1 is a diagram showing the relationship between the importance of a sample video and a scene.

FIG. 2 is a diagram showing voting of effective parameters to parameter potential.

FIG. 3 is a flowchart showing an entire process of the present invention.

FIG. 4 is a flowchart illustrating an importance calculation process according to the present invention.

FIG. 5 is a flowchart showing screen display and user selection processing of the present invention.

FIG. 6 is a flowchart illustrating a parameter potential updating process by reinforcement learning according to the present invention.

FIG. 7 is a diagram showing a configuration of a PVR to which the present invention is applied.

FIG. 8 is a diagram showing an application example of the present invention to a soccer video.

FIG. 9 is a diagram showing an example of numerical data used in the present invention.

FIG. 10 is a diagram showing a relationship between a scene and a summary when the present invention is applied.

[Explanation of symbols]

 500 PVR 505 Video input 510 Video storage unit 520 Video summarization unit 530 Video buffer 535 Video output

──────────────────────────────────────────────────続 き Continued on the front page (72) Inventor Tomita Alberto 1623-14 Shimotsuruma, Yamato-shi, Kanagawa Prefecture IBM Japan, Ltd. Address 14 IBM Japan, Ltd. Tokyo Research Laboratory (56) References JP-A-2000-350165 (JP, A) JP-A-2000-299829 (JP, A) (58) Fields studied (Int. . ^7, DB name) H04N 5/76 - 5/956 H04N 5/262 - 5/278

Claims (8)

(57) [Claims] 1. A method for creating a digest from a video, comprising the steps of: extracting numerical data from the target video; and, if the target video includes semantic data, the semantic data and the numerical value. Determining the importance of each scene based on the data, and, if the target video does not include semantic data, determining the importance of each scene based on the numerical data; Creating a summary from portions that are greater than or equal to a predetermined threshold. 2. A method for creating a digest from a video, the method comprising: extracting numerical data from a target video; receiving data representing a user's preference while displaying the target video; Determining the importance of each scene based on the obtained numerical data and data representing the user's preference; and generating a summary from a portion where the importance is equal to or greater than a predetermined threshold. 3. A method for creating an abstract from a video, comprising: extracting numerical data from a target video; determining the importance of each scene based on the extracted numerical data; A method comprising: creating a summary from a portion whose degree is equal to or greater than a predetermined threshold; displaying the created summary; and changing the importance by user input while displaying the summary. 4. An apparatus for creating a summary from a video, comprising: means for extracting numerical data from a target video; and, when the target video includes semantic data, the semantic data and the numerical value. Means for determining the importance of each scene based on the data, and when the target video does not include semantic data, means for determining the importance of each scene based on the numerical data; Means for creating a summary from portions that are greater than or equal to a predetermined threshold. 5. An apparatus for creating a digest from a video, comprising: means for extracting numerical data from a target video; means for receiving data representing user's preference while displaying the target video; A means for determining the importance of each scene based on the obtained numerical data and data representing the user's preference; and a means for creating a summary from a portion where the importance is equal to or greater than a predetermined threshold. 6. An apparatus for creating a digest from a video, comprising: means for extracting numerical data from a target video; means for determining importance of each scene based on the extracted numerical data; An apparatus comprising: means for creating a summary from a portion having a degree equal to or greater than a predetermined threshold; means for displaying the created summary; and means for changing the importance by user input while the summary is being displayed. 7. A system for creating a summary from a video, the system comprising: a processor, a memory, a storage device for storing an input video, and a video summary device, wherein the video summary device comprises: Means for extracting numerical data from the target video, and when the target video includes semantic data, determine the importance of each scene based on the semantic data and the numerical data. In the case where the video does not include semantic data, means for determining importance of each scene based on the numerical data, and means for creating a summary from a portion where the importance is equal to or greater than a predetermined threshold value are included. system. 8. A computer-readable recording medium for storing a program for creating an abstract from a video, wherein the program causes a computer to extract numerical data from a target video; Receiving data representing the user's preference during display; determining the importance of each scene based on the extracted numerical data and data representing the user's preference; and wherein the importance is greater than or equal to a predetermined threshold. Creating a summary from the parts;