KR101341808B1 - Video summary method and system using visual features in the video - Google Patents

Abstract

The present invention relates to an image summarization method and system using visual features in an image. More specifically, after detecting at least one video shot using a low-level feature vector from an image, a keyframe representing the detected video shot is generated. Detecting a key frame; A character detection step of detecting a face in the detected keyframe and clustering the keyframe in which the face is detected to detect a character; A keyframe including keyframes having a distance between keyframes less than a predetermined threshold based on the number of times the character is exposed in the keyframe in which the character is detected and the low-level feature vector of the keyframe in which the face is detected. A importance calculation step of detecting a cluster and calculating importance for each keyframe in the detected keyframe cluster; And a selection step of selecting and extracting corresponding key frames in order of high importance according to the number of key frames requested by the user. With this configuration, the image summarization method and system using the visual features in the image of the present invention can be applied to keyframes in the image using visual features such as characters in the image to be summarized, shot lengths of keyframes and color information. After calculating the importance and selecting keyframes based on the calculated importance, the image can be easily summarized.

Description

Video summary method and system using visual features in the video}

The present invention relates to an image summarization method and system using visual features in an image, and in particular, an image capable of efficiently summarizing an image using visual features such as characters included in the image, shot length of keyframes, and color information. The present invention relates to a method and system for summarizing images using my visual features.

Recently, with the rapid development of electronics, communication, and technology, and the rapid increase in image data due to the development of camera technology and the spread of smart phones, the amount of information generated from multimedia data such as image, video, and audio data has exploded. Increased. Accordingly, there is a growing demand for a technique for accurately and quickly extracting information desired by a user in using multimedia data.

In particular, such multimedia data is composed of various media such as still images, moving images, graphics, animations, sounds, music, and texts, and the information included in the multimedia data is much larger than conventional text-oriented information. However, it is difficult to provide an objective and easy retrieval environment for multimedia data using only existing annotations.

As described above, the prior art of an image summarization method and system using visual features in an image is as follows.

Prior art 1 relates to Korean Patent Registration No. 0792016 (December 28, 2007), which is a character based video summarization apparatus using audio and video information and a method thereof. The prior art 1 includes a speaker recognition unit that detects a speaker by speaker recognition using auditory information and provides a summary of a specific actor unit; A face recognition unit detecting a key frame in which a specific person appears through face region detection and face recognition using visual information; Video summarization unit for performing character-based video summarization using speaker-centered video summarization results using auditory information in the speaker recognition unit and face recognition results using visual information in the face recognition unit; In the data summary method, a video summary by character is provided using audio and video information.

In addition, the prior art 2, Korean Registered Patent No. 0708337 (April 10, 2007), relates to an automatic video summarizing apparatus and method using fuzzy-based OC-SVM. This prior art 2 reflects human subjective judgment for effective video summary generation, proposes a method for generating flexible video summary information that is flexible to the user's environment or requirements, and extracts important video segments from a given video. By extracting a series of key frames from it, you can see the content of the video at a glance and have instant access to the desired video scene.

In order to solve the above problems of the prior art, the present invention calculates the importance of the keyframe of the image by using visual features such as the characters included in the image, shot length and color information of the keyframe. To provide an image summarization method and system using visual features in an image that can easily perform image summarization based on the calculated importance.

According to an embodiment of the present invention, an image summarization method using a visual feature in an image detects at least one video shot using a low-level feature vector, and then represents a detected video shot. A key frame detection step of detecting a key frame; A character detection step of detecting a face in the detected keyframe and clustering the keyframe in which the face is detected to detect a character; A keyframe including keyframes having a distance between keyframes less than a predetermined threshold based on the number of times the character is exposed in the keyframe in which the character is detected and the low-level feature vector of the keyframe in which the face is detected. A importance calculation step of detecting a cluster and calculating importance for each keyframe in the detected keyframe cluster; And a selection step of selecting and extracting corresponding key frames in order of high importance according to the number of key frames requested by the user.

More preferably, it is determined whether there is a character in the detected video shot, and a video shot corresponding to a case where there is a character in the detected video shot is detected as a keyframe, or a character in the detected video shot. The key frame detection step of detecting the first frame of the detected video shot as a key frame, if there is no present.

In particular, it may include a low level feature vector that is a color histogram or a color correlogram.

More preferably, a face detection process of detecting a face based on the edge portion of the eyes, nose and mouth from the key frame; And a character detection process of detecting a character in the key frame by clustering the key frames in which the face is detected according to color information.

More preferably, the character distance calculation process of calculating the similarity between the characters according to the number of exposure of the characters in the key frame, the character is detected, and the distance between the characters in the two key frames according to the calculated similarity ; An inter-frame distance calculation process of calculating a distance between the two key frames based on the weights for the characters, the low-level feature vectors of the two key frames, and the distances between the characters in the two key frames; A hierarchical tree which detects a keyframe cluster including keyframes having a distance between the two keyframes less than a preset threshold and forms a hierarchical tree structure for keyframes in the detected keyframe cluster. Structure formation process; And an importance calculation step of calculating importance for each keyframe in the hierarchical tree structure.

More preferably, a hierarchical tree structure is formed using a semi Hausdorff distance algorithm to detect a keyframe cluster including keyframes having a distance between two keyframes having a distance smaller than a predetermined threshold. Process may be included.

In particular, a three-level hierarchical tree having one higher key frame, at least one lower key frame included in the upper key frame, and at least one lowest key frame included in the lower key frame. It may include a structure.

In particular, the shot length of the upper key frame may be a total of the shot lengths of at least one lower key frame included in the upper key frame.

In particular, the method may include an importance calculation process of calculating importance for each key frame by multiplying a distance between each key frame and the higher key frame in the hierarchical tree structure and a shot length of each key frame.

According to an embodiment of the present invention, an image summarization system using a visual feature in an image detects at least one video shot using a low-level feature vector from an image, and then represents a detected video shot. A key frame detector for detecting a key frame; A character detection unit detecting a face in the detected keyframe and detecting a character by grouping the detected keyframes; A keyframe including keyframes having a distance between keyframes smaller than a predetermined threshold value based on the number of exposures of the character in the keyframe in which the character is detected and the low level feature vector of the keyframe in which the face is detected. A importance calculator for detecting a cluster and calculating importance for each keyframe in the detected keyframe cluster; And a selection unit configured to select and extract the corresponding key frames in the order of high importance as the number of key frames requested from the user.

An image summarization method and system using a visual feature in an image of the present invention calculates the importance of the keyframe in the image using visual features such as characters in the image to be summarized, shot lengths of keyframes, and color information. By selecting and summarizing the keyframes based on the calculated importance, the image can be easily summarized.

In addition, the image summarization method and system using the visual features in the image of the present invention are more efficient by grouping the keyframes according to the color information included in at least one keyframe representing the image and removing a plurality of similar keyframes. There is an effect that can summarize the image.

1 is a block diagram of an image summary system using visual features in an image according to an exemplary embodiment.
2 is a flowchart of an image summarization method using visual features in an image according to another exemplary embodiment.
FIG. 3 is a flowchart illustrating a detailed process of the importance calculation step of FIG. 2.
4 is a diagram illustrating a video shot detection process for an image.
5 is a view showing a detection result of the character.
6 is a diagram illustrating exposure of a character in a keyframe.
7 is a graph showing the relationship between the appearance frequency of the characters and weights for the characters.
8 shows a hierarchical tree structure for keyframes.
9 is a view showing a summary image results using the present invention.
10 is a view showing the image summary results using the prior art and the present invention, respectively.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT Hereinafter, the present invention will be described in detail with reference to preferred embodiments and accompanying drawings, which will be easily understood by those skilled in the art. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein.

Hereinafter, an image summary system using visual features in an image according to an exemplary embodiment will be described in detail with reference to FIG. 1.

1 is a block diagram of an image summary system using visual features in an image according to an exemplary embodiment.

As shown in FIG. 1, the image summarization system 100 using the visual features in the image of the present invention includes a key frame detector 120, a character detector 140, a importance calculator 160, and a selector 180. Include.

The key frame detector 120 detects at least one video shot using a low level feature vector from the image, and detects at least one key frame representing the detected video shot.

The character detecting unit 140 detects the face of the character in the detected keyframe and performs clustering on the keyframe in which the face is detected.

The importance calculation unit 160 performs keyframes having a distance between keyframes smaller than a predetermined threshold based on the number of exposures of the characters in the keyframe in which the face is detected and the low level feature vector of the keyframe in which the face is detected. Detects a keyframe cluster comprising a plurality of keyframes, forms a tree structure for each keyframe in the detected keyframe cluster, and calculates importance for each keyframe formed in the tree structure.

The selector 180 selects and extracts the corresponding keyframes in order of high importance as the number of keyframes requested from the user.

Hereinafter, a method of summarizing an image using visual features in an image according to another exemplary embodiment will be described in detail with reference to FIG. 2.

2 is a flowchart of an image summarization method using visual features in an image according to another exemplary embodiment.

As shown in FIG. 2, first, the keyframe detector 120 detects at least one video shot in an image to be summarized using a low level feature vector, and then detects a keyframe representing the detected video shot. (S210). In this case, the keyframe detector 120 may detect at least one video shot using a low level feature vector such as a color histogram or a color correlogram.

Thereafter, the key frame detection unit 120 that detects the video shot detects a key frame representing each video shot from the detected video shot. For example, the key frame detection unit 120 first checks whether there is a character in the video shot detected, and if there is a character in the video shot, the frame in which the character exists is displayed in the video. Detects with keyframe of shot. However, in contrast, when there is no character in the video shot detected by the key frame detector 120, the first frame of the video shot is detected as a key frame representing the video shot.

As such, the detected video shot may detect at least one video shot that does not have a common feature or does not exist as shown in FIG. 4.

Thus, as can be seen through FIG. 4, it can be seen that the common characters or the common background do not exist in the detected video shots 1 to 5.

Thereafter, the character detecting unit 140 detects the face of the character in the detected keyframe, and performs a clustering of the keyframes in which the face is detected (S220). The character detection unit 140 is an edge corresponding to the obituary of the characters existing in the key frame from the key frame detected by the key frame detection unit 120, ie, eyes, nose, and mouth through the process S210. ) To detect the presence of a face. Thereafter, the character detection unit 140 detects the character by grouping the key frames where the face is detected according to color information. In such a clustering process, since a keyframe in which a human face is detected may repeatedly appear in a background having the same person or a similar color, when the face is clustered according to color information, the keyframes are present in the keyframe. Characters can be effectively detected.

As such, the keyframe in which the character is detected may appear as shown in FIG. 5.

5 is a view showing a detection result of the character.

As illustrated in FIG. 5, a face is detected from a plurality of keyframes using edges of eyes, nose, and mouth, and characters are displayed based on color information displayed on the face and the background of the detected keyframes. 1 to 4 can be detected.

In particular, it can be seen that the keyframes detected for each character all have the same character on a similar background, and the keyframes corresponding to the characters 1 to 4 have different backgrounds and characters for each character. It can be seen that it is included.

Subsequently, the importance calculator 160 has two keyframes having a minimum distance based on the number of exposures of the characters representing how many times the characters in the detected keyframes have been exposed and the low-level feature vectors of the faces detected. Is detected and the importance level for the detected keyframe is calculated (S230).

Hereinafter, referring to FIG. 3, the importance calculation for keyframes will be described in more detail.

FIG. 3 is a flowchart illustrating a detailed process of the importance calculation step of FIG. 2.

As shown in FIG. 3, the importance calculation unit 160 first calculates an exposure frequency to check how many times the characters in the key frame are exposed with respect to the detected key frame. Hereinafter, referring to FIG. 6, a detailed description will be given of a process of calculating the number of exposures of characters in a key frame.

6 is a diagram illustrating exposure of a character in a keyframe.

As shown in FIG. 6, when the key frames are detected from 1 to 6, the character 1 represents a, the character 2 represents b, the character 3 represents c, and the set of key frames in which the character 1 appears A, assume that the set of keyframes in which the character 2 appears is B, and the set of keyframes in which the character 3 appears is C. In addition, as can be seen from FIG. 6, the keyframes in which the character 1 appears are f ₂ , f ₃ , f ₄ , and f ₅ , and a set of such key frames is represented by A. FIG. Similarly, the keyframes in which the character 2 appears are f ₁ , f ₃ , f ₅ , the set of such keyframes is represented by B, the keyframes in which the character 3 appears are f ₄ , f ₅ , This set of keyframes is represented by C.

Accordingly, among the number of exposures for each character, the number of exposures corresponding to set A is n (A) = 4, and the number of exposures corresponding to set B is n (B) = 3, corresponding to set C. The number of exposures is n (C) = 2 and the number of times the character is not exposed is n (O) = 1. In addition, the number of characters 1 and 2 exposed simultaneously in the keyframe is n (A, B) = n (B, A) = 2, the number of characters 1 and 3 exposed simultaneously is n (A, C) = n (C, A) = 2, and the number of times characters 2 and 3 were exposed simultaneously is n (C, B) = n (B, C) = 1.

Then, the importance calculation unit 160 calculates the similarity between each of the characters by using the number of exposures to each of the characters through Equation 1 below.

[Equation 1]

S (A, B) =

S (A, C) =

S (B, C) =

S (A, O) =

Therefore, the importance calculator 160 calculates the distance P (A, B) between the characters in two key frames by applying the calculation result according to the similarity operation between each character to Equation 2 below (S231).

&Quot; (2) "

P (A, B) = 1-S (A, B)

Subsequently, the importance calculation unit 160 multiplies the value obtained by subtracting the weight α for the character from 1 with the low-level feature vector C (f _i , f _j ) between two keyframes, as shown in Equation 3 below. The weight α for is multiplied by the distance P (f _i , f _j ) between the characters in the two keyframes calculated by Equation 2, and then subtracted the weight α for the character from 1 The distance D (f _i , f _j ) between the two key frames is calculated by adding to the value multiplied by the low-level feature vector C (f _i , f _j ) between the two key frames (S232). This calculation process can be expressed as Equation 3 below.

&Quot; (3) "

D (f _i , f _j ) = (1-α) × C (f _i , f _j ) + α × P (f _i , f _j )

In this case, the low-level feature vector is calculated through C (f _i , f _j ) = correlogram (f _i ) -correlogram (f _j ), and represents a difference in color correlogram between two key frames. In addition, the weight α for the character used at this time depends on the frequency FP of the character. As shown in Equation 4 below, the frequency FP of the character is calculated by dividing the length of the entire character by the total length of the image and then percentage-creating the frequency of the character FP for the characters in the image. have.

&Quot; (4) "

The weight α for the characters according to the frequency of appearance of the characters calculated as described above is as follows. When the frequency count FP for the character calculated through Equation 4 is 50 or more, the weight α for the character is assigned 0.1, and if the frequency count for the character is FP less than or equal to 0.5, the character The weight α for is assigned 0.8. Or if the frequency FP for the character is greater than 0.5 and less than 50, the value obtained by subtracting a value obtained by multiplying the calculated frequency FP for the character by 0.02 by 1 is assigned as the weight α for the character.

The relationship between the appearance frequency and the weight of the character will be described with reference to FIG. 7.

7 is a graph showing the relationship between the appearance frequency of the characters and weights for the characters.

As shown in FIG. 7, the weight α for the character according to the appearance frequency FP of the character is 0.1 when the appearance frequency FP of the character is 50%, and the appearance frequency FP of the character is 5%. It can be seen that the weight α for the character is 0.8. At this time, the solid line graph of the graph shown in Figure 7 is the optimum value obtained through the experiment, the dotted line graph can be seen as an approximation value.

As such, the importance calculator 160 detects a keyframe cluster including keyframes having a distance smaller than a predetermined threshold as the keyframe 160 continuously calculates a distance between two keyframes previously calculated while changing a keyframe. In this case, the distance between two keyframes may be calculated using the Semi Hausdorff distance algorithm. In particular, the preset threshold may be 0.06 or 0.3, and the threshold may be changed by the user.

All keyframes are contained in one cluster of multiple clusters

Subsequently, the importance calculation unit 160 divides each keyframe in the keyframe cluster detected above into a hierarchical layer, and forms a tree structure (S233). Referring to FIG. 8, each key in the keyframe cluster is described below. Let's take a closer look at the hierarchical tree structure of frames.

8 shows a hierarchical tree structure for keyframes.

As shown in FIG. 8, the hierarchical tree structure for a key frame includes one higher key frame 1 and at least one lower key frame 2, 3, 4 included in the higher key frame 1. And at least three lowest level key frames 5 to 11 included in the lower key frames 2, 3, and 4, respectively. In other words, it can be seen that a hierarchical tree structure is formed for all keyframes.

In this case, the shot duration of the subkey frames 2, 3, and 4 is the shot length of at least one least significant key frame 5 to 11 included in the subkey frames 2, 3, and 4, respectively. Is equal to the sum of In addition, the shot length of the upper key frame 1 is also equal to the sum of the shot lengths of at least one lower key frame 2, 3, 4 included in the upper key frame 1.

For example, the shot length of the subkey frame 2 is 10, which is the shot length of the lowest key frame 5 included in the sub key frame 2, and the other lowest key included in the sub key frame 2, respectively. It can be seen that 25 is the sum of the shot length 15 of the frame 6. In addition, the shot length of the upper key frame 1 is 25, the shot length of the lower key frame 2 included in the upper key frame 1, and the other lower key frame included in the upper key frame 1. It can be seen that the sum of the shot length 30 of (3) and the shot length 5 of another lower key frame 4 included in the upper key frame 1 are 60, which is the sum of the sums.

As described above, the importance calculation unit 160 that calculates the hierarchical tree structure for the keyframe may use the above-described higher keyframe for all key frames included in the hierarchical tree structure. The multiplier distance D _i is multiplied by the shot length W _i corresponding to the key frame to calculate the importance IF _i for the key frame (S234). In this case, each keyframe in the keyframe cluster may be represented as a hierarchical tree structure.

&Quot; (5) "

IF _i = (D _i × W _i )

Accordingly, the selector 180 sorts the importance IF corresponding to each keyframe in descending order, and then selects and extracts the keyframes as many as the number of keyframes requested from the user in the order of high importance (S240).

Hereinafter, an image summary experiment to which the present invention is applied will be described.

Table 1 shows the image information used in this experiment. As shown in Table 1 below, a total of four program genres such as TV entertainment, drama, documentary, and news, and two images of each genre were adopted to conduct a video summary experiment. The selected image is about 30 minutes in average, and the summary result shows the storyboard with the number of still images you want arranged in chronological order.

Program title Length The frequency of characters Bit rate TV entertainment I'm a man 40 minutes 32% 30 fps Man's qualifications 40 minutes 35% 30 fps drama LOST 30 minutes 55% 30 fps All my love 35 minutes 60% 30 fps documentary World theme travel_
European alps 30 minutes 3% 30 fps World theme travel_
Mudangsan, the sacred place of Taoism 30 minutes 5% 30 fps news MBC News Desk 40 minutes 25% 30 fps CBS No Cut News 20 minutes 28% 30 fps

Comparing the number of events reported in the original image with the number of events included in the summary results, it can be seen that the average detection accuracy is about 90%.

In the original video
Number of events included The number of events included in the video summary results accuracy News 1
MBC News Desk 19 18 94% News 2
MBC noon news 10 9 90% News 3
KBS 9 News 25 22 88% News 4
CBS No Cut News 20 18 90%

Accordingly, as shown in FIG. 9 (a), it can be seen that the result of summarizing five images is shown. As shown in FIG. 9 (b), the result of summarizing 15 images is shown. It can be seen that indicates.

In addition, the user satisfaction was measured by comparing the experimental example applying the present invention and the experimental example applying the prior art.

Kim's Algorithm Shingo's Algorithm Marian's Algorithm Invention TV entertainment 3.5 3.6 4.0 4.5 drama 3.0 3.9 4.5 4.5 documentary 3.0 3.0 4.0 4.8 news 3.6 3.6 4.3 4.8

In order to measure the user satisfaction, 10 users were provided storyboards of the original image and the summarized still image, and the satisfaction level of 1 to 5 was measured for each image. As can be seen from Table 3, it can be seen that the image summary according to the present invention has a relatively high average satisfaction compared to the prior art used in the test.

In addition, it is also possible to confirm the difference in the image summary result between the prior art and the present invention through FIG. 10.

10 is a view showing the image summary results using the prior art and the present invention, respectively.

As shown in FIG. 10, the left image is a TV entertainment with a relatively high frequency of characters with 32% of the characters, and the right image has a frequency of less than 5% of the characters. This is a relatively low documentary video.

With respect to the left image (a), all 15 images summarized using the first prior art Marian method exist, and there is a person. The left image (b) summarized using the second conventional technology Shingo method is illuminated. It can be seen that a scene that is not helpful for grasping the contents of the image, such as a strong scene, is selected. In addition, since the left image (c) uses only color features in the image, when the color changes due to lighting, the adjacent shot may be repeated. However, the left image (d) to which the present invention is applied can properly improve the user's satisfaction by properly showing only the main scenes in which the characters and the scene change are certain.

In addition, the image on the right, which shows a documentary image having a frequency of appearance of less than 5%, is different from the (a) to (c) images using the conventional art, which repeatedly shows a similar natural environment. By properly including and summarizing interview scenes and other scenes in which characters are present, it can be seen that the user satisfaction is improved through effective video summarization even for documentary images having a relatively low frequency of characters.

In addition, an image summarization method and system using the visual features in the image may be stored in a computer readable recording medium having recorded thereon a program for execution by a computer. At this time, the computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer system is stored. Examples of the computer readable recording medium include ROM, RAM, CD-ROM, DVD 占 ROM, DVD-RAM, magnetic tape, floppy disk, hard disk, optical data storage, and the like. The computer readable recording medium can also be distributed over network coupled computer devices so that the computer readable code is stored and executed in a distributed fashion.

An image summarizing method and system using a visual feature in an image of the present invention calculates the importance of a keyframe in the image by using visual features such as characters in the image to be summarized, shot lengths of keyframes, and color information. By selecting and summarizing the keyframes based on the calculated importance, the image can be easily summarized.

In addition, the image summarization method and system using the visual features in the image of the present invention are more efficient by grouping the keyframes according to the color information included in at least one keyframe representing the image and removing a plurality of similar keyframes. There is an effect that can summarize the image.

While the present invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, Do.

120: key frame detection unit 140: character detection unit
160: importance calculation unit 180: selection unit

Claims (11)

A key frame detection step of detecting at least one video shot using a low level feature vector from the image, and then detecting a key frame representative of the detected video shot;
A character detection step of detecting a face in the detected keyframe and clustering the keyframe in which the face is detected to detect a character;
A keyframe including keyframes having a distance between keyframes less than a predetermined threshold based on the number of times the character is exposed in the keyframe in which the character is detected and the low-level feature vector of the keyframe in which the face is detected. A importance calculation step of detecting a cluster and calculating importance for each keyframe in the detected keyframe cluster; And
A selection step of selecting and extracting corresponding key frames in order of high importance according to the number of key frames requested by the user;
Image summary method using a visual feature in the image comprising a.
The method of claim 1,
The key frame detection step
Check whether there is a character in the detected video shot, and detect a video shot corresponding to the case where there is a character in the detected video shot as a keyframe, or if there is no character in the detected video shot. And a first frame of the detected video shot as a key frame.
The method of claim 1,
The low level feature vector
A method of summarizing an image using visual features in an image, characterized in that it is a color histogram or a color correlogram.
The method of claim 1,
The character detection step
A face detection process of detecting a face based on edges of eyes, nose and mouth from the key frame; And
A character detection process of detecting characters in the key frame by clustering the key frames in which the face is detected according to color information;
Image summarization method using a visual feature in the image, characterized in that it comprises a.
The method of claim 1,
The importance calculation step
Calculating distance between characters according to the number of exposures of the characters in the key frame in which the characters are detected, and calculating distances between characters in the two key frames according to the calculated similarity;
An inter-frame distance calculation process of calculating a distance between the two key frames based on the weights for the characters, the low-level feature vectors of the two key frames, and the distances between the characters in the two key frames;
A hierarchical tree which detects a keyframe cluster including keyframes having a distance between the two keyframes less than a preset threshold and forms a hierarchical tree structure for keyframes in the detected keyframe cluster. Structure formation process; And
A importance calculation process of calculating importance for each keyframe in the hierarchical tree structure;
Image summarization method using a visual feature in the image, characterized in that it comprises a.
The method of claim 5,
The hierarchical tree structure formation process
A visual feature in an image is detected by using a semi Hausdorff distance algorithm to detect a keyframe cluster including keyframes having a distance less than a predetermined threshold between two keyframes. Image summary method used.
The method of claim 5,
The hierarchical tree structure
An image comprising three levels, one upper key frame, at least one lower key frame included in the upper key frame, and at least one lowest key frame included in the lower key frame. Video summary method using my visual features.
The method of claim 7, wherein
And a shot length of the upper key frame is a sum of shot lengths of at least one lower key frame included in the upper key frame.
The method of claim 7, wherein
The importance calculation process
An image summarization method using a visual feature in an image, wherein the importance of each keyframe is calculated by multiplying a distance between each keyframe and the upper keyframe in the hierarchical tree structure by the shot length of each keyframe. .
A computer-readable recording medium having recorded thereon a program for executing a method according to any one of claims 1 to 9 with a computer.
A key frame detector for detecting at least one video shot using a low level feature vector from an image, and then detecting a key frame representative of the detected video shot;
A character detection unit detecting a face in the detected keyframe and detecting a character by grouping the detected keyframes;
A keyframe including keyframes having a distance between keyframes smaller than a predetermined threshold value based on the number of exposures of the character in the keyframe in which the character is detected and the low level feature vector of the keyframe in which the face is detected. A importance calculator for detecting a cluster and calculating importance for each keyframe in the detected keyframe cluster; And
A selection unit which selects and extracts corresponding key frames in order of high importance as the number of key frames requested by the user;
Image summary system using a visual feature in the image comprising a.

Images