JP2002199332A - Audio summary information, and extraction device therefor, reproduction device and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide audio summary information and the extraction device therefor, the reproduction device and the recording medium, which extracts summary information having a length designated by audio or audio video information uniformly and speedily from the whole contents. SOLUTION: A contents analytic part 1 analyzes the hourly structure of video information of inputted audio video contents. An audio level evaluation part 2 evaluates the audio level of audio information attached to the video information the hourly structure of which is analyzed. A movement activity evaluation part 3 evaluates the movement activity of a shot which is evaluated to be the candidate of the summary information by the part 2. A summary information registering part 4 registers audio video summary information evaluated by the movement activity evaluation part.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to audio information for extracting, from uncompressed or compressed audio information, or compressed audio / video information, summary information (summary) for efficiently grasping the contents thereof. And a recording medium for extracting audio / video summary information. Also, to provide high-speed and efficient browsing of audio information or audio-video information by extracting summary information at high speed in a compressed data area from uncompressed or compressed audio information or compressed audio-video information. Which is capable of reproducing audio-video summary information.

[0002]

2. Description of the Related Art Automatic summarization of videos is described, for example, by Tanaka, Wakimoto and Kanda in "Development of Automatic Summarization / Browsing Technology of Video Information by Scene Detection", IEICE Technical Report, IE99-20 (1999). Research presentation (first conventional technology)
In this research presentation, after detecting a scene change point of a video, a hierarchical structure is formed, and a video summary is automatically created by giving a priority to each scene. In particular, scenes immediately after the turning point of the topic are given a high priority. In the hierarchical structure, the upper layer is set so that the priority is higher.

[0003] Also, "Usi" by J. Saarela and B. Merialdo
ng content models to build audio-video summarie
s '', SPIE Conference on Strorage and Retrieval for
Imageand Video Databases VII (1999) (second prior art) considers the creation of general video summary information (summary) as a restricted optimization problem. Restrictions include minimum shot length, audio and video synchronization, video continuity, and audio and video redundancy. Then, the content model (symbol description) of the video is manually constructed, and a summary is created.

Also, "Video Abstra" by R. Lienhart et al.
cting ”, Communications of ACM, Vol. 40, No. 12 (199
In (7) (third prior art), the purpose is to create summary information specialized for a movie trailer. The main creation procedure is to split the video into shots, analyze clips including special events, select clips, and aggregate clips. Special events include recognition of actor faces
It identifies conversations, extracts textual information from titles, and extracts events such as shootings and explosions.

[0005]

The extraction of the outline information of the audio / video according to the prior art has the following problems. First, in the first prior art,
Although video summarization provides efficient summarization, it must be done manually, and for long audio / video information, it can be used as a pre-process required to extract summary information. , Can take a lot of time. In addition, since the assignment of the priority is performed depending on the hierarchy to which the scene belongs, it is often necessary to substantially require human labor. Also, since it is intended for video alone, it can be extended to audio video, but since it does not use the characteristics as audio, in some cases it does not use audio information containing important contents, It is possible that appropriate summary information cannot be obtained.

In the second prior art, the construction of the content model of the video must be performed manually, and no method for effectively utilizing the characteristics of the audio is adopted. At the same time, the classification of video segments requires advanced recognition technology, which is difficult to realize on compressed data, and therefore, the processing required to extract summary information is expected to increase. The above-mentioned third prior art also requires these advanced processes (entering into the meaning of the contents).

As described above, in the prior art, in the process from input of audio-video information to output of summary information,
In many cases, manual processing is involved, and audio information is not effectively used. Therefore, generation of summary information based on audio characteristics is not considered. Also,
As for automatic extraction of summary information from compressed or uncompressed audio alone, no effective method has been studied so far.

Also, from the structural aspect of the outline information, there is no guarantee that the outline information is uniformly extracted from the entire audio / video information. Further, in the first and second prior arts, the outline information is specified from outside. A method of performing control to approach the summary information length described above has not been sufficiently adopted.

SUMMARY OF THE INVENTION The present invention has been made in view of the above-mentioned prior art, and has as its object to efficiently grasp the contents of uncompressed or compressed audio information or compressed audio / video information. In the apparatus for extracting summary information (summary), the spatio-temporal characteristics of audio and video are analyzed in the compressed data area, and if necessary, these are integratedly evaluated, and audio or audio-video information is specified. An object of the present invention is to provide an audio outline information and audio video outline information extraction device, a reproduction device, and a recording medium that enable uniform and high-speed extraction of outline information having a length.

[0010]

In order to achieve the above-mentioned object, the present invention provides an audio-video summary information extracting apparatus for analyzing the temporal structure of video information of input audio-video content. Audio level evaluation means for evaluating the audio level of audio information associated with the video information whose temporal structure has been analyzed, and summary information registration means for registering audio / video summary information evaluated by the audio level evaluation means The first feature is that it has

According to the present invention, there is provided an audio outline information extracting apparatus, comprising: means for extracting sub-band data from input compressed audio content; and calculating a sum of sub-band energies weighted by a band from the sub-band data. The second characteristic lies in that the present invention comprises means for performing summing, a means for evaluating the sum of sub-band energies per unit time, and a means for extracting audio summary information.

Also, the present invention provides a function of analyzing a temporal structure of video information of input audio / video content and a function of evaluating an audio level of audio information accompanying the video information whose temporal structure has been analyzed. When,
A computer-readable recording program for causing a computer to execute, including a function of evaluating motion activity in an image, and a summary information registration function of registering audio-video summary information on which the audio level and the motion activity have been evaluated. A third feature is that a recording medium is provided.

A fourth feature of the present invention is that the audio-video summary information reproducing apparatus includes means for synchronously reproducing the video element and the audio element extracted by the audio-video summary information extracting apparatus. There is.

[0014]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below in detail with reference to the drawings. FIG. 1 is a block diagram illustrating a configuration of an audio-video summary information extraction device according to a first embodiment of the present invention.

First, compressed audio / video information A
V is input to a content analysis unit 1 that analyzes the temporal structure of the audio / video information AV. The content analysis unit 1 has a configuration as shown in FIG. 2, and the compressed audio / video information AV is first input to the shot division unit 11. The shot division unit 11
Divides video information into shot units S, then counts the number of shots for the entire content, and holds the total number of shots as NS. Next, the division number determination unit 12
Is the total number of shots N input from the shot division unit 11
Using S, the content length CL obtained from the audio-video information AV, and the outline information length SL specified from the outside, the division number NP is determined by the following equation (1).

NP = NS × SL / CL (1) Here, it is assumed that SL> CL / NS. For example, when the content length CL is 1 hour and the summary information length SL is 5 minutes, when the total number of shots NS is 240, the division number NP is 20. Then, the content is divided into NPs in the content division 13. The equally divided sections SAV and the shots S belonging to each section are mapped.

In the following, processing is performed for each section SAV input by the divided section input 14. First, the first shot Sn is input in the shot input 15. Shot length evaluation unit 1
In step 6, when the shot length SHLn of the input shot Sn is less than or equal to the length T specified in advance, the shot information is excluded from the candidates for the summary information SUM, and the process proceeds to the input processing of the next shot Sn + 1. On the other hand, when the shot length SHLn of the shot Sn is longer than T, the corresponding shot Sn is added to the summary information SUM.
And sends it to the representative frame extracting unit 17.

In the representative frame extracting section 17, the shot Sn
As the representative frame RFSn of the shot, the first frame SF of the shot
Extract Sn or feature frame KFSn. Feature frame KF
For the extraction of Sn, for example, the method described in Japanese Patent Application No. 2000-065259 can be used. Further, the representative frame characteristic value extracting unit 18 extracts the characteristic value CHFSn of the frame RFSn extracted by the representative frame extracting unit 17, and the shot characteristic value extracting unit 19 extracts the characteristic value CHSn of the shot Sn.
Is extracted.

The characteristic value of the leading frame SFSn or the characteristic frame KFSn as the representative frame RFSn for the shot Sn
One or both of CHFSn and the feature value CHSn of the shot Sn are sent to the representative shot evaluation unit 1A. As the feature value CHFSn of the first frame SFSn or the feature frame KFSn, for example, MPEG-7 (Moving Picture Experts Group pha
Descriptors specified in se 7) can be used.

In the representative shot evaluation section 1A, the characteristic value CHF of the representative frame RF regarding all shots already registered as the representative shots RS in the target divided section SAV and the representative frame characteristic value extraction section 18 send the CHF. The similarity is determined with respect to the feature value CHFSn of the representative frame RFs of the shot Sn. Here, when it is determined that the similarity between the feature value CHF and the feature value CHFSn of the input shot Sn is large, it is regarded as a repetitive shot, and the corresponding shot Sn is excluded from the candidates for the summary information SUM. Then, the processing shifts to the input processing of the next shot Sn + 1. On the other hand, when it is determined that the similarity between the feature value CHF and the feature value CHFSn of the shot Sn is small, the shot is regarded as an independent shot, the corresponding shot Sn is set as a candidate for the summary information SUM, and the representative shot registration 1B is set. After registering a shot, the shot is sent to the audio level evaluation unit 2. Note that feature values relating to the shot Sn registered as the representative shot are separately held and used for evaluation of the subsequent representative shots.

In the determination of the shot similarity, similarly, the characteristic value CHS of the representative shot related to the shot already registered as the representative shot RS and the characteristic value CHSn of the shot Sn sent from the shot characteristic value extracting unit 19 are substituted. Or may be used together.

In the audio level evaluation section 2, with respect to the shot Sn input from the content analysis section 1, the subband data extraction section 21 in FIG. 3 extracts the subband data SDSn of the audio portion of the shot Sn. Then, the silence analyzing unit 22 calculates a feature value as silence, and the silence determining unit 23 calculates all the sections of the shot Sn or the shots.
When it is determined that Y% or more of the Sn section is silent, the corresponding shot Sn is excluded from the candidates for the summary information SUM, and the process proceeds to the input processing of the next shot Sn + 1. On the other hand, in the determination by the silence determination section 23, when the audio portion of the shot Sn is (100-Y)% or more of the section of the shot Sn having sound, the shot Sn is sent to the low-level sound analysis section 24. Examples of the silent analysis method in the silent analysis unit 22 and the silent determination method in the silent determination unit 23 include, for example, Japanese Patent Application No. Hei 10-23554.
The method described in No. 3 can be used.

In the low-level sound analysis unit 24, the sub-band data SD similarly extracted by the sub-band data extraction unit 21
The level LSn of the corresponding audio is estimated from Sn, and if the audio below the designated sufficiently low level THLL occupies all the sections of the shot Sn or Z% or more of the section of the shot Sn, the corresponding shot Sn is determined. It is excluded from the candidates for the summary information SUM, and the process proceeds to the input processing of the next shot Sn + 1. On the other hand, when the audio part of the shot Sn is audio that exceeds the audio level THLL by (100-Z)% or more of the section of the shot Sn in the low-level sound analysis unit 24, the shot Sn is regarded as a candidate for the summary information SUM. And sends it to the high-level sound analysis unit 26.

In the high-level sound analysis section 26, based on the sub-band data SDSn relating to the shot Sn obtained from the sub-band data extraction section 21, the unit time density calculation section 261 shown in FIG. Calculate the unit time density Dsn of audio. At this time, if the audio information is encoded with MPEG audio,
The unit time density Dsn per second can be obtained, for example, as follows.

Dsn = (NAF _THHL / NAF) × AAL _THHL (2) Here, NAF _THHL is the number of frames per second having a level of THHL or more, NAF is the number of audio frames per second, AAL
_THHL is the average level for NAF _THHL .

The sub-band energy sum calculation unit 2
At 62, the sum SBE of the sub-band energies weighted by the sub-bands is calculated, and the unit time sub-band energy calculator 263 calculates the sum of the sub-band energies SBsn per unit time based on the calculated sum SBE. The unit time sub-band energy SBsn per second can be obtained, for example, as in the following equation (3). Here, αk is a weight for the subband k, and sbk is the energy of the subband k in a certain frame.

Next, if the corresponding shot Sn has a unit time density Dsn exceeding a certain threshold THD, the unit time sub-band energy determining unit 26
Go to 5. Unit time subband energy determination unit 2
At 65, when there is a unit time sub-band energy SBsn exceeding the threshold THSB, the shot Sn including the corresponding audio is determined as a candidate for the outline information, the high-level sound analysis routine is exited, and the process proceeds to the motion activity evaluation unit 3. .

On the other hand, when the unit time density DSn of the audio having the audio level THHL or higher is less than the threshold value THD, or when the unit time subband energy SBSn is less than the threshold value THSB, the shot Sn including the corresponding audio is included in the summary information. It is excluded from the candidates, and the process proceeds to the input processing of the next shot Sn + 1.

Note that, in the configuration of FIG.
2, low-level sound analysis unit 24 and high-level sound analysis unit 2
6 is not always necessary, and it is sufficient if at least one is provided.

The motion activity evaluator 3 extracts the motion vector information MV of all the frames belonging to the shot Sn by the motion vector extractor 31 of FIG. 5 for the shot Sn input from the audio level evaluator 2. Then, the motion activity calculator 32 calculates the motion activity MASn as the entire shot Sn using the motion vector information MV, and uses the calculated motion activity MASn as a unit time in the unit time motion activity calculator 33 (for example, 1 second, 1 second). Movement activity MA in frames
Is calculated. The processing in the movement activity calculation unit 32 and the unit time movement activity 33 is performed, for example, by MPEG.
It can be expressed as follows for the encoded video. Here, we assume movement activity per second. Here, ASMV is the sum of absolute values of motion vectors having a size of X or more in a frame, NMB is the number of macroblocks having a motion vector of a size of X or more in a frame, and NPSn is the number of predicted coded frames in a shot. , NVF is the number of video frames per second.

In the movement activity determination section 34,
The calculated motion activity MA is compared with a specified threshold value THMA, and when MA exceeds THMA, the corresponding shot Sn is excluded from the candidates for the summary information SUM, and the input processing of the next shot Sn + 1 is performed. Transition. On the other hand, when the motion activity determination unit 34 determines that the motion activity MA of the shot Sn in a certain unit time does not exceed the threshold THMA, the corresponding shot Sn is set as a candidate for the summary information SUM.

With the above processing, audio-video information
In the section SAV obtained by dividing the AV at equal intervals, the shot S selected as a candidate for the summary information SUM is input to the summary information registration unit 4, and the information of the shot S is stored in the shot memory 4 in FIG.
1 is stored. Next, the determination unit 42 determines whether or not processing of all shots in the section SAV has been completed.
If this determination is negative, the processing for the next shot is performed.

On the other hand, when the processing of all the shots in the section SAV is completed, the sum total SSL of the shot lengths of all the shots S in the section SAV is determined by the summary information length determination section 43 by the average summary information length MSL ( = SL / NP). If the shot length total SSL and the average summary information length MSL in the section SAV are considered to be sufficiently close, the summary information extraction processing in the section SAV is terminated, and the time information and the like are registered in the summary information registration 44. . Then, the processing shifts to the input processing of the next section SAVn + 1. At this time, various characteristic values related to the representative shot registered in the representative shot registration 1B are cleared. On the other hand, if SSL and MSL are not considered to be close to each other, the threshold value changing unit 47 changes some or all of the threshold values used to determine the summary information SUM as a candidate, and has been extracted up to that time. The processing from the shot length evaluator 16 to the motion activity evaluator 3 is recursively repeated for the candidates for the summary information SUM until SSL and MSL become sufficiently close.

These processes are performed for all sections SAV, and finally, summary information SUM of audio video information AV
Then, the summary information SUM is registered. At this time, if the description of the summary information has been designated, the process proceeds to the summary information description unit 5, and if not, the process is terminated.

As shown in FIG. 7, in the summary information description section 5, at least the time information of all shots S extracted as the summary information SUM is described in the summary information description section 51. The output unit 52 outputs it as a summary information description file. As a format to be described, for example, a format defined by MPEG-7 can be used. When the description has been completed for all the sections SAV, all the processing ends. Also, summary information SU
All shots S extracted as M can be combined and saved as a separate file. At this time, it can be saved as an audio video file, or audio and video can be saved separately.

Next, the function of the audio-video summary information extracting device of the above-described embodiment can be realized by software (program). The software can be recorded on a recording medium such as an optical disk, a floppy (registered trademark) disk, and a hard disk.

FIG. 8 shows an example of a program recorded on the recording medium 100. The recording medium 100 has a content analysis function 1 for compressed audio / video information.
11, an audio level evaluation function 112, a motion activity evaluation function 113, a summary information registration function 114, and a summary information description function 115 are recorded. Note that the movement activity evaluation function 113 may be omitted.

Further, the content analysis function 111 comprises:
A function of dividing video information into shots, a function of dividing input content into equally spaced sections according to a certain criterion,
It can be composed of at least one of a function for evaluating the length of a shot included in the equally-spaced section and a function for determining a repetitive shot. Further, the audio level evaluation function 112 can be composed of a function of determining silence, a function of determining low-level sound, and a function of determining high-level sound.

The motion activity evaluation function 1
Reference numeral 13 denotes a function of extracting motion vector data of a frame belonging to the shot, a function of calculating a motion activity from the extracted motion vector data, a function of calculating a motion activity in a unit time, and a function of calculating a motion in the unit time. And a function of determining a candidate for summary information using an activity.

It is possible to combine the extracted summary information as audio video or audio and video separately, and record the combined summary information as a separate file on the recording medium 100. .

The recording medium 100 includes a transmission medium such as a network for temporarily recording and holding data.

FIG. 9 is a block diagram showing a configuration of an audio outline information extracting apparatus according to a second embodiment of the present invention.

First, when the compressed audio information CA is input, the sub-band data extracting unit A1 extracts the sub-band data SD. The extracted sub-band data SD is sent to the high-level sound evaluation section A3. The operation of the sub-band data extraction unit A1 is the same as the operation of the silent band shown in the first embodiment.
Sub-band data extraction unit 5 in low-level sound evaluation unit 5
Same as 1. On the other hand, when the uncompressed audio information UA is input, the input audio is sub-band analyzed by the sub-band analysis unit A2, and the sub-band data SD as a result of the analysis is similarly sent to the high-level sound evaluation unit A3. .

The high-level sound evaluation section A3 has the same functions as the sub-band energy sum calculation section 262 and the unit time sub-band energy sum calculation section 263 included in the high-level sound analysis section 26 shown in the first embodiment. Figure 10
From the input sub-band data SD, the sub-band energy sum SBE and the sub-band energy sum SB per unit time are calculated by the sub-band energy sum calculation unit A31 and the unit time sub-band energy sum calculation unit A32. .

Next, in the outline information start time determination unit A33, a time position where the sum of the unit time subband energies SB is maximum is determined as the outline information start time T_start. In the summary information end time determination unit A34, the unit time subband energy sum SB is α times the maximum value (0 <α <
The time position of 1) is determined as the outline information end time T_end. At this time, T_start> T_end.

The outline information registration unit A4 registers outline information based on the outline information start time T_start and the outline information end time T_end determined by the high-level sound evaluation unit A3. Then, at least the time information is described in the summary information description section A5 and output as a summary information description file.
As a format to be described, for example, a format defined by MPEG-7 can be used.

When there are a plurality of pieces of audio information,
The above processing is performed on all audio information.

The function of the audio summary information extracting device of the above-described embodiment can be realized by software (program), and the software can be recorded on a recording medium 100 such as an optical disk, a floppy disk, and a hard disk. Further, the extracted audio summary information can be recorded on the recording medium 100 as an individual file.

FIG. 11 shows an example of a program recorded on the recording medium 100.
The sub-band data extraction function 121 and / or the sub-band analysis function 122, the high-level sound evaluation function 12
3. The summary information registration function 124 and the summary information description function 125 are recorded.

FIG. 12 is a block diagram showing an embodiment of an audio / video summary information reproducing apparatus according to the present invention.

When the audio / video summary information SUM extracted by the above means is input, the audio / video separation unit P1 separates the audio / video summary information into a video element VSUM and an audio element ASUM of the summary information. Next, the video speed converter P
In 2, the video element VSUM is spatially thinned out according to the conversion speed parameter SP given from the outside to convert the video playback speed. Similarly, the audio element ASUM is temporally thinned out at the same rate as the video element VSUM according to the conversion speed parameter SP in the audio speed conversion unit P3,
Convert audio playback speed. The conversion of the audio playback speed can be realized by, for example, periodic skipping of an audio frame or a combination of repeated playback and periodic skipping. Here, when making the audio 1.5 times faster, in the former, <frame number to be played> 1, 2, 4, 5, 7, 8, 10, 1
This is achieved by playing back two consecutive frames with 1, ... and skipping the next one frame. In the latter, <frame number to be played back> 1, 1, 4, 4, 7, 7, 10, 1
Repeat the same frame as 0, ... twice, and then continue
Achieved by skipping two frames.

The video element VSUM 'and the audio element ASUM' whose speed has been converted are input to the audio / video multiplexing / synchronizing unit P4, where multiplexing and synchronization processing is performed, and the speed-converted audio / video summary information SUM. 'Is obtained. The obtained audio / video summary information SUM ′ is displayed and reproduced.

[0053]

As is apparent from the above description, according to the present invention, the contents of uncompressed or compressed audio information or compressed audio / video information can be grasped quickly and efficiently. Can be extracted. In addition, this extraction enables high-speed browsing of audio-video information.

The length of the outline information to be extracted can be arbitrarily specified, and at the same time, since the outline information is uniformly extracted from the audio / video information, the entire contents can be efficiently grasped. Become.

By describing the time information and the like included in the summary information, it is possible to describe the feature as the summary information of the corresponding audio / video information, and it is also possible to standardize the content description to MPEG-7 or the like. It is possible to apply.

Further, it is possible to provide high-performance reproduction of the outline information, such as converting the display speed of the extracted outline information.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an overall configuration of an embodiment of the present invention.

FIG. 2 is a block diagram illustrating a detailed configuration of a content analysis unit in FIG. 1;

FIG. 3 is a block diagram illustrating a detailed configuration of an audio level evaluation unit in FIG. 1;

FIG. 4 is a block diagram illustrating a detailed configuration of a high-level sound analysis unit in FIG. 3;

FIG. 5 is a block diagram illustrating a detailed configuration of a motion activity evaluation unit in FIG. 1;

FIG. 6 is a block diagram illustrating a detailed configuration of a summary information registration unit in FIG. 1;

FIG. 7 is a block diagram illustrating a detailed configuration of a summary information description unit in FIG. 1;

FIG. 8 is a diagram showing an outline of a program recorded on a recording medium.

FIG. 9 is a block diagram illustrating a configuration of an audio summary information extraction device according to another embodiment of the present invention.

FIG. 10 is a block diagram illustrating a detailed configuration of a high-level sound evaluation unit in FIG. 8;

FIG. 11 is a diagram showing an outline of a program recorded on a recording medium.

FIG. 12 is a block diagram showing a configuration of an audio summary information reproducing device according to another embodiment of the present invention.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Content analysis part, 2 ... Audio level evaluation part, 3 ... Motion activity evaluation part, 4 ... Summary information registration part, 5 ... Summary information description part, A1 ... Subband Data extraction unit, A2: sub-band analysis unit, A3: high-level sound evaluation unit, A4: summary information registration unit, A5: summary information description unit, P1: audio / video separation unit, P2: video speed converter, P3: audio speed converter, P4: audio / video multiplexing / synchronizing unit, 100: recording medium.

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Hiromasa Yanagihara 2-1-15 Ohara, Kamifukuoka-shi, Saitama F-term in C-D Laboratory Inc. (reference) 5C053 FA14 GB19 HA30 JA01 5D044 AB02 BC01 BC03 CC04 DE91 GK11 HL02 5L096 BA20 GA08 GA59 HA04 HA08 JA11

Claims (28)

[Claims] 1. Content analysis means for analyzing the temporal structure of video information of input audio-video content, and audio level evaluation for evaluating the audio level of audio information associated with the video information whose temporal structure has been analyzed. Means for registering audio-video summary information evaluated by the audio-level evaluation means. 2. The audio-video summary information extracting device according to claim 1, wherein the content analyzing unit divides the video information into shots and divides the input content into equally spaced sections according to a certain criterion. And at least one of a means for evaluating the length of a shot included in the equally-spaced section and a means for determining a repetitive shot, and when the length of the shot is smaller than a predetermined value or when the repetitive shot Wherein the shot is excluded from the summary information when the determination is made. 3. The audio-video summary information extraction apparatus according to claim 2, wherein the means for determining the repetitive shot includes means for determining a representative frame of the divided shot, and a feature value of the representative frame. Means for extracting, and a means for performing a similar image search by comparing the characteristic value with the characteristic value of another representative frame. When the similar image is determined to be a similar image by the similar image search, the shot is set as a repetitive shot. An audio-video summary information extraction device characterized by being excluded. 4. The audio / video summary information extracting apparatus according to claim 2, wherein the means for determining the repetitive shot includes means for extracting a feature value of the divided shot, and a feature value and another shot. Means for performing a similar shot search by comparing the feature value with the feature value.
An apparatus for extracting audio / video summary information, wherein the shot is determined to be a repetitive shot. 5. The audio-video summary information extraction device according to claim 2, wherein the audio level evaluation unit includes at least one of a unit that determines silence and a unit that determines low-level sound. An audio / video summary information extraction device, wherein the shot is excluded from the summary information when the sound is evaluated as a low-level sound. 6. The audio-video summary information extracting apparatus according to claim 5, further comprising: means for determining a high-level sound, wherein the means for determining the high-level sound comprises a sub-band from the compressed audio information. Means for extracting data, means for estimating an audio level from the sub-band data, and means for calculating a time density of the audio level from the audio level exceeding a certain threshold value, and using the audio level time density. An audio / video summary information extracting apparatus for determining a candidate for summary information. 7. The audio-video summary information extracting apparatus according to claim 5, further comprising: means for judging a high-level sound, wherein the means for judging the high-level sound comprises a sub-band from compressed audio information. Means for extracting data, means for calculating the sum of sub-band energies weighted by bands from the sub-band data, and means for calculating the sum of sub-band energies per unit time, An audio-video summary information extraction apparatus, wherein summary information candidates are determined using at least one of the sum and the sub-band energy sum in the unit time. 8. The audio-video summary information extraction device according to claim 1, further comprising: a motion activity evaluation unit that evaluates a motion in an image; An audio-video summary information extraction device, which registers audio-video summary information evaluated by a level evaluation unit and the motion activity evaluation unit. 9. The audio / video summary information extraction device according to claim 8, wherein the motion activity evaluation means extracts motion vector data of a frame belonging to the shot,
Means for calculating a motion activity from the extracted motion vector data, and means for calculating a motion activity per unit time, wherein a candidate for summary information is determined using the motion activity per unit time. Audio-video summary information extraction device. 10. The audio / video summary information extraction device according to claim 1, wherein the outline information specified from outside is controlled by recursively controlling a threshold value as a criterion for determining the summary information. An audio / video summary information extraction device, wherein a determination process is repeated until the length becomes longer. 11. The audio-video summary information extracting apparatus according to claim 1, wherein the time information of the extracted summary information is a start time and an end time or a start time and a continuation time of the summary information. An audio / video summary information extraction device, comprising: means for at least describing time. 12. The audio-video summary information extraction device according to claim 1, wherein the extracted summary information is combined as audio video or audio and video separately. Means for outputting and storing the combined summary information as a separate file. 13. The audio / video summary information extraction apparatus according to claim 1, wherein: a means for extracting a head frame or a feature frame of a shot as a representative frame as a video element of the extracted summary information; An audio / video summary information extraction apparatus, comprising: means for extracting audio information accompanying a shot as an audio element of the summary information. 14. The audio / video summary information extracting apparatus according to claim 13, wherein a time position in the content is described as time information of the extracted video element, and a time of the extracted audio element. An audio-video summary information extraction apparatus, comprising: means for describing at least a start time and an end time or a start time and a duration in a content as information. 15. A means for extracting sub-band data from input compressed audio content, a means for calculating a sum of sub-band energies weighted by a band from the sub-band data, and a sum of sub-band energies per unit time. An audio outline information extraction device, comprising: an evaluation unit; and an audio outline information extraction unit. 16. A means for extracting sub-band data from the input compressed audio content, a means for calculating a sum of sub-band energies weighted by a band from the sub-band data, and a unit of the sum of the sub-band energies Means for calculating the sum in time, means for determining the start time of the summary information, means for determining the end time of the summary information, and means for extracting audio information of the section between the start time and the end time as the summary information. A device for extracting audio summary information, comprising: 17. A means for converting input audio content into sub-band data; a means for calculating a sum of sub-band energies weighted by a band from the sub-band data; An audio outline information extracting device, comprising: an evaluation unit; and an audio outline information extracting unit. 18. A unit for converting input audio content into sub-band data, a unit for calculating a sum of sub-band energies weighted by a band from the sub-band data, and a unit time of the sum of the sub-band energies , A means for determining the start time of the summary information, a means for determining the end time of the summary information, and a means for extracting audio information of the section between the start time and the end time as the summary information. A device for extracting audio summary information, characterized in that: 19. The audio outline information extracting device according to claim 15, wherein the time information of the extracted outline information is a start time and an end time or a start time and a duration time of the outline information. Characterized by at least a means for describing audio information. 20. The audio summary information extracting apparatus according to claim 15, further comprising means for outputting and saving the extracted summary information as an individual file. Extraction equipment. 21. A playback apparatus for audio-video summary information, comprising: means for synchronizing and playing back video and audio elements extracted by the audio-video summary information extraction device according to claim 13. Description: Audio-video summary information playback device. 22. A playback device for audio-video summary information, comprising: means for temporally thinning out audio elements extracted by the audio-video summary information extraction device according to claim 13 to convert audio playback speed; An audio / video summary information reproducing apparatus, comprising: means for synchronously reproducing the converted audio element and video element. 23. A playback device for audio-video summary information, comprising: means for temporally thinning out video information of a shot as the extracted summary information to convert a video playback speed; and a shot as the extracted summary information. Means for converting audio reproduction speed by temporally thinning out the audio information at the same rate as video information, and means for synchronizing and reproducing the video information and audio information whose reproduction speed has been converted. Characteristic audio-video summary information playback device. 24. The audio / video summary information reproducing apparatus according to claim 22, wherein the audio reproduction speed conversion is achieved by periodically skipping audio frames. Video summary information playback device. 25. The audio-video summary information reproducing apparatus according to claim 22, wherein the audio reproduction speed conversion is achieved by combining repetitive reproduction of audio frames and periodic skipping. Characteristic audio-video summary information playback device. 26. A function of analyzing a temporal structure of video information of input audio-video content, a function of evaluating an audio level of audio information associated with the video information whose temporal structure has been analyzed, and A computer-readable recording medium recording a program to be executed by a computer, the program including a summary information registering function for registering audio-video summary information evaluated for level and motion activity. 27. The recording medium according to claim 26, further comprising a function of evaluating a motion activity in an image, wherein the summary information registering function includes the audio-video summary information evaluated for the audio level and the motion activity. A recording medium characterized by registering the following. 28. At least one of a sub-band data extraction function for extracting sub-band data from input compressed audio content and a sub-band analysis function for converting input audio content to sub-band data, and evaluating a high-level sound And a function of registering the outline information of the audio content.