JPH11266428A - Method and device for picture division and recording medium with picture division program recorded - Google Patents

Abstract

PROBLEM TO BE SOLVED: To roughly divide pictures by extracting the feature quantities of sections including no music neither voice in sound information of pictures to calculate degrees of similarity of feature quantities and dividing pictures, which include sections having high degrees of similarity and sound information interposed between these sections, as one segment. SOLUTION: Pictures are stored in a picture storage part 102, and it is discriminated by a music detection part 103 whether sound information of sounds is music or not, and it is discriminated by a voice detection part 104 whether the sound information is voice or not if it is not music; and if it is not voice, sound information in this period is decided as background sounds including no music neither voice, and features of segments corresponding to background sounds are extracted by a feature extraction part 105. Long-term average spectrums of sound information are obtained by a feature extraction part 105, and the correlation between the preceding long-term average spectrum and current than is obtained by a picture division part 106, and they are regarded as the same scene to perform labeling if the correlation is high, and label information is preserved in the picture storage part 102 together with time information of segments.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video dividing method and apparatus for analyzing a background sound of sound information contained in a video and dividing the video based on the similarity of the characteristic amounts thereof, and a recording in which a video dividing program is recorded. Regarding the medium.

[0002]

2. Description of the Related Art A method of dividing an image mainly uses image information. For example, there is a method of detecting a cut point at which a camera is switched and dividing the image into shots.

[0003]

As an application example of a technique for dividing image information using a method of detecting a cut point, a leading image of a shot is set to a representative still image (representative image) representing the shot. There is a video expression method that can be displayed spatially side-by-side and display the contents of the video.However, since there are frequent cut points, when targeting a long video, the number of representative images is reduced. There was a problem that it would increase too much. In order to reduce the number of representative images, the video needs to be roughly divided.

[0004] From the viewpoint of video production, a set of shots is a scene, and it is conceivable to divide the video by capturing the scene. However, it is difficult to automatically divide the normal scene because the same scene is connected. Met.

SUMMARY OF THE INVENTION An object of the present invention is to roughly divide a video using the feature that background sounds are likely to be similar in the same scene.

[0006]

In order to achieve the above object, according to the present invention, an image is input, the input image is stored, music and sound are detected from sound information of the image,
In the sound information, a feature amount is extracted for a section that does not include music and voice, and a similarity between the extracted feature amounts is calculated.
By dividing a video including a section having high similarity and sound information sandwiched between the sections as one segment,
The video is roughly divided.

Further, the similarity of video segments is obtained from background sound by using a long-term average spectrum of sound information.

[0008]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram illustrating a schematic configuration of a video division device according to an embodiment of the present invention. The video dividing apparatus according to the present embodiment includes a video input unit 101 for inputting video, a video storage unit 102 for storing video, a music detection unit 103 for detecting music, a voice detection unit 104 for detecting audio, And the section without sound
A feature extracting unit 105 for extracting a feature amount; and a video dividing unit for calculating a similarity between the extracted feature amounts, and dividing a video including sound information sandwiched between the high similarity and the segment as one segment as one segment. 106.

FIG. 2 is a flowchart showing the flow of processing of the video dividing apparatus according to one embodiment of the present invention. The same processing flow is used when the present invention is implemented by software. One loop of processing is performed on a video segment of about one second.

First, a video is stored in a video storage process 201, and a music detection process 202 is performed on sound information of the video.
In the judgment 203, it is judged whether or not the music is music. If it's not music,
The voice detection processing 204 is performed. In the judgment 205, it is judged whether or not it is a voice. Music detection uses the characteristic that the peak of the frequency spectrum of sound information is temporally stable in the frequency direction, and voice detection uses a comb filter (Minami et al., “Sound Video Indexing by Analysis ",
IEICE General Conference, D-12-64, 199
7) is effective.

[0011] The "video indexing by sound analysis" is to automatically detect voice and music from the sound information included in the video, extract only the portion including these, and summarize the video. This is effective, for example, when the user wants to hear only the song without listening to the talk of the song program. When music is present, the peak of the frequency spectrum has the characteristic of being temporally stable in the frequency direction, so the music is detected by detecting the peak and calculating the temporal continuity. can do.

If it is not a voice, the feature extraction processing 206 is performed as a background sound that does not include music and voice during the period, that is, as a segment corresponding to the background sound. In the feature extraction processing 206, the sound information is frequency-analyzed,
Find the long-term average spectrum. The long-term average spectrum is a temporal average value of the spectrum power at each frequency.

Next, in the video segmentation processing 207, the correlation between the long-term average spectrum calculated one loop before and the current long-term average spectrum is determined. If the correlation is high, the scene is regarded as the same scene, and labeling is performed. I do. The music or audio segments present in the two segments for which the correlation has been determined are also labeled as belonging to the same scene. The label information is stored in the video storage unit 102 together with the segment time information.
Is stored in

Although the above description has been made on the division of an image, the division can be held in the form of a program that can be executed by the data processing device, and the present invention also includes a recording medium on which the program is recorded.

[0015]

(1) The first, third and fifth aspects of the present invention
A video is input, the input video is stored, music and audio are detected from the audio information of the video, and a feature amount is extracted from a section of the audio information that does not include the music and audio, and the extracted features are extracted. By calculating the degree of similarity, it is possible to divide a video including a section with high similarity and sound information sandwiched between the sections as one segment, and roughly divide the video.

(2) The inventions of claims 2, 4 and 6 make it possible to determine the similarity of video segments from background sound by using a long-term average spectrum of sound information.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a schematic configuration of a video division device according to an embodiment of the present invention.

FIG. 2 is a flowchart showing a processing flow of the video dividing apparatus according to the embodiment of the present invention and a processing flow when the present invention is realized by software.

[Explanation of symbols]

 Reference Signs List 101 video input unit 102 video storage unit 103 music detection unit 104 audio detection unit 105 feature extraction unit 106 video division unit 201 video storage process 202 music detection process 203 music determination process 204 audio detection process 205 audio determination process 206 feature extraction process 207 video Division processing 208 Image end determination processing

Claims (6)

[Claims] 1. A method for dividing a given video according to a scene, comprising: a video input stage for inputting a video; a video storage stage for storing a video; and music for detecting music from sound information in the video. A detection step; a voice detection step of detecting voice from the voice information; and a section of the voice information that does not include music and voice.
A feature extraction step of extracting a feature amount; calculating a similarity of the extracted feature amount to obtain a video of a section having a high similarity and a video including sound information sandwiched between the sections.
A video dividing step of dividing the video into a plurality of segments by combining them into one segment. 2. The video segmentation method according to claim 1, wherein in the feature extraction step, a long-term average spectrum of sound information is extracted as a feature amount. 3. A device for dividing a given video according to a scene, a video input unit for inputting a video, a video storage unit for storing a video, and music for detecting music from sound information in the video. A detecting unit, a voice detecting unit that detects voice from the voice information, and a section of the voice information that does not include music and voice.
A feature extraction unit for extracting a feature amount; calculating a similarity degree of the extracted feature amount to obtain a video of a section having a high similarity degree and a video including sound information sandwiched between the sections;
A video dividing unit that divides a video into a plurality of segments by combining the video into one segment. 4. The video dividing apparatus according to claim 3, wherein the feature extracting unit extracts a long-term average spectrum of the sound information as a feature amount. 5. A recording medium on which a program for dividing a given video according to a scene is recorded, comprising: a video input process for inputting a video, a video storage process for storing a video, and sound information in the video. A music detection process for detecting music, a voice detection process for detecting voice from sound information, and a section of the sound information that does not include music and voice.
A feature extraction process of extracting a feature amount; calculating a similarity of the extracted feature amount to obtain a video of a section having a high similarity and a video including sound information sandwiched between the sections;
A recording medium characterized by recording a video dividing program for causing a computer to execute a video dividing process of dividing a video into a plurality of segments by putting together as one segment. 6. The recording medium according to claim 5, wherein said feature extraction process extracts a long-term average spectrum of sound information as a feature amount.