JP2001147697A - Method and device for acoustic data analysis - Google Patents

Abstract

PROBLEM TO BE SOLVED: To detect a voice section with high precision even in the case of a mixed sound signal like a broadcasting program. SOLUTION: The sound signal of a program is inputted to an acoustic data input part 101, and the feature quantity or the like is used to divide it into proper sections by a section division part 102. Acoustic data for every divided section is classified to three patterns of 'voice', 'non-voice', and 'mixture' by a discrimination part 103, and the genre of the program is determined by the pattern of classified acoustic data by a discrimination part 104. The start time, the end time, the degree of importance, etc., of the section are added on the basis of the determined genre to output the analysis result by an output part 107.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to a technique for automatically extracting additional information necessary for implementing skimming, searching, and filtering of digital content information such as digital broadcast programs.

[0002]

2. Description of the Related Art In recent years, with the development of digital technology, it has become possible to transmit and store a large amount of digital content information such as sound and video. Skimming, searching, filtering, and the like are known as methods for efficiently extracting target information from a large amount of digital content information. To realize this method, the structure of the digital content information must be determined in advance. Must be analyzed and additional information according to the structure needs to be added. If such additional information had to be relied on manually, it would be too costly.Therefore, research and development of technology to automatically analyze the structure of digital content information and to provide additional information suitable for the structure has been conducted. ing.

[0003] For example, USP-5,918,223 (Method and artic
le of manufacture for content-based analysis, stor
age, retrieval, and segmentation of audio informat
In ion. June 29, 1999), a patent is registered for a sound analysis method and a search method. In the above-mentioned conventional example, feature amounts such as volume, pitch, brightness, band, and mel-frequency cepstrum coefficient (MFCC) are statistically processed to determine the type of acoustic information. This classifies single sound source audio files (or segments) on the Internet, for example.
This is an effective method for searching. In addition, three types of methods for segmenting a continuous sound using the above technique are shown.

[0004] The first method is a method of segmenting into a region having a similarity with a representative value of a sound presented by the user (a statistic of a feature vector) equal to or larger than a threshold value and a region other than the region. With this method, the continuous sound is divided into two types of regions. The second is a method of segmenting into a region that is exactly the same as the sound presented by the user and a region that is not. The first method calculates the similarity using the statistic of the feature vector as a representative value, whereas the second method calculates the similarity using the feature vector itself. Three
The second method is to compare adjacent regions of a continuous sound and detect a point (scene change) at which the similarity is equal to or less than a threshold value.

[0005]

However, when analyzing the acoustic data of a broadcast program or the like, the sound of a single sound source is small, and the sound of the mixed sound and music, the noise of the audience and the voice of the announcer are mixed. It can be seen that a plurality of sounds such as sounds are mixed. In this way, since there are actually many mixed sounds,
There is a problem that the technique of only determining the type of sound is not practical for contents such as broadcast programs.

[0006] Further, since sound data is diverse, if a scene change is detected only by the similarity between adjacent areas,
Excessive detection. Furthermore, it is difficult to determine which segments are important during the program.

[0007]

According to the present invention, there is provided a sound data analyzing apparatus which divides sound data into meaningful sections and determines the importance of the sections. Sound data dividing means for dividing into sections, section type determining means for determining the type of the section, section type pattern determining means for determining a genre of the sound data using a statistical processing result of the section type, and A control information management unit that manages control information.

According to the present invention, a simple type of a sound section (a loudness of a sound, a sound including a sound, a sound including no sound, and a silent sound) is once discriminated, and the type of the sound section is determined based on the type pattern of the sound section. Since the genre of the program is estimated, a mixed sound model suitable for the genre can be set. As a result, it is possible to make a detailed type determination and also to determine a degree of importance suitable for a genre. Also,
Instead of using the similarity between the feature amounts of adjacent sections, the change pattern (increase / decrease) of the feature amount of the adjacent section, the value of the feature amount before the change, the value of the feature amount after the change, and the magnitude of the change are used. Thus, excessive section detection can be suppressed.

[0009]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention according to claim 1 of the present invention provides an audio data dividing means for dividing audio data into a plurality of sections, a section type determining means for determining the type of the section, It is characterized by comprising a section type pattern discriminating means for discriminating a genre of a program by using a statistical processing result, and a control information management unit for managing control information according to the genre. This has the effect that importance determination suitable for the genre can be made.

According to a second aspect of the present invention, in the acoustic data analysis apparatus according to the first aspect, there is provided a section integrating means for integrating successive sections of the same type. When sections are continuous, it has an effect that they can be combined into one.

According to a third aspect of the present invention, the acoustic data dividing means divides the acoustic data into a plurality of sections by using an autocorrelation coefficient of the acoustic data, and determines a speech section. Can be easily and accurately performed.

According to a fourth aspect of the present invention, there is provided the acoustic data analysis apparatus according to the first aspect, wherein any one of a sound section including voice, a sound section including no voice, and a silent section is used. It is characterized by comprising a section type determining means for determining, and has a function of performing a type determination before determining a genre with high accuracy and generating a pattern effective for determining a genre.

According to a fifth aspect of the present invention, there is provided an audio data dividing means for dividing audio data into a plurality of sections, a section type determining means for determining the type of the section, and control information corresponding to a genre. And a control information management unit that manages the sound information, and has an effect that the importance of a sound segment can be estimated using genre information.

According to a sixth aspect of the present invention, there is provided an audio data dividing means for dividing audio data into a plurality of sections, and a section type determining means for determining a type of the section. , The genre of the program can be estimated.

An embodiment of the present invention will be described below with reference to the drawings.

(Embodiment 1) System for performing genre determination FIG. 1 is a configuration diagram of an acoustic data analyzer according to Embodiment 1 of the present invention. The acoustic data analysis device according to the first embodiment includes an acoustic data input unit 101 that reads audio data in various formats for each program, and a section dividing unit 102 that divides the read audio data using the change information of the acoustic feature amount. If the genre is unknown, the type of section is determined to be one of a voice section, a mixed sound section with voice, a voiced section without voice, and a silent section. Section type determination unit 103 to perform
A genre discriminating standard pattern storage unit 105 that stores section type judgment results of programs whose genres are known, a section type pattern discriminating unit 104 that discriminates a program genre based on a section type pattern, and a type calculation method suitable for the genre And a program genre-based control information management unit 106 for managing the method of calculating the degree of importance and an analysis result output unit 107 for outputting an analysis result of audio data.

FIG. 2 shows a processing flow of the above acoustic data analyzer. First, sound data is input using the sound data input unit 101, and the input sound data is divided into a plurality of sections using the section dividing unit 102. The above is the section dividing means (st
ep201). Next, the section type determination unit 103
Is used to determine the type of a section as one of a voice section, a mixed sound section with voice, a voiced section without voice, and a silent section,
The determination result is output to the section type pattern determination unit 104.
The above corresponds to the section type determination means (step 202). The section type pattern determination unit 104 compares the section type pattern output from the section type determination unit 103 with the section type pattern of a program whose genre is known and stored in the genre determination standard pattern storage unit 105, The top K programs with high similarity are selected. The genre that is the largest among the genres of the selected program is the genre of the audio data input from the audio data input unit 101. The genre estimation result is input to the control information management unit 106 for each program. The above corresponds to the section type pattern determination means (step 203). The program genre-based control information management section 106 outputs section type information and important section determination information suitable for the program genre to the section type determination section 103 as control information. The section type determination unit 103 calculates the section type and section importance using the genre-specific control information, and outputs the calculated section type and section importance to the analysis result output unit 107. The above is the genre adaptive section type determination means (st
ep204). The analysis result output unit 107 converts the analysis result into an appropriate format and outputs it. For example, a section start time, a section end time, a section importance value, and a section type number are output for each row.

Specific section dividing means (step 201)
Will be described with reference to the flowchart of FIG. An autocorrelation analysis is performed using the acoustic data as input, and a normalized first-order autocorrelation coefficient is output (step 301). Using the normalized primary autocorrelation coefficient, the division position of the acoustic data is detected, and the division positions are ranked (step 31).
0). The sound data is divided according to the rank of the division position (step 320). Further, the dividing position detecting means (ste
p310) calculates the maximum value of the normalized primary autocorrelation coefficient in the preceding and succeeding sections at all positions of the acoustic data (step 31).
1) In addition to calculating the difference between the maximum values of adjacent divided sections (step 312), a pattern (a rising position) in which the maximum value increases in the time direction is found (step 312).
p313). Further, N-stage ranking is performed according to the magnitude of the change in the rising position (step 314),
Within a few seconds (for example, within 2 seconds) before and after the division position above a certain rank, it is determined whether there is a division position below a certain rank (step 315), and if a division position below a certain rank is found, the rank of that division position is determined. Is lowered by one rank (step 316).

Specific section type determination determining means (step)
202) will be described with reference to the flowchart of FIG.
Percentage of primary autocorrelation coefficient below threshold in section unit, 1
The average of the second order autocorrelation coefficient, the acoustic feature amount such as the peak of the sound volume and the like are calculated (step 401), and it is determined whether the ratio of the first order autocorrelation coefficient equal to or smaller than the threshold is equal to or larger than α and equal to or smaller than (s).
(step 402), if the conditions are met, it is regarded as a voice section. In addition, it is determined whether the ratio of the primary autocorrelation coefficient equal to or smaller than the threshold value is equal to or smaller than α (step 403). Further, it is determined whether or not the volume peak is equal to or more than γ in the section where the condition is not satisfied in the above step (step 404). If the condition is satisfied, the section is regarded as a sound section without sound. A section that does not meet any of the conditions in the above steps is regarded as a silent section.

Specific section type pattern determining means (st
ep203) will be described with reference to the flowchart of FIG. For each program, the rate of each section type, the rate of sections whose section length is 10 seconds or more / 1 minute or more / 3 minutes or more, the rate of section length of 10 seconds or less / 1 minute or less / 3 minutes or less, and The program type statistics such as the ratio of the section length for each section type are calculated (s
(Step 501), the similarity to the genre-determining standard pattern is calculated by using the program type statistics obtained in the previous step, and the top N high similarities are selected (Step 50).
2). The genre that is the largest among the selected N programs is the genre of the sound data. If the number of genres is the same, the genre of the program with the highest similarity is set (step
503).

A specific genre-adaptive section type determining means (step 204) will be described with reference to the flowchart of FIG. Read control information (section type information and important section information) suitable for the genre prepared in advance (step
p601), and calculate a feature amount suitable for the genre (ste
p602), the section type is determined, and the importance according to the section type is calculated (step 603). The section type information suitable for the genre is, for example, in a music program, there are a music section and a talk section as section types, and a feature amount for determining a feature of the music section, a threshold thereof, and a talk section. And its threshold value. As feature amounts for determining the section type, volume, pitch, brightness,
The autocorrelation coefficient is used in addition to the band and the mel frequency cepstrum. For example, in order to determine a music section of a genre of a music program, the ratio of the partial section in which the normalized primary autocorrelation coefficient is 0.01 or less in each divided section is 0.1 or less and the normalized primary autocorrelation coefficient is Set the condition of 0.1 or more. The important section determination information corresponds to, for example, information that a music program has a high importance in a music section and a low importance in a talk section.

(Embodiment 2) When genre information is input FIG. 7 is a system configuration diagram of an acoustic data analyzer according to an embodiment when genre information is input together with audio data. The acoustic data analysis device according to the first embodiment includes an acoustic data input unit 701 that reads audio data in various formats for each program, and a section dividing unit 7 that divides the read audio data using the change information of the acoustic feature amount.
02, a section type determination unit 703 that determines a type suitable for the genre, and a program genre-based control information management unit 70 that manages a type calculation method and an importance calculation method suitable for the genre.
4 and an analysis result output unit 705 for outputting the analysis result of the acoustic data.

In order to construct the above-described acoustic data analyzer, first, acoustic data is input using the acoustic data input unit 701, and the input acoustic data is divided into a plurality of sections using the section dividing unit 702. Next, the section type determination unit 703 outputs the genre estimation result to the program-specific control information management unit 704. The program-specific control information management unit 704 outputs a section type calculation method and an important section determination method suitable for the genre to the section type determination unit 703 as control information. The section type determination unit 703 calculates the section type and section importance using the genre-specific control information, and outputs the calculated section type and section importance to the analysis result output unit 705. The analysis result output unit 705 converts the analysis result into an appropriate format and outputs it.

The difference from the first embodiment is whether genre information is known in advance. Embodiment 2
In this case, it is assumed that the genre information is known in advance, and the configuration according to the first embodiment does not include the device for estimating the genre.

[0025]

As described above, the acoustic data analyzing apparatus and method of the present invention provide a simple classification of a sound section (volume of sound, sound including sound, sound including no sound,
Since no sound is determined and the genre of the program is estimated based on the result of the determination, the type of mixed sound in the divided section and the importance can be determined.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of an acoustic data analysis device according to a first embodiment of the present invention.

FIG. 2 is a flowchart showing the entire processing of the acoustic data analyzer.

FIG. 3 is a flowchart of a section dividing unit.

FIG. 4 is a flowchart of section type determination means (1).

FIG. 5 is a flowchart of a section type pattern determination unit.

FIG. 6 is a flowchart of genre adaptive section type determination means.

FIG. 7 is a system configuration diagram of an acoustic data analysis device according to a second embodiment of the present invention;

[Explanation of symbols]

 Reference Signs List 101 Sound data input unit 102 Section division unit 103 Section type determination unit 104 Section type pattern determination unit 105 Standard pattern storage unit for genre determination 106 Program genre-specific control information management unit 107 Analysis result output unit

Claims (12)

[Claims] 1. An audio data dividing unit that divides audio data into a plurality of sections, a section type determining unit that determines a type of the section, and a section type that determines a genre of a program using a statistical processing result of the section type. An acoustic data analysis device comprising: a pattern discriminating unit; and a control information management unit that manages control information according to a program genre. 2. The acoustic data analyzer according to claim 1, further comprising a section integrating means for integrating continuous sections of the same type. 3. An acoustic data dividing apparatus, wherein acoustic data is divided into a plurality of sections by using an autocorrelation coefficient of the acoustic data. 4. The acoustic data analysis apparatus according to claim 1, further comprising: a section type determining unit that determines any one of a sound section including a voice, a voice section including no voice, and a silent section. An acoustic data analyzer characterized by the following. 5. An audio data dividing unit that divides audio data into a plurality of sections, a section type determining unit that determines the type of the section, and a control information management unit that manages control information according to a program genre. An acoustic data analysis device comprising: 6. A section type pattern discriminating apparatus comprising: a sound data dividing unit that divides sound data into a plurality of sections; and a section type determining unit that determines a type of the section. 7. A section for dividing sound data into a plurality of sections, a section type determining step for determining the type of the section, and a section for determining a genre of the sound data using a statistical processing result of the section type. A type pattern determination process;
A genre-adaptive section type determination step of determining a section type using control information corresponding to a genre. 8. The acoustic data analysis method according to claim 6, further comprising a section integration step of integrating successive sections of the same type. 9. A method for dividing sound data, wherein the sound data is divided into a plurality of sections using an autocorrelation coefficient of the sound data. 10. The acoustic data analysis method, wherein in the section type determining step, the type is determined to be any of a voiced section including voice, a voiced section not including voice, and a silent section. 11. An audio data dividing step of dividing audio data into a plurality of sections, a section type determining step of determining the type of the section, and a control information managing step of managing control information according to a genre. A method for analyzing acoustic data, characterized in that: 12. A section type pattern discriminating method comprising: an audio data dividing step of dividing sound data into a plurality of sections; and a section type determining step of determining a type of the section.