JP2002366176A - Device and method for information extraction and device and method for information retrieval - Google Patents

Abstract

PROBLEM TO BE SOLVED: To automatically and efficiently retrieve a conversation section of speakers using AV data. SOLUTION: In an information extracting device 20, a voice signal D11 of AV data inputted from an input part 11 is inputted to a cepstrum extraction part 12 to take an LPC analysis and an obtained LPC coefficient is converted into an LPC cepstrum coefficient. Part D12 of the LPC cepstrum coefficient is inputted to a vector quantization part 13, which performs vector quantization. Its quantization distortion D14 is inputted to and evaluated by a speaker identification part 14 and further threshold data D16 are used to identify and determines a speaker by specified recognition blocks. The identified speaker D16 is inputted to a speaker determination frequency calculation part 15, which calculate the recognized speaker determination frequencies of respective speakers in the specified evaluation sections, and outputs them as appearance frequency information D17 of the speakers. An information retrieving device retrieves a part, etc., where a desired speaker converses at a desired frequency according to the appearance frequency information.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information extraction apparatus and method, and an information retrieval apparatus and method, and more particularly, to an information extraction apparatus and method for retrieving audio data or audio image data, and a method thereof. The present invention relates to an information retrieval device and a method thereof.

[0002]

2. Description of the Related Art With the spread of multimedia in recent years,
Efficiently manage a large amount of AV (Audio Visual) data,
The need to perform classification, search, extraction, etc. has increased. For example, it is necessary to retrieve a scene of a certain character or a conversation scene of the person from a large amount of AV data, or to extract and reproduce only a conversation scene of a certain person from the AV data.

Conventionally, when searching for a position on a time axis at which a specific speaker is talking in such AV data, a human is directly watching the AV data while looking at the position on the time axis. And sections.

On the other hand, as a technique for identifying a speaker of voice, an automatic speaker identification / verification technique has been studied. An outline of a conventional technique regarding this technique will be described. First, speaker recognition includes speaker identification and speaker verification. What is speaker identification?
It is to determine which of the pre-registered speakers the input voice is, and the speaker verification is to compare the input voice with data of the pre-registered speakers to identify the speaker. Is determined. The speaker recognition includes a utterance content dependent type in which words (keywords) to be uttered at the time of recognition are determined in advance, and an occurrence content independent type in which an arbitrary word is uttered for recognition.

[0005] As general speech recognition techniques, for example, the following techniques are often used. First, a feature quantity representing the personality of a speaker's voice signal is extracted and recorded in advance as learning data. At the time of inquiry / identification, the input speaker's voice is analyzed, and the feature amount representing the personality is extracted, and the similarity with the training data is evaluated. Do. Here, cepstrum (Cepstrum) or the like is often used as a feature amount representing the personality of the voice. The cepstrum is obtained by performing an inverse Fourier transform on a logarithmic spectrum, and can express an envelope of a voice spectrum by using coefficients of lower-order terms. Further, a polynomial expansion coefficient of a cepstrum time series is called a delta cepstrum, and this is also often used as a feature quantity expressing a temporal change of a speech spectrum. In addition, pitch, delta pitch (polynomial expansion coefficient of pitch), and the like may be used.

The LPC (Linear) extracted in this way
Predictive Coding) Learning data is created using a feature amount such as a cepstrum as a standard pattern. Typical methods include a method based on vector quantization distortion and a method based on a Hidden Markov Model (HMM).

In the method based on vector quantization distortion, feature amounts for each speaker are grouped in advance, and the center of gravity is stored as an element (code vector) of a codebook. Then, the feature amount of the input voice is vector-quantized by the codebook of each speaker, and the average quantization distortion of each codebook for the entire input voice is obtained.

In the case of speaker identification, the speaker of the codebook having the smallest average quantization distortion is selected. In the case of speaker verification, the average quantization distortion by the codebook of the corresponding speaker is set as a threshold. A comparison is made to determine whether the person is the person.

As described above, in the conventional speaker recognition technology, the speech LPC cepstrum is extracted as a feature amount,
A method for performing speaker identification using the vector quantization distortion of the feature amount will be described in detail.

First, an input speech signal is subjected to LPC analysis (linear prediction analysis) for each block, and a linear prediction coefficient (L
PC coefficient). As the analysis block length, about 20 to 30 milliseconds are generally used for voice. Samples x _t of the input signal to predict the equation (1) below from the past P samples. In general, the order P of linear prediction is about the order of 10 to 20.

[0011]

(Equation 1)

Then, a linear prediction coefficient a _i that minimizes the linear prediction error ε = x ′ _t −x _t is obtained by the least square method. There are two methods for finding the solution of the least-squares method: the covariance method and the autocorrelation method.In particular, the autocorrelation method guarantees the positive definiteness of the coefficient matrix and can always find the solution. It is widely used because it can be obtained efficiently by the recursive method. Based on the obtained P linear prediction coefficients, the generation function of the all-pole type speech model estimated is expressed as the following equation (2).

[0013]

(Equation 2)

Since the cepstrum is the inverse Fourier transform of the logarithmic spectrum of the speech, the cepstrum of the speech model obtained by the LPC analysis is the Fourier transform of the cepstrum.
If C (ω), it is expressed by the following equation (3).

[0015]

(Equation 3)

If the Fourier transform is extended to a Z transform and generalized, it can be described as in equation (4).

[0017]

(Equation 4)

The inverse Z transform c _i of C (z) is called a complex cepstrum. Here, the method of converting the LPC coefficients _{a i} directly to the complex cepstrum _{c i} are known. That is, a complex cepstrum can be sequentially obtained from a recurrence formula such as the following formulas (5), (6), and (7). In this way, c _n obtained from the LPC analysis is particularly L
Called PC cepstrum.

[0019]

(Equation 5)

Next, vector quantization is performed on the features (such as LPC cepstrum) extracted as described above, and the speaker is identified using the quantization distortion. Basically,
The obtained feature amount vector is subjected to vector quantization using codebooks of a plurality of speakers, and a codebook that minimizes the average quantization distortion is selected. The details will be described below.

Firstly i-th and P number of feature quantity following expression vectors _{x i} in LPC analysis block (8). As the elements of the feature amount vector, for example, the LPC cepstrum of order 1 to P as described above is used.

[0022]

(Equation 6)

Also, the code book CB_kJ-th sen
Toroid (code vector) r_j ^kTo the following equation (9)
I do.

[0024]

(Equation 7)

[0025] defined here, the weighting distance between the feature quantity vector _{x i} and the centroid _r ^{j k} such that the following equation (10).

[0026]

(Equation 8)

The vector quantization distortion d _k (i) by the codebook CB _k of the i-th block is obtained as in the following equation (11).

[0028]

(Equation 9)

Vector quantization distortion d for each block
_k (i) is obtained, and the codebook CB in all blocks (i = 1, 2,... L) of the speaker recognition section is obtained.
The average quantization distortion D _k of _k obtained as the following equation (12).

[0030]

(Equation 10)

A code book CB _{k ′} that minimizes the average quantization distortion D _k is obtained, and a speaker corresponding to the code book is selected as a speaker in a speaker evaluation section.

On the other hand, in the method based on the HMM, the feature amount of the speaker obtained in the same manner as described above is used as a hidden Markov model (H
MM) is expressed by the transition probability between the states and the appearance probability of the feature amount in each state, and the determination is made based on the average likelihood with the model in the entire input voice section.

In the case of speaker identification that includes an unspecified speaker that is not registered in advance, the determination is made by a method that combines the above-described speaker identification and speaker verification. That is, the most similar speaker is selected from the registered speaker set as a candidate, and the quantization distortion or the likelihood of the candidate is compared with a threshold to determine whether the speaker is the speaker.

In speaker verification or speaker identification including an unspecified speaker, in order to determine the identity of the speaker, the likelihood or quantization distortion of the speaker is determined by comparing it with a threshold. The value of the input data and the learning data (model) vary greatly even for the same speaker due to the temporal fluctuation of the feature amount, the difference in the utterance sentence, and the influence of noise. Even if it is set, a sufficient recognition rate cannot be obtained stably.

Therefore, in speaker recognition in the HMM, it is common practice to normalize the likelihood. For example, there is a method of using a log likelihood ratio LR as shown in the following equation (13) for determination.

[0036]

[Equation 11]

In equation (13), L (X / S _c ) is
Collation target speaker is a likelihood for the input speech X of _{S c (person), L (X / S r} ) is a likelihood for the input speech X of the speaker _{S c} other than the speaker _{S r.} That is, the threshold value is dynamically set in accordance with the likelihood of the input voice X, and the system is robust against differences in utterance content and time variation.

Alternatively, a method of making a determination by posterior establishment as shown in the following equation (14) using the concept of posterior probability has been studied. Here, P (S _c ), P (S
_r ) is the appearance probability of the speakers S _c and S _r , respectively.

[0039]

(Equation 12)

The method of likelihood normalization using these HMMs is described in detail in the following document [4].

On the other hand, there is a method in which the likelihood described in the above-described HMM method is obtained from the Mahalanobis distance of the characteristic amount extracted from the standard pattern of the characteristic amount and the input data.

Using the feature vector x of the input data X and the vector r of the standard pattern of the feature, the input data X
The likelihood L (X / S) for the speaker S is expressed by the following equation (1).
It is required as in 5).

[0043]

(Equation 13)

Here, the feature quantity vector x and the feature pattern standard pattern vector r are expressed by the following equations (1)
6), given by equation (17).

[0045]

[Equation 14]

In equation (15), P is the vector order,
Σ is a covariance matrix of the feature amount data of the speaker S. Also,
(Xr)^TΣ^-1(Xr) is the Mahalanobis distance and
Called. From equation (15), the covariance coefficient of the feature amount of the speaker
Is obtained in advance, the likelihood of the input data X is obtained.
You. In addition, the speaker S as described above will now be described._cAnd speaker S
_rIs the Mahalanobis of each speaker.
Expressed by the difference in distance. That is, as described above
Log likelihood in the likelihood normalization for
Value and a threshold for the difference in Mahalanobis distance.
Is equivalent to specifying For details, see the literature
[5] and so on.

The prior art relating to speaker recognition is described in detail in, for example, the following documents. [1] Furui: "Speaker Recognition by Statistical Features of Cepstrum", IEICE, volJ65-A, No.2 183-190 (1982) [2] FKSoong and AERosenberg: “On the Use of I
nstantaneous and Transitional Spectral Information
in Speaker Recognition. ”, IEEE Trans. ASSP, Vol.
36, No.6, pp.871-879 (1988) [3] Furui: "Story of the voice's personality", Journal of the Acoustical Society of Japan, 51,1
1, pp.876-881, (1995) [4] Matsui: "HMM-based speaker recognition," IEICE Technical Report, Vol.95,
No.467, (SP95 109-116) pp.17-24 (1996) [5] THE DIGITAL SIGNAL PROCESSING HANDBOOK, IEEE P
RESS (CRC Press), 1998 On the other hand, in the specification and the drawings of Japanese Patent Application No. 2000-363547, the technology of speaker recognition is applied to the AV data,
A technique for detecting a continuous conversation section of the same speaker and a speaker switching position has been proposed. This patent application 2000-363
In the specification and drawings of 547, the audio signal of the AV data is classified and identified into speaker groups for each small section (about 1 to 2 seconds), and several continuous recognition sections (several seconds to about 10 seconds).
The change of the discrimination frequency of the speaker group is obtained within the section, and the position where the frequency is above or below the threshold is detected to detect the switching position of the speaker, and the section during the switching of the speaker is determined. It is detected as the same speaker continuous conversation section of the speaker.

[0048]

The conventional speaker recognition technology has been researched and developed with the main application of identification and collation of a single speaker in a security system or the like, and a plurality of speakers are included in one voice data. However, it cannot be applied to actual complex conversation scenes, such as alternately uttering in a short time, occasionally uttering simultaneously, and music or noise in the background. Therefore, if the search for the conversation section of the speaker in the AV data is performed automatically by the conventional speaker recognition technique, there is a problem that the discrimination performance is significantly reduced.

Also, the aforementioned Japanese Patent Application No. 2000-36354.
7, the technology described in the specification and the drawings of FIG.
Since the switching of the speaker is detected based on the change in the discrimination frequency for each frequency evaluation section of about 10 seconds, the same speaker needs to have a substantially continuous conversation during the evaluation section. That is, the same speaker has an evaluation section length of 10
It can be applied when the discrimination error rate is sufficiently low for a period of about two seconds alone, but it can be applied, but multiple speakers speak alternately within a short time, for example, within a few seconds. If the speaker's recognition error is large due to frequent or simultaneous utterances or background noise, music, etc., it is not possible to accurately detect the switch position of the speaker, and appropriate detection of the conversation section There was a problem that can not be.

Further, in the specification and drawings of Japanese Patent Application No. 2000-363547, since speakers are identified by a method of allocating input data to any of the speaker groups, unknown speakers that have not been registered are identified. Classification can be performed, but since the identity of the speaker is not verified, classification errors are likely to occur, and even if data other than voice is input, the data is classified into any speaker group. was there.

In such speaker detection of AV data, there is a speaker verification technique as a method for avoiding erroneous identification of unregistered speakers or input data other than voice.
A conventional speaker matching method based on likelihood normalization of a speaker is HMM.
Can be applied when the likelihood is obtained by using, but there is a problem that it cannot be applied as it is by an identification method using vector quantization distortion, which can perform identification more easily.

Further, in order to obtain the likelihood using the Mahalanobis distance from the standard pattern, the covariance coefficient of the speaker and the like must be known, the calculation is very complicated, and furthermore, this method is applied to vector quantization. Applying the case where distortion is used requires complicated calculations such as obtaining a covariance coefficient in advance, and is not practical.

The present invention has been proposed in view of such a conventional situation, and automatically and efficiently detects a talk section of a speaker in AV data and extracts information for efficient search. It is an object to provide an apparatus and a method thereof, and an information retrieval apparatus and a method thereof.

[0054]

In order to achieve the above-mentioned object, an information extraction apparatus according to the present invention comprises: an information extraction apparatus for extracting desired information from a predetermined information source;
With respect to the audio signal as the information source, a speaker identification unit for identifying a speaker for each evaluation section based on the similarity of the audio in the audio signal, and a frequency of the speaker determined for each evaluation section are obtained. A speaker discrimination frequency calculation unit that obtains the discrimination frequency information of the speaker in the frequency section of the information source that is the section, and detects the appearance frequency information of the speaker in the frequency section.

Here, the information extracting apparatus uses L.sub.L as a feature amount for evaluating the similarity of the voice in the voice signal of the information source.
An LPC cepstrum obtained by PC analysis is used, a vector quantization of a feature quantity by a plurality of codebooks is used as an identification method, and the vector quantization distortion is used as a measure of identification.

In the information extracting apparatus, the minimum quantization distortion is subtracted from the sum or average of the minimum quantization distortion, which is the minimum value of the vector quantization distortion, and a plurality of vector quantization distortions other than the minimum quantization distortion. The identification value is determined by comparing the set values with threshold values set in advance.

Such an information extracting device identifies a speaker for each evaluation section based on the feature amount of the speaker's voice in the voice signal, and also determines the frequency of the speaker determined for each evaluation section. The appearance frequency of the speaker in the frequency section which is the section to be obtained is detected, and the appearance frequency information of the speaker is generated.

According to another aspect of the present invention, there is provided an information extraction method for retrieving predetermined information from a predetermined information source. A speaker identification step of determining a speaker for each evaluation section based on the similarity of the voice in the voice signal, and a frequency section in the information source for obtaining a frequency of the speaker determined for each evaluation section And a speaker discriminating frequency calculating step of obtaining the speaker discriminating frequency information in step (a), and detecting the appearance frequency information of the speaker in the frequency section.

Here, in the information extraction method, L is used as a feature value for evaluating the similarity of the voice in the voice signal of the information source.
An LPC cepstrum obtained by PC analysis is used, a vector quantization of a feature quantity by a plurality of codebooks is used as an identification method, and the vector quantization distortion is used as a measure of identification.

In the information extraction method, the minimum quantization distortion is subtracted from the sum or average of the minimum quantization distortion, which is the minimum value of the vector quantization distortion, and a plurality of vector quantization distortions other than the minimum quantization distortion. The identification value is determined by comparing the set values with threshold values set in advance.

According to such an information extraction method, a speaker is identified for each evaluation section based on the feature amount of the speaker's voice in the audio signal, and the frequency of the speaker determined for each evaluation section is determined. The appearance frequency of the speaker in the frequency section which is the section to be obtained is detected, and the appearance frequency information of the speaker is generated.

Further, in order to achieve the above-mentioned object, the information retrieval apparatus according to the present invention, for a speech signal as an information source, relies on the similarity of speech in the speech signal to identify a speaker in each evaluation section. Discrimination, and the appearance frequency information of the speaker in the frequency section obtained by obtaining the speaker identification frequency information in the frequency section which is the section for obtaining the frequency of the speaker determined for each evaluation section is determined in advance. What is claimed is: 1. An information search device for searching for desired information from a recorded recording medium, comprising: speaker appearance frequency reading means for reading speaker appearance frequency information recorded on the recording medium; A search condition input unit for inputting a condition, and a speaker appearance section output for comparing the input search condition with information read from the recording medium and outputting information corresponding to the search condition as speaker appearance section information. It is characterized in that it comprises a stage.

Here, the LPC cepstrum obtained by the LPC analysis is used as the characteristic amount for evaluating the similarity of the voice in the voice signal of the information source when obtaining the speaker discrimination frequency information. As a method, vector quantization of feature amounts by multiple codebooks is used, and the vector quantization distortion is used as a measure of identification,
In addition, a minimum quantization distortion, which is the minimum value of the vector quantization distortion, and a value obtained by subtracting the minimum quantization distortion from a sum or an average of a plurality of vector quantization distortions other than the minimum quantization distortion are set in advance. The identification is determined by comparing with the threshold value.

Such an information search apparatus compares the appearance frequency information of a desired speaker with the input search condition,
A part where a desired speaker is talking at a desired frequency is searched.

Further, in order to achieve the above-mentioned object, the information search method according to the present invention, for a voice signal as an information source, identifies a speaker in each evaluation section based on the similarity of voice in the voice signal. Discrimination, and the appearance frequency information of the speaker in the frequency section obtained by obtaining the speaker identification frequency information in the frequency section which is the section for obtaining the frequency of the speaker determined for each evaluation section is determined in advance. An information retrieval method for retrieving desired information from a recorded recording medium, comprising: a speaker appearance frequency reading step of reading speaker appearance frequency information recorded on the recording medium; and a search for a desired speaker. A search condition input step of inputting a condition, a speaker appearance section output for comparing the input search condition with information read from the recording medium, and outputting information corresponding to the search condition as speaker appearance section information. It is characterized by having a degree.

Here, the LPC cepstrum obtained by the LPC analysis is used as a feature amount for evaluating the similarity of the voice in the voice signal of the information source when obtaining the speaker discrimination frequency information. As a method, vector quantization of feature amounts by multiple codebooks is used, and the vector quantization distortion is used as a measure of identification,
In addition, a minimum quantization distortion, which is the minimum value of the vector quantization distortion, and a value obtained by subtracting the minimum quantization distortion from a sum or an average of a plurality of vector quantization distortions other than the minimum quantization distortion are set in advance. The identification is determined by comparing with the threshold value.

In such an information search method, the appearance frequency information of a desired speaker is compared with the input search condition.
A part where a desired speaker is talking at a desired frequency is searched.

[0068]

Embodiments of the present invention will be described below in detail with reference to the drawings.

First, FIG. 1 shows a conceptual configuration diagram of the information extraction device according to the present embodiment. As shown in FIG. 1, in the information extraction device, a speech signal serving as an information source is input to speaker identification means 1, and a speaker is identified by evaluating vector quantization distortion.

The speaker identified by the speaker identification means 1 is input to the speaker identification frequency calculation means 2 and the recognized speaker identification frequency of each speaker in the section is calculated for each predetermined evaluation section. You. The obtained speaker discrimination frequency is output as speaker appearance frequency information.

FIG. 2 shows a specific configuration example of the information extraction device shown in FIG. As shown in FIG. 2, the information extraction device 10 includes an input unit 11 for inputting an audio signal of AV (Audio Visual) data, and an LPC (Linear) by analyzing the audio signal.
Predictive Coding) A cepstrum extraction unit 12 that extracts cepstrum coefficients, a vector quantization unit 13 that vector-quantizes LPC cepstrum coefficients, a speaker identification unit 14 that evaluates vector quantization distortion to identify a speaker, and recognizes And a speaker discrimination frequency calculation unit 15 for obtaining the appearance frequency of the speaker using the determined speaker discrimination frequency.

In FIG. 2, the code book group CB
Is a table in which codebook data of each speaker used for vector quantization is stored, and a threshold table file TF is a table in which threshold data for discriminating the speakers is stored. Recorded in the department. The speaker frequency file SF records the frequency of each speaker in each section.

The operation of the information extracting apparatus 10 configured as described above will be described below. AV input from the input unit 11
The data audio signal D11 is input to the cepstrum extraction unit 12 in block units, subjected to LPC analysis, and the obtained LPC coefficients are converted into LPC cepstrum coefficients.

Part of the obtained LPC cepstrum coefficient D
Numeral 12 is input to the vector quantization unit 13, and is subjected to vector quantization using the codebook data D13 of each speaker from the codebook group CB. The result (quantization distortion) D14 of the vector quantization performed by each codebook is input to the speaker identification unit 14 and evaluated,
Further, the threshold data D read from the threshold table file TF
15, the speaker is identified and determined for each predetermined recognition block.

The identified speaker D16 is input to the speaker discrimination frequency calculator 15, and the recognized speaker discrimination frequency of each speaker in the section is calculated for each predetermined evaluation section. The obtained speaker discrimination frequency is recorded as speaker appearance frequency information D17 in a speaker frequency file SF for each AV data, each speaker, and each evaluation section in a recording format as shown in FIG. 3, for example. Is done. The speaker frequency file SF may be communicated via a communication line by a transmitting / receiving unit (not shown), or may be a recording medium such as a magnetic disk, a magneto-optical disk, or a storage medium such as a semiconductor memory. It may be stored in a storage medium.

The recording format of FIG. 3 includes an AV data name of an audio signal input from the input unit 11, an identification name for identifying each registered speaker, a start time of a frequency section, and an end time of the same section. And the speaker identification frequency in the frequency section of the AV data as information. This recording format is
This is an example, and the present invention is not limited to the information shown in FIG.

The processing for identifying the speaker and obtaining the speaker discrimination frequency will be described in more detail with reference to FIGS. 2 and 4.

The audio signal of the input AV data is converted by the cepstrum extraction unit 12 into an LPC as shown in FIG.
LPC analysis is performed for each analysis block AB, and the obtained LPC coefficients are converted to extract LPC cepstrum coefficients. The block length a of the LPC analysis block AB is
In the case of an audio signal, usually about 20 to 30 ms is often used. In addition, in order to improve the analysis performance, it often overlaps with an adjacent block slightly.

The speaker recognition block RB in FIG. 4 is the minimum unit for identifying a speaker, and identifies a speaker in this block unit. The block length b of the speaker recognition block RB is desirably about several seconds. Therefore, one speaker recognition block R
B includes about 50 to several hundred LPC analysis blocks AB. The speaker recognition block RB may also slightly overlap with the adjacent section. The overlap length is usually about 10% to 50% of the section length.

The frequency section FI in FIG. 4 indicates the appearance frequency of the speaker.
This is the required evaluation unit.
Based on the discrimination frequency of the speaker identified in block RB, each
Find the appearance frequency of the speaker. Frequency section FI_IAt the start of the section
The time is S_I, The section end time is E _IAnd the section length (E_I-
S_I) Is suitably from several minutes to several tens of minutes. Also rated
The section may also slightly overlap with the adjacent section.

FIG. 5 is a flowchart showing the operation of the information extracting apparatus 10. First, in step S10, the section number I is set to 0 as initialization processing. The section number I is a serial number assigned to the frequency section FI for obtaining the frequency of the speaker.

Next, in step S11, speaker candidates are identified and speaker candidates are selected in units of the above-described speaker recognition blocks RB. A method for selecting speaker candidates will be described later in detail.

In step S12, it is determined whether or not the selected speaker candidate is a correct speaker. That is, when an unknown unspecified speaker or data other than voice is input, the candidate speaker selected in step S11 selects the speaker most similar to the input voice as a candidate. But he is not necessarily the speaker himself. Therefore, step S12
Then, by evaluating the vector quantization distortion and comparing it with threshold data recorded in the threshold table file TF shown in FIG. 2, it is determined whether or not the data is the data of the selected speaker candidate. The determination method will be described later in detail. In step S12, if it is determined that the speaker is the speaker, the speaker candidate is determined as the speaker in this speaker recognition block, and if it is determined that the speaker is not the speaker, the speaker in the speaker recognition block is determined as the unknown speaker. Is determined.

Subsequently, in step S13, the frequency section FI
_It is determined whether or not processing has been performed up to the last speaker recognition block RB of _I. If it is not the last speaker recognition block RB in step S13, the process proceeds to the next speaker recognition block RB in step S14, and returns to step S11. If it is determined in step S13 that the block is the last speaker recognition block RB, the process proceeds to step S15.

In step S15, the current frequency section FI
_{In I} , the determination frequency of each registered speaker is obtained as appearance frequency information. Note that the speaker recognition block RB determined to be an unknown speaker is not included in the frequency calculation. The obtained speaker appearance frequency is stored in the speaker frequency file SF shown in FIG.
It is recorded in a recording format such as

In step S16, it is determined whether the end of the data has been reached. If the end of the data has been reached, the process ends. If the end of the data has not been reached, the process proceeds to step S17.

In step S17, the section number I is incremented by one, the process proceeds to the next frequency section, and the process returns to step S11.

FIG. 6 shows details of the method for identifying speaker candidates in step S11 of FIG. First, in step S20, audio data is read from input data for each of the above-described LPC analysis blocks AB.

Next, in step S21, it is determined whether or not the processing has been completed up to the last LPC analysis block AB of the speaker recognition block RB.
If the processing has been completed, the process proceeds to step S26. If it is not the last LPC analysis block AB in step S21, the process proceeds to step S22.

In step S22, the data of the obtained LPC analysis block AB is evaluated to determine whether or not this block is a voice block. If it is determined in step S22 that the LPC analysis block AB is a silence block or a non-speech block, the analysis of this block is skipped, the process proceeds to step S25, the process proceeds to the next LPC analysis block AB, and the process proceeds from step S20. Perform processing. A method of determining whether a block is an audio block includes, for example,
As the simplest method, the power average and the maximum value of the block may be evaluated to simply detect whether or not the block is a silent block. In addition, there are various methods for analyzing the average power of the signal, the number of zero crossings, the presence or absence of a pitch, the spectrum shape, and the like to determine whether or not the data is audio data. Alternatively, this step may be omitted.

If it is determined in step S22 that the block is a voice block, then in step S23,
The LPC analysis of this block is performed, and the obtained LPC coefficient is converted to extract the LPC cepstrum coefficient. Here, low-order cepstrum coefficients of the first to fourteenth orders are extracted.

Next, in step S24, the LPC cepstrum coefficients obtained in step S23 are vector-quantized using a plurality of codebooks created in advance. Each codebook has a one-to-one correspondence with a registered speaker. Here, let d _k be the vector quantization distortion of the LPC cepstrum coefficient of this block by the codebook CB _k .

In step S25, the process proceeds to the next LPC analysis block AB, returns to step S20, and similarly repeats the processing from step S20 to step S25.

In step S26, the speaker recognition block R
An average quantization distortion D _k which is an average of the quantization distortion d _k of each codebook CB over the entire B is _obtained .

Subsequently, in step S27, a codebook CB _{k ′} corresponding to the speaker S _{k ′} that minimizes the average quantization distortion D _k is selected, and in step S28, the code book CB _{k ′} is selected.
and it outputs a _{k 'as} a speaker candidate _{S c.}

[0096] Thus, among the speakers codebook is registered, the voice of the most input data speakers are similar, selects a speaker candidate S _c at the speaker recognition block RB.

[0097] Next, FIG. 7 shows the details of the matching determination method of the speaker candidates _{S c} at step S12 in FIG. 5. First, in step S30, the average quantization distortion of the speaker candidates _{S c} and _{D 0.} Next, in step S31, the speaker candidate S
_The average quantization distortion by each codebook other than _c is rearranged in ascending order.
D ₁ , D ₂ ,... D _n (D ₀ <D ₁ <D ₂ <.
D _n ). The value of n can be arbitrarily selected.

[0098] Subsequently, in Step S32, as a measure of evaluation, for n quantization distortion otherwise the quantization distortion _{D 0} of the speaker candidates _{S c,} using the following equation (18) or (19) To obtain the distortion difference amount ΔD.

[0099]

(Equation 15)

[0100] Equation (18), in the formula (19), for example, when n is 1, to determine the difference between the quantization distortion of the speaker candidates _S quantization distortion Following _c is smaller _{D 1} and _{D 0} Become.

[0102] Subsequently, in Step S33, Reads a threshold data corresponding to the speaker candidate S _c from the threshold table file TF shown in FIG.

In the threshold table file TF, for example, a format as shown in FIG. 8 is recorded for each registered speaker. That is, as shown in FIG. 8, the speaker identifier of each registered speaker, the maximum distortion absolute value D _max of the quantization distortion as the threshold data, and the minimum distortion difference ΔD _min are recorded in advance.

[0103] Returning to FIG. 7, in step S34, the threshold value data _{D max} yelling read, the [Delta] D _{mi n,} determined as compared to the _{D 0} and [Delta] D obtained. That is, step S34
, The absolute value D ₀ of the quantization distortion is equal to the threshold data D
_{If the} difference is smaller than _max and the distortion difference ΔD is larger than the threshold data ΔD _min , the process proceeds to step S35, the person is determined to be the person, and the candidate is determined. Otherwise,
Proceeding to step S36, the speaker is determined to be an unknown speaker, and the candidate is rejected. Thus, the average quantization distortion D ₀ of the speaker candidate _Sc is
By comparing the threshold value and the distortion difference amount ΔD with the threshold value, the identification error of the voice data of the registered speaker is reduced, and voice data other than the registered speaker can be determined as an unknown speaker.

As described above, the information extracting apparatus according to the present embodiment identifies the speaker for each speaker recognition block based on the feature amount of the speaker's voice in the audio signal of the AV data. The appearance frequency of the speaker in a predetermined section is detected, and the appearance frequency information of the speaker is generated. By supplying the appearance frequency information to an information search device described later via a communication line or a recording medium, desired information can be effectively searched in the information search device.

Next, an information retrieval apparatus according to the present embodiment will be described. First, a conceptual configuration diagram of the information search device is shown in FIG. As shown in FIG. 9, in the information retrieval apparatus, the appearance frequency information of the speaker generated by the above-described information extraction apparatus is read by the speaker appearance frequency reading means 3, and the search condition is determined by the search condition. It is input to the input means 4. The speaker appearance section output means 5 outputs the speaker appearance section information searched based on the appearance frequency information and the search condition.

FIG. 10 shows a specific configuration example of the information extracting apparatus shown in FIG. As shown in FIG. 10, the information search device 20 includes a condition input unit 21 for inputting search conditions, a data search unit 22 for searching information from the input conditions,
An output unit 23 that outputs a search result. Further, the speaker frequency file SF is a file in which speaker appearance frequency information is recorded, and the information is recorded in a recording format as shown in FIG. 3 described above.

[0107] The operation of the information retrieval apparatus 20 thus configured will be described below. The condition input unit 21 inputs search conditions for searching AV data. The search conditions include, for example, the name and identification number of a desired speaker, the conversation frequency of the speaker, and AV data to be searched.

The input search condition D22 is input to the data search unit 22, and information matching the search condition is searched. The data search unit 22 reads the speaker appearance frequency information D23 with reference to the speaker frequency information file SF, compares it with the search condition D22, and supplies a search result D24 to the output unit 23. The output unit 23 outputs the search result D24 as speaker appearance section information D25.

As described above, the information retrieval apparatus according to the present embodiment uses the AV input via a communication line or a recording medium.
By comparing the appearance frequency information of the desired speaker in the audio signal of the data with the input search condition, it is possible to effectively search for a portion where the desired speaker is talking at a desired frequency. .

It should be noted that the present invention is not limited to only the above-described embodiment, and it is needless to say that various modifications can be made without departing from the spirit of the present invention.

[0111]

As described in detail above, the information extracting apparatus according to the present invention is an information extracting apparatus for extracting desired information from a predetermined information source. Speaker identification means for determining a speaker for each evaluation section based on the similarity of the voice in the signal;
Speaker discrimination frequency calculation means for obtaining the discrimination frequency information of the speaker in the frequency section of the information source, which is a section for obtaining the frequency of the speaker discriminated in each of the evaluation sections. It is characterized by detecting appearance frequency information of a person.

Here, the information extracting apparatus uses L.sub.L as a feature amount for evaluating the similarity of the voice in the voice signal of the information source.
An LPC cepstrum obtained by PC analysis is used, a vector quantization of a feature quantity by a plurality of codebooks is used as an identification method, and the vector quantization distortion is used as a measure of identification.

In the information extracting apparatus, the minimum quantization distortion is subtracted from the sum or average of the minimum quantization distortion, which is the minimum value of the vector quantization distortion, and a plurality of vector quantization distortions other than the minimum quantization distortion. The identification value is determined by comparing the set values with threshold values set in advance.

According to such an information extracting apparatus, a speaker is identified for each evaluation section based on the feature amount of the speaker's voice in the audio signal, and the frequency of the speaker determined for each evaluation section is determined. Can be detected, and the appearance frequency information of the speaker can be generated. By supplying the appearance frequency information to the information search device via a communication line or a recording medium, desired information can be effectively searched in the information search device.

The information extraction method according to the present invention is the information extraction method for retrieving predetermined information from a predetermined information source, wherein the audio signal as the information source has a similarity of audio in the audio signal. A speaker identification step of identifying a speaker for each evaluation section, and the speaker identification frequency information in a frequency section of the information source, which is a section for obtaining the frequency of the speaker determined for each evaluation section And a speaker discrimination frequency calculation step of determining the speaker frequency in the frequency section.

Here, in the information extraction method, L is used as a feature amount for evaluating the similarity of the voice in the voice signal of the information source.
An LPC cepstrum obtained by PC analysis is used, a vector quantization of a feature quantity by a plurality of codebooks is used as an identification method, and the vector quantization distortion is used as a measure of identification.

In the information extraction method, the minimum quantization distortion is subtracted from the sum or average of the minimum quantization distortion, which is the minimum value of the vector quantization distortion, and a plurality of vector quantization distortions other than the minimum quantization distortion. The identification value is determined by comparing the set values with threshold values set in advance.

According to such an information extraction method, the speaker is identified for each evaluation section based on the feature amount of the speaker's voice in the audio signal, and the frequency of the speaker determined for each evaluation section is determined. Can be detected, and the appearance frequency information of the speaker can be generated. By supplying the appearance frequency information to the information search device via a communication line or a recording medium, desired information can be effectively searched in the information search device.

Further, the information retrieval apparatus according to the present invention determines a speaker for each evaluation section based on the similarity of the voice in the voice signal as the information source, and determines the speaker for each evaluation section. From the recording medium in which the appearance frequency information of the speaker in the frequency section obtained by obtaining the speaker identification frequency information in the frequency section that is the section for obtaining the frequency of the An information retrieval apparatus for retrieving information, comprising: speaker appearance frequency reading means for reading speaker appearance frequency information recorded on the recording medium; and search condition input means for inputting a desired speaker search condition. Speaker input section output means for comparing the input search condition with information read from the recording medium, and outputting information corresponding to the search condition as speaker appearance section information. That.

Here, the LPC cepstrum obtained by the LPC analysis is used as a feature value for evaluating the similarity of the voice in the voice signal of the information source when obtaining the appearance frequency information of the speaker. As a method, vector quantization of feature amounts by multiple codebooks is used, and the vector quantization distortion is used as a measure of identification,
In addition, a minimum quantization distortion, which is the minimum value of the vector quantization distortion, and a value obtained by subtracting the minimum quantization distortion from a sum or an average of a plurality of vector quantization distortions other than the minimum quantization distortion are set in advance. The identification is determined by comparing with the threshold value.

According to such an information search apparatus, by comparing the appearance frequency information of a desired speaker with the input search condition, a portion where the desired speaker is talking at a desired frequency can be effectively determined. Can be searched.

Further, in the information search method according to the present invention, a speaker is determined for each evaluation section based on the similarity of the voice in the voice signal as the information source, and the speaker is determined for each evaluation section. From the recording medium in which the appearance frequency information of the speaker in the frequency section obtained by obtaining the speaker identification frequency information in the frequency section that is the section for obtaining the frequency of the An information retrieval method for retrieving information, comprising: a speaker appearance frequency reading step of reading speaker appearance frequency information recorded on the recording medium; and a search condition input step of inputting a desired speaker search condition. Comparing the input search conditions with the information read from the recording medium, and outputting information corresponding to the search conditions as speaker appearance section information. That.

Here, the LPC cepstrum obtained by the LPC analysis is used as the characteristic amount for evaluating the similarity of the voice in the voice signal of the information source when obtaining the speaker discrimination frequency information. As a method, vector quantization of feature amounts by multiple codebooks is used, and the vector quantization distortion is used as a measure of identification,
In addition, a minimum quantization distortion, which is the minimum value of the vector quantization distortion, and a value obtained by subtracting the minimum quantization distortion from a sum or an average of a plurality of vector quantization distortions other than the minimum quantization distortion, are preset. The identification is determined by comparing with the threshold value.

According to such an information search method, by comparing the appearance frequency information of a desired speaker with the input search condition, a portion where the desired speaker is talking at a desired frequency can be effectively determined. Can be searched.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating a conceptual configuration of an information extraction device according to the present embodiment.

FIG. 2 is a diagram illustrating a configuration example of the information extraction device.

FIG. 3 is a diagram illustrating an example of a recording format of speaker appearance frequency information in the information extraction device.

FIG. 4 is a diagram illustrating a relationship among a frequency evaluation section, a speaker recognition block, and an LPC analysis block in the information extraction device.

FIG. 5 is a flowchart illustrating the operation of the information extraction device.

FIG. 6 is a flowchart illustrating speaker identification processing in units of speaker recognition blocks in the information extraction device.

FIG. 7 is a flowchart illustrating a speaker verification determination process in the information extraction device.

FIG. 8 is a diagram illustrating an example of a recording format of threshold data for speaker verification determination in the information extraction device.

FIG. 9 is a diagram illustrating a conceptual configuration of an information search device according to the present embodiment.

FIG. 10 is a diagram illustrating a configuration of the information search device.

[Explanation of symbols]

1 speaker identification means, 2 speaker discrimination frequency calculation means, 3 speaker appearance frequency reading means, 4 search condition input means, 5
Speaker appearance section output means, 10 information extraction device, 11 input section, 12 cepstrum extraction section, 13 vector quantization section, 14 speaker identification section, 15 speaker discrimination frequency calculation section,
Reference Signs List 20 information search device, 21 condition input unit, 22 data search unit, 23 output unit

Claims (12)

[Claims] 1. An information extracting apparatus for extracting desired information from a predetermined information source, comprising: a speaker for each evaluation section of a voice signal as the information source, based on a similarity of voice in the voice signal. Speaker identification means for determining the frequency of the speaker determined for each evaluation section; and speaker determination frequency calculation means for obtaining the speaker determination frequency information in a frequency section of the information source, which is a section for obtaining the frequency of the speaker determined for each evaluation section. An information extraction device comprising: detecting appearance frequency information of the speaker in the frequency section. 2. An LPC cepstrum obtained by LPC analysis is used as a feature value for evaluating the similarity of the voice in the voice signal of the information source, and a vector quantization of the feature value by a plurality of codebooks is used as an identification method. The information extraction apparatus according to claim 1, wherein the vector quantization distortion is used as a measure of identification. 3. A minimum quantization distortion, which is a minimum value of the vector quantization distortion, and a value obtained by subtracting the minimum quantization distortion from a sum or an average of a plurality of vector quantization distortions other than the minimum quantization distortion, 3. The information extraction device according to claim 2, wherein the identification determination is performed by comparing each with a preset threshold value. 4. An information extraction method for retrieving predetermined information from a predetermined information source, wherein, for a speech signal as the information source, a speaker is determined for each evaluation section by a similarity of speech in the speech signal. A speaker identification step of determining the frequency of the speaker determined in each of the evaluation sections; An information extraction method comprising: detecting appearance frequency information of the speaker in the frequency section. 5. An LPC cepstrum obtained by LPC analysis is used as a feature for evaluating the similarity of the voice in the voice signal of the information source, and a vector quantization of the feature using a plurality of codebooks is used as an identification method. 5. The method according to claim 4, wherein the vector quantization distortion is used as a measure of identification. 6. A minimum quantization distortion, which is a minimum value of the vector quantization distortion, and a value obtained by subtracting the minimum quantization distortion from a sum or an average of a plurality of vector quantization distortions other than the minimum quantization distortion, 6. The information extraction method according to claim 5, wherein identification determination is performed by comparing each of the threshold values with a preset threshold value. 7. A section in which a speaker is determined for each evaluation section based on the similarity of the voice in the voice signal as the information source, and a frequency of the speaker determined for each evaluation section is obtained. From the recording medium in which the appearance frequency information of the speaker in the frequency section obtained by obtaining the determination frequency information of the speaker in the frequency section in the information source is previously recorded, desired information is searched. A speaker appearance frequency reading means for reading the appearance frequency information of the speaker recorded on the recording medium; a search condition input means for inputting a desired speaker search condition; An information search device comprising: speaker appearance section output means for comparing the search condition with information read from the recording medium and outputting information corresponding to the search condition as speaker appearance section information. . 8. An LPC cepstrum obtained by LPC analysis is used as a feature for evaluating the similarity of the voice in the voice signal of the information source, and vector quantization of the feature using a plurality of codebooks is used as an identification method. The information retrieval apparatus according to claim 7, wherein the vector quantization distortion is used as a measure of identification. 9. A minimum quantization distortion, which is a minimum value of the vector quantization distortion, and a value obtained by subtracting the minimum quantization distortion from a sum or an average of a plurality of vector quantization distortions other than the minimum quantization distortion, 9. The information search apparatus according to claim 8, wherein the identification is determined by comparing each with a preset threshold value. 10. A section in which a speaker is determined for each evaluation section based on the similarity of the voice in the voice signal, and a frequency of the speaker determined for each evaluation section is determined. From the recording medium in which the appearance frequency information of the speaker in the frequency section obtained by obtaining the determination frequency information of the speaker in the frequency section in the information source is previously recorded, desired information is searched. A speaker appearance frequency reading step of reading speaker appearance frequency information recorded on the recording medium; a search condition input step of inputting a desired speaker search condition; A speaker appearance section output step of comparing the search condition with information read from the recording medium and outputting information corresponding to the search condition as speaker appearance section information. Law. 11. An LPC cepstrum obtained by LPC analysis is used as a feature value for evaluating the similarity of the voice in the voice signal of the information source, and vector quantization of the feature value using a plurality of codebooks is used as an identification method. 11. The information retrieval method according to claim 10, wherein the vector quantization distortion is used as a measure of identification. 12. A minimum quantization distortion which is a minimum value of the vector quantization distortion, and a value obtained by subtracting the minimum quantization distortion from a sum or an average of a plurality of vector quantization distortions other than the minimum quantization distortion, 12. The information search method according to claim 11, wherein the identification is determined by comparing each with a preset threshold value.