JP4973352B2 - Voice processing apparatus and program - Google Patents

Description

  The present invention relates to a technique for classifying (clustering) a plurality of sections obtained by dividing an audio signal on a time axis for each speaker.

If voice signals collected in an environment where a plurality of speakers speak at any time (for example, a conference) can be classified and classified for each speaker, it is convenient to use it for the preparation of conference minutes, for example. Patent Document 1 discloses a technique for extracting acoustic feature amounts for each of a plurality of sections obtained by dividing an audio signal at predetermined time intervals, and classifying a plurality of sections having similar feature quantities as the voice of the same speaker. It is disclosed.
JP 2005-321530 A

However, for example, in a configuration in which each section is selected and classified in order of short time or chronological order, it is difficult to accurately classify a plurality of sections for each speaker. In view of the above circumstances, an object of the present invention is to solve the problem of accurately classifying a plurality of sections into which audio signals are divided.

In order to solve the above-described problems, the speech processing apparatus according to the present invention includes speech classification means for partitioning a speech signal into a plurality of variable length sections on the time axis with a valley portion of a waveform envelope of the speech signal as a boundary. When a cluster designation means for sequentially specifying the target cluster as the grouping destination of each section, and the cluster information generating means for generating a cluster information including an acoustic model of the target cluster, unclassified audio signal during specified target cluster Section selection means for sequentially selecting each section as a selection section in order of long time, similarity determination means for determining similarity between the feature amount of the audio signal in the selection section and the acoustic model of the target cluster, A section classifying section that classifies the selected section into a target cluster when the rejection determination section determines that it is similar, and a target class based on the feature amount of the audio signal in the selection section when the similarity determination section determines that the selection section is similar. Comprising an updating means for updating the acoustic model of. According to the above configuration, the uncategorized sections are selected in the order of the longest time among the plurality of variable length sections that segment the speech signal with the valleys of the envelope of the waveform of the speech signal as a boundary. Since the feature quantities of the selected sections classified into clusters are used to update the acoustic model, the acoustic model is more reliable compared to a configuration in which unclassified sections are selected in the order of short time or chronological order, for example. It is possible to classify each section with high accuracy by improving the performance at an early stage.

  In a preferred aspect of the present invention, the cluster information generating means generates an acoustic model based on the feature amount of the speech signal in the unclassified longest section (for example, step SA2 to step SA4 in FIGS. 3 and 6). According to this aspect, the reliability of the acoustic model can be improved at an early stage as compared with a configuration in which the unclassified shortest section or the section that is first in time is used for creating the acoustic model.

In a preferred aspect of the present invention, the cluster information of the target cluster depends on the acoustic model including the first model and the second model, each generated by a separate method, and the time length of the section classified into the target cluster. The similarity determination means determines whether the classification interval length in the cluster information exceeds a threshold value, and if the determination result is affirmative, the feature amount of the audio signal in the selected interval And the first model are determined, and if the determination result is negative, the similarity between the feature amount of the audio signal in the selected section and the second model is determined. According to the above configuration, since the advantages of both the first model and the second model can be used, the accuracy of classification of each section can be further increased. Note that an acoustic model (for example, a mixed model such as a Gaussian mixture model) having a property of improving reliability particularly when the classification section length is long is preferably used for the first model, and the classification section length is used for the second model. An acoustic model (for example, a codebook) having a property that can ensure reliability even when the number is short is preferably used. The classification section length is, for example, the total time length of the sections classified into the target cluster or the total number of feature quantities extracted from the sections classified into the target cluster.

  In a specific aspect of the present invention, the cluster information of the target cluster includes a classification section length corresponding to the time length of the section classified into the target cluster, and the similarity determination unit is characterized by the feature of the audio signal in the selected section By comparing the similarity index value (for example, average likelihood L or VQ distortion D) between the quantity and the acoustic model of the target cluster with the similarity determination threshold (for example, threshold Lth or Dth), the similarity of both is determined. Threshold setting means for variably setting the similarity determination threshold according to the classification section length in the information is provided. According to the above aspect, since the similarity determination threshold value is variably controlled, there is an advantage that omission of classification or erroneous determination of similarity can be effectively prevented.

  The speech processing apparatus according to a preferred aspect of the present invention includes a speaker number specifying means for specifying the number of speakers, and finishes the classification when each section is classified into a number of clusters corresponding to the number of speakers. . The speech processing apparatus according to another aspect includes a speaker number specifying unit that specifies the number of speakers, and the total number of clusters is equal to or less than the number of speakers when the total number of clusters into which each section is classified exceeds the number of speakers. And cluster merging means for merging a plurality of clusters. According to each aspect described above, each section can be easily classified into a number of clusters corresponding to the number of speakers.

  Furthermore, the speech processing apparatus according to a preferred aspect is configured such that an unclassified section that the similarity determination unit does not classify into any cluster is the most characteristic feature of the speech signal in the unclassified section among the plurality of existing clusters. Unclassified section processing means for classifying into clusters of similar acoustic models is provided. According to this aspect, there is an advantage that the classification omission of each section can be effectively prevented.

The audio processing apparatus according to the present invention is realized by hardware (electronic circuit) such as a DSP (Digital Signal Processor) dedicated to audio processing, and a general-purpose arithmetic processing apparatus such as a CPU (Central Processing Unit). It is also realized through collaboration with the program. The program according to the present invention includes a voice classification process for classifying a voice signal into a plurality of variable length sections on a time axis with a valley portion of a waveform envelope of the voice signal as a boundary, and a target cluster that is a classification destination of each section Cluster specification processing (for example, steps SA1 and SA14 in FIGS. 3 and 6) and cluster information generation processing for generating cluster information including the acoustic model of the target cluster (for example, step SA4 in FIGS. 3 and 6). And a section selection process (for example, step SA5 in FIGS. 3 and 6) for selecting each section of the unclassified speech signal as a selected section in order of long time while the target cluster is designated, and speech within the selected section. Similarity determination processing (for example, step SA6 to step SA10 in FIGS. 3 and 6) for determining the similarity between the feature amount of the signal and the acoustic model of the target cluster and the similarity determination processing are selected. When it is determined that the sections are similar to each other by the section classification process for classifying the sections into the target clusters (for example, step SA11 in FIGS. 3 and 6) and the similarity determination process, Update processing (for example, step SA12 in FIGS. 3 and 6) for updating the acoustic model is executed by the computer. Even with the above program, the same operations and effects as those of the speech processing apparatus according to the present invention are exhibited. The program of the present invention is provided to the user in a form stored in a computer-readable recording medium and installed in the computer, or is provided from the server device in the form of distribution via a communication network. To be installed.

<A: First Embodiment>
FIG. 1 is a block diagram showing the configuration of the speech processing apparatus according to the first embodiment of the present invention. As shown in FIG. 1, the voice processing device 100 is a computer system that includes a control device 10 and a storage device 20. The control device 10 is an arithmetic processing device that executes a program. The storage device 20 stores a program executed by the control device 10 and various data used by the control device 10. A known recording medium such as a semiconductor storage device or a magnetic storage device is arbitrarily adopted as the storage device 20. An input device 25 and an output device 30 are connected to the control device 10. The input device 25 notifies the control device 10 of the content of the operation by the user. The output device 30 is a display device that displays various images under the control of the control device 10.

  The storage device 20 stores an audio signal S representing a waveform of the audio on the time axis. A waveform on the time axis of the audio signal S is illustrated in part (A) of FIG. The voice represented by the voice signal S of this embodiment is voice recorded by using a sound collection device in a conference where a plurality of participants speak at any time.

  The control device 10 in FIG. 1 creates a meeting minutes from the audio signal S by executing a program stored in the storage device 20. The minutes are conference records in which the contents of each participant's remarks are arranged in time series for each participant. As shown in FIG. 1, the control device 10 functions as a voice classification unit 12, a feature extraction unit 13, a classification processing unit 14, and a voice recognition unit 16. Each function of the control device 10 is also realized by an electronic circuit such as a DSP dedicated to voice processing. Further, the control device 10 may be distributed and mounted on a plurality of integrated circuits.

  As shown in part (D) of FIG. 2, the voice classifying unit 12 converts the voice signal S stored in the storage device 20 into a plurality of sound generation periods PA (PA1, PA2,...) And a plurality of voices on the time axis. It is divided into a non-sounding section PB. Each sounding section PA is a variable-length section that is estimated to be continuously uttered by one speaker. The non-sounding section PB is a variable length section in which the volume of the uttered sound is sufficiently small (or there is no uttered sound).

  The voice classification unit 12 executes a first process and a second process. As shown in part (B) of FIG. 2, the first process is a process of detecting a section in which the S / N ratio and the volume (amplitude) of the audio signal S exceed the threshold as the sound generation section PA. A section other than the sounding section PA is a non-sounding section PB.

  When utterances by a plurality of speakers are continuous without being spaced apart or partially overlapped, it is difficult to classify the audio signal S for each speaker by only the first process. Therefore, as shown in part (C) and part (D) of FIG. 2, the audio classification unit 12 uses each of a plurality of valleys D appearing in an envelope E of the waveform of the audio signal S as a boundary. A second process for dividing the sound generation period PA is executed. In a series of utterances by humans, generally, there is a tendency that the volume gradually increases from the start point of the utterance and gradually decreases from an intermediate point to the end point of the utterance. Therefore, according to the configuration in which the sound generation section PA is divided with the valley portion D as a boundary, even if a plurality of utterances are continuous or overlapped, the utterances by each speaker are divided into separate sound generation sections PA. In the following, the total number of sounding sections PA (PA1, PA2,...) After classification by the voice classification unit 12 is J (J is a positive number of 2 or more). As shown in part (D) of FIG. 2, the non-sound generation section PB is not divided in the second process even if there is a valley D inside.

  The feature extraction unit 13 in FIG. 1 extracts the acoustic feature amount of the audio signal S for each of the J sound generation sections PA and stores it in the storage device 20. In this embodiment, the feature quantity extracted from one sounding section PA is a MFCC (Mel Frequency Cepstral Coefficient) vector (hereinafter referred to as “feature vector”) calculated from the audio signal S of each frame that divides the sounding section PA. It is a time series of x.

  The classification processing unit 14 classifies the J pronunciation sections PA defined by the voice classification unit 12 for each speaker (conference participant) based on the feature vector x extracted by the feature extraction unit 13. That is, among the J pronunciation intervals PA, the pronunciation intervals PA that are likely to be uttered by the same speaker are classified into a common set (cluster). The non-sounding section PB is excluded from the classification target.

  The voice recognition unit 16 specifies the content of the utterance for each speaker as a character from the voice signal S of each pronunciation section PA after the classification by the classification processing unit 14. For the process of recognizing characters from the speech signal S in each sound generation section PA, a known speech recognition technique is arbitrarily employed. For example, the speech recognition unit 16 firstly updates (speaker adaptation) the initial acoustic model according to the acoustic feature amount of the speech signal S of each sound generation section PA classified into one cluster. Then, an acoustic model that uniquely reflects the features of the speaker corresponding to the cluster is generated, and secondly, the feature value extracted from the acoustic model after speaker adaptation and the speech signal of each pronunciation section PA in the cluster By comparing, the character spoken by the speaker is specified. The control device 10 outputs the processing result by the voice recognition unit 16 to the output device 30. The output device 30 displays an image of the minutes in which the time of utterance, the identification code of the utterer (for example, the name of the utterer), and the characters identified by the voice recognition unit 16 regarding the content of the utterance are arranged in time series. To do.

  Next, the operation of the classification processing unit 14 will be described in detail with reference to FIG. When the user instructs the creation of the minutes by operating the input device 25, the processing of FIG. 3 is started following the classification of the audio signal S by the audio classification unit 12 and the extraction of the feature vector x by the feature extraction unit 13. .

  When instructing the creation of the minutes, the user inputs the number of speakers V at the time of recording the audio signal S by appropriately operating the input device 25. The classification processing unit 14 acquires the number of speakers V input by the user from the input device 25 (step SA0). Further, the classification processing unit 14 initializes the number n of each cluster (hereinafter referred to as “cluster number”) n that is a candidate for the classification destination of each sound generation section PA to “1” (step SA1). That is, the classification processing unit 14 designates the cluster CL1 whose cluster number n is “1”. Hereinafter, the cluster currently designated by the classification processing unit 14 is particularly referred to as a “target cluster”.

  Next, the classification processing unit 14 selects one pronunciation section PA having the longest time length among the pronunciation sections PA that are not classified into any cluster at the current stage (step SA2). Then, the classification processing unit 14 classifies the sound generation section PA selected in step SA2 into the target cluster CLn (step SA3). That is, the classification processing unit 14 stores the start point and end point times of the sound generation section PA selected in step SA2 in association with the current cluster number n in the storage device 20. The part (E1) in FIG. 2 illustrates a state in which the sound generation section PA4 is classified into the target cluster CL1 in step SA3 immediately after the processing in FIG. 3 is started.

Further, the classification processing unit 14 newly creates cluster information CINF [n] of the target cluster CLn and stores it in the storage device 20 (step SA4). One piece of cluster information CINF [n] includes a classification section length T and an acoustic model X as shown in FIG. The classification section length T is the sum of the time lengths of the sound generation sections PA classified into the target cluster CLn. In step SA4, only the sounding section PA selected in step SA2 is classified as the target cluster CLn. Therefore, the classification processing unit 14 sets the time length of the sounding section PA as the classification section length T. The acoustic model X models the acoustic feature quantity of the sound generation section PA to be classified into the target cluster CLn. Acoustic model X of this embodiment is composed of a mixture model λ and codebook (codebook) C ^A.

The mixed model λ is a function that models the time series of the feature vector x extracted by the feature extraction unit 13 from the speech signal S of the sound generation section PA selected in step SA2 as a weighted sum of M probability distributions. In this embodiment, a Gaussian mixture model expressed by the following equation (1) is adopted as a mixture model λ as a weighted sum of M normal distributions. A known technique such as an EM (Expectation-Maximization) algorithm is arbitrarily employed to generate the mixed model λ.
λ = {pi, μi, Σi} (i = 1 to M) (1)
In the equation (1), pi is a weight value (weight value) of the i-th normal distribution. The sum of the weights p1 to pM is 1. In Expression (1), μi is an average vector of the i-th normal distribution, and Σi is a covariance matrix of the i-th normal distribution. It should be noted that even if a symbol actually means a vector, such as μi in equation (1), this specification means that the symbol means a vector, for example, by clearly expressing it as an `` average vector ''. The vector symbol (the arrow pointing right on the character) is omitted.

Codebook C ^A defines a plurality of centroid vector corresponding to the time series of feature vector x in the pronunciation section PA selected in step SA2 (code vectors). The creation of the code book C ^A, known techniques such as k-means method and LBG algorithm are optionally employed.

  When the new cluster information CINF [n] is created by the above procedure, the classification processing unit 14 selects one pronunciation section having the longest time length from the pronunciation sections PA that are not classified into any cluster at the present stage. PA is selected (step SA5). Next, the classification processing unit 14 determines the similarity between the time series of the feature vector x of the pronunciation section PA (hereinafter, specifically referred to as “selected section PA”) selected in step SA5 and the acoustic model X of the cluster information CINF [n]. Determination is made (from step SA6 to step SA10). Details of the processing from step SA6 to step SA10 will be described later.

  If it is determined that the feature vector x of the selected section PA and the acoustic model X are similar, the classification processing unit 14 classifies the selected section PA into the target cluster CLn (step SA11). That is, the classification processing unit 14 stores the start point and end point times of the selected section PA in association with the current cluster number n in the storage device 20. The part (E2) in FIG. 2 illustrates a state in which the sound generation section PA5 is classified into the target cluster CL1.

Further, the classification processing unit 14 updates the cluster information CINF [n] of the target cluster CLn based on the feature vector x of the selected section PA (Step SA12). That is, the classification processing unit 14 includes the feature vectors x of all the sounding sections PA previously classified into the target cluster CLn and the feature vectors x of the selected section PA newly classified into the target cluster CLn in the immediately preceding step SA11. after having generated a mixture model λ and codebook C ^a stored in the storage device 20 as the acoustic model X after the update based on and. Further, the classification processing unit 14 adds the time length of the current selected section PA to the classification section length T of the cluster information CINF [n]. When the above update is completed, the classification processing unit 14 proceeds to step SA13.

  On the other hand, if it is determined that the feature vector x of the selected section PA and the acoustic model X are not similar, the classification processing unit 14 proceeds to step SA13 without passing through step SA11 and step SA12. That is, the sound generation section PA whose acoustic features are not similar to the acoustic model X of the target cluster CLn is not classified into the target cluster CLn.

  In step SA13, the classification processing unit 14 determines whether or not the processing from step SA5 to step SA12 has been completed for all unclassified pronunciation sections PA. If the result of step SA13 is negative, the classification processing unit 14 selects the longest uncategorized pronunciation section PA as a new selection section PA in step SA5, and then executes the processes after step SA6. The part (E3) of FIG. 2 illustrates a state in which the sounding section PA1 is classified into the target cluster CL1 in the second and subsequent steps SA11. As described above, the classification processing unit 14 selects each uncategorized pronunciation section PA as a selection section PA in order of long time while designating one target cluster CLn, and the selected section PA becomes the target cluster CLn. Each time classification is performed, the cluster information CINF [n] (acoustic model X and classification section length T) of the target cluster CLn is updated based on the feature vector x of the selected section PA.

By the way, when the number of feature vectors x used to generate the mixed model λ and the code book C ^A is small, the code book C ^A faithfully reflects the acoustic features of the speech signal S compared to the mixed model λ. There is a tendency to do. On the other hand, the more the number of feature vector x, the mixed model lambda, faithfully reflect the acoustic characteristics of the audio signal S as compared to the codebook C ^A. Therefore, as the acoustic model X to be compared with the feature vector x in the selected section PA, the code book C ^A is suitable when the number of the feature vectors x used for generating the code book C ^A and the mixed model λ is small. The mixed model λ is suitable when a sufficient number of feature vectors x are secured. Therefore, in the determination of the similarity with the selected section PA in steps SA6 to SA10 of this embodiment, when the classification section length T of the cluster information CINF [n] exceeds the threshold Tth (that is, used for generating the acoustic model X). The mixed model λ is used when the number of feature vectors x is large), and the codebook C ^A is used when the classification section length T is equal to or smaller than the threshold Tth (that is, when the number of feature vectors x is small). . Further details are as follows.

  In step SA6, the classification processing unit 14 determines whether or not the classification section length T of the cluster information CINF [n] exceeds the threshold Tth. If the result of step SA6 is positive, the classification processing unit 14 calculates the average likelihood L based on the mixed model λ of the cluster information CINF [n] and the time series of the feature vector x extracted from the selected section PA. (Step SA7). As described in detail below, the average likelihood L is obtained by averaging the probability (likelihood) that each feature vector x in the selected section PA appears from the mixed model λ with respect to all the feature vectors x in the selected section PA. It is a numerical value.

When one feature vector x is a D-dimensional vector, the likelihood that the feature vector x appears from the mixed model λ is calculated by the following equation (2).

The classification processing unit 14 calculates the average likelihood L by substituting the K feature vectors x (x1 to xK) extracted by the feature extraction unit 13 for the selected section PA into the following equation (3). As understood from the equation (3), the average likelihood L increases as the acoustic feature represented by the mixed model λ is similar to the acoustic feature represented by the feature vector x.

  Next, the classification processing unit 14 determines whether or not the average likelihood L calculated in Step SA7 exceeds the threshold Lth (Step SA8). If the result of step SA8 is affirmative (that is, if the mixed model λ and the feature vector x of the selected section PA are similar), the classification processing unit 14 proceeds to step SA11, and the result of step SA8 is negative. If so, the process proceeds to step SA13.

On the other hand, if the result of step SA6 is negative (that is, if the number of feature vectors x used to create the acoustic model X is still small), the classification processing unit 14 uses the code book C ^{A of the} cluster information CINF [n]. And VQ (Vector-Quantization) distortion D is calculated based on the time series of the feature vectors x (x1 to xK) extracted from the selected section PA (step SA9). The VQ distortion D is calculated by, for example, the following formula (4).

In equation (4), | C ^A | is the size of codebook C ^A , and C ^A (i) is the i-th centroid vector in codebook C ^A. D (X, Y) means the Euclidean distance between the vector X and the vector Y. As described above, the VQ distortion D is the minimum value (min) between | C ^A | centroid vectors in the codebook C ^A and the feature vector x of the selected section PA over K feature vectors x1 to xK. It is an averaged number. Therefore, VQ distortion D as the acoustic characteristics similar representative acoustic feature and the feature vector x representing the codebook C ^A decreases.

The classification processing unit 14 determines whether or not the VQ distortion D calculated in Step SA9 is less than the threshold value Dth (Step SA10). Classification processing unit 14, the process proceeds when the result of step SA10 is affirmative (i.e. if the codebook C ^A feature vector x of the selected section PA is similar) to step SA11, the result of step SA10 If negative, the process proceeds to step SA13.

  If it is determined in step SA13 after the similarity determination described above that the processing of all uncategorized pronunciation sections PA has been completed, the classification processing unit 14 adds “1” to the cluster number n (step SA14). ). That is, a new target cluster CLn is sequentially designated every time the processing from step SA5 to step SA12 is completed for all uncategorized pronunciation intervals PA.

  Next, the classification processing unit 14 determines whether or not the cluster number n after addition in step SA14 exceeds the number of speakers V acquired in step SA0 (step SA15). If the result of step SA15 is negative, the classification processing unit 14 classifies each sound generation interval PA into a new target cluster CLn by executing the processing after step SA2 for the cluster number n after the update in step SA14.

  For example, in step SA3 immediately after a new target cluster CL2 is selected in step SA14, the unclassified and longest pronunciation section PA2 is classified as the target cluster CL2 as shown in part (E4) of FIG. Then, in the next step SA4, cluster information CINF [2] is generated in the storage device 20. That is, each time a new target cluster CLn is designated, new cluster information CINF [n] is sequentially generated as shown in FIG. Further, the part (E5) of FIG. 2 illustrates a state in which the sounding section PA3 is classified into the target cluster CL2 in step SA11 while the target cluster CL2 is being specified.

  On the other hand, when the result of step SA15 is affirmative (that is, when the pronunciation interval PA is classified into a cluster having the number of speakers V), the classification processing unit 14 proceeds to step SA16. In step SA16, the classification processing unit 14 selects a pronunciation section PA that is not yet classified into any cluster (hereinafter referred to as “unclassified section PA” in particular) from among the existing V clusters CL1 to CLV. Unclassified section processing is performed for classifying the cluster CL of the mixed model λ with the most similar time series of the feature vectors x of the classified section PA. Hereinafter, specific contents of the unclassified section process will be described with reference to FIG.

  When the unclassified section process is started, the classification processing unit 14 determines whether or not an unclassified section PA exists (step SB1). If the result of step SB1 is affirmative, the classification processing unit 14 determines the mixed model for each of all (V) pieces of cluster information CINF (CINF [1] to CINF [V]) stored in the storage device 20. The average likelihood L between λ and the time series of the feature vector x of one unclassified section PA is calculated based on the equation (3) (step SB2). Next, the classification processing unit 14 selects the maximum value Lmax from the V average likelihoods L calculated in step SB2 (step SB3), and determines whether or not the maximum value Lmax exceeds the threshold value TH ( Step SB4). The threshold value TH is set to a numerical value smaller than the threshold value Lth in step SA8.

  If the result of step SB4 is affirmative (similar), the classification processing unit 14 classifies the unclassified section PA into the cluster CL whose average likelihood L is the maximum value Lmax (step SB5). On the other hand, if the result of step SB4 is negative, the classification processing unit 14 does not execute step SB5. That is, the unclassified section PA is not classified into any of the clusters CL1 to CLV, and is processed in the same manner as the non-sound generation section PB. The processing from step SB2 to step SB5 is repeated for all unclassified sections PA. The classification processing unit 14 ends the unclassified section processing when the processing is completed for all unclassified sections PA (step SB1: NO).

  As described above, in this embodiment, since the speech signal S is classified into variable-length pronunciation intervals PA for each utterer's utterance, the speech signal S is classified into a fixed length. Compared with the configuration of 1, it is possible to classify each sound generation section PA for each speaker with high accuracy. Therefore, an accurate minutes can be created from the audio signal S.

  Further, the uncategorized sound generation intervals PA are sequentially selected as the selection interval PA in the order of long time, and the selection interval PA classified into the target cluster CLn is used for updating the cluster information CINF [n]. As the time length of the sound generation section PA is longer, a larger number of feature vectors x are extracted from the sound generation section PA. For example, the uncategorized sound generation section PA is selected as the selection section PA in the order of short time or chronological order. The acoustic model X of the cluster information CINF [n] is rapidly compared with the reliable acoustic model reflecting a large number of feature vectors x (that is, the characteristics of the actual uttered sound of each speaker are faithfully reflected). (Acoustic model). Further, since the uncategorized and longest sounding section PA is used for the new creation of the cluster information CINF [n] (acoustic model X) in step SA4, for example, the sounding section PA having the shortest time length or the most temporally longest sounding section PA is used. Compared with the configuration in which the previous sound generation section PA is used to create the cluster information CINF [n], the acoustic model X newly created in step SA4 has high reliability. As described above, since the reliability of the acoustic model X is ensured at an early stage of classification, according to the present embodiment, there is an advantage that each sound generation section PA can be classified with high accuracy.

Further, optionally used in accordance with the feature vector x and the acoustic model determines similarity between the X classification section length and mixed model λ and codebook C ^A is in (steps SA6 step SA10) T selection section PA Therefore, the advantage of the mixed model λ that the reliability is particularly high when a large number of feature vectors x are used for generation, and the reliability is ensured even if the number of feature vectors x used for generation is small. and advantages of the codebook C ^a both can enjoy. Therefore, in comparison with the configuration in which only one type of acoustic model X (one of the mixed model λ and codebook C ^A) is used, it is possible to increase the accuracy of the classification of each sound segment PA.

  In this embodiment, since the classification is completed when the number of classifications in the pronunciation section PA reaches the number of speakers V, it is easy to classify the pronunciation section PA into the total number of actual speakers. Furthermore, since the unclassified section PA at the end of classification is classified into the cluster CL having the most similar acoustic characteristics, for example, compared with a configuration in which the unclassified section PA is discarded (a configuration in which step SA16 is omitted). The omission of the classification of the sound generation interval PA is effectively suppressed. In the meeting minutes, lack of statements (missing description) can be a particularly serious problem, so the above effect of unclassified section processing is particularly effective.

<B: Second Embodiment>
Next, a second embodiment of the present invention will be described. The first embodiment exemplifies a configuration in which classification is terminated when the number of classifications (number of clusters) in the pronunciation period PA reaches the number of speakers V. On the other hand, in this embodiment, the classification of the sounding section PA is continued until all of the J sounding sections PA are classified into any cluster CL. In addition, about the element in which an effect | action and a function are equivalent to 1st Embodiment in each following form, the same code | symbol as the above is attached | subjected and each detailed description is abbreviate | omitted suitably.

  FIG. 6 is a flowchart showing the operation of the classification processing unit 14. As shown in the figure, the processing from step SA0 to step SA14 is the same as in the first embodiment. Subsequent to step SA14, the classification processing unit 14 determines whether or not all (J) pronunciation sections PA have been classified (step SA17). If the result of step SA17 is negative, the classification processing unit 14 executes the processing after step SA2 for the target cluster CLn newly specified in the immediately preceding step SA14. That is, the processing from step SA2 to step SA14 is repeated until all the sound generation sections PA are classified into any cluster CL. Therefore, the final classification number (cluster number) N may exceed the number of speakers V.

  If the result of step SA17 is affirmative, the classification processing unit 14 determines whether or not the classification number N at the current stage exceeds the number of speakers V specified in step SA0 (step SA18). When the result of step SA18 is negative (that is, when the pronunciation interval PA is classified into a cluster with the number of speakers V), the classification processing unit 14 ends the processing of FIG. On the other hand, if the result of step SA18 is affirmative, the classification processing unit 14 sequentially merges a plurality of clusters until the classification number N becomes equal to or less than the number of speakers V (steps SA19 and SA20).

In step SA19, the classification processing unit 14 sets numerical values (hereinafter referred to as “similarity” between the acoustic models X for all combinations ( _N C ₂ ) of selecting two clusters out of _N. Cluster similarity index value ”). For example, the cluster similarity index value is calculated by comparing the mixed model λ of the cluster information CINF for two clusters. That is, for each of the M average vectors μa in the mixed model λ of one cluster CLa, the classification processing unit 14 distances the average vector μa among the M average vectors μb in the mixed model λ of the other cluster CLb. M combinations of the average vectors μa and μb are determined by selecting the average vector μb having the smallest (for example, Euclidean distance) without overlapping. Then, the classification processing unit 14 calculates a numerical value obtained by adding the distances between the average vectors μa and μb belonging to each combination for M combinations as a cluster similarity index value. Therefore, the cluster similarity index value decreases as the mixed models λ are similar between the clusters CLa and CLb.

In step SA20, the classification processing unit 14 subtracts “1” from the classification number N after merging two clusters belonging to the combination having the smallest cluster similarity index value among the _N C _two combinations. A known technique is arbitrarily employed for merging the clusters. The result of step SA18 changes to negative by repeating the merging of clusters described above. In the step SA19, it may calculate a cluster such absence index value by utilizing instead of the average vector μ mixing model λ a centroid vector in the codebook C ^A of each cluster. In this embodiment, the same effect as that of the first embodiment is obtained.

<C: Third Embodiment>
Next, a third embodiment of the present invention will be described. In each of the above embodiments, the threshold value Lth used in step SA8 and the threshold value Dth used in step SA10 are fixed values. On the other hand, in this embodiment, the classification processing unit 14 variably controls the threshold value Lth and the threshold value Dth.

  FIG. 7 is a flowchart partially showing the operation of the classification processing unit 14. As shown in the figure, in this embodiment, step SC1 is executed when the result of step SA6 is affirmative, and step SC2 is executed when the result of step SA6 is negative. Processes other than steps SC1 and SC2 are the same as those in the first and second embodiments.

  In step SC1, the classification processing unit 14 variably controls the threshold Lth according to the classification section length T in the cluster information CINF [n] of the target cluster CLn. Since the reliability of the mixed model λ increases as the classification interval length T increases, the difference in the average likelihood L increases when the feature vector x of the selected interval PA and the mixed model λ are similar or dissimilar. Therefore, the classification processing unit 14 sets the threshold value Lth to a smaller numerical value as the classification section length T is longer. According to the above configuration, there is an increased possibility that it can be determined to be similar in step SA8 even if the characteristics (for example, pitch or volume) of a uttered sound by a specific speaker change. Therefore, it is possible to prevent the omission of the classification of the sounding section PA (a state where the sounding section PA that should be classified into the target cluster CLn is not classified).

  On the other hand, in step SC2, the classification processing unit 14 variably controls the threshold value Dth according to the classification section length T in the cluster information CINF [n]. For example, the classification processing unit 14 sets the threshold value Dth to a larger numerical value as the classification section length T is longer. According to the above configuration, the possibility of omission of the classification of the sounding section PA is reduced as in step SC1.

  In addition, although the case where priority was given to prevention of omission of classification in the above was illustrated, in the case where priority should be given to strictness of similarity determination, the threshold Lth and Dth are the above examples for the classification interval length T. It may be changed in the reverse direction. That is, according to the configuration in which the classification processing unit 14 increases the threshold value Lth and decreases the threshold value Dth as the classification section length T is longer, the acoustic model X corresponding to a specific speaker is similar to the feature vector x of another person. It is possible to reduce the possibility of erroneous determination. Further, the timing for controlling the thresholds Lth and Dth is not limited to the above examples. For example, a configuration in which both the thresholds Lth and Dth are set according to the classification section length T prior to the determination in step SA6 can be employed.

<D: Modification>
Various modifications can be made to each of the above embodiments. An example of a specific modification is as follows. Two or more aspects may be arbitrarily selected from the following examples and combined.

(1) Modification 1
In each of the above forms, the uncategorized and longest pronunciation section PA is used for creating the cluster information CINF [n] in step SA4. However, the condition of the pronunciation section PA used for newly creating the cluster information CINF [n] is used. Are appropriately changed. For example, the cluster information CINF [n] may be created from the feature vector x of the pronunciation segment PA after selecting the uncategorized pronunciation segment PA that is earliest (oldest) on the time axis in step SA2. . However, according to the configuration in which the longest sound generation section PA is used in step SA4 as in the above embodiments, there is an advantage that the reliability of the acoustic model X can be improved at an early stage as described above.

(2) Modification 2
In each embodiment described above is a mixed model λ and codebook C ^A was adopted as the acoustic model X, aspects of the acoustic model X may be appropriately changed. In a configuration in which any one of a plurality of acoustic models is selected according to the classification section length T (total number of feature vectors x), an acoustic model that provides sufficient reliability when the classification section length T is long, and the classification section length A configuration in which an acoustic model capable of obtaining reliability even when T is short is selectively used. However, a configuration that selectively uses a plurality of acoustic models is not essential in the present invention. That is, only the mixed model λ and configured to be used as an acoustic model X (for example, step SA6, the configuration is omitted SA9 and SA10), configured to use only the codebook C ^A as the acoustic model X (for example, step SA6, SA7 and SA8 Is also adopted.

(3) Modification 3
In the first embodiment, the similarity between the feature vector x of the unclassified section PA and the mixed model λ is determined in the unclassified section processing (FIG. 5). The feature vector x of the unclassified section PA, the code book CA, A configuration for determining the similarity is also adopted. That is, the classification processing unit 14, selects the minimum value Dmin of the VQ distortion D with calculating the VQ distortion D of the code book C ^A unclassified section PA and all of the cluster CL (step SB2) (Step SB3) If the minimum value Dmin is lower than the threshold value TH (step SB4: YES), the unclassified section PA is classified into the cluster for which the minimum value Dmin is calculated (step SB5).

(4) Modification 4
In the third embodiment, the threshold values Lth and Dth are controlled according to the classification section length T. However, the numerical value serving as a reference for determining the threshold values Lth and Dth is not limited to the classification section length T. For example, a configuration in which the threshold values Lth and Dth are controlled based on the SN ratio of the audio signal S is also employed. That is, the classification processing unit 14 calculates the SN ratio from the audio signal S in the selected section PA selected in step SA5, and sets the threshold value Lth to a smaller value and the threshold value Dth to a larger value as the SN ratio is lower. . Since the feature vector x of the audio signal S is affected by noise, if the threshold values Lth and Dth are fixed, the lower the S / N ratio of the audio signal S, the lower the SN ratio of the selection section PA that is actually similar to the acoustic model X. There is a high possibility that the feature vector x is erroneously determined to be dissimilar in steps SA8 and SA10. According to the configuration in which the thresholds Lth and Dth are variably controlled according to the SN ratio, there is an advantage that the possibility of erroneous determination due to the SN ratio is reduced. Note that the SN ratio of the selected section PA (sound generation section PA) is calculated as a relative ratio between the average intensity of the audio signal S in the selected section PA and the average intensity of the non-sound generation section PB immediately before the selected section PA.

(5) Modification 5
In addition to the above examples, a known technique is arbitrarily adopted for the classification of the audio signal S. For example, a configuration in which the audio signal S is divided into the sound generation section PA and the non-sound generation section PB according to only the SN ratio, the sound volume, and the threshold value (configuration in which the sound classification unit 12 executes only the first process) is also employed. The Further, it is not always necessary to distinguish between the sounding section PA and the non-sounding section PB. For example, a configuration in which the audio signal S is divided into a plurality of sections with only the valley portion D of the envelope E as a boundary (a configuration in which the audio classification unit 12 executes only the second process) is also employed.

(6) Modification 6
The classification section length T need only be a numerical value that serves as an index of reliability of the acoustic model X, and is not limited to the sum of the time lengths of the sound generation sections PA classified into the target cluster CLn. For example, since the reliability of the acoustic model X increases as the number of feature vectors x used to generate the acoustic model X increases (the classification section length T increases), the sound generation section PA classified into the target cluster CLn is increased. The total number of feature vectors x and the total number of frames in the pronunciation section PA may be used as the classification section length T.

(7) Modification 7
A printing device that prints the minutes created by the voice processing device 100 may be adopted as the output device 30. However, it is not necessary to output the processing result by the speech processing apparatus 100 in the form of minutes (characters), and for example, the classification result by the classification processing unit 14 can be output. For example, according to the configuration in which the sound signal S in the sound generation section PA including the time designated by the user among the plurality of sound generation sections PA is output as a sound wave from the sound emitting device (for example, a speaker), the user It is possible to effectively support the task of creating the minutes of the meeting while selectively listening to the comments and confirming them appropriately. In addition, a configuration in which the audio signal S is stored in the storage device 20 in a state of being divided into a plurality of sections, and a feature vector x (feature amount) extracted from each section of the audio signal S is stored in the storage device 20 in advance. The structure made is also adopted. As described above, the speech classification unit 12, the feature extraction unit 13, and the speech recognition unit 16 are not essential elements for the speech processing apparatus 100.

(8) Modification 8
In each of the above embodiments, the audio signal S stored in advance in the storage device 20 is the target of processing, but is sequentially supplied via the audio signal S supplied from the sound collection device (microphone) and the communication network. The processing may be executed in real time for the audio signal S.

(9) Modification 9
The type of sound represented by the audio signal S is not limited to a human voice. For example, if an audio signal S obtained by collecting performance sounds when a plurality of types of musical instruments are sequentially played is set as an object of processing by the audio processing device 100, a plurality of performance sound intervals for each instrument are classified for each instrument type. It becomes possible to classify into clusters.

It is a block diagram which shows the structure of the audio processing apparatus which concerns on 1st Embodiment of this invention. It is a conceptual diagram for demonstrating the specific example of operation | movement of an audio | voice classification part and a classification | category process part. It is a flowchart which shows operation | movement of a classification | category process part. It is a conceptual diagram for demonstrating the content of cluster information. It is a flowchart which shows the content of an unclassified area process. It is a flowchart which shows operation | movement of the classification | category process part in 2nd Embodiment of this invention. It is a flowchart which shows operation | movement of the classification | category process part in 3rd Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 ... Voice processing apparatus, 10 ... Control apparatus, 12 ... Voice classification part, 13 ... Feature extraction part, 14 ... Classification processing part, 16 ... Voice recognition part, 20 ... Memory | storage device, 25 ... Input device, 30... Output device, S... Voice signal, PA (PA1, PA2,...) ... Sound generation section, CINF [n] ... Cluster information, CL (CL1, CL2,...) ... Cluster, X ... Acoustic model, λ ... Mixed model, C ^A ... Codebook, T ... Classification interval length.

Claims (8)

Voice classification means for dividing the voice signal into a plurality of variable length sections on the time axis with a valley of the envelope of the waveform of the voice signal as a boundary;
Cluster designation means for sequentially designating target clusters to be classified into the respective sections;
Cluster information generating means for generating cluster information including an acoustic model of the target cluster;
Section selection means for selecting each of the unclassified sections of the audio signal as a selection section in order of long time during the designation of the target cluster;
Similarity determination means for determining similarity between the feature amount of the audio signal in the selected section and the acoustic model of the target cluster;
Section classification means for classifying the selected section into the target cluster when the similarity determination section determines that they are similar;
An audio processing apparatus comprising: an updating unit configured to update an acoustic model of the target cluster based on a feature amount of the audio signal in the selected section when the similarity determination unit determines that the similarity is similar. The audio processing apparatus according to claim 1, wherein the cluster information generation unit generates the acoustic model based on a feature amount of an audio signal in an unclassified longest section. The cluster information of the target cluster is
The acoustic model comprising a first model and a second model, each generated in a separate manner;
Including a classification section length according to the time length of the section classified into the target cluster,
The similarity determination means includes
Determine whether the classification section length in the cluster information exceeds a threshold,
When the determination result is affirmative, the similarity between the feature amount of the audio signal in the selected section and the first model is determined, and when the determination result is negative, the feature in the selected section The speech processing device according to claim 1, wherein the similarity between the feature amount of the speech signal and the second model is determined. The cluster information of the target cluster includes a classification section length according to the time length of the section classified into the target cluster,
The similarity determination means determines the similarity of both by comparing the similarity index value of the feature quantity of the audio signal in the selected section and the acoustic model of the target cluster with an similarity determination threshold,
The voice processing device according to any one of claims 1 to 3, further comprising a threshold setting unit that variably sets the similarity determination threshold according to the classification section length in the cluster information. A number-of-speakers identifying means for identifying the number of speakers,
The speech processing apparatus according to any one of claims 1 to 4, wherein the classification is terminated when the sections are classified into a number of clusters corresponding to the number of speakers. A speaker number identifying means for identifying the number of speakers,
5. A cluster merging means for merging a plurality of clusters until the total number of clusters becomes equal to or less than the number of speakers when the total number of clusters into which the sections are classified exceeds the number of speakers. Any of the voice processing devices. The unclassified section that has not been classified into any cluster by the similarity determination unit is not classified into an acoustic model cluster that is most similar to the feature amount of the audio signal in the unclassified section among the plurality of existing clusters. The speech processing apparatus according to claim 1, further comprising a classification section processing unit. On the computer,
A voice classification process for dividing the voice signal into a plurality of variable length sections on the time axis with a valley of the envelope of the waveform of the voice signal as a boundary;
A cluster designation process for sequentially designating a target cluster as a classification destination of each section;
A cluster information generating process for generating cluster information including an acoustic model of the target cluster;
A section selection process for sequentially selecting the sections of the audio signal that are not classified during the designation of the target cluster as a selection section in order of long time,
Similarity determination processing for determining similarity between the feature amount of the audio signal in the selected section and the acoustic model of the target cluster;
A section classification process for classifying the selected section into the target cluster when it is determined to be similar in the similarity determination process;
A program for executing an update process for updating an acoustic model of the target cluster based on a feature amount of the audio signal in the selected section when it is determined that the similarity is similar in the similarity determination process.

Images