JP5672155B2 - Speaker discrimination apparatus, speaker discrimination program, and speaker discrimination method - Google Patents

Description

  The present invention relates to a speaker discrimination device, a speaker discrimination program, and a speaker discrimination method.

  There is known a technique for discriminating who is speaking from each speaker in each scene of a conversation made by a plurality of speakers.

  An example of a technique for realizing such speaker determination by threshold determination is a voice recognition device. A microphone is connected to this speech recognition apparatus corresponding to each participant. Under such a configuration, the speech recognition apparatus records the speech signal in the section from when the power of the speech signal output by the microphone exceeds the power threshold to below it as a speech recognition target in a predetermined area of the storage unit. . In addition, the voice recognition device recognizes the voice signal recorded in the storage unit, and then associates the microphone identification information as data for specifying the speaker, and displays the result of the voice recognition in the minutes area of the storage unit. To record.

  An example of a technique for realizing speaker discrimination by sound source localization is an utterance event separation system. In this utterance event separation system, a microphone array having a plurality of microphones radially directed in different directions is used. The utterance event separation system uses a sound source localization algorithm to analyze multi-channel audio data recorded by a microphone array and estimate the direction of arrival of sound at each time. Also, the utterance event separation system estimates the existence range of speakers as sound sources. Then, the utterance event separation system identifies which speaker is speaking at each time from the result of sound source localization and the estimation result of the presence range of the speaker.

JP 2008-309856 A JP 2007-233239 A

  However, the above-described prior art has a problem that it is not possible to easily and accurately determine the speaker as described below.

  For example, the speech recognition apparatus described above determines whether or not the speaker is speaking depending on whether or not the power of the speech signal exceeds the power threshold. For this reason, in the above speech recognition apparatus, the accuracy of determining the speaker depends on the power threshold, but since there are individual differences in the speech uttered by humans, it is difficult to set an appropriate value for the power threshold. is there. Therefore, the above speech recognition apparatus cannot accurately determine the speaker.

  In the above utterance event separation system, it is necessary to use a complicated algorithm to estimate the direction of arrival of sound by sound source localization. Furthermore, in the above utterance event separation system, it is necessary to learn in advance the number of people participating in the conference in order to estimate the range of speakers. Therefore, in the above utterance event separation system, it is not possible to easily determine the speaker.

  The disclosed technique has been made in view of the above, and an object thereof is to provide a speaker discrimination device, a speaker discrimination program, and a speaker discrimination method capable of easily and accurately discriminating a speaker. To do.

  The speaker discrimination device disclosed in the present application includes an acquisition unit that acquires each voice data from a plurality of microphones arranged for each speaker. Further, the speaker discrimination device has a framing unit that frames the voice data acquired by the acquisition unit into frames of a predetermined section. Furthermore, the speaker discrimination device has a first identification unit that identifies whether the frame framed by the framing unit is a voiced sound region or an unvoiced sound region based on a first probability model. . Further, the speaker discrimination device has the largest energy among the frames identified as the voiced sound area in the same section when the voiced sound area is identified in duplicate in the frame of the same section in each audio data. An enabling unit that validates the frame identification result; Further, the speaker discriminating device uses a second probability model to identify a speech area and a silence area in each speech data from a frame identification result after being validated by the validation unit. It has an identification part.

  According to one aspect of the speaker discrimination device disclosed in the present application, it is possible to easily and accurately perform speaker discrimination.

FIG. 1 is a block diagram illustrating a functional configuration of the conversation analysis apparatus according to the first embodiment. FIG. 2 is a diagram illustrating an example of voiced and unvoiced sounds. FIG. 3 is a diagram illustrating an example of a speech area and a silence area. FIG. 4 is a diagram for explaining a speaker discrimination method. FIG. 5 is a diagram illustrating an example of a state transition diagram in the hidden Markov model. FIG. 6 is a diagram illustrating an example of identification results of voiced sound areas and unvoiced sound areas. FIG. 7 is a diagram illustrating an example of a replacement result of the identification result illustrated in FIG. FIG. 8 is a diagram illustrating an example of a state transition diagram in the hidden Markov model. FIG. 9 is a flowchart illustrating the procedure of the conversation analysis process according to the first embodiment. FIG. 10 is a flowchart illustrating the procedure of the conversation analysis process according to the first embodiment. FIG. 11 is a flowchart illustrating the procedure of the activation process according to the first embodiment. FIG. 12 is a schematic diagram illustrating an example of a computer that executes a speaker discrimination program according to the first and second embodiments.

  Embodiments of a speaker discrimination device, a speaker discrimination program, and a speaker discrimination method disclosed in the present application will be described below in detail with reference to the drawings. Note that this embodiment does not limit the disclosed technology. Each embodiment can be appropriately combined within a range in which processing contents are not contradictory.

  First, the functional configuration of the conversation analysis device including the speaker discrimination device according to the present embodiment will be described. FIG. 1 is a block diagram illustrating a functional configuration of the conversation analysis apparatus according to the first embodiment. The conversation analysis apparatus 10 shown in FIG. 1 uses a plurality of voice data collected through close-talking microphones 30A to 30C provided corresponding to the speaker A, the speaker B, and the speaker C, respectively. ~ Analysis of conversation style by extracting characteristics related to the conversation of speaker C.

  Three conversation microphones 30A to 30C are connected to the conversation analysis apparatus 10. The close-talking microphones 30A to 30C are close-talking microphones worn by speakers. Examples of such close-talking microphones include a lapel microphone and a headset microphone. Hereinafter, the close microphones 30 </ b> A to 30 </ b> C may be collectively referred to as “close microphone 30” when collectively referred to without distinction.

  In the example of FIG. 1, the case of using a close-talking microphone is illustrated, but it is not always necessary to use a close-talking microphone, as long as each speaker is arranged closer to the other speakers. Any microphone can be used. Further, in the example of FIG. 1, a case where three conversations of speakers A to C are collected using three microphones is illustrated, but two conversations are collected using two microphones. It is also possible to collect four or more conversations using four or more microphones.

  The registration unit 31 is a processing unit that registers the audio signal collected by the close-talking microphone 30 in the storage unit 11 of the conversation analysis device 10. As an aspect, the registration unit 31 performs A / D (Analog / Digital) conversion on an analog signal input by voice from the close-talking microphone 30, converts the analog signal into a digital signal, and registers the digital signal in the voice storage unit 11. . Hereinafter, a digital signal obtained by A / D-converting an analog signal input from the close-talking microphone 30A may be referred to as “first audio data”. In addition, a digital signal obtained by A / D-converting an analog signal input from the close-talking microphone 30B may be referred to as “second audio data”. Furthermore, a digital signal obtained by performing A / D conversion on an analog signal input from the close-talking microphone 30C may be referred to as “third audio data”.

  As shown in FIG. 1, the conversation analysis device 10 includes a voice storage unit 11, an extraction unit 13, and an analysis unit 14. The conversation analysis apparatus 10 controls communication with other apparatuses, including various functional units included in known computers other than the functional units illustrated in FIG. 1, such as various input devices and voice output devices. It shall have functional parts, such as a communication interface.

  The voice storage unit 11 is a storage unit that stores voice data. The voice storage unit 11 stores first voice data 12A, second voice data 12B, and third voice data 12C.

  The first audio data 12A, the second audio data 12B, and the third audio data 12C are obtained by A / D-converting the audio signals collected by the close-talking microphone 30 worn by the speakers A to C. Digital data. Of these, the first audio data 12A may include not only the voice of the speaker A but also the voices of the speaker B and the speaker C, but the distance from the speaker A to the close-up microphone 30A is the speaker B. And speaker C are closer. Therefore, the voice included in the first voice data 12A has the highest energy of the voice uttered by the speaker A even when the speaker A, the speaker B, and the speaker C are simultaneously speaking. Get higher. Similarly, the voice included in the second voice data 12B has the highest energy of the voice uttered by the speaker B, and the voice included in the third voice data 12C is the voice uttered by the speaker C. The energy becomes the highest.

  Note that a semiconductor memory element or a storage device can be adopted as the storage unit such as the voice storage unit 11. For example, examples of the semiconductor memory element include a video random access memory (VRAM), a random access memory (RAM), a read only memory (ROM), and a flash memory. Examples of the storage device include storage devices such as a hard disk and an optical disk.

  Here, voiced and unvoiced sounds uttered by the speaker will be described. FIG. 2 is a diagram illustrating an example of voiced and unvoiced sounds. In the example of FIG. 2, voice data acquired using a close-talking microphone with a sampling frequency of 16 kHz is shown. In the example of FIG. 2, the horizontal axis indicates time, the vertical axis indicates frequency, and the shading in the figure indicates the magnitude of spectral entropy.

  As shown in FIG. 2, the voiced sound V (Voiced) is a sound having a large spectrum entropy change and a frequency lower than that of the unvoiced sound U (Unvoiced). Examples of voiced sounds include vowels “a”, “i”, “u”, “e”, “o”, and the like. The unvoiced sound U is a sound having a higher frequency than the voiced sound V. Examples of unvoiced sounds include sounds other than vowels, such as “s”, “p”, “h”, and the like. The characteristics of these voiced sounds and unvoiced sounds do not depend on the language spoken by the speaker, and are common to arbitrary languages such as Japanese, English, and Chinese.

  Next, the relationship between voiced and unvoiced sounds and utterance areas and silence areas will be described. FIG. 3 is a diagram illustrating an example of a speech area and a silence area. The utterance area refers to an area where an utterance is made by a speaker, and includes an unvoiced sound area and a voiced sound area. Note that the example of FIG. 3 shows a case where the speaker utters “WaTaShiWa Chou DeSu”.

  In the example of the utterance shown in FIG. 3, there are a silence area 43 and a silence area 44 between the utterance area 40 of “WaTaShiwa”, the utterance area 41 of “Chou”, and the utterance area 42 of “DeSu”. Indicates. Among these, in the utterance area 40, unvoiced sound “W”, voiced sound “a”, unvoiced sound “T”, voiced sound “a”, unvoiced sound “Sh”, voiced sound “i”, unvoiced sound “W”, voiced sound “ a ”is included. Further, the utterance area 41 includes an unvoiced sound “Ch” and a voiced sound “ou”. Furthermore, the speech area 42 includes unvoiced sound “D”, voiced sound “e”, unvoiced sound “S”, and voiced sound “u”.

  Returning to the description of FIG. 1, the conversation analyzing apparatus 10 includes a speaker discriminating apparatus 50 that discriminates who is speaking out of each speaker in each scene of a conversation made by a plurality of speakers.

  Here, the speaker discrimination device 50 according to the present embodiment acquires the first voice data, the second voice data, and the third voice data from the close-talking microphones 30A to 30C. Furthermore, the speaker discrimination device 50 according to the present embodiment frames the first voice data, the second voice data, and the third voice data into frames of a predetermined section. Furthermore, the speaker discrimination device 50 according to the present embodiment identifies whether the frame is a voiced sound area or an unvoiced sound area based on the first probability model. Furthermore, the speaker discrimination device 50 according to the present embodiment, when the voiced sound area is identified redundantly in the frames of the same section in each audio data, among the frames identified as the voiced sound area in the same section Enable identification results for frames with maximum energy. Then, the speaker discriminating apparatus 50 according to the present embodiment identifies the speech area and the silence area in each voice data from the identification result of the validated frame based on the second probability model.

  The above speaker discrimination method will be described with reference to FIG. FIG. 4 is a diagram for explaining a speaker discrimination method. The upper part of FIG. 4 shows the identification result of the voiced sound area or the unvoiced sound area of each frame. The middle part of FIG. 4 shows the identification result of each frame after the identification result of the voiced sound area of the frame having the maximum energy is validated. The lower part of FIG. 4 shows the results of identifying speech areas and silence areas in each audio data.

  As shown in the upper part of FIG. 4, the speaker discriminating device 50 has each frame formed from the first voice data, the second voice data, and the third voice data based on the first probability model. It is identified whether it is a voice sound area or an unvoiced sound area. Here, in the example of FIG. 4, the symbol “●”, the symbol “◯”, and the symbol “△” each represent a frame of audio data, and the symbol “●” and the symbol “◯” indicate that they are voiced sound regions. , Symbol “Δ” indicates an unvoiced sound region. The frame indicated by the symbol “●” shown in FIG. 4 has higher energy than the frame indicated by the symbol “◯” shown in FIG. From the identification results of the first voice data, the second voice data, and the third voice data, the speaker B and the speaker C are talking among the speakers A to C, and the speaker B It can be estimated that the speaker speaks louder than the speaker C. In the following, each frame obtained from the first audio data may be described as first frame (1)... First frame (n) in the order of observation. Also, each frame obtained from the second audio data may be described as second frame (1)... Second frame (m) in the order of observation. Furthermore, the frames obtained from the third audio data may be described as the third frame (1)... Third frame (m) in the order of observation.

  In addition, as shown in the middle part of FIG. 4, the speaker discrimination device 50 validates the identification result of the frame having the maximum energy when the voiced sound regions overlap in the frames of the same section of each voice data. . In this example, among the frames constituting the second audio data and the third audio data, as described below, the identification results are identified as voiced sound regions in the frames of the same section as described below. That is, the second frame (1) and the third frame (1), the second frame (6) and the third frame (6), and the second frame (10) and the third frame (10) have mutual identification results. It is identified as a voice sound area. Further, in the second frame (13) and the third frame (13), and in the second frame (18) and the third frame (18), the mutual identification results are identified as voiced sound regions. In this case, since the energy of the second audio data is higher in any frame, the third frame (1), the third frame (6), the third frame (10), and the third frame (13). And the identification result of the third frame (18) is replaced from voiced sound to unvoiced sound.

  Furthermore, as shown in the lower part of FIG. 4, the speaker discriminating device 50 identifies the speech area and the silence area in each voice data from the identification result of the validated frame based on the second probability model. In this example, the underlined region of the second audio data frame is identified as the speaker B's speech region. Furthermore, the underlined region of the third audio data frame is identified as the utterance region of the speaker C. In this case, a frame in which the utterance area of the speaker B and the utterance area of the speaker C overlap, that is, a section from the second frame (7) to the second frame (13) is determined as the simultaneous utterance.

  As described above, the speaker discrimination device 50 according to the present embodiment effectively uses only the identification result of the frame having the maximum energy when the voiced sound area is identified in duplicate in the frame of the same section in each audio data. To identify the speech area and silence area of each voice data. For this reason, the speaker discriminating apparatus 50 according to the present embodiment can discriminate a speaker without using a threshold between frames in the same section constituting each audio data. Further, in the speaker discrimination device 50 according to the present embodiment, it is not necessary to use a complicated algorithm for speaker discrimination, and it is not necessary to perform learning in advance. Therefore, according to the speaker discriminating apparatus 50 according to the present embodiment, the speaker can be discriminated easily and accurately.

  Further, the speaker discrimination device 50 according to the present embodiment is identified as the voiced sound area regardless of the magnitude of energy when the voiced sound area is identified independently in the frames of the same section in each audio data. Maintain identification results. In general, utterances are composed of a mixture of voiced and unvoiced sounds, so even if multiple speakers speak at the same time, it is unlikely that voiced sound areas will overlap completely. There is a high probability that an opportunity to be identified alone will remain. For example, in the example in the lower part of FIG. 4, even if the volume of the utterance of the speaker C is lower than the volume of the utterance of the speaker B, the third frame (7), the third frame (9), and the third frame The identification result of (12) is maintained as a voiced sound. Therefore, the speaker discrimination device 50 according to the present embodiment can discriminate simultaneous utterances even when there is a gap in the volume of the utterance by the speaker.

  Further, the speaker discrimination device 50 will be described in detail. As shown in FIG. 1, the speaker discrimination device 50 includes an acquisition unit 51, a framing unit 52, a first identification unit 53, an validation unit 54, and a second identification unit 55.

  The acquisition unit 51 is a processing unit that acquires first audio data, second audio data, and third audio data. As an aspect, the acquisition unit 51 reads the first audio data 12A, the second audio data 12B, and the third audio data 12C stored in the audio storage unit 11. As another aspect, the acquisition unit 51 can also acquire the first audio data, the second audio data, and the third audio data that have been A / D converted by the registration unit 31 as stream data. As a further aspect, the acquisition unit 51 can also acquire first audio data, second audio data, and third audio data from an external device (not shown) via a network.

The framing unit 52 is a processing unit that frames the first audio data 12A, the second audio data 12B, and the third audio data 12C acquired by the acquisition unit 51 into frames of a predetermined section. As an aspect, the framing unit 52 compares the lengths of the first audio data 12A, the second audio data 12B, and the third audio data 12C. Then, if the difference in length between the first audio data 12A, the second audio data 12B, and the third audio data 12C is not within the allowable error range, the framing unit 52 gives an error to a display unit (not shown) or the like. Output a message and do not perform further processing. On the other hand, when the lengths of the first audio data 12A, the second audio data 12B, and the third audio data are the same or within an allowable error range, the framing unit 52 is as follows. Perform appropriate processing. That is, the framing unit 52 frames the first audio data 12A, the second audio data 12B, and the third audio data 12C. For example, the framing unit 52 uses the following formulas (1) and (2) to frame each audio data with a length of 256 ms. At this time, the framing unit 52 sets the length of the overlapped portion of the preceding and following frames to 128 ms. Note that the length of the frame and the length of the overlapping portion of the preceding and following frames are merely examples, and arbitrary values can be adopted.
S = floor (Y / X) ... Formula (1)
m = floor ((S-256) / 128) +1 (2)
Note that “floor (x)” is a function for calculating the maximum integer equal to or smaller than x, and Y is the first audio data 12A, the second audio data 12B, and the third audio data 12C. It is a data amount (byte), and X is a length (ms) corresponding to 1 (byte) data.

  The first identifying unit 53 is a processing unit that identifies whether the frame framed by the framing unit 52 is a voiced sound region or an unvoiced sound region based on the first probability model. As an aspect, the first identification unit 53 includes the first frame (1) to the first frame (m), the second frame (1) to the second frame (m), and the third frame (1) to third. The following processing is executed for each audio data of the frame (m). That is, the first discriminating unit 53 extracts three feature quantities including the number of peaks of the autocorrelation coefficient, the maximum value of the peak of the autocorrelation coefficient, and the spectral entropy. Further, the first identification unit 53 calculates an average value and a standard deviation of each of the three feature quantities extracted previously for each audio data. In addition, the first identifying unit 53 identifies the voiced sound area and the unvoiced sound area for each voice data using a hidden Markov model (HMM) which is a probability model.

Here, a method for identifying the voiced sound area and the unvoiced sound area will be described. FIG. 5 is a diagram illustrating an example of a state transition diagram in the hidden Markov model. As shown in FIG. 5, the first identification unit 53 uses the above-described three feature quantities, and the average value and standard deviation of each feature quantity as observation results (observation), and uses an EM method (Expectation-Maximization algorithm). using calculates the state transition probability (transition possibility) P _t.

The state transition probability P _t is, for example, the probability of remaining a voiced sound state, the probability of transitioning from a voiced sound state to an unvoiced sound state, the probability of remaining an unvoiced sound state, and the state of a voiced sound to a voiced sound state Indicates the probability of transition to. In the example shown in FIG. 5, assuming that both voiced and unvoiced sounds start with the same probability, the probability of the state of voiced and unvoiced sounds at the start of the utterance is both “0.5”. Is set. Furthermore, as the initial state transition probability P _t , the probability that the voiced sound state remains is set to “0.95”, and the probability that the voiced sound state changes to the unvoiced sound state is “0.05”. Is set to Furthermore, as the initial state transition probability P _t , the probability of remaining unvoiced sound is set to “0.95”, and the probability of transition from unvoiced sound to voiced sound is set to “0.05”. Is set. Under such setting, the first identification unit 53 repeats the calculation of the state transition probability P _{t a} predetermined number of times. Thereby, it is possible to calculate the state transition probability P _t with high accuracy.

Furthermore, the first identifying unit 53, the three feature amounts described above, as well as the mean and standard deviation of each feature quantity and observation results, the Viterbi algorithm (Viterbi algorithm), the observation probability (observation possibility) P _o Calculation is performed for each audio data. Here, the observation probability P _o is, for example, the probability of outputting observation from the voiced sound state, the probability of outputting not observed from the voiced sound state, and the probability of outputting observation from the unvoiced sound state. And the probability of outputting non-observation from the state of unvoiced sound. The observation probability is also referred to as output probability.

After calculating the state transition probability P _t and the observation probability _Po , the first identification unit 53 executes the following process using the Viterbi algorithm based on the above three feature amounts. In other words, the first identification unit 53 identifies whether the sound being uttered in each frame where the utterance is being performed is a voiced sound or an unvoiced sound. In addition, the first identification unit 53 sets a region identified as a voiced sound as a voiced sound region and a region identified as an unvoiced sound as an unvoiced sound region.

  As described above, the first identifying unit 53 identifies the voiced sound region and the unvoiced sound region using the feature quantity such as the number of peaks of the autocorrelation coefficient, the maximum value of the peak of the autocorrelation coefficient, and the spectral entropy. . Therefore, in the 1st identification part 53, it can suppress that the precision which identifies a voiced sound area | region and an unvoiced sound area | region by the influence of surrounding noise falls. In addition, since the first identification unit 53 uses a feature amount that is strong against ambient noise, when the first audio data 12A, the second audio data 12B, and the third audio data 12C are framed, The number can be reduced. Therefore, the first identification unit 53 can identify the voiced sound area and the unvoiced sound area with simpler processing.

  The validation unit 54 validates the identification result of the frame having the maximum energy among the frames identified as the voiced sound area in the same section when the voiced sound areas overlap in the same section frame in each audio data. To do.

  As one aspect, the enabling unit 54 compares the identification results obtained by the first identifying unit 53 with frames in the same section in each audio data. At this time, when the voiced sound area overlaps in the frame of the same section, the enabling unit 54 calculates the energy of the frame identified as the voiced sound area. Then, the enabling unit 54 specifies the frame having the maximum energy among the frames identified as the voiced sound area in the same section. Then, the enabling unit 54 replaces the identification result other than the frame having the maximum energy from the voiced sound area to the unvoiced sound area. Thereafter, the validation unit 54 repeatedly executes comparison of identification results, identification of frames, and replacement of identification results until all frames in the same section are processed between the respective audio data. Note that the above energy is calculated by performing frequency analysis by performing fast Fourier transform, so-called FFT (Fast Fourier Transform), on each audio data frame, and then averaging the amplitude value for each frequency component. The

  Here, the replacement procedure of the identification result by the validation unit 54 will be described. FIG. 6 is a diagram illustrating an example of identification results of voiced sound areas and unvoiced sound areas. FIG. 7 is a diagram illustrating an example of a replacement result of the identification result illustrated in FIG. As shown in FIG. 6, in the section from “12: 00: 00.000” to “12: 00: 0.0010”, all the identification results of the first frame, the second frame, and the third frame are displayed. It is identified as a voiced sound area. In this case, as shown in FIG. 7, the enabling unit 54, except for the first frame having the maximum energy among the first frame, the second frame, and the third frame, The identification result is replaced from the voiced sound area “V” to the unvoiced sound area “U”. Further, in the section from “12: 00: 0.0010 seconds” to “12: 00: 0.0020 seconds” shown in FIG. 6, the identification results of the first frame and the second frame are identified as voiced sound regions. Yes. In this case, the enabling unit 54 maintains the identification result of the first frame having the maximum energy among the first frame and the second frame, as shown in FIG. The identification result is replaced from the voiced sound area “V” to the unvoiced sound area “U”. Further, as shown in FIG. 6, in the section from “12: 00: 0.020 second” to “12: 00: 0.0030 second”, only the identification result of the second frame is identified as the voiced sound region. Yes. In this case, the enabling unit 54 maintains the identification result of the second frame, as shown in FIG. 7, because the voiced sound regions do not overlap in the frames of the same section.

  The second identification unit 55 is a processing unit that identifies a speech region and a silence region in each audio data from the identification result of the frame after the validation by the validation unit 54 based on the second probability model. is there.

Here, a method for identifying a speech area and a silence area will be described. FIG. 8 is a diagram illustrating an example of a state transition diagram in the hidden Markov model. The state transition probability P _t and the observation probability P _o shown in FIG. 8 are predetermined values. The state transition probability P _t is, for example, the probability of staying in a silence state that is a silence state, the probability of transitioning from the silence state to the utterance state that is an utterance state, the probability of remaining in the utterance state, and the silence from the utterance state Indicates the probability of transition to a state. In the example shown in FIG. 8, assuming that both voiced and unvoiced sounds start with the same probability, both the silence state and the utterance state probabilities at the start of the utterance are set to “0.5”. Has been. Further, as the state transition probability P _t , the probability of remaining silent is set to “0.999”, and the probability of transition from the silent state to the speech state is set to “0.001”. Further, as the state transition probability P _t , the probability of remaining in the speech state is set to “0.999”, and the probability of transition from the speech state to the silence state is set to “0.001”.

The observation probability _Po is, for example, the probability that an unvoiced sound is detected in the silence state, the probability that the voiced sound is detected in the silence state, the probability that the unvoiced sound is detected in the utterance state, and the voiced sound in the utterance state. Indicates the probability. In the example of FIG. 8, the probability that an unvoiced sound is detected in the silence state is set to “0.99” as the observation probability _Po , and the probability that the voiced sound is detected in the silence state is “0.01”. "Is set. Further, as the observation probability P _o , the probability that an unvoiced sound is detected in an utterance state is set to “0.5”, and the probability that a voiced sound is detected in an utterance state is set to “0.5”. .

  In the example of FIG. 8, the case where both the probability that an unvoiced sound is detected in the utterance state and the probability that the voiced sound is detected in the utterance state is set to “0.5” is illustrated. It is also assumed that the unvoiced sound of a speaker who makes an utterance whose volume is lower than that of other speakers increases. Therefore, by setting the probability that an unvoiced sound is detected in an utterance state to be larger than “0.5”, it is possible to suppress an increase in the unvoiced sound of a speaker who makes an utterance whose volume is lower than that of other speakers.

  Under such a setting, the second identification unit 55 uses the Viterbi algorithm from the voiced sound and the unvoiced sound after being validated by the validation unit 54 in the silence area and the speech area in each voice data. Identify if there is. Thus, the utterance area and silence area of the speaker A in the first audio data, the utterance area and silence area of the speaker B in the second audio data, and the utterance area of the speaker C in the third audio data And silence regions are identified.

  Returning to the description of the conversation analysis device 10, the extraction unit 13 is a processing unit that extracts conversation characteristics from each voice data. As one aspect, the extraction unit 13 uses the average number of voiced sound regions and the number of voiced sound regions based on the utterance region of the speaker A in the first voice data identified by the second identification unit 55. The standard deviation of the value and the length of the voiced sound area is calculated. The extraction unit 13 also determines the number of utterance regions, the average value of the utterance regions, and the utterance region based on the utterance region of the speaker A in the first voice data identified by the second identification unit 55. Calculate the standard deviation of the length. Further, the extraction unit 13 uses the silence area of the speaker A in the first speech data identified by the second identification unit 55 to determine the number of silence areas, the average value of the silence area lengths, and the silence areas. The standard deviation of the length of is calculated.

  Further, the extraction unit 13 calculates the ratio of the length of the utterance time of the speaker A to the length of time of the entire conversation. At this time, the extraction unit 13 calculates the above ratio by regarding the total length of the utterance area of the speaker A as the length of the utterance time of the speaker A. Further, the extraction unit 13 calculates the ratio of the speaking time of the speaker A to the speaking time of the speaker B. Further, the extraction unit 13 also calculates the ratio of the utterance time of the speaker A to the utterance time of the speaker C. Further, the extraction unit 13 calculates the standard deviation of the volume and the standard deviation of the spectral entropy based on the utterance area of the speaker A. Furthermore, the extraction unit 13 calculates the sum of the standard deviation of the volume calculated based on the utterance area of the speaker A and the standard deviation of the spectral entropy as the degree of change. Here, the case where the conversation characteristic of the speaker A is extracted is illustrated, but the conversation characteristic is also extracted for the speaker B and the speaker C in the same manner as the speaker A described above.

Each conversation characteristic of the number of voiced sound areas, the average value of the length of the voiced sound area, and the standard deviation of the length of the voiced sound area thus calculated indicates how long the length of the voiced sound area is. It becomes an indicator. In addition, the number of utterance areas, the average value of the utterance area lengths, and the standard deviations of the utterance area lengths indicate that the corresponding person always speaks for a long time in the conversation or speaks little. It becomes an index indicating whether or not. In addition, the number of silence areas, the average value of the silence area lengths, and the standard deviation of the silence area lengths, the conversation characteristics, whether the speaker speaks for a long time, or is often interrupted (silence). It becomes an index that shows how to talk. Further, each conversation characteristics of speech time ratio R _t of a person to the length speech time parts and other persons in the speech time of a person to the length of the entire conversation time index indicating the participation status of the conversation It becomes. In addition, the conversation characteristics of volume standard deviation, spectral entropy standard deviation, and degree of change indicate whether the speaker is a passionate speaker with a strong emotional change or a quiet speaker with a small emotional change. It becomes an indicator to show.

The analysis unit 14 is a processing unit that analyzes the conversation style based on the conversation characteristics extracted by the extraction unit 13. As an embodiment, the analysis portion 14, the ratio R _t of utterance time a person with respect to speech time of the other person, a predetermined value, if for example 1.5 or more, this "a person" is a conversation Analyzes that the person speaks well. Further, when the ratio R _t is a predetermined value, for example, 0.66 or less, the analysis unit 14 analyzes that this “certain person” is a so-called hearing person who does not speak much in conversation. Incidentally, the analysis portion 14, the ratio R _t is a predetermined value, for example greater than 0.66, when it is less than 1.5, the analysis and both are equal in participation for conversation.

  As another aspect, the analysis unit 14 determines that the ratio of the number of voiced sound regions to the number of utterance regions of a certain person and the average value of the length of the utterance regions are the voiced sound regions with respect to the number of utterance regions of other persons. When the ratio is larger than the average value of the ratio of the number of utterances and the length of the utterance region, analysis is performed as follows. That is, the analysis unit 14 analyzes that “a certain person” is a person who tends to talk for a long time in the conversation. The analysis unit 14 also determines that the average value of the silence area length of a person is larger than the average value of the silence area length of another person, and the standard deviation of the length of the silence area of a person is a predetermined value. For example, when it is 6.0 or more, the analysis is performed as follows. That is, the analysis unit 14 analyzes that “a certain person” is a person whose utterance length is not constant because he / she utters the utterance in accordance with the contents of the other party by listening to the other party ’s story.

  As a further aspect, when the standard deviation of the volume of a certain person, the standard deviation of the spectral entropy, or the degree of change is equal to or greater than the corresponding reference value, the analysis unit 14 determines that “a certain person” is an emotion. Analyzed as a passionate speaker with drastic changes. In addition, the analysis unit 14 indicates that “a certain person” has a quiet and small emotional change when the standard deviation of the volume of the person, the standard deviation of the spectral entropy, or the degree of change is less than the corresponding reference value. Analyzing to be a good speaker.

As another aspect, the analysis unit 14 can also analyze the relationship between a certain person and another person. For example, the analysis portion 14, the ratio R _t is a predetermined value of the speech time of a person with respect to speech time of another person, if for example 1.0 or higher, "a person" whereas "other person" You can analyze that the relationship between one person and another person is a friend or family member. On the other hand, when the ratio R _t is a predetermined value, for example, less than 1.0, this “one person” is going to listen to the story of “another person”. Analyze your company colleagues and business partners.

  As a further aspect, the analysis unit 14 has a case where the average value of the length of the utterance area of a certain person in a conversation between a certain person and another person is a predetermined value, for example, 1.85 (s) or more. Can analyze that the relationship between a person and another person is a friend or family member. This is because “a certain person” often talks to “another person”. On the other hand, if the average value of the length of the utterance area of a person in a conversation between a person and another person is less than a predetermined value, for example, 1.85 (s), the analysis unit 14 Analyzes relationships with other people as company colleagues or business partners.

  As another aspect, the analysis unit 14 has a case where the average value of the silence area length of a person in a conversation between the person and another person is a predetermined value, for example, 3.00 (s) or less. For the same reason, it can be analyzed that the relationship between a person and another person is a friend or family member. On the other hand, if the average value of the length of the silence area of a certain person is greater than a predetermined value, for example, 3.00 (s), the analysis unit 14 determines that the relationship between a certain person and another person Can be analyzed as a business partner.

  As a further aspect, the analysis unit 14 has the same reason when the degree of change of a person in a conversation between a person and another person is a predetermined value, for example, 0.33 or more. Analyzes that the relationship between a person and another person is a friend or family member. On the other hand, when the degree of change of a certain person is less than a predetermined value, for example, less than 0.33, the analysis unit 14 can analyze that the relationship between a certain person and another person is a company colleague or a business partner. .

  After performing these analyses, the analysis unit 14 outputs the analysis results to a predetermined output destination device, such as a display unit included in the conversation analysis device 10 or an information processing device used by the speakers A to C. be able to.

  Note that various integrated circuits and electronic circuits can be adopted for the speaker discrimination device 50, the extraction unit 13, and the analysis unit 14. In addition, a part of the functional unit included in the speaker discrimination device 50 can be another integrated circuit or an electronic circuit. For example, an ASIC (Application Specific Integrated Circuit) is an example of the integrated circuit. Examples of the electronic circuit include a central processing unit (CPU) and a micro processing unit (MPU).

  Next, the process flow of the conversation analysis apparatus according to this embodiment will be described. Here, after (1) conversation analysis processing executed by the conversation analysis device 10 is described, (2) validation processing executed by the speaker discrimination device 50 will be described.

(1) Conversation Analysis Processing FIGS. 9 and 10 are flowcharts illustrating the procedure of conversation analysis processing according to the first embodiment. As an example, the conversation analysis process is started when an instruction to execute the conversation analysis process is received from an input unit (not shown).

  As illustrated in FIG. 9, the acquisition unit 51 acquires the first audio data 12A, the second audio data 12B, and the third audio data 12C (step S101). Then, the framing unit 52 determines whether the lengths of the first audio data 12A, the second audio data 12B, and the third audio data 12C are the same (step S102). Here, “same” includes the case where the difference in length is within the allowable error range.

  At this time, if the lengths of the respective audio data are not the same (No at Step S102), the framing unit 52 outputs an error message to a display unit (not shown) (Step S103) and ends the process.

  On the other hand, when the lengths of the respective audio data are the same (Yes at step S102), the framing unit 52 frames the first audio data 12A, the second audio data 12B, and the third audio data 12C. (Step S104).

  After that, the first identification unit 53 extracts three feature quantities of the number of autocorrelation coefficient peaks, the maximum value of the autocorrelation coefficient peaks, and the spectral entropy for each audio data (step S105). Then, the first identification unit 53 calculates an average value and a standard deviation of each of the three feature amounts extracted for each audio data (step S106).

Subsequently, the first identifying unit 53 sets 0 in the variable N (step S107), and sets the initial state transition probability P _t for the state transitions of the voiced and unvoiced sound in the hidden Markov model (step S108).

Then, the first identification unit 53 increments the value of the variable N by one (step S109). At this time, if the value of the variable N is not 5 or more (No at Step S110), the first identification unit 53 extracts the above three feature amounts extracted for each audio data and the average of the feature amounts. Using the value and standard deviation as observation results, the state transition probability P _t is calculated using the EM method (step S111), and the process proceeds to step S109.

On the other hand, when the value of the variable N is 5 or more (Yes at Step S110), the first identification unit 53 extracts the above three feature amounts extracted for each audio data and the average of the feature amounts. Using the value and the standard deviation as the observation result, the state transition probability P _t is calculated using the EM method (step S112).

Then, the first identification unit 53 uses the above three feature quantities extracted for each voice data, and the average value and standard deviation of each feature quantity as observation results, and uses the Viterbi algorithm to obtain an observation probability P _o is calculated (step S113).

  Thereafter, the first identification unit 53 performs the following process using the Viterbi algorithm based on the above three feature values extracted for each audio data. That is, the first identifying unit 53 identifies whether the sound being uttered is a voiced sound or an unvoiced sound in each frame where the utterance is being performed. Then, the first identification unit 53 sets the area where the voiced sound is detected as the voiced sound area, and sets the area where the unvoiced sound is detected as the unvoiced sound area (step S114).

  Here, when the voiced sound area overlaps the frames in the same section in each audio data, the enabling unit 54 identifies the frame having the maximum energy among the frames identified as the voiced sound area in the same section. An “validation process” is performed to validate (step S115).

  After that, the second identification unit 55 detects whether it is a silence state or a speech state by using the Viterbi algorithm based on the voiced sound and the unvoiced sound after the activation by the activation unit 54. Then, the silence area and the speech area are identified (step S116).

  Subsequently, as shown in FIG. 10, the extraction unit 13 determines the number of voiced sound regions, the average value of the lengths of the voiced sound regions, and the length of the voiced sound regions from the frame identified as being uttered by a certain speaker. Is calculated (step S117).

  Further, the extraction unit 13 calculates the number of utterance areas, the average value of the lengths of the utterance areas, and the standard deviation of the length of the utterance areas from the frame specified that a certain speaker has uttered (step S118). Thereafter, the extraction unit 13 calculates the number of silence areas, the average value of the silence area lengths, and the standard deviation of the silence area lengths from the frame of the silence area of a certain speaker (step S119).

  Then, the extraction unit 13 calculates the ratio of the length of the utterance time of a certain speaker to the length of time of the entire conversation (step S120). Further, the extraction unit 13 calculates the ratio of the utterance time of a certain speaker to the utterance time of another speaker (step S121).

  Subsequently, the extraction unit 13 calculates the standard deviation of the sound volume and the standard deviation of the spectral entropy from the frame specified that a certain speaker has spoken (step S122). The extraction unit 13 calculates, as the degree of change, the sum of the standard deviation of the volume calculated from the frame identified as being spoken by a certain speaker and the standard deviation of the spectral entropy (step S123).

  Until the conversation characteristics of all the speakers are extracted (No at Step S124), the processes from Step S117 to Step S123 are repeated. Thereafter, when the conversation characteristics of all the speakers are extracted (Yes at Step S124), the analysis unit 14 analyzes the conversation style based on the conversation characteristics extracted by the extraction unit 13 (Step S125). Finally, the analysis unit 14 outputs the analysis result to a predetermined output destination device (step S126), and ends the process.

(2) Validation Process FIG. 11 is a flowchart illustrating the validation process procedure according to the first embodiment. This validation process is a process corresponding to step S115 shown in FIG. 9, and starts after the voiced sound area and the unvoiced sound area are identified.

  As illustrated in FIG. 11, the enabling unit 54 compares the identification results obtained by the first identifying unit 53 with frames in the same section in each audio data (step S301). At this time, if the voiced sound regions overlap in the same frame (Yes in step S302), the enabling unit 54 calculates the energy of the frame identified as the voiced sound region (step S303). If the voiced sound areas do not overlap in the same frame (No in step S302), the process proceeds to step S306.

  Then, the enabling unit 54 specifies the frame having the maximum energy among the frames identified as the voiced sound area in the same section (step S304). Then, the enabling unit 54 replaces the identification result other than the frame having the maximum energy from the voiced sound region to the unvoiced sound region (step S305).

  Thereafter, the processing from step S301 to step S305 is repeated until all the frames in the same section are processed between the audio data (No in step S306). Then, when all the frames in the same section are processed between the respective audio data (Yes at step S306), the process is terminated.

[Effect of Example 1]
As described above, the speaker discriminating apparatus 50 according to the present embodiment can identify only the identification result of the frame having the maximum energy when the voiced sound area is identified in duplicate in the same section frame in each voice data. Is activated to identify the speech area and silence area of each audio data. For this reason, the speaker discriminating apparatus 50 according to the present embodiment can discriminate a speaker without using a threshold between frames in the same section constituting each audio data. Further, in the speaker discrimination device 50 according to the present embodiment, it is not necessary to use a complicated algorithm for speaker discrimination and it is not necessary to perform learning in advance. Therefore, according to the speaker discriminating apparatus 50 according to the present embodiment, the speaker can be discriminated easily and accurately.

  Further, the speaker discrimination device 50 according to the present embodiment is identified as the voiced sound area regardless of the magnitude of energy when the voiced sound area is identified independently in the frames of the same section in each audio data. Maintain identification results. In general, utterances are composed of a mixture of voiced and unvoiced sounds, so even if multiple speakers speak at the same time, it is unlikely that voiced sound areas will overlap completely. There is a high probability that an opportunity to be identified alone will remain. Therefore, the speaker discrimination device 50 according to the present embodiment can discriminate simultaneous utterances even when there is a gap in the volume of the utterance by the speaker.

  Furthermore, the speaker discrimination device 50 according to the present embodiment replaces the identification result other than the frame having the maximum energy among the frames identified as the voiced sound region in the same section with the unvoiced sound region. For this reason, in the speaker discriminating apparatus 50 according to the present embodiment, only the discriminating result of the frame having the maximum energy can be validated by a simple process of replacing the discriminating information, so that the discrimination of the speaker can be realized easily.

  Although the embodiments related to the disclosed apparatus have been described above, the present invention may be implemented in various different forms other than the above-described embodiments. Therefore, another embodiment included in the present invention will be described below.

[energy]
For example, in the first embodiment, the case where only the identification result of the frame having the maximum energy is validated is illustrated. However, only the identification result of the frame where the other index related to energy is maximized may be validated. it can. As an example, the disclosed apparatus can validate only the identification result of the frame that has the largest difference between the maximum and minimum amplitudes observed in the frame. In this case, the replacement of the identification result can be realized by a simpler calculation than the energy calculation process.

[Microphone]
In the first embodiment, the case where a close-talking microphone is applied has been exemplified. However, the disclosed apparatus is not limited to this, and it is not necessary to keep other speakers other than the speaker wearing the microphone away from the microphone. There is no. For example, a microphone having directivity can be applied. In this case, the speaker A or the directional microphone is arranged so that the sensitivity in the direction in which the speaker A speaks is stronger than the sensitivity in the other direction, and the same applies to the speaker B and the speaker C. A directional microphone may be used. In the case of using a directional microphone, the number of speakers may be plural, and the disclosed device can be applied to two or more speakers.

[Distribution and integration]
In addition, each component of each illustrated apparatus does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured. For example, the speaker discrimination device 50, the extraction unit 13, or the analysis unit 14 may be connected as an external device of the conversation analysis device via a network. Further, another device may have the speaker discrimination device 50, the extraction unit 13 or the analysis unit 14, and the functions of the speaker discrimination device described above may be realized by being connected to a network and cooperating. .

[Speaker discrimination program]
The various processes described in the above embodiments can be realized by executing a prepared program on a computer such as a personal computer or a workstation. In the following, an example of a computer that executes a speaker discrimination program having the same function as that of the above embodiment will be described with reference to FIG.

  FIG. 12 is a schematic diagram illustrating an example of a computer that executes a speaker discrimination program according to the first and second embodiments. As shown in FIG. 12, the computer 100 includes an operation unit 110a, a speaker 110b, a microphone 110c, a display 120, and a communication unit 130. Further, the computer 100 includes a CPU 150, a ROM 160, an HDD 170, and a RAM 180. These units 110 to 180 are connected via a bus 140.

  As shown in FIG. 12, the HDD 170 includes the acquisition unit 51, the framing unit 52, the first identification unit 53, the validation unit 54, and the second identification unit 55 shown in the first embodiment. A speaker discrimination program 170a that performs the same function as is stored in advance. About this speaker discrimination | determination program 170a, as each component of each acquisition part 51 shown in FIG. 1, the framing part 52, the 1st identification part 53, the validation part 54, and the 2nd identification part 55, You may integrate or isolate | separate suitably. In other words, all data stored in the HDD 170 need not always be stored in the HDD 170, and only data necessary for processing may be stored in the HDD 170.

  Then, the CPU 150 reads the speaker discrimination program 170 a from the HDD 170 and expands it in the RAM 180. Thereby, as shown in FIG. 12, the speaker discrimination program 170a functions as a speaker discrimination process 180a. The speaker determination process 180a expands various data read from the HDD 170 in an area allocated to itself on the RAM 180 as appropriate, and executes various processes based on the expanded data. The speaker discrimination process 180a is performed by the acquisition unit 51, the framing unit 52, the first identification unit 53, the validation unit 54, and the second identification unit 55 shown in FIG. The process shown in FIGS. 9-11 is included. In addition, each processing unit virtually realized on the CPU 150 does not always require that all processing units operate on the CPU 150, and only a processing unit necessary for the processing needs to be virtually realized.

  Note that the speaker discrimination program 170a is not necessarily stored in the HDD 170 or the ROM 160 from the beginning. For example, each program is stored in a “portable physical medium” such as a flexible disk inserted into the computer 100, so-called FD, CD-ROM, DVD disk, magneto-optical disk, or IC card. Then, the computer 100 may acquire and execute each program from these portable physical media. In addition, each program is stored in another computer or server device connected to the computer 100 via a public line, the Internet, a LAN, a WAN, etc., and the computer 100 acquires and executes each program from these. It may be.

DESCRIPTION OF SYMBOLS 10 Conversation analyzer 11 Voice memory | storage part 12A 1st audio | voice data 12B 2nd audio | voice data 12C 3rd audio | voice data 30A, 30B, 30C Close-talking microphone 31 Registration part 50 Speaker discrimination | determination apparatus 51 Acquisition part 52 Frame-izing part 53 First identification unit 54 Validation unit 55 Second identification unit

Claims (4)

An acquisition unit for acquiring each voice data from a plurality of microphones arranged in each speaker;
A framing unit that frames the audio data acquired by the acquiring unit into frames of a predetermined section;
A first identification unit that identifies whether the frame framed by the framing unit is a voiced sound region or an unvoiced sound region based on a first probability model;
Effective when the voiced sound area is identified in duplicate in the same interval frame in each audio data, and the identification result of the frame having the maximum energy among the frames identified as the voiced sound area in the same interval is valid And
A second identification unit for identifying a speech area and a silence area in each voice data from the identification result of the frame after being validated by the validation unit based on a second probability model, Speaker discrimination device.   2. The speaker discrimination device according to claim 1, wherein the validation unit replaces an identification result other than a frame having the maximum energy among frames identified as a voiced sound region in the same section with an unvoiced sound region. . On the computer,
Acquire each voice data from multiple microphones placed in each speaker,
The acquired audio data is framed into frames of a predetermined section,
Identifying whether the frame is a voiced sound region or an unvoiced sound region based on a first probability model;
When the voiced sound area is identified in duplicate in the same interval frame in each audio data, the identification result of the frame having the maximum energy among the frames identified as the voiced sound area in the same interval is enabled,
A speaker discrimination program for executing each process for identifying a speech area and a silence area in each voice data from a validated frame identification result based on a second probability model. Computer
Acquire each voice data from multiple microphones placed in each speaker,
The acquired audio data is framed into frames of a predetermined section,
Identifying whether the frame is a voiced sound region or an unvoiced sound region based on a first probability model;
When the voiced sound area is identified in duplicate in the same interval frame in each audio data, the identification result of the frame having the maximum energy among the frames identified as the voiced sound area in the same interval is enabled,
A speaker discrimination method, comprising: executing each process for identifying a speech area and a silence area in each voice data based on a validated frame identification result based on a second probability model.

Images