JP6268916B2 - Abnormal conversation detection apparatus, abnormal conversation detection method, and abnormal conversation detection computer program - Google Patents

Description

  The present invention relates to an abnormal conversation detection device, an abnormal conversation detection method, and an abnormal conversation detection computer program for determining whether or not a conversation is abnormal based on, for example, audio signals obtained by recording conversations of a plurality of speakers.

  In recent years, fraudulent acts and malicious solicitation activities aimed at deceiving money, which are performed using telephone lines, have become a social problem. Therefore, a technique for estimating the psychological state of a speaker based on voice during a call via a telephone line has been proposed (see, for example, Patent Documents 1 and 2).

  For example, the utterance state detection device disclosed in Patent Document 1 extracts a high frequency component from the result of frequency analysis of the utterance data of a speaker, and calculates the degree of variation per unit time for the high frequency component. And this utterance state detection device detects the utterance state of a specific speaker based on a statistic for each predetermined section calculated based on a plurality of fluctuation degrees in a predetermined period obtained from the utterance data of the specific speaker To do.

  Moreover, the suppression state detection apparatus disclosed by patent document 2 analyzes the input audio | voice for every some frame, and calculates the average value of the analysis result. The suppression state detection device determines a threshold value based on statistical data regarding the average value of the analysis results and the cumulative frequency distribution of the analysis results for each of the plurality of speakers stored in advance, and the calculated average value of the analysis results. The appearance frequency of the analysis result having a value larger than the threshold value among the analysis results is calculated. And the suppression state detection apparatus determines the tension state of the vocal cord which utters sound based on the appearance frequency.

  These techniques are based on the premise that the voice of the transmitting speaker and the voice of the receiving speaker can be obtained separately. In order to acquire separately the voice of the speaker on the transmitting side and the voice of the speaker on the receiving side, for example, a call recording adapter is connected between the telephone body and the handset. Then, the state estimating device acquires the transmitting side audio signal and the receiving side audio signal from the adapter, and estimates the state of the speaker. In this case, the audio signal that can be acquired from the call recording adapter is limited to the audio signal of the call using the telephone connected to the call recording adapter. Therefore, if multiple telephones are connected to one telephone line, and the call recording adapter is connected to only one of the telephone lines, the state estimation device can detect the state of the speaker from a call using another telephone. Cannot be estimated. On the other hand, if a call recording adapter is connected between the modular rosette and the distributor, and the state estimation device acquires an audio signal from the call recording adapter, even if a plurality of telephones are connected to the distributor, The voice signal of the call can be acquired. However, in this case, the voice signal obtained from the call recording adapter is a mixture of the voice of the transmitting speaker and the voice of the receiving speaker. Therefore, even if the above-described technique is applied to such a voice signal, the voice of the transmitting speaker and the voice of the receiving speaker are separately obtained. It is difficult to obtain sufficient estimation accuracy. This is because the voice of one speaker is superimposed on the voice of the other speaker, so the feature of the voice for estimating the state of one speaker includes the voice feature of the other speaker. Because it ends up. On the other hand, a technique for separating sounds from two sound sources by estimating parameters of a sine wave superposition model has been proposed (see, for example, Patent Document 3).

JP2011-242755A JP 2012-168296 A JP 2008-304718 A

  In the technique described in Patent Document 3, the sine wave model does not include a term representing noise. However, in an actual call, noise emitted from a sound source around the speaker is superimposed on the voice of the speaker, so the technique described in Patent Document 3 May not be able to accurately separate the voices of the speakers.

  In view of this, an object of one aspect of the present invention is to provide an abnormal conversation detection apparatus that can determine whether or not a conversation is abnormal based on an audio signal obtained by recording conversations of a plurality of speakers.

  According to one embodiment, an abnormal conversation detection device is provided. This abnormal conversation detecting device includes a voice input unit that inputs a voice signal including conversations of a plurality of speakers, a storage unit, and a voice characteristic of a human voice in units of a frame having a predetermined time length. Extracting at least two feature quantities to be represented, storing the combination of the at least two feature quantities in a storage unit, and a probability of the same number of speakers as the distribution of the feature quantities stored in the storage unit Determine whether the fitting section that fits the distribution and the probability distribution of the same number as the number of speakers can approximate the distribution of the feature amount, and the probability distribution of the same number as the number of speakers approximates the distribution of the feature amount If not, the determination unit determines that the conversation is an abnormal conversation.

The objects and advantages of the invention will be realized and attained by means of the elements and combinations particularly pointed out in the appended claims.
It should be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the invention as claimed.

  The abnormal conversation detection device disclosed in this specification can determine whether or not the conversation is abnormal based on an audio signal obtained by recording conversations of a plurality of speakers.

It is a figure which shows an example of the position which connects the abnormal conversation detection apparatus by one Embodiment to a telephone line. It is a schematic block diagram of the abnormal conversation detection apparatus by 1st Embodiment. It is a functional block diagram of the process part which an abnormal conversation detection apparatus has. (A) is a figure which shows an example of distribution of the feature-value of the audio | voice when the conversation of two speakers is normal conversation, (b) is the case when the conversation of two speakers is abnormal conversation. It is a figure which shows an example of distribution of the feature-value of an audio | voice. It is an operation | movement flowchart of the abnormal conversation detection process by 1st Embodiment. It is a functional block diagram of the process part by 2nd Embodiment. It is an operation | movement flowchart of the abnormal conversation detection process by 2nd Embodiment. It is a schematic block diagram of the mobile telephone by which the abnormal conversation detection apparatus by any embodiment or its modification was mounted. It is a schematic block diagram of the server client system by which the abnormal conversation detection apparatus by any embodiment or its modification was mounted.

Hereinafter, the abnormal conversation detecting apparatus will be described with reference to the drawings.
The inventor divides a voice signal including the voice of the speaker when the speaker is speaking in a normal state in units of frames, and extracts two or more feature amounts representing the characteristics of the human voice from each frame. We found that the distribution of the feature amount of the speaker's voice can be approximated by one probability distribution such as a normal distribution. Furthermore, when the psychological state of the speaker is no longer normal and the psychological state is reflected in the speaker's voice, the inventor's voice feature distribution is one probability distribution. I found that I could not approximate.

Therefore, this abnormal conversation detection apparatus divides a voice signal in which conversations of a plurality of speakers are recorded in units of frames, and extracts two or more feature amounts representing characteristics of a human voice from each frame. The abnormal conversation detection device determines that each speaker is in a normal state and normal conversation is performed when the distribution of the feature amount can be approximated by the same probability distribution as the number of speakers. To do. On the other hand, when the distribution of the feature amount cannot be approximated by the same number of probability distributions as the number of speakers, the abnormal conversation detection device determines that the conversation is an abnormal conversation.
Note that the abnormal conversation is a conversation in which at least one of the speakers participating in the conversation is in an abnormal psychological state. Moreover, the state where the speaker's psychological state is abnormal is a state where the speaker cannot keep calm, for example, a state where he / she takes action such as angry, barking, or crying.

  In the first embodiment, the abnormal conversation detection device determines whether or not the conversation is an abnormal conversation based on an audio signal obtained by recording a call between two speakers using a telephone line. However, the abnormal conversation detection device determines whether or not the conversation is an abnormal conversation based on an audio signal recorded by a mobile phone, a video conference system, or a voice recorder and including a conversation between two speakers. Also good.

  FIG. 1 is a diagram illustrating an example of a position where an abnormal conversation detection apparatus according to one embodiment is connected to a telephone line. In this example, for convenience of explanation, the side on which the abnormal conversation detecting device is attached is set as the transmitting side, and the other side of the call with the transmitting side via the telephone line is set as the receiving side. In this embodiment, the abnormal conversation detection apparatus 1 acquires a voice signal from a call recording adapter 5 connected between a distributor 3 to which two telephones 2-1 and 2-2 are connected and a modular rosette 4. To do. Therefore, regardless of which of the telephones 2-1 and 2-2 is used by the sending speaker, the voice signal including the voice of the sending speaker passes through the call recording adapter 5. To do. A voice signal including the voice of the speaker on the receiving side is transmitted from the telephone line 6 to any telephone through the modular rosette 4 and the call recording adapter 5. Therefore, the audio signal output from the call recording adapter 5 to the abnormal conversation detecting device 1 is an analog signal in which the voice of the transmitting speaker and the voice of the receiving speaker are mixed.

  FIG. 2 is a schematic configuration diagram of the abnormal conversation detection apparatus according to the first embodiment. The abnormal conversation detection apparatus 1 includes an interface unit 11, an analog / digital converter 12, a processing unit 13, a storage unit 14, a digital / analog converter 15, and a speaker 16.

  The interface unit 11 is an example of a voice input unit and includes an audio interface. The interface unit 11 acquires an audio signal, which is an analog signal, from the call recording adapter 5 and outputs the audio signal to the analog / digital converter 12 (hereinafter referred to as an A / D converter). The A / D converter 12 digitizes the audio signal by sampling the analog audio signal at a predetermined sampling rate. Then, the A / D converter 12 outputs the digitized audio signal to the processing unit 13.

  The processing unit 13 includes, for example, one or a plurality of processors, a memory circuit, and a peripheral circuit. The processing unit 13 determines whether the conversation is an abnormal conversation based on the digitized audio signal. Details of processing by the processing unit 13 will be described later.

  The storage unit 14 includes, for example, a readable / writable nonvolatile semiconductor memory and a readable / writable volatile semiconductor memory. The storage unit 14 stores various data used in the abnormal conversation detection process executed on the processing unit 13 and various data generated during the abnormal conversation detection process. The storage unit 14 also stores a warning sound signal output from the speaker 16 when the processing unit 13 determines that the conversation is an abnormal conversation.

  A digital / analog converter 15 (hereinafter referred to as a D / A converter) converts a warning voice signal output from the processing unit 13 into an analog signal when the processing unit 13 determines that the conversation is an abnormal conversation. 16 output. The speaker 16 reproduces an analog warning sound signal.

FIG. 3 is a functional block diagram of the processing unit 13. The processing unit 13 includes a spectrum calculation unit 21, a feature amount extraction unit 22, a fitting unit 23, a determination unit 24, and a warning unit 25.
Each of these units included in the processing unit 13 is, for example, a functional module realized by a computer program that operates on a processor included in the processing unit 13.

  The spectrum calculation unit 21 divides a digitized audio signal (hereinafter simply referred to as an audio signal) into frames having a predetermined length. The frame length is set to 32 msec, for example. The spectrum calculation unit 21 may overlap a part of two consecutive frames. In this case, when moving from the current frame to the next frame, the spectrum calculation unit 21 may set the frame shift amount newly taken into the frame to, for example, 10 msec to 16 msec.

The spectrum calculation unit 21 converts the audio signal from the time domain to the frequency domain spectrum signal using time-frequency conversion for each frame. The spectrum calculation unit 21 can use, for example, Fast Fourier Transform (FFT) or Modified Discrete Cosign Transform (MDCT) as time-frequency conversion. The spectrum calculation unit 21 may perform time-frequency conversion after multiplying each frame by a window function such as a Hamming window or a Hanning window.
For example, if the frame length is 32 msec and the sampling rate of the A / D converter 12 is 8 kHz, since 256 sample points are included in one frame, the spectrum calculation unit 21 executes 256-point FFT.

  The spectrum calculation unit 21 outputs the spectrum signal to the feature amount extraction unit 22 every time a spectrum signal of a frame is obtained.

  For each frame, the feature amount extraction unit 22 extracts two or more feature amounts representing the characteristics of a human voice from the spectrum signal of the frame. In the present embodiment, the feature amount extraction unit 22 extracts a power integrated value and a pitch frequency in a frequency band in which a human voice is included as a feature amount representing a feature of a human voice.

For each frame, the feature amount extraction unit 22 calculates an integrated value of power in a frequency band including a human voice, for example, according to the following equation.
Here, S (f) is a spectrum signal at frequency f, and | S (f) | ² is a power spectrum at frequency f. Fmin and fmax represent the lower limit and the upper limit of the frequency band in which a human voice is included, respectively. P is an integrated value of power. Further, the feature amount extraction unit 22 may directly obtain the integrated power value from the square sum of the sample points for each frame without performing the time-frequency conversion of the frame.

  Also, the feature quantity extraction unit 22 calculates the maximum value (except for the peak value with a time difference of 0) of the autocorrelation function or the modified autocorrelation function for each frame in order to calculate the pitch frequency. . A frame corresponding to human voiced sound has a relatively high degree of autocorrelation, whereas a frame corresponding to unvoiced sound or background noise has a low degree of autocorrelation. Therefore, the feature amount extraction unit 22 compares the maximum value of the peak value with a predetermined threshold value, and determines that the frame contains the voiced sound of the speaker if the maximum value is larger than the predetermined threshold value. Then, the feature quantity extraction unit 22 uses the reciprocal of the time difference corresponding to the maximum value of the peak value as the pitch frequency. Note that the autocorrelation function is obtained by performing an inverse Fourier transform on the power spectrum of each frequency. The modified autocorrelation function is obtained by inverse Fourier transform of the power spectrum filtered using a linear predictive coding filter. Note that the feature quantity extraction unit 22 can also directly obtain the pitch frequency from the frame in the time domain by obtaining the autocorrelation function using the sample points for each frame without performing Fourier transform on the frame. As described above, when the feature amount extraction unit 22 directly calculates a feature amount from the signal value of each sample point of the time domain frame without using the spectrum signal, the spectrum calculation unit 21 may be omitted.

  The feature amount extraction unit 22 stores a set of pitch frequency and power integrated values for each frame in the storage unit 14 as a set of feature amounts.

  Note that the feature amount extraction unit 22 may use a normalized pitch frequency obtained by dividing the pitch frequency by a predetermined value as the feature amount. Similarly, the feature amount extraction unit 22 may use a normalized power integration value obtained by dividing the power integration value by a predetermined value as the feature amount. In addition, the feature amount extraction unit 22 compares the power integrated value with a noise determination threshold value Thn representing a noise component included in the frame, and only when the power integrated value is larger than the noise determination threshold value Thn, A set of pitch frequencies may be stored in the storage unit 14. As a result, the feature set extracted from the frame when no speaker is speaking is not used for fitting the feature distribution based on the probability distribution described later. It can be determined whether or not is an abnormal conversation.

The noise determination threshold value Thn is preferably set adaptively according to the background noise level of the call voice. Therefore, the feature quantity extraction unit 22 determines a frame in which neither the transmitting speaker nor the receiving speaker is speaking as a silent frame including only background noise. For example, if the integrated value of the power spectrum of the entire frequency band of the frame is less than a predetermined power threshold, the feature amount extraction unit 22 determines that the frame is a silent frame. Then, the feature quantity extraction unit 22 estimates the background noise level based on the integrated value of the power of the silent frame. For example, the feature quantity extraction unit 22 estimates the background noise level according to the following equation.
Here, Ps is the integrated value of the power of the latest silent frame, and noiseP is the background noise level before update. And noiseP 'is the background noise level after the update. In this case, the noise determination threshold value Thn is set according to the following equation, for example.
Here, γ is a constant set in advance, and is set to 2 to 3 [dB], for example.

  The fitting unit 23 determines whether or not the number of feature value pairs stored in the storage unit 14 has reached a predetermined number (for example, 100 to 1000) sufficient to fit the feature value distribution with the probability distribution. To do. Then, when the number of feature quantity sets reaches the predetermined number, the fitting unit 23 fits the feature quantity distribution with a mixed distribution including probability distributions equal to the number of speakers. In the present embodiment, a two-dimensional two-mixed Gaussian distribution having a pitch frequency and a power integrated value as one dimension each is used as the mixed distribution. The two-mixed Gaussian distribution is one of the mixed normal distributions.

  Then, the fitting unit 23 performs maximum likelihood estimation of a plurality of parameters representing each Gaussian distribution included in the two-mixed Gaussian distribution, using a set of pitch frequency and power integrated value obtained from each frame as a learning sample. For this purpose, for example, the fitting unit 23 uses an EM algorithm (also called an expected value maximization method). For example, the fitting unit 23, for each of the Gaussian distributions included in the two-mixed Gaussian distribution, a weighting coefficient that is a probability that each learning sample is generated by the Gaussian distribution, an average value vector (that is, an average value of each feature value) Set) and the maximum likelihood estimate of the covariance matrix.

Note that the fitting unit 23 may use a lognormal distribution as a probability distribution used for fitting the distribution of feature values. Also in this case, the fitting unit 23 uses the EM algorithm to calculate the weighting coefficient, average value vector, and covariance matrix for each of the lognormal distributions of the same number as the number of speakers included in the mixed lognormal distribution. Find the maximum likelihood estimate.
Further, the fitting unit 23 may use a Markov chain Monte Carlo method or simulated annealing instead of the EM algorithm as an algorithm used to obtain a probability distribution for fitting the distribution of the feature amount.

  The fitting unit 23 notifies the determination unit 24 of the weighting coefficient, average vector, and maximum likelihood estimation value of the covariance matrix of each probability distribution obtained by fitting the feature amount distribution.

  The determination unit 24 calculates a degree of fitness that represents the degree to which the same number of probability distributions as the number of speakers fitting the feature amount distribution match the feature amount distribution. If the degree of fitness is equal to or greater than the fitness determination threshold value, the determination unit 24 can approximate the feature distribution with the same number of probability distributions as the number of speakers, and thus determines that the conversation is normal conversation. On the other hand, if the fitness is less than the fitness determination threshold, the distribution of feature quantities cannot be approximated by the same probability distribution as the number of speakers, so the determination unit 24 determines that the conversation is an abnormal conversation.

FIG. 4A is a diagram illustrating an example of a distribution of voice feature values when two speakers are talking in a normal state. On the other hand, FIG. 4B is a diagram showing an example of the distribution of the voice feature amount when at least one of the two speakers is talking in an abnormal psychological state.
4 (a) and 4 (b), the horizontal axis represents the integrated power value, and the vertical axis represents the pitch frequency. Each point 400 represents one feature set. As shown in FIG. 4A, when two speakers are talking in a normal state, that is, when the conversation is a normal conversation, distribution of feature amounts indicated by ellipses 401 and 402 The distribution of the feature quantity is approximated relatively well by the two normal distributions fitted with. For this reason, the degree of fitness is also increased.

  On the other hand, if the psychological state of at least one of the two speakers is not normal and the conversation becomes abnormal, each speaker will make a rough voice, and the voice characteristics will change from normal. The distribution of feature values varies. As a result, the two normal distributions obtained by fitting the distributions of feature amounts indicated by the ellipse 403 and the ellipse 404 cannot approximate the distribution of feature amounts well. As a result, the fitness is also low.

In the present embodiment, the determination unit 24 calculates the average log likelihood for the two-dimensional vector sequence as the fitness according to the following equation.
Here, P (x _n | Ω) represents the probability that the n-th two-dimensional vector x _n (corresponding to each learning sample in this embodiment) is output from the parameter Ω of the probability distribution. N represents the total number of learning samples. w _i (i = 1, 2) represents the maximum likelihood estimate of the weighting coefficient of each Gaussian distribution. μ _i represents a maximum likelihood estimation value of an average value vector of each Gaussian distribution (that is, a set of average values of each feature amount). Σi represents the covariance matrix of each Gaussian distribution.

  The determination unit 24 compares the average log likelihood with the fitness determination threshold value Thf. Then, the determination unit 24 determines that the conversation is a normal conversation if the average log likelihood is equal to or greater than the fitness determination threshold value Thf. Note that the suitability determination threshold Thf is a lower limit value of the average log likelihood when the feature distribution can be regarded as being approximated by the same number of probability distributions as the number of speakers, and is experimentally performed in advance, for example. On the other hand, if the average log likelihood is less than the fitness determination threshold value Thf, the determination unit 24 determines that the conversation is an abnormal conversation. Then, the determination unit 24 notifies the warning unit 25 that the conversation is an abnormal conversation.

  When the determination unit 24 notifies the determination unit 24 that the conversation is an abnormal conversation, the warning unit 25 reads a warning sound signal from the storage unit 14. Then, the warning unit 25 outputs the warning sound signal to the speaker 16 via the D / A converter 15.

  The abnormal conversation detection apparatus 1 may have a warning light source. In this case, when it is determined that the conversation is an abnormal conversation, the warning unit 25 may warn the speaker on the transmission side by turning on or blinking the light source.

  FIG. 5 is an operation flowchart of the abnormal conversation detection process. The processing unit 13 executes abnormal conversation detection processing according to the following operation flowchart for each call. As an initialization process, the processing unit 13 erases the pitch frequency and the power integrated value stored in the storage unit 14.

  The spectrum calculation unit 21 calculates the spectrum signal of the current frame by performing time-frequency conversion on the current frame, which is the latest frame cut out from the audio signal (step S101). The spectrum calculation unit 21 outputs the spectrum signal of the current frame to the feature amount extraction unit 22.

  The feature amount extraction unit 22 extracts two or more feature amounts representing human voice features such as an integrated power value and a pitch frequency based on the spectrum signal of the current frame (step S102). Then, the feature quantity extraction unit 22 stores the extracted feature quantity set in the storage unit 14.

  The fitting unit 23 determines whether or not the number of feature value sets stored in the storage unit 14 has reached a predetermined number (step S103). If the number of feature quantity groups has not reached the predetermined number (step S103-No), the processing unit 13 sets the next frame as the current frame (step S104). And the process part 13 repeats the process after step S101.

  On the other hand, if the number of feature quantity sets stored in the storage unit 14 has reached a predetermined number (step S103—Yes), the fitting unit 23 sets the distribution of feature quantities to the same number as the number of speakers. Fitting is performed with a mixed distribution including a probability distribution (step S105). Then, the fitting unit 23 determines a maximum likelihood estimated value of each parameter (for example, a weighting coefficient, an average value vector, and a covariance matrix of each normal distribution included in the mixed distribution) representing a probability distribution obtained by fitting the feature amount distribution. 24 is notified.

  The determination unit 24 calculates the fitness of the probability distribution obtained by fitting the feature amount distribution (step S106). Then, the determination unit 24 determines whether or not the fitness level is equal to or greater than the fitness determination threshold value Thf (step S107). When the fitness level is equal to or higher than the fitness determination threshold value Thf (step S107—Yes), the determination unit 24 determines that each probability distribution can approximate the distribution of the feature amount. That is, the determination unit 24 determines that the conversation is a normal conversation (step S108).

On the other hand, when the fitness level is less than the fitness determination threshold value Thf (No in step S107), the determination unit 24 determines that each probability distribution cannot approximate the distribution of the feature amount. That is, the determination unit 24 determines that the conversation is an abnormal conversation (step S109). Then, the determination unit 24 notifies the warning unit 25 that the conversation is an abnormal conversation. The warning unit 25 issues a warning to the speaker on the transmission side (step S110).
After step S108 or S110, the processing unit 13 ends the abnormal conversation detection process.

Table 1 is a diagram showing an experimental result of determination as to whether or not a speaker is in an abnormal state according to the related art disclosed in JP 2013-011830 A and this embodiment. In this experiment, 100 voice signals in which conversations by any two of 25 speakers were recorded were used.

  As shown in Table 1, in the conventional technology, the correct answer rate was 47% for both the abnormal conversation and the normal conversation, whereas in this embodiment, the correct answer rate for the abnormal conversation and the normal conversation was It became 70% and 69%. As described above, it has been shown that the abnormal conversation detection apparatus according to the present embodiment can detect abnormal conversation more accurately than the prior art.

  As described above, this abnormal conversation detection apparatus uses two or more types of feature quantity distributions extracted from a speech signal including a plurality of speaker voices in the same probability distribution as the number of speakers. Whether or not the conversation is an abnormal conversation is determined based on whether or not the approximation is possible. Therefore, the abnormal conversation detection apparatus can determine whether or not the conversation is abnormal even if the voice signal includes voices of a plurality of speakers.

  If the number of speakers is three or more and is known in advance, the fitting unit 23 may fit the distribution of feature values with a mixed distribution including probability distributions corresponding to the number of speakers. .

  Next, an abnormal conversation detecting apparatus according to the second embodiment will be described. The abnormal conversation detection apparatus according to the second embodiment determines whether or not the conversation is an abnormal conversation based on an audio signal including conversations of an unspecified number of two or more speakers.

  FIG. 6 is a functional block diagram of the processing unit of the abnormal conversation detecting apparatus according to the second embodiment. The processing unit 13 ′ includes a spectrum calculation unit 21, a feature amount extraction unit 22, a fitting unit 23, a determination unit 24, a warning unit 25, and a speaker number estimation unit 26. Compared with the processing unit 13 according to the first embodiment shown in FIG. 3, the processing unit 13 ′ according to the second embodiment includes a speaker number estimation unit 26, a fitting unit 23, and a determination unit 24. The processing of is different. Therefore, hereinafter, the speaker number estimation unit 26, the fitting unit 23, and the determination unit 24 will be described. For other components of the abnormal conversation detecting device, refer to the description of the corresponding components of the abnormal conversation detecting device according to the first embodiment.

The speaker number estimation unit 26 estimates the number of speakers participating in the conversation. For example, the speaker number estimation unit 26 is extracted from each frame as disclosed in Daben Liu et al., “ONLINE SPEAKER CLUSTERING”, in Proceedings of ICASSP2004, vol. I, pp.333-336, 2004. The set of feature quantities is clustered using a genetic algorithm or the like. Then, the speaker number estimation unit 26 sets the obtained number of clusters as the number of speakers.
Note that the speaker number estimation unit 26 may estimate the number of speakers according to another method for estimating the number of speakers from the voice signal.
The speaker number estimation unit 26 notifies the determination unit 24 of the estimated number of speakers.

The fitting unit 23 variously changes the number of probability distributions included in the mixed distribution, and calculates a value of Akaike's Information Criterion (AIC) as the fitness for each number of the probability distributions. The value of AIC is calculated by the following formula.
Here, L is the maximum likelihood (for example, the likelihood for the sample used for fitting after fitting the feature distribution sample with the probability distribution using the EM algorithm), and ln (L) is, for example, This is the maximum value of the average log likelihood according to equation (4) when the probability distribution of the number of interest is included in the mixed distribution. k is the number of free parameters, and increases as the number of probability distributions included in the mixture distribution increases. For example, when a mixed Gaussian distribution or a mixed lognormal distribution is used as the mixed distribution, parameters such as a weight coefficient, an average value vector, and a covariance matrix are required to define one probability distribution. Therefore, every time the probability distribution increases, k increases by the number of those parameters.

The fitting unit 23 may calculate Bayesian information criteria (BIC) instead of calculating AIC. The value of BIC is calculated by the following formula.
Here, L is the maximum likelihood (like the equation (5), the likelihood for the sample used for fitting after fitting the feature distribution sample with the probability distribution using the EM algorithm), and k is The number of free parameters. M represents the size of the sample, that is, the number of sets of feature quantities used as learning samples.

  In this case, it is estimated that the probability distribution of the number when the value of AIC or the number of BIC is minimum is most suitable for the distribution of the feature amount. Therefore, the fitting unit 23 obtains the number of probability distributions when the value of AIC or the number of BIC is minimized. The number of probability distributions corresponds to the number of probability distributions most suitable for fitting the feature amount distribution. Then, the fitting unit 23 notifies the determination unit 24 of the number of probability distributions.

  The determination unit 24 compares the number of speakers notified from the speaker number estimation unit 26 with the number of probability distributions notified from the fitting unit 23 and most suitable for fitting the distribution of feature values. If the number of probability distributions is equal to the number of speakers, the determination unit 24 can determine that the distribution of feature quantities can be approximated by the probability distribution of the number of speakers, so that the conversation is determined to be a normal conversation. . On the other hand, if the number of probability distributions is larger than the number of speakers, the distribution of feature values cannot be approximated by the probability distribution of the number of speakers, so the determination unit 24 determines that the conversation is an abnormal conversation. .

  FIG. 7 is an operational flowchart of abnormal conversation detection processing according to the second embodiment. The processing unit 13 ′ executes the abnormal conversation detection process according to the following flowchart instead of steps S <b> 105 to S <b> 110 in the abnormal conversation detection process according to the first embodiment shown in FIG. 5.

In step S103, when the number of feature value pairs stored in the storage unit 14 has reached a predetermined number, the speaker number estimation unit 26 estimates the number of speakers participating in the conversation ( Step S201). Then, the speaker number estimation unit 26 notifies the determination unit 24 of the estimated number of speakers.
The fitting unit 23 calculates the number of probability distributions most suitable for fitting the feature amount distribution with the probability distribution (step S202). Then, the fitting unit 23 notifies the determination unit 24 of the number of probability distributions.

  The determination unit 24 determines whether or not the number of probability distributions is larger than the number of speakers (step S203). When the number of probability distributions is equal to the number of speakers (step S203—No), the determination unit 24 determines that the distribution of feature quantities can be approximated with the same number of probability distributions as the number of speakers. That is, the determination unit 24 determines that the conversation is a normal conversation (step S204).

On the other hand, when the number of probability distributions is larger than the number of speakers (step S203—Yes), the determination unit 24 determines that the distribution of feature quantities cannot be approximated with the same probability distribution as the number of speakers. That is, the determination unit 24 determines that the conversation is an abnormal conversation (step S205). Then, the determination unit 24 notifies the warning unit 25 that the conversation is an abnormal conversation. The warning unit 25 issues a warning to the speaker on the transmission side (step S206).
After step S204 or S206, the processing unit 13 ′ ends the abnormal conversation detection process. Note that the order of the processing in step S201 and the processing in step S202 may be reversed, or the processing in step S201 and the processing in step S202 may be performed in parallel.

  According to the second embodiment, the abnormal conversation detection device can appropriately determine whether or not the conversation is an abnormal conversation even if the number of speakers participating in the conversation is an unspecified number of two or more. .

  Note that according to the modification of the second embodiment, the number of speakers participating in the conversation may be input via a user interface (not shown) such as a touch panel. In this case, the speaker number estimation unit 26 may be omitted.

In addition, according to the modification of each of the above-described embodiments, the feature amount extraction unit 22 uses the norm of the delta cepstrum as a feature representing a human voice for each frame instead of the power integrated value or together with the power integrated value. May be calculated. The norm of the delta cepstrum is calculated by the following formula.
Here, C _t ⁽ⁿ⁾ represents the n-th order cepstrum of the frame t, and ΔC (n) represents the delta cepstrum.

In addition, the feature quantity extraction unit 22 uses a ratio of the geometric mean of the spectrum and the arithmetic mean as a feature quantity representing a human voice for each frame, instead of the pitch frequency or together with the pitch frequency, as in the following equation. The represented spectral flatness-measure may be calculated.
Here, f _k is a spectrum signal at the frequency k (= 1,..., N), and N is the total number of frequencies at which the spectrum signal is calculated (ie, half the number of sampling points included in the frame). Represents. Ξ (f) is a flatness-measure. The flatness-measure is, for example, Hayakawa et al., “Speaker recognition using personality information included in the harmonic structure of the linear prediction residual spectrum”, IEICE Journal A, Vol. J80-A, No. .9, pp.1360-1367, 1997.

  In this case, the fitting unit 23 may fit the obtained delta cepstrum norm and flatness-measure distribution with probability distributions equal to the number of speakers. Alternatively, the fitting unit 23 may fit the distribution of three or more types of feature amounts among the obtained power, the delta cepstrum norm, the pitch frequency, and the flatness-measure with a probability distribution equal to the number of speakers.

According to another modification, the processing unit may continue the abnormal conversation detection process until the conversation ends even if it is determined that the conversation is a normal conversation. In this case, the feature value extraction unit 22 continues to extract feature value sets for each frame, and the fitting unit 23 performs the latest predetermined number (for example, 100 to 1000) of feature value sets. The feature amount distribution may be fitted with the probability distribution of the number of speakers. According to this modification, the abnormal conversation detecting device can warn the transmitting speaker during the call that the conversation has become an abnormal conversation, so that the transmitting speaker remains in an abnormal psychological state. , Can interrupt the call before incurring any disadvantages, or give a chance to return to normal.
In addition, the abnormal conversation detection device according to each of the above embodiments or modifications has an abnormal psychological state of any speaker participating in the conversation when a condition for determining that the conversation is an abnormal conversation is satisfied. It may be determined that

The abnormal conversation detection device may be mounted on a mobile phone.
FIG. 8 is a schematic configuration diagram of a mobile phone in which the abnormal conversation detection device according to any one of the above-described embodiments or modifications thereof is mounted. The cellular phone 30 includes a microphone 31, a communication unit 32, a storage medium access device 33, a storage unit 34, a user interface unit 35, a processing unit 36, and a speaker 37.

  The microphone 31 is an example of an audio input unit. The microphone 31 collects audio uttered by a speaker on the transmission side around the microphone 31 to generate an analog audio signal, and the analog audio signal is converted into an A / D converter (see FIG. (Not shown). The A / D converter generates a digital audio signal by sampling an analog audio signal at a predetermined sampling rate and digitizing it. Then, the A / D converter outputs the digitized audio signal to the processing unit 36.

  The communication unit 32 includes a wireless communication circuit for connecting the mobile phone 30 to a telephone line via a base station. And the communication part 32 receives the data stream containing the downstream audio | voice signal converted into the electrical signal from the audio | voice which the speaker of the receiving side received via the base station from the telephone line. And the communication part 32 extracts a downstream audio | voice signal from the data stream. Then, the communication unit 32 outputs the downlink audio signal to the processing unit 36.

  The storage medium access device 33 is a device that accesses a storage medium 38 such as a semiconductor memory card. The storage medium access device 33 reads, for example, a computer program executed on the processing unit 36 stored in the storage medium 38 and passes it to the processing unit 36. For example, the storage medium access device 33 may read a computer program for detecting abnormal conversation from the storage medium 38 and pass it to the processing unit 36.

  The storage unit 34 includes, for example, a readable / writable nonvolatile semiconductor memory and a readable / writable volatile semiconductor memory. The storage unit 34 stores various application programs executed on the processing unit 36 and various data. The storage unit 34 may store a computer program for executing the abnormal conversation detection process according to each of the above-described embodiments or modifications, and various data used for the abnormal conversation detection process. Further, the storage unit 34 may store an audio signal obtained by synthesizing the audio signal acquired via the microphone 31 and the downlink audio signal acquired via the communication unit 32.

  The user interface unit 35 includes, for example, an input device such as a plurality of operation keys and a display device such as a liquid crystal display. Alternatively, the user interface unit 35 may include a device in which an input device and a display device are integrated, such as a touch panel display. Then, the user interface unit 35 generates an operation signal corresponding to the operation of the input device by the speaker on the transmission side, and outputs the operation signal to the processing unit 36. The user interface unit 35 displays various information received from the processing unit 36 on the display device. Furthermore, the user interface unit 35 is an example of an output unit that outputs a warning, receives a warning message from the processing unit 36 when the processing unit 36 determines that the conversation is an abnormal conversation, and displays the warning message on the display device. Also good.

  The processing unit 36 includes one or a plurality of processors, a memory circuit, and a peripheral circuit. The processing unit 36 is connected to each unit of the mobile phone 30 via a signal line, and controls each unit of the mobile phone 30. Further, the processing unit 36 performs various processes for performing call setting and maintaining communication in response to an operation by a speaker or an incoming call signal. When the call is started, the processing unit 36 acquires a voice signal obtained by synthesizing the voice signal acquired via the microphone 31 and the downlink voice signal acquired via the communication unit 32. And the process part 36 determines whether a conversation is abnormal conversation by performing the abnormal conversation detection process performed by the process part of the abnormal conversation detection apparatus in said embodiment with respect to the synthesized speech signal. . In this example, the processing unit 36 can know the start and end of the conversation via the application programming interface (API) of the telephone application of the mobile phone 30.

In this example, when the processing unit 36 determines that the conversation is an abnormal conversation, the processing unit 36 displays a warning message on the display device of the user interface unit 35. Alternatively, the processing unit 36 may cause the speaker 37, which is another example of the output unit, to reproduce the audio signal of the warning message.
Alternatively, when the processing unit 36 determines that the conversation is an abnormal conversation, the processing unit 36 uses the electronic mail function of the mobile phone 30 to indicate that the conversation is determined to be an abnormal conversation to a mail address of a designated party. A warning mail may be automatically sent.

Furthermore, the abnormal conversation detection device according to each of the above-described embodiments or modifications thereof may be implemented in a server client system.
FIG. 9 is a schematic configuration diagram of a server client system in which the abnormal conversation detection device according to each of the above-described embodiments or modifications thereof is mounted.
The server client system 100 includes a terminal 110 and a server 120, and the terminal 110 and the server 120 can communicate with each other via a communication network 130. A plurality of terminals 110 included in the server client system 100 may exist. Similarly, a plurality of servers 120 included in the server client system 100 may exist.

  The terminal 110 includes a voice input unit 111, a storage unit 112, a communication unit 113, a control unit 114, and a speaker 115. The voice input unit 111, the storage unit 112, the communication unit 113, and the speaker 115 are connected to the control unit 114 via a bus, for example.

  The audio input unit 111 includes, for example, an audio interface and an A / D converter. Then, the voice input unit 111 acquires a voice signal that is an analog signal including a conversation from a call recording adapter connected between the modular rosette and the telephone, and samples the voice signal at a predetermined sampling rate. Digitize audio signals. Then, the voice input unit 111 outputs the digitized voice signal to the control unit 114.

  The storage unit 112 includes, for example, a nonvolatile semiconductor memory and a volatile semiconductor memory. The storage unit 112 stores a computer program for controlling the terminal 110, identification information of the terminal 110, various data used in abnormal conversation detection processing, a computer program, and the like.

  The communication unit 113 includes an interface circuit for connecting the terminal 110 to the communication network 130. Then, the communication unit 113 transmits the set of feature values received from the control unit 114 to the server 120 via the communication network 130 together with the identification information of the terminal 110. Further, the communication unit 113 receives a determination result as to whether or not the conversation is an abnormal conversation from the server 120 via the communication network 130 and passes the result to the control unit 114.

The control unit 114 includes one or a plurality of processors and their peripheral circuits. And the control part 114 implement | achieves the function of the spectrum calculation part 21, the feature-value extraction part 22, and the warning part 25 among each function of the process part by said each embodiment or modification. That is, the control unit 114 divides the audio signal into frames, and extracts two or more types of feature amounts representing the characteristics of a human voice from each frame. And the control part 114 transmits the group of the feature-value for every flame | frame to the server 120 via the communication part 113 and the communication network 130 with the identification information of the terminal 110. FIG.
When the control unit 114 receives a determination result that the conversation is an abnormal conversation from the server 120 via the communication network 130 and the communication unit 113, the control unit 114 outputs a warning sound via the speaker 115.

  The server 120 includes a communication unit 121, a storage unit 122, and a processing unit 123. The communication unit 121 and the storage unit 122 are connected to the processing unit 123 via a bus.

  The communication unit 121 includes an interface circuit for connecting the server 120 to the communication network 130. The communication unit 121 receives a set of feature amounts for each frame and the identification information of the terminal 110 from the terminal 110 via the communication network 130 and passes them to the processing unit 123. The communication unit 121 transmits a determination result that the conversation received from the processing unit 123 is an abnormal conversation to the terminal 110 via the communication network 130 based on the identification information of the terminal 110.

  The storage unit 122 includes, for example, a nonvolatile semiconductor memory and a volatile semiconductor memory. The storage unit 122 stores a computer program for controlling the server 120 and the like. In addition, the storage unit 122 may store a computer program for executing abnormal conversation detection processing and a set of feature values for each frame received from each terminal.

  The processing unit 123 includes one or a plurality of processors and their peripheral circuits. And the process part 123 implement | achieves the function of the fitting part 23 and the determination part 24 among each function of the process part by said each embodiment or modification. Further, the processing unit 123 may realize the function of the speaker number estimation unit 26. That is, the processing unit 123 fits the feature amount distribution with the same number of probability distributions as the number of speakers from the feature amount set for each frame received from the terminal 110. Then, the processing unit 123 determines that the conversation is a normal conversation if the fitted probability distributions can approximate the distribution of the feature amount, while the fitted probability distributions cannot approximate the distribution of the feature amount. For example, it is determined that the conversation is an abnormal conversation. Then, the processing unit 123 transmits the determination result to the terminal 110 via the communication unit 121 and the communication network 130.

  According to this embodiment, each terminal 110 extracts a set of feature values for each frame from an audio signal recording a conversation and transmits it to the server 120, and the determination result as to whether or not the conversation is an abnormal conversation. Can be obtained.

  A computer program that causes a computer to realize the functions of the processing unit of the abnormal conversation detection apparatus according to each of the above embodiments or modifications is provided in a form recorded on a computer-readable medium such as a magnetic recording medium or an optical recording medium. May be.

  All examples and specific terms listed herein are intended for instructional purposes to help the reader understand the concepts contributed by the inventor to the present invention and the promotion of the technology. It should be construed that it is not limited to the construction of any example herein, such specific examples and conditions, with respect to showing the superiority and inferiority of the present invention. Although embodiments of the present invention have been described in detail, it should be understood that various changes, substitutions and modifications can be made thereto without departing from the spirit and scope of the present invention.

The following supplementary notes are further disclosed regarding the embodiment described above and its modifications.
(Appendix 1)
A voice input unit for inputting a voice signal including conversations of a plurality of speakers;
A storage unit;
Extracting at least two feature amounts representing human voice features in units of frames having a predetermined time length from the audio signal, and storing the set of at least two feature amounts in the storage unit And
A fitting unit that fits the distribution of feature values stored in the storage unit with the same number of probability distributions as the number of speakers;
It is determined whether or not the probability distribution of the same number as the number of speakers can approximate the distribution of the feature amount, and the probability distribution of the same number as the number of speakers cannot approximate the distribution of the feature amount A determination unit that determines that the conversation is an abnormal conversation;
An abnormal conversation detecting device having
(Appendix 2)
The fitting unit fits the distribution of the set of at least two feature quantities with a mixed distribution including the same number of probability distributions as the number of speakers.
The determination unit calculates a fitness indicating the degree to which each probability distribution included in the mixed distribution is compatible with the distribution of the feature amount, and the probability distribution approximates the distribution of the feature amount. The abnormal conversation detection device according to appendix 1, wherein the conversation is determined to be an abnormal conversation when the conversation is less than a threshold value corresponding to a lower limit of the fitness when it is made.
(Appendix 3)
The fitting unit calculates the fitness for each of the probability distributions while changing the number of probability distributions included in the mixed distribution, and is most suitable for the feature amount distribution based on the fitness. Find the number of probability distributions,
The abnormal conversation detection device according to appendix 1, wherein the determination unit determines that the conversation is an abnormal conversation when the number of the probability distributions is larger than the number of the speakers.
(Appendix 4)
The fitting unit calculates the Akaike information criterion or the Bayesian information criterion as the fitness, and the number of probability distributions when the Akaike information criterion or the Bayesian information criterion is minimized is the best fit to the feature distribution. The abnormal conversation detection device according to appendix 3, which is obtained as the number of probability distributions.
(Appendix 5)
The abnormal conversation detection device according to appendix 3 or 4, further comprising a speaker number estimation unit that estimates the number of speakers from the voice signal.
(Appendix 6)
Acquire an audio signal containing conversations of multiple speakers,
Extracting at least two feature quantities representing the characteristics of a human voice in units of frames having a predetermined time length for the audio signal;
Fitting the extracted feature distribution with the same number of probability distributions as the number of speakers;
It is determined whether or not the probability distribution of the same number as the number of speakers can approximate the distribution of the feature amount, and the probability distribution of the same number as the number of speakers cannot approximate the distribution of the feature amount , Determining that the conversation is an abnormal conversation,
An abnormal conversation detection method including the above.
(Appendix 7)
Acquire an audio signal containing conversations of multiple speakers,
Extracting at least two feature quantities representing the characteristics of a human voice in units of frames having a predetermined time length for the audio signal;
Fitting the extracted feature distribution with the same number of probability distributions as the number of speakers;
It is determined whether or not the probability distribution of the same number as the number of speakers can approximate the distribution of the feature amount, and the probability distribution of the same number as the number of speakers cannot approximate the distribution of the feature amount , Determining that the conversation is an abnormal conversation,
A computer program for detecting an abnormal conversation for causing a computer to execute the operation.

DESCRIPTION OF SYMBOLS 1 Abnormal conversation detection apparatus 2-1, 2-2 Telephone 3 Distributor 4 Modular rosette 5 Call recording adapter 6 Telephone line 11 Interface part 12 A / D converter 13, 13 'Processing part 14 Storage part 15 D / A converter 16 Speaker DESCRIPTION OF SYMBOLS 21 Spectrum calculation part 22 Feature-value extraction part 23 Fitting part 24 Judgment part 25 Warning part 26 Speaker number estimation part 30 Cellular phone (abnormal conversation detection apparatus)
31 microphone 32 communication unit 33 storage medium access device 34 storage unit 35 user interface unit 36 processing unit 37 speaker 38 storage medium 100 server client system 110 terminal 111 voice input unit 112 storage unit 113 communication unit 114 control unit 115 speaker 120 server 121 communication Unit 122 Storage unit 123 Processing unit 130 Communication network

Claims (6)

A voice input unit for inputting a voice signal including conversations of a plurality of speakers;
A storage unit;
Extracting at least two feature amounts representing human voice features in units of frames having a predetermined time length from the audio signal, and storing the set of at least two feature amounts in the storage unit And
A fitting unit that fits the distribution of feature values stored in the storage unit with the same number of probability distributions as the number of speakers;
It is determined whether or not the probability distribution of the same number as the number of speakers can approximate the distribution of the feature amount, and the probability distribution of the same number as the number of speakers cannot approximate the distribution of the feature amount A determination unit that determines that the conversation is an abnormal conversation;
An abnormal conversation detecting device having The fitting unit fits the distribution of the set of at least two feature quantities with a mixed distribution including the same number of probability distributions as the number of speakers.
The determination unit calculates a fitness indicating the degree to which each probability distribution included in the mixed distribution is compatible with the distribution of the feature amount, and the probability distribution approximates the distribution of the feature amount. The abnormal conversation detection apparatus according to claim 1, wherein the conversation is determined to be an abnormal conversation when the conversation is less than a threshold value corresponding to a lower limit of the fitness level when it is made. The fitting unit changes the number of probability distributions included in the mixed distribution, and the degree of fitness representing the degree to which each probability distribution included in the mixed distribution is adapted to the distribution of the feature amount for each number of the probability distributions And calculating the number of probability distributions most suitable for the distribution of the feature amount based on the fitness,
The abnormal conversation detection apparatus according to claim 1, wherein the determination unit determines that the conversation is an abnormal conversation when the number of the probability distributions is larger than the number of the speakers.   The abnormal conversation detection apparatus according to claim 3, further comprising a speaker number estimation unit that estimates the number of speakers from the voice signal. Acquire an audio signal containing conversations of multiple speakers,
Extracting at least two feature quantities representing the characteristics of a human voice in units of frames having a predetermined time length for the audio signal;
Fitting the extracted feature distribution with the same number of probability distributions as the number of speakers;
It is determined whether or not the probability distribution of the same number as the number of speakers can approximate the distribution of the feature amount, and the probability distribution of the same number as the number of speakers cannot approximate the distribution of the feature amount , Determining that the conversation is an abnormal conversation,
An abnormal conversation detection method including the above. Acquire an audio signal containing conversations of multiple speakers,
Extracting at least two feature quantities representing the characteristics of a human voice in units of frames having a predetermined time length for the audio signal;
Fitting the extracted feature distribution with the same number of probability distributions as the number of speakers;
It is determined whether or not the probability distribution of the same number as the number of speakers can approximate the distribution of the feature amount, and the probability distribution of the same number as the number of speakers cannot approximate the distribution of the feature amount , Determining that the conversation is an abnormal conversation,
A computer program for detecting an abnormal conversation for causing a computer to execute the operation.