JP7095414B2 - Speech processing program, speech processing method and speech processing device - Google Patents

Description

The present invention relates to a voice processing program and the like.

In recent years, call centers have recorded conversations between operators and customers and accumulated information on the recorded conversations. The accumulated conversation information is used for feedback to the operator in order to improve the service.

In the prior art, there is a technique for determining whether or not a call is a nuisance call based on voice information from the start time to the end time of the call. In this conventional technique, the time of the entire call, the ratio of the voice section of the entire call, the stress evaluation value, and the number of times a predetermined keyword is detected are input to the model learned in advance to identify the nuisance call. ing.

Japanese Unexamined Patent Publication No. 2005-12831 International Publication No. 2008/032787 International Publication No. 2014/069122

However, in the above-mentioned conventional technique, there is a problem that it is not possible to determine whether the conversation situation is a normal conversation situation or an abnormal conversation situation.

In one aspect, it is an object of the present invention to provide a voice processing program, a voice processing method, and a voice processing apparatus capable of determining whether a conversation situation is a normal conversation situation or an abnormal conversation situation. ..

In the first plan, the computer is made to perform the following processing. The computer sets a set time set for each predetermined time interval from the start time of the conversation to be judged included in the voice information, and multiple feature quantities are selected from a plurality of voice information from the start time to each set time. calculate. The computer supports multiple features by inputting multiple features calculated for each set time into a model generated based on the features of voice information from the start time to the end time of the conversation. Calculate multiple output values of the model for each set time. The computer determines whether or not the conversation to be determined is an abnormal conversation situation based on a plurality of output values.

It is possible to determine whether the conversation situation is a normal conversation situation or an abnormal conversation situation.

FIG. 1 is a diagram for explaining the processing of the voice processing device according to the first embodiment. FIG. 2 is a functional block diagram showing a configuration of the voice processing device according to the first embodiment. FIG. 3 is a schematic diagram for explaining the model information according to the first embodiment. FIG. 4 is a diagram showing an example of the data structure of the output value storage buffer according to the first embodiment. FIG. 5 is a functional block diagram showing the configuration of the feature amount calculation unit according to the first embodiment. FIG. 6 is a diagram showing an example of the data structure of the pitch power storage unit according to the first embodiment. FIG. 7 is a diagram showing an example of a data structure of detection frequency information according to the second embodiment. FIG. 8 is a diagram for explaining variation 1 of the determination process. FIG. 9 is a diagram for explaining variation 2 of the determination process. FIG. 10 is a diagram for explaining variation 3 of the determination process. FIG. 11 is a diagram for explaining variation 4 of the determination process. FIG. 12 is a flowchart (1) showing a processing procedure of the voice processing apparatus according to the first embodiment. FIG. 13 is a flowchart (2) showing a processing procedure of the voice processing apparatus according to the first embodiment. FIG. 14 is a diagram for explaining the effect of the voice processing device according to the first embodiment. FIG. 15 is a diagram for explaining other processes of the conversation time management unit. FIG. 16 is a diagram for explaining the process of Reference Technique 2. FIG. 17 is a diagram for explaining the processing of the voice processing device according to the second embodiment. FIG. 18 is a functional block diagram showing the configuration of the voice processing device according to the second embodiment. FIG. 19 is a diagram showing an example of the data structure of the output value storage buffer according to the second embodiment. FIG. 20 is a functional block diagram showing the configuration of the feature amount calculation unit according to the second embodiment. FIG. 21 is a diagram showing an example of a data structure of detection frequency information according to the second embodiment. FIG. 22 is a flowchart (1) showing a processing procedure of the voice processing apparatus according to the second embodiment. FIG. 23 is a flowchart (2) showing a processing procedure of the voice processing apparatus according to the second embodiment. FIG. 24A is a diagram for explaining the first locus. FIG. 24B is a diagram for explaining the second locus. FIG. 24C is a diagram for explaining the third locus. FIG. 25 is a diagram showing an example of a hardware configuration of a computer that realizes a function similar to that of a voice processing device.

Hereinafter, examples of the voice processing program, the voice processing method, and the voice processing apparatus disclosed in the present application will be described in detail with reference to the drawings. The present invention is not limited to this embodiment.

Before explaining the voice processing device according to the first embodiment, the reference technique 1 for determining whether or not the conversation between the operator and the customer is an abnormal conversation situation will be described. This reference technique 1 is not a conventional technique.

The reference technique 1 determines whether the conversation situation is a normal conversation situation or an abnormal conversation situation based on the voice information (voice information of the entire conversation) from the start time to the end time of the conversation. Here, the "abnormal conversation situation" includes an "unusual situation" in which the customer feels dissatisfied, angry, or threatens.

This reference technique 1 seems to be an abnormal conversation situation by inputting the time of the whole conversation, the ratio of the voice section of the whole conversation, the stress evaluation value, and the number of times when a predetermined keyword is detected into the model learned in advance. Specify the output value that indicates. Reference Technique 1 determines that the conversation situation is an abnormal conversation situation when this output value is equal to or greater than the threshold value.

Here, if the customer gets angry only at the end of the conversation, or if the customer complains during the conversation, but the operator calms it down by speaking, it is preferable to judge that it is an "abnormal conversation situation". .. However, in Reference Technique 1, since it is determined whether or not the conversation is abnormal comprehensively by using the evaluation value and the analysis result for the entire conversation, the abnormal conversation situation is included in a part of the conversation. However, as a whole, the output value indicating the appearance of an abnormal conversation situation does not increase, and it may not be possible to determine that the conversation situation is abnormal.

Next, an example of processing of the voice processing device according to the first embodiment will be described. The voice processing device performs "a process of learning a model" for determining the appearance of an abnormal conversation situation and "a process of determining whether or not the conversation situation is abnormal".

When the voice processing device learns the model, the model is learned by using the evaluation values and the analysis results for the voice information of the entire conversation in the same manner as in Reference Technique 1.

Subsequently, when the voice processing device determines whether or not the conversation is abnormal, the evaluation values and analysis results from the conversation start time to each set time are input to the model at regular time intervals. Calculate and accumulate the output value of the model. The voice processing device determines whether it is a normal conversation situation or an abnormal conversation situation by using the locus obtained from the accumulated output value.

FIG. 1 is a diagram for explaining the processing of the voice processing device according to the first embodiment. In FIG. 1, the vertical axis corresponds to the output value of the model, and the horizontal axis corresponds to the conversation time. The output value 10a is an evaluation value of conversation in the section from time 0 to time t ₁ , and an output value when the analysis result is input to the model. The output value 10b is an output value when the evaluation value and the analysis result of the conversation in the section from time 0 to time t ₂ are input to the model. The output value 10c is an output value when the evaluation value and the analysis result of the conversation in the section from the time 0 to the time t3 _are input to the model. The output value 10d is an output value when the evaluation value and the analysis result of the conversation in the section from the time 0 to the time _t4 are input to the model. The output value 10e is an output value when the evaluation value and the analysis result of the conversation in the section from the time 0 to the time _t5 are input to the model. The output value 10f is an output value when the evaluation value and the analysis result of the conversation in the section from the time 0 to the time _t6 are input to the model.

The output value 10g is an evaluation value of conversation in the section from time 0 to time _t7 , and an output value when the analysis result is input to the model. The output value 10h is an evaluation value of conversation _in the section from time 0 to time t8, and an output value when the analysis result is input to the model. The output value 10i is an output value when the evaluation value and the analysis result of the conversation in the section from time 0 to time t ₉ are input to the model. The output value 10j is an evaluation value of conversation in the section from time 0 to time _t10 , and an output value when the analysis result is input to the model. The output value 10k is an output value when the evaluation value and the analysis result of the conversation in the section from the time 0 to the time t ₁₁ are input to the model.

The voice processing device determines whether it is a normal conversation situation or an abnormal conversation situation based on the loci of each output value 10a to 10k. As a result, if an abnormal conversation situation is included in a part of the conversation, the locus is changed, so that it is possible to determine whether the conversation situation is a normal conversation situation or an abnormal conversation situation. On the other hand, in the above reference technique 1, it is a normal conversation situation by comparing the evaluation value of the conversation during the entire conversation time (time 0 to t ₁₁ ) and the output value 10k input to the model of the analysis result. In order to determine whether the conversation situation is abnormal, even if a part of the conversation contains an abnormal conversation situation, it cannot be specified.

FIG. 2 is a functional block diagram showing a configuration of the voice processing device according to the first embodiment. As shown in FIG. 2, the voice processing device 100 includes a communication unit 110, a storage unit 120, and a control unit 130.

The communication unit 110 is a processing unit that executes data communication with an external device via a network. For example, the communication unit 110 receives voice information from a server device (not shown) that collects voice information including a conversation between a customer and an operator. The communication unit 110 outputs the received voice information to the control unit 130. The communication unit 110 is an example of a communication device. In the first embodiment, as an example, the conversation of voice information is a conversation between a customer and an operator, but the conversation is not limited to this, and a conversation between users may be used.

The storage unit 120 has a voice buffer 120a, model information 120b, and an output value storage buffer 120c. The storage unit 120 corresponds to a semiconductor memory element such as a RAM (Random Access Memory), a ROM (Read Only Memory), and a flash memory (Flash Memory), and a storage device such as an HDD (Hard Disk Drive).

The voice buffer 120a is a buffer for storing voice information of a conversation between a customer and an operator. When the voice processing device 100 performs the "process for learning the model", the voice information for learning is stored in the voice buffer 120a. On the other hand, when the voice processing device 100 performs the "processing for determining the conversation status", the voice information to be determined is accumulated in the voice buffer 120a.

The model information 120b is information on a model that outputs an output value indicating the degree to which the conversation included in the voice information is an abnormal conversation situation. FIG. 3 is a schematic diagram for explaining the model information according to the first embodiment. As shown in FIG. 3, this model information 120b has a neural network structure, and has an input layer 20a, a hidden layer 20b, and an output layer 20c. The input layer 20a, the hidden layer 20b, and the output layer 20c have a structure in which a plurality of nodes are connected by edges. The hidden layer 20b and the output layer 20c have a function called an activation function and a bias value, and the edge has a weight.

When the feature amount of voice information is input to each node included in the input layer 20a, the conversation is transmitted from each node of the output layer 20c through the hidden layer 20b with the probability "Ot" that the conversation is in an abnormal conversation situation. The probability "On", which is a normal conversation situation, is output.

In this embodiment, the output value V output from the model information 120b is defined by the equation (1). P (t) included in the equation (1) is a value defined by the equation (2). P (n) included in the equation (1) is a value defined by the equation (3).

V = logP (t) -logP (n) ... (1)

P (t) = exp (Ot) / {exp (Ot) + exp (On)} ... (2)
P (n) = exp (On) / {exp (Ot) + exp (On)} ... (3)

The output value storage buffer 120c is a buffer for storing the output value calculated based on the model information 120b. FIG. 4 is a diagram showing an example of the data structure of the output value storage buffer according to the first embodiment. As shown in FIG. 4, the output value storage buffer 120c associates the time with the output value. The time indicates the time of the voice information from which the feature amount is extracted (the elapsed time from the start time of the conversation). The output value indicates the output value V obtained when the feature amount calculated from the voice information at the corresponding time is input to the model information 120b. For example, in the example shown in FIG. 4, the output value obtained when the feature amount calculated from the voice information of the time “ ₀ to t ₁ ” is input to the model information 120b is the output value V1.

The control unit 130 includes an acquisition unit 130a, a feature amount calculation unit 130b, a model learning unit 130c, a conversation time management unit 130d, an output value calculation unit 130e, and a determination unit 130f. The control unit 130 can be realized by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like. Further, the control unit 130 can also be realized by hard-wired logic such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

The acquisition unit 130a is a processing unit that acquires voice information and stores the acquired voice information in the voice buffer 120a. For example, when the voice processing device 100 performs the "process for learning a model", the acquisition unit 130a acquires the voice information for learning and stores the voice information for learning in the voice buffer 120a. When the voice processing device 100 performs the "processing for determining the conversation status", the acquisition unit 130a acquires the voice information to be determined and stores the voice information in the voice buffer 120a.

The feature amount calculation unit 130b is a processing unit that calculates the feature amount based on the voice information stored in the voice buffer 120a. For example, the feature amount calculated by the feature amount calculation unit 130b is a stress evaluation value, the number of times a keyword is detected, and the elapsed time from the start time of conversation. The stress evaluation value, the number of detections, and the elapsed time will be described later.

When the voice processing device 100 performs the "process for learning the model", the feature amount calculation unit 130b outputs the feature amount to the model learning unit 130c. When the voice processing device 100 performs the "processing for determining the conversation status", the feature amount calculation unit 130b outputs the feature amount to the output value calculation unit 130e.

FIG. 5 is a functional block diagram showing the configuration of the feature amount calculation unit according to the first embodiment. As shown in FIG. 5, the feature amount calculation unit 130b includes a voice acquisition unit 131a, a frame processing unit 131b, a pitch extraction unit 132, a power calculation unit 133, a pitch power storage unit 134, and a stress evaluation value. It has a calculation unit 135. Further, the feature amount calculation unit 130b has a voice recognition unit 136, a recognition result storage unit 137, and a conversation time calculation unit 138.

The voice acquisition unit 131a acquires the voice information stored in the voice buffer 120a and outputs the read voice information to the frame processing unit 131b. In the following description, the voice information of the digital signal read by the voice acquisition unit 131a is simply referred to as "voice information".

The frame processing unit 131b takes out the signal time series of the voice information acquired from the voice acquisition unit 131a as a "frame" for each predetermined number of samples, and multiplies the frame by an analysis window such as a hanning window.

For example, the frame processing unit 131b takes out the sample N in a section of 32 ms at a sampling frequency of 8 kHz as a frame. For example, N = 256. Let each sample included in the frame be "s (0), s (1), s (2), ..., S (N-1)". The frame processing unit 131b multiplies each of the above samples by a humming window. For example, the humming window is represented by equation (4).

The sample obtained by multiplying each sample by the humming window is defined as "x (0), x (1), x (2), ..., X (N-1)". In the following description, the sample "x (0), x (1), x (2), ..., X (N-1)" obtained by multiplying the humming window is referred to as a sample value. The frame processing unit 131b outputs the sample value obtained by multiplying the frame by the Hanning window to the pitch extraction unit 132, the power calculation unit 133, and the voice recognition unit 136. The frame processing unit 131b may output sample value information and assign a frame identification number to the frame in frame units.

The pitch extraction unit 132 is a processing unit that extracts the fundamental frequency (pitch) of the frame based on the sample value of the frame. The pitch extraction unit 132 stores pitch information for each frame in the pitch power storage unit 134.

For example, the pitch extraction unit 132 calculates an autocorrelation function using each sample value of the frame. The pitch extraction unit 132 calculates the autocorrelation function φ (m) based on the equation (5). The m shown in the equation (5) indicates the delay time.

The pitch extraction unit 132 specifies the value of the delay time m at which the autocorrelation function becomes the maximum value for the equation (5) except for the delay time m = 0. The delay time m at which the autocorrelation function is maximized is expressed as "delay time m'". The pitch extraction unit 132 calculates the pitch based on the equation (6) after calculating the delay time m'.

Pitch = 1 / delay time m'... (6)

The pitch extraction unit 132 repeatedly executes the above processing for the sample value of each frame to calculate the pitch from each frame. However, when the maximum value of the autocorrelation function is equal to or less than a predetermined threshold value, the pitch and power of the frame are not used for the subsequent processing as a silent section.

The power calculation unit 133 is a processing unit that calculates the power of the frame based on the sample value of the frame. The power calculation unit 133 stores power information for each frame in the pitch power storage unit 134.

For example, the power calculation unit 133 is a logarithm with respect to the sum of the squared values of each sample value "x (0), x (1), x (2), ..., X (N-1)" of the frame. By taking, the power of the frame is calculated. Specifically, the power calculation unit 133 calculates the power of the frame based on the equation (7).

The pitch power storage unit 134 is a buffer that stores pitch information extracted by the pitch extraction unit 132 and power information calculated by the power calculation unit 133. FIG. 6 is a diagram showing an example of the data structure of the pitch power storage unit according to the first embodiment. As shown in FIG. 6, the pitch power storage unit 134 associates the frame identification number with the pitch and the power. However, the frame set as the silent section in the pitch extraction unit 132 is not included in the buffer.

The stress evaluation value calculation unit 135 is a processing unit that calculates a stress evaluation value for each set time based on the pitch and power information stored in the pitch power storage unit 134. For example, in the stress evaluation value calculation unit 135, the greater the variation in the sample of the current pitch and power set, the greater the variation in the sample when the user's normal pitch and power set is used as a sample. The stress evaluation value is increased, and the smaller the stress evaluation value is, the smaller the stress evaluation value is.

The stress evaluation value calculation unit 135 performs the following processing when the voice processing device 100 performs the “processing for determining the conversation status”. The stress evaluation value calculation unit 135 models and estimates the mixed Gaussian distribution by maximum likelihood estimation using a sample of a set of pitch powers accumulated from the start time of the conversation to the time when the output control signal is received. The average log-likelihood of the model for the sample used is multiplied by -1 to calculate as a stress evaluation value, and the calculated stress evaluation value is output to the output value calculation unit 130e. The "output control signal" is a signal output from the conversation time management unit 130d. For example, the stress evaluation value calculation unit 135 uses the EM algorithm (expected value maximization method) described in Japanese Patent Application Laid-Open No. 2015-082093 to perform modeling by maximum likelihood estimation.

The voice recognition unit 136 is a processing unit that detects whether or not a predetermined keyword is included in the voice information by, for example, performing word spotting type voice recognition. The voice recognition unit 136 performs a process of adding 1 to the number of detections corresponding to the predetermined keyword each time the predetermined keyword is detected from the voice information. The voice recognition unit 136 stores information in which a predetermined keyword is associated with the number of detections in the recognition result storage unit 137. A predetermined keyword is a keyword that is often spoken when a customer is dissatisfied or angry.

Further, the voice recognition unit 136 starts the voice section detection process for performing voice recognition, and outputs the information of the start time and the end time of the voice section when the voice section is detected to the conversation time calculation unit 138. ..

The recognition result accumulating unit 137 holds information (detection number information) of the number of detections of each keyword (predetermined keyword) detected by the voice recognition unit 136. FIG. 7 is a diagram showing an example of a data structure of detection frequency information according to the first embodiment. As shown in FIG. 7, the detection number information 137a associates the keyword with the detection number.

When the voice processing device 100 performs the "process for learning the model", the recognition result accumulating unit 137 performs the following processing. The recognition result storage unit 137 outputs the detection number information 137a at the end time of the conversation from the start time of the conversation to the model learning unit 130c.

When the voice processing device 100 performs the "processing for determining the conversation status", the recognition result accumulating unit 137 performs the following processing. The recognition result storage unit 137 outputs the detection number information 137a from the start time of the conversation to the time when the output control signal is received to the output value calculation unit 130e.

The conversation time calculation unit 138 is a processing unit that calculates the elapsed time of the conversation from the start time of the conversation. For example, the conversation time calculation unit 138 acquires time information from a timer (not shown) and measures the elapsed time from the start time of the conversation. The conversation time calculation unit 138 may estimate the elapsed time based on the cumulative number of samples included in each frame. The conversation time calculation unit 138 outputs the information of the start time and the elapsed time from the start time to the conversation time management unit 130d.

For example, the conversation time calculation unit 138 specifies the start time at which the information on the start time of the detected voice section is first received from the voice recognition unit 136 as the conversation start time. If the conversation time calculation unit 138 does not newly accept the information that the word has been detected even after a predetermined time has elapsed from the end time when the information of the detected voice section was last received, the conversation is terminated. judge. When the conversation time calculation unit 138 determines that the conversation has ended, the conversation time calculation unit 138 specifies the end time at which the detected voice section was last received as the end time. The conversation time calculation unit 138 outputs information on the end time of the conversation to the conversation time management unit 130d.

When the voice processing device 100 performs the "process for learning the model", the conversation time calculation unit 138 performs the following processing. The conversation time calculation unit 138 outputs information on the elapsed time from the conversation start time to the conversation end time to the model learning unit 130c.

When the voice processing device 100 performs the "processing for determining the conversation status", the conversation time calculation unit 138 performs the following processing. The conversation time calculation unit 138 outputs information on the elapsed time from the conversation start time to the time when the output control signal is received to the output value calculation unit 130e.

Returning to the description of FIG. The model learning unit 130c is a processing unit that generates (learns) model information 120b using a feature amount calculated from voice information for learning. When the model learning unit 130c generates the model information 120b, it is assumed that the correct answer data corresponding to the voice information for learning is stored in advance. For example, if the voice information of learning includes "abnormal conversation situation", the value of "Ot (probability of abnormal conversation situation)" of the correct answer data is "On (probability of normal conversation situation)". ) ”, Which is larger than the value. On the other hand, if the voice information of learning is the voice information of "normal conversation situation", the value of "Ot (probability of abnormal conversation situation)" of the correct answer data is "On (probability of normal conversation situation)". ) ”, Which is smaller than the value.

The model learning unit 130c inputs the feature amount calculated from the learning voice information into the input layer 20a of the model information 120b so as to reduce the difference between the value output from the output layer 20c and the correct answer data. , The bias value of the hidden layer 20b and the output layer 20c, and the weight of the edge are adjusted. The model learning unit 130c learns the model information 120b by repeatedly executing the above processing using the voice information for each learning and the correct answer data corresponding to the voice information for each learning. For example, the model learning unit 130c may learn the model information 120b by using an algorithm such as the Back Propagation method.

The conversation time management unit 130d acquires the conversation start time and the elapsed time from the conversation start time from the conversation time calculation unit 138, and determines whether or not the predetermined time T has elapsed. The conversation time management unit 130d outputs an "output control signal" to the stress evaluation value calculation unit 135, the recognition result storage unit 137, the conversation time calculation unit 138, and the output value calculation unit 130e each time the time T elapses.

When the conversation time management unit 130d receives the information on the end time of the conversation from the conversation time calculation unit 138, the conversation time management unit 130d outputs the information on the end time of the conversation to the determination unit 130f.

The output value calculation unit 130e is a processing unit that calculates an output value based on the feature amount acquired from the feature amount calculation unit 130b and the model information 120b. The output value calculation unit 130e stores the calculated output value in the output value storage buffer 120c.

For example, the output value calculation unit 130e acquires a feature amount from the feature amount calculation unit 130b at the timing when the output control signal is acquired from the conversation time management unit 130d, and inputs the acquired feature amount to the input layer 20a of the model information 120b. do. When the feature amount is input to the input layer 20a of the model information 120b, the output value calculation unit 130e acquires the values of the probability “Ot” and the probability “On” output from the output layer 20c, and obtains the values of the equation (1). )-Equation (3) is used to calculate the output value V.

Each time the output value calculation unit 130e acquires an output control signal from the conversation time management unit 130d, the output value calculation unit 130e repeatedly executes the above processing to sequentially calculate and calculate the output value V corresponding to the feature amount of each elapsed time. The information of the output value V is stored in the output value storage buffer 120c. The output value calculation unit 130e associates the elapsed time (time) with the accumulated output value V.

The determination unit 130f is a processing unit that determines whether the conversation is in an abnormal conversation situation or a normal conversation situation based on the locus of the output value stored in the output value storage buffer 120c. The determination unit 130f may output the determination result to a display device (not shown) and display it, or may notify the external device via the communication unit 110.

There are various variations in the determination process performed by the determination unit 130f. Hereinafter, variations 1 to 4 of the determination process performed by the determination unit 130f will be described. It is assumed that the user sets in advance which variation is used for the determination process.

FIG. 8 is a diagram for explaining variation 1 of the determination process. In FIG. 8, the vertical axis corresponds to the output value, and the horizontal axis corresponds to the conversation time. The determination unit 130f provides a threshold value 50, and the threshold value 50 provides a range in which the output value can be taken, a region 50a and a region 50b. When the output value exceeds the threshold value 50, it is highly possible that the conversation situation is an abnormal conversation situation. The threshold value 50 is a preset threshold value.

The determination unit 130f compares the locus of the output value with the threshold value 50, and determines that the conversation is in an abnormal conversation situation when the locus of the output value exceeds the threshold value 50 and is included in the region 50a.

When the determination unit 130f compares the locus 30a of the output value with the threshold value 50, the conversation ends without the locus 30a exceeding the threshold value 50. The determination unit 130f determines that the conversation corresponding to the locus 30a of the output value is a "normal conversation situation".

When the locus 30b of the output value is compared with the threshold value 50, the determination unit 130f exceeds the threshold value 50, temporarily enters the region 50a, and then returns to the region 50b. The determination unit 130f determines that the conversation corresponding to the locus 30b is an "abnormal conversation situation" because the locus 30b returns to the area 50b at the end of the conversation but exceeds the threshold value 50 once.

When the determination unit 130f compares the locus 30c of the output value with the threshold value 50, the locus 30c exceeds the threshold value 50 and invades the region 50a. The determination unit 130f determines that the conversation corresponding to the locus 30c is an "abnormal conversation situation".

FIG. 9 is a diagram for explaining variation 2 of the determination process. In FIG. 9, the vertical axis corresponds to the output value, and the horizontal axis corresponds to the conversation time. The determination unit 130f is provided with threshold values 50 and 51, and the regions 50b, 51a and 51b are provided by the threshold values 50 and 51. If the output value exceeds the threshold value 50, it is highly possible that the conversation is in an abnormal conversation situation. When the output value exceeds the threshold value 51, it is highly possible that the conversation is in an abnormal conversation situation (certainly, it is an abnormal conversation situation). The threshold values 50 and 51 are preset threshold values.

The determination unit 130f compares the locus of the output value with the threshold values 50 and 51, and determines that the conversation is in an abnormal conversation situation when the locus of the output value exceeds the threshold value 51 and is included in the region 51b. .. The determination unit 130f compares the locus of the output value with the threshold values 50 and 51, and if the entire locus of the output value is included in the area 51a, determines that the conversation is an abnormal conversation situation. .. The determination unit 130f compares the locus of the output value with the threshold values 50 and 51, and if a part of the locus of the output value is included in the area 50b, determines that the conversation is a normal conversation situation. do.

The determination unit 130f includes a part of the locus 31a in the region 50b when compared with the locus 31a of the output value and the threshold values 50 and 51. Therefore, the determination unit 130f determines that the conversation corresponding to the locus 31a is a "normal conversation situation".

When the determination unit 130f compares the locus 31b of the output value with the threshold values 50 and 51, the locus 31b exceeds the threshold value 51 and invades the region 51b. The determination unit 130f determines that the conversation corresponding to the locus 31b is an "abnormal conversation situation" because the locus 31b returns to the area 50b at the end of the conversation but exceeds the threshold value 50 once.

When the determination unit 130f compares the locus 31c of the output value with the threshold values 50 and 51, the entire locus 31c of the output value is included in the region 51a. Therefore, the determination unit 130f determines that the conversation corresponding to the locus 31c is an "abnormal conversation situation".

FIG. 10 is a diagram for explaining variation 3 of the determination process. In FIG. 10, the vertical axis corresponds to the output value, and the horizontal axis corresponds to the conversation time. The determination unit 130f is provided with threshold values 50 and 52, and the regions 50a, 52a and 52b are provided by the threshold values 50 and 52. If the output value exceeds the threshold value 50, it is highly possible that the conversation is in an abnormal conversation situation. When the output value is equal to or less than the threshold value 52, it is highly possible that the conversation is in a normal conversation situation (certainly, it is a normal conversation situation). The threshold values 50 and 52 are preset threshold values.

The determination unit 130f compares the locus of the output value with the threshold values 50 and 52, and determines that the conversation is a normal conversation situation when the locus of the output value falls below the threshold value 52 and is included in the region 52a. .. The determination unit 130f compares the locus of the output value with the threshold values 50 and 52, and if the locus is not included in the region 52a and exceeds the threshold value 50, the determination unit 130f determines that the conversation is in an abnormal conversation situation. ..

When the locus 32a of the output value is compared with the threshold values 50 and 52, the determination unit 130f never falls below the threshold value 52, and a part of the locus is included in the region 50a. Therefore, the determination unit 130f determines that the conversation corresponding to the locus 32a is an "abnormal conversation situation".

When the determination unit 130f compares the locus 32b of the output value with the threshold values 50 and 52, the locus 32b has a time zone in which a part of the locus is included in the region 50a but is below the threshold value 52. Therefore, the determination unit 130f determines that the conversation with respect to the locus 32b is a "normal conversation situation".

FIG. 11 is a diagram for explaining variation 4 of the determination process. In FIG. 11, the vertical axis corresponds to the output value, and the horizontal axis corresponds to the conversation time. The determination unit 130f is provided with threshold values 50, 51, 52, and the regions 51a, 51b, 52a, 52b are provided by the threshold values 50, 51, 52. When the output value exceeds the threshold value 51, it is highly possible that the conversation is in an abnormal conversation situation (certainly, it is an abnormal conversation situation). When the output value is equal to or less than the threshold value 52, it is highly possible that the conversation is in a normal conversation situation (certainly, it is a normal conversation situation). The threshold values 50, 51, and 52 are preset threshold values.

The determination unit 130f compares the locus of the output value with the threshold values 50, 51, 52, and when a part of the locus is included in the regions 51a and 52b, priority is given to the one closer to the end time of the conversation. For example, when the locus of the output value exceeds the threshold value 51 first and then becomes the threshold value 52 or less, the determination unit 130f gives priority to the fact that the locus becomes the threshold value 52 or less, and "normal conversation". It is determined that the situation is "situation". When the locus of the output value first becomes the threshold value 52 or less and then exceeds the threshold value 51, the determination unit 130f gives priority to the fact that the locus becomes the threshold value 51 or more, and in an "abnormal conversation situation". Judge that there is.

When the determination unit 130f compares the locus 33a of the output value with the threshold values 50, 51, 52, the locus 33a first exceeds the threshold value 51 and then becomes the threshold value 52 or less. The determination unit 130f gives priority to "the locus 33a is equal to or less than the threshold value 52" near the end time of the conversation, and determines that the conversation corresponding to the locus 33a is the "normal conversation situation".

When the determination unit 130f compares the locus 33b of the output value with the threshold values 50, 51, 52, the locus 33b first becomes the threshold value 52 or less, and then exceeds the threshold value 51. The determination unit 130f gives priority to "the locus 33b exceeds the threshold value 51" near the end time of the conversation, and determines that the conversation corresponding to the locus 33b is an "abnormal conversation situation".

Next, an example of the processing procedure of the voice processing apparatus 100 according to the first embodiment will be described. FIG. 12 is a flowchart showing a processing procedure of the voice processing apparatus according to the first embodiment. As shown in FIG. 12, the feature amount calculation unit 130b of the voice processing device 100 executes frame processing and extracts a frame from the voice information (step S101). The feature amount calculation unit 130b extracts the pitch of the frame (step S102) and calculates the power (step S103).

The feature amount calculation unit 130b accumulates pitch and power values (step S104), and proceeds to step S107. On the other hand, the feature amount calculation unit 130b executes voice recognition (step S105), updates the detection number information (step S106), and proceeds to step S107.

The conversation time management unit 130d of the voice processing device 100 determines whether or not it is time to calculate the output value (step S107). If it is not the time to calculate the output value (steps S107, No), the conversation time management unit 130d shifts to step S101.

When it is time to calculate the output value (step S107, Yes), the voice processing device 100 calculates the stress evaluation value (step S108), and proceeds to step S109. The output value calculation unit 130e of the voice processing device 100 calculates the output value of the model and stores it in the output value storage buffer 120c (step S109). The determination unit 130f of the voice processing device 100 calculates the locus of the output value (step S110), and proceeds to step S111 of FIG.

The description shifts to FIG. The determination unit 130f determines whether or not there is an abnormal conversation situation based on the locus (step S111). When the determination result is confirmed (step S112, Yes), the determination unit 130f shifts to step S115.

If the determination result is not finalized (step S112, No), the determination unit 130f determines whether or not the conversation has ended (step S113). If the conversation is not completed (step S113, No), the determination unit 130f proceeds to step S101 in FIG.

When the determination result is confirmed (step S113, Yes), the determination unit 130f determines whether or not there is an abnormal conversation situation based on the locus (step S114). The determination unit 130f outputs the determination result (step S115).

Next, the effect of the voice processing device 100 according to the first embodiment will be described. The voice processing device 100 sets a set time at predetermined time intervals from the start time of the conversation included in the voice information, and calculates a plurality of feature quantities from the voice information from the start time to each set time. The voice processing device 100 inputs each feature amount into the model information 120b, and determines whether or not the conversation is in an abnormal conversation situation based on the locus of each output value obtained from the model information 120b. This makes it possible to determine whether the conversation situation is normal or abnormal.

The voice processing device 100 divides the range that the locus of the output value can take into an abnormal area that is taken when the conversation situation is abnormal and a normal area that is taken when the conversation situation is normal, and the locus of the output value. And, based on the abnormal area and the normal area, it is determined whether or not the conversation is in an abnormal conversation situation. As a result, even if an abnormal situation is included in a part of the conversation, it is possible to accurately determine whether or not the conversation situation is abnormal.

FIG. 14 is a diagram for explaining the effect of the voice processing device according to the first embodiment. FIG. 14 shows graphs 60a, 60b, 60c. In each of the graphs 60a to 60c, the vertical axis corresponds to the output value, and the horizontal axis corresponds to the conversation time. The description of the threshold values 50 and 51 is the same as that of FIG. The threshold value 55 is a threshold value used by Reference Technique 1 when determining an abnormality or normal conversation situation.

Each locus shown in the graph 60a is an experimental result showing a typical locus for an abnormal conversation situation, and one locus corresponds to one conversation. Among the loci shown in the graph 60a, those included in the area 61a can be determined by the determination unit 130f that the conversation is in an abnormal conversation situation. Further, even if the locus is not included in the region 61a, most of the loci are not included in the region 61b, so that it can be determined that the conversation is in an abnormal conversation situation. For example, it can be accurately determined by the determination process based on the variation 2 described with reference to FIG.

Each locus shown in the graph 60b is an experimental result showing a typical locus with respect to a normal conversation situation. Of the loci shown in the graph 60b, most of the loci are included in the region 62b, and there is no locus included in the region 62a. Therefore, it can be determined that the conversation is a normal conversation situation. For example, it can be accurately determined by the determination process based on the variation 2 described with reference to FIG.

Each locus shown in the graph 60c is an experimental result of the locus for an abnormal conversation situation. Since all the loci are below the threshold value 55 at the end of the conversation, it cannot be determined that the conversation situation is abnormal by the determination based on the reference technique 1. On the other hand, according to the determination unit 130f according to the first embodiment, since the locus from the start time to the end time of the conversation is not included in the area 63b, it is determined that the conversation is in an abnormal conversation situation. can. For example, it can be accurately determined by the determination process based on the variation 2 described with reference to FIG.

By the way, since the value of the feature amount is not stable immediately after the start of the conversation, the output value obtained when the feature amount is output to the model information 120b may not be stable. Therefore, the conversation time management unit 130d uses the stress evaluation value calculation unit 135, the recognition result storage unit 137, and the conversation time calculation unit 138 for the "output control signal" from the reception of the conversation start time until the predetermined time elapses. , The process of outputting to the output value calculation unit 130e may be suppressed. As a result, the determination unit 130f can determine the conversation status using a stable output value.

FIG. 15 is a diagram for explaining other processes of the conversation time management unit. In FIG. 15, the vertical axis corresponds to the output value, and the horizontal axis corresponds to the conversation time. The conversation time management unit 130d outputs the "output control signal" to the stress evaluation value calculation unit 135, the recognition result storage unit 137, and the conversation time calculation unit at predetermined time intervals from the time when a predetermined time ta has elapsed from the start time 0. 138, output to the output value calculation unit 130e. As a result, the determination unit 130f can determine the conversation status based on the stable output value after the time ta. The description of the threshold values 50 and 51 and the loci 31a to 31c shown in FIG. 15 is the same as that of FIG.

Before explaining the voice processing device according to the second embodiment, the reference technique 2 for determining whether or not the conversation between the operator and the customer is an abnormal conversation situation will be described. This reference technique 2 is not a conventional technique. The voice information is divided at predetermined time intervals from the start time of the conversation, and the feature amount obtained from each divided voice information is input to the model information to calculate the output value.

FIG. 16 is a diagram for explaining the process of Reference Technique 2. As shown in FIG. 16, the reference technique 2 divides the voice information into a plurality of voice information 12a to 12k. The reference technique calculates the output values 11a to 11k by inputting each feature amount calculated in the section of each voice information 12a to 12k into the model. The model for inputting the feature amount corresponds to the model information 120b described in the first embodiment. In this way, when the voice information is divided into predetermined time intervals and the output values 11a to 11k are calculated, as shown in FIG. 16, since each output value is not stable, it may not be possible to accurately determine the conversation situation.

Next, an example of processing of the voice processing device according to the second embodiment will be described. FIG. 17 is a diagram for explaining the processing of the voice processing device according to the second embodiment. The horizontal axis of FIG. 17 is the axis corresponding to the conversation time, and the vertical axis is the axis corresponding to the output value. For example, the voice processing device divides the voice information every 30 seconds, inputs the feature amount of each divided voice information into the model information, and obtains each output value 11a to 11n. Each divided voice information is an example of the divided voice information. Further, the voice processing device inputs the feature amount of the voice information from the start time to the current time into the model information, and obtains an output value (not shown). The voice processing device determines the conversation status in real time. Let the current time be "Tc".

The voice processing device uses the average value of each output value from the start time to the current time Tc, the minimum value of each output value included within a predetermined time (for example, 5 minutes before) from the current time Tc, and the start time. The conversation status is determined based on the output value up to the current time Tc.

In the example shown in FIG. 17, the average value of each output value from the start time to the current time Tc is the average value of each output value 11a to 11n included in the time zone B1. The minimum value of each output value included from the current time Tc to a predetermined time before is the minimum value of the output values 11c to 11n included in the time zone B2. The output value of the current time Tc is an output value obtained by inputting the feature amount of the voice information in the section from the time 0 to the time Tc into the model.

The voice processing device according to the second embodiment determines that the conversation is in an abnormal conversation situation when "condition 2 and condition 1 are satisfied" or "condition 2 and condition 3 are satisfied". Th1 to Th3 included in the conditions 1 to 3 are preset threshold values. The magnitude relationship of each threshold value is Th3> Th1> Th2.

Condition 1: Mean value of each output value from the start time to the current time Tc> Th1
Condition 2: Minimum value of each output value included from the current time Tc to a predetermined time before> Th2
Condition 3: Output value of current time Tc from start time> Th3

The voice processing device according to the second embodiment determines the conversation status by using the above conditions 1 to 3 to determine whether or not the conversation status is abnormal even when the output value is not stable. It can be judged accurately.

FIG. 18 is a functional block diagram showing the configuration of the voice processing device according to the second embodiment. As shown in FIG. 18, the voice processing device 200 has a communication unit 210, a storage unit 220, and a control unit 230.

The communication unit 210 is a processing unit that executes data communication with an external device via a network. For example, the communication unit 210 receives voice information from a server device (not shown) that collects voice information including a conversation between a customer and an operator. The communication unit 210 outputs the received voice information to the control unit 230. The communication unit 210 is an example of a communication device.

The storage unit 220 has a voice buffer 220a, model information 220b, and an output value storage buffer 220c. The storage unit 220 corresponds to a semiconductor memory element such as RAM, ROM, and flash memory, and a storage device such as an HDD.

The voice buffer 220a is a buffer for storing voice information of a conversation between a customer and an operator. When the voice processing device 200 performs the "process for learning the model", the voice information for learning is stored in the voice buffer 220a. On the other hand, when the voice processing device 200 performs the "processing for determining the conversation status", the voice information to be determined is accumulated in the voice buffer 220a.

The model information 220b is information on a model that outputs an output value indicating the degree to which the conversation included in the voice information is an abnormal conversation situation. The other description of the model information 220b is the same as the description of the model information 120b described in the first embodiment.

The output value storage buffer 220c is a buffer for storing the output value output based on the model information 220b. FIG. 19 is a diagram showing an example of the data structure of the output value storage buffer according to the second embodiment. As shown in FIG. 19, the output value storage buffer 220c has tables 221a and 221b.

Table 221a associates the time with the output value. The time in the table 221a indicates the time of the voice information from which the feature amount is extracted. The output value indicates the output value V obtained when the feature amount extracted from the voice information at the corresponding time is input to the model information 220b.

Table 221b associates the time with the output value. The time in the table 221b indicates the time of the voice information from which the feature amount is extracted (the elapsed time from the start time of the conversation). The output value indicates the output value V obtained when the feature amount calculated from the voice information at the corresponding time is input to the model information 220b.

Returning to the description of FIG. The control unit 230 includes an acquisition unit 230a, a feature amount calculation unit 230b, a model learning unit 230c, a conversation time management unit 230d, an output value calculation unit 230e, and a determination unit 230f. The control unit 230 can be realized by a CPU, an MPU, or the like. Further, the control unit 230 can also be realized by hard-wired logic such as ASIC or FPGA.

The acquisition unit 230a is a processing unit that acquires voice information and stores the acquired voice information in the voice buffer 220a. For example, when the voice processing device 200 performs the "process for learning a model", the acquisition unit 230a acquires the voice information for learning and stores the voice information for learning in the voice buffer 220a. When the voice processing device 200 performs the "processing for determining the conversation status", the acquisition unit 230a acquires the voice information to be determined and stores the voice information in the voice buffer 220a.

The feature amount calculation unit 230b is a processing unit that calculates the feature amount based on the voice information stored in the voice buffer 220a. For example, the feature amount calculated by the feature amount calculation unit 230b is a stress evaluation value, the number of times a keyword is detected, and the elapsed time from the start time of conversation. The stress evaluation value, the number of detections, and the elapsed time will be described later.

When the voice processing device 200 performs the "process for learning the model", the feature amount calculation unit 230b outputs the feature amount to the model learning unit 230c. When the voice processing device 200 performs the "processing for determining the conversation status", the feature amount calculation unit 230b outputs the feature amount to the output value calculation unit 230e.

FIG. 20 is a functional block diagram showing the configuration of the feature amount calculation unit according to the second embodiment. As shown in FIG. 20, the feature amount calculation unit 230b includes a voice acquisition unit 231a, a frame processing unit 231b, a pitch extraction unit 232, a power calculation unit 233, a pitch power storage unit 234, and a stress evaluation value. It has a calculation unit 235. Further, the feature amount calculation unit 230b has a voice recognition unit 236, a recognition result storage unit 237, and a conversation time calculation unit 238.

The voice acquisition unit 231a acquires the voice information stored in the voice buffer 220a and outputs the read voice information to the frame processing unit 231b. In the following description, the voice information of the digital signal read by the voice acquisition unit 231a is simply referred to as "voice information".

The frame processing unit 231b takes out the signal time series of the voice information acquired from the voice acquisition unit 231a as a "frame" for each predetermined number of samples, and extracts the frame information from the pitch extraction unit 232 and the power calculation unit 233. It is output to the voice recognition unit 236. The processing of the frame processing unit 231b corresponds to the processing of the frame processing unit 131b of the first embodiment.

The pitch extraction unit 232 is a processing unit that extracts the fundamental frequency (pitch) of the frame based on the sample value of the frame. The pitch extraction unit 232 stores pitch information for each frame in the pitch power storage unit 234. The processing of the pitch extraction unit 232 corresponds to the processing of the pitch extraction unit 132 of the first embodiment.

The power calculation unit 233 is a processing unit that calculates the power of the frame based on the sample value of the frame. The power calculation unit 233 stores power information for each frame in the pitch power storage unit 234. The processing of the power calculation unit 233 corresponds to the processing of the power calculation unit 133 of the first embodiment.

The pitch power storage unit 234 is a buffer that stores pitch information extracted by the pitch extraction unit 232 and power information calculated by the power calculation unit 233. The data structure of the pitch power storage unit 234 is the same as the data structure of the pitch power storage unit 134 shown in FIG.

The stress evaluation value calculation unit 235 is a processing unit that calculates the stress evaluation value for each set time based on the pitch and power information stored in the pitch power storage unit 234. For example, the stress evaluation value calculation unit 235 calculates the stress evaluation value in the same manner as the stress evaluation value calculation unit 135.

When the voice processing device 200 performs the "processing for determining the conversation status", the stress evaluation value calculation unit 235 performs the following processing. The stress evaluation value calculation unit 235 models and estimates the mixed Gaussian distribution by maximum likelihood estimation using a sample of a set of pitch powers accumulated from the start time of the conversation to the time when the output control signal is received. The average log-likelihood of the model for the sample used is multiplied by -1 to calculate as a stress evaluation value, and the calculated stress evaluation value is output to the output value calculation unit 130e as the first stress value. The “output control signal” is a signal output from the conversation time management unit 230d.

Further, the stress evaluation value calculation unit 235 uses a sample of a set of pitch powers accumulated from the time when the output control signal was received last time to the time when the output control signal is received this time to maximum likelihood the mixed Gaussian distribution. Modeled by estimation, the average log-likelihood of the model for the sample used for estimation multiplied by -1 is calculated as the stress evaluation value, and the calculated stress evaluation value is used as the second stress value in the output value calculation unit 230e. Output to.

The voice recognition unit 236 is a processing unit that detects whether or not a predetermined keyword is included in the voice information by, for example, performing word spotting type voice recognition. The voice recognition unit 236 performs a process of adding 1 to the number of detections corresponding to the predetermined keyword each time the predetermined keyword is detected from the voice information. The voice recognition unit 236 stores information in which a predetermined keyword is associated with the number of detections in the recognition result storage unit 237. A predetermined keyword is a keyword that is often spoken when a customer is dissatisfied or angry.

For example, the voice recognition unit 236 distinguishes between the "first detection number" and the "second detection number" and stores them in the recognition result storage unit 237. The first detection number indicates the number of detections of a predetermined keyword detected in the voice section from the start time of the conversation to the time when the output control signal is received. The second number of detections indicates the number of detections of a predetermined keyword detected in the voice section from the time when the previous output control signal was received to the time when the output control signal was received this time.

Further, the voice recognition unit 236 starts the voice section detection process for performing voice recognition, and outputs the time information when the voice section is detected to the conversation time calculation unit 238.

The recognition result storage unit 237 holds information on the number of detections (information on the number of first detections and the number of second detections) of each keyword (predetermined keyword) detected by the voice recognition unit 236. FIG. 21 is a diagram showing an example of a data structure of detection frequency information according to the second embodiment. As shown in FIG. 21, the detection number information 237a has a table 237b and a table 237c.

Table 237b associates the keyword with the number of first detections. The first detection number indicates the number of detections of a predetermined keyword detected in the voice section from the start time of the conversation to the time when the output control signal is received.

Table 237c associates the time with the keyword and the second detection count. Time indicates the time interval at which each output control signal is received. The second number of detections indicates the number of detections of a predetermined keyword detected in the voice section from the time when the previous output control signal was received to the time when the output control signal was received this time.

When the voice processing device 200 performs the "process for learning the model", the recognition result accumulating unit 237 performs the following processing. The recognition result storage unit 237 outputs the information of the table 237b at the end time of the conversation from the start time of the conversation to the model learning unit 230c.

When the voice processing device 200 performs the "processing for determining the conversation status", the recognition result accumulating unit 237 performs the following processing. The recognition result storage unit 237 outputs the information of the table 237b from the start time of the conversation to the time when the output control signal is received to the output value calculation unit 230e. Further, the recognition result storage unit 237 records the record of the time corresponding to the time corresponding to the time when the output control signal is received this time from the time when the previous output control signal was received among the records in the table 237c to the output value calculation unit. Output to 230e. For example, assuming that the time when the output control signal was received last time is "t ₁ " and the time when the output control signal is received this time is "t ₂ ", the recognition result storage unit 237 has the time "t ₁ " among the records in the table 237c. The record corresponding to "~ t ₂ " is output to the output value calculation unit 230e.

The conversation time calculation unit 238 is a processing unit that calculates the elapsed time of the conversation from the start time of the conversation. For example, the conversation time calculation unit 238 acquires time information from a timer (not shown) and measures the elapsed time from the start time of the conversation. The conversation time calculation unit 238 may estimate the elapsed time based on the cumulative number of samples included in each frame. The conversation time calculation unit 238 outputs information on the start time and the elapsed time from the start time to the conversation time management unit 230d.

When the voice processing device 200 performs the "process for learning the model", the conversation time calculation unit 238 performs the following processing. The conversation time calculation unit 238 outputs information on the elapsed time from the conversation start time to the conversation end time to the model learning unit 230c.

When the voice processing device 200 performs the "processing for determining the conversation status", the conversation time calculation unit 238 performs the following processing. The conversation time calculation unit 238 outputs information on the elapsed time from the conversation start time to the time when the output control signal is received to the output value calculation unit 230e. Further, the conversation time calculation unit 238 outputs the information of the time interval from the time when the previous output control signal is received to the time when the output signal is received this time to the output value calculation unit 230e.

Returning to the description of FIG. The model learning unit 230c is a processing unit that generates (learns) model information 220b using a feature amount calculated from voice information for learning. The process of generating the model information 220b by the model learning unit 230c corresponds to the process of the model learning unit 130c described in the first embodiment.

The conversation time management unit 230d acquires the conversation start time and the elapsed time from the conversation start time from the conversation time calculation unit 238, and determines whether or not the predetermined time T has elapsed. The conversation time management unit 230d outputs the "output control signal" to the stress evaluation value calculation unit 235, the voice recognition unit 236, the recognition result storage unit 237, the conversation time calculation unit 238, and the output value calculation unit each time the time T elapses. Output to 230e.

When the conversation time management unit 230d receives the information on the end time of the conversation from the conversation time calculation unit 238, the conversation time management unit 230d outputs the information on the end time of the conversation to the determination unit 230f.

The output value calculation unit 230e is a processing unit that calculates an output value based on the feature amount acquired from the feature amount calculation unit 230b and the model information 220b. The output value calculation unit 230e stores the calculated output value in the output value storage buffer 220c.

For example, the output value calculation unit 230e acquires the feature amount from the feature amount calculation unit 230b at the timing when the output control signal is acquired from the conversation time management unit 230d. This feature amount includes a first feature amount and a second feature amount.

The first feature amount is a feature amount extracted based on the voice information from the start time of the conversation to the time when the output control signal is received this time. The first feature amount includes the first stress evaluation value, the information of the first detection number, and the information of the elapsed time from the start time of the conversation to the time when the output control signal is received this time.

The output value calculation unit 230e acquires the first feature amount from the feature amount calculation unit 230b at the timing when the output control signal is acquired from the conversation time management unit 230d, and the acquired first feature amount is used as the input layer 20a of the model information 220b. Enter in. When the feature amount is input to the input layer 20a of the model information 220b, the output value calculation unit 230e acquires the values of the probability “Ot” and the probability “On” output from the output layer 20c, and obtains the values of the equation (1). )-Equation (3) is used to calculate the output value V. The output value calculation unit 230e registers the information of the output value V calculated from the first feature amount in the table 221b.

Each time the output value calculation unit 230e acquires an output control signal from the conversation time management unit 230d, the output value calculation unit 230e repeatedly executes the above processing to sequentially calculate the output value V corresponding to the first feature amount of each elapsed time. , The calculated output value V information is stored in the table 221b and updated.

On the other hand, the second feature amount is a feature amount extracted based on the voice information in the section from the time when the previous output control signal is received to the time when the output control signal is received this time. The second feature amount includes a second stress evaluation value, information on the number of second detections, and information on the elapsed time from the time when the previous output control signal was received to the time when the output control signal was received this time.

The output value calculation unit 230e acquires the second feature amount from the feature amount calculation unit 230b at the timing when the output control signal is acquired from the conversation time management unit 230d, and the acquired second feature amount is used as the input layer 20a of the model information 220b. Enter in. When the feature amount is input to the input layer 20a of the model information 220b, the output value calculation unit 230e acquires the values of the probability “Ot” and the probability “On” output from the output layer 20c, and obtains the values of the equation (1). )-Equation (3) is used to calculate the output value V. The output value calculation unit 230e registers the information of the output value V calculated from the second feature amount in the table 221a in association with the corresponding time.

For example, when the output value calculation unit 230e calculates the output value V ₂ from the second feature amount extracted from the voice information of the time "t ₁ to t ₂ ", the output value calculation unit 230e sets the time as "t ₁ to t ₂ ". , And the output value V2 _are associated with each other and registered in the table 221a.

Each time the output value calculation unit 230e acquires an output control signal from the conversation time management unit 230d, the output value calculation unit 230e repeatedly executes the above processing to sequentially calculate the output value V corresponding to the second feature amount of each time interval. , The calculated output value V information is stored in the table 221a.

The determination unit 230f is a processing unit that determines whether the conversation is in an abnormal conversation situation or a normal conversation situation based on the information of the output value stored in the output value storage buffer 220c. The determination unit 130f calculates each value used in the above-mentioned conditions 1 to 3 and determines whether or not the conversation situation is abnormal.

The process in which the determination unit 230f calculates the average value of each output value from the start time to the current time Tc will be described. The determination unit 230f calculates the average value of each output value stored in the table 221a of FIG. 19 from the start time to the current time Tc.

A process in which the determination unit 230f calculates the minimum value of each output value included from the current time Tc to a predetermined time before will be described. The determination unit 230f extracts a plurality of output values included in the output values stored in the table 221a of FIG. 19 from the current time Tc to a predetermined time before. The determination unit 230f calculates the minimum output value among the extracted plurality of output values as the minimum value.

The process of specifying the output value of the current time Tc by the determination unit 230f will be described. The determination unit 230f specifies the latest output value stored in the table 221b of FIG. 19 as the output value of the current time Tc.

The determination unit 230f calculates each value used in the conditions 1 to 3, and the conversation is in an abnormal conversation situation in "when the condition 2 and the condition 1 are satisfied" or "when the condition 2 and the condition 3 are satisfied". Is determined. The determination unit 230f determines that the conversation is a normal conversation situation when "condition 2 and condition 1 are not satisfied" and "condition 2 and condition 3 are not satisfied". The determination unit 230f may output the determination result to a display device (not shown) and display it, or may notify the external device via the communication unit 210.

Next, an example of the processing procedure of the voice processing apparatus 200 according to the second embodiment will be described. 22 and 23 are flowcharts showing the processing procedure of the voice processing apparatus according to the second embodiment. As shown in FIG. 22, the feature amount calculation unit 230b of the voice processing device 200 executes frame processing and extracts a frame from the voice information (step S201). The feature amount calculation unit 230b extracts the pitch of the frame (step S202) and calculates the power (step S203).

The feature amount calculation unit 230b accumulates pitch and power values (step S204), and proceeds to step S207. On the other hand, the feature amount calculation unit 230b executes voice recognition (step S205), updates the detection number information (step S206), and proceeds to step S207.

The conversation time management unit 230d of the voice processing device 200 determines whether or not it is time to calculate the output value (step S207). If it is not the time to calculate the output value (step S207, No), the conversation time management unit 230d shifts to step S201.

When it is time to calculate the output value (step S207, Yes), the voice processing apparatus 200 calculates the stress evaluation value (step S208), and proceeds to step S209. The output value calculation unit 230e of the voice processing device 200 calculates the output value of the model based on the first feature amount and the second feature amount, stores it in the output value storage buffer 220c (step S209), and steps in FIG. 23. Move to S210.

The description shifts to FIG. 23. The determination unit 230f calculates a value for satisfying the conditions 1 to 3 (step S210). When the conversation is in an abnormal conversation situation (steps S211 and Yes), the determination unit 230f shifts to step S214.

On the other hand, if the conversation is not in an abnormal conversation situation (step S211 and No), the determination unit 230f determines whether or not the conversation has ended (step S212). If the conversation is not completed (step S212, No), the determination unit 230f shifts to step S201 in FIG. 22.

When the conversation is completed (step S212, Yes), the determination unit 230f determines that the conversation is in a normal conversation situation (step S213). The determination unit 230f outputs the determination result (step S214).

Next, the effect of the voice processing device 200 according to the second embodiment will be described. The voice processing device 200 is based on the average value of each output value from the start time to the current time Tc, the minimum value of each output value included from the current time Tc to a predetermined time before, and the output value of the current time Tc. And judge the conversation situation. As a result, in addition to the output value for the feature amount of the voice information from the start time to the current time Tc, the information of the output value for the feature amount of the voice information in the interval of each time interval can be used for the determination. Can be accurately determined.

By the way, the voice processing device 200 for the above-described second embodiment determines the conversation status in real time, but the present invention is not limited to this, and when the conversation ends, offline processing is executed and the conversation status is executed. May be determined. In the following description, the voice processing device 200 that executes offline processing is simply referred to as "voice processing device 200".

For example, when the conversation ends at time Te, the voice processing device 200 executes offline processing to obtain the following three loci (first locus, second locus, and third locus).

FIG. 24A is a diagram for explaining the first locus. The horizontal axis of FIG. 24A is the axis corresponding to the conversation time, and the vertical axis is the axis corresponding to the output value. Similar to the voice processing device 100 of the first embodiment, the voice processing device 200 inputs the feature amount of the voice information in the section from the start time to the set time into the model information 220b and calculates the output value. do. In the example shown in FIG. 24A, the output values 12a to 12r are calculated by the end time Te. The voice processing device 200 specifies the maximum value of the output values 12a to 12r. For example, the maximum value is 12 g. The maximum value of the first locus is referred to as "first maximum value".

FIG. 24B is a diagram for explaining the second locus. The horizontal axis of FIG. 24B is the axis corresponding to the conversation time, and the vertical axis is the axis corresponding to the output value. The voice processing device 200 inputs the feature amount of the voice information (the feature amount of the voice information between the set time before and after) divided for each set time into the model information 220b, and calculates the output value. In the example shown in FIG. 24B, the output values 13a to 13r are calculated by the end time Te. The voice processing device 200 specifies the minimum value of the output values 13a to 13r. For example, the minimum value is 13 m. The minimum value of the second locus is referred to as "second minimum value".

FIG. 24C is a diagram for explaining the third locus. The horizontal axis of FIG. 24C is the axis corresponding to the conversation time, and the vertical axis is the axis corresponding to the output value. In the same manner as in FIG. 24B, the voice processing device 200 inputs the feature amount of voice information (feature amount of voice information between the set time before and after) divided for each set time into the model information 220b, and outputs values 13a to 13r. Is calculated. Then, the voice processing device 200 calculates the average value 14a to 14r of each output value calculated from the start time to the set time for each set time. The average value 14a to 14r is the third locus. For example, the average value 14a corresponds to the output value 13a. The average value 14b is an average value of the output values 13a and 13b. The average value 14c is an average value of the output values 13a to 13c. The average value 14d is an average value of the output values 13a to 13d. The average value 14e is an average value of the output values 13a to 13e.

Similarly, the average value 14f is an average value of the output values 13a to 13f. The average value of 14 g is an average value of output values 13a to 13 g. The average value 14h is an average value of the output values 13a to 13h. The average value 14i is an average value of the output values 13a to 13i. The average value 14j is an average value of the output values 13a to 13j. The average value 14k is an average value of the output values 13a to 13k. The average value 14l is an average value of the output values 13a to 13l. The average value of 14 m is an average value of output values 13a to 13 m. The average value 14n is an average value of the output values 13a to 13n. The average value 14o is an average value of the output values 13a to 13o. The average value 14p is an average value of the output values 13a to 13p. The average value 14q is an average value of the output values 13a to 13q. The average value 14r is an average value of the output values 13a to 13r.

The voice processing device 200 specifies the maximum value of the average value 14a to 14r. For example, the maximum value is 14d. The maximum value of the third locus is referred to as "third maximum value".

The voice processing device 200 determines that the conversation is in an abnormal conversation situation when "when the conditions 5 and 4 are satisfied" or "when the conditions 5 and 6 are satisfied". Th1 to Th3 included in the conditions 4 to 6 are preset threshold values. The magnitude relationship of each threshold value is Th1> Th3> Th2.

Condition 4: Maximum value of the locus (first maximum value) of each output value from the start time to the end time Te> Th1
Condition 5: Minimum value (second minimum value) of each output value included from the start time to the end time Te> Th2
Condition 6: From the start time to the end time Te, the maximum value of the trajectory obtained by averaging the output values up to that point for each time interval (third maximum value)> Th3

The voice processing apparatus 200 according to the second embodiment determines not only the output value for the feature amount of the voice information from the start time to the end time Te, but also the statistical value of the output value for the feature amount of the voice information in each time interval section. Therefore, the conversation status can be accurately determined by determining whether or not the conversation status is abnormal by using the above conditions 4 to 6.

Next, an example of the hardware configuration of the computer that realizes the same functions as the voice processing devices 100 and 200 shown in the above embodiment will be described. FIG. 25 is a diagram showing an example of a hardware configuration of a computer that realizes a function similar to that of a voice processing device.

As shown in FIG. 25, the computer 300 has a CPU 301 for executing various arithmetic processes, an input device 302 for receiving data input from a user, and a display 303. Further, the computer 300 has a reading device 304 that reads a program or the like from a storage medium, and an interface device 305 that exchanges data with another computer via a wired or wireless network. For example, the interface device 305 is connected to a communication device or the like. Further, the computer 300 has a RAM 306 for temporarily storing various information and a hard disk device 307. Then, each of the devices 301 to 307 is connected to the bus 308.

The hard disk device 307 reads the acquisition program 307a, the feature amount calculation program 307b, the model learning program 307c, the conversation time management program 307d, the output value calculation program 307e, and the determination program 307f and deploys them in the RAM 306.

The acquisition program 307a functions as the acquisition process 306a. The feature amount calculation program 307b functions as a feature amount calculation process 306b. The model learning program 307c functions as a model learning process 306c. The conversation time management program 307d functions as a conversation time management process 306d. The output value calculation program 307e functions as an output value calculation process 306e. The determination program 307f functions as the determination process 306f.

The processing of the acquisition process 306a corresponds to the acquisition units 130a and 230a. The processing of the feature amount calculation process 306b corresponds to the feature amount calculation units 130b and 230b. The processing of the model learning process 306c corresponds to the model learning units 130c and 230c. The processing of the conversation time management process 306d corresponds to the conversation time management units 130d and 230d. The process of the output value calculation process 306e corresponds to the output value calculation units 130e and 230e. The processing of the determination process 306f corresponds to the determination units 130f and 230f.

The programs 307a to 307f do not necessarily have to be stored in the hard disk device 307 from the beginning. For example, each program is stored in a "portable physical medium" such as a flexible disk (FD), a CD-ROM, a DVD disk, a magneto-optical disk, or an IC card inserted into a computer 300. Then, the computer 300 may read and execute each program 307a to 307f.

The following additional notes will be further disclosed with respect to the embodiments including each of the above embodiments.

(Appendix 1) Multiple features from a plurality of voice information from the start time to each set time based on a set time set at predetermined time intervals from the start time of the conversation to be judged included in the voice information. Calculate the amount,
By inputting a plurality of features calculated for each set time into a model generated based on the features of voice information from the start time to the end time of the conversation, the said above corresponding to the plurality of features. Calculate multiple output values of the model for each set time,
A voice processing program characterized by causing a computer to execute a process of determining whether or not the conversation to be determined is an abnormal conversation situation based on the plurality of output values.

(Appendix 2) In the determination process, the range that can be taken by the loci of the plurality of output values is set to an abnormal area that is taken when the conversation situation is abnormal and a normal area that is taken when the conversation situation is normal. Additional note that it is divided and it is determined whether or not the conversation to be determined is an abnormal conversation situation based on the loci of the plurality of output values and the abnormal region and the normal region. The voice processing program according to 1.

(Appendix 3) In the determination process, the abnormal region is divided into a first region and a second region corresponding to a region having a larger output value than the first region, and one of the loci of the plurality of output values. When the unit is included in the second region, or when the entire locus of the plurality of output values is included in the first region, it is determined that the conversation to be determined is an abnormal conversation situation. The voice processing program described in Appendix 2 as a feature.

(Appendix 4) In the determination process, the normal region is divided into a third region and a fourth region corresponding to a region having a smaller output value than the third region, and one of the loci of the plurality of output values. The voice processing program according to Appendix 2 or 3, wherein when the unit is included in the fourth area, it is determined that the conversation to be determined is in a normal conversation situation.

(Appendix 5) In the determination process, it is determined whether or not the conversation to be determined is in an abnormal conversation situation based on the order in which the locus of the output value passes through the normal region or the abnormal region. The voice processing program according to Appendix 2, wherein the voice processing program is described.

(Appendix 6) The start time is set to any one of the appendices 1 to 5, characterized in that the start time is a predetermined time after the time when the start of the conversation to be determined included in the voice information is detected. The described voice processing program.

(Appendix 7) In the process of calculating the feature amount, the voice information is divided into the predetermined time intervals, and a plurality of feature amounts are further calculated from the divided plurality of divided voice information.
In the process of calculating the output value, a plurality of output values are further calculated by inputting a plurality of feature amounts calculated from the plurality of divided voice information into the model.
In the determination process, the average value of a plurality of output values obtained from the feature quantities of the plurality of divided voice information from the start time to the current time up to the current time, and the time before a predetermined time from the current time are described. The determination target is based on the minimum value of a plurality of output values obtained from the feature quantities of a plurality of divided voice information up to the current time and the output value obtained from the feature quantities of the voice information from the start time to the current time. The voice processing program according to Appendix 1, wherein it is determined whether or not the conversation is in an abnormal conversation situation.

(Appendix 8) In the process of calculating the feature amount, the voice information is divided into the predetermined time intervals, and a plurality of feature amounts are calculated from the divided plurality of divided voice information.
In the process of calculating the output value, a plurality of output values are calculated by inputting a plurality of feature amounts calculated from the plurality of divided voice information into the model.
The determination process includes the maximum value of the locus obtained by calculating the average value from the start time to the set time for each of the plurality of output values obtained from the feature quantities of the plurality of divided voice information, and a plurality of loci. The determination is based on the minimum value of the plurality of output values obtained from the feature amount of the divided voice information and the maximum value of each output value obtained from the feature amount of the voice information from the start time to each set time. The voice processing program according to Appendix 1, wherein it is determined whether or not the target conversation is in an abnormal conversation situation.

(Appendix 9) A voice processing method executed by a computer.
Based on the set time set at predetermined time intervals from the start time of the conversation to be judged included in the voice information, a plurality of feature quantities are calculated from the plurality of voice information from the start time to each set time. ,
By inputting a plurality of features calculated for each set time into a model generated based on the features of voice information from the start time to the end time of the conversation, the said above corresponding to the plurality of features. Calculate multiple output values of the model for each set time,
A voice processing method comprising executing a process of determining whether or not the conversation to be determined is an abnormal conversation situation based on the plurality of output values.

(Appendix 10) In the determination process, the range that can be taken by the loci of the plurality of output values is set to an abnormal area that is taken when the conversation situation is abnormal and a normal area that is taken when the conversation situation is normal. Additional note that it is divided and it is determined whether or not the conversation to be determined is an abnormal conversation situation based on the loci of the plurality of output values and the abnormal region and the normal region. 9. The voice processing method according to 9.

(Appendix 11) In the determination process, the abnormal region is divided into a first region and a second region corresponding to a region having a larger output value than the first region, and one of the loci of the plurality of output values. When the unit is included in the second region, or when the entire locus of the plurality of output values is included in the first region, it is determined that the conversation to be determined is an abnormal conversation situation. The voice processing method according to Appendix 10, which is a feature.

(Appendix 12) In the determination process, the normal region is divided into a third region and a fourth region corresponding to a region having a smaller output value than the third region, and one of the loci of the plurality of output values. The voice processing method according to Appendix 10 or 11, wherein when the unit is included in the fourth region, it is determined that the conversation to be determined is in a normal conversation situation.

(Appendix 13) In the determination process, it is determined whether or not the conversation to be determined is in an abnormal conversation situation based on the order in which the locus of the output value passes through the normal region or the abnormal region. The voice processing method according to Appendix 10, wherein the voice processing method is performed.

(Appendix 14) The start time is set to any one of the appendices 9 to 13, characterized in that the start time is a predetermined time after the start of the conversation to be determined, which is included in the voice information, is detected. The described voice processing method.

(Appendix 15) In the process of calculating the feature amount, the voice information is divided into the predetermined time intervals, and a plurality of feature amounts are further calculated from the divided plurality of divided voice information.
In the process of calculating the output value, a plurality of output values are further calculated by inputting a plurality of feature amounts calculated from the plurality of divided voice information into the model.
In the determination process, the average value of a plurality of output values obtained from the feature quantities of the plurality of divided voice information from the start time to the current time up to the current time, and the time before a predetermined time from the current time are described. The determination target is based on the minimum value of a plurality of output values obtained from the feature quantities of a plurality of divided voice information up to the current time and the output value obtained from the feature quantities of the voice information from the start time to the current time. The voice processing method according to Appendix 9, wherein it is determined whether or not the conversation is in an abnormal conversation situation.

(Appendix 16) In the process of calculating the feature amount, the voice information is divided into the predetermined time intervals, and a plurality of feature amounts are calculated from the divided plurality of divided voice information.
In the process of calculating the output value, a plurality of output values are calculated by inputting a plurality of feature amounts calculated from the plurality of divided voice information into the model.
The determination process includes the maximum value of the locus obtained by calculating the average value from the start time to the set time for each of the plurality of output values obtained from the feature quantities of the plurality of divided voice information, and a plurality of loci. The determination is based on the minimum value of the plurality of output values obtained from the feature amount of the divided voice information and the maximum value of each output value obtained from the feature amount of the voice information from the start time to each set time. The voice processing method according to Appendix 9, wherein it is determined whether or not the target conversation is in an abnormal conversation situation.

(Appendix 17) Multiple features from a plurality of voice information from the start time to each set time based on a set time set at predetermined time intervals from the start time of the conversation to be determined included in the voice information. A feature amount calculation unit that calculates the amount, and
By inputting a plurality of features calculated for each set time into a model generated based on the features of voice information from the start time to the end time of the conversation, the said above corresponding to the plurality of features. An output value calculation unit that calculates multiple output values of the model for each set time,
A voice processing device comprising a determination unit for determining whether or not the conversation to be determined is an abnormal conversation situation based on the plurality of output values.

(Appendix 18) The determination unit divides the range that the loci of the plurality of output values can take into an abnormal area that is taken when the conversation situation is abnormal and a normal area that is taken when the conversation situation is normal. The appendix 17 is characterized in that it is determined whether or not the conversation to be determined is an abnormal conversation situation based on the loci of the plurality of output values and the abnormal region and the normal region. The voice processing device described in.

(Appendix 19) The determination unit divides the abnormal region into a first region and a second region corresponding to a region having a larger output value than the first region, and a part of the loci of the plurality of output values. Is included in the second region, or when the entire locus of the plurality of output values is included in the first region, it is determined that the conversation to be determined is an abnormal conversation situation. The voice processing apparatus according to Appendix 18.

(Appendix 20) The determination unit divides the normal region into a third region and a fourth region corresponding to a region having a smaller output value than the third region, and a part of the loci of the plurality of output values. The voice processing apparatus according to Supplementary note 18 or 19, wherein when is included in the fourth region, it is determined that the conversation to be determined is in a normal conversation situation.

100,200 Audio processing device 110, 210 Communication unit 120, 220 Storage unit 120a, 220a Voice buffer 120b, 220b Model information 120c, 220c Output value storage buffer 130, 230 Control unit 130a, 230a Acquisition unit 130b, 230b Feature amount calculation unit 130c, 230c Model learning unit 130d, 230d Conversation time management unit 130e, 230e Output value calculation unit 130f, 230f Judgment unit

Claims (10)

Based on the set time set at predetermined time intervals from the start time of the conversation to be judged included in the voice information , a plurality of feature quantities are applied to a plurality of voice information from the start time to each set time. Calculate and
By inputting a plurality of features calculated for each set time into a model generated based on the features of voice information from the start time to the end time of the conversation, the said above corresponding to the plurality of features. Calculate multiple output values of the model for each set time,
A voice processing program characterized by causing a computer to execute a process of determining whether or not the conversation to be determined is an abnormal conversation situation based on the plurality of output values. The determination process divides the range that can be taken by the loci of the plurality of output values into an abnormal area to be taken when the conversation situation is abnormal and a normal area to be taken when the conversation situation is normal. The first aspect of the present invention is to determine whether or not the conversation to be determined is an abnormal conversation situation based on the loci of a plurality of output values and the abnormal region and the normal region. Voice processing program. In the determination process, the abnormal region is divided into a first region and a second region corresponding to a region having a larger output value than the first region, and a part of the loci of the plurality of output values is the first region. A claim characterized in that it is determined that the conversation to be determined is an abnormal conversation situation when it is included in two regions or when the entire locus of the plurality of output values is included in the first region. Item 2. The voice processing program according to item 2. In the determination process, the normal region is divided into a third region and a fourth region corresponding to a region having a smaller output value than the third region, and a part of the loci of the plurality of output values is the first region. The voice processing program according to claim 2 or 3, wherein when it is included in the four areas, it is determined that the conversation to be determined is in a normal conversation situation. The determination process is characterized in that it is determined whether or not the conversation to be determined is in an abnormal conversation situation based on the order in which the locus of the output value passes through the normal region or the abnormal region. The voice processing program according to claim 2. The voice according to any one of claims 1 to 5, wherein the start time is a predetermined time after the start of the conversation to be determined, which is included in the voice information, is detected. Processing program. In the process of calculating the feature amount, the voice information is divided into the predetermined time intervals, and a plurality of feature amounts are further calculated from the divided plurality of divided voice information.
In the process of calculating the output value, a plurality of output values are further calculated by inputting a plurality of feature amounts calculated from the plurality of divided voice information into the model.
In the determination process, the average value of a plurality of output values obtained from the feature quantities of the plurality of divided voice information from the start time to the current time up to the current time, and the time before a predetermined time from the current time are described. The determination is based on the minimum value of a plurality of output values obtained from the feature quantities of a plurality of divided voice information up to the current time and the output value obtained from the feature quantities of the voice information from the start time to the current time. The voice processing program according to claim 1, wherein it is determined whether or not the target conversation is in an abnormal conversation situation. In the process of calculating the feature amount, the voice information is divided into the predetermined time intervals, and a plurality of feature amounts are calculated from the divided plurality of divided voice information.
In the process of calculating the output value, a plurality of output values are calculated by inputting a plurality of feature amounts calculated from the plurality of divided voice information into the model.
The determination process includes the maximum value of the locus obtained by calculating the average value from the start time to the set time for each of the plurality of output values obtained from the feature quantities of the plurality of divided voice information and a plurality of loci. The determination is based on the minimum value of the plurality of output values obtained from the feature amount of the divided voice information and the maximum value of each output value obtained from the feature amount of the voice information from the start time to each set time. The voice processing program according to claim 1, wherein it is determined whether or not the target conversation is in an abnormal conversation situation. It is a voice processing method executed by a computer.
Based on the set time set at predetermined time intervals from the start time of the conversation to be judged included in the voice information , a plurality of feature quantities are applied to a plurality of voice information from the start time to each set time. Calculate and
By inputting a plurality of features calculated for each set time into a model generated based on the features of voice information from the start time to the end time of the conversation, the said above corresponding to the plurality of features. Calculate multiple output values of the model for each set time,
A voice processing method comprising executing a process of determining whether or not the conversation to be determined is an abnormal conversation situation based on the plurality of output values. Based on the set time set at predetermined time intervals from the start time of the conversation to be judged included in the voice information , a plurality of feature quantities are applied to a plurality of voice information from the start time to each set time. The feature amount calculation unit to be calculated and
By inputting a plurality of features calculated for each set time into a model generated based on the features of voice information from the start time to the end time of the conversation, the said above corresponding to the plurality of features. An output value calculation unit that calculates multiple output values of the model for each set time,
A voice processing device comprising a determination unit for determining whether or not the conversation to be determined is an abnormal conversation situation based on the plurality of output values.

Images