JP2021124530A - Information processor, information processing method and program - Google Patents

Abstract

To understand an utterance content and emotion at that time when voice uttered by a speaker is voice-recognized.SOLUTION: A text data acquisition part 33 of a computer 3 voice-recognizes voice data of speaker's utterance, and acquires text data indicating a content of the utterance on the basis of a result of voice recognition. An acoustic data acquisition part 34 acquires voice data of the speaker's utterance as acoustic data. An emotion recognition part 36 recognizes speaker's emotion on the basis of the acoustic data and the text data, and acquires emotion information indicating the recognition result.SELECTED DRAWING: Figure 3

Description

The present invention relates to an information processing device, an information processing method and a program.

In recent years, in the telephone reception counter business of customer centers, etc., voice recognition technology is used to separate the voice data of calls exchanged between the customer and the operator of the telephone counter for voice recognition, and the amount of information is in a compact character format. It is stored as data, for example, text data. The saved text data will be used later, for example, as a correspondence history.

By the way, when the customer's response history is confirmed with the stored text data, it is often difficult to grasp what kind of emotion the customer was talking about when responding to the phone call from the text data. For example, if you are angry but use polite language, it will not appear in the text data.

Japanese Unexamined Patent Publication No. 2019-153961

In this way, when the voice uttered by a person is voice-recognized and converted into text data, the content of the utterance itself can be confirmed from the text data, but the emotion of the person who uttered may not be transmitted.

The present invention has been made in view of such a situation, and an object of the present invention is to be able to understand the content of the utterance and the emotion at that time in recognizing the voice uttered by a person.

In order to achieve the above object, the information processing device of one aspect of the present invention is
A text acquisition means that recognizes the voice data of the speaker's utterance and acquires text data indicating the content of the utterance based on the result of the voice recognition.
An acoustic acquisition means for acquiring the voice data of the utterance of the speaker as acoustic data,
An emotion recognition means that recognizes the emotion of the speaker based on the acoustic data and the text data and obtains emotion information indicating the recognition result.
Information processing device equipped with.

The information processing method and program corresponding to the information processing apparatus according to one aspect of the present invention are also provided as the information processing method and program according to one aspect of the present invention.

According to the present invention, in recognizing a voice spoken by a speaker, the content of the utterance and the emotion of the speaker at that time can be understood.

It is a figure which shows the example of the structure of the information processing system including the computer which concerns on one Embodiment of the information processing apparatus of this invention. It is a block diagram which shows an example of the hardware composition of the computer which concerns on the information processing apparatus of this invention among the information processing system of FIG. FIG. 5 is a functional block diagram showing an example of the functional configuration of the computer of FIG. 2 in the information processing system of FIG. 1 of the embodiment. It is a figure which shows an example of the feature quantity used for constructing an emotion recognition model. It is a figure which shows an example of the emotion recognition means. It is a figure which shows the detailed structure (an example of a hidden layer 52a) of the emotion recognition model of FIG. It is a figure which shows the detailed structure (an example of a hidden layer 52b) of the emotion recognition model of FIG. It is a flowchart which shows the operation of the computer of FIG. 1 and FIG. It is a figure for demonstrating the evaluation method of emotion recognition accuracy.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a diagram showing a configuration of an information processing system including a computer according to an embodiment of the information processing device of the present invention.

The information processing system shown in FIG. 1 is configured by connecting a telephone 1, a headset 2, a computer 3, and a learning device 4 to each other via a network N. This information processing system can provide an emotion recognition service based on the voice spoken by the speaker.

The telephone device 1 has a call function including a microphone and a speaker, and a user U who makes an inquiry calls the inquiry center, and an operator OP and a voice wearing a headset 2 connected via an exchange of the inquiry center. It is a communication device that communicates with each other, that is, makes a call. The telephone device 1 receives the voice (analog signal) spoken by the user U.

The headset 2 has a call function and a recording function including a microphone and a speaker, and is a call device for the operator OP to make a call with the user U who has received an inquiry. The headset 2 receives the voice data (analog signal) spoken by the operator OP, records each voice data, and sends the voice data to the computer 3 through the network N.

The computer 3 functions as a voice recognition device that recognizes the voice of at least one of the user U and the operator OP received from the headset 2 and outputs emotional information regarding the text data of the voice recognition result. As the emotion information, an emotion label that classifies emotions so as to include at least emotion types such as "joy", "anger", "sorrow", and "normal" is output. The details of the functional configuration and processing of the computer 3 will be described later with reference to the drawings of FIGS. 3 and later.

The learning device 4 performs machine learning and models a plurality of voice data classified for each emotion type prepared in advance. The learning device 4 generates a learning model by machine learning one or more (many) voice data for learning, and stores it in the storage unit 18 of the computer 3.

For example, a neural network or the like can be applied to the learning model. The neural network is only an example, and other machine learning methods may be applied. Furthermore, the learning model is not limited to the machine learning model, and a determination device that makes a determination by a predetermined algorithm may be adopted.

FIG. 2 is a block diagram showing an example of a computer hardware configuration according to a first embodiment of the information processing apparatus of the present invention in the information processing system of FIG.

The computer 3 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a bus 14, an input / output interface 15, an output unit 16, and an input unit 17. A storage unit 18, a communication unit 19, and a drive 20 are provided.

The CPU 11 executes various processes according to the program recorded in the ROM 12 or the program loaded from the storage unit 18 into the RAM 13.
Data and the like necessary for the CPU 11 to execute various processes are appropriately stored in the RAM 13.

The CPU 11, ROM 12 and RAM 13 are connected to each other via the bus 14. An input / output interface 15 is also connected to the bus 14. An output unit 16, an input unit 17, a storage unit 18, a communication unit 19, and a drive 20 are connected to the input / output interface 15.
The output unit 16 is composed of a display, a speaker, a printer, and the like, and outputs output information such as voice data and text data. If the output unit 16 is, for example, a printer or the like, the output information can be printed.
The input unit 17 is composed of a keyboard, a mouse, and the like, and inputs various information according to a user's instruction operation. The storage unit 18 is composed of a hard disk device or the like, and stores various information data.

The communication unit 19 communicates with another target (for example, the headset 2 or the learning device 4 in FIG. 1) via the network N.
A removable media 21 made of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is appropriately mounted on the drive 20. The program read from the removable media 21 by the drive 20 is installed in the storage unit 18 as needed. Further, the removable media 21 can also store various data stored in the storage unit 18 in the same manner as the storage unit 18.

Although not shown, the learning device 4 of the information processing system of FIG. 1 has basically the same configuration as the hardware configuration of the computer 3 shown in FIG. Therefore, the description of the hardware configuration of the learning device 4 will be omitted.
Further, for convenience of explanation, the computer 3 is provided separately from the learning device 4, but the present invention is not particularly limited to this, and the functions of the learning device 4 and the computer 3 are integrated into one information processing device. May be good.

FIG. 3 is a functional block diagram showing an example of the functional configuration of the computer 3 of FIG. 2 in the information processing system of FIG. 1 of the embodiment.

As shown in FIG. 3, the CPU 11 of the computer 3 functions as a conversion unit 31, a voice recognition model 32, a text data acquisition unit 33, an acoustic data acquisition unit 34, an emotion recognition model 35, an emotion recognition unit 36, and the like. Further, the voice recognition model 32, the emotion recognition model 35, and the like are stored in the storage unit 18 of the computer 3.

The conversion unit 31 converts the speaker's spoken voice input to the computer 3, that is, analog voice data, into digital voice data. When the voice received from the headset 2 is the voice data mixed in the call of the operator OP and the user U, the conversion unit 31 separates the voice data spoken by each speaker and determines which of the emotion recognition targets. Either one or each is output separately.
By setting in advance to output one voice (for example, only the voice of the user U), it is possible to output only the voice data of the user U to the subsequent stage.
In FIG. 3, the two voice data output from the conversion unit 31 are voice data input by the same speaker in the same time series.

The voice recognition model 32 is a trained model constructed so that the learning device 4 learns voice data in word units in advance and outputs text data corresponding to the input voice data as a recognition result. That is, the voice recognition model 32 may be generated by learning to output text data corresponding to the voice data when the voice data is input.

The text data acquisition unit 33 voice-recognizes the voice data of the speaker's utterance, and acquires text data indicating the content of the utterance based on the result of the voice recognition. Specifically, the text data acquisition unit 33 is composed of words and phrases output as a result of voice recognition in the voice recognition model 32 by inputting voice data of the speaker's utterance into the voice recognition model 32 of the storage unit 18. The text data is concatenated to acquire the text data indicating the content of the utterance.

The acoustic data acquisition unit 34 acquires the voice data of the speaker's utterance as acoustic data. Specifically, noise and the like unnecessary for voice recognition included in the voice data of the received speaker's utterance are removed and acquired as acoustic data.

The emotion recognition model 35 is an emotion-learned model constructed by learning a plurality of voice data (hereinafter referred to as “learning data”) classified in advance for each emotion type and given correct emotion labels. be. FIG. 4 shows the features used for constructing the emotion recognition model 35. The emotion recognition model 35 was learned using the average of the features shown in FIG. 4 and the standard deviation value.

The emotion recognition unit 36 recognizes the emotion of the speaker based on the input acoustic data and the text data, and obtains emotion information indicating the recognition result.

More specifically, the emotion recognition unit 36 uses the emotion recognition model 35 to describe the emotions of the speaker based on the acoustic data acquired by the acoustic data acquisition unit 34 and the text data acquired by the text data acquisition unit 33. Is recognized, and emotional information indicating the recognition result is output.

Here, the emotion recognition unit 36 decomposes the text data into morphemes by morphological analysis, converts the decomposed morphemes into hiragana based on preset word determination conditions, and converts the converted hiragana into hiragana and a feature vector in advance. Using the text-learned model in which the relationship is learned, it is converted into a feature vector which is a second feature value.

In addition, the emotion recognition unit 36 extracts a feature vector, which is the first feature value, from the acoustic data. Then, the emotion recognition unit 36 recognizes the emotion of the speaker based on the feature vector extracted from the text data and the feature vector extracted from the acoustic data.

The emotion recognition unit 36 stores the emotion information in the storage unit 18 in association with the text data of the voice recognition result. Emotion information may be given to the entire text data, may be given to each sentence break (phrase), or may be given to each word or a certain character string unit.

As emotional information, the emotion recognition unit 36 expresses the emotions of the speaker, for example, "joy" indicating human joy, "anger" indicating anger, "sadness" indicating sadness, "normal" indicating normal heart, and the like. An emotion label is output, which is classified into one and classified so as to include at least these four emotion types. As the emotion information, in addition to the emotion label, for example, a probability value indicating the accuracy of the classified emotion label may be output in association with the emotion label.

Here, an emotion recognition means composed of an emotion recognition model and an emotion recognition unit will be described with reference to FIGS. 5 to 7. 5 is a diagram showing an emotion recognition means, FIG. 6 is a diagram showing an example of the hidden layer 52a of the emotion recognition model of FIG. 5, and FIG. 7 is a diagram of the hidden layer 52b of the emotion recognition model of FIG. It is a figure which shows an example.

The emotion recognition means is composed of the emotion recognition model 35 and the emotion recognition unit 36 shown in FIG. The emotion recognition model 35 has a hidden layer 52a, a hidden layer 52b, a weighting means 53 (composed of a data coupling circuit, a dimension number adjusting circuit, and the like), and an output layer 54 to which the Softmax function is applied. The data coupling circuit sequentially couples the data of the calculation results of the hidden layer 52a and the hidden layer 52b. The dimension adjustment circuit applies a SOFTMAX function to the data combined by the data coupling circuit to generate 8-dimensional vector data.
As shown in FIG. 5, the emotion recognition unit 36 includes a feature vector extraction unit 51a for processing acoustic data and a feature vector extraction unit 51b for processing text data.
The feature vector extraction unit 51a extracts the feature vector, which is the first feature value, from the input acoustic data.
At this time, the feature vector extraction unit 51a divides the input acoustic data into fine voices such as voice 1, voice 2, ... voice N, etc., which are units for performing voice section detection (VAD), and each voice m. For, feature values are extracted using Mel Frequency Cepstrum Coefficients (MFCC: Mel Frequency Cepstrum Cofficients) and the like, and as a result, a 21-dimensional feature vector is obtained.
The feature vector extraction unit 51a further multiplies the feature vector for each dimension of the 21-dimensional feature vector by 15 types of functions (average, maximum, minimum, etc.) and outputs the feature vector to the hidden layer 52a of the emotion recognition model 35. The value of the function takes one of the output average, maximum, and minimum of voice 1 to voice N. As a result, a 21 × 15 dimensional feature vector is obtained.

The feature vector extraction unit 51b extracts a feature vector, which is a second feature value, from the input text data.
More specifically, the feature vector extraction unit 51b decomposes the text data into morphemes by morphological analysis, and refers to the Japanese evaluation polarity dictionary or the Japanese word emotion polarity correspondence table for the decomposed morphemes, and refers to the polarity of the morphemes. Is converted and the morpheme is decomposed.
Then, the feature vector extraction unit 51b converts the decomposed morphemes into hiragana (or keeps the kanji of the morphemes) based on preset word determination conditions, and converts the converted hiragana (character data) into hiragana (character data). Convert to a feature vector using a model that has been text-learned in advance.
Further, the feature vector extraction unit 51b adds the polarity of the morpheme to the converted feature vector to generate a second feature value, and inputs the generated second feature value to the hidden layer 52b of the emotion recognition model 35.
The text-learned model is obtained by learning the correct answer data in which the hiragana and the feature vector are related in advance by the learning device 4.

As shown in FIG. 5, the hidden layer 52a of the emotion recognition model 35 is composed of four layers of LSTM (Long Short-Term Memory). The number of nodes in each layer is 256-512-512-256, the number of epoches is 5000, the batch size is 500, and the dropout is 0.7. The input is, for example, 252 feature quantities (feature vectors). That is, a 252 dimensional feature vector extracted from the acoustic data is input to the hidden layer 52a.
The hidden layer 52a performs a 4-layer LSTM calculation using the input 252 feature vectors, and then inputs the hidden layer 52a to the data coupling circuit of the weighting means 53.
The hidden layer 52b of the emotion recognition model 35 is composed of two layers of BILSTM. The number of nodes in each layer is 1024-256, the number of epoches is 5000, and the maximum patch is 250. The input is, for example, 770 features. That is, 770 feature quantities (feature vectors) extracted from the text data are input to the hidden layer 52b.
The hidden layer 52b performs a Bi-direction-RNN calculation using the input 770 feature vectors, and then inputs the hidden layer 52b to the data coupling circuit of the weighting means 53.
The weighting means 53 weights the outputs of the hidden layer 52a and the hidden layer 52b, and then outputs the weight to the output layer 54 to which the Softmax function is applied.

The output layer 54 inputs the feature vector weighted by the weighting means 53 into the Softmax function, and based on the value obtained from the Softmax function, indicates “joy” indicating human joy, “anger” indicating anger, and sadness. An emotion label Y classified into any of four types of emotions, "sadness" and "normal" indicating normal feelings, is output. The value obtained from the Softmax function is a numerical value in which the sum of each component is 1 in the range of 0 or more and 1 or less. A probability value indicating the accuracy of the emotion label Y may be assigned and output simultaneously with or separately from the emotion label Y.

As shown in FIG. 6, the hidden layer 52a processes the frame-based 21-dimensional feature vectors X1 to X30 in order, and outputs an emotion label Y classified into any of the above four types of emotions from the output layer 54. It is configured to output. Among the marks shown in FIG. 6, the square indicates the calculation, the white circle indicates the input gate, and the circle with a broken line indicates the tanh hyperbolic tangent function.
As shown in FIG. 7, the hidden layer 52b has a Bi-directional RNN in which a plurality of recurrent neural networks (hereinafter referred to as “RNN”) are interconnected, and an integrated portion Σ. The hidden layer 52b is configured such that the Bi-directional RNN learns information from the future in addition to learning information from the past, and the learning results are integrated by the integration unit Σ.

Next, the voice recognition process executed by the computer 3 will be described with reference to FIG. FIG. 8 is a flowchart illustrating an example of a flow of emotion recognition processing executed by the computer 3 having the functional configuration of FIG.
In the case of this information processing system, the computer 3 executes an emotion recognition process for recognizing emotions from the voices of speakers such as the operator OP and the user U as follows.

When the speaker's voice data recorded in the headset 2 is received by the computer 3 through the network N, the text data acquisition unit 33 of the computer 3 uses the voice recognition model 32 in step S11 to display the speaker's voice data. The voice data of the utterance is recognized by voice, and the text data indicating the content of the utterance is acquired based on the result of the voice recognition.

Further, in step S12, the acoustic data acquisition unit 34 acquires the voice data of the speaker's utterance as acoustic data.

Then, in step S13, the emotion recognition unit 36 uses the emotion recognition model 35, and based on the acoustic data acquired by the acoustic data acquisition unit 34 and the text data acquired by the text data acquisition unit 33, the speaker Recognize emotions and obtain emotional information indicating the recognition result.
As the emotion information, an emotion label indicating the speaker's emotions, for example, one of "joy", "anger", "sorrow", and "normal", and a probability value indicating the probability thereof can be obtained. The obtained emotion information is stored in the storage unit 18 in correspondence with the text data of the voice recognition result.

By reading out the pair of emotion information and text data stored in the storage unit 18 as necessary and displaying them side by side on the screen of the computer 3, an emotion label is given to the text data at the timing when the speaker utters. Therefore, it becomes clear that the speaker was angry when inquiring with the expression of this text.

Here, an evaluation method of emotion recognition accuracy will be described with reference to FIG. FIG. 9 is a diagram for explaining a method of evaluating emotion recognition accuracy.

As shown in FIG. 9, 10-fold cross-validation (10-fold cv) is used to evaluate the emotion recognition accuracy.

Cross-validation is a method for verifying and confirming how much data analysis can really deal with the population. In this verification, sample data is divided in statistics, a part of it is analyzed first, and the remaining part is tested for the analysis, and the validity of the analysis itself is verified and confirmed.

Especially when it is difficult to collect more samples, it is necessary to carefully confirm the support by performing cross-validation etc. in order to estimate from the data and derive the verification result.
For example, in k1 partition cross-validation, the sample group is divided into k pieces. Then, it is common to use one of them as a test case and the remaining k1 as a training case. In cross-validation, each sample group divided into k pieces is used as a test case and verified k times. One estimation result is obtained by averaging the results of k times obtained in this way. This time, we will proceed with verification by 10-fold cross-validation with k = 10.

For verification, the correlation table 81 shown in FIG. 9 is used, and the emotion recognition accuracy can be evaluated by obtaining the precision rate P (precision), the recall rate R (recall), and the F value (Fmease). Correlation table 81 is a table showing the relationship between the predicted result and the true result.
In the correlation table 81 shown in FIG. 9, when the prediction result is "positive" and the true result is "positive", TP is used, when the prediction result is "positive" and the true result is "negative", FP is used, and when the prediction result is "negative". If the true result is "positive", it is FN, and if the predicted result is "negative" and the true result is "negative", it is TN.

Here, the precision ratio P refers to the ratio of the data predicted to be “positive” that is actually “positive”, and can be expressed by P = TP / (TP + FP).
The recall rate R refers to the ratio of those that are actually “positive” and those that are predicted to be “positive”, and can be expressed by R = TP / (TP + FN).
The F value is a harmonic mean of the precision rate P and the recall rate R, and can be expressed by F = (2 * R * P) / (R + P).
The precision P and the recall R have a trade-off relationship with each other, and a high F value indicates that both the precision and the recall are well-balanced and high.

As described above, according to the present embodiment, the text data acquisition unit 33 voice-recognizes the voice data of the speaker's speech, and acquires the text data indicating the content of the speech based on the result of the voice recognition. The acoustic data acquisition unit 34 acquires the voice data of the speaker's speech as acoustic data, and the emotion recognition unit 36 recognizes the speaker's emotions based on the acoustic data and the text data, and obtains the recognition result. Since the emotional information to be shown (for example, emotional labels classified into "joy", "anger", "sorrow", "normal", etc.) is obtained, the voice spoken by a person is obtained from the text data of the recognition result and the emotional information classified into several categories. You will be able to understand the content of the utterance and the feelings of the speaker at that time.

In the above embodiment, the learning model is stored in the computer 3, but it may be stored in the learning device 4 or in another computer (server or the like) connected to the network N.

Further, in the above embodiment, the conversation exchanged between the operator OP and the user U who makes an inquiry is taken as an example, but in addition to this, for example, a caregiver working in a care facility and a user receiving care by the caregiver Since there is a problem of communication such as the caregiver cannot understand the meaning of the words spoken by the elderly (for example, the elderly), this patent can be applied to understand the feelings of the speaker rather than the words themselves. ..

In other words, the information processing apparatus to which the present invention is applied can take various embodiments having the following configurations.
That is, the information processing device to which the present invention is applied (for example, the computer 3 as shown in FIG. 3) recognizes the voice data of the speaker's speech, and based on the result of the voice recognition, text data indicating the content of the speech. (For example, the voice recognition model 32 and the text data acquisition unit 33 as shown in FIG. 3) and the text data acquisition unit 33.
An acoustic data acquisition means (for example, an acoustic data acquisition unit 34 in FIG. 3 or the like) for acquiring the voice data of the utterance of the speaker as acoustic data, and
An emotion recognition means (for example, an emotion recognition model 35 and an emotion recognition unit 36 in FIG. 3 or the like) that recognizes the emotion of the speaker based on the acoustic data and the text data and obtains emotion information indicating the recognition result.
To be equipped.
As a result, from the text data of the recognition result and the emotional information, the utterance content of the voice uttered by the speaker and the emotion at that time can be understood.
The emotion recognition means (for example, the emotion recognition model 35 and the emotion recognition unit 36 in FIG. 3 and the like) is
As the emotional information, at least one of an emotional label that classifies the speaker's emotions so as to include at least "joy", "anger", "sorrow", and "normal" and a probability value indicating the accuracy of the classified emotional label. To output,
As a result, it can be seen that although the speaker's voice data expresses politely, the speaker actually expresses something contrary to the words, such as being angry.
The emotion recognition means
The speaker's emotion is recognized by using the first feature value (feature vector) extracted from the acoustic data and the second feature value (feature vector) extracted from the text data.
As a result, the speaker's emotions are recognized from the acoustic data and the text data, so that the emotion recognition accuracy can be improved.
The emotion recognition means
The text data is decomposed into morphemes by morphological analysis.
Converts the decomposed morphemes into hiragana based on preset word judgment conditions.
A feature value extraction means (for example, a feature vector extraction unit 51b in FIG. 5 or the like) that converts the converted hiragana into the second feature value using a model in which text has been learned in advance.
Further prepare,
As a result, the expression of the emotion obtained from the text data can be added to the emotion obtained from the acoustic data, so that the recognition accuracy of the emotion can be improved.

The series of processes described above can be executed by hardware or software.
In other words, the functional configuration of FIG. 3 is only an example and is not particularly limited.
That is, it suffices if the information processing system or the information processing device is provided with a function capable of executing the above-mentioned series of processes as a whole, and what kind of functional block is used to realize this function is particularly an example of FIG. Not limited to. Further, the location of the functional block is not particularly limited to FIG. 3, and may be arbitrary. For example, the functional block of the learning device 4 may be transferred to a computer 3 or the like. Further, the functional block of the computer 3 may be transferred to the learning device 4 or the like. Furthermore, the computer 3 and the learning device 4 may have the same hardware.

Further, for example, when a series of processes are executed by software, a program constituting the software is installed on a computer or the like from a network or a recording medium.
The computer may be a computer embedded in dedicated hardware. Further, the computer may be a computer capable of executing various functions by installing various programs, for example, a general-purpose smartphone or a personal computer in addition to a server.

Further, for example, a recording medium containing such a program is not only composed of a removable medium (not shown) distributed separately from the device main body in order to provide the program to the user, but is also preliminarily incorporated in the device main body. It is composed of a recording medium or the like provided to the user in the state.

In the present specification, the steps for describing a program recorded on a recording medium are not necessarily processed in chronological order, but also in parallel or individually, even if they are not necessarily processed in chronological order. It also includes the processing to be executed.
Further, in the present specification, the term of the system means an overall device composed of a plurality of devices, a plurality of means, and the like.

1 ... Phone, 2 ... Headset, 3 ... Computer, 4 ... Learning device, 11 ... CPU, 18 ... Storage unit, 31 ... Conversion unit, 32 ... Speech recognition model, 33 ... text data acquisition unit, 34 ... acoustic data acquisition unit, 35 ... emotion recognition model, 36 ... emotion recognition unit

Claims (6)

A text data acquisition means that recognizes the voice data of the speaker's utterance and acquires text data indicating the content of the utterance based on the result of the voice recognition.
An acoustic data acquisition means for acquiring the voice data of the utterance of the speaker as acoustic data,
An emotion recognition means that recognizes the emotion of the speaker based on the acoustic data and the text data and obtains emotion information indicating the recognition result.
Information processing device equipped with. The emotion recognition means
At least one of an emotion label that classifies the speaker's emotions so as to include at least "joy", "anger", "sorrow", and "normal" as the emotion information, and a probability value indicating the accuracy of the classified emotion label. To output,
The information processing device according to claim 1. The emotion recognition means
The emotion of the speaker is recognized by using the first feature value extracted from the acoustic data and the second feature value extracted from the text data.
The information processing device according to claim 1 or 2. The emotion recognition means
The text data is decomposed into morphemes by morphological analysis.
Converts the decomposed morphemes into hiragana based on preset word judgment conditions.
A feature value extraction means for converting the converted hiragana into the second feature value using a model in which text has been learned in advance.
The information processing apparatus according to claim 3, further comprising. It is an information processing method executed by an information processing device.
A step of voice-recognizing the voice data of the speaker's utterance and acquiring text data indicating the content of the utterance based on the result of the voice recognition.
A step of acquiring the voice data of the utterance of the speaker as acoustic data, and
A step of recognizing the emotion of the speaker based on the acoustic data and the text data, and obtaining emotion information indicating the recognition result.
Information processing methods including. A program that lets a computer perform processing
The computer
A text acquisition means that recognizes the voice data of the speaker's utterance and acquires text data indicating the content of the utterance based on the result of the voice recognition.
An acoustic acquisition means for acquiring the voice data of the utterance of the speaker as acoustic data,
An emotion recognition means that recognizes the emotion of the speaker based on the acoustic data and the text data, and obtains emotion information indicating the recognition result.
A program that functions as.

Images