JP2024037614A - Program, device, system and method for generating an emotion dictionary for a predetermined object, and emotion estimation program and device - Google Patents

Abstract

The present invention provides a program that can automatically generate emotion dictionary information for generating or updating an emotion dictionary for a predetermined object. [Solution] This emotion dictionary generation program includes a conversion means for converting voice data of a predetermined target into text data, and an emotion determination means for determining information related to the emotion of the target from this voice data using an emotion identification model. , emotion dictionary information for generating or updating an emotion dictionary in which target words, phrases, clauses, or sentences and information related to emotions are registered in association with each other, words, phrases, or words included in the converted text data; A computer is caused to function as a dictionary generation means that generates and outputs emotion dictionary information in which clauses or sentences are associated with information related to determined emotions. Here, the dictionary generation means uses a set of the converted text data and the information about the emotion determined from the audio data related to the text data, for which the probability of emotion identification is higher than a predetermined value. It is also preferable to generate emotion dictionary information. [Selection diagram] Figure 1

Description

The present invention relates to techniques for generating and utilizing a sentiment lexicon.

In various fields such as medical care, nursing care, education, and marketing, the need to estimate the emotions of a target person is rapidly increasing. If the emotions of the target person can be appropriately understood, it becomes possible to provide the target person with a more suitable service that matches the emotion, for example. Currently, a method using a sentiment lexicon is widely adopted for such emotion estimation. Here, the emotion dictionary is a language/emotion correspondence model in which words and phrases used are registered in association with the types of emotions contained therein.

Emotion estimation using such an emotion dictionary is as disclosed in Non-Patent Document 1. The rationale is that it is likely to be strongly influenced by the person's emotional state. In other words, by analyzing the words and phrases used by the subject, it is possible to obtain information about the subject's current emotional state.

As an example of such emotion estimation, Patent Document 1 discloses that an emotion dictionary is used to assign a positive or negative label (for emotion) to an initial set of seed words in input text data. A technique is disclosed for determining the emotion of the target by assigning the emotion to the target. The emotion dictionary used here is a general-purpose model generated from an unspecified number of data.

US Patent Application Publication No. 2012/0046938

James W. Pennebaker, Matthias R. Mehl, and Kate G. Niederhoffer, “Psychological Aspects of Natural Language Use: Our Words, Our Selves”, Annual Review of Psychology, Volume 54, pp 547-577, <https://doi .org/10.1146/annurev.psych.54.101601.145041>, 2003 T. Rahman and C. Busso, "A personalized emotion recognition system using an unsupervised feature adaptation scheme," 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5117-5120, <https://doi. org/10.1109/ICASSP.2012.6289072＞, 2012

In this way, in many cases, emotion dictionaries have been used as general-purpose models for emotion estimation. On the other hand, in order to more accurately grasp the emotional state of a subject, it is ideal to use a model dedicated to the subject. For this reason, emotion dictionaries have traditionally been personalized (individualized) through heavy manual updating work.

Regarding the individualization of such models, for example, Non-Patent Document 2 discloses an attempt to make a system for identifying emotions from speech data suitable for each individual subject. Specifically, an attempt was made to resolve potential discrepancies between the acoustic (speech) features of a pre-trained model and the acoustic (speech) features of a specific subject. There is. However, in general, no attempt has been made to automatically personalize the emotional dictionary itself without manual intervention.

Therefore, the present invention provides an emotional dictionary generation program, an emotional dictionary generation device, an emotional dictionary generation system, and an emotional dictionary generation method that can automatically generate emotional dictionary information for generating or updating an emotional dictionary for a predetermined target. The purpose is to provide. Another object of the present invention is to provide an emotion estimation program and an emotion estimation device that estimate the emotion of the target using the emotion dictionary generated in this way.

According to the invention,
a conversion means for converting the acquired audio data of a predetermined target into text data;
emotion determination means for determining information related to the emotion of the target from the voice data using an emotion identification model;
Emotion dictionary information for generating or updating an emotion dictionary in which words, phrases, clauses, or sentences of the target and information related to the emotions of the target are registered in association with words and phrases included in the text data. An emotion dictionary generation program is provided that causes a computer to function as a dictionary generation means for generating and outputting emotion dictionary information in which clauses or sentences are associated with information related to determined emotions.

In the emotion dictionary generation program according to the present invention, the dictionary generation means has a predetermined probability of emotion identification among a set of the text data and the information related to the emotion determined from the audio data related to the text data. It is also preferable to generate the emotion dictionary information using a set that is high enough to satisfy the condition.

According to the present invention, the emotion dictionary generated or updated by the emotion dictionary generation program described above is used to determine the emotion of the estimation target data including the word, phrase, clause, or sentence expressed by the target. A program that estimates information,
Among the words, phrases, clauses, or sentences included in the acquired estimation target data, a predetermined condition is applied to one of the plurality of preset types of information related to the emotion registered in the emotion dictionary. A means for identifying words, phrases, clauses, or sentences that are strongly associated with each other to meet the criteria, such as emotional phrases;
Information regarding the presence or absence of the specified word, phrase, clause, or sentence in the estimation target data, and if yes, at least one corresponding word, phrase, clause, or sentence that is associated with the specified word, phrase, clause, or sentence. An emotion estimation program that causes a computer to function as an emotion estimation means that determines emotion information about the estimation target data or about a word, phrase, clause, or sentence included in the estimation target data based on emotion information. is provided.

In the emotion estimation program according to the present invention, the emotion word/phrase identification means selects one of a plurality of preset types of information related to the emotion from among the words, phrases, clauses, or sentences included in the estimation target data. In addition, it is also preferable to identify words, phrases, clauses, or sentences that are strongly correlated in a way that greatly deviates from the average degree of correspondence until a predetermined condition is satisfied.

Further, as an embodiment of the emotion estimation program according to the present invention, the emotion estimation means includes:
Sequentially slide the window within the estimation target data, and if at least one of the identified words, phrases, clauses, or sentences exists in the window at each position, the identified word, phrase, or clause is Alternatively, among the information related to the one emotion associated with the sentence, one that appears frequently until a predetermined condition is met is determined as the information related to the emotion for the window,
For each word, phrase, clause, or sentence included in the estimation target data, information related to emotions determined for at least one window containing the word, phrase, clause, or sentence until a predetermined condition is met. It is also preferable to determine information with a high frequency of appearance as information related to the emotion of each word, phrase, clause, or sentence included in the estimation target data.

Furthermore, in the embodiment using the window described above, when the estimation target data includes a plurality of sentences, the emotion estimation means adds a padding element before and after each sentence included in the estimation target data. After that, it is also preferable to sequentially slide windows within the estimation target data, and for each sentence, determine emotion-related information for each window that is sequentially slid within the sentence that includes the assigned padding element. .

According to the invention, further:
a conversion means for converting the acquired audio data of a predetermined target into text data;
emotion determination means for determining information related to the emotion of the target from the voice data using an emotion identification model;
Emotion dictionary information for generating or updating an emotion dictionary in which words, phrases, clauses, or sentences of the target and information related to the emotions of the target are registered in association with words and phrases included in the text data. An emotion dictionary generation device is provided, which includes a dictionary generation unit that generates and outputs emotion dictionary information in which clauses or sentences are associated with information related to determined emotions.

According to the present invention, there is also a device for estimating information related to emotion regarding estimation target data including a word, phrase, clause, or sentence expressed by a predetermined object,
a conversion means for converting the acquired target audio data into text data;
emotion determination means for determining information related to the emotion of the target from the voice data using an emotion identification model;
Emotion dictionary information for generating or updating an emotion dictionary in which words, phrases, clauses, or sentences of the target and information related to the emotions of the target are registered in association with words and phrases included in the text data. , dictionary generation means for generating and outputting emotion dictionary information in which clauses or sentences are associated with information related to the determined emotion;
Among the words, phrases, clauses, or sentences included in the acquired estimation target data, those that are registered in the emotion dictionary generated or updated using the emotion dictionary information and that relate to multiple preset types of emotions. A means for specifying emotional phrases, etc., for specifying words, phrases, clauses, or sentences that are strongly associated with one of the pieces of information to meet a predetermined condition;
Information regarding the presence or absence of the specified word, phrase, clause, or sentence in the estimation target data, and if yes, at least one corresponding word, phrase, clause, or sentence that is associated with the specified word, phrase, clause, or sentence. An emotion estimation device is provided that includes emotion estimation means for determining emotion information about the estimation target data or about a word, phrase, clause, or sentence included in the estimation target data based on the emotion information. Ru.

According to the invention, further:
a conversion means for converting the acquired audio data of a predetermined target into text data;
emotion determination means for determining information related to the emotion of the target from the voice data using an emotion identification model;
Emotion dictionary information for generating or updating an emotion dictionary in which words, phrases, clauses, or sentences of the target and information related to the emotions of the target are registered in association with words and phrases included in the text data. An emotion dictionary generation system is provided, which includes a dictionary generation unit that generates and outputs emotion dictionary information in which clauses or sentences are associated with information related to determined emotions.

According to the present invention, furthermore,
converting the acquired voice data of a predetermined target into text data, and determining information related to the emotion of the target from the voice data using an emotion identification model;
Emotion dictionary information for generating or updating an emotion dictionary in which words, phrases, clauses, or sentences of the target and information related to the emotions of the target are registered in association with words and phrases included in the text data. An emotion dictionary implemented by a computer, comprising the steps of: generating emotion dictionary information that associates passages or sentences with information related to the determined emotion; and generating or updating the emotion dictionary using the emotion dictionary information. A generation method is provided.

According to the emotional dictionary generation program, emotional dictionary generation device, emotional dictionary generation system, and emotional dictionary generation method according to the present invention, emotional dictionary information for generating or updating an emotional dictionary for a predetermined target can be automatically generated. Can be done. Furthermore, according to the emotion estimation program and emotion estimation device according to the present invention, it is possible to estimate the emotion of this target using the emotion dictionary generated in this way.

FIG. 1 is a functional block diagram showing an embodiment of an emotion dictionary generation device and an emotion estimation device according to the present invention. FIG. 2 is a schematic diagram for explaining an embodiment of the emotion dictionary generation method according to the present invention. FIG. 2 is a schematic diagram for explaining an embodiment of emotion word/phrase identification processing according to the present invention. FIG. 2 is a schematic diagram for explaining one embodiment of emotion estimation processing according to the present invention. FIG. 2 is a schematic diagram for explaining one embodiment of emotion estimation processing according to the present invention.

Hereinafter, embodiments of the present invention will be described in detail using the drawings.

[Emotion dictionary generation device/emotion estimation device]
FIG. 1 is a functional block diagram showing an embodiment of an emotion dictionary generation device and an emotion estimation device according to the present invention.

The smartphone 1 shown in FIG. 1 is an embodiment of an emotion dictionary generation device and an emotion estimation device according to the present invention. Using audio data related to utterances, "emotion dictionary information" (as a component of a sentiment dictionary) for generating or updating a "sentiment lexicon" exclusively for this subject is automatically generated without manual intervention. generate.

Furthermore, in the present embodiment, the smartphone 1 transmits the generated "emotion dictionary information" to an external emotion dictionary database (DB) 2 via communication to generate or update a target-specific "emotion dictionary". be. Here, the "emotion dictionary" dedicated to this target includes words, phrases, clauses, or sentences in the target (for example, expressed by the target) and information related to the target's emotions contained therein (for example, "anger", This is a language/emotion correspondence model that is registered in association with one of the five emotion types: "joy," "sadness," "surprise," and "neutral."

Incidentally, in this embodiment, the emotion dictionary DB2 collectively stores and manages a large number of "emotion dictionaries", each of which is a dedicated dictionary for each user (target person). The emotional dictionary generation device according to the present invention receives "emotional dictionary information" linked to a predetermined user ID, and generates and updates an "emotional dictionary" exclusively for the user (target person) corresponding to the user ID. be. In addition, as a modification, the smartphone 1 is equipped with at least one "emotion dictionary" dedicated to one user (target person) or each of a plurality of users, and generates this "emotion dictionary" within itself.・You may update it.

Here, the smartphone 1 (emotional dictionary generation device) specifically generates and updates such an "emotional dictionary" by:
(A) a text conversion unit 111 that converts "audio data" related to the speech/utterance of a predetermined target acquired (for example, by the microphone 102) into "text data";
(B) From this "voice data", an emotion identification model is used to obtain "information related to the emotion" of this target, and in this embodiment, a plurality of preset emotion types (for example, "anger,""joy," and "sadness") , "surprise" and "neutral");
(C) "Emotional dictionary information" for generating or updating the "Emotional dictionary", which includes words, phrases, clauses, or sentences included in the "text data" in (A) above, and determined in (B) above. and a dictionary generation unit 113 that generates and outputs "emotion dictionary information" in which "information related to emotions" (identified emotion types) are associated with each other.

In this way, the smartphone 1 determines a word, phrase, clause, or sentence related to a predetermined target from the "voice data" of this target, and "information related to emotion" (for example, emotion type) of this target, and This makes it possible to automatically generate and output "emotional dictionary information" (as a constituent element of the emotional dictionary) for generating or updating the "emotional dictionary" dedicated to this subject. Here, the target-specific "emotion dictionary" that is generated and updated can be regarded as a model that can provide information about, for example, what kind of words, etc. will be selected when a given target is in what emotional state. . By the way, of course, "information related to emotions" is not limited to one of the five emotion types mentioned above, but for example, one of "positive," "negative," and "neutral." It may be.

In addition, in this embodiment, by collecting, for example, what this target says about various topics as "audio data" of a predetermined target, an expanded version that can also be used for various topics. It is also possible to automatically generate and update the ``emotion dictionary''. For example, the registration of "proper nouns" such as people's names, company names, place names, etc. that frequently appear on a given topic can be done automatically without manual intervention. Here, for example, in user A's "emotion dictionary", "Iidabashi" is more strongly associated with "joy", while in user B's "emotion dictionary", "Iidabashi" is more strongly associated with "surprise". It is also possible that there is a strong correspondence.

In the example shown later, the "emotion dictionary" that is generated and updated is one in which one word is associated with an emotion type, but for example, two or more (consecutive sentences) are associated. It is also possible to associate a group of words with an emotion type. Similarly, one or more (consecutive) clauses are associated with an emotion type, one or more (consecutive) clauses are associated with an emotion type, or one or more (consecutive) phrases are associated with an emotion type, or one or more (consecutive) clauses are associated with an emotion type, ) It is also possible to associate sentences with emotion types. Furthermore, all parts of speech, such as nouns, verbs, adjectives, and adverbs, may be adopted as words to which emotion types should be associated (to be registered), or only specific parts of speech, such as nouns, verbs, adjectives, and adverbs. It is also possible to adopt only Furthermore, for declensions of nouns and verbs (e.g. "tries", "trying", "tried", etc.), those with the same stem may be registered as one (e.g. "try"), or , each variant may also be registered individually.

Further, the above-mentioned "predetermined object" is not limited to one specific person, for example, in the case of a person. For example, multiple people who are classified into one type with respect to a certain attribute, such as members of a predetermined group or residents of a predetermined area, are defined as a "predetermined target," and a set of these multiple people (for example, one group) is used exclusively. It is also possible to generate and update an "emotion dictionary."

Furthermore, in the present embodiment, the smartphone 1 uses a generated or updated "emotion dictionary" dedicated to a predetermined object to estimate the emotion of the object from the "estimation target data" related to this object. Specifically, information related to the target emotion is extracted from the "estimation target data" through processing that will be explained in detail later. "," "surprise," and "neutral." Here, "estimated target data" related to this target includes text data of sentences written or created by the target, and audio data related to the target's utterances (text data generated by converting). This is text data or audio data that includes words, phrases, clauses, or sentences expressed/selected by this target.

Furthermore, the smartphone 1 (emotion dictionary generation device, emotion estimation device) of this embodiment is capable of supporting various languages such as Japanese, English, and Chinese. For example, "emotion dictionary information" with a corresponding user ID and language ID is output, and the "emotion dictionary information" corresponding to the language used by the user (target person) corresponding to the user ID (the language corresponding to the language ID) is output. It is also possible to generate and update an "emotion dictionary" exclusively for the user (target person). In addition, when such a variety of "emotion dictionaries" are available, by appropriately using the "emotion dictionaries" from each of the "estimation target data" related to various languages by multiple users (subjects), It also becomes possible to more reliably estimate the emotions of each user (target person).

As a modification, the emotion dictionary generation device and the emotion estimation device according to the present invention may be separate devices, as will be described later. Furthermore, as will be described later, the emotion dictionary generation device and emotion estimation device according to the invention are not limited to smartphones, but can be used, for example, in the emotion dictionary DB 2, document DB 3, voice DB 4, SNS server 5, or e-mail server. It may be a device mounted on or attached to 6. Furthermore, one, two, or all of the text conversion unit 111 in (A) above, the emotion determination unit 112 in (B) above, and the dictionary generation unit 113 in (C) above may be replaced by other components. It may be included in a separate device. For example, all three may be included in three different servers. In any case, in such a case, the entirety of the above (A) to (C) constitutes the emotion dictionary generation system according to the present invention.

[Device functional configuration, emotion dictionary generation program and method, emotion estimation program and method]
Hereinafter, the functional configuration of the smartphone 1 as an embodiment of the emotion dictionary generation device and emotion estimation device according to the present invention will be described in more detail. Similarly, in the functional block diagram of FIG. 1, the smartphone 1 includes a communication interface section 101, a microphone 102, an audio data storage section 103, a text data storage section 104, an emotion estimation result storage section 105, and a user interface (U. I) unit 106, and a processor/memory (arithmetic processing system with memory function).

Here, this processor memory stores an embodiment of the emotion dictionary generation program and emotion estimation program according to the present invention, and also has a computer function to execute the emotion dictionary generation program and emotion estimation program. By executing this, emotion dictionary generation processing and emotion estimation processing are implemented. For this reason, the emotion dictionary generation device and emotion estimation device according to the present invention may be a mobile terminal such as a smartphone as in the present embodiment, but the emotion dictionary generation device and emotion estimation device according to the present invention may be equipped with the emotion dictionary generation program and emotion estimation program according to the present invention. For example, it may be a tablet computer, a notebook computer, a personal computer (PC), or a wearable device such as an HMD (Head Mounted Display), smart glasses, or a smart watch. Furthermore, as described above, it is also possible to use a non-cloud server or a cloud server.

Further, in this embodiment, the above processor memory includes a text conversion section 111, an emotion determination section 112, a dictionary generation section 113, an emotion word/phrase identification section 114, an emotion estimation section 115, and a communication control section 121. , functions as the input/output control section 122. Here, the text conversion section 111, the emotion determination section 112, and the dictionary generation section 113 can be regarded as functions realized by executing the emotion dictionary generation program according to the present invention stored in the processor memory. Furthermore, the emotion word/phrase identification unit 114 and the emotion estimation unit 115 can be regarded as functions implemented by executing the emotion estimation program according to the present invention, which is also stored in the processor memory. Furthermore, the process flow shown by connecting the functional components of the smartphone 1 with arrows in the functional block diagram of FIG. 1 can be understood as an embodiment of the emotion dictionary generation method and emotion estimation method according to the present invention.

As a modification, the emotion dictionary generation device and the emotion estimation device according to the present invention may be different devices each equipped with an emotion dictionary generation program and an emotion estimation program according to the present invention. That is, both the emotion dictionary generation device and the emotion estimation device according to the present invention may be separate devices having the above-described device configuration.

Similarly, in the functional block diagram of FIG. 1, in this embodiment, the audio data storage unit 103 acquires audio data related to the speech/speech of the target person as an output of the microphone 102 or from the audio DB 4 etc. to the communication interface unit 101 and It is acquired via the communication control unit 121, and stored and managed after giving the corresponding user ID.

In this embodiment, the text data storage unit 104 acquires text data written and created by the target person from the document DB 3, SNS server 5, email server 6, etc. via the communication interface unit 101 and the communication control unit 121. Then, save and manage the information after assigning the corresponding user ID. It also acquires text data (as a result of converting audio data) output from the text converter 111, which will be described later, and stores and manages the data after giving it a corresponding user ID.

<Emotion dictionary generation process>
Similarly, in the functional block diagram of FIG. 1, in this embodiment, a text conversion unit 111 converts audio data of a predetermined target acquired from the audio data storage unit 103 into text data. Here, this conversion process can also be performed using a known speech-to-text application, such as Google Docs (registered trademark). Alternatively, for example, the acquired voice data may be converted into digital data in a predetermined format, and this digital data may be converted into text data using a linguistic unit extraction model constructed using machine learning algorithms such as neural networks. It's okay. In addition, in this embodiment, the text conversion unit 111 performs morphological analysis, etc., if necessary, to identify and understand words (morphemes), phrases, etc. included in the text data, as well as their part-of-speech types. Good too.

In the present embodiment, the emotion determining unit 112 uses an "emotion identification model" to determine the emotion type at the time of the utterance/utterance of a predetermined target from the audio data related to the utterance/utterance of a predetermined target acquired from the audio data storage unit 103. (In this embodiment, one of the following five: "anger," "joy," "sadness," "surprise," and "neutral.") is determined. For example, this audio data is converted into digital data in a predetermined format, such as data in the same format as audio digital data recorded in a known audio data corpus, and this digital data is applied to a trained "emotion identification model". (a) the (estimated) emotion type at the time of the target's utterance/speech, and (b) a confidence score representing the certainty of this emotion type.

Here, as the "emotion identification model", various models constructed using machine learning algorithms can be adopted, such as Mel-Frequency Cepstral Coefficients (MFCC), pitch, i-vectors, etc. , LLD (Low-Level Descriptors) and other techniques are used to extract audio features from digital data in a predetermined format generated (converted) from audio data, and then from these audio features, a deep neural network (DNN, Deep The estimated emotion type will be output using a discriminator/classifier constructed using Neural Networks), Gaussian Mixture Models (GMM), Support Vector Machines (SVM), etc. be able to.

Similarly, in the functional block diagram of FIG. 1, in this embodiment, the dictionary generation unit 113 is emotion dictionary information for generating or updating the emotion dictionary (stored in the emotion dictionary DB2) described above,
(a) Words, phrases, clauses, or sentences included in the text data generated by converting the “predetermined target audio data” obtained from the text data storage unit 104;
(b) Generate emotion dictionary information that associates information related to emotion (emotion type in this embodiment) determined by the emotion determination unit 112 with respect to the same "predetermined target audio data" as in (a) above. Output. In this embodiment, this generated emotion dictionary information is transmitted to the emotion dictionary DB2 via the communication control unit 121 and the communication interface unit 101.

In addition, the dictionary generation unit 113 generates the emotion type (b) determined from the text data (a) (words, phrases, clauses, or sentences included in the text data) and the audio data related to this text data (emotion type). information), the emotion dictionary information is generated and output using the group in which the reliability related to emotion identification (in this embodiment, the reliability score output from the emotion identification model) is high until a predetermined condition is satisfied. is also preferable. For example, emotion dictionary information may be generated and output using only pairs with reliability scores of 0.8 (80%) or higher.

FIG. 2 is a schematic diagram for explaining an embodiment of the emotion dictionary generation method according to the present invention. In the following embodiments including this embodiment, English is used as the language for speech and text. However, as described above, even when other languages such as Japanese or Chinese are used, the emotion dictionary generation process can be performed in the same way.

As shown in FIG. 2(A), in this example, the subject first utters "I hate you." The dictionary generation unit 113
(a) receiving from the emotion determining unit 112 the emotion type “anger” and the confidence score (=0.8) determined for the audio data related to this utterance acquired via the microphone 102;
(b) confirming that the received confidence score (=0.8) is 0.8 or higher and deciding to generate emotion dictionary information;
(c) Generate emotion dictionary information that associates each of the words “I,” “hate,” and “you” received from the text conversion unit 111 with the received emotion type “anger,” and direct it to the emotion dictionary DB2. is output.

In addition, the emotion dictionary DB2 that received this emotion dictionary information increments the count number in the emotion type "anger" column by 1 for each of the words "I", "hate", and "you" registered in the emotion dictionary table. (Here, the count number is set to 1).

Next, as shown in FIG. 2(B), the subject utters "I hate this train." The dictionary generation unit 113 receives the emotion type “anger” and the confidence score (=0.5) determined for the audio data related to this utterance from the emotion determination unit 112, and determines that the received confidence score (=0.5) is less than 0.8. After confirming this, we have decided not to generate emotion dictionary information. As a result, the emotion dictionary DB2 does not register this utterance in the emotion dictionary.

After that, as shown in FIG. 2(C), the subject utters "I love you. I hate you." Here, the dictionary generation unit 113 determines the emotion type "joy" and confidence score (=0.9) determined for the audio data related to the utterance "I love you." and the audio data determined for the utterance "I hate you." The emotion type "anger" and the confidence score (=0.9) are received from the emotion determining unit 112. In addition, as the reliability scores (=0.9) received were all 0.8 or higher,
(a) Associate each of the words “I”, “love”, and “you” received from the text conversion unit 111 with the received emotion type “joy”, and further,
(b) Generate emotion dictionary information that associates each of the words “I,” “hate,” and “you” received from the text conversion unit 111 with the received emotion type “anger,” and direct it to the emotion dictionary DB2. is output.

In addition, the emotion dictionary DB 2 that has received this emotion dictionary information increments the count numbers in the column of emotion type "joy" and the column of emotion type "anger" by 1 for the word "I" registered in the emotion dictionary table, For the word "hate", increment the count number in the emotion type "anger" column by 1, and for the word "you", increment the count number in the emotion type "joy" column and the emotion type "anger" column by 1. Furthermore, for the word "love", the count number in the emotion type "joy" column is incremented by one. In the emotion dictionary, different emotion types are registered in association with one word (for example, "I") because this word is often selected and used in different emotional states of the subject. That is quite possible.

The dictionary generation unit 113 repeats the above-described processing, generates emotion dictionary information having a reliability higher than a predetermined value, outputs it to the emotion dictionary DB2, and generates emotional dictionary information having a reliability higher than a predetermined value for the emotion dictionary DB2 in a table format. It is possible to generate and update emotion dictionaries. Here, the "emotion dictionary" exclusively for the target person that is generated and updated in this way is a model that can provide information about, for example, what words and phrases the target person selects in what emotional state. It can also be captured.

Further, the processes in the text conversion unit 111, emotion determination unit 112, and dictionary generation unit 113 as described above can be performed at all times as, for example, the background of the device (smartphone 1) (in a standby state). ) may be performed, for example, as soon as the target person's voice data is acquired from the voice DB 4 or the microphone 102. As a result, the subject's emotion dictionary is updated as appropriate, and becomes more accurate and has a larger number of registered words.

<Emotion estimation processing>
Returning to the functional block diagram of FIG. 1, in this embodiment, the emotion word/phrase identification unit 114 selects words, phrases, clauses, or sentences of this target from among the words, phrases, clauses, or sentences included in the acquired "estimation target data of a predetermined target". The information is registered in the emotion dictionary, and in the emotion dictionary, the information corresponds strongly to one of the preset multiple types of emotion information (in this embodiment, multiple emotion types) until a predetermined condition is met. Identify the word, phrase, clause, or sentence that appears. Specifically, in the present embodiment, words, phrases, clauses, or sentences whose "t-score" (described later) for the one emotion type exceeds a predetermined threshold (for example, 3.5) are identified. It is.

Here, the above-mentioned "estimated target data of a given target" refers to text data of sentences written/created by this target, and text data generated by converting audio data related to utterances/utterances of this target. ) is text data or audio data that includes words, phrases, clauses, or sentences expressed or selected by the target.

in particular,
(a) Text data such as essays and papers written and created by the subject received from the document DB3,
(b) audio data related to the target utterance/utterance received from the audio DB 4 (for example, text data generated by converting it with the text conversion unit 111);
(c) Posted data posted by the target received from the SNS (Social Networking Service) server 6;
(d) email data sent by the target received from the email server 6;
(e) Audio data related to the target's utterances/utterances collected by the microphone 102 (for example, text data generated by converting it with the text conversion unit 111),
(f) Text data, etc. input by the target using the keyboard or touch panel of the user interface 106 can be adopted as the "estimation target data of the predetermined target."

In this way, the smartphone 1 of this embodiment is capable of handling various types of "estimation target data". In other words, as can be seen from a comparison with conventional emotion identification models using voice data, the emotion estimation device is compatible with various modalities.

Here, the above "t-score" ts for one emotion type in a certain word etc. is calculated using the following formula (1) ts=(e _i -μ)/(σ×n ^-0.5 )
This is an index calculated by In the above formula (1), e _i is the count number (value in the relevant column) of this emotion type i in this word, etc., and μ and σ are the average of the counts in this word, etc. over all emotion types, respectively. value and standard deviation, and n is the number of emotion types (for example, n = 5 if the emotion types are "anger", "joy", "sadness", "surprise", and "neutral"). .

In this target emotion dictionary, words, phrases, clauses, or sentences whose t-score ts for a certain emotion type exceeds a predetermined threshold (for example, 3.5) are It can be said that they are strongly correlated in such a way that they deviate significantly until a predetermined condition is met. In this embodiment, the emotion word/phrase identification unit 114 identifies words, phrases, clauses, or sentences included in the estimation target data that are strongly associated with a certain emotion type. , it is specified as an "emotional word", "emotional phrase", "emotional clause", or "emotional sentence" related to the certain emotion type.

Note that, instead of the above-described t-score ts, various indexes indicating the degree of deviation from the average degree of correspondence, such as a deviation value, can be used.

FIG. 3 is a schematic diagram for explaining an embodiment of the emotion word/phrase identification process according to the present invention.

According to FIG. 3A, the emotion word/phrase identification unit 114 obtains the emotion dictionary information of a predetermined target stored in the emotion dictionary DB2, and calculates the t-score for each emotion type for each registered word. ts is calculated, and the emotion dictionary DB2 generates and updates a t-score-based emotion dictionary (the count number in each column is rewritten to t-score ts) based on this calculation result. Incidentally, as a modification, the emotion word/phrase identification unit 114 may generate and update a t-score-based emotion dictionary based on the above calculation result, and store and manage it by itself. It is also possible to calculate the t-score ts only for words for which the t-score ts needs to be calculated, and to save and manage the results. Furthermore, the emotion dictionary DB2 may generate and update the t-score-based emotion dictionary by itself.

Next, as shown in FIG. 3B, the emotion word/phrase identification unit 114 uses this t-score-based emotion dictionary to determine the estimation target data: “Today will be rough. I have English class, but my teacher is Abc-san. He always tries to embarrass me.”

Specifically, the emotion word/phrase identification unit 114 identifies words that constitute this estimation target data and are registered in the t-score-based emotion dictionary, and whose t-score value for a certain emotion type exceeds 3.5. The words "English" and "Abc-san" are specified as emotional words related to "joy" and "sadness," respectively.

Returning to the functional block diagram of FIG. 1, the emotion estimation unit 115 in this embodiment:
(a) Information regarding the presence or absence of a specified emotion word, emotion phrase, emotion clause, or emotion sentence in the estimation target data, and
(b) If yes in (a) above, information related to at least one emotion (emotion type in this embodiment) that is associated with the specified emotion word, emotion phrase, emotion clause, or emotion sentence.
Based on this, information related to emotions (emotion type) about the estimation target data or about a word, phrase, clause, or sentence included in the estimation target data is determined.

Specifically, the emotion estimation unit 115
(a) Sequentially slide windows within the estimation target data, and if there is at least one emotion word, emotion phrase, emotion clause, or emotion sentence specified in the window at each position, this Among the emotion types associated with the identified emotion words, emotion phrases, emotion clauses, or emotion sentences, those that appear frequently until a predetermined condition is met (in this embodiment, the emotion type that appears most frequently) are selected. , determine the emotion type for the window,
(b) For each word, phrase, clause, or sentence included in the estimation target data, it appears before a predetermined condition is met among the emotion types determined for at least one window containing the word, phrase, clause, or sentence. A frequently occurring emotion type (in this embodiment, an emotion type whose appearance frequency exceeds 50%) is determined as an emotion type for each word, phrase, clause, or sentence included in this estimation target data.

Here, in this embodiment, the reason why the emotion estimation process is performed using a window is that the subject does not put his or her emotions into words or descriptions of a single word in a sentence, but rather words related to emotion. This is because it is often for a predetermined range within a sentence that contains the text. By sliding the window, it is possible to more accurately capture this predetermined area that is filled with emotion. Note that the size of the window can be set to four words, for example, when the emotional words within the window are focused on by sequentially sliding word by word. Of course, the size may be less than three words or more than five words. By the way, if the language used is English, it is preferable to use a window the size of four words, as this makes it easier to extract relationships with surrounding words and determine a single emotion type (emotion word). I know this from experience.

Furthermore, in this embodiment, when the estimation target data includes a plurality of sentences, the emotion estimation unit 115 adds padding elements before and after each sentence included in the estimation target data, as shown in FIGS. 5), and then slide the windows sequentially within this estimation target data, and for each sentence, the emotion type for each window that is sequentially slid within the sentence that includes the assigned padding element. It is also preferable to determine (information related to emotions). Note that a sentence break can be determined by the presence of a period (period in English), a question mark, or the like in the text data.

Here, the padding element is added so that words at the beginning and end of a sentence can be treated equally to other words in the sentence in a group of windows related to one sentence. Treated as an unmatched word. For this reason, it is also preferable that the larger the window size, the more padding elements are added before and after each sentence. For example, when using a window whose size is four words, it is also preferable to add two padding elements "_" before and after each sentence.

Furthermore, among the words, phrases, clauses, or sentences included in the estimation target data, the emotion type is determined as in (a) above if the words, phrases, clauses, or sentences are registered in the emotion dictionary. It is also preferable to limit the total count number (for all emotion types) to 10 or more. In this case, the window in (a) above will skip that position and slide to the next position when more than a predetermined number (for example, 75% or more) of the included words have a total count of less than 10. The setting may be set to

Hereinafter, the emotion estimation process using the window and padding elements described above will be specifically explained using FIGS. 4 and 5.

4 and 5 are schematic diagrams for explaining one embodiment of emotion estimation processing according to the present invention.

Here, in this example, the same data as the example of emotional word identification processing shown in FIG. 3 is used: “Today will be rough. I have English class, but my teacher is Abc-san. He always tries to embarrass me.” is the estimation target data related to the target, and as shown in Figure 3 (B), among this estimation target data, “English” and “Abc-san” are respectively “joyful”. It has been identified as an emotion word related to ``,'' and an emotion word related to ``sadness''.

First, Figure 4 shows the results obtained by sequentially sliding a window with a size of four words on data in which two padding elements "_" are added before and after each sentence included in this estimation target data. , three window groups are shown. Here, each window group is composed of a plurality of windows from the first two windows having padding elements "_" to the last two windows having padding elements "_". That is, these three windows correspond to three sentences included in the estimation target data, respectively.

Similarly, in FIG. 4, the first window group W1-1 to W1-5 corresponds to the sentence "Today will be rough.", and each window group (W1-1, W2-2, ..., W1- 5) contains two to four words included in this sentence. Here, in this example, none of the windows of this first window group contain any emotion words ("English", "Abc-san") specified for this object.

As a result, the emotion estimation unit 115
(a) Emotion type related to the sentence “Today will be rough.”
(b) Emotion type related to each window (W1-1, W1-2, ..., W1-5), and (c) each word included in the sentence (“Today”, “will”, “be”) , “rough”) are all set to “neutral.” Note that it is also possible to decide here that the emotion type cannot be determined for these.

Also, as shown in FIG. 4, the same results as for the first window group are obtained for the third window groups W3-1 to W3-7, and as a result, the emotion estimation unit 115
(a) Emotion type related to the sentence “He always tries to embarrass me.”
(b) Emotion type related to each window (W3-1, W3-2, ..., W3-7), and (c) each word included in the sentence (“He”, “always”, “tries”) , “to,” “embarrass,” “me”) are determined to be “neutral.” Note that it is also possible here that the emotion type cannot be determined for these. By the way, if "embarrass" is an emotion word associated with "sadness", the emotion type associated with each of windows W3-4, W3-5, W3-6, and W3-7 would be "sadness". will be determined. Furthermore, by using the method described in detail later, the emotion type related to the words "to" and "me" is also determined to be "sadness."

Also in FIG. 4, the second window group W2-1 to W2-10 correspond to the sentence "I have English class, but my teacher is Abc-san." Here, among the included windows (W2-1, W2-2, ..., W2-10), four windows (W2-2, W2-3, W2-4, W2-5) are "joy" three windows (W2-8, W2-9, W2-10) contain one emotion word (“Abc-san”) related to “sadness”. There is. Further, the other windows (W2-1, W2-6, W2-7) do not contain any emotion words.

As a result, the emotion estimation unit 115
(a) For each of the four windows (W2-2, W2-3, W2-4, W2-5), it is the emotion type that appears most frequently (in this example, it appears only in each window). Decide “joy” as the emotion type of the window,
(b) For each of the three windows (W2-8, W2-9, W2-10), select “sadness”, which is the emotion type that appears most frequently (in this example, it appears only in each window). , determine the emotion type of the window,
(c) For each of the other windows (W2-1, W2-6, W2-7), the emotion type "neutral" is determined as the emotion type of the window.

In addition, the emotion estimation unit 115 determines that this group corresponds to the second window group W2-1 to W2-10 because this group has the largest number of windows (four) corresponding to the emotion type "joy". The emotion type of the sentence "I have English class, but my teacher is Abc-san." may be determined to be "joy." Or, since there is no emotion type for which the number of applicable windows exceeds 50% (within the group) (even ``joy'' has 40%), the emotion type of this sentence is determined to be ``neutral.'' Alternatively, the setting may be such that the emotion type cannot be determined.

Next, a process of determining the emotion type for each word included in the sentence “I have English class, but my teacher is Abc-san.” to which this second window group W2-1 to W2-10 corresponds. will be explained using FIG.

As shown in FIG. 5(A), in the second window group, there are one window (W2-1), four windows (W2-2, W2-3, W2-4, W2-5), and two windows. The emotion types of the window (W2-6, W2-7) and the three windows (W2-8, W2-9, W2-10) are "neutral", "joy", "neutral", and "sad", respectively. It has been decided.

First, the process of determining the emotion type for the word "I" included in the sentence to which the second window group corresponds will be described. As shown in FIG. 5B, there are three windows that include the word "I": W2-1, W2-2, and W2-3. Further, among these three windows, two windows include the specified emotional word "English". Therefore, in these three windows, the frequency of appearance of the emotional word "English" is approximately 67% (=2/3). As a result, the emotion word "English" corresponds to the set condition: "an emotion word with an appearance frequency of more than 50%," so the emotion estimation unit 115 determines the emotion type of the word "I" in this embodiment by The emotion type ``joy'' (corresponding to the emotion word ``English'') has been determined.

Next, a process for determining the emotion type for the word "my" included in the sentence corresponding to the second window group will be described. As shown in FIG. 5C, there are four windows that include the word "my": W2-5, W2-6, W2-7, and W2-8. here,
(a) Among these four windows, only one contains the specified emotional word "English". Therefore, in these four windows, the frequency of appearance of the emotional word "English" is 25% (=1/4). Also,
(b) Among these four windows, one also includes the specified emotion word "Abc-san." Therefore, in these four windows, the frequency of appearance of the emotional word "Abc-san" is also 25% (=1/4).

As a result, since neither the emotional word "English" nor the emotional word "Abc-san" falls under the set condition: "emotional word with an appearance frequency exceeding 50%," the emotional estimation unit 115 uses the word The emotion type of “my” is determined to be “neutral,” which is not a prominent emotion type (“anger,” “joy,” “sadness,” and “surprise”). Note that as a modification, it is also possible to make it impossible to determine the emotion type for the word "my" in such a case.

Here, if we assume that the emotion word "English" is included not only in window W2-5 but also in windows W2-6 and W2-7, the appearance frequency of the emotion word "English" is 75% (=3/ 4), so it falls under the set condition: "Emotion word with an appearance frequency of over 50%." In this case, the emotion type of the word "my" is determined to be the emotion type "joy" associated with the emotion word "English" (despite the fact that window W2-8 includes the emotion word "Abc-san"). will be done. Furthermore, if there are a plurality of emotion words whose frequency of appearance exceeds 50%, the emotion type associated with the emotion word with the highest frequency of appearance among them may be adopted. Furthermore, if there are multiple emotion words that have a frequency of occurrence exceeding 50% and have the maximum value (for example, when all of them have a frequency of 75%), the emotion type "neutral" may be adopted, or the emotion type It may not be possible to make a decision.

Furthermore, using the method described above, it is possible to determine the emotion type for other words included in the sentence. For example, for the word "but", the frequency of appearance of the emotion word "English" is 50% (= 2/4), so the emotion type is determined to be "neutral" (or the emotion type cannot be determined) . Furthermore, for example, for the word "teacher", the frequency of appearance of the emotion word "Abc-san" is 50% (= 2/4), so the emotion type is determined to be "neutral" (or the emotion type cannot be determined). ).

As a modification, in the above-described process of determining the emotion type of a word included in the sentence, a window determined to be "neutral" may not be considered in calculating the frequency of appearance of an emotion word. For example, in determining the emotion type of the word "I", among the three windows (W2-1, W2-2, W2-3) in FIG. 5(B), as shown in FIG. 5(A), " The window W2-1 determined to be "neutral" does not need to be included in the calculation of the appearance frequency of the emotion word "English". In this case, the frequency of appearance of the emotional word “English” is 100% (=2/2), and the emotional word “English” falls under “emotional words with an appearance frequency of more than 50%,” so as a result, the word “I The emotion type of ``is determined to be ``joy''.

In addition, in this modification mode, when determining the emotion type of the word "my", one of the four windows (W2-5, W2-6, W2-7, W2-8) in FIG. There are two determined windows, W2-6 and W2-7. Therefore, the frequency of appearance of emotion words is calculated in the remaining two windows (W2-5, W2-8). As a result, both the emotional word "English" and the emotional word "Abc-san" have an appearance frequency of 50% (= 1/2), which does not fall under the category of "emotional words with an appearance frequency of over 50%." Therefore, the emotion type of the word “my” is determined to be “neutral” (or the emotion type cannot be determined).

Furthermore, in this modification, for the word "but" mentioned above, the frequency of appearance of the emotion word "English" is 100% (=2/2), so the emotion type is determined to be "joy". Also, regarding the word "teacher" mentioned above, the frequency of appearance of the emotion word "Abc-san" is also 100% (=2/2), so the emotion type is determined to be "sadness".

In addition, as a further modification, in the process of determining the emotion type of the word included in the sentence, the frequency of appearance in the window containing the word to be determined is the highest (for example, the frequency of appearance is 25% as the maximum value) The emotion type related to the emotion word may be determined as the emotion type of this word. In this case, the absolute value of the frequency of appearance does not serve as a criterion for determining the emotion type. In addition, if there are multiple emotion words that all have the highest frequency of occurrence (for example, they all have a maximum frequency of 25%), the emotion type of this word may be determined to be "neutral." (or it may be assumed that the emotion type cannot be determined).

Returning to the functional block diagram of FIG. 1, in this embodiment, the emotion estimation unit 115 uses information on the emotion type determined for each word, each window, and/or (each) sentence related to the estimation target data to It can be made into a data set with words, etc., and output to the emotion estimation result storage unit 105 for storage and management. Alternatively, the data may be output to the user interface unit 106 via the input/output control unit 122 and displayed on the display of the user interface unit 106.

Furthermore, the data set of emotion type information determined in this way and corresponding words, etc. is linked with a user ID and is transmitted from the communication interface unit 101 to an external information processing device via the communication control unit 121. may be used in the device. Furthermore, this data set may further be associated with the frequency of appearance of emotion words related to the determined emotion type.

Furthermore, the emotion estimation unit 115 uses the emotion estimation results (emotion-related information). This data set contains information about the probability that the words, etc. that are the target of emotion estimation correspond to each emotion type.

As described in detail above, according to the present invention, emotional dictionary information (as a component of an emotional dictionary) for generating or updating an emotional dictionary for a predetermined target is automatically generated from voice data of a predetermined target. can be generated and output automatically. Although this is a preferred embodiment, it is also possible to use a generated or updated "emotion dictionary" for a predetermined target to estimate the target's emotion from the estimation target data related to this target. Become.

The emotion dictionary generation/updating function and the emotion estimation function according to the present invention are suitable for business fields and specific areas where there is a need to understand the emotional state of a target person (for example, in real time) from the target person's utterances and written contents. It can be of great help in fields such as medicine, nursing care, education, various public services, and marketing. For example, in counseling in psychosomatic medicine, etc., it is possible to obtain emotional information about a patient that is useful for treatment and advice, based on the patient's utterances and writings, using an emotional dictionary created specifically for the patient. Furthermore, for example, the conversation robot identifies the user with whom it is interacting (for example, based on facial information), selects a pre-generated emotion dictionary exclusively for this user, and uses the selected emotion dictionary to identify the user. By estimating the user's emotional state, it is also possible to perform actions and behaviors that match the user's emotional state.

In addition, for example, in order to provide children with high-quality education that is tailored to their individual emotional tendencies, the present invention may be used to estimate children's emotional states and emotions (from children's utterances and writings). Information on trends and history can also be utilized. In other words, the present invention contributes to Goal 4 of the Sustainable Development Goals (SDGs) led by the United Nations, "Provide inclusive, equitable and quality education for all and promote lifelong learning opportunities." It is also possible to do so.

Furthermore, in order to provide adults with decent work that does not harm the environment and jobs that are of high quality and match their individual emotional tendencies, the present invention It is also possible to utilize information about adults' emotional states, emotional trends, and histories that are estimated (from adults' utterances and writings). In other words, according to the present invention, it is also possible to contribute to Goal 8 of the SDGs led by the United Nations, "Promote inclusive and sustainable economic growth, employment and decent work for all." be.

Furthermore, in order to provide consumers with education on sustainable consumption and lifestyles in line with their individual emotional tendencies, the present invention can be used to estimate (from consumers' utterances and written materials) It is also possible to utilize information on consumers' emotional states, emotional trends, and history. In other words, according to the present invention, it is also possible to contribute to Goal 12 of the SDGs led by the United Nations, "Ensure sustainable consumption and production patterns."

Regarding the various embodiments of the present invention described above, various changes, modifications, and omissions within the scope of the technical idea and viewpoint of the present invention can be easily made by those skilled in the art. The above explanation is just an example and is not intended to be any restriction. The invention is limited only by the claims and their equivalents.

1 Smartphone (emotion estimation device, emotion dictionary generation device)
101 Communication interface unit 102 Microphone 103 Voice data storage unit 104 Text data storage unit 105 Emotion estimation result storage unit 106 User interface (U/I) unit 111 Text conversion unit 112 Emotion determination unit 113 Dictionary generation unit 114 Emotional phrase etc. identification unit 115 Emotion estimation section 121 Communication control section 122 Input/output control section 2 Emotion dictionary database (DB)
3 Document DB
4 Audio DB
5 SNS server 6 Email server

Claims (10)

a conversion means for converting the acquired audio data of a predetermined target into text data;
emotion determination means for determining information related to the emotion of the target from the voice data using an emotion identification model;
Emotion dictionary information for generating or updating an emotion dictionary in which words, phrases, clauses, or sentences of the target and information related to the emotions of the target are registered in association with words and phrases included in the text data. An emotion dictionary generation program characterized by causing a computer to function as a dictionary generation means for generating and outputting emotion dictionary information in which clauses or sentences are associated with information related to determined emotions. The dictionary generating means uses a set of the text data and the information related to the emotion determined from the audio data related to the text data for which the probability of emotion identification is high until a predetermined condition is satisfied. 2. The emotion dictionary generation program according to claim 1, wherein the emotion dictionary generation program generates the emotion dictionary information. Using the emotion dictionary generated or updated by the emotion dictionary generation program according to claim 1 or 2, information related to emotions regarding estimation target data including words, phrases, clauses, or sentences expressed by the target is estimated. A program,
Among the words, phrases, clauses, or sentences included in the acquired estimation target data, a predetermined condition is applied to one of the plurality of preset types of information related to the emotion registered in the emotion dictionary. A means for identifying words, phrases, clauses, or sentences that are strongly associated with each other to meet the criteria, such as emotional phrases;
Information regarding the presence or absence of the specified word, phrase, clause, or sentence in the estimation target data, and if yes, at least one corresponding word, phrase, clause, or sentence that is associated with the specified word, phrase, clause, or sentence. The computer is characterized by functioning as an emotion estimation means that determines emotion-related information about the estimation target data or about a word, phrase, clause, or sentence included in the estimation target data based on the emotion-related information. Emotion estimation program. The emotion word/phrase identification means selects one of the plurality of preset types of emotion-related information among the words, phrases, clauses, or sentences included in the estimation target data, based on the average degree of correspondence. 4. The emotion estimation program according to claim 3, wherein words, phrases, clauses, or sentences that are strongly associated with each other in a manner that significantly deviates from each other until a predetermined condition is satisfied are identified. The emotion estimation means includes:
Sequentially slide the window within the estimation target data, and if at least one specified word, phrase, clause, or sentence exists in the window at each position, the specified word, phrase, clause Alternatively, among the information related to the one emotion associated with the sentence, one that appears frequently until a predetermined condition is met is determined as the information related to the emotion for the window,
For each word, phrase, clause, or sentence included in the estimation target data, information related to emotion determined for at least one window containing the word, phrase, clause, or sentence until a predetermined condition is met. 4. The emotion estimation program according to claim 3, wherein a word with a high frequency of appearance is determined as information related to emotion for each word, phrase, clause, or sentence included in the estimation target data. When the estimation target data includes a plurality of sentences, the emotion estimation means adds a padding element before and after each sentence included in the estimation target data, and then adds a padding element to each sentence included in the estimation target data. Emotion estimation according to claim 5, characterized in that the windows are sequentially slid and, for each sentence, information related to the emotion of each window that is sequentially slid inside the sentence including the given padding element is determined. program. a conversion means for converting the acquired voice data of a predetermined target into text data;
emotion determination means for determining information related to the emotion of the target from the voice data using an emotion identification model;
Emotion dictionary information for generating or updating an emotion dictionary in which words, phrases, clauses, or sentences of the target and information related to the emotions of the target are registered in association with words and phrases included in the text data. 1. An emotional dictionary generation device comprising: dictionary generation means for generating and outputting emotional dictionary information in which clauses or sentences are associated with information related to determined emotions. A device for estimating information related to emotions regarding estimation target data including words, phrases, clauses, or sentences expressed by a predetermined target,
a conversion means for converting the acquired target audio data into text data;
emotion determination means for determining information related to the emotion of the target from the voice data using an emotion identification model;
Emotion dictionary information for generating or updating an emotion dictionary in which words, phrases, clauses, or sentences of the target and information related to the emotions of the target are registered in association with words and phrases included in the text data. , dictionary generation means for generating and outputting emotion dictionary information in which clauses or sentences are associated with information related to the determined emotion;
Among the words, phrases, clauses, or sentences included in the acquired estimation target data, those that are registered in the emotion dictionary generated or updated using the emotion dictionary information and that relate to multiple preset types of emotions. A means for specifying emotional phrases, etc., for specifying words, phrases, clauses, or sentences that are strongly associated with one of the pieces of information to meet a predetermined condition;
Information regarding the presence or absence of the specified word, phrase, clause, or sentence in the estimation target data, and if yes, at least one corresponding word, phrase, clause, or sentence that is associated with the specified word, phrase, clause, or sentence. An emotion characterized by having an emotion estimation means that determines emotion-related information about the estimation target data or about a word, phrase, clause, or sentence included in the estimation target data based on the emotion-related information. Estimation device. a conversion means for converting the acquired audio data of a predetermined target into text data;
emotion determination means for determining information related to the emotion of the target from the voice data using an emotion identification model;
Emotion dictionary information for generating or updating an emotion dictionary in which words, phrases, clauses, or sentences of the target and information related to the emotions of the target are registered in association with words and phrases included in the text data. An emotion dictionary generation system comprising: dictionary generation means for generating and outputting emotion dictionary information in which clauses or sentences are associated with information related to determined emotions. converting the acquired voice data of a predetermined target into text data, and determining information related to the emotion of the target from the voice data using an emotion identification model;
Emotion dictionary information for generating or updating an emotion dictionary in which words, phrases, clauses, or sentences of the target and information related to the emotions of the target are registered in association with words and phrases included in the text data. , generating emotion dictionary information that associates clauses or sentences with information related to the determined emotion, and generating or updating the emotion dictionary using the emotion dictionary information, by a computer. The emotion dictionary generation method to be implemented.

Images