RU2762702C2 - System and method for automated assessment of intentions and emotions of users of dialogue system - Google Patents

Abstract

FIELD: computer technology.

SUBSTANCE: invention relates to the field of computer technology. This result is achieved by the fact that the processing of the input text data is performed, in which the data is cleared, normalized and tokenized; form at least one supply vector based on the received data tokens; sentiment analysis is performed, in which the type of sentence is determined based on the mentioned vectors of sentences using their processing by a machine learning model, while the type of sentence is: negative, positive, neutral or conversational; performing the extraction of dialog acts, in which the general intention in the incoming proposals is determined by processing the said vector of the proposal with a machine learning model; process the mentioned tokens of sentences obtained as a result of the analysis of sentiment and dialog acts in order to identify at least one of: subject, object, action or their combinations in each sentence and determine a specific intention and / or reason, as well as emotions of the object and subject based on the processing of said offer tokens.

EFFECT: invention is aimed at providing real-time automated analysis of user messages to select the most relevant response for an automatic response from the dialogue system.

14 cl, 3 dwg

Description

FIELD OF TECHNOLOGY

[0001] The present invention relates to the field of computer technology, in particular to solutions for the implementation of dialogue systems for automated assessment of the user's intentions and emotions.

LEVEL OF TECHNOLOGY

[0002] Determining the intentions and emotions of the user - information that can significantly improve the quality of message processing in conversational systems. Similar analysis mechanisms are now present in many popular chat bots, for example, Gunrock (Chen C.Y. et al. Gunrock: Building A Human-Like Social Bot By Leveraging Large Scale Real User Data. - 2018. https: // pdfs. semanticscholar.org/b402/b85ad45e3ac51flda8ee718373082ce24f47.pdf)

[0003] The architecture of the dialog system includes sub-modules responsible for classifying and extracting given commands from the user (intent), emotions (sentiment), highlighting topics of the user's conversation (topic modeling). The system shows high quality metrics, however, it operates only with specified sets of emotions, commands, topics, while user behavior is not deterministic and can potentially cover an unlimited number of topics.

[0004] Another example of this kind of systems is the Xiaoice solution (Zhou L. et al. The design and implementation of Xiaoice, an empathetic social chatbot // arXiv preprint arXiv: 1812.08989. - 2018. https://arxiv.org/abs/ 1812.08989). This dialog system also operates with submodules responsible for the classification of commands from the user (intent), emotions (sentiment), highlighting the topics of the user's conversation (topic modeling), highlighting a given set of intentions / dialog acts (dialog acts). It also has a dedicated storage system for user opinions and related objects.

[0005] The system shows high quality metrics and regularly updates the architecture due to a large amount of feedback from users. However, it also operates only with specified sets of emotions, commands, topics, while user behavior is not deterministic and can potentially cover an unlimited number of topics.

[0006] At the same time, there are neural network models with an open set of possible conclusions (including potentially - intentions, emotions), in particular, the Event2Mind model (Rashkin N. et al. Event2mind: Commonsense inference on events, intents, and reactions // arXiv preprint arXiv: 1805.06939. - 2018.https: //arxiv.org/abs/1805.06939). This system solves the problem of logical inference and generates possible emotional states of a person (open list) correlated with the events described in the text in natural language.

[0007] At the input, the system is given a text describing an event (for example, Person X drinks coffee in the morning), the system generates intentions, or the reason for the event (Person X wants to cheer up / wake up or Person X falls asleep) and the psycho-emotional state and reactions of the subjects events (Person X feels tired). A potentially trained system is able to correlate expressed emotions and states with the subject and object (if they are present in the message). The authors of this solution have collected a dataset for the English language, which contains about 25,000 events and the corresponding reactions and intentions of the participants in the events. The system is capable of generating a relevant response and shows high quality metrics on the test sample. However, such a dataset does not exist for the Russian language, and the morphology of the Russian language is richer, which can potentially cause problems for generation.

SUMMARY OF THE INVENTION

[0008] To solve the existing technical problem, the claimed system architecture and method of its operation is proposed, which is capable of real-time analyzing a message from a user for not only the actual meaning of the utterance, but also emotions, sub-emotions, causes of events in the utterance, an open list of intentions, and based on this information, responding to maintain a conversation with the user is more valid and empathic.

[0009] The technical result is to provide real-time automated analysis of user messages to select the most relevant response for an automatic response from the dialog system.

[0010] The claimed result is achieved using a system for automated assessment of intentions and emotions of users of the dialogue system, which contains:

at least one processor;

at least one memory means;

a text preprocessing module capable of processing input data, in which text data is cleaned, normalized and tokenized;

a vectorization module that provides the formation of a vector of sentences based on tokens transmitted from the text preprocessing module;

a sentiment analysis module, configured to determine the type of sentence based on the resulting vector using a machine learning model, wherein the type of sentence is: negative, positive, neutral or conversational;

a module for extracting dialog acts, configured to determine the general intention in incoming proposals by processing said proposal vector with a machine learning model;

an event handling module, configured

processing tokens of proposals in order to identify at least one of: subject, object, action, or their combination, in each proposal;

determination of a specific intention and / or reason, as well as the emotions of the object and the subject, based on the processing of the said offer tokens.

[0011] In one of the particular examples of the implementation of the system, tokenization breaks the input data into words, numbers and / or punctuation marks.

[0012] In another particular example of the implementation of the system, the division of sentences is carried out by spaces and punctuation marks, taking into account abbreviations and initials.

[0013] In another particular example of the system implementation, when clearing data, tokens that are not alphabet letters, punctuation marks and / or numbers are removed.

[0014] In another particular example of the implementation of the system, the text preprocessing module during normalization performs correction of typos in tokens.

[0015] In another particular example of the system implementation, the text preprocessing module performs token lemmatization during normalization.

[0016] In another particular example of the implementation of the system, the text preprocessing module during normalization builds a syntactic graph for each sentence to determine the binary relations of control, agreement and contiguity between the words of the sentences.

[0017] The claimed technical result is also achieved using a method for automated assessment of the intentions and emotions of users of the dialogue system, which is performed using a processor and contains the steps in which:

perform processing of the input text data, in which

data cleaning, normalization and tokenization;

form at least one supply vector based on the received data tokens;

sentiment analysis is performed, in which the type of sentence is determined based on the mentioned vectors of sentences using their processing by a machine learning model, while the type of sentence is: negative, positive, neutral or conversational;

performing the extraction of dialog acts, in which the general intention in the incoming proposals is determined by processing the said vector of the proposal with a machine learning model;

processing said tokens of the proposals in order to identify at least one of: subject, object, action, or combinations thereof, in each proposal; and

determine the specific intent and / or reason, as well as the emotions of the object and the subject, based on the processing of said offer tokens.

[0018] In one of the particular examples of the implementation of the method, during tokenization, the input data is divided into words, numbers and / or punctuation marks.

[0019] In another particular example of the implementation of the method, the division of sentences is carried out by spaces and punctuation marks, taking into account abbreviations and initials.

[0020] In another particular example of the implementation of the method, when clearing data, tokens that are not alphabet letters, punctuation marks and / or numbers are removed.

[0021] In another particular example of the implementation of the method, during text normalization, typos in tokens are corrected.

[0022] In another particular example of the method implementation, during text normalization, morphological analysis and lemmatization of tokens are performed.

[0023] In another particular example of the implementation of the method, during the normalization of the text, a syntax graph is built for each sentence to determine the binary relations of control, agreement and contiguity between the words of the sentences.

BRIEF DESCRIPTION OF DRAWINGS

[0024] FIG. 1 illustrates a general view of the claimed system.

[0025] FIG. 2 illustrates the architecture of the event handling module.

[0026] FIG. 3 illustrates a general view of a computing device.

CARRYING OUT THE INVENTION

[0027] FIG. 1 shows the architecture of the claimed system (100) for automated assessment of intentions and emotions of users of a dialogue system, which can be implemented on the basis of a computing device, for example, a personal computer, a server, etc. System (100) contains a set of modules providing processing of incoming replicas from the user (10), which interacts with the dialogue system (110).

[0028] The text preprocessing module (101) is a module performing the primary processing of user requests, in particular, it performs mandatory sequential procedures for cleaning and preparing incoming text from the dialogue system when conducting a conversation with a user. The preprocessing module (101) performs the division of the incoming text into sentences and their tokenization. Tokenization is understood as splitting the text into minimum functional units, tokens, which are words, numbers and punctuation marks.

[0029] The input data is supplied to the input of the module (101) in txt format. The user (10) can interact with the dialogue system (110) using voice commands, for example, if the voice system (110) is a voice system. In this case, the voice commands of the user (10) are converted into text format using known solutions in the prior art, for example, Google Speech-to-Text. Using open libraries in the python3 language, the resulting text is divided into sentences (using https://pypi.org/project/rusenttokenize/) and the sentences are divided into tokens by breaking sentences by spaces and separating punctuation marks from them.

[0030] After the primary processing of the text, the module (101) generates proposals, in which the lists of tokens of the proposal.

Example:

"All people are mortal. Socrates is a man. Therefore, Socrates is mortal." □ [["all", "people", "mortal", "."], ["Socrates", "-", "man", "."], ["Hence", ",", "Socrates" , "mortal", "."]].

In this system, a token is considered not a string data type, but an object of the Token class, which has the following attributes:

asis - this is where the token text is stored "as is"

morph - the result of morphological analysis of the token (normal form of a word, part of speech, case, etc.)

spellcheck - the result of the spellcheck (from the English Spellcheck) token.

If the spellchecking module did not detect an error, this attribute will be empty, if it does, it will contain the most likely fix.

Start_index, end_index - the ordinal number in the text of the first and last character in the Coref token - a field for coreference (the relationship of the token with others within the text, if the token is a pronoun). The field is empty by default.

Pseudocode:

[0031] Next, using the module (101), the text is cleared from special characters. During this procedure, incoming tokens are filtered from special characters that are not included in the list of Cyrillic and Latin letters, numbers and symbols from a standard 105-key keyboard.

[0032] After clearing the text from special characters, the module (101) performs typo correction. At this stage, incoming tokens are checked for typos and corrected. Open technologies are used - a model based on the distance Damerau Levenshtein 1 (Model from the DeepPavlov Damerau Levenshtein 1 + lm repository). When performing the procedure for correcting typos, the list of objects of the Token class is processed, for which, as a result of the check, the spellcheck attribute is filled in those tokens where a typo is possible.

[0033] After that, the module (101) determines the parts of speech (morphological analysis), in which the tokens are normalized. During normalization, tokens are reduced to the nominative singular form for nouns, the infinitive for verbs, etc., as well as the morphological characteristics of the token form (case, number, person, gender, etc.) are fixed. The analysis is carried out using open technologies - the RnnMorph library on recurrent neural networks https://pypi.org/project/rnnmorph/. This neural network architecture provides the best analysis quality for the Russian language. Based on the analysis results, the following information is saved for each token:

• Part of speech (POS, part of speech);

• Morphological characteristics (fulltag);

• Normal form of lemmatization (lemma).

Analysis example:

Asis

"tokens"

Morph:

[{"grammar": {'POS'i'NOUN', 'fulltag': 'Case = Nom | Gender = Masc | Number = Plur', 'wordform': 'tokens', 'lemma': 'token'}] ...

[0034] Next, using the module (101), parsing is performed. During this stage, a higher-level analysis of the incoming text is performed - it builds a syntactic graph on each sentence, defining binary relations of control, agreement and contiguity between words. The open source Udpipe library https://pypi.org/project/ufal.udpipe/ is used for parsing. In the course of the analysis, lists of vertices and edges of the syntactic graph are formed for the sentence assembled from the list of tokens passed to the input. As a result of processing, a row-by-row matrix of syntactic graph edges is formed on the original list of tokens, String. The parsing information can be used optionally and is stored in binding to the offer token list.

[0035] The vectorization module (102) receives as input the text processed by the module (101), and performs its transformation into a vector using the ELMo model. The ELMO model provides the formation of a vector for Russian sentences. The resulting vector containing contextual information about the order of words in the text and their importance is then transmitted to the input of modules (103) and (104).

[0036] The ELMo text vectorization model (https://allennlp.org/elmo) for the Russian language is pre-trained on huge data bodies and is an architecture that stores multi-level information about the language - symbolic, lexical and semantic (information about the context). Due to this, it is possible to make high-level generalizations about the properties of incoming phrases even in the absence of a large number of examples when training the classifier.

[0037] The sentiment analysis module (103) is executed based on the neural network model of the classifier, trained to determine the emotions of the message. The determination is performed according to a given list of classes recognized by the module model (103). The use of the module (102) allows you to more empathically respond to the user's messages in the course of his interaction with the dialogue system, accumulate his opinions on various issues, change the tone of the message in response, influence the negative, trying to translate into more positive aspects, etc. One of the examples of system implementation (100) can use the sentiment analysis model from the open repository DeepPavlov (https: //github.eom/deepmipt/DeepPavlov/blob/0.2.0/deeppavlov/configs/classifiers/rusentiment_elmo.json).

[0038] The convolutional neural network of the module (103) is trained to assign to the sentence vector a certain class selected from the group: 'positive', 'negative', 'neutral', 'speech', 'skip'.

• Positive, neutral, negative - for strongly positive, neutral and negative emotions

• Speech - for neutral conversational phrases ("Hello, bye, thanks")

• Skip - for cases implicit for the model (for which it is impossible to perform classification).

[0039] The neural network-based model in module (103) is trained on data from the open source RuSentiment (contains texts of social media in Russian). Since the model works with vectors of whole sentences and short texts, the generalizing ability of the model allows it to perceive the entire text, and not be tied to specific words, since their meaning in the context can vary greatly: warm (positive sentiment) + beer (neutral sentiment) = warm beer (negative sentiment).

[0040] The module for extracting dialog acts (104) is also a neural network architecture of a classifier that can define on a message from 1 to 13 different dialog acts, that is, classes that determine intentions in a user's message - an informative question, a question about an opinion, a personal opinion, cause, effect, argument, agreement, disagreement, apology, response to an apology, gratitude, response to gratitude, greeting, goodbye, etc. Module (104) accepts as input a sentence vectorized using module (102). Further, the vectorized proposal is processed by the classifier to recognize the general intention of the user in the incoming proposals.

[0041] The module classifier model (104) operates on a convolutional neural network trained to assign 1 to 13 class labels to a sentence vector (they are not mutually exclusive, multiclass labeling).

[0042] The model is trained on examples of custom dialogs accumulated during testing of dialog systems and open data dialogs from OpenSubtitles, tagged with rules and trained markers. Based on the results of the processing of the vector of proposals by the module (104), a list of labels of the proposal is formed.

[0043] The module for extracting dialog acts (104) allows more accurate processing of information about the pragmatics of the statement (with an accuracy of 92%) and the correct response to the user's message. For example, saying "how much does this phone cost?" requires an exact answer, the statement "how much can such a phone cost?" requires a value judgment in response (including about the characteristics of the phone, and not about the price), and the statement "but what is it that phones cost so much!" requires a reaction to negativity. On the given examples, the module classifier model (104) gives various classes - an informative question, a question about an opinion, a personal opinion.

[0044] The event processing module (105), built on the architecture of the event2mind model, extracts the event from the text and generates a relevant description of the causes of the event and the psychoemotional state of the event participants in the moment. FIG. 2 shows an example of the architecture of the mentioned module.

[0045] The module (105), upon processing the tokens of the user's proposals (10), is able to determine a specific intention and / or reason, as well as the emotions of the object and subject of the proposal. Events are highlighted using a syntactic parser (UDPipe model for Russian, trained on the syntagrus http://ufal.mff.cuni.cz/udpipe), using templates of the form "subject" (subj) + "action" (verb - verb) + " object "(obj). Events of this type (“I want coffee” → “X (subj) tired”) are extracted from the user's text and fed to the neural network of the module (105). Receiving the event text as input, the model aims to generate three pragmatic conclusions: the intention / causes of the event, the reaction of Person X, and the reactions of other participants in the event (Person Y).

[0046] The operation of the neural network of the module (105) is as follows: 1) the input is a description of the event in the form of text; 2) the event is encoded into the vector hE representation by an autoencoder; 3) then several RNN decoders are used, which accept a vector representation of an event as input and generate text entities. For example, an event intent / cause sequence (vi = vi ^ (0) vi ^ (1) ...) is calculated as follows: vi ^ (t + 1) = softmax (Wi RNN (vi ^ (t), hi, dec ^ ( t)) + bi).

[0047] The event processing module (105) allows both to take the results of the modules (103) and (104) as features for training, and to autonomously generate new replicas reflecting the user's emotional state (10). For the module (105) to work, a tagged dataset for the Russian language with a description of events and pragmatic conclusions from them is required. The neural encoder-decoder model has shown that it can successfully compose vector representations of events that were not previously present in the training sample and generate probabilistic intentions and reactions of event participants.

[0048] The system (100) can receive as input not only the preprocessed text of the user (10), but also the context - the previous messages in the conversation. It is possible to implement both the response directly, through the event processing module (105), which specifies the emotional state of a person (empathic message), and to use the information received during the operation of modules (103) - (105) for a more correct choice of response, the implementation of various modules of proactivity in dialogue system (110), accumulation of knowledge about events in the user's life, etc.

[0049] Potentially the claimed system (100) can be built into modern architectures of dialog systems (110) and supplement and improve their quality of work; the system can also work autonomously and its data based on the analysis of a large array of user reviews can be useful for analytics, tracking trends, helping call center operators, etc.

[0050] Various solutions can act as the dialogue system (110), for example, voice assistants, chat bots, robotic call centers, and other technologies that embody an automated process of communicating with a user.

[0051] FIG. 3 shows a general view of the computing device (200). On the basis of the device (200), a user device can be implemented for forming a communication process with a dialogue system (110), a system (100) for implementing the claimed solution and other unrepresented devices that can participate in the general information architecture of the claimed solution.

[0052] In the General case, the computing device (200) contains one or more processors (201) united by a common data exchange bus, memory means such as RAM (202) and ROM (203), input / output interfaces (204), devices input / output (205), and a device for networking (206).

[0053] The processor (201) (or multiple processors, multi-core processor) can be selected from a range of devices currently widely used, for example, Intel ™, AMD ™, Apple ™, Samsung Exynos ™, MediaTEK ™, Qualcomm Snapdragon ™ and etc. The processor (201) can also include a graphics processor or work in conjunction with a graphics accelerator, for example, Nvidia, AMD Radeon, etc., which can be used to perform computational operations when executing machine learning algorithms.

[0054] RAM (202) is a random access memory and is intended for storing machine-readable instructions executed by the processor (201) for performing the necessary operations for logical data processing. RAM (202), as a rule, contains executable instructions of the operating system and corresponding software components (applications, software modules, etc.).

[0055] ROM (203) is one or more persistent storage devices, such as a hard disk drive (HDD), solid state data storage device (SSD), flash memory (EEPROM, NAND, etc.), optical storage media ( CD-R / RW, DVD-R / RW, BlueRay Disc, MD), etc.

[0056] Various types of I / O interfaces (204) are used to organize the operation of the components of the device (200) and to organize the operation of external connected devices. The choice of the appropriate interfaces depends on the specific version of the computing device, which can be, but are not limited to: PCI, AGP, PS / 2, IrDa, FireWire, LPT, COM, SATA, IDE, Lightning, USB (2.0, 3.0, 3.1, micro, mini, type C), TRS / Audio jack (2.5, 3.5, 6.35), HDMI, DVI, VGA, Display Port, RJ45, RS232, etc.

[0057] To ensure user interaction with the computing device (200), various means (205) I / O information are used, for example, a keyboard, display (monitor), touch display, touch-pad, joystick, mouse manipulator, light pen, stylus, touch panel, trackball, speakers, microphone, augmented reality, optical sensors, tablet, light indicators, projector, camera, biometric identification (retina scanner, fingerprint scanner, voice recognition module), etc.

[0058] The means of networking (206) allows the device (200) to transmit data via an internal or external computer network, for example, Intranet, Internet, LAN, and the like. One or more means (206) may be used, but not limited to: Ethernet card, GSM modem, GPRS modem, LTE modem, 5G modem, satellite communication module, NFC module, Bluetooth and / or BLE module, Wi-Fi module and dr.

[0059] Additionally, satellite navigation aids can also be used as part of the device (300), for example, GPS, GLONASS, BeiDou, Galileo.

[0060] The presented application materials disclose preferred examples of the implementation of the technical solution and should not be construed as limiting other, particular examples of its implementation, not going beyond the scope of the claimed legal protection, which are obvious to specialists in the relevant field of technology.

Claims (29)

1. A system for the automated assessment of the intentions and emotions of users of the dialogue system, containing at least one processor; at least one memory means; a text preprocessing module capable of processing input data, in which text data is cleaned, normalized and tokenized; a vectorization module that provides the formation of a vector of sentences based on tokens transmitted from the text preprocessing module; a sentiment analysis module, configured to determine the type of sentence based on the resulting vector using a machine learning model, wherein the type of sentence is: negative, positive, neutral or conversational; a module for extracting dialog acts, configured to determine the general intention in incoming proposals by processing said proposal vector with a machine learning model; an event handling module, configured processing tokens of sentences received from the sentiment analysis module and the module for extracting dialog acts in order to identify at least one of: subject, object, action or their combinations in each sentence; determining a specific intention and / or reason, as well as the emotions of the object and the subject, based on the processing of the said offer tokens. 2. The system according to claim 1, characterized by the fact that during tokenization the input data is divided into words, numbers and / or punctuation marks. 3. The system according to claim 2, characterized by the fact that the division of sentences is carried out by spaces and punctuation marks, taking into account abbreviations and initials. 4. The system according to claim 1, characterized by the fact that when clearing the data, tokens that are not letters of the alphabet, punctuation marks and / or numbers are deleted. 5. The system according to claim 1, characterized by the fact that the text preprocessing module corrects typos in tokens during normalization. 6. The system according to claim 1, characterized by the fact that the text preprocessing module performs lemmatization of tokens during normalization. 7. The system according to claim 1, characterized by the fact that the text preprocessing module during normalization builds a syntactic graph for each sentence to determine the binary relations of control, agreement and contiguity between the words of the sentences. 8. A method for the automated assessment of the intentions and emotions of users of the dialogue system, performed with the help of a processor and containing the stages at which: processing the input text data, in which the data is cleared, normalized and tokenized; form at least one supply vector based on the received data tokens; sentiment analysis is performed, in which the type of sentence is determined based on the mentioned vectors of sentences using their processing by a machine learning model, while the type of sentence is: negative, positive, neutral or conversational; performing the extraction of dialog acts, in which the general intention in the incoming proposals is determined by processing the said vector of the proposal with a machine learning model; process the said tokens of sentences received as a result of sentiment analysis and dialog acts in order to identify at least one of: subject, object, action or their combinations in each sentence and determine the specific intent and / or reason, as well as the emotions of the object and the subject, based on the processing of said offer tokens. 9. The method according to claim 8, characterized in that during tokenization, the input data is broken into words, numbers and / or punctuation marks. 10. The method according to claim 9, characterized in that the division of sentences is carried out by spaces and punctuation marks, taking into account abbreviations and initials. 11. The method according to claim 8, characterized in that when clearing the data, tokens that are not alphabet letters, punctuation marks and / or numbers are deleted. 12. The method according to claim 8, characterized in that during the text normalization, typos in tokens are corrected. 13. The method according to claim 8, characterized in that during the normalization of the text, lemmatization of tokens is performed. 14. The method according to claim 8, characterized in that during the normalization of the text, a syntactic graph is built for each sentence to determine the binary relations of control, agreement and contiguity between the words of the sentences.

Images