RU2796208C1 - Method and system for digital assistant text generation - Google Patents

Abstract

FIELD: computer technology.

SUBSTANCE: invention relates to a method and system for generating text for a digital assistant in interactive systems. The technical result from the implementation of the claimed solution is to increase the semantic accuracy of generating stylized text from the source text. The specified technical result is achieved due to the implementation of a method for automatically generating text for a digital assistant in interactive systems, containing the following steps: receiving input data containing the source text in natural language and the target style of the digital assistant's replicas; carrying out encoding of the source text; performing token vectorization; processing vector representations of source text tokens using a machine learning model based on a neural network trained on stylized, in accordance with a given target style, text replicas of a digital assistant, during which an array of vectorised stylized texts is formed; performing decoding of each vectorised stylized text from the array, and in the course of decoding, at least converting the vectorised stylized text into tokens and detokenization are performed; performing filtering of the array of stylized texts; ranking the filtered stylized texts and selecting the best stylized text, the selection of the best text being based on the pairwise distance between the original text and each of the possible stylized texts; sending stylized text to the dialog system.

EFFECT: increase the semantic accuracy of generating stylized text from the source text.

8 cl, 3 dwg

Description

FIELD OF TECHNOLOGY

[0001] The claimed technical solution, in general, relates to the field of computer technology, and in particular to a method and system for generating text for a digital assistant in interactive systems.

BACKGROUND OF THE INVENTION

[0002] As a result of the functioning of the language, its varieties have developed, belonging to a certain style of text presentation, which has certain features, language tools, genres, etc. So, when publishing a text in a scientific journal, such a text will have features of a scientific style, while in informal communication, the text will have features of a conversational style, for example, informal appeals to “you”, simplicity of sentence construction, use of slang, etc. .

[0003] At present, with the development of information technology, speech style transfer technologies have been actively developed in the field of natural language processing (NLP) and today they are trying to integrate it into a variety of areas. Automating the process or part of the process of styling text in a particular style can provide significant efficiency gains in areas such as journalism, publishing houses, such as text editors, content creation for media platforms and virtual assistants, etc.

[0004] However, despite the demand for this technology, there are a number of difficulties that do not allow, for example, to generate stylized text with high accuracy. Thus, the problems of text styling are to ensure the safety of the original information, to ensure the absence of new facts in the generated texts, to preserve the semantic load of the original text, etc. In addition, one of the important problems is also the possibility of ensuring the universality of the technology, which allows generating text not only in one style, but also provides the ability to style the source text in several styles, depending on the scope. Therefore, the creation of an efficient and accurate way to automatically generate text in given styles is an essential task.

[0005] So, the method of transferring the text style disclosed in the source [1] is known from the prior art. This method provides the ability to generate stylized text from source text by solving a machine translation problem. In this case, the style of the text acts as the "language" into which the source text needs to be translated.

[0006] The disadvantages of this solution include the high complexity of implementation and the narrow focus of this solution, due to the huge set of required training data, and the inability to adapt to different styles due to the peculiarities of the language translation technology. In addition, this solution also does not provide high accuracy, because in the process of such a “translation”, the meaning of the original phrase may be lost due to the change in all the words of the source text.

[0007] The prior art also knows a method for providing logical answers that mimic the user's speech style, disclosed in RF patent No. RU 2693332 C1 (LIMITED LIABILITY COMPANY "YANDEX"), publ. 07/02/2019. This method provides the ability to select a contextual answer to a question, depending on the context of the question, by analyzing the vector representation of the contextual question and searching for the closest answer from the set of answers in the database.

[0008] The disadvantages of this solution are the impossibility of generating stylized text based on the source text, high computational power and a large amount of memory required to form a database (DB), limited styling by DB patterns, low accuracy of generating stylized text due to the selection of pre-created and saved in Database of stylistic responses.

[0009] The common disadvantages of existing solutions is the lack of an effective way to generate stylized text with high accuracy, ensuring the safety of the original information and the absence of new facts in the generated texts. Also, this method should ensure the preservation of the semantic load of the source text. In addition, this method should ensure the universality of the text styling technology, which allows generating text not only in one style, but also provides the ability to style the source text in several styles, depending on the scope of application.

DISCLOSURE OF THE INVENTION

[0010] The claimed technical solution proposes a new approach to generating text for a digital assistant in interactive systems. This solution uses a machine learning algorithm that allows generating stylized text for a digital assistant from the source text with high accuracy, ensuring the semantic similarity of the source and stylized text and excluding the distortion of the stylized text by new facts.

[0011] Thus, the technical problem of enabling the generation of stylized text is solved.

[0012] The technical result achieved by solving this problem is to increase the semantic accuracy of generating stylized text from the source text.

[0013] An additional technical result, manifested in solving the above problem, is the possibility of generating a plurality of stylized texts from one source.

[0014] These technical results are achieved through the implementation of a computer-implemented method for automatically generating text for a digital assistant in interactive systems, performed by at least one computing device, and comprising the steps of:

a) receive input data containing the source text in natural language and the target style of the speech of the digital assistant;

b) performing encoding of the original text, and in the course of encoding, at least the tokenization of text data is performed;

c) vectorize the tokens obtained in step b);

d) process the vector representations of the source text tokens obtained in step c), using a machine learning model based on a neural network trained on stylized, in accordance with a given target style, text replicas of a digital assistant, during which an array of vectorized stylized texts;

e) performing the decoding of each vectorized stylized text from the array obtained in step d), and during the decoding perform at least the transformation of the vectorized stylized text into tokens and detokenization;

f) filtering the array of stylized texts obtained in step e);

g) ranking the filtered stylized texts and selecting the best stylized text, with the selection of the best text based on the pairwise distance between the original text and each of the possible stylized texts;

h) sending the stylized text obtained in step g) to the dialog system.

[0015] In one of the private implementations of the method, filtering an array of stylized texts is performed using regular expressions and a morphological analyzer.

[0016] In another particular embodiment of the method, the array of stylized texts is filtered based on the matching of proper names in the source text and each stylized text from the array.

[0017] In another particular implementation of the method, proper name matching is determined using the number of unchanged named entities contained in the source text and the styled text.

[0018] In another particular embodiment of the method, the number of immutable named entities is determined based on the recognition of named entities in the source text and each styled text from the array.

[0019] In another particular implementation, the method further comprises the step of checking for text styling.

[0020] In another particular implementation of the method, the presence of text styling is checked based on determining whether the target style indicators have been replaced.

[0021] In addition, the claimed technical results are achieved through an automatic text generation system for a digital assistant in interactive systems, comprising:

•at least one processor;

• at least one memory connected to the processor, which contains machine-readable instructions, which, when executed by at least one processor, provide a method for generating text for digital assistants in conversational systems.

BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The features and advantages of the present invention will become apparent from the following detailed description of the invention and the accompanying drawings.

[0023] FIG. 1 illustrates a block diagram of a general view of the claimed system.

[0024] FIG. 2 illustrates a block diagram of the claimed method.

[0025] FIG. 3 illustrates an example of a general view of a computing device that provides the implementation of the claimed solution.

IMPLEMENTATION OF THE INVENTION

[0026] The concepts and terms necessary for understanding this technical solution will be described below.

[0027] A model in machine learning (ML) is a set of artificial intelligence methods, the characteristic feature of which is not the direct solution of a problem, but learning in the process of applying solutions to many similar problems.

[0028] Named-entity recognition (NER) is an information extraction subtask that seeks to find and classify references to named entities in unstructured text into predefined categories such as proper names, character names, organizations, locations, monetary values, percentages, etc.

[0029] Vector representation of words (word embeddings, embeddings) - a common name for various approaches to language modeling and learning representations in natural language processing, aimed at matching words (and possibly phrases) from some dictionary of vectors from an n-dimensional real space R_n.

[0030] Tokenization is the process of splitting text into text units or tokens (most often words are such units, but it can also be letters, parts of a sentence, combinations of words, etc.).

[0031] A language model is a probability distribution over a set of vocabulary sequences. In this patent, the term "language model" is used to describe neural network language models that are designed to model a language by estimating the probability of a given sequence of characters.

[0032] The claimed technical solution offers a new approach that improves the semantic accuracy of generating stylized text, which consists in maintaining the semantic content in the stylized text, as well as excluding the addition of stylized text with new facts and logical units of the language. One of the features of the claimed technical solution is the ability to generate many stylized texts from a single source text, which also automates the process of generating stylized texts and significantly reduces the time of text styling compared to manual generation of each individual stylized text.

[0033] The claimed technical solution can be implemented on a computer, in the form of an automated information system (AIS) or a machine-readable medium containing instructions for performing the above method.

[0034] The technical solution can also be implemented as a distributed computer system or computing device.

[0035] In this solution, a system means a computer system, a computer (electronic computer), CNC (numerical control), PLC (programmable logic controller), computerized control systems and any other devices capable of performing a given, well-defined sequence of computing operations (actions, instructions).

[0036] A command processing device is an electronic unit or an integrated circuit (microprocessor) that executes machine instructions (programs)/

[0037] The instruction processor reads and executes machine instructions (programs) from one or more storage devices, such as random access memory (RAM) and/or read only memory (ROM). ROM can be, but not limited to, hard drives (HDDs), flash memory, solid state drives (SSDs), optical storage media (CDs, DVDs, BDs, MDs, etc.), etc.

[0038] A program is a sequence of instructions intended to be executed by a computer control device or command processing device.

[0039] The term "instructions" as used in this application may refer, in general, to program instructions or program instructions that are written in a given programming language to perform a specific function, such as, for example, encoding and decoding texts, filtering, ranking , translation of texts into a dialogue system, etc. The instructions may be implemented in a variety of ways, including, for example, object-oriented methods. For example, instructions can be implemented using the Python programming language, C++, Java, Python, various libraries (eg MFC; Microsoft Foundation Classes), etc. Instructions that carry out the processes described in this solution can be transmitted over both wired and wireless data transmission channels, such as Wi-Fi, Bluetooth, USB, WLAN, LAN, etc.

[0040] In FIG. 1 shows a general view of a text generation system 100 for a digital assistant in conversational systems. The system 100 includes basic functional elements such as: an encoding/decoding module 101, a style transfer module 102, a stylized text filtering module 103, and a stylized text ranking module 104. The elements of system 100 are discussed in more detail in FIG. 3.

[0041] System 120 can be used as a conversational system and can be various solutions, such as voice assistants, chatbots, robotic call centers, and other technologies that implement an automated process of communicating with a user, such as user 110. It is worth noting that that under the dialogue systems in this solution should be understood any automated human-machine system operating in the dialogue mode, in which it responds to each user command and, as necessary, refers to it for information.

[0042] As digital assistants (virtual digital assistants), user interaction automation systems implemented on the basis of artificial intelligence in a dialog format (chat bot, skills for a voice assistant, etc.) can act. Thus, in one particular embodiment, the digital assistant may be a computer system that simulates a conversation with users in a conversational format.

[0043] Styling text in this solution refers to the generation of text by converting the received source text into text that has stylistic speech features. So, stylistic features can be the emotional color of the text (cheerful, sad, etc.). In another particular embodiment, stylistic speech features may be the condition and goals of communication in some area of social activity, for example, official business activities, journalistic activities, colloquial, artistic, etc. It should be noted that stylization of the text can also be the giving of characteristic features to the text, inherent in the peculiarities of communication of individual personalities, characters, literary heroes, etc., without being limited. So, in another particular embodiment, text styling can be the transformation of the source text into a stylized one, in accordance with the given communication style of a particular digital assistant, which has conversational style features, for example, the use of certain pronouns that emphasize the informal/formal communication style (“You” and "You", gender, number, etc.

[0044] Encoding/decoding module 101 may be implemented on at least one computing device equipped with appropriate software and include a set of models for tokenizing and detokenizing text, vectorizing tokenized text, and converting tokens to text, for example, one or more models machine learning for converting text information into vector form, for example, BERT, ELMo, ULMFit, XLNet, RoBerta, RuGPT3 and others. In one particular embodiment, module 101 may be implemented based on system 300, which is discussed in more detail in FIG. 3. It is worth noting that the specific method of tokenization and vectorization depends on the chosen language model, on the basis of which the module 102 is implemented. For example, when using the RuGPT3 model, tokenization is carried out using the BPE (Byte Pair Encoding) method, and subsequent vectorization - by replacing each token with its index in the dictionary of the language model, compiled at the stage of initial training of the model. In addition, in yet another particular embodiment, word tokenization may be used as a tokenization method.

An example of tokenization by words and coding of words by indexes in the dictionary: 'mom washed the frame' → ['mother', 'soap', 'frame'] → [235, 376, 1056]

[0045] Module 102 can be implemented on the basis of at least one neural network, pre-trained on specific sets of stylized, in accordance with the specified styles, texts. As a machine learning model that implements the function of generating stylized texts in accordance with a given style, for example, a generative language model such as RuGPT3, XLNet, etc. can be used. In one particular embodiment, when implementing the claimed solution, the MO was the Russian-language generative language model RuGPT3-Large. The model is trained on sources from different domains: Wikipedia, books, news, Russian Common Crawl, etc. The model was trained for 14 days on 128 GPUs with a context window of 1024 and an additional few days on 16 GPUs with a context of 2048. The final model has a perplexity of 13.8 on the test dataset. At this stage of learning, the result of learning the language model was the ability to predict the probability of the next token based on the previous initial text fragment. So, if during the training process the model often encountered a certain phrase in the training data, then when predicting the next token after the token known from the phrase, the model will predict with high probability exactly the token from the phrase in the training data set.

[0046] Further, to perform directly the process of generating stylized text from the source text, additional training of the trained model was performed. The fine-tune procedure was used to further train the model. At this stage, the weights of the trained model were adjusted in accordance with the problem being solved. So, when the original sentence appears in the model, due to the changed weight coefficients, the most likely sentence for the model will be a rephrase of this sentence in a certain style, i.e. increasing the probability of continuing a text fragment in the format in which the stylized text should be generated. When retraining the model, datasets with various stylized replicas of digital assistants were used. So, in one particular embodiment, the problem of stylizing the source text for three speech styles of digital assistants was solved. The initial data for additional training contained 2174, 2436 and 32242 cues in the style of each assistant, respectively. At the same time, the initial data contained only replicas in the style of specific assistants, but did not contain the source text from which such stylized replicas are generated.

[0047] To form a dataset (training dataset) in a format suitable for retraining the model, consisting of pairs of “original replica” - “assistant-style replica”, a paraphrase was applied to the stylized replicas, for example, a paraphrase based on the RuT5 generative model. The paraphraser model is known from the prior art and is disclosed, for example, in a source available via the Internet link: https://huggingface.co/cointegrated/rut5-base-paraphraser. For each stylized replica, 10 initial variants of the paraphrase were generated, from which the 2 closest initial replicas were then selected based on a semantic metric, for example, the LabSe metric. This metric evaluates the cosine similarity between the vector representations of sentences obtained using the model, which corresponds to semantic similarity. Thus, the final datasets for additional training of RuGPT3 for the style transfer task contained 4348, 4872, 64484 pairs of sentences (“initial replica” - “replica in the style of an assistant”). In addition, a tag was also added to the data that characterizes a particular digital assistant with the corresponding speech style to provide the ability to determine in which style the stylized text will be generated. Based on the obtained data, the model was retrained for 5 epochs.

[0048] The resulting model was evaluated on a test set separately for each assistant. As a test data set, 1097 replicas were used for each assistant from the assistant sets. The following metrics were used to evaluate the quality: 1) BLEU (Papineni, K.; Roukos, S.; Ward, T.; Zhu, W. J. (2002). BLEU: a method for automatic evaluation of machine translation (PDF). ACL- 2002: 40th Annual meeting of the Association for Computational Linguistics, pp.311-318 CiteSeerX 10.1.1.19.9416). The BLEU algorithm compares the double translation phrases with the phrases it finds in the reference variant and weights the number of matches. These matches are independent of position. A higher degree of match indicates a higher degree of similarity with the reference translation and a higher score. Intelligibility and grammar are not taken into account. 2) The number of common N-Gram, where N-Gram was taken as a sequence of words of length from 3 to 8. 3) Levenshtein - Levenshtein distance (V. I. Levenshtein. Binary codes with correction of dropouts, insertions and substitutions of characters. Reports of the Academies of Sciences USSR, 1965. 163.4:845-848.) (editing distance, editing distance). The specified metric that modulo the difference between two sequences of characters. It is defined as the minimum number of single-character operations (namely insertions, deletions, substitutions) required to transform one sequence of characters into another. 4) Jaccard index - Jaccard index -272.), which was calculated as the number of common tokens in the stylized replica and the original sentence, divided by the union of tokens in these two fragments of text.

[0049] The results of model evaluation for three digital assistants, each with a different communication style, are shown in Table 1.

[0050] Table 1 shows that the BLEU score, the number of total n-gram and the Jaccard index have a level of more than 95%. The Levenshtein distance for all assistants is also on average less than 2, which indicates that the generated sentences differ on average from the sample answers by no more than 2 characters.

[0051] Module 103 may be implemented on at least one computing device and include a morphological parser, such as a Unigram parser, an N-gram parser, a regular expression parser, and so on. Also, module 103 may include a neural network trained to solve a NER (Named Entity Recognition) problem, such as Slovnet BERT NER, DeepPavlov BERT NER, etc., without being limited. In one particular embodiment, module 103 may contain a heavy BERT model and be trained on a small hand-annotated dataset.

[0052] The stylized text ranking module 104 may be implemented on the basis of at least one computing device equipped with appropriate software for calculating the Levenshtein distance. So, the specified module 104 is configured to implement an algorithm for calculating pairwise distances between the original text and stylized texts.

[0053] Those skilled in the art will appreciate that while the modules described above are presented as separate devices, these modules can also be combined as part of a single device, such as system 300.

[0054] FIG. 2 is a block diagram of a method 200 for automatically generating text for a digital assistant in conversational systems, which is described step by step in more detail below. Said method 200 consists in performing steps aimed at processing various digital data. Processing is typically performed by a system, such as system 100, which may also represent, for example, a server, a computer, a mobile device, a computing device, and so on.

[0055] In step 210, system 100 receives input containing natural language source text and the target digital assistant speech style. Thus, input may be received from the dialog system 120 over data links such as the Internet. The source text received from the conversational system 120 may represent, for example, a conversational cue, such as an answer to a user's question, a question to the user, and so on. The target style of the digital assistant's remarks can represent certain characteristic features of the speech style, for example, a conversational style of communication, which is characterized, for example, by the presence of informal addresses to the user, a business style of communication, which is inherent in formal addresses, etc. Also, in one particular embodiment, the target speech style may indicate the emotional coloring of the text, for example, sad, cheerful, neutral, etc. The target speech style of the digital assistant may be determined by the settings of the dialog system 120. Thus, the dialog system, such as system 120, may contain a set of digital assistants, each of which has a different communication style. When starting a dialogue, the user can select a specific assistant in the system settings. Said data about the selected assistant is also transmitted to the system 100.

[0056] At step 220, the source text is encoded, wherein at least tokenization of the text data is performed during encoding. Said step 220 may be performed by module 101. The input text may be divided into tokens. A token in this solution should be understood as a sequence of characters in the text that is important for analysis. In another particular embodiment, text tokenization can be performed using the BPE (Byte Pair encoding) algorithm. In yet another particular embodiment, tokenization may be the splitting of text into words by a space between words. Next, a dictionary of fixed size tokens (for example, 30,000 tokens) is compiled, where each token is matched with its index in the dictionary.

An example of tokenization for words:

['Here's what I found on your <application' → '<Here><what><I><found><on><your><application>']

[0057] At the specified step 230, vectorization of tokenized texts is performed. As mentioned above, the method of tokenization depends on the language model, which is used in module 102 at step 240. Thus, for example, when using the RuGPT3 language model, each token is mapped to its index in the dictionary. Thus, the tokenized text fragment (the list of tokens) after vectorization is mapped to the vector of token data indices in the dictionary. Let's use vectorization for tokenization according to the words: 'mother washed the frame' → ['mother', 'soap', 'frame'] → [235, 376, 1056]

[0058] Next, method 100 proceeds to step 240.

[0059] At step 240, vector representations of source text tokens are processed using module 103, during which an array of vectorized stylized texts is formed. As mentioned above, the machine learning model based on a neural network was retrained on stylized, in accordance with a given target style, text replicas of digital assistants. At step 240, a vector representation of the original text is input to the model. At the output, the model generates several variants of stylized texts (candidates) in the form of vector representations. The specified vector representations are stored in an array of stylized texts. The number of candidates generated by the model depends on the sampling methods used. In one particular embodiment, the following sampling criteria were used for the model: top P sample (top_p=0.92), top K sample (top_k=50), temperature sample (temperature=0.85). In more detail, these sampling methods are disclosed in the source, found on the Internet at the link: https://towardsdatascience.com/how-to-sample-from-language-mod. The general principle of the model is the ability to predict the probability of the next token in a certain context. Thus, at the first stage of learning, the language model is able to predict the probability of the next token based on the previous initial text fragment. To implement the possibility of generating stylized texts by the specified model, then its weights are changed so that the probabilities for the next token that the model predicts correspond to the current task of text styling. The specified change in weights is based on the training data set. The retrained model is then able to generate from the source text a set of stylized texts with varying degrees of probability of tokens in this text. Since the generated set of stylized texts can be very large, and the probability distribution in some of them will be too small, such candidates (generated stylized texts) can be cut off based on the sampling criteria indicated above. As a result, an array of stylized texts is formed at the output of the model.

[0060] It is worth noting that making it possible to generate an array of styled texts rather than a single styled text improves the accuracy of text styling, as the possibility of further verification and processing of all variants of the stylized text is provided to select the most semantically close, to the original, text from the array. In addition, the generation of an array of stylized texts makes it possible in the future to exclude the addition of new facts and / or incorrect changes in certain words based on further processing of such texts.

[0061] In step 250, the generated array of vectorized styled text is supplied to module 101. In step 250, each vectorized styled text from the array is decoded, at least converting the vectorized styled text to tokens and detokenization are performed during decoding. So, for example, during the specified process, each fixed-length vector, based on its dimension, is mapped to a token by a dictionary index, which allows each vector to be represented as a token. The process of detokenization is the reverse process to tokenization and consists of combining tokens into text. As a result of this step 250, the array of vectorized stylized texts is converted into an array of stylized natural language texts. Example:

Original text ['Here's what I found on your application]

An array of stylized texts: ['Here's what I found on your application', 'Here's what I found on your application', 'Here's what I found on your application', 'What I found on your application'…]

[0062] At step 260, the array of stylized texts is filtered. At the specified step 260, using the module 103, the specified array is filtered. So, at this stage, those candidates who do not satisfy the given characteristic style features are excluded. So, for example, when generating a stylized replica for a digital assistant imitating colloquial informal communication as a female, there will be at least the following criteria: by gender (feminine) and address (Tu). So, as a result of generating a stylized text for a female character who adheres to a business style of communication (there are characteristic stylistic features of a business style of communication), the candidate 'Here's what I found on your application' is excluded. The specified filtering also improves the accuracy of text styling. To check the correctness, regular expressions written on the basis of the re library and the MorphAnalyzer morphological analyzer from the pymorphy2 library are used. At the output of this module, we have a filtered list of possible candidates that have passed the correctness check. Example: ['Here's what I found on the application', 'What I found on your application'…].

[0063] In addition, in one particular embodiment, the array of styled texts is filtered based on a match of named entities (eg, proper names) in the source text and each styled text from the array. To do this, in the source and each of the stylized texts, an algorithm for recognizing named entities is performed, for example, using module 103, and the number of unchanged named entities contained in the source text and the stylized text is compared. As mentioned above, at step 240, an array of stylized texts is generated. To increase semantic accuracy between the styled text and the original text, the styled text should not distort facts or be supplemented with new logical relationships. In turn, the specified array of stylized texts may contain relevant stylized texts that satisfy the characteristic stylistic features, but such texts will be incorrect, for example, due to changes in the facts of such text.

Example:

Source text: I think you should read the work of Alexander Sergeevich Pushkin "Eugene Onegin"

Incorrect stylized text: I think you should read the work of Alexander Vasilyevich Pushkin "Eugene Onegin"

[0064] As can be seen from the example, the generated text satisfies the stylistic features of the official style (referring to "You"), i.e. is relevant, however, changes the facts of the text (the name of the author of the work).

[0065] To solve this problem, an approach was proposed that consists in comparing named entities in the source text and each styled text from the set. To do this, in the first step, named entities (eg, names, toponyms, and organizations) are recognized in the source text and stored in memory, eg, in the memory of the system 100, as a data file. The next step is to recognize named entities in each styled text from the array. After that, a comparison of immutable entities between the original and stylized texts is performed. Accordingly, candidates that differ from the source text in terms of named entities are discarded, which further increases the semantic accuracy of text styling by eliminating changes/additions of facts in the stylized text.

[0066] Thus, at step 260, the array of stylized texts is filtered.

[0067] At step 270, the filtered styled texts are ranked and the best styled text is selected, with the best text selection based on the pairwise distance between the original text and each of the possible styled texts. At block 270, the set of filtered stylized texts is ranked by character-by-character proximity to the original text. The specified ranking is based on the Levenshtein distance. To do this, the pairwise distance between the original neutral phrase and each of the possible candidates is calculated. The candidate with the minimum Levenshtein distance is selected as the best candidate. The Levenshtein distance is calculated using the distance function from the Levenshtein library. As a result, we get a stylized replica of the digital assistant, with the least change in the original text fragment, which accordingly increases the accuracy of the entire styling algorithm and reduces the likelihood of adding new facts.

[0068] In addition, in one particular embodiment of the claimed solution, the resulting text can be additionally checked for styling. Checking for text styling is based on determining whether the target style indicators have been replaced. For example, system 100 may further comprise a styling checker implemented on a computing device. The specified module is designed to check for a pair (original cue, stylized cue) whether during styling the replacement of target style indicators, for example, gender / number or number in circulation You / You, compared to the original phrase (there are many neutral phrases for which replacement is not required). This module is designed to be replaced only if at least one change occurs. Thus, if a replacement occurs, then at the output we get a stylized replica, and if the replacement does not occur, then the original one, which is neutral and suitable for any style.

[0069] At step 280, the stylized text is transmitted to the dialog system. At the specified step 280, a stylized replica of the digital assistant can be stored in the system memory as a file and sent to the dialog system via, for example, a data link for subsequent display to the user, for example, using the input / output interface of the dialog system 120.

[0070] Thus, in the above materials, a system and method for generating text for digital assistants in interactive systems has been described, providing high semantic accuracy in generating stylized text.

[0071] In addition, it is worth noting that, thanks to the implementation of the claimed solution, the universality of text styling is also provided, which allows generating text not only in one style, but also provides the ability to style the source text in several styles, depending on the scope. This feature eliminates the need for a separate formation of a unique stylized cue for each style. Thanks to the application of the claimed solution, it is possible to submit one source text and generate different stylized texts based on it.

[0072] Now consider one of the examples of the implementation of the claimed technical solution.

[0073] One of the possible applications of the system is to style the initially neutral response to the style of a given assistant. So, as mentioned above, dialogue systems may contain a set of digital assistants, each of which has its own style of communication. When requested by the user, the dialog system generates an initially neutral replica (source text). Said method 200 makes it possible to generate, from said neutral cue, such as a digital assistant response, a stylized cue (stylized text) in accordance with a given digital assistant style. This eliminates the need to generate many stylized replicas for each digital assistant. Also, another advantage of this approach is the flexibility in choosing a style, the generation of a text response is a computationally complex operation, it is necessary to train a large model for it. The claimed solution, in turn, allows you to style the generated initial response using the system 100 under different styles.

[0074] In FIG. 3 shows an example of a general view of a computing system (300), which provides the implementation of the claimed method or is a part of a computer system, for example, modules 101-103, a server, a personal computer, a part of a computing cluster that processes the necessary data to implement the claimed technical solution.

[0075] In general, the system (300) includes components such as: one or more processors (301), at least one memory (302), data storage media (303), input/output interfaces (304), means B /B (305), a means of networking (306), which are combined through a universal bus.

[0076] The processor (301) performs the basic computational operations necessary to process the data when executing the method (200). The processor (301) executes the necessary machine-readable instructions contained in the main memory (302).

[0077] the Memory (302), as a rule, is made in the form of RAM and contains the necessary software logic to provide the required functionality.

[0078] The data storage means (303) can be implemented in the form of HDD, SSD disks, raid array, flash memory, optical information storage devices (CD, DVD, MD, Blue-Ray disks), etc. The means (303) allow perform long-term storage of various types of information, such as generated stylized replicas, user IDs, digital assistant IDs, etc.

[0079] Various types of I/O interfaces (304) are used to organize the operation of system components (300) and organize the operation of external connected devices. The choice of appropriate interfaces depends on the particular design of the computing device, which can be, but not limited to: PCI, AGP, PS/2, IrDa, FireWire, LPT, COM, SATA, IDE, Lightning, USB (2.0, 3.0, 3.1, micro, mini, type C), TRS/Audio jack (2.5, 3.5, 6.35), HDMI, DVI, VGA, Display Port, RJ45, RS232, etc.

[0080] The choice of interfaces (304) depends on the specific implementation of the system (300), which can be implemented based on a wide class of devices, for example, a personal computer, mainframe, laptop, server cluster, thin client, smartphone, server, etc.

[0081] Data I/O (305) can be: keyboard, joystick, display (touchscreen), monitor, touchscreen, touchpad, mouse, light pen, stylus, touchpad, trackball, speakers, microphone, augmented reality tools, optical sensors, tablet, indicator lights, projector, camera, biometric identification tools (retinal scanner, fingerprint scanner, voice recognition module), etc.

[0082] The means of networking (306) are selected from devices that provide network reception and transmission of data, for example, an Ethernet card, WLAN / Wi-Fi module, Bluetooth module, BLE module, NFC module, IrDa, RFID module, GSM modem, etc. .P. With the help of the means (305), the organization of data exchange between, for example, the system (300), presented in the form of a server, and the user's computing device, which can display the received data (generated stylized replica of a digital assistant) via a wired or wireless data transmission channel, is provided, for example , WAN, PAN, LAN, Intranet, Internet, WLAN, WMAN or GSM.

[0083] The specific choice of elements of the system (300) for the implementation of various software and hardware architectural solutions may vary while maintaining the required functionality provided.

[0084] The submitted application materials disclose preferred examples of the implementation of the technical solution and should not be interpreted as limiting other, particular examples of its implementation that do not go beyond the scope of the requested legal protection, which are obvious to specialists in the relevant field of technology. Thus, the scope of this technical solution is limited only by the scope of the attached claims.

INFORMATION SOURCES

1. An Empirical Study on Multi-Task Learning for Text Style Transfer and Paraphrase Generation, Pawel Bujnowskia, Kseniia Ryzhovac, Hyungtak Choib, Katarzyna Witkowskad, Jaroslaw Piersaa, Tymoteusz Krumholca and Katarzyna Beksa. Found on the Internet at the link: https://aclanthology.org/2020.coling-industry.6.pdf, 04/20/2022.

Claims (18)

1. A method for automatically generating text for a digital assistant in interactive systems, performed by at least one computing device and comprising the steps of: a) receive input data containing the source text in natural language and the target style of the speech of the digital assistant, which characterizes the stylistic features inherent in the specified target style; b) performing encoding of the original text, and in the course of encoding, at least the tokenization of text data is performed; c) perform vectorization of texts by tokens obtained in step b); d) processing vector representations of the source text tokens obtained in step c) using a machine learning model based on a neural network trained on stylized, in accordance with a given target style, text replicas of a digital assistant, and the trained model generates, predicting the probability the next token based on the previous initial text fragment, during which an array of vectorized stylized texts is formed; e) performing the decoding of each vectorized stylized text from the array obtained in step d), and during the decoding perform at least the transformation of the vectorized stylized text into tokens and detokenization; f) filtering the array of stylized texts obtained in step e); g) ranking the filtered stylized texts and selecting the best stylized text, with the selection of the best text based on the pairwise distance between the original text and each of the possible stylized texts; h) sending the stylized text obtained in step g) to the dialog system. 2. The method according to claim 1, characterized in that the array of stylized texts is filtered using regular expressions and a morphological analyzer. 3. The method according to claim 1, characterized in that the array of stylized texts is filtered based on the matching of proper names in the source text and each stylized text from the array. 4. The method according to claim 3, characterized in that the proper name match is determined by the number of unchanged named entities contained in the source text and the stylized text. 5. The method according to claim 4, characterized in that the number of immutable named entities is determined based on the recognition of named entities in the source text and each styled text from the array. 6. The method according to claim. 1, characterized in that it further comprises the step of checking for the presence of text styling. 7. The method of claim. 7, characterized in that the check for the presence of text styling is carried out on the basis of determining whether the target style indicators have been replaced. 8. A system for automatic text generation for a digital assistant in interactive systems, comprising: • at least one processor; • at least one memory connected to the processor, which contains machine-readable instructions, which, when executed by at least one processor, provide execution of the method according to any one of paragraphs. 1-7.

Images