KR20210087384A - The server, the client device, and the method for training the natural language model - Google Patents

Abstract

Provided are a server and client device for learning a language model, and an operating method thereof. Provided, in one embodiment of the present disclosure, are the server and client device for learning a natural language understanding model and an operating method thereof, wherein the operating method: identifies a word or a phrase composed of an object name and the like for which a user finds difficulty in pronouncing accurately, or is difficult to pronounce correctly among an input text used for learning of a natural language understanding model; allows the user to be expected to utter the identified word or phrase; generates a text candidate used for learning the natural language understanding model by substituting the word or the phrase with a high phonetic similarity; and uses the generated text candidate.

Description

Server, client device, and operation method for learning natural language understanding model {THE SERVER, THE CLIENT DEVICE, AND THE METHOD FOR TRAINING THE NATURAL LANGUAGE MODEL}

The present disclosure relates to a server, a client device, and an operating method thereof for learning a natural language understanding model.

As multimedia technology and network technology develop, a user can receive various services using a device. In particular, as voice recognition technology develops, a user may input a voice (eg, utterance) into a device and receive a response message according to the voice input through a voice service providing agent.

When understanding the intention included in the user's voice input, artificial intelligence (AI) technology may be utilized, and rule-based natural language understanding (NLU) technology may be utilized. When a device or server receives a user's speech input and performs Automatic Speech Recognition (ASR) on the received speech input, a mismatch occurs between the ASR result and the text input to the natural language understanding model, and the precise intention of the speech cannot be identified. There is a problem that I cannot. In the conventional natural language understanding technology, for example, when the user utters with inaccurate pronunciation, or when the user utters inaccurately because he or she does not clearly know the named entity, place, place name, movie title, or game name, etc. If the output text of ASR is inaccurate because it is clearly pronounced but the ASR model is not trained on the object name, place, place name, movie title, or game name uttered by the user, the output text of the ASR is written through the natural language understanding model. There was a problem that did not match the learned text. The conventional natural language understanding technology may not provide an appropriate response according to the user's intention due to a problem that the ASR output text and the text of the natural language understanding model do not match.

The present disclosure discloses a word composed of an object name that a user pronounces incorrectly or is difficult for a user to pronounce correctly among input texts used for learning a natural language understanding model, an object name that is highly likely to cause an error when converted into text through ASR, etc. Or by identifying a phrase and replacing the identified word or phrase with a word or phrase that is expected to be uttered by the user and has a high phonological similarity to generate and generate text candidates used for learning the natural language understanding model An object of the present invention is to provide a server, a client device, and a method of operating the same for learning a natural language understanding model using the candidate text.

In order to solve the above-described technical problem, an embodiment of the present disclosure provides a method for a server to learn a natural language understanding model using text. The method may include receiving input text input by a user from a client device, identifying a replacement target text requiring replacement from among at least one word included in the input text, with respect to the identified replacement target text, the user generating an alternative text expected to be uttered and having a high phonological similarity; generating at least one learning candidate text by replacing the replacement target text among the input text with the generated replacement text and using the input text and the at least one learning candidate text as training data to train a natural language understanding model.

For example, the step of identifying the replacement target text may include receiving a user input for selecting at least one word or phrase from among the input text from the client device, and selecting from the client device based on the user input from the input text. It may include identifying at least one word or phrase, and determining the identified at least one word or phrase as the replacement target text.

For example, the step of identifying the replacement target text includes parsing the received input text into words, morphemes, and phrases, and information on pronunciation sequence information or embedding vectors for a plurality of words. retrieving at least one parsed word from a dictionary DB (Dictionary DB) that is not searched for, based on a search result of the dictionary DB, or a word whose frequency of use is lower than a preset first threshold as the replacement target It may include the step of determining with text.

For example, the step of identifying the replacement target text may include detecting a domain into which the input text is classified by interpreting the input text using a pre-learned natural language understanding model, and the pre-learned natural language understanding. Detecting an intent from the input text by interpreting the input text using a model; interpreting the input text using the pre-learned natural language understanding model, thereby creating a slot in the input text and performing slot tagging, and determining a text corresponding to the slot as the replacement target text.

For example, generating the at least one learning candidate text may include determining the generated replacement text to be the same slot as the slot identified from the input text.

For example, the generating of the replacement text includes extracting a phoneme sequence for the replacement target text, and extracting the phoneme sequence from the words pre-stored in the dictionary DB based on phonetic relevance of the pronunciation sequence. The method may include searching for text similar to the pronunciation string, and generating the alternative text using at least one text having a high similarity to the extracted pronunciation string based on the search result.

For example, generating the replacement text includes converting the replacement target text into an embedding vector using a word embedding model, and using a neural network model to embed the converted text. It may include generating text having a vector value similar to a vector, and generating the replacement text by using the generated text.

For example, the generating of the replacement text may include converting the replacement text into a wave signal using a Text-to-Speech model, and outputting the converted sound signal. , converting the output sound signal into output text using an ASR model, and generating the replacement text by replacing the replacement target text using the converted output text.

For example, the training of the natural language understanding model may include transmitting the input text and the at least one learning candidate text to the client device, from the client device, among the input text and the at least one learning candidate text. receiving an identification value related to at least one text selected by a user input, selecting at least one text of the input text and the at least one learning candidate text based on the received identification value, and the The method may include learning the natural language understanding model by using the selected at least one text as training data.

In order to solve the above-described technical problem, an embodiment of the present disclosure provides a method for providing an application (application) for a client device to learn a natural language understanding model. The method includes: displaying a first graphical user interface (GUI) for receiving a user input for inputting input text for learning the natural language understanding model on a display unit of the client device; transmitting the input text inputted through the server to a server; displaying a second GUI for receiving a user input for selecting at least one of at least one replacement target text identified from the input text; Receiving at least one training candidate text generated by replacing the at least one replacement target text selected through the second GUI with the replacement text expected to be uttered by the user from the server, and at least one of the at least one training candidate text The method may include displaying a third GUI for receiving a user input for selecting one on the display unit.

For example, the method may include receiving a user input selecting at least one of context information including at least one of age, gender, region, spoken language, and dialect of a user being considered for generating the alt text; The method may further include displaying a fourth GUI for

In order to solve the above-described technical problem, an embodiment of the present disclosure provides a server for learning a natural language understanding model using text. The server includes a communication interface for performing data communication with a client device, a memory storing a program including one or more instructions, and a processor executing one or more instructions of the program stored in the memory, the processor receives the input text input by the user from the client device through the communication interface, identifies the replacement target text requiring replacement from among at least one word included in the input text, and relates to the identified replacement target text , generating at least one learning candidate text by generating an alternative text that is expected to be uttered by a user and having a high phonological similarity, and replacing the replacement target text among the input text with the generated replacement text and, by using the input text and the at least one learning candidate text as training data, a natural language understanding model may be trained.

For example, the processor receives a user input for selecting at least one word or phrase from the input text from the client device using the communication interface, and based on the user input from the input text The selected at least one word or phrase may be identified, and the identified at least one word or phrase may be determined as the replacement target text.

For example, the memory stores a dictionary DB (DB) including pronunciation sequence information for a plurality of words or information on embedding vectors, and the processor converts the received input text into words, morphemes, and syntax units. parse, search the at least one parsed word in the dictionary DB, and determine a word that is not searched based on a search result or has a frequency of use lower than a preset first threshold as the replacement target text can

For example, the memory stores at least one pre-trained natural language understanding model, and the processor interprets the input text by using any one natural language understanding model among the at least one natural language understanding model pre-stored in the memory. , by detecting a domain and an intent into which the input text is classified, and interpreting the input text using the pre-learned natural language understanding model, to identify a slot in the input text, Slot tagging may be performed, and the text corresponding to the slot may be determined as the replacement target text.

For example, the processor extracts a phoneme sequence related to the replacement target text, and based on phonetic relevance of the pronunciation sequence, extracts the extracted words from the dictionary DB stored in the memory. A text similar to the selected pronunciation string may be searched for, and the alternative text may be generated using at least one text having a high similarity to the extracted pronunciation string based on the search result.

For example, the memory stores a word embedding model, and the processor uses the word embedding model to convert the replacement target text into an embedding vector, and a neural network model (Neural Network). model), a text having a vector value similar to the converted embedding vector may be generated, and the replacement text may be generated using the generated text.

For example, the memory stores a text-to-speech model for converting text into a wave signal, and the processor converts the replacement target text into a sound signal using the TTS model. and outputting the converted sound signal, using an ASR model pre-stored in the memory, converting the output sound signal into an output text, and replacing the replacement target text using the converted output text, The alternative text may be generated.

For example, the processor transmits the input text and the at least one learning candidate text to the client device using the communication interface, and from the client device, a user of the input text and the at least one learning candidate text. Receive an identification value for at least one text selected by an input, and select at least one of the input text and the at least one learning candidate text based on the received identification value, and select the at least one selected text The natural language understanding model may be learned by using the text as training data.

In order to solve the above technical problem, an embodiment of the present disclosure provides a client device that provides an application for learning a natural language understanding model. The client device includes a display unit, a user input unit configured to receive a user input for inputting input text, a communication interface for performing data communication with a server, a memory storing a program including one or more instructions, and the a processor for executing one or more instructions of a program stored in a memory, wherein the processor controls the display unit to display a first graphical user interface (GUI) for receiving a user input for inputting the input text and control the display unit to display a second GUI for receiving a user input for selecting at least one of the at least one replacement target text identified from the input text, and at least one of the input text selected through the second GUI controlling the communication interface to receive, from a server, at least one learning candidate text generated by replacing one replacement target text with an alternative text expected to be uttered by the user, and selecting at least one of the at least one training candidate text The display unit may be controlled to display a third GUI for receiving a user input.

For example, the processor receives a user input for selecting at least one of context information including at least one of age, gender, region, spoken language, and dialect of a user to be considered for generating the alternative text. The display unit may be controlled to display a fourth GUI for

In order to solve the above-described technical problem, another embodiment of the present disclosure provides a computer-readable recording medium in which a program for execution in a computer is recorded.

1 is a conceptual diagram illustrating operations for a server and a client device to learn a language model according to an embodiment of the present disclosure.
2 is a block diagram illustrating a configuration of a server according to an embodiment of the present disclosure.
3 is a block diagram illustrating a configuration of a client device according to an embodiment of the present disclosure.
4 is a flowchart illustrating an embodiment in which a server of the present disclosure learns a natural language understanding model.
5 is a flowchart of an embodiment in which a server of the present disclosure determines an alternative target text based on a user input.
6 is a flowchart of an embodiment in which a server of the present disclosure identifies a text to be replaced based on a search result of a dictionary DB.
7 is a flowchart of an embodiment in which a server of the present disclosure automatically identifies an alternative target text using a natural language understanding model.
8 is a diagram illustrating an embodiment in which a server of the present disclosure generates an alternative text from an alternative text.
9 is a flowchart of an embodiment in which a server of the present disclosure generates an alternative text from an alternative text.
10 is a diagram illustrating an embodiment in which a server of the present disclosure generates an alt text from an alt text by using a neural network model.
11 is a flowchart of an embodiment in which a server of the present disclosure generates an alternative text using a neural network model.
12 is a diagram illustrating an embodiment in which a server of the present disclosure generates an alternative text from an alternative text.
13 is a flowchart of an embodiment in which a server of the present disclosure generates an alternative text from an alternative text.
14 is a flowchart illustrating a method in which a server of the present disclosure interprets an input text and at least one learning candidate text using a natural language understanding model, and generates information about an interpretation result.
15 is a flowchart of an embodiment in which a server of the present disclosure determines text input as training data to a natural language understanding model.
16 is a flowchart of an embodiment in which a server of the present disclosure learns a natural language understanding model based on the number of input texts and at least one learning candidate text.
17 is a flowchart illustrating an operation of a client device of the present disclosure.
18 is a diagram illustrating an example of a GUI displayed by a client device of the present disclosure.
19A is a diagram illustrating an example of a GUI displayed by a client device of the present disclosure.
19B is a diagram illustrating an example of a GUI displayed by a client device of the present disclosure.
20 is a diagram illustrating an example of a GUI displayed by a client device of the present disclosure.
21 is a diagram illustrating an example of a GUI displayed by a client device of the present disclosure.
22 is a block diagram illustrating a configuration of a language understanding service according to an embodiment of the present disclosure.

Terms used in the embodiments of the present specification have been selected as currently widely used general terms as possible while considering the functions of the present disclosure, which may vary depending on the intention or precedent of a person skilled in the art, the emergence of new technology, etc. . In addition, in specific cases, there are also terms arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the corresponding embodiment. Therefore, the terms used in the present specification should be defined based on the meaning of the term and the contents of the present disclosure, rather than the name of a simple term.

The singular expression may include the plural expression unless the context clearly dictates otherwise. Terms used herein, including technical or scientific terms, may have the same meanings as commonly understood by one of ordinary skill in the art described herein.

In the present disclosure, when a part "includes" a certain component, it means that other components may be further included, rather than excluding other components, unless otherwise stated. In addition, terms such as "... unit", "... module", etc. described in this specification mean a unit that processes at least one function or operation, which is implemented as hardware or software, or is a combination of hardware and software. can be implemented.

As used herein, the expression “configured to (or configured to)” depends on the context, for example, “suitable for”, “having the capacity to” It can be used interchangeably with "," "designed to", "adapted to", "made to", or "capable of". The term “configured (or configured to)” may not necessarily mean only “specifically designed to” in hardware. Instead, in some circumstances, the expression “a system configured to” may mean that the system is “capable of” with other devices or components. For example, the phrase "a processor configured (or configured to perform) A, B, and C" refers to a dedicated processor (eg, an embedded processor) for performing the operations, or by executing one or more software programs stored in memory; It may refer to a generic-purpose processor (eg, a CPU or an application processor) capable of performing corresponding operations.

In the present disclosure, a 'Natural Language Understanding (NLU)' is a model trained to interpret text converted from a voice signal to obtain a domain and an intent corresponding to the text. By interpreting the text, the natural language understanding model can identify information about a slot as well as a domain and an intent.

In the present disclosure, a 'Natural Language Understanding Model' is an expression of at least one of a 'Language Model', a 'Language Understanding Model', a 'Natural Language Understanding Model', a 'Natural Language Processing Model', a 'Language Processing Model', and a 'Language Understanding Technology Model'. can be used instead of

In the present disclosure, a 'domain' is a category or area related to a user's intention identified by analyzing text. The domain can be detected by interpreting the text using a natural language understanding model. A domain may be associated with an intent detected from text using a natural language understanding model. In one embodiment, domains may be classified according to text-related services. The domain may include one or more areas, such as a category to which text corresponds, for example, a movie domain, a music domain, a book domain, a game domain, an aviation domain, and a food domain.

In the present disclosure, an 'intent' is information indicating a user's intention determined by analyzing text. The intent is information indicating the user's intention to speak, and may include information indicating an action or function that the user intends to execute using the device. The intent may be determined by interpreting the text using a Natural Language Understanding (NLU) model. For example, if the text converted from the user's voice input is "Search for the release date of the movie Avengers: Endgame", the domain may be 'movie' and the intent may be 'movie search'. The intent may include an intent action and an intent object.

The intent may include not only information indicating the user's utterance intention (hereinafter, intention information), but also a numerical value corresponding to information indicating the user's intention. The numerical value may indicate a probability that the text is associated with information indicating a particular intention. When a plurality of pieces of information indicating the user's intention are obtained as a result of analyzing the text using the natural language understanding model, intention information having a maximum numerical value corresponding to each intention information may be determined as an intent.

In the present disclosure, a 'slot' means variable information for obtaining detailed information related to an intent or determining a detailed operation. A slot is information related to an intent, and a plurality of types of slots may correspond to one intent. A slot may include a numerical value indicating a probability that the text is associated with the variable information. In an embodiment, as a result of analyzing the text using the natural language understanding model, a plurality of variable information indicating slots may be obtained. In this case, variable information having a maximum numerical value corresponding to each variable information may be determined as a slot. For example, when the text is "Search for the release date of the movie Avengers Endgame", the slots obtained from the text may be 'Movie Title (Avengers Endgame)' and 'Film Release Date'.

Domains, intents, and slots may be automatically identified or detected using a natural language understanding model, but is not limited thereto. In an embodiment of the present disclosure, the domain, the intent, and the slot may be manually designated or determined by a user input input through a client device, respectively.

1 is a conceptual diagram illustrating operations for learning a language model by a server 1000 and a client device 2000 according to an embodiment of the present disclosure. The server 1000 and the client device 2000 may perform operations for learning the natural language understanding model.

Referring to FIG. 1 , a server 1000 may transmit/receive data to/from a client device 2000 through a wired or wireless communication method.

The client device 2000 may be a computing device that provides a developer software tool for learning a natural language understanding model from a user. The client device 2000 is, for example, a smart phone, a tablet PC, a PC, a laptop, a smart TV, a mobile phone, a personal digital assistant (PDA), a media player, a global positioning system (GPS) device, an e-book terminal, or a digital broadcasting terminal. , navigation, kiosks, MP3 players, digital cameras, home appliances and other mobile or non-mobile computing devices.

The software tool for developers provided by the client device 2000 receives input text through an application, and displays at least one learning candidate text generated by the server with respect to the input text through interaction with the server 1000 . may be displayed on 2510 . The at least one learning candidate text is obtained by replacing a word or phrase in the input text with a word or phrase having a similar pronunciation, or a similar pronunciation string, that a user using the voice assistant function is expected to pronounce, thereby generating a server text generated by The at least one learning candidate text may be input data used to train the natural language understanding model 1320 . The natural language understanding model 1320 may be a pre-trained NLU model according to a specific domain, but is not limited thereto.

The server 1000 receives the input text from the client device 2000, generates at least one learning candidate text related to the received input text, and inputs the input text and the generated at least one learning candidate text as training data. Training on the natural language understanding model may be performed. The server 1000 may update the previously learned natural language understanding model 1320 or generate a new natural language understanding model by using the input text and at least one learning candidate text as training data. In one embodiment, the server 1000 includes an ASR model 1310 , a natural language understanding model 1320 , a TTS model 1330 , an alternative target text identification module 1340 , an alternative text generation module 1350 , and a learning candidate text. A generation module 1360 may be included.

Hereinafter, data transmission/reception and operation between the server 1000 and the client device 2000 will be described.

In step S110 , the client device 2000 receives input text from the user and transmits the input text to the server 1000 . In an embodiment, the client device 2000 displays a first graphical user interface (GUI) 2010 for receiving input text on the display unit 2510 , and displays the first GUI 2010 . to receive input text.

In step S120 , the server 1000 generates an alternative text and reconstructs the input text using the alternative text, thereby transmitting the generated at least one learning candidate text to the client device 2000 . In an embodiment, the server 1000 may identify the replacement target text requiring replacement from among the input text by using the data and the program code of the replacement target text identification module 1340 . In an embodiment, the server 1000 generates an alternative text that is expected to be uttered by the user with respect to the identified alternative text and has a high phonological similarity by using the data and the program code of the alternative text generating module 1350 . can create In an embodiment, the server 1000 uses the data and program code of the training candidate text generation module 1360 to replace the replacement target text identified in the input text with the replacement text, thereby generating the natural language understanding model 1320 . ) can generate a learning candidate text input to the learning of.

For example, if the input text input by the user through the client device 2000 is "Deliver 3 pepperoni pizzas to 30 gil in Unjuro", the server 1000 identifies 'pepperoni' as the replacement target text, and ' 'Pepperoni', 'Pepperoni', 'Pepperoni', etc., which are predicted to be pronounced by the user with respect to 'Pepperoni' and have similar pronunciation columns, are generated as alternative texts, , "deliver 3 slices of pepperoni pizza by 30 gil", "deliver 3 slices of pepperoni pizza by 30 gil", "deliver 3 slices of pepperoni pizza by 30 gil". can

In the embodiment illustrated in FIG. 1 , it has been described that only 'pepperoni' is identified as the replacement target text from the input text, but is not limited thereto. In an embodiment, the server 1000 may identify a plurality of replacement target texts from the input text using data and program codes of the replacement target text identification module 1340 . For example, the server 1000 may identify 'Eonjuro 30gil', which is a slot indicating an address, as the replacement target text from the input text of "3 pieces of pepperoni pizza delivered to Eonjuro 30gil". When 'Eonju-ro 30-gil' is identified as the replacement target text, the server 1000 predicts that the user will pronounce 'Eonju-ro 30-gil' by using the data and program code of the alternative text generation module 1350 . By creating alternative texts such as 'Anju-ro 30-gil', 'Unju-ro 30-gil', 'Eonjo-ro 30-gil', etc. with similar pronunciation columns, and replacing the replacement target text with the generated alternative text, "Pepperoni Pizza 3 You can create learning candidate texts such as "Deliver 3 slices of pepperoni pizza to 30 gil with a side dish", "Deliver 3 slices of pepperoni pizza to 30 gil with Unjuro", and "Deliver 3 slices of pepperoni pizza to 30 gil with Eonjo".

When a plurality of replacement target texts are identified, the server 1000 may generate an alternative text for each of the identified plurality of replacement target texts. For example, when 'pepperoni', a slot indicating the type of pizza topping, and 'Eonjuro 30-gil', a slot indicating an address, are both identified as replacement target text, the server 1000 generates an alternative text generating module 1350. can be used to generate alternative texts for each of 'Pepperoni' and 'Eonju-ro 30-gil'. As in the above example, the server 1000 selects 'pepperoni', 'paperoni', and 'peporoni' for 'pepperoni', and 'anju-ro 30-gil' and 'unju-ro 30-gil' for 'Eonju-ro 30-gil'. By generating 'Ro 30-gil' and 'Eonjo-ro 30-gil' as alternative texts and combining the generated alternative texts, it is possible to generate a learning candidate text replacing the input text.

The server 1000 may transmit the generated at least one learning candidate text to the client device 2000 .

In step S130 , the client device 2000 receives at least one learning candidate text from the server 1000 , and transmits at least one text selected by the user among the at least one learning candidate text to the server 1000 . In an embodiment, the client device 2000 may display the second graphical user interface 2020 indicating a new sentence list including at least one learning candidate text received from the server 1000 on the display unit 2510 . can The client device 2000 receives a user input for selecting at least one text among at least one learning candidate text included in the new sentence list through the second graphical user interface 2020, and based on the received user input, at least You can select one text.

The client device 2000 may transmit an identification value regarding at least one selected text to the server 1000 .

In an embodiment, the server 1000 may transmit a result (domain, intent, and slot information) learned using the input text and at least one learning candidate text to the client device 2000 . In an embodiment, the server 1000 may identify a domain, an intent action, an intent object, and a slot into which the input text and at least one learning candidate text are classified using the natural language understanding model 1320 . In one embodiment, the server 1000 may automatically perform slot tagging on slots in the input text and at least one learning candidate text based on the identified domain, the intent action, and the intent object. may be

In step S140, the server 1000 performs learning by inputting the text selected based on the user input received through the client device 2000 into the natural language understanding model 1320 as learning data, and provides information on completion of learning. It is transmitted to the client device 2000 . In an embodiment, the server 1000 uses the natural language understanding model 1320 to identify a domain into which the input text and at least one learning candidate text are classified, and a plurality of natural languages stored in the memory 1300 (refer to FIG. 2 ). A natural language understanding model specialized for the identified domain among the understanding models 1320a to 1320c (refer to FIG. 2 ) may be selected. In an embodiment, the server 1000 may perform learning by inputting the selected text among the input text and the at least one learning candidate text to the selected natural language understanding model as training data, and may update the natural language understanding model. However, the present invention is not limited thereto, and the server 1000 may generate a new natural language understanding model using the input text and at least one learning candidate text.

2 is a block diagram illustrating a configuration of a server 1000 according to an embodiment of the present disclosure.

Referring to FIG. 2 , the server 1000 may include a communication interface 1100 , a processor 1200 , and a memory 1300 .

The communication interface 1100 may perform data communication with the client device 2000 under the control of the processor 1200 . The communication interface 1100 may perform data communication not only with the client device 2000 but also with other servers. Communication interface 1100 is, for example, wired LAN, wireless LAN (Wireless LAN), Wi-Fi (Wi-Fi), Bluetooth (Bluetooth), Zigbee (zigbee), WFD (Wi-Fi Direct), infrared communication (IrDA, Infrared Data Association), BLE (Bluetooth Low Energy), NFC (Near Field Communication), WiBro (Wireless Broadband Internet, Wibro), WiMAX (World Interoperability for Microwave Access, WiMAX), SWAP (Shared Wireless Access Protocol), WiGig Data communication may be performed with the client device 2000 or another server using at least one of a data communication method including (Wireless Gigabit Allicance, WiGig) and RF communication.

The processor 1200 may execute one or more instructions of a program stored in the memory 1300 . The processor 1200 may be configured as a hardware component that performs arithmetic, logic, input/output operations and signal processing. The processor 1200 may include, for example, a central processing unit (Central Processing Unit), a microprocessor, a graphic processing unit (ASIC), Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), and Digital Signal Processors (DSPDs). Signal Processing Devices), PLDs (Programmable Logic Devices), and FPGAs (Field Programmable Gate Arrays) may be configured as at least one, but is not limited thereto.

The memory 1300 is, for example, a flash memory type, a hard disk type, a multimedia card micro type, or a card type memory (eg, SD or XD memory). etc.), non-volatile memory including at least one of ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, magnetic disk, optical disk, and It may include a volatile memory such as random access memory (RAM) or static random access memory (SRAM).

The memory 1300 may store instructions, data structures, and program codes that the processor 1200 can read. In the following embodiment, the processor 1200 may be implemented by executing instructions or codes of a program stored in a memory. In addition, the memory 1300 includes an ASR model 1310 , an NLU model 1320 , a TTS model 1330 , an alternative target text identification module 1340 , an alternative text generation module 1350 , and a learning candidate text generation module 1360 ). , a dictionary DB (Dictionary DB, 1370), and data and program command codes corresponding to the word embedding model 1380 may be stored.

The processor 1200 may receive the input text input by the user from the client device 2000 using the communication interface 1100 . The input text may be acquired through a typing input through the user input unit 2410 (refer to FIG. 3 ) of the client device 2000, but is not limited thereto. In one embodiment, the input text may be obtained by converting a voice input received through the microphone 2420 (refer to FIG. 3 ) of the client device 2000 into text using the ASR model 1310 . In this case, the server 1000 may receive a voice signal converted from a voice input from the client device 2000 .

In one embodiment, the processor 1200 performs Automatic Speech Recognition (ASR) using the data and the command code regarding the ASR model 1310, and converts the voice signal received from the client device 2000 into input text. can The ASR model 1310 is a voice recognition model for recognizing the user's voice, and may convert the user's voice input into input text and output it. The ASR model 1310 may be, for example, an artificial intelligence model including an acoustic model, a pronunciation dictionary, and a language model. Alternatively, the ASR model 1310 may be, for example, an end-to-end speech recognition model having a structure including an integrated neural network without separately including an acoustic model, a pronunciation dictionary, and a language model. The end-to-end ASR model uses an integrated neural network to convert speech into text without a process of converting phonemes into text after recognizing phonemes from speech.

A plurality of natural language understanding models 1320 may be stored in the memory 1300 . Although only the first natural language understanding model 1320a, the second natural language understanding model 1320b, and the third natural language understanding model 1320c are illustrated in the drawings, this is for convenience of description. The number of natural language understanding models stored in the memory 1300 is not limited as shown in the drawings. In an embodiment, only the first natural language understanding model 1320a may be stored in the memory 1300 .

The first natural language understanding model 1320a to the third natural language understanding model 1320c may be pre-trained NLUs. Each of the first natural language understanding model 1320a to the third natural language understanding model 1320c may be an artificial intelligence model learned based on a specific domain. For example, the first natural language understanding model 1320a is a model trained using a word or text related to a movie, and label values (correct values) for intents and slots specialized in a movie domain, and the second natural language understanding model ( 1320b) is a model learned by using game-related words or texts and label values of intents and slots specialized for game domains, and the third natural language understanding model 1320c is a word or text related to control of home appliances and home appliances. It may be a model trained using the label values of intents and slots specialized for the control domain. However, the present invention is not limited thereto, and at least one of the first natural language understanding model 1320a to the third natural language understanding model 1320c is not specific to a specific domain, and uses general-purpose word, text and label values of intents and slots. This may be a trained model.

In an embodiment, the first natural language understanding model 1320a to the third natural language understanding model 1320c is a new natural language understanding learned using a new domain, a new intent, and a new slot and input text newly created by the user. It could be a model.

Hereinafter, components will be described based on the first natural language understanding model 1320a.

The first natural language understanding model 1320a may include a labeling model 1321a, a domain identification model 1322a, an intent action identification model 1323a, an intent object identification model 1324a, and a slot tagging model 1325a. can However, this is exemplary, and the first natural language understanding model 1320a is the labeling model 1321a, domain identification model 1322a, intent action identification model 1323a, intent object identification model 1324a shown in the figure. , and the slot tagging model 1325a may not be included, and only some models may be included.

The labeling model 1321a may include data and program codes configured to label information about domains, intents, and slots detected from input text. For example, the labeling model 1321a may be used by a user to add his or her own tags to intents and slots. The processor 1200 may automatically determine a label for a domain, an intent, and a slot detected from the input text by using the data and the program code of the labeling model 1321a.

However, the present invention is not limited thereto, and the processor 1200 receives, from the client device 2000 , information about a domain, an intent, and a slot determined for the input text based on a user input using the communication interface 1100 . You may. In this case, the processor 1200 may use the data and program code of the labeling model 1321a to label the received domain, intent, and slot in the input text.

The processor 1200 may allow a user to create a new domain, an Intent action, an Intent object, and a slot by using the data and program code of the labeling model 1321a. For example, the labeling model 1321a may be used by the user to add arbitrary tags for intents and slots, or to select automatically determined tags. The labeling model 1321a may use a domain identification model 1322a , an intent action identification model 1323a , an intent object identification model 1324a , and a slot tagging model 1325a .

The domain identification model 1322a is a model used to identify a domain corresponding to the input text by interpreting the input text. The processor 1200 may interpret the input text by using the data and the program code of the domain identification model 1322a to identify a domain to which the input text belongs or is category information to which it can be classified. In one embodiment, the processor 1200 may identify one or more domains from the input text. In an embodiment, the processor 1200 may calculate the relation between the input text and the domain as a numerical value by interpreting the input text using the domain identification model 1322a. In an embodiment, the domain identification model 1322a is configured to calculate a degree of relevance between the input text and one or more domains as a probability value, and to determine a domain having a high probability value among the calculated probability values as a domain into which the input text is classified. can

The intent action identification model 1323a is a model configured to predict an intent action from the input text by interpreting the input text. The intent action refers to an action performed by the input text, for example, search, posting, play, purchase, order, and the like. The processor 1200 may identify the intent action from the input text by performing syntactic analysis or semantic analysis using the data and program code of the intent action identification model 1323a. . In one embodiment, the processor 1200 parses the input text into units of morphemes, words, or phrases using the data and program code of the intent action identification model 1323a, The meaning of the word or phrase extracted from the parsed text may be inferred by using the linguistic features (eg, grammatical elements) of the parsed morpheme, word, or phrase. The processor 1200 may determine an intent action corresponding to the meaning of the inferred word or phrase by comparing the meaning of the inferred word or phrase with predefined intents provided from the natural language understanding model.

The intent object identification model 1324a is a model configured to identify an intent object from the input text by interpreting the input text. The intent object means an object related to the identified intent action. The intent object is an object that is the target of the detected intent action, and may mean, for example, a movie, a photo, sports, weather, or aviation. The processor 1200 may identify the intent object related to the intent action from the input text by interpreting the input text by using the program code and the data related to the intent object identification model 1324a.

The intent action identification model 1323a and the intent object identification model 1324a may identify an intent action and an intent object based on a result of learning specialized for a specific domain.

For example, if the input text is "Search for the release date of the movie Avengers: Endgame", the processor 1200 interprets the input text using the intent action identification model 1323a, and determines the intent action as a result of the interpretation. 'search', and using the intent object identification model 1324a, the intent object may be detected as a 'movie'. As another example, if the input text is "Order a pepperoni pizza at Domino's Pizza~", the processor 1200 interprets the input text using the intent action identification model 1323a, and the result of the interpretation is the intent The action may be identified as 'order' and the intent object may be detected as 'food' using the intent object identification model 1324a.

In an embodiment, the intent action identification model 1323a and the intent object identification model 1324a may identify an intent action and an intent object from the input text, respectively, based on the matching model. The relation between each word or phrase parsed from the input text and the intent action and intent object may be calculated as a predetermined numerical value. In an embodiment, a relationship between a word or phrase of the input text and an intent (including an intent action and an intent object) may be calculated as a probability value. In an embodiment, the processor 1200 applies the matching model of the intent action identification model 1323a to the input text, thereby generating a plurality of numerical values indicating the degree of relevance between the word parsed from the input text and the pre-stored intent action. obtained, and an intent action having a maximum value among a plurality of numerical values may be identified as an intent action related to the input text. For example, when the input text is "Order 3 slices of pepperoni pizza at Domino's Pizza", the processor 1200 applies the matching model of the intent action identification model 1323a to the input text, thereby making the input text and the intent As an action, the matching degree between 'order' can be calculated as a numerical value of 0.9, and the matching degree between the input text and 'search' can be calculated as a numerical value of 0.1, respectively. The processor 1200 may determine the intent action calculated as the maximum value among the numerical values as the intent action related to the input text, but is not limited thereto. In an embodiment, the processor 1200 may determine an intent action regarding the input text based on a user input of selecting one of a plurality of intents.

Since the method of the processor 1200 for identifying the intent object from the input text is the same as the method of identifying the intent action, a redundant description will be omitted.

In FIG. 2 , the intent action identification model 1323a and the intent object identification model 1324a are illustrated as separate models, but the present disclosure is not limited thereto. In an embodiment, the intent action identification model 1323a and the intent object identification model 1324a may be integrated into one intent identification model.

The slot tagging model 1325a is a model configured to identify slots from input text and perform slot tagging associating the identified slots with domains, intent actions, and intent objects. The slot means variable information for obtaining detailed information related to a domain, an intent action, and an intent object from an input text or determining a detailed action. A slot is information related to an intent action and an intent object, and a plurality of types of slots may correspond to one intent. Slot tagging refers to an operation of identifying a slot in an input text and mapping the identified slot to an element related to a domain, an Intent action, and an Intent object.

In one embodiment, the processor 1200 uses the slot tagging model 1325a to identify a slot from the input text and automatically map the identified slot to a slot element associated with a domain, an Intent action, and an Intent object. can do it For example, if the input text is "Search for the release date of the movie Avengers Endgame", the processor 1200 uses the data and program code of the slot tagging model 1325a to extract 'Avengers Endgame' and To identify 'release date' as a slot, tag 'Avengers Endgame' to 'movie title', a slot related to the domain and movie search intent of the movie, and tag 'release date' to the slot for 'movie release date'. can As another example, if the input text is "Order for delivery at 30 gil with 3 slices of pepperoni pizza from Domino's pizza", the processor 1200 uses the data and program code of the slot tagging model 1325a, the input text From 'Domino Pizza', 'Pepperoni Pizza', '3rd Edition', and 'Eonju-ro 30-gil' are identified as slots, and as slots related to the domain of food and the intent of the food order, 'Domino Pizza' is defined as 'restaurant'. You can tag 'Pepperoni Pizza' as 'Pizza Topping', '3 pieces' as 'order quantity', and 'Eonju-ro 30-gil' as 'place'.

The processor 1200 may automatically perform slot tagging on the input text using the slot tagging model 1325a, but is not limited thereto. The processor 1200 may receive a user input for inputting a specific slot or adding a new slot with respect to a word or phrase in the input text from the client device 2000 through the communication interface 1100 . In this case, the processor 1200 may tag the word or phrase of the input text to the slot determined or added by the user input.

When the processor 1200 receives the input text input by the user from the client device 2000 through the communication interface 1100 , the input text from the first natural language understanding model 1320a to the third natural language understanding model 1320c may calculate a probability corresponding to a specific domain, and select any one natural language understanding model from among the first natural language understanding model 1320a to the third natural language understanding model 1320c based on the calculated probability value. The processor 1200 may perform intent detection and slot tagging of the input text, identify the replacement target text, and generate the replacement text by using the selected natural language understanding model. This will be described in detail with reference to FIG. 16 .

The processor 1200 may transmit an intent matching the input text, slot tagging information, and a calculated numerical value to the client device 2000 using the communication interface 1100 . The transmitted intent, slot tagging information, and each calculated numerical value may be displayed on the display unit 2510 (refer to FIG. 3 ) of the client device 2000 . This will be described in detail with reference to FIG. 18 .

The Text-to-Specch (TTS) model 1330 is a model configured to convert text into a wave signal. In an embodiment, the processor 1200 converts text into a sound signal by using data and a program code related to the TTS model 1330 , and converts the converted sound signal into a binary data streaming type signal. can be printed out.

In an embodiment, the TTS model 1330 may include a personalized TTS model reflecting a context including information of a region or a specific person. The personalized TTS model may convert a text into an acoustic signal and output it by reflecting at least one characteristic of a person's tone, intonation, and dialect according to a specific region, age, gender, etc., for example.

The replacement target text identification module 1340 is a module configured to identify replacement target text requiring replacement among input texts. In an embodiment, the processor 1200 may identify the replacement target text requiring replacement from among at least one word included in the input text by using the data and the program code related to the replacement target text identification module 1340 . . 'Substitute text' is input to the natural language understanding model 1320 because a user using an application including a language model pronounces it incorrectly when uttering, or the text output as a result of performing ASR through the ASR model 1310 is inaccurate. If it is, it may mean a word or phrase that is difficult to learn accurately. The replacement target text may include, for example, a named entity, a keyword, a place, a place name, a movie title, or a game term included in the input text. The replacement target text may be, for example, a slot.

The replacement target text identification module 1340 may be trained to identify one or more replacement target texts from the input text. For example, if the input text is "Order for delivery by 30 gil with 3 slices of pepperoni pizza", the processor 1200 uses the replacement target text identification module 1340 to obtain 'pepperoni' or 'unjug' from the input text. '30-gil' can be identified.

The replacement target text identification module 1340 may be configured to obtain an analysis result of the input text from the natural language understanding model 1320 and automatically identify the replacement target text from the input text based on the analysis result of the input text. In an embodiment, the processor 1200 interprets the input text using the pre-learned natural language understanding model to identify a domain, an intent action, an intent object, and a slot into which the input text is classified, and select the identified slot. The domain, the intent action, and the slot related to the intent object are tagged, and the text corresponding to the tagged slot may be determined as the replacement target text by using the replacement target text identification module 1340 . In this case, the replacement text replacing the replacement target text may be tagged with the same slot as the slot in which the replacement target text is tagged.

However, the present invention is not limited thereto, and the replacement target text identification module 1340 may be configured to determine a word or phrase manually determined by a user input as the replacement text. In an embodiment, the processor 1200 receives a user input for selecting at least one word or phrase among the input text from the client device 2000 through the communication interface 1100 , and the replacement target text identification module 1340 . may be used to identify at least one word or phrase selected based on the user input, and determine the identified at least one word or phrase as the replacement target text. In an embodiment, the replacement target text identification module 1340 may identify at least one word or phrase selected based on a user input as an entity name.

In an embodiment, the replacement target text identification module 1340 is configured to identify the entity name included in the input text by performing part-of-speech tagging (POS Tagging, Part-Of-Speech Tagging) of words identified from the input text. can be However, the embodiment of the present disclosure is not limited thereto, and the replacement target text identification module 1340 is configured to identify the entity name from the input text using various known named entity recognition (NER) methods. can be

In an embodiment, the replacement target text identification module 1340 may identify the replacement target text based on a search result related to a word pre-stored in the dictionary DB 1370 . The dictionary DB (Dictionary Database) 1370 may be a database that stores pronunciation sequence information for a plurality of words or information on an embedding vector converted for each of the plurality of words. The dictionary DB 1370 is a database that stores pronunciation column information on a plurality of words and embedding vectors on pronunciation columns included in each of the plurality of words. In one embodiment, the processor 1200 parses the input text into words, morphemes, and phrases by using the data and program code of the replacement target text identification module 1340 , and at least parsed in the dictionary DB 1370 . A word may be searched for, and a word that is not searched or has a frequency of use lower than a preset first threshold may be determined as the replacement target text based on a search result of the dictionary DB 1370 . The first threshold is a set value related to the frequency of use for determining the replacement target text as a result of searching the dictionary DB 1370 . In one embodiment, the first threshold may be set based on a user input. For example, the first threshold may be set by a user input received through a UI displayed through the client device 2000 . A UI for setting the first threshold will be described in detail with reference to FIG. 18 .

For example, if the first threshold is set to be relatively low, fewer words with a frequency of use lower than the first threshold will be searched as a result of a search of the dictionary DB 1370 , and thus the number of replacement target texts may be relatively small. Conversely, if the first threshold is set to be relatively high, even if the frequency of use of the search result of the dictionary DB 1370 is lower than the first threshold, there is a high probability of being determined as the replacement target text, so a relatively large number of replacement target texts can be determined.

The alternative text generating module 1350 is a module configured to generate an alternative text that is expected to be uttered by the user and has a high phonological similarity with respect to the alternative text identified by the alternative text identification module 1340 . . In one embodiment, the processor 1200 obtains the identified replacement target text from the replacement target text identification module 1340 , and uses the data and program code of the replacement text generation module 1350 to replace ( You can generate alternative text that can be replaced. In an embodiment, the processor 1200 may generate at least one alternative text using the alternative text generation module 1350 .

'Alternative text' is an example of text that is phonetically similar or predicted by a user to pronounce a specific entity name, keyword, or word or phrase that can be identified as a slot by the natural language understanding model 1320 . The alternative text may be an example of text that is expected to be output after a voice input input from the user is converted through the ASR model 1310 . For example, if the alt text is 'Pepperoni Pizza', the alt text is 'Pepperoni', 'Paperoni', 'Peperoni', which may be uttered with incorrect or similar pronunciation when the user pronounces the alt text. 'poroni' and the like. For example, if the replacement target text is 'Eonju-ro 30-gil', it is predicted that the user will pronounce 'Eonju-ro 30-gil', and the pronunciation column is similar to 'Anju-ro 30-gil' and 'Unju-ro 30-gil'. , 'Eonjo-ro 30-gil', etc.

In an embodiment, the processor 1200 extracts a phoneme sequence related to the replacement target text by using the data and the program code of the replacement text generation module 1350, and adds the phoneme sequence to the phonetic relevance of the pronunciation sequence. Based on the search for text similar to the extracted pronunciation column among the words pre-stored in the dictionary DB 1370, an alternative text is generated using at least one text having a high similarity to the extracted pronunciation column based on the search result. can create Here, the similarity may be calculated as a confidence score. A detailed method for the processor 1200 to generate the alternative text based on the similarity of the pronunciation column will be described in detail with reference to FIGS. 8 and 9 .

In an embodiment, the processor 1200 converts the replacement target text into an embedding vector by using the data and the program code of the alt text generation module 1350, and uses a neural network model to , a text having a vector value similar to the converted embedding vector may be generated, and alternative text may be generated using the generated text. The processor 1200 may convert the replacement target text into an embedding vector by using the word embedding model 1380 . The word embedding model 1380 is a model configured to quantify at least one word constituting the input text. The word embedding model 1380 is a model for digitizing words as vector values, and for example, a CBOW (Continuous Bag of Words) embedding model, Skip-Gram embedding model, word2vec, etc. It may include a well-known embedding model. The processor 1200 may generate an alternative text having a vector value similar to an embedding vector using, for example, a generative network. A specific method for the processor 1200 to generate the alternative text by using the generation model will be described in detail with reference to FIGS. 10 and 11 .

In an embodiment, the processor 1200 converts the replacement target text into a sound signal using the TTS model 1330 , outputs the converted sound signal, and outputs the outputted text using the ASR model 1310 . It is possible to convert an acoustic signal into output text. The processor 1200 may generate the replacement text by replacing the replacement target text with the output text using the data and the program code of the replacement text generation module 1350 . A specific method for the processor 1200 to generate the alternative text using the ASR model 1310 , the TTS model 1330 , and the alternative text generation module 1350 will be described in detail with reference to FIGS. 12 and 13 .

The training candidate text generation module 1360 is configured to input the replacement text generated by the replacement text generation module 1350 to the replacement target text identified from the input text, thereby inputting the learning by the natural language understanding model 1320 . It is a module configured to generate learning candidate texts. In an embodiment, the processor 1200 uses the data and program code of the learning candidate text generation module 1360 to replace the replacement target text in the input text with the replacement text, or the remaining words except the replacement target text in the input text. Alternatively, by performing paraphrasing of concatenating a phrase with an alternative text, at least one learning candidate text may be generated. The 'learning candidate text' means text that becomes a candidate of training data used as input data for learning by the natural language understanding model 1320 .

For example, if the input text is "Order for delivery at 30 gil with 3 slices of pepperoni pizza", the processor 1200 uses the learning candidate text generation module 1360 to generate 'pepperoni' identified as the replacement target text. By replacing 'Pepperoni', 'Paperoni', 'Pephoroni', etc. respectively, "Order 3 slices of pepperoni pizza for delivery at 30 gil", "Order for delivery at 30 gil with 3 slices of pepperoni pizza." You can create three learning candidate texts that include "Do it" or "Order delivery by 30 gil with 3 slices of pepperoni pizza." However, the present invention is not limited thereto, and when a plurality of replacement target texts are identified from the input text, the processor 1200 may generate a learning candidate text by using the replacement text for each of the plurality of replacement target texts. For example, from the input text "Order for delivery of 3 pepperoni pizzas by Unjuro 30 gil", 'Pepperoni', a slot indicating the type of pizza topping, and 'Eonjuro 30 gil', a slot indicating an address, are replaced with text. When identified, the processor 1200 generates alternative texts for each of 'Pepperoni' and 'Eonjuro 30-gil' using the alternative text generation module 1350, and for 'Pepperoni', 'Pepperoni' and 'Pape' 'Roni' and 'Pephoroni', and for 'Eonju-ro 30-gil', 'Anju-ro 30-gil', 'Unju-ro 30-gil', and 'Eonjo-ro 30-gil' can be created as alternative texts. As in the above example, when three alternative texts are generated for each of 'Pepperoni' and 'Eonju-ro 30-gil', the processor 1200 combines the alternative texts using the learning candidate text generation module 1360, so that a total of 9 Can generate learning candidate texts.

When the training candidate text is generated by replacing the replacement target text identified as a slot in the input text with the replacement text, the training candidate text generation module 1360 converts the domain, intent, and slot information of the training candidate text into the natural language understanding model. can be provided to In an embodiment, the training candidate text generation module 1360 identifies a slot of the alt text replaced within the training candidate text as the same slot as the slot of the replacement target text, and divides the slot information and the replacement target text into the natural language understanding model 1320 ) can be provided.

In an embodiment, the learning candidate text generation module 1360 may provide the replacement text, slot information of the replacement text, and the replacement target text identified from the input text to the natural language understanding model. The slot information may include, for example, a type of an identified slot, a slot value, and a slot replacement value. Here, the slot replacement value means the replacement target text having the same slot value as the slot in which the replacement text is identified, that is, the original text. For example, if the learning candidate text is "Order for delivery by 30 gil with 3 slices of pepperoni pizza", "Order for delivery by 30 gil with 3 slices of pepperoni pizza", "Order for delivery with 3 slices of pepperoni pizza by 30 gil" In the case of "Order delivery by road", the learning candidate text generation module 1360 generates 'pizza topping' which is a slot type for each of 'peparoni', 'paperoni', and 'peporoni' which are alternative texts among the learning candidate texts. , a slot value for the pizza topping, and 'pepperoni', which is the original text replaced with the alternative text, may be provided to the natural language understanding model.

The processor 1200 may perform training by inputting at least one training candidate text to the natural language understanding model 1320 as input data. In one embodiment, when the intent action, intent object, and slot of the input text are all identified by the natural language understanding model 1320 , the learning candidate text is added to the intent action, intent object, and slot of the input text. By inheriting all related information, it is possible to have the same intent action, intent object, and slot information as the input text. In an embodiment, the processor 1200 may learn the natural language understanding model by using the target text and slot information including the slot type and slot value of the replacement text in the training candidate text. In this case, the processor 1200 may select any one natural language understanding model corresponding to the domain and intent of at least one learning candidate text from among the plurality of pre-trained natural language understanding models 1320a to 1320c. The processor 1200 may update the natural language understanding model by inputting at least one learning candidate text, an intent action, an intent object, and slot information into the selected natural language understanding model as training data to perform learning. .

When the processor 1200 does not interpret the input text using the pre-learned natural language understanding model, the processor 1200 interprets the at least one learning candidate text using the pre-learned natural language understanding model, so that at least one learning The domain of the candidate text, the intent action, and the intent object may be identified, and slot tagging may be performed. In this case, the processor 1200 may perform learning and update the natural language understanding model by using the identified domain, intent action, intent object, and slot information as training data of the natural language understanding model.

In an embodiment, the processor 1200 may generate a new natural language understanding model without updating the previously learned natural language understanding model. The processor 1200 counts the number of the input text and the at least one learning candidate text, and when the counted number is greater than a third preset threshold, the input text and the at least one learning candidate text and the identified domain, intent, and by performing learning using slot information as input data for learning, a new natural language understanding model may be generated. A specific method for the processor 1200 to update any one of the plurality of pre-learned natural language understanding models 1320a to 1320c or to generate a new natural language understanding model will be described in detail with reference to FIG. 16 .

In an embodiment, the processor 1200 may perform learning by not using all of the at least one learning candidate text, but by selecting only a part and inputting it as input data to the natural language understanding model. In this case, the processor 1200 transmits at least one learning candidate text to the client device 2000 through the communication interface 1100, and the text selected based on a user input for selecting at least one of the at least one learning candidate text. An identification value of , may be received from the client device 2000 . The processor 1200 may select at least one text based on the received identification value, and input the selected at least one text and the input text to the natural language understanding model as input data to learn.

In an embodiment, the processor 1200 may generate a text normalizer module when learning using the input text and the learning candidate text is performed. The text normalization module may include a rule table that receives the replacement text in the training candidate text and outputs the replacement target text that is the original text of the replacement text. For example, when the replacement text in the learning candidate text is 'Pepperoni', 'Papperoni', and 'Pephoroni', and the replacement target text is 'Pepperoni', the text normalization module is 'Pepperoni' as the input text as shown in Table 1 below. It may include a rule table defining 'Pepperoni', 'Pepperoni', 'Pephoroni', and 'Pepperoni' as output text.

Input Output pepperoni pepperoni paperoni pepperoni pepperoni pepperoni

The text normalization module may provide input text and output text to a Dialog Manager 3300 (see FIG. 22 ) or a Response Generator model 3400 (see FIG. 22 ).

In the above-described embodiment, although it has been described that the text normalization module includes a rule table, it is not limited thereto. The text normalization module may be implemented as a neural network model. When the text normalization module is implemented as a neural network model, the neural network model can be trained by receiving the alternative text in the training candidate text as input data and the original text of the alternative text as output data that is the groundtruth. . When an alternative text is input to the pre-trained neural network model, the original text may be output through training. For example, when alternative texts 'Pepperoni', 'Paperoni', and 'Pephoroni' are input to the pre-trained neural network model, the original text 'Pepperoni' may be output through training.

3 is a block diagram illustrating a configuration of a client device 2000 according to an embodiment of the present disclosure. The configuration shown in FIG. 3 is a partial configuration of the client device 2000 , and the client device 2000 may further include a configuration not shown in FIG. 3 .

The client device 2000 may be a computing device that receives a user input for inputting input text, and displays a graphical user interface indicating a result of learning using the input text and at least one learning candidate text. . The client device 2000 is, for example, a smart phone, a tablet PC, a PC, a laptop, a smart TV, a mobile phone, a personal digital assistant (PDA), a media player, a global positioning system (GPS) device, an e-book terminal, or a digital broadcasting terminal. , navigation, kiosks, MP3 players, digital cameras, home appliances and other mobile or non-mobile computing devices. Also, the client device 2000 may be a wearable device such as a watch, glasses, a hair band, and a ring having a communication function and a data processing function. However, the present invention is not limited thereto, and the client device 2000 may include any type of computing device capable of transmitting and receiving data through a network with the server 1000 for voice recognition.

Referring to FIG. 3 , the client device 2000 may include a communication interface 2100 , a processor 2200 , and a memory 2300 , an input unit 2400 , and an output unit 2500 .

The communication interface 2100 may include one or more components for data communication with the server 1000 or an external device (not shown). The communication interface 2100 may transmit/receive data for voice recognition and voice assistant service to and from the server 1000 or an external device (not shown) under the control of the processor 2200 . Communication interface 2100 is, for example, wired LAN, wireless LAN (Wireless LAN), Wi-Fi (Wi-Fi), Bluetooth (Bluetooth), Zigbee (zigbee), WFD (Wi-Fi Direct), infrared communication (IrDA, Infrared Data Association), BLE (Bluetooth Low Energy), NFC (Near Field Communication), WiBro (Wireless Broadband Internet, Wibro), WiMAX (World Interoperability for Microwave Access, WiMAX), SWAP (Shared Wireless Access Protocol), WiGig Data communication may be performed with the server 1000 or another external device (not shown) using at least one of a data communication method including (Wireless Gigabit Allicance, WiGig) and RF communication.

The processor 2200 may execute one or more instructions of a program stored in the memory 2300 , and a program code. The processor 2200 may include hardware components that perform arithmetic, logic, input/output operations, and signal processing. The processor 2200 is, for example, a central processing unit (Central Processing Unit), a microprocessor (microprocessor), a graphic processor (Graphic Processing Unit), ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), and FPGAs (Field Programmable Gate Arrays) may be configured as at least one, but is not limited thereto.

The memory 2300 may include, for example, a flash memory type, a hard disk type, a multimedia card micro type, or a card type memory (eg, SD or XD memory). etc.), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, magnetic disk, optical disk, and It may include volatile memory of random access memory (RAM) or static random access memory (SRAM).

The memory 2300 stores instructions and program codes readable by the processor 1200 . In the following embodiment, the processor 2200 may be implemented by executing instructions or codes of a program stored in the memory 2300 .

The memory 2300 may include an operating system 2310 and an application driving module 2320 . The operating system 2310 is software used to control an operation or function of the client device 2000 . The processor 2200 may execute an application on the operating system 2310 .

The application driving module 2320 is a module used to drive an application program. The application driving module 2320 may be used to drive the language model learning application 2330 and the plurality of applications 2332 and 2334 . The processor 2200 may load one or more application programs from the memory 2300 using data and program codes of the application driving module 2320 and execute the one or more application programs. In an embodiment, the processor 2200 may execute the language model learning application 2330 using the application driving module 2320 .

The language model learning application 2330 is a program provided to a developer to learn or develop a new language model or to update a previously learned language model. The language model learning application 2330 may include a tool for developers to interact with the server 1000 for learning the language model. The language model learning application 2330 may be, for example, an application for learning a natural language understanding model.

In an embodiment, the processor 2200 executes the language model learning application 2330 to display a graphic user interface (GUI) that receives a user input for inputting input text for learning the language model on the display unit ( 2510) can be displayed. The processor 2200 may receive the input text through the user input unit 2410 and store it in the memory 2300 . In an embodiment, the processor 2200 may transmit the input text received through the graphical user interface to the server 1000 by controlling the communication interface 2100 .

In an embodiment, the processor 2200 may receive domain, intent, and slot tagging information identified by the server 1000 from the input text by controlling the communication interface 2100 . The processor 2200 may display the identified domain, intent, and slot tagging information on the display unit 2510 by executing the language model learning application 2330 . In this case, the processor 2200 may display a numerical value indicating a confidence score for each of one or more domains and one or more intents together. Similarly, the processor 2200 may display a numerical value indicative of a confidence score for one or more slots automatically tagged by the server 1000 .

However, the present invention is not limited thereto, and the client device 2000 stores the previously learned natural language understanding model in the memory 2300 , and the processor 2200 stores the previously learned natural language understanding model stored in the memory 2300 . It can be used to detect domains and intents from input text and perform slot tagging. The processor 2200 detects the domain, intent, and slot from the input text. Since the specific method of performing slot tagging is the same as the method performed by the processor 1200 (refer to FIG. 2 ) of the server 1000, a detailed description thereof is omitted.

In one embodiment, the processor 2200 executes the language model learning application 2330, thereby a graphical user for receiving user input for selecting or determining, respectively, a domain, an intent action, and an intent object corresponding to the input text. The interface may be displayed on the display unit 2510 . The user input unit 2410 may receive a user input for inputting a domain into which input text is classified, an intent action, and an intent object through a graphical user interface. The processor 2200 may determine a domain, an intent action, and an intent object related to the input text based on a user input received through the user input unit 2410 .

The language model learning application 2330 is identified by the server 1000 or based on a domain determined based on user input, for example, based on movies, photos, sports, weather, aviation, etc., one or more application program interfaces (Applications). Program Interface, API). In one embodiment, the language model learning application 2330 may include example data (eg, example text) used to select or determine an existing domain, as well as extend an existing domain, or create a new domain. may be In one embodiment, the processor 2200 may receive user input selecting one or more application program interfaces associated with the identified or determined domain with respect to the input text by executing the language model learning application 2330 .

The processor 2200 may display, on the display unit 2510 , a graphical user interface configured to receive a user input for selecting a replacement target text within the input text by executing the language model learning application 2330 . In this case, the user input unit 2410 may receive any one of a mouse input, a touch input, and a drag input for selecting a specific word or phrase in the input text. The processor 2200 may determine the selected word or phrase as the replacement target text based on the user input received through the user input unit 2410 .

In an embodiment, the processor 2200 may receive a user input for selecting a word or phrase corresponding to a slot related to a domain and an intent detected from the input text through the graphical user interface. The processor 2200 executes the language model learning application 2330 to display at least one example slot of slot tagging corresponding to the word or phrase determined as the slot, and a user input for selecting any one of the at least one example slot Slot tagging may be performed based on . An embodiment of determining a slot from an input text based on a user input and performing slot tagging will be described in detail with reference to FIG. 19A .

In one embodiment, the processor 2200 is a graphical user receiving a user input selecting at least one of context information considered to generate an alt text to be substituted for the replacement target text when the replacement target text is selected. The display unit 2510 may be controlled to display the interface. The context information may include, for example, at least one of the age, gender, region, language, and dialect of the application user using the alternative text. In an embodiment, the processor 2200 may receive a user input for selecting a characteristic of an application in consideration of context information. An embodiment of selecting context information used to generate alternative text based on a user input will be described in detail with reference to FIG. 19B .

The processor 2200 controls the communication interface 2100 to receive, from the server 1000, at least one learning candidate text generated by replacing the replacement target text among the input text with the replacement text expected to be uttered by the user. can The at least one learning candidate text is generated by the processor 1200 of the server 1000 and is the same as the at least one learning candidate text described with reference to FIG. 2 , and thus a redundant description will be omitted.

In FIG. 2 , it has been described that the server 1000 is the operating subject for generating at least one learning candidate text by identifying the replacement target text among the input texts and replacing the identified replacement target text with the replacement text, but is limited thereto. it is not In an embodiment, the client device 2000 may generate at least one learning candidate text by identifying the replacement target text from the input text and replacing the identified replacement target text with the replacement text.

The processor 2200 executes the language model learning application 2330 to display a graphical user interface on the display unit 2510 for receiving a user input for selecting at least one of the input text and the at least one learning candidate text. can In this case, the user input unit 2410 may receive a user input for selecting at least one of the input text and the at least one learning candidate text through the graphic user interface displayed on the display unit 2510 . In one embodiment, the processor 2200 executes the language model learning application 2330, so that the word corresponding to the alternative text replaced by the server 1000 among the words included in the at least one learning candidate text is another word. They can be displayed to distinguish them from others.

The processor 2200 may select at least one text based on the received user input and transmit an identification value related to the selected at least one text to the server 1000 by controlling the communication interface 2100 .

In an embodiment, the processor 2200 executes the language model learning application 2330 to display at least one learning candidate text output through a personalized TTS model reflecting context information of a specific region or a specific person on the display unit 2510 ) can be displayed on the At least one learning candidate text output through the personalized TTS model may be received from the server 1000 . In an embodiment, the processor 2200 may reflect the context information of at least one learning candidate text output from the server 1000 through the personalized TTS model, and output it as an audio signal through the sound output unit 2520 . have. A specific embodiment of displaying at least one learning candidate text output through the personalized TTS model or outputting it as an audio signal will be described in detail with reference to FIG. 21 .

The input unit 2400 may include a user input unit 2410 and a microphone 2420 . The user input unit 2410 may include, for example, a mouse, a keyboard, a key pad, a dome switch, and a touch pad (contact capacitive method, pressure resistance film method, infrared sensing method, surface ultrasonic conduction). method, integral tension measurement method, piezo effect method, etc.), a jog wheel, a jog switch, etc., but is not limited thereto. When the user input unit 2410 is a touch pad, it may be integrated with the display unit 2510 to form a touch screen. In this case, the user input unit 2410 may receive at least one of a touch input, a pinch input, a drag input, a swipe input, and a scroll input through the touch screen. In an embodiment, the user input unit 2410 may receive a user's gesture, for example, a stretch gesture, a pinch gesture, a selection and hold gesture, a guide gesture, a swiping action, a dragging action, and the like.

The microphone 2420 may receive the user's utterance and process it as an electrical voice signal. The processor 2200 may receive a voice input from the user through the microphone 2420 and obtain a voice signal from the received voice input. In an embodiment, the processor 2200 converts a sound received through the microphone 2420 into an acoustic signal, and removes noise (eg, non-voice component) from the acoustic signal to obtain a voice signal. .

The output unit 2500 may include a display unit 2510 and a sound output unit 2520 .

The display unit 2510 may display information processed by the client device 2000 under the control of the processor 2200 . For example, the display unit 2510 may display a graphical user interface (GUI) for learning a language model output by the language model learning application 2330 . In an embodiment, the display unit 2510 may form a layer structure with a touch pad to configure a touch screen. In this case, the display unit 2510 may be used as an input device in addition to an output device.

The display unit 2510 includes a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, and a three-dimensional display ( 3D display) and electrophoretic display (electrophoretic display) may be configured as at least one.

The sound output unit 2520 may output audio data. The sound output unit 2520 may include, for example, a speaker, a buzzer, or the like.

In the embodiment disclosed through FIGS. 1 to 3 , the server 1000 determines that the output text of the ASR model 1310 (see FIG. 2 ) does not match the text included in the plurality of pre-trained natural language understanding models 1320a to 1320c. Substitute text that is uttered incorrectly or is difficult to pronounce correctly because it is not correctly known, or that the user knows incorrectly when uttering, with an alternative text that is phonetically similar and predicted to be pronounced, and, as a result of the substitution, By learning the natural language understanding model by generating at least one learning candidate text that can be a replacement candidate, it is possible to provide a more accurate natural language understanding model when the natural language understanding model is trained using only the input text.

The client device 2000 of the present disclosure allows a user, that is, a developer who wants to learn a language model to generate a new natural language understanding model or update an existing natural language understanding model, domain, intent, and slot tagging for input text. By providing a tool for developers to perform through user input, it is possible to develop a language model specialized for a desired application by the developer. In addition, the client device 2000 of the present disclosure provides a developer tool for selecting at least one of the at least one learning candidate text generated by the server 1000 , thereby performing language model learning with only the text desired by the developer. This can improve the accuracy of language model learning.

4 is a flowchart illustrating an embodiment in which the server 1000 of the present disclosure learns a natural language understanding model.

In step S410 , the server 1000 receives the input text input by the user from the client device 2000 . The client device 2000 may receive input text through a typing input through the user input unit 2410 (refer to FIG. 3 ) and transmit the input text to the server 1000 . However, the present invention is not limited thereto. In an embodiment, the client device 2000 may receive a voice input from the user through the microphone 2420 (refer to FIG. 3 ) and transmit a voice signal including the voice input to the server 1000 . The processor (1200, see Fig. 2) of the server 1000 uses the ASR model (1310, see Fig. 2) to perform ASR (Auto Speech Recognition) on the received voice signal, thereby converting the voice signal into text, It can receive input text from it.

In operation S420 , the server 1000 may identify a replacement target text requiring replacement among at least one word included in the input text. 'Substitute text' is a word that a user who uses an application including a language model that is updated through learning or that is newly created for a voice command is uttered incorrectly or is difficult to pronounce correctly ( word) or phrase. For example, the replacement target text may include a named entity, a keyword, a place, a place name, a movie title, or a game term. In an embodiment, the replacement target text may be a slot identified by interpreting the input text using a pre-learned natural language understanding model. For example, if the input text is "Order for delivery of 3 pepperoni pizzas by Unjuro 30 Gil", the replacement target text may be 'Pepperoni' or 'Eonjuro 30 Gil' identified by an entity name or slot.

In one embodiment, the replacement target text may be determined based on a user input. In an embodiment, the client device 2000 may receive a user input for selecting a word or phrase included in the input text, and transmit identification information of the selected word or phrase to the server 1000 based on the received user input. . The processor 1200 of the server 1000 may identify a word or phrase selected by a user input based on the identification information received from the client device 2000 , and determine the identified word or phrase as the replacement target text.

In an embodiment, the server 1000 may identify the replacement target text based on a search result for a word pre-stored in the dictionary DB 1370 (refer to FIG. 2 ). In one embodiment, the processor 1200 of the server 1000 parses the input text in units of words, morphemes, and phrases, searches for at least one word parsed in the dictionary DB 1370 , and the dictionary DB 1370 . Based on the search result of , a word that is not searched or has a frequency of use lower than a preset first threshold may be determined as the replacement target text.

In an embodiment, the processor 1200 of the server 1000 may identify a slot from the input text by interpreting the input text using a pre-learned natural language understanding model, and may determine the identified slot as the replacement target text.

A specific embodiment in which the server 1000 identifies the text to be replaced will be described in detail with reference to FIGS. 5 to 7 .

In step S430 , the server 1000 generates the replacement target text that is expected to be uttered by the user with respect to the replacement target text and has a high phonological similarity. The 'alternative text' is an example of a text that is phonetically similar to a specific entity name, keyword, or a word or phrase identified as a slot by a pre-learned natural language understanding model, or is predicted to be pronounced by the user. For example, if the alt text is 'Pepperoni Pizza', the alt text is 'Pepperoni', 'Paperoni', 'Peperoni', which may be uttered with incorrect or similar pronunciation when the user pronounces the alt text. 'poroni' and the like.

The server 1000 generates the replacement target text that can replace the replacement target text identified in step S420 . In an embodiment, the processor 1200 may generate an alternative text capable of replacing the replacement target text by using the alternative text generation module 1350 (refer to FIG. 2 ).

When the replacement target text corresponds to a slot automatically identified as a result of analyzing the input text using the pre-learned natural language understanding model, the replacement text may acquire the same slot information as the slot of the replacement target text. In other words, the replacement text may not only replace the replacement target text, but also inherit and acquire slot information of the replacement target text. For example, if the slot identified from the input text is 'Pepperoni', which means the topping of pizza, the replacement text for 'Pepperoni', 'Pepperoni', 'Pepperoni', and 'Pephoroni' will also be replaced. The same topping of pizza as slot information of the text 'pepperoni' may be obtained as slot information.

In an embodiment, the server 1000 extracts a phoneme sequence related to the replacement target text, and based on phonetic relevance of the pronunciation sequence, a word pre-stored in the dictionary DB 1370 (refer to FIG. 2 ). Text similar to the extracted pronunciation string may be searched for, and an alternative text may be generated using at least one text having a high similarity to the extracted pronunciation string based on the search result.

In another embodiment, the server 1000 converts the replacement target text into an embedding vector, and uses a neural network model to generate text having a vector value similar to the transformed embedding vector, Alternative text can be created using the generated text.

In another embodiment, the server 1000 converts the replacement target text into a sound signal (wave signal) using the TTS model 1330 (see FIG. 2), and outputs the converted sound signal, and the ASR model 1310, 2) may be used to convert the output sound signal into output text. The server 1000 may generate an alternative text by using the output text.

A specific embodiment in which the server 1000 generates the alternative text will be described in detail with reference to FIGS. 8 to 13 .

In step S440 , the server 1000 generates at least one learning candidate text to be used for learning by replacing the replacement target text with the replacement text. The 'training candidate text' means text that becomes a candidate of training data used as input data for learning a language model. In an embodiment, the processor 1200 uses the replacement text generation module 1350 to replace the replacement target text in the input text with the replacement text, or to replace the remaining words or phrases in the input text except for the replacement target text with the replacement text. By performing concatenating paraphrasing, at least one learning candidate text may be generated.

For example, if the input text is "Order for delivery at 30 gil with 3 slices of pepperoni pizza", the server 1000 returns 'pepperoni' identified as the replacement target text to 'pepperoni', 'paperoni', ' Pepperoni', etc., "Order for delivery at 30 gil with 3 slices of pepperoni pizza", "Order for delivery with 3 slices of pepperoni pizza at 30 gil", or "3 slices of pepperoni pizza You can create 3 learning candidate texts that include "Order delivery to 30 Gil as Eonjuro."

In operation S450, the server 1000 trains a natural language understanding model by using at least one text among the input text and the at least one learning candidate text as training data.

Between steps S440 and S450, the server 1000 interprets the input text and at least one learning candidate text using the pre-learned natural language understanding model, thereby generating a domain from the input text and the at least one learning candidate text. An intent and a slot may be identified. In an embodiment, the server 1000 may perform learning by using the identified domain, intent, and slot information as input data to the natural language understanding model.

In one embodiment, the server 1000 includes a plurality of natural language understanding models 1320a to 1320c pre-stored in the memory 1300 (see FIG. 2 ) based on the domains and intents identified from the input text and at least one learning candidate text. Any one natural language understanding model may be selected, and learning may be performed using the selected natural language understanding model.

However, the present invention is not limited thereto, and in another embodiment, the server 1000 may generate a new natural language understanding model through learning using the input text and at least one learning candidate text.

In an embodiment, the server 1000 may perform learning by not using all of the at least one learning candidate text, but selecting only a part and inputting it as input data to the natural language understanding model. The server 1000 transmits the at least one learning candidate text generated in step S440 to the client device 2000, and an identification value regarding the selected text based on a user input for selecting at least one of the at least one learning candidate text. It may be received from the client device 2000 . The server 1000 may select at least one text based on the received identification value, and input the selected at least one text and the input text as input data to the natural language understanding model to learn.

5 is a flowchart of an embodiment in which the server 1000 of the present disclosure determines an alternative target text based on a user input. The steps of the method shown in FIG. 5 embodied step S420 of FIG. 4 , and step S510 is performed after step S410 shown in FIG. 4 is performed. After step S530 of FIG. 5 is performed, step S430 shown in FIG. 4 is performed.

In operation S510 , the server 1000 receives, from the client device 2000 , a user input for selecting at least one word among input texts. In an embodiment, the client device 2000 may receive a user input for selecting a word or phrase included in the input text. For example, the client device 2000 may receive a mouse click, a drag input, and a touch and drag input for selecting a specific word or phrase from among input text. The client device 2000 may select a word or phrase from the input text based on the received user input. The word or phrase selected by the user input may include, for example, an entity name, a keyword, a place, a place name, a movie title, or a game term, but is not limited thereto.

In an embodiment, the client device 2000 may receive a user input for selecting an alternative target text based on a development purpose of an application using a language model. For example, when the purpose of developing the application is to consider the dialect of a specific region, considering the language used (eg, English, Spanish, etc.), or considering users of a specific age group and a specific gender, the client device It is possible to receive a user input for selecting a word that is incorrectly pronounced as a word not included in the natural language understanding model due to a heavy dialect or is highly likely to be replaced according to the language used as the replacement target text. In an embodiment, the client device 2000 receives a user input for selecting an application development purpose in consideration of an application user's age, region, language, gender, etc., and automatically recommends an alternative target text according to the received user input You may. A specific embodiment in which the client device 2000 receives a user input for selecting an alternative text will be described in detail with reference to FIG. 18 .

The client device 2000 may transmit identification information regarding the selected word or phrase as the replacement target text to the server 1000 . The server 1000 may receive identification information about a word or phrase selected by a user input from the client device 2000 .

In step S520, the server 1000 identifies at least one selected word based on the user input. In an embodiment, the server 1000 may identify the word or phrase selected by the user input from among the input text by analyzing the identification information received from the client device 2000 .

In step S530, the server 1000 determines the identified word as the replacement target text. In an embodiment, the server 1000 may determine not only the identified word but also the phrase as the replacement target text.

6 is a flowchart of an embodiment in which the server 1000 of the present disclosure identifies a text to be replaced based on a search result of the dictionary DB 1370 (refer to FIG. 2 ). The steps of the method shown in FIG. 6 embodied step S420 of FIG. 4 , and step S610 is performed after step S410 shown in FIG. 4 is performed. After step S630 of FIG. 6 is performed, step S430 shown in FIG. 4 is performed.

In step S610, the server 1000 extracts at least one word by parsing the input text. In one embodiment, the server 1000 parses the input text into word, morpheme, and syntactic units, and uses linguistic features (eg, grammatical elements) of the morpheme, word, or phrase to at least from the parsed text. One word can be extracted.

In step S620, the server 1000 searches at least one word extracted as a result of the parsing in a dictionary DB 1370 (refer to FIG. 2) that stores pronunciation string information or embedding vector information. do. The dictionary DB 1370 may be configured as a database that stores pronunciation column information on words and embedding vectors on pronunciation columns included in each of the words. The dictionary DB 1370 may be, for example, a pronunciation column dictionary. In an embodiment, the dictionary DB 1370 may include a look-up table including pronunciation column information and embedding vectors for each word.

In an embodiment, the server 1000 may convert at least one word extracted as a result of parsing into a pronunciation sequence or into an embedding vector. In an embodiment, the server 1000 may convert at least one word into an embedding vector using a word embedding method. For example, the server 1000 may convert at least one word into an embedding vector using an embedding model such as Bag of Words or word2vec.

The server 1000 may search the dictionary DB 1370 for pronunciation column information or embedding vectors converted from at least one word.

In step S630 , the server 1000 determines, as the replacement target text, a word that is not searched or has a low frequency of use, based on the search result of the dictionary DB 1370 . In an embodiment, the server 1000 searches the dictionary DB 1370 for at least one word extracted as a result of parsing the input text, and the search is not performed because it is not included in the search result dictionary DB 1370, or the dictionary DB A word whose frequency of use of the word 1370 is lower than a preset first threshold may be determined as the replacement target text.

7 is a flowchart of an embodiment in which the server 1000 of the present disclosure automatically identifies an alternative target text using a natural language understanding model. Among the steps of the method shown in FIG. 7 , steps S710 to S730 are steps performed between steps S410 and S420 of FIG. 4 . Step S740 of FIG. 7 is a concrete embodiment of step S420 illustrated in FIG. 4 . After step S740 of FIG. 7 is performed, step S430 shown in FIG. 4 is performed.

In step S710 , the server 1000 detects a domain into which the input text is classified by interpreting the input text using the natural language understanding model. In an embodiment, the server 1000 uses a domain identification model included in any one of the pre-trained first natural language understanding model 1320a (refer to FIG. 2) to the third natural language understanding model 1320c (refer to FIG. 2). By analyzing the input text, it is possible to detect a domain to which the input text belongs or is category information that can be classified. In one embodiment, the server 1000 may identify one or more domains from the input text. In an embodiment, the server 1000 may calculate the relation between the input text and the domain as a numerical value by interpreting the input text using the domain identification model. In an embodiment, the server 1000 may calculate a degree of relevance between each of the one or a plurality of domains and the input text as a probability value, and determine a domain having a high probability value among the calculated probability values as a domain into which the input text is classified.

In step S720 , the server 1000 detects the intent of the input text by interpreting the input text using the natural language understanding model. In an embodiment, the server 1000 may detect an intent from the input text using an intent identification model of a pre-learned natural language understanding model. In an embodiment, the server 1000 detects an intent action from the input text using an intent action identification model included in any one of the first natural language understanding model 1320a to the third natural language understanding model 1320c, and , an intent object can be detected from the input text using the intent object identification model.

The intent action refers to an action performed by the input text, for example, search, posting, play, purchase, order, and the like. The server 1000 may detect an intent action from the input text by performing syntactic analysis or semantic analysis using the intent action identification model. In an embodiment, the server 1000 parses the input text into units of morphemes, words, or phrases using the intent action identification model, and uses the parsed morpheme, word, or phrase. The meaning of a word or phrase extracted from the parsed text can be inferred using linguistic features (eg, grammatical elements). The server 1000 may determine an intent action corresponding to the meaning of the inferred word or phrase by comparing the meaning of the inferred word or phrase with predefined intents provided from the natural language understanding model.

The intent object means an object related to the identified intent action. The intent object is an object that is the target of the detected intent action, and may mean, for example, a movie, a photo, sports, weather, or aviation. The server 1000 may detect an intent object related to an intent action from the input text by interpreting the input text using the intent object identification model.

In step S730, the server 1000 identifies a slot from the input text using the natural language understanding model, and performs slot tagging. In one embodiment, the server 1000 identifies a slot from the input text using a slot tagging model included in any one of the pre-trained first natural language understanding model 1320a to the third natural language understanding model 1320c, Slot tagging may be performed that associates the identified slots with domains, intent actions, and intent objects. The slot means variable information for obtaining detailed information related to a domain, an intent action, and an intent object from an input text or determining a detailed action. For example, the slot may include a named entity, a keyword, a place, a place name, a movie title, or a game term. According to an embodiment, a slot is information related to an intent action and an intent object, and a plurality of types of slots may correspond to one intent.

In one embodiment, the server 1000 may identify one or more slots from the input text.

In step S740, the server 1000 determines the text corresponding to the identified slot as the replacement target text. When there are a plurality of slots identified from the input text, the server 1000 may determine any one slot as the replacement target text.

In an embodiment, the server 1000 may determine the selected at least one slot as the replacement target text based on a user input for selecting at least one of the plurality of slots. A user input for selecting at least one of the plurality of slots may be received through the client device 2000 . The client device 2000 displays a list including a plurality of slots identified from the input text, receives a user input for selecting at least one of the plurality of slots included in the list, and receives at least one based on the received user input You can select a slot of The client device 2000 may transmit identification information regarding at least one selected slot to the server 1000 . A specific embodiment in which the client device 2000 receives a user input for selecting at least one slot to be determined as the replacement target text from among a plurality of slots will be described in detail with reference to FIG. 18 .

In an embodiment, the server 1000 may determine any one of the plurality of slots as the replacement target text based on the characteristics of the application using the replacement text. In an embodiment, the server 1000 selects any one of a plurality of slots based on characteristic information such as age, gender, region, language, dialect, and intonation of a user who uses an application using a language understanding model. It can be determined by the text to be replaced. In an embodiment, the client device 2000 may receive a user input for selecting at least one of characteristic information of a plurality of applications, and transmit the selected characteristic information to the server 1000 based on the received user input. The server 1000 may determine any one of the plurality of slots as the replacement target text based on the characteristic information of the application received from the client device 2000 . A specific embodiment in which the client device 2000 receives a user input for selecting application characteristic information will be described in detail with reference to FIG. 19B .

Although not shown in FIG. 7 , the server 1000 may generate an alternative text that is expected to be uttered by the user with respect to the replacement target text and has a high phonological similarity (refer to step S430 of FIG. 4 ). In this case, the replacement text may be tagged with the same slot as the slot in which the replacement target text is tagged. In other words, the replacement text may inherit slot information of the replacement target text as it is.

The domain detected in step S710, the intent detected in step S720, and slot tagging information performed in step S730 may be provided as input data to the natural language understanding model.

Although steps S710 to S730 shown in FIG. 7 are shown to be performed before the step S420 of determining the replacement target text, the embodiment of the present disclosure is not limited to FIG. 7 . Detecting the domain and intent from the input text and performing slot tagging may be performed after generating the learning candidate text. A specific embodiment related thereto will be described in detail with reference to FIG. 14 .

8 is a diagram illustrating an embodiment in which the server 1000 of the present disclosure obtains the alternative text 830 from the alternative text 810 .

Referring to FIG. 8 , the server 1000 may obtain the replacement text 830 from the replacement target text 810 by using the dictionary DB 800 . The dictionary DB 800 stores information about a pronunciation column 804 for each of the plurality of words 802 and information about an embedding vector 806 obtained by converting each of the plurality of words 802 into a vector value. It is a database that The dictionary DB 800 may be, for example, a pronunciation column dictionary. In an embodiment, the dictionary DB 800 may be replaced with a look-up table including the pronunciation column 804 information and the embedding vector 806 for each of the plurality of words 802 . .

In an embodiment, the processor 1200 (refer to FIG. 2 ) of the server 1000 divides the replacement target text 810 into word units or morpheme units, and as a result of the division, a pronunciation column 820 from a corpus composed of word units and morpheme units. ) can be extracted. In an embodiment, the processor 1200 may extract the pronunciation column 820 from the replacement target text 810 using a rule-based defined model. Since the method of extracting the pronunciation column 820 from the replacement target text 810 is a known technique, a detailed description thereof will be omitted. For example, when the replacement target text 810 is 'Solomon', the processor 1200 divides 'Sulomone' into word units, so that a pronunciation string such as '[S_B OW_I L_I R_I OW_I M_I OW_I NN_E]' 820 can be extracted.

The server 1000 may acquire the alternative text 830 to replace the replacement target text 810 based on phonetic relevance of the pronunciation column. In an embodiment, the processor 1200 may search for the replacement target text 810 from the dictionary DB 800 , and determine the replacement text based on a similarity 808 of a pronunciation column as a result of the search. In an embodiment, the processor 1200 generates a list by extracting at least one word having a similarity 808 equal to or greater than a preset second threshold among the words searched in the dictionary DB 800 , and generates a list of at least one word included in the list. A word having the highest similarity among words may be determined as the alternative text 830 . The second threshold is a standard value for determining the similarity between the word searched in the dictionary DB 800 and the replacement target text 810 . In an embodiment, the second threshold may be set by a user input input through the client device 2000 . The UI for setting the second threshold will be described in detail with reference to FIG. 18 .

For example, when the second threshold is set to be relatively low, there is a high probability that words similar to the replacement target text 810 will be searched for in the dictionary DB 800 , so that a relatively large number of replacement texts 830 are generated. can be decided. Conversely, when the second threshold is set to be relatively high, the probability of searching for words similar to the replacement target text 810 in the dictionary DB 800 is low, so that a relatively small number of replacement texts 830 is can be decided.

In the embodiment shown in FIG. 8 , as a result of searching for 'Solomon' through the dictionary DB 800, the similarity with 'Solomon' was found to be 0.7822, and the similarity exceeds a preset second threshold, the processor 1200 may obtain 'Solomon' as an alternative text 830 for replacing 'Solomon'.

9 is a flowchart of an embodiment in which the server 1000 of the present disclosure obtains an alternative text from an alternative text. The steps of the method shown in FIG. 9 embody step S430 of FIG. 4 , and step S910 is performed after step S420 shown in FIG. 4 is performed. After step S930 of FIG. 9 is performed, step S440 shown in FIG. 4 is performed.

In step S910 , the server 1000 extracts a phoneme sequence related to the replacement target text. In an embodiment, the server 1000 may divide the replacement target text into word units or morpheme units, and extract a pronunciation sequence from a corpus composed of word units and morphemes as a result of the division. In an embodiment, the server 1000 may extract a pronunciation column from the replacement target text using a rule-based defined model, but is not limited thereto. The server 1000 may extract a pronunciation column from the replacement target text by using a known method.

In step S920 , the server 1000 searches for text having a pronunciation column similar to the extracted pronunciation column among a plurality of words included in the dictionary DB based on phonetic relevance of the pronunciation column.

In step S930, the server 1000 obtains an alternative text by using at least one text having a high similarity to the extracted pronunciation string among the searched texts. In an embodiment, the server 1000 may generate a list by searching for a replacement target text in the dictionary DB, and extracting at least one word having a similarity greater than or equal to a preset second threshold among words searched in the dictionary DB. The server 1000 may determine, as the replacement text, a word having the highest similarity to the replacement target text among at least one word included in the list.

10 is a diagram illustrating an embodiment in which the server 1000 of the present disclosure generates the replacement text 1050 from the replacement target text 1010 using a neural network model.

Referring to FIG. 10 , the server 1000 may convert the replacement target text 1010 into an embedding vector 1030 . In an embodiment, the processor 1200 (refer to FIG. 2 ) of the server 1000 may convert the replacement target text 1010 into an embedding vector 1030 using the word embedding model 1020 . The word embedding model 1020 is a model configured to digitize words constituting text into vector values. For example, the word embedding model 1020 may convert the replacement target text 1010 into an embedding vector 1030 using an embedding model such as Bag of Words or word2vec. However, the present invention is not limited thereto, and the processor 1200 may convert the replacement target text 1010 into an embedding vector 1030 using all known embedding models. In the embodiment shown in FIG. 10, the processor 1200 uses the word embedding model 1020 to convert the replacement target text 1010 called 'Solomon' to [0.00002645. 0.23621, 0.00499278, … ] can be converted into an N-dimensional embedding vector 1030 having a vector value of .

The server 1000 may generate the alternative text 1050 having a vector value similar to the transformed embedding vector 1030 using the neural network model. In an embodiment, the processor 1200 may generate text having a vector value similar to the embedding vector 1030 of the replacement target text 1010 by using the generative network 1040 . The generative model 1040 may include a generator model 1044 .

The generative model 1040 may be configured as a generative adversarial network (GAN), for example. When the generative model 1040 is configured as a generative adversarial neural network model, the generative model 1040 may include a discriminator model 1042 and a generator model 1044 . The discriminator model 1042 and the generator model 1044 may be trained adversarially to each other. The generator model 1044 is a fake that can fool the discriminator model 1042 based on the embedding vector 1030 of the replacement object text 1010 with the embedding vector 1030 input to the generative adversarial neural network model. ) is a model trained to generate text. The discriminator model 1042 is a model trained to discriminate fake text generated by the generator model 1044 as fake. The discriminator model 1042 and the generator model 1044 may be trained until they reach a balance point with each other. While the discriminator model 1042 and the generator model 1044 reach a balance point, the generative adversarial neural network model can be used to sufficiently simulate the data probability distribution of the true text, that is, the replacement target text 1010 . The loss function of can be appropriately modified.

In one embodiment, the probability that fake texts generated by the generator model 1044 are determined by the discriminator model 1042 to be non-fake texts that can replace the replacement target text 1010 will converge to 0.5. can In this case, the processor 1200 may determine the text generated by the generator model 1044 as the alternative text 1050 . In the embodiment illustrated in FIG. 10 , the processor 1200 may generate 'Solomon' as the alternative text 1050 for 'Solomon', which is the replacement target text 1010 .

Although it has been described in FIG. 10 that the replacement text 1050 similar to the embedding vector 1030 of the replacement target text 1010 is generated using the generation model 1040, the present disclosure is not limited thereto. In one embodiment, the server 1000 is a Convolutional Neural Network (CNN), Deep Neural Network (DNN), Recurrent Neural Network (RNN), Restricted Boltzmann Machine (RBM), Deep Belief Network (DBN), Bidirectional Recurrent Deep (BRDNN). The alternative text 1050 may be generated by using a neural network model of at least one of a Neural Network and Deep Q-Networks.

11 is a flowchart of an embodiment in which the server 1000 of the present disclosure generates an alternative text using a neural network model. The steps of the method shown in FIG. 11 embodied step S430 of FIG. 4 , and step S1110 is performed after step S420 shown in FIG. 4 is performed. After step S1130 of FIG. 11 is performed, step S440 shown in FIG. 4 is performed.

In step S1110, the server 1000 converts the replacement target text into an embedding vector. In an embodiment, the server 1000 may convert the replacement target text into an embedding vector by using a word embedding method, which is a method of digitizing one word constituting text into a vector value. For example, the server 1000 may convert the replacement target text into an embedding vector by using an embedding model such as Bag of Words or word2vec.

In step S1120, the server 1000 uses a neural network model to generate text having a vector value similar to the transformed embedding vector. Server 1000 is, for example, CNN (Convolutional Neural Network), DNN (Deep Neural Network), RNN (Recurrent Neural Network), GAN (Generative Adversarial Network), DCGAN (Deep Convolutional Generative Adversarial Network), RBM (Restricted Boltzmann) Machine), DBN (Deep Belief Network), BRDNN (Bidirectional Recurrent Deep Neural Network), or deep Q-Networks using at least one neural network model, a vector value similar to the embedding vector of the target text You can create text with

In an embodiment, the server 1000 may generate text having a vector value similar to an embedding vector of the replacement target text by using the generation model. The generative model may consist of, for example, a generative adversarial neural network model. In one embodiment, the server 1000 is configured to generate a discriminator model 1042, 1042, for fake text generated by a generator model 1044 (see FIG. 10) included in the generative adversarial neural network model based on the embedding vector of the replacement target text. 10) can be used to perform adversarial training to determine whether the text is fake. As a result of the training, when the probability that the fake texts generated by the generator model 1044 are not fake by the discriminator model 1042, but are determined as texts that can replace the replacement target text converges to 0.5, the server 1000 ) can print text.

In step S1130 , the server 1000 generates an alternative text by using the generated text.

12 is a diagram illustrating an embodiment in which the server 1000 of the present disclosure generates an alternative text 1230 from the replacement target text 1210 by using the ASR model 1310 and the TTS model 1330 .

Referring to FIG. 12 , the processor 1200 (refer to FIG. 2 ) of the server 1000 may convert the replacement target text 1210 into a wave signal 1220 using the TTS model 1330 . . The TTS model 1330 is a model configured to convert text into an acoustic signal. In an embodiment, the processor 1200 converts the replacement target text 1210 into an acoustic signal 1220 using the TTS model 1330 , and converts the converted acoustic signal 1220 into binary data streaming. It can be output in the form of a signal. In the embodiment shown in FIG. 12 , the processor 1200 uses the TTS model 1330 for 'Sulomone', which is the replacement target text 1210 among the input text 'Order sweets sulomone~', with the sound signal 1220 ), and the converted sound signal 1220 may be encoded as a binary data streaming signal and output.

In an embodiment, the processor 1200 may convert the replacement target text 1210 into a custom sound signal using a personalized TTS model. The personalized TTS model is a model that generates a sound signal from text by reflecting personalized features such as the speaker's age, gender, region, dialect, intonation, and pronunciation (eg, British English, American English). The personalized TTS model may be generated by recording a specific user's acoustic signal and training by inputting the recorded acoustic signal and text into the deep neural network model as training data. The personalized TTS model may be pre-stored in the memory 1300 (refer to FIG. 2 ). In an embodiment, the server 1000 may receive, from the client device 2000, a personalized characteristic value selected based on a user input for inputting information such as the usage region, age, gender, intonation, dialect, and pronunciation of the application user. have. The processor 1200 of the server 1000 may convert the replacement target text 1210 into an acoustic signal 1220 by reflecting the personalized characteristic value received from the client device 2000 using the personalized TTS model. have.

A specific embodiment in which the client device 2000 receives a personalized characteristic value selected based on a user input for inputting information such as an application user's usage region, age, gender, intonation, dialect, and pronunciation will be described in detail with reference to FIG. 19B . do it with

The server 1000 may convert the output sound signal 1220 into text using the ASR model 1310 . In the embodiment shown in FIG. 12 , the processor 1200 converts the sound signal 1220 outputted using the ASR model 1310 into text, and replaces 'Solomon', which is the replacement target text 1210 . 'Solomon' may be generated as text 1230 .

13 is a flowchart of an embodiment in which the server 1000 of the present disclosure generates an alternative text from an alternative text. The steps of the method shown in FIG. 13 embodied step S430 of FIG. 4 , and step S1310 is performed after step S420 shown in FIG. 4 is performed. After step S1340 of FIG. 13 is performed, step S440 shown in FIG. 4 is performed.

In step S1310, the server 1000 converts the input text into an acoustic signal using the TTS model. The TTS model is a model for converting text into a wave signal, and the server 1000 may convert the replacement target text included in the input text into a sound signal using the TTS model.

In an embodiment, the server 1000 may convert the input text into an acoustic signal using the personalized TTS model. The personalized TTS model is a model that generates a sound signal from text by reflecting personalized features such as the speaker's age, gender, region, dialect, intonation, and pronunciation (eg, British English, American English). In an embodiment, the server 1000 receives, from the client device 2000, a personalized characteristic value selected based on a user input for inputting information such as an application user's usage region, age, gender, intonation, dialect, and pronunciation, The replacement target text among the input texts may be converted into an acoustic signal based on the received personalized characteristic value.

In step S1320, the server 1000 outputs the converted sound signal. In an embodiment, the server 1000 may encode the sound signal into a binary data streaming signal composed of binary data including 0 and 1, and output the encoded binary data streaming signal.

In step S1330, the server 1000 converts the output sound signal into output text using the ASR model. The ASR model is a model that converts a voice input or voice signal into text.

In step S1340, the server 1000 generates an alternative text by using the converted output text. In an embodiment, the alternative text may be a text in which personalized characteristics such as the speaker's age, gender, region, dialect, intonation, and pronunciation (eg, British English or American English) are reflected.

14 is a flowchart illustrating a method in which the server 1000 of the present disclosure interprets an input text and at least one learning candidate text using a natural language understanding model, and generates information about an interpretation result. The steps of the method shown in FIG. 14 may be performed between steps S440 and S450 of FIG. 4 . Step S1410 of FIG. 14 may be performed after step S440 illustrated in FIG. 4 , and step S450 of FIG. 4 may be performed after step S1440 .

In step S1410, the server 1000 detects a domain by interpreting the input text and at least one learning candidate text using the natural language understanding model. In an embodiment, the server 1000 uses a domain identification model included in any one of the pre-trained first natural language understanding model 1320a (refer to FIG. 2) to the third natural language understanding model 1320c (refer to FIG. 2). By analyzing the input text and the at least one learning candidate text, respectively, it is possible to detect a domain that is category information to which the input text and the at least one learning candidate text belong or can be classified. In an embodiment, the server 1000 may identify one or more domains from the input text and the at least one learning candidate text. In an embodiment, the server 1000 may calculate the relation between the text and the domain as a numerical value by interpreting the input text and at least one learning candidate text using the domain identification model. In one embodiment, the server 1000 calculates a degree of relevance between each of one or a plurality of domains, the input text and at least one learning candidate text as a probability value, and selects a domain having a high probability value among the calculated probability values in which the input text is classified. domain can be determined.

The same domain may be detected from the input text and the at least one learning candidate text, but is not limited thereto. In an embodiment, the input text and the at least one learning candidate text may be classified into different domains, and each of the at least one learning candidate text may also be classified into different domains.

In step S1420, the server 1000 detects an intent by interpreting the input text and at least one learning candidate text using the natural language understanding model. The server 1000 may detect an intent from the input text and at least one learning candidate text by using the intent identification model of the pre-trained natural language understanding model. In an embodiment, the server 1000 uses an intent action identification model included in any one of the first natural language understanding model 1320a to the third natural language understanding model 1320c to input text and at least one learning candidate text. Intent action can be detected from In an embodiment, the server 1000 uses an intent object identification model included in any one of the first natural language understanding model 1320a to the third natural language understanding model 1320c to input text and at least one learning candidate text. You can detect an Intent object from .

The intent action refers to an action performed by the input text, for example, search, posting, play, purchase, order, and the like. The server 1000 may detect an intent action from the input text and at least one learning candidate text by performing syntactic analysis or semantic analysis using the intent action identification model. In one embodiment, the server 1000 parses each of the input text and at least one learning candidate text into a unit of a morpheme, a word, or a phrase using the intent action identification model, The meaning of the word or phrase extracted from the parsed text may be inferred by using the linguistic features (eg, grammatical elements) of the parsed morpheme, word, or phrase. The server 1000 may determine an intent action corresponding to the meaning of the inferred word or phrase by comparing the meaning of the inferred word or phrase with predefined intents provided from the natural language understanding model.

The intent object means an object related to the identified intent action. The intent object is an object that is the target of the detected intent action, and may mean, for example, a movie, a photo, sports, weather, or aviation. The server 1000 may detect an intent object related to an intent action from the input text and the at least one learning candidate text by interpreting each of the input text and the at least one learning candidate text using the intent object identification model. .

The same intent may be detected from the input text and at least one learning candidate text, but is not limited thereto. In an embodiment, different intents may be detected from the input text and the at least one training candidate text, and different intents may be detected from each of the at least one training candidate text.

In step S1430 , the server 1000 identifies a slot in the input text and at least one learning candidate text using the natural language understanding model, and performs slot tagging. In one embodiment, the server 1000 uses a slot tagging model included in any one of the pre-trained first natural language understanding model 1320a to third natural language understanding model 1320c to input text and at least one learning candidate. Slot tagging can be performed by identifying slots from text and associating the identified slots with domains, intent actions, and intent objects. The slot means variable information for obtaining detailed information related to a domain, an intent action, and an intent object from an input text or determining a detailed action. For example, the slot may include a named entity, a keyword, a place, a place name, a movie title, or a game term. According to an embodiment, a slot is information related to an intent action and an intent object, and a plurality of types of slots may correspond to one intent.

In an embodiment, the server 1000 may identify one or more slots from each of the input text and the at least one learning candidate text.

In steps S1410, S1420, and S1430, it has been described that the server 1000 detects a domain and an intent from the input text and performs slot tagging, but is not limited thereto. In an embodiment, the server 1000 may determine a domain and an intent corresponding to the input text based on a user input received through the client device 2000 , and perform slot tagging. In an embodiment, the server 1000 interprets the input text by using the plurality of pre-trained natural language understanding models 1320a to 1320c, so that at least one intent candidate related to the input text, and at least for slot tagging One slot may be detected, and the detected at least one intent candidate and the at least one slot may be transmitted to the client device 2000 . The client device 2000 displays a graphical user interface (GUI) related to at least one intent candidate and at least one slot candidate received from the server 1000 , and an intent selected based on a user input received through the GUI. and slot tagging information may be transmitted to the server 1000 . The server 1000 may determine an intent related to the input text based on the received intent and slot tagging information and perform slot tagging. A specific embodiment related thereto will be described in detail with reference to FIG. 18 .

In step S1440, the server 1000 provides information about the domain, intent, and slot as input data for learning the natural language understanding model. In an embodiment, the server 1000 may select a natural language understanding model related to the input text and at least one learning candidate text from among a plurality of pre-stored natural language understanding models based on the detected domain and intent.

15 is a flowchart according to an embodiment in which the server 1000 of the present disclosure determines text input as training data to a natural language understanding model based on a user input received by the client device 2000 . The steps of the method shown in FIG. 15 embodied step S450 of FIG. 4 , and step S1510 is performed after step S440 shown in FIG. 4 is performed.

In step S1510 , the server 1000 transmits the input text and at least one learning candidate text to the client device 2000 .

In operation S1520 , the server 1000 receives, from the client device 2000 , an identification value regarding at least one text selected by a user input among the input text and the at least one learning candidate text. In an embodiment, the client device 2000 receives the input text and the at least one learning candidate text from the server 1000 , and a list including the at least one learning candidate text and a user who selects at least one text from the list A graphical user interface for receiving input may be displayed. The client device 2000 may select at least one of the at least one learning candidate text based on a user input received through the graphical user interface, and transmit an identification value regarding the selected at least one learning candidate text to the server 1000 . have. A specific embodiment in which the client device 2000 receives a user input for selecting at least one of at least one learning candidate text will be described in detail with reference to FIG. 20 .

In step S1530, the server 1000 selects at least one text based on the received identification value. The server 1000 may identify at least one of the at least one learning candidate text included in the list based on the identification value received from the client device 2000 and select the identified at least one learning candidate text.

In step S1540, the server 1000 learns a natural language understanding model by using the selected at least one text as training data. The server 1000 may perform training by inputting the input text and at least one text selected in step S1530 as training data to the natural language understanding model. In one embodiment, the server 1000 is provided with information about a domain, an intent, and a slot identified from the input text and the learning candidate text in step S1440 (see FIG. 14 ), and the provided domain, intent, and slot Learning may be performed using the information and at least one text selected in step S1530 as input data. However, the present invention is not limited thereto, and the server 1000 determines the domain detected in step S710 (see FIG. 7 ), the intent detected in step S720 (see FIG. 7 ), and the slot performed in step S730 (see FIG. 7 ). Learning may be performed by receiving tagging information and using domain, intent, and slot tagging information and at least one text selected in step S1530 as input data.

A specific example of learning the natural language understanding model will be described in detail with reference to FIG. 16 .

16 is a flowchart of an embodiment in which the server 1000 of the present disclosure learns a natural language understanding model based on the number of input texts and at least one learning candidate text. The steps of the method shown in FIG. 16 embody step S450 shown in FIG. 4 . Also, steps S1610 to S1640 are performed between steps S1530 and S1540 shown in FIG. 15 .

In step S1610, the server 1000 counts the number of the input text and the at least one learning candidate text, and as a result of the count, the number of the input text and the at least one learning candidate text exceeds the third threshold value α. judge The third threshold value α is the sum of the number of input texts and at least one learning candidate text, and may be set by a user input. The third threshold value α may be set by a user input received through a UI displayed on the client device 2000 . A UI for setting the third threshold value α will be described in detail with reference to FIG. 18 .

The preset third threshold value α may be, for example, 10, but is not limited thereto.

When the number of input texts and at least one learning candidate text as a result of the count in step S1610 exceeds a third preset threshold α (step S1620), the server 1000 determines the detected domain, intent, and slot information is used to train a new natural language understanding model. In an embodiment, the server 1000 receives the input text and at least one learning candidate text corresponding to each of the label values and the label values for domains, intents, and slots detected from the input text and the at least one learning candidate text. A new natural language understanding model can be trained by using it as training data. The natural language understanding model may be trained through a rule-based system using the input text and the learning candidate text and the label values for the domain, intent, and slot corresponding to each as input data, but is not limited thereto. does not Natural language understanding models are, for example, neural network-based systems (eg, convolutional neural networks (CNNs), recurrent neural networks (RNNs), support vector machines (SVMs), linear regression). (linear regression), logistic regression (logistic regression), naive Bayes classification (Naive Bayes), random forest (random forest), decision tree, or k-nearest neighbor algorithm, or a combination of the foregoing or other It can also be trained using artificial intelligence models.

For example, the input text "Order for delivery by 30 gil with 3 pepperoni pizzas" and multiple learning candidate texts "Order for delivery by 30 gil with 3 pepperoni pizzas", "3 slices of pepperoni pizza If the number of texts including "Order for delivery by 30 gil in Unjuro" and "Order for delivery in 3 slices of pepperoni pizza in Unjuro 30 gil" exceeds a preset third threshold (α), the server ( 1000) may train a new natural language understanding model with a domain related to 'order' or 'delivery'. The server 1000 inputs 'food order' as the label value of the intent related to the input text and the plurality of learning candidate texts, and inputs 'pepperoni', 'pepperoni', 'paperoni', and 'peporoni' By inputting the text and the plurality of learning candidate texts as label values of slots, a new natural language understanding model can be learned.

When the number of input texts and at least one learning candidate text as a result of the count in step S1610 is less than or equal to a third threshold α (step S1630), the server 1000 performs a pre-trained NLU model ), a natural language understanding model corresponding to the detected domain and intent information is selected. In an embodiment, the server 1000 is configured to provide a plurality of pre-learned natural language understanding models 1320a to 1320c stored in the memory 1300 (refer to FIG. 2 ) based on the domain detected from the input text and at least one learning candidate text. , see FIG. 2), any one of the natural language understanding models may be selected. In an embodiment, each of the plurality of pre-trained natural language understanding models 1320a to 1320c is a model trained specifically for a specific domain, and the server 1000 is an input of the plurality of pre-trained natural language understanding models 1320a to 1320c. A natural language understanding model trained with a domain similar to a domain detected from the text and at least one learning candidate text may be selected. A similarity between a domain detected from the input text and at least one learning candidate text and a domain from which the plurality of pre-trained natural language understanding models 1320a to 1320c have been learned may be calculated as a probability value.

For example, the first natural language understanding model 1320a is a model trained in the movie domain, the second natural language understanding model 1320b is a model trained in the game domain, and the third natural language understanding model 1320c is home appliance control. It may be a model trained with a domain. In an embodiment, the server 1000 interprets the input text and the at least one learning candidate text using the pre-learned natural language understanding model, so that an intent action and an action related to a movie from the input text and the at least one learning candidate text are performed. It is possible to detect tent objects, identify slots such as movie titles, movie actors names, and the like, and perform slot tagging. In this case, the server 1000 calculates a first probability value associated with the movie domain, a second probability value associated with the game domain, and a third probability value associated with the home appliance control domain for the input text and the at least one learning candidate text, respectively, , a first probability value among the calculated probability values may be determined as a maximum value. The server 1000 may select the first natural language understanding model 1320a from among the first natural language understanding model 1320a to the third natural language understanding model 1320c based on the first probability value.

In step S1640 , the server 1000 updates the natural language understanding model by inputting the input text and at least one learning candidate text as training data to the selected natural language understanding model to perform learning. In an embodiment, the server 1000 may input text and at least one text, domain, and intent to the first natural language understanding model 1320a selected from the first natural language understanding model 1320a to the third natural language understanding model 1320c. By inputting label values for , , and slots, and performing learning using the input data, the first natural language understanding model may be updated. In an embodiment, a domain identification model 1322a (see FIG. 2 ), an intent action identification model 1323a (see FIG. 2 ), and an intent object identification model 1324a (see FIG. 2 ) included in the first natural language understanding model 1320a ), and the four models including the slot tagging model 1325a (refer to FIG. 2 ) may be updated.

In one embodiment, the server 1000 configures the intent action identification model 1323a to support any new intent actions not yet included in the intent action identification model 1323a of the first natural language understanding model 1320a. ), the intent action identification model 1323a may be updated by inputting an input text and at least one learning candidate text, and inputting a label value of a new intent action corresponding thereto for training. In one embodiment, the intent actions associated with the new domain may already be supported by the intent action identification model 1323a before being updated.

In one embodiment, the server 1000 configures the Intent object identification model 1324a to support any new Intent objects not yet included in the Intent object identification model 1324a of the first natural language understanding model 1320a. ), the intent object identification model 1324a may be updated by inputting an input text and at least one learning candidate text, and inputting a label value of a new intent object corresponding thereto for training. In one embodiment, the Intent object associated with the new domain may already be supported by the Intent object identification model 1324a before being updated.

In an embodiment, the server 1000 may update the slot tagging model 1325a by inputting the input text and the slot identified from the at least one learning candidate text and a label value related to the slot for training.

For example, if the input text is "Search for the movie Avengers Endgame release date", the plurality of learning candidate texts are "Find the movie Avengers Endgame release date", "Search the movie Avengers Endgame release date", "The Movie Avengers" In the case of "Search for the release date of the juice endgame", the server 1000 detects a 'movie' domain from the input text and a plurality of learning candidate texts, and the first natural language understanding model 1320a to the third natural language understanding model 1320c. A probability value indicating a degree of association with a domain may be calculated, and the first natural language understanding model 1320a may be selected. In an embodiment, the server 1000 may include a label value for 'search', which is a new intent action that is not yet included in the intent action identification model 1323a of the first natural language understanding model 1320a, and an input By training using the text and the plurality of learning candidate texts, the intent action identification model 1323a may be updated. In one embodiment, the server 1000 includes a label value for 'movie release date', a new Intent object that is not yet included in the intent object identification model 1324a of the first natural language understanding model 1320a, and the input text and By training using a plurality of learning candidate texts, the intent object identification model 1324a may be updated. In one embodiment, the server 1000 is configured to provide slot elements that are not yet included in the slot tagging model 1325a of the first natural language understanding model 1320a 'Avengers Endgame', 'Avengers Endgame', and Avengers End The slot tagging model 1325a may be updated by training using the label values for 'Game' and 'Avenjuice Endgame', input text, and a plurality of learning candidate texts.

17 is a flowchart illustrating an operation of the client device 2000 of the present disclosure.

In step S1710 , the client device 2000 displays a first graphical user interface (GUI) for receiving a user input for inputting input text. The processor 2200 (refer to FIG. 3) of the client device 2000 displays the first GUI on the display unit 2510 (refer to FIG. 3), and inputs input text from the user through the user input unit 2410 (refer to FIG. 3). can receive

In step S1720 , the client device 2000 transmits the input text received through the first GUI to the server 1000 . In an embodiment, the processor 2200 may control the communication interface 2100 (refer to FIG. 3 ) to transmit the input text received from the user to the server 1000 .

In operation S1730, the client device 2000 displays a second GUI for receiving a user input for selecting at least one of the at least one replacement target text identified from the input text. In an embodiment, the at least one replacement target text is identified by the server 1000 , and the client device 2000 uses the communication interface 2100 to transmit an identification value related to the at least one replacement target text to the server 1000 . ) can be received from In an embodiment, the processor 2200 may display a list including at least one replacement target text on the display unit 2510 . In an embodiment, the processor 2200 may display a second GUI for receiving a user input for selecting at least one of at least one replacement target text included in the list on the display unit 2510 . The processor 2200 may receive a user input for selecting at least one of at least one replacement target text included in the list through the user input unit 2410 .

However, the present invention is not limited thereto, and the client device 2000 may identify the replacement target text from the input text by itself. Since the method for the processor 2200 of the client device 2000 to identify the replacement target text from the input text is the same as the method performed by the processor 1200 (refer to FIG. 2 ) of the server 1000 , a redundant description will be omitted.

In step S1740 , the client device 2000 replaces at least one learning candidate text generated by replacing at least one replacement target text selected through the second GUI among the input texts with text expected to be uttered by the server 1000 . receive from In an embodiment, the at least one learning candidate text is generated by the server 1000 , and the processor 2200 uses the communication interface 2100 to generate an identification value related to the at least one learning candidate text to the server 1000 . can be received from

However, the present invention is not limited thereto, and the client device 2000 may generate at least one learning candidate text by itself replacing the replacement target text with the replacement text expected to be uttered by the user. A specific method of generating at least one learning candidate text by the processor 2200 of the client device 2000 replacing the replacement target text with the replacement text is the same as the method performed by the processor 1200 of the server 1000, A description will be omitted.

In operation S1750, the client device 2000 displays a third GUI for receiving a user input for selecting at least one of the at least one learning candidate text. In an embodiment, the processor 2200 may display a list including at least one learning candidate text on the display 2510 . In an embodiment, the processor 2200 may display a third GUI for receiving a user input for selecting at least one of at least one learning candidate text included in the list on the display unit 2510 .

The processor 2200 may receive a user input for selecting at least one of at least one learning candidate text included in the list through the user input unit 2410 .

18 is a diagram illustrating an example of a GUI displayed by the client device 2000 of the present disclosure.

Referring to FIG. 18 , the client device 2000 includes a first GUI 2511 , a second GUI 2512 , a third GUI 2513 - 1 , a fourth GUI 2514 , and a second GUI on the display unit 2510 . A fifth GUI 2515 and a sixth GUI 2516 may be displayed. The first GUI 2511 , the second GUI 2512 , the third GUI 2513 - 1 , the fourth GUI 2514 , the fifth GUI 2515 , and the sixth GUI 2516 receive user input. It is a graphical interface where the user can be a developer who wants to learn a language model for inclusion in an application to be developed.

The first GUI 2511 is a graphical user interface for receiving a user input for entering input text. The processor 2200 (refer to FIG. 3) of the client device 2000 displays the first GUI 2511 on the display unit 2510, and receives input text from the developer through the user input unit 2410 (refer to FIG. 3). can In the embodiment shown in FIG. 18 , the processor 2200 may receive input text from the developer, "Deliver 3 pepperoni pizzas to Eonjuro 30 Gil".

The second GUI 2512 is a graphical user interface that displays information about an intent detected from the input text. In one embodiment, the intent is detected from the input text by the server 1000 , and the processor 2200 of the client device 2000 transmits information about the detected intent through the communication interface 2100 (refer to FIG. 3 ). can receive The processor 2200 may display the second GUI 2512 indicating information about the intent received from the server 1000 on the display unit 2510 . However, the present invention is not limited thereto, and the processor 2200 may detect an intent from the input text by interpreting the input text using the pre-learned natural language understanding model. An intent may include an intent action and an intent object.

The second GUI 2512 may include at least one intent candidate, a confidence score for each of the at least one intent candidate, and a selection option GUI 2512a. The 'reliability score' is a numerical value indicating whether the intent is correctly detected with respect to the input text. In an embodiment, the confidence score may be a number in a range between 0 and 1, and a higher confidence score may mean that the intent is more related to the utterance intention of the input text. The selection option GUI 2512a provides the developer with at least one intent candidate detected from the input text, and the processor 2200 selects any one of the at least one intent candidate from the developer through the selection option GUI 2512a. Or it may receive an input for determining. 18 , the second GUI 2512 displays 'food order' and 'food search' as intent candidates detected from the input text, and the confidence score for the food order is 0.9, and the food search A confidence score of 0.1 can be displayed. The client device 2000 may receive an input for determining a food order among two intent candidates as an intent through the selection option GUI 2512a.

Although not shown in FIG. 18 , the second GUI 2512 may further include a graphical user interface for adding a new intent.

The third GUI 2513 - 1 may include at least one slot detected from the input text, a confidence score for the at least one slot, and selection option GUIs 2513a , 2513b , and 2513c . The confidence score is a numerical value indicating the accuracy with respect to slot tagging of the slots detected from the input text. The selection options GUIs 2513a, 2513b, and 2513c provide the developer with at least one slot detected from the input text, and the processor 2200 provides each of the at least one slot from the developer via the selection options GUI 2513a, 2513b, 2513c. may receive an input for performing tagging. 18, the third GUI 2513-1 displays 'Pepperoni', '3', and 'Eonjuro 30 Gil' as slots detected from the input text, and a slot related to 'Pepperoni' Toppings and person names may be displayed as a tagging result, a number may be displayed as a slot tagging result for '3', and an address and traffic may be displayed as a slot tagging result for 'Eonju-ro 30-gil'. In the third GUI 2513 - 1 , a numerical value for slot tagging of 'pepperoni' may be 0.8 for a topping and 0.2 for a person's name. In the third GUI 2513 - 1 , the number of '3' for slot tagging may be 1.0. In the third GUI 2513-1, numerical values related to slot tagging of 'Eonju-ro 30-gil' may have an address of 0.9 and a traffic of 0.1. The processor 2200 may receive an input for selecting one of a topping and a person's name displayed as a slot tagging result regarding 'pepperoni' from the developer through the selection option GUI 2513a. The processor 2200 may receive an input for selecting the number to be displayed as a slot tagging result regarding '3' from the developer through the selection option GUI 2513b. The client device 2000 may receive, from the developer, an input for selecting one of an address and traffic displayed as a slot tagging result for 'Eonju-ro 30-gil' through the selection option GUI 2513c.

Although not shown in FIG. 18 , the third GUI 2513 - 1 may further include a graphic user interface for adding a new slot.

The fourth GUI 2514 may include a list of replacement target texts and a selection option GUI 2514a for selecting each of a plurality of replacement target texts included in the list. The replacement target text displayed through the fourth GUI 2514 may be a slot identified from the input text by the server 1000 , but is not limited thereto. In one embodiment, the replacement target text may be a slot identified from the input text by the processor 2200 of the client device 2000 . In another embodiment, the processor 2200 receives an input from the developer for selecting a word or phrase included in the input text through the user input unit 2410 (refer to FIG. 3 ), and selects the selected word or phrase based on the received input. It can be determined by the text to be replaced For example, a developer may select 'pepperoni' included in the input text through a touch and drag input or a click and drag input through a mouse.

In the embodiment shown in FIG. 18 , the processor 2200 performs the replacement target text for each of 'pepperoni', '3', and 'Eonjuro 30-gil' included in the replacement target text list through the selection option GUI 2514a. You can receive input from the developer to select .

The fifth GUI 2515 may include a selection option GUI 2515a that receives an input for selecting at least one of a list listing application characteristics and an application characteristic included in the list. The characteristics of the application may indicate a development purpose for which a developer intends to develop an application including the learned language model. The purpose of the development of the application is, for example, at least among the age, gender, region, language, accent, and dialect of the application user, such as the development of the application considering the local dialect, the development of the application considering the foreigner, and the development of the application that needs to be considered for kids. It may be determined based on context information including one. The processor 2200 may receive an input for selecting any one of the development purposes of the application from the developer through the selection option GUI 2515a.

The client device 2000 may determine the replacement target text and the development purpose of the application based on the input received from the developer, and generate the replacement text in consideration of the determined replacement target text of the application and context information according to the development purpose. In an embodiment, the processor 2200 of the client device 2000 may determine the replacement target text based on the development purpose of the application selected based on the user input. The processor 2200 uses a pre-trained deep neural network model using the application development purpose and the replacement target text as training data, or a rule-based model defining a pairing relationship between the application development purpose and the replacement target text. -based model), it is possible to determine an alternative target text according to an application development purpose selected through a user input. For example, when an app that requires consideration for foreigners is selected as the purpose of application development through the fifth GUI 2515, the processor 2200 uses a pre-trained deep neural network model or a rule-based model to be a word included in the replacement target text list Among them, "pepperoni", which is likely to be pronounced relatively inaccurately, may be determined as the replacement target text.

The client device 2000 may generate the replacement text in consideration of the determined replacement target text and context information according to the purpose of application development. For example, when the replacement target text is 'pepperoni' and the development purpose of the application is determined to be 'application that needs consideration for foreigners', the processor 2200 considers context information such as the region and language of the application user to be 'pepperoni' can be generated as an alternative text with texts frequently pronounced by foreigners, such as 'peparoni' and 'peporoni'.

The sixth GUI 2516 is a user interface for receiving a user input for setting a threshold. In one embodiment, the sixth GUI 2516 includes a first threshold UI 2516a for receiving a user input for setting a first threshold, a second threshold UI 2516b for receiving a user input for setting a second threshold; and a third threshold UI 2516c for receiving a user input for setting a third threshold.

The first threshold UI 2516a searches the dictionary DB 1370 (refer to FIG. 2 ) for each of a plurality of words included in the input text input through the first GUI 2511 , and as a result, a reference value for determining the replacement target text This is a user interface that receives a user input for setting the threshold of the frequency of use. The first threshold UI 2516a is, for example, a bar consisting of a low level, a medium level, and a high level, and a UI of a cursor for adjusting the threshold. may include The user may set or adjust the threshold higher or lower, for example by moving the cursor. However, the present invention is not limited thereto, and the first threshold UI 2516a may be configured as a UI for directly inputting a numerical value of the frequency of use.

For example, if the first threshold is set to be relatively low through the first threshold UI 2516a, as a result of the search of the dictionary DB 1370, fewer words with a frequency of use lower than the first threshold will be searched, so the replacement target text may be relatively small. In the embodiment shown in FIG. 18 , when the first threshold is set low, only 'pepperoni' among the input texts may be determined as the replacement target text.

As a converse example, if the first threshold is set to be relatively high through the first threshold UI 2516a, even if the frequency of use of the search result of the dictionary DB 1370 is lower than the first threshold, the probability of being determined as the replacement target text is high. , a relatively large number of replacement target texts may be determined. In the embodiment shown in FIG. 18 , when the first threshold is set to be relatively high, all of 'pepperoni', '3', and 'Eonjuro 30gil' among the input texts may be determined as the replacement target text.

The second threshold UI 2516b is a user interface that receives a user input for setting a threshold of similarity, which is a criterion for determining the replacement text, as a result of searching the dictionary DB 1370 for the replacement target text identified from the input text. The second threshold is a standard value for determining the degree of similarity between the word searched in the dictionary DB 1370 and the text to be replaced. In one embodiment, the second threshold UI 2516b, like the first threshold UI 2516a, consists of a bar consisting of a low step, a medium step, and a high step, and a UI of a cursor for adjusting the threshold. may include. The user may set or adjust the second threshold higher or lower, for example by moving the cursor.

For example, when the second threshold is set to be relatively low through the second threshold UI 2516b, there is a high probability that words similar to the replacement target text will be searched for in the dictionary DB 1370, so a relatively large number of Alternative text may be determined. Conversely, when the second threshold is set to be relatively high, the probability of searching for words similar to the replacement target text in the dictionary DB 1370 is low, so that a relatively small number of replacement texts may be determined.

The third threshold UI 2516c is a user interface for receiving a user input for setting a reference value of the number of texts for which training is performed using a natural language understanding model among input texts and at least one training candidate text. The third threshold is the sum of the input text and the number of at least one learning candidate text, and may be set by a user input received through the third threshold UI 2516c. The third threshold UI 2516c, like the first threshold UI 2516a and the second threshold UI 2516b, includes a bar consisting of a low step, a medium step, and a high step, and a cursor for adjusting the threshold. ) of the UI. The user may set or adjust the third threshold higher or lower, for example by moving the cursor. However, the present invention is not limited thereto, and the third threshold UI 2516c may include a UI for directly inputting the number of learning candidate texts to be used as input data for learning the natural language understanding model.

19A is a diagram illustrating an example of a GUI displayed by the client device 2000 of the present disclosure.

Referring to FIG. 19A , the client device 2000 may display the first GUI 2511 and the slot tagging GUI 2513 - 2 on the display unit 2510 . The client device 2000 may receive input text from a developer through the first GUI 2511 . The client device 2000 may receive an input for tagging a selected slot or detected from the input text through the slot tagging GUI 2513 - 2 .

In one embodiment, the slot tagging GUI 2513-2 lists the slot candidates that can be detected from the input text or can be tagged with respect to the selected slot, and can receive input from the developer to select any one of the listed slot candidates. have. In the embodiment shown in FIG. 19A , for 'pepperoni', which is detected from the input text "deliver 3 slices of pepperoni pizza to 30 gil," or the selected slot element, the slot tagging GUI 2513-2 is the pizza topping. (PizzaTopping), a location (Location), and a number (Number) may be displayed as candidates for slots to be tagged. The processor 2200 of the client device 2000 may receive, from a developer, an input for selecting a slot to be tagged with 'pepperoni' among pizza toppings, locations, and numbers through the slot tagging GUI 2513 - 2 .

19B is a diagram illustrating an example of a GUI displayed by the client device 2000 of the present disclosure.

Referring to FIG. 19B , the client device 2000 may display the first GUI 2511 and the application characteristic selection GUI 2517 on the display unit 2510 . The processor 2200 may receive input text from a developer through the first GUI 2511 . The processor 2200 may display characteristic information of at least one application through the application characteristic selection GUI 2517 and receive an input for selecting at least one of characteristics of the at least one application.

The application characteristic selection GUI 2517 may be a graphical user interface configured to list context information of a user using an application using the learned language model and receive an input for selecting at least one of the listed context information. The context information may include, for example, at least one of an application user's age, gender, region, language, accent, and dialect. In the embodiment shown in FIG. 19B , the application characteristic selection GUI 2517 displays a graphical user interface that selects the application user's region of use, the application user's age, and the application user's gender, and the processor 2200 provides the application characteristic An input for selecting a usage region, age, and gender of an application user may be received from a developer through the selection GUI 2517 .

In an embodiment, the processor 2200 of the client device 2000 obtains context information of the application user based on an input received through the application characteristic selection GUI 2517, and selects an alternative target text based on the context information. You can create alt text to be replaced. For example, when the region of the application user is a foreigner, the alternative text may be generated using a personalized TTS model and an ASR model specialized for pronunciation for each country. Similarly, alternative texts may be generated by using the personalized TTS model and the ASR model specialized for the age and gender of the application user. A method of generating the alternative text using the personalized TTS model and the ASR model is the same as the method described with reference to FIGS. 12 and 13 , and thus a redundant description is omitted.

In an embodiment, the processor 2200 of the client device 2000 may transmit the context information of the application user selected through the application characteristic selection GUI 2517 to the server 1000 . The server 1000 selects a personalized TTS model based on the context information, converts the replacement target text into a sound signal using the selected personalized TTS model, outputs a sound signal, and converts the output sound signal back to the ASR model (1310, see FIG. 2) may be used to convert the text into text, thereby generating an alternative text. The server 1000 may transmit the generated alternative text to the client device 2000 . In another embodiment, the server 1000 converts the replacement target text into an acoustic signal using a general TTS model 1330 (see FIG. 2 ), and converts the TTS model 1330 using VTLP (Vocal Tract Length Perturbation) technology. Alternative text may be generated by converting the sound signal output through the text into alternative text reflecting context information. For example, using VTLP technology, a frequency is converted to reflect the voice of a child, adult, or grandmother with respect to the output sound signal of the TTS model 1330, and the frequency-converted sound signal is converted into the ASR model 1310. ) through ASR, an alternative text reflecting context information including at least one of age, gender, region, language used, intonation, and dialect may be generated.

However, the present invention is not limited thereto, and the client device 2000 may generate an alternative text reflecting context information by using its own personalized TTS model.

In one embodiment, the application characteristic selection GUI 2517 may be displayed in the form of a pop-up window.

20 is a diagram illustrating an example of a GUI displayed by the client device 2000 of the present disclosure.

Referring to FIG. 20 , the client device 2000 may display a first GUI 2511 , a sixth GUI 2516 , and a new sentence list GUI 2518 on the display unit 2510 . The client device 2000 may receive input text from a developer through the first GUI 2511 . The client device 2000 may display a sixth GUI 2516 for receiving a user input for setting a threshold. The client device 2000 may display a list including at least one learning candidate text through the new sentence list GUI 2518 .

The sixth GUI 2516 includes a first threshold UI 2516a that receives a user input for setting a first threshold that is a reference value of the frequency of use for determining the replacement target text, and a second threshold UI 2516a that receives a reference value of similarity for obtaining the replacement text. A second threshold UI 2516b for receiving a user input for setting a threshold, and a third for receiving a user input for setting a third threshold that is a reference value of the number of learning candidate texts to be used as input data for learning a natural language understanding model At least one of the threshold UI 2516c may be included. A description of each of the first threshold UI 2516a , the second threshold UI 2516b , and the third threshold UI 2516c is the same as the UIs shown in FIG. 18 , and thus a redundant description will be omitted. For example, in the case where the sixth GUI 2516 is a UI for setting a first threshold regarding the frequency of use of words in the dictionary DB 1370 for determining the replacement target text, among the input texts according to the setting of the first threshold Only 'Pepperoni' may be determined as the replacement target text, or 'Pepperoni', '3rd edition', and 'Eonjuro 30gil' may all be determined as the replacement target text.

In one embodiment, the new sentence list GUI 2518 may include a list 2518a and a selection options GUI 2518b. The list 2518a is a list including at least one learning candidate text. In an embodiment, the processor 2200 of the client device 2000 receives at least one learning candidate text generated by the server 1000 from the server 1000, and includes the received at least one learning candidate text. List 2518a may be displayed. However, the present invention is not limited thereto, and the processor 2200 may generate at least one learning candidate text by identifying the replacement target text from the input text and replacing the identified replacement target text with the replacement text. The processor 2200 of the client device 2000 may display a list 2518a including at least one learning candidate text generated by itself on the display unit 2510 .

The selection options GUI 2518b may be a graphical user interface for receiving an input for selecting at least one of the at least one learning candidate text. The processor 2200 may receive an input for selecting at least one of the at least one learning candidate text from the developer through the selection option GUI 2518b.

In the embodiment shown in FIG. 20 , the client device 2000 via the new sentence list GUI 2518 "Deliver 3 slices of pepperoni pizza to 30 gil", "Deliver 3 slices of pepperoni pizza to 30 gil" Display a list 2518a including three learning candidate texts, such as ", and "Deliver 3 pieces of pepperoni pizza to 30 gil in Unjuro", and display the first learning candidate text "Peppa" through the selection option GUI 2518b. An input may be received from the developer to select "Deliver 3 slices of roni pizza by 30 gil" and the third learning candidate text "deliver 3 slices of pepperoni pizza by 30 gil".

21 is a diagram illustrating an example of a GUI displayed by the client device 2000 of the present disclosure.

Referring to FIG. 21 , the client device 2000 may display a first GUI 2511 , a sixth GUI 2516 , and a new sentence list GUI 2519 on the display unit 2510 . The processor 2200 of the client device 2000 may receive input text from a developer through the first GUI 2511 . The processor 2200 may display a sixth GUI 2516 for receiving a user input for setting a threshold on the display unit 2510 , and may receive a threshold input from a developer through the sixth GUI 2516 . The processor 2200 may display a list including at least one learning candidate text through the new sentence list GUI 2519 .

The sixth GUI 2516 includes a first threshold UI 2516a that receives a user input for setting a first threshold that is a reference value of the frequency of use for determining the replacement target text, and a second threshold UI 2516a that receives a reference value of similarity for obtaining the replacement text. A second threshold UI 2516b for receiving a user input for setting a threshold, and a third for receiving a user input for setting a third threshold that is a reference value of the number of learning candidate texts to be used as input data for learning a natural language understanding model At least one of the threshold UI 2516c may be included. A description of each of the first threshold UI 2516a , the second threshold UI 2516b , and the third threshold UI 2516c is the same as the UIs shown in FIG. 18 , and thus a redundant description will be omitted.

In one embodiment, the new sentence list GUI 2519 may include an alternative text GUI 2519a, a context GUI 2519b, and a voice output GUI 2519c. The alt text GUI 2519a is a graphical user interface that displays an alt text that is detected from the input text or replaces the selected substitute target text. The alternative text GUI 2519a may display the alternative text of the learning candidate text to be distinguished from other words or phrases. For example, the alt text GUI 2519a may display the alt text in a different color from other words or phrases, display the alt text in bold, have a different font, or highlight an area surrounding the alt text. It can be processed to be visually distinct from other words or phrases in the learning candidate text. In the embodiment shown in FIG. 21 , alternative texts 'Dungkin Donatsu', 'Dunking Donuts', and 'Dumpling Donuts' may be displayed with yellow highlighting, unlike other words.

The context GUI 2519b may display context information that is considered for generating the alt text. The context information may include, for example, at least one of an application user's age, gender, region, language, accent, and dialect. In the embodiment shown in FIG. 21 , the context GUI 2519b may indicate information about the region of the application user considered for generating the alternative text. For example, since 'Deungkin Donatsu' in the alternative text is a Busan regional dialect, the context information is the Busan region that is used by the application user, and the context GUI 2519b may display information 'Busan region'.

The voice output GUI 2519c is a graphical user interface configured to convert an alternative text into an acoustic signal using the TTS model, and output the converted acoustic signal. In an embodiment, the client device 2000 may select a personalized TTS model based on context information, and convert the alternative text into an acoustic signal using the selected personalized TTS model. For example, if the context information is a Busan area or a Busan area dialect, the client device 2000 converts 'Dungkin Donatsu' into an acoustic signal using a personalized TTS model that has been pre-recorded and trained in the Busan area dialect, The converted sound signal can be output.

22 is a block diagram illustrating a configuration of a language understanding service 3000 according to an embodiment of the present disclosure.

Referring to FIG. 22 , the language understanding service 3000 may include an ASR model 3100 , a natural language understanding model 3200 , a dialog manager 3300 , and a response generator model 3400 . The language understanding service 3000 shown in FIG. 22 may be pre-stored in the memory 1300 (refer to FIG. 2 ) of the server 1000 shown in FIG. 2 , but is not limited thereto. All or some components of the language understanding service 3000 may be pre-stored in the memory 2300 (refer to FIG. 3 ) of the client device 2000 .

Application 3500 may be configured to interact with language understanding service 3000 . In one embodiment, the application 3500 may include a Natural User Interface (NUI) for interacting with the language understanding service 3000 . Combinations of natural language dialogs and other non-verbal modalities expressing intent through application 3500 , such as gestures, touch, gaze, images, or video, may be used in interaction with language understanding service 3000 . . In one embodiment, the language understanding service 3000 may be configured to receive a voice input, such as a user's utterance, through the application 3500 , and to understand and respond to the received voice input, thereby communicating with the user. can

The ASR model 3100 is a model for converting a user's voice input received through the application 3500 into text. The ASR model 3100 may provide the text converted from the voice input to the natural language understanding model 3200 .

Although the configuration for providing the text to the natural language understanding model 3200 in FIG. 22 is illustrated as the ASR model 3100, the present invention is not limited thereto. In another embodiment, the language understanding service 3000 may include an input unit that receives a user input other than the voice input and provides the received user input to the natural language understanding model 3200 . The input unit may receive, for example, at least one of a user's touch input, a gesture input, and a text input received from an application, and provide text information corresponding to the received input to the natural language understanding model 3200 .

The natural language understanding model 3200 is a model configured to interpret or analyze the text obtained from the ASR model 3100 . The natural language understanding model 3200 is an artificial intelligence model trained to tag text, perform semantic analysis, detect domains and intents from text, and identify information about slots. can be The natural language understanding model 3200 may be a model learned through each of the embodiments of FIGS. 2 to 16 of the present disclosure. In an embodiment, the natural language understanding model 3200 identifies at least one replacement target text included in the input text, and generates the replacement text that is expected to be uttered by the user with respect to the replacement target text and has a high phonological similarity. and generating at least one learning candidate text by replacing the replacement target text with the generated replacement text, and performing learning using the input text and the at least one learning candidate text, it may be a generated or updated model. .

The natural language understanding model 3200 may provide information about a domain, an intent, and a slot detected from the text to the dialog manager 3300 .

Although not shown in the drawings, the language understanding service 3000 may further include a text normalizer module. The text normalization module may include a rule table that receives the replacement text in the training candidate text and outputs the replacement target text that is the original text of the replacement text. For example, the input text is "Search for new products from Dunkin' Donuts", the replacement target text identified from the input text is 'Dunkin Donuts', and the alt text is 'Dungkin Donuts', 'Dunkin Donuts', 'Dungkin Donuts'. ', the text normalization module defines the replacement text 'Dunkin Donuts', 'Dunking Donuts', and 'Dunkin Donuts' as the input text as the input text as shown in Table 2 below, and the replacement target text 'Dunkin Donuts' as the output text. It may include a rule table to

Input Output Dungkin Dnath Dunkin Donuts Dunking Donuts Dunkin Donuts chunky donuts Dunkin Donuts

In one embodiment, the text normalization module may be disposed between the natural language understanding model 3200 and the dialog manager 3300 . The natural language understanding model 3200 provides slot information and replacement target text of the replacement text to the text normalization module, the text normalization module outputs the replacement target text, which is the original text of the replacement text, and sends the output replacement target text to the dialog manager ( 3300) can be provided.

However, the present invention is not limited thereto, and the text normalization module may provide the replacement target text to the response generator model 3400 .

Through the text normalization module, even if 'Dunkin' Donuts', 'Dunkin' Donuts' and 'Dunkin' Donuts' are input, the original text 'Dunkin' Donuts' is output, and the dialog manager 3300 displays the original text of 'Dunkin' Donuts. can determine the machine action for '.

In one embodiment, the text normalization module may be a non-language understanding services (3000) stored in third-party (3 ^rd party) server.

The dialog manager 3300 is a model configured to manage a dialog by determining a response action in response to an intent and a slot output from the natural language understanding model 3200 . In an embodiment, the dialog manager 3300 may determine a machine action responding to an intent and a slot of the obtained text based on a predefined question-answer pair.

The response generator model 3400 is a model that determines a response to be provided to the user and outputs the determined response. In an embodiment, the response generator model 3400 may include a Natural Language Generator (NLG) and a Text-to-Speech (TTS) model.

In an embodiment, the response generator model 3400 may determine the type of response to be output based on the characteristics of the application 3500 or the characteristics of the device driving the application 3500 . For example, when the application 3500 receives a text response, the response generator model 3400 generates a response text using the NLG model, and provides the generated response text to the application 3500 in text form. have. When the application 3500 is driven through a device having a voice output function, for example, an AI speaker, the response generator model 3400 may convert the response text into a voice signal using the TTS model and output it.

Unlike the conventional voice assistant service, the language understanding service 3000 of the present disclosure uses the natural language understanding model 3200 to input a user input even if the user pronounces a voice command or a utterance incorrectly or pronounces the name inaccurately because he or she does not know the name correctly. , and output a response message using the dialog manager 3300 and the response generator model 3400 .

In an embodiment, the natural language understanding model 3200 of the language understanding service 3000 may be learned using context information including at least one of the user's age, gender, region, language used, and dialect. . For example, if the natural language understanding model 3200 is a model learned by reflecting context information of a user who speaks a Busan dialect, the user inputs a voice command "Search for a new product, Deungkin Denatsu" through the application 3500 When the language understanding service 3000 interprets the voice command using the natural language understanding model 3200, the intent of 'product search' and the slot of 'Dunkin Donuts' are detected from the voice command, and the dialog manager 3300 and The result of searching for new products of Dunkin Donuts may be output using the response generator model 3400 . In the case of the conventional voice assistant service, the language understanding service ( 3000) can accurately grasp the user's utterance intention and personalized context information, thereby improving user satisfaction and providing an advanced user experience (UX).

The program executed by the server 1000 or the client device 2000 described herein may be implemented as a hardware component, a software component, and/or a combination of the hardware component and the software component. A program may be executed by any system capable of executing computer readable instructions.

Software may include a computer program, code, instructions, or a combination of one or more thereof, which configures a processing device to operate as desired or is independently or collectively processed You can command the device.

The software may be implemented as a computer program including instructions stored in a computer-readable storage medium. The computer-readable recording medium includes, for example, a magnetic storage medium (eg, read-only memory (ROM), random-access memory (RAM), floppy disk, hard disk, etc.) and an optically readable medium (eg, CD-ROM). (CD-ROM), DVD (Digital Versatile Disc), etc. The computer-readable recording medium is distributed among computer systems connected through a network, so that the computer-readable code can be stored and executed in a distributed manner. The medium may be readable by a computer, stored in a memory, and executed on a processor.

The computer-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' means that the storage medium does not include a signal and is tangible, and does not distinguish that data is semi-permanently or temporarily stored in the storage medium.

In addition, the program according to the embodiments disclosed in the present specification may be provided by being included in a computer program product. Computer program products may be traded between sellers and buyers as commodities.

The computer program product may include a software program, a computer-readable storage medium in which the software program is stored. For example, a computer program product is a product (eg, a downloadable application) in the form of a software program distributed electronically through a manufacturer of a device or an electronic market (eg, Google Play Store, App Store). may include. For electronic distribution, at least a portion of the software program may be stored in a storage medium or may be temporarily generated. In this case, the storage medium may be a server of a manufacturer, a server of an electronic market, or a storage medium of a relay server temporarily storing a software program.

The computer program product may include a storage medium of a server or a storage medium of a device in a system including the server 1000 and the client device 2000 . Alternatively, when there is a third device (eg, a smartphone) that is communicatively connected to the server 1000 or the client device 2000 , the computer program product may include a storage medium of the third device. Alternatively, the computer program product may include a software program itself transmitted from the server 1000 to the device or a third device, or transmitted from the third device to the device.

In this case, one of the server 1000 , the client device 2000 , and the third device may execute a computer program product to perform the method according to the disclosed embodiments. Alternatively, two or more of the server 1000 , the client device 2000 , and the third device may execute a computer program product to distribute the method according to the disclosed embodiments.

For example, the server 1000 executes a computer program product stored in the memory 1300 (see FIG. 2 ) to control the client device 2000 communicatively connected with the server 1000 to perform the method according to the disclosed embodiments. can do.

As another example, the third device may execute a computer program product to control the device communicatively connected to the third device to perform the method according to the disclosed embodiment.

When the third device executes the computer program product, the third device may download the computer program product from the server 1000 and execute the downloaded computer program product. Alternatively, the third device may execute the computer program product provided in a preloaded state to perform the method according to the disclosed embodiments.

As described above, although the embodiments have been described with reference to the limited embodiments and drawings, various modifications and variations are possible from the above description by those skilled in the art. For example, the described techniques are performed in an order different from the described method, and/or the described components, such as computer systems or modules, are combined or combined in a different form than the described method, other components or equivalents Appropriate results can be achieved even if substituted or substituted by

Claims (23)

In a method for a server to learn a language model using text,
receiving input text input by a user from a client device;
identifying replacement target text requiring replacement from among at least one word included in the input text;
generating, with respect to the identified replacement target text, replacement text expected to be spoken by a user and having a high phonological similarity;
generating at least one learning candidate text by replacing the replacement target text among the input text with the generated replacement text;
training a natural language understanding model by using the input text and the at least one learning candidate text as training data;
A method comprising According to claim 1,
The step of identifying the replacement target text includes:
receiving a user input for selecting at least one word or phrase from the input text from the client device;
identifying at least one word or phrase selected based on the user input from among the input text; and
determining the identified at least one word or phrase as the replacement target text;
A method comprising According to claim 1,
The step of identifying the replacement target text includes:
parsing the received input text into words, morphemes, and phrases;
searching for at least one parsed word in a dictionary DB (Dictionary DB) including pronunciation sequence information for a plurality of words or information on embedding vectors; and
determining, as the replacement target text, a word that is not searched or has a frequency of use lower than a preset threshold based on a search result of the dictionary DB;
A method comprising According to claim 1,
The step of identifying the replacement target text includes:
detecting a domain into which the input text is classified by analyzing the input text using a pre-learned natural language understanding model;
detecting an intent from the input text by interpreting the input text using the pre-learned natural language understanding model;
interpreting the input text using the pre-learned natural language understanding model, thereby identifying a slot in the input text and performing slot tagging; and
determining the text corresponding to the slot as the replacement target text;
A method comprising 5. The method of claim 4,
The generating of the at least one learning candidate text comprises:
determining the generated replacement text to be the same slot as the slot identified from the input text;
A method comprising According to claim 1,
The step of generating the alternative text comprises:
extracting a phoneme sequence for the replacement target text;
searching for text similar to the extracted pronunciation column among words previously stored in a dictionary DB based on phonetic relevance of the pronunciation column; and
generating the replacement text using at least one text having a high similarity to the extracted pronunciation string based on a search result;
A method comprising According to claim 1,
The step of generating the alternative text comprises:
converting the replacement target text into an embedding vector using a word embedding model;
generating text having a vector value similar to the transformed embedding vector by using a neural network model; and
generating the alternative text by using the generated text;
A method comprising According to claim 1,
The step of generating the alternative text comprises:
converting the replacement target text into a wave signal using a text-to-speech (TTS) model;
outputting the converted sound signal;
converting the output sound signal into output text using an ASR model; and
generating the replacement text by replacing the replacement target text using the converted output text;
A method comprising According to claim 1,
The step of learning the natural language understanding model comprises:
transmitting the input text and the at least one learning candidate text to the client device;
receiving, from the client device, an identification value for at least one text selected by a user input among the input text and the at least one learning candidate text;
selecting at least one of the input text and the at least one learning candidate text based on the received identification value; and
learning the natural language understanding model by using the selected at least one text as training data;
A method comprising In a method for a client device to provide an application for learning a language model,
displaying a first graphical user interface (GUI) for receiving a user input for inputting input text for learning the language model on a display unit of the client device;
transmitting the input text received through the first GUI to a server;
displaying a second GUI for receiving a user input for selecting at least one of the at least one replacement target text identified from the input text;
receiving, from a server, at least one learning candidate text generated by replacing at least one replacement target text selected through the second GUI among the input text with an alternative text expected to be uttered by a user; and
displaying a third GUI for receiving a user input for selecting at least one of the at least one learning candidate text on the display unit;
A method comprising 11. The method of claim 10,
Display a fourth GUI for receiving a user input for selecting at least one of context information including at least one of age, gender, region, spoken language, and dialect of a user to be considered for generating the alternative text to do;
A method further comprising: In a server for learning a language model using text,
a communication interface for performing data communication with the client device;
a memory storing a program including one or more instructions; and
a processor executing one or more instructions of a program stored in the memory;
including,
The processor is
receiving the input text input by the user from the client device through the communication interface, identifying the replacement target text requiring replacement from among at least one word included in the input text, with respect to the identified replacement target text, generating at least one learning candidate text by generating an alternative text expected to be spoken by a user and having a high phonological similarity, and replacing the replacement target text among the input text with the generated replacement text; , A server for learning a natural language understanding model (Natural Language Understanding) by using the input text and the at least one learning candidate text as training data. 13. The method of claim 12,
The processor is
receiving a user input for selecting at least one word or phrase among the input text from the client device using the communication interface;
identify at least one word or phrase selected based on the user input among the input text;
determining the identified at least one word or phrase as the replacement target text. 13. The method of claim 12,
The memory stores a dictionary DB (dictionary DB) including information on pronunciation sequence information or embedding vectors for a plurality of words,
The processor is
Parsing the received input text into words, morphemes, and phrases, and searching for the parsed at least one word in the dictionary DB;
A server that determines, as the replacement target text, a word that is not searched or has a frequency of use lower than a preset threshold based on a search result. 13. The method of claim 12,
The memory stores at least one pre-trained natural language understanding model,
The processor is
Detecting a domain and an intent into which the input text is classified by analyzing the input text using any one of the natural language understanding models pre-stored in the memory;
By interpreting the input text using the pre-learned natural language understanding model, a slot in the input text is identified, and slot tagging is performed,
Determining the text corresponding to the slot as the replacement target text, the server. 13. The method of claim 12,
The processor is
Extracting a phoneme sequence for the replacement target text, and searching for text similar to the extracted pronunciation sequence among words included in the dictionary DB pre-stored in the memory based on phonetic relevance of the pronunciation sequence and,
Based on a search result, the server generates the replacement text by using at least one text having a high similarity to the extracted pronunciation string. 13. The method of claim 12,
The memory stores a word embedding model,
The processor is
converting the replacement target text into an embedding vector using the word embedding model;
By using a neural network model, a text having a vector value similar to the transformed embedding vector is generated,
A server that generates the alternative text by using the generated text. 13. The method of claim 12,
The memory stores a TTS model (Text-to-Speech) that converts text into a wave signal,
The processor is
using the TTS model to convert the replacement target text into a sound signal, output the converted sound signal, and convert the output sound signal into an output text using the ASR model pre-stored in the memory;
and generating the replacement text by replacing the replacement target text using the converted output text. 13. The method of claim 12,
The processor is
transmit the input text and the at least one learning candidate text to the client device using the communication interface, and at least one selected by a user input among the input text and the at least one learning candidate text from the client device receive an identification value relating to the text;
select at least one text among the input text and the at least one learning candidate text based on the received identification value;
A server for learning the natural language understanding model by using the selected at least one text as training data. A client device that provides an application for learning a language model, comprising:
display unit;
a user input unit configured to receive a user input for entering input text;
a communication interface for performing data communication with the server;
a memory storing a program including one or more instructions; and
a processor executing one or more instructions of a program stored in the memory;
including,
The processor is
control the display unit to display a first graphical user interface (GUI) for receiving a user input for inputting the input text, and select at least one of at least one replacement target text identified from the input text generating by controlling the display unit to display a second GUI for receiving a user input, and replacing at least one replacement target text selected through the second GUI among the input text with an alternative text expected to be uttered by the user controlling the communication interface to receive at least one learning candidate text from a server, and controlling the display unit to display a third GUI for receiving a user input for selecting at least one of the at least one learning candidate text, client device. 21. The method of claim 20,
The processor is
Display a fourth GUI for receiving a user input for selecting at least one of context information including at least one of age, gender, region, spoken language, and dialect of a user to be considered for generating the alternative text A client device that controls the display unit to do so. A computer-readable recording medium in which a program for executing the method of claim 1 in a computer is recorded. A computer-readable recording medium recording a program for executing the method of claim 10 on a computer.

Images