KR20220069219A - Functional dialog filter system applying sentence reconstruction and contrastive loss and the method thereof - Google Patents

Abstract

The present invention relates to a functional dialog filter system and a method thereof for determining a sentence corresponding to a specific domain by learning only a domain function according to a purpose. The system comprises: a reception unit receiving an input sentence; and a sentence determination unit determining the input sentence as an in-domain sentence or an out-of-domain sentence by comparing language vectors for the input sentence processed by a language model converting input into a vector with restored vectors restored by an auto-encoder receiving an in-domain input vector among the language vectors.

Description

FUNCTIONAL DIALOG FILTER SYSTEM APPLYING SENTENCE RECONSTRUCTION AND CONTRASTIVE LOSS AND THE METHOD THEREOF

The present invention relates to a functional dialog filter system and method to which sentence reconstruction and contrast loss are applied, and more particularly, to a technique for determining a sentence corresponding to a specific domain by learning only a domain function according to the purpose. will be.

Recently, due to the development of network infrastructure and improvement of performance of electronic devices, online services beyond information communication are being provided.

Among them, a chat service is provided as a place for active communication between users, and various search systems capable of effectively searching a vast amount of data are provided.

In the case of the existing functional AI (Artificial Intelligence) chat service, it performs instructions and commands for the text content entered by the user. The service was provided in a structure to deliver information to the user in the form of simply answering only the content already entered in the database for the text content entered.

Since this existing technology provides a chatting service based on preset data, there is a limitation in responding only to the user's limited text or providing a response by inferring only the previous sentence. In addition, the existing technology has a limitation in that it cannot continue a conversation about a specific domain due to an error in sentence recognition in the process of recognizing a sentence combined with many words.

Korean Patent Application Laid-Open No. 10-2014-0054494 (published on May 9, 2014), “Method and Apparatus for Providing Conversation Services”

By discriminating input sentences only with domain functions assigned according to the purpose of development, the present invention shares a wide and deep conversation on a specific topic compared to existing technologies, and provides improved conversational ability such as everyday conversations or sympathetic conversations. do.

A functional dialogue filter system according to an embodiment of the present invention receives a language vector for the input sentence processed by a receiving unit for receiving an input sentence and a language model for converting an input into a vector, and an input vector within a domain among the language vectors and a sentence determining unit that compares the restored vectors restored by the auto-encoder to determine whether the input sentence is an in-domain or out-of-domain sentence.

The sentence determining unit may be a combination of the language model for converting a sentence into a vector and the auto encoder for reducing and restoring the language vector processed through the language model.

The sentence determining unit compares the language vector processed by the language model with the restored vector restored by the auto-encoder, and if the language vector and the restored vector represent more than a preset numerical value, the input sentence is converted to a sentence outside the domain. It can be determined, otherwise, it can be identified as a sentence within the domain.

In addition, the functional dialogue filter system according to an embodiment of the present invention may further include a control unit for learning the auto-encoder by extracting only the input vector within the domain from among the language vectors.

The control unit trains through the reconstruction loss and contrast loss of the auto-encoder that restores the reconstructed vector after reducing the language vector, and the auto-encoder is learned only from the input vector within the domain and restores only the input vector within the domain. can do.

The controller may perform a contrastive task so that the language model classifies and processes the input within the domain and the input outside the domain.

The controller may perform a contrast learning algorithm for learning the same cluster or a different cluster by using an input vector within the domain and an input vector outside the domain among the language vectors.

A functional dialog filter method according to an embodiment of the present invention includes receiving an input sentence, a language vector for the input sentence processed by a language model that converts the input into a vector, and receiving an input vector in a domain from among the language vectors and comparing the restored vectors restored by the auto-encoder to determine whether the input sentence is an in-domain or out-of-domain sentence.

In addition, the functional dialogue filter method according to an embodiment of the present invention may further include the step of learning the auto-encoder by extracting only the input vector within the domain from among the language vectors.

According to an embodiment of the present invention, it is possible to share a wide and deep conversation on a specific topic by processing only the assigned domain function, and to provide an improved conversational ability such as a conversation in a common conversation or a form of empathy.

1 is a block diagram showing a detailed configuration of a functional dialogue filter system according to an embodiment of the present invention.
2 illustrates a process for determining an input sentence in a functional dialog filter system according to an embodiment of the present invention.
3 is a diagram to explain an example of discriminating a sentence according to a specific domain using the functional conversation filter system according to an embodiment of the present invention.
4 is a flowchart illustrating an operation of a functional dialog filter method according to an embodiment of the present invention.

Advantages and features of the present invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be embodied in various different forms, and only these embodiments allow the disclosure of the present invention to be complete, and common knowledge in the art to which the present invention pertains It is provided to fully inform those who have the scope of the invention, and the present invention is only defined by the scope of the claims.

The terminology used herein is for the purpose of describing the embodiments, and is not intended to limit the present invention. In this specification, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, "comprises" and/or "comprising" refers to the presence of one or more other components, steps, operations and/or elements mentioned. or addition is not excluded.

Unless otherwise defined, all terms (including technical and scientific terms) used herein may be used with the meaning commonly understood by those of ordinary skill in the art to which the present invention belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless specifically defined explicitly.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the accompanying drawings. The same reference numerals are used for the same components in the drawings, and repeated descriptions of the same components are omitted.

The gist of the embodiments of the present invention is to learn only the domain function according to the purpose to determine the sentence within the domain or outside the domain.

Functional conversation filter system and method according to an embodiment of the present invention can determine whether a sentence corresponds to a specific domain by recognizing a user's voice or by recognizing a user's chatting input through a terminal device possessed by the user. .

A terminal device is a PC, a smartphone, a tablet PC, etc., and refers to any terminal device capable of accessing a web/mobile site related to the functional dialog filter system and method according to an embodiment of the present invention or installing and executing a service-only application can do. In this case, the terminal device may perform overall service operations such as service screen configuration, data input, data transmission/reception, and data storage under the control of a web/mobile site or a dedicated application.

Furthermore, the present invention is not limited to online or application chatting through a terminal device, and various products or services capable of providing response data to dialog form data recognized from a user's voice, gesture, and facial expression is applicable to

Hereinafter, the present invention will be described in more detail with reference to FIGS. 1 to 4 .

1 is a block diagram illustrating a detailed configuration of a functional dialogue filter system according to an embodiment of the present invention.

Referring to FIG. 1 , the function conversation filter system according to an embodiment of the present invention learns only a domain function according to a purpose to determine a sentence corresponding to the domain.

To this end, the functional dialogue filter system 100 according to an embodiment of the present invention includes the receiving unit 110 and the sentence determining unit 130 , and may further include a control unit 120 and a database 140 .

The receiving unit 110 receives an input sentence. In this case, the input sentence may be any one of a declarative sentence, an interrogative sentence, a command sentence, a request sentence, an exclamation sentence, and a promise sentence, and may be a sentence form in which a plurality of words are combined.

At this time, the receiving unit 110 receives not only the input sentences of text and morphemes based on the user's voice or the user's chatting form data input through chatting, but also receives the conversation form data such as emoticons or stickers used by the user. You may.

The sentence determining unit 130 is restored by an autoencoder that receives a language vector for an input sentence processed by a language model that converts an input into a vector, and an input vector within a domain among language vectors. The reconstructed vectors are compared to determine whether the input sentence is in-domain or out-of-domain. Here, the sentence determining unit 130 may be a combination of a language model that converts a sentence into a vector and an auto encoder that reduces and restores a language vector processed through the language model.

In this case, the language model refers to a model that maps sentences to vectors, and the autoencoder refers to a vector mapped by the language model after reducing the vector mapped to the original vector. Furthermore, the auto-encoder is learned only from input vectors within the domain and only reconstructs the input vectors within the domain, but cannot reconstruct the input vectors outside the domain.

Accordingly, the sentence determining unit 130 according to an embodiment of the present invention compares the language vector processed by the language model with the restored vector restored by the auto-encoder, and when the language vector and the restored vector represent more than a preset numerical value, the input sentence can be determined as a sentence outside the domain, otherwise it can be determined as a sentence within the domain.

In more detail, the sentence discrimination unit 130 determines that the similarity between the two vectors is low when the difference between the vector values between the language vector and the reconstructed vector is greater than or equal to a preset numerical value, and determines the sentence as a non-domain sentence. It can be determined that the similarity between the sentences is high, so that the sentences are within the domain. For example, the sentence discrimination unit 130 compares the input sentence with x, which is a language vector converted by the language model, and the restored vector x' restored by the auto-encoder, and the language vector x and the restored vector x' are constant values. If an abnormality is recorded, the sentence can be determined as a sentence input outside the domain, otherwise, it can be determined as a sentence input within the domain.

The controller 120 may train the auto-encoder by extracting only an input vector within a domain from among the language vectors.

The control unit 120 can train the auto-encoder by extracting and using only an input vector (In-Domain) from among language vectors for reconstruction loss and contrast loss of the auto-encoder that restores the reconstructed vector after reducing the language vector. In addition, the auto-encoder is trained only with input vectors within the domain, so that only the input vectors within the domain can be restored. In addition, the controller 120 may perform a contrastive task so that the language model classifies and processes the input within the domain and the input outside the domain. The contrast learning is an algorithm that learns 'same cluster', otherwise 'different cluster' if they are in matching clusters.

Accordingly, the controller 120 may perform a contrast learning algorithm for learning the same cluster or a different cluster using an input vector within a domain and an input vector outside the domain among language vectors.

The database 140 of the functional dialogue filter system 100 according to an embodiment of the present invention builds data for at least one or more specific domains, and basic grammar data for classifying input sentences received by the user into words and morphemes. can be saved and maintained. Furthermore, the database 140 accumulates and builds dialogue form data and response data generated due to a conversation between a user corresponding to a specific domain and a counterpart, and data stored in the system 100 and received from an external server. You can save and maintain updated data.

2 illustrates a process for determining an input sentence in a functional dialog filter system according to an embodiment of the present invention.

Referring to FIG. 2 , the functional dialogue filter system according to an embodiment of the present invention generates a language model 220 that converts an input sentence 210 into a vector and a language vector 230 that is converted by the language model 230 . After reduction, it consists of an auto-encoder 240 that restores the original vector.

As shown in FIG. 2 , when an input sentence 210 of an intra-domain input and an out-of-domain input enters the data arrangement together, it is classified into an intra-domain input and an out-of-domain input. Inputs of are converted into a language vector 230 of a fixed dimension through the language model 220 . In this case, the language vector 230 may include an in-domain input vector 231 and an out-of-domain input vector 232 for the input sentence 210 .

Accordingly, in the present invention, only the in-domain input vector (In-Domain) 231 included in the input sentence 210 is extracted from among the language vectors 230 including the in-domain input vector 231 and the out-of-domain input vector 232 . to learn the auto encoder 240 . For this reason, since the auto-encoder 240 is trained only with the input vector 231 within the domain, only the input vector 231 within the domain can be restored, and the input vector outside the domain cannot be restored or cannot be properly restored. .

In addition, in the present invention, in order for the language model 220 to classify and transform the input within the domain and the input outside the domain in different directions, a contrastive task 250 is performed. Contrast learning 250 extracts and uses both the in-domain input vector (In-Domain, 231) and the out-of-domain input vector (Out-of-Domain, 232) from the language vector 230, and uses the intra-domain input vector 231 ) and the out-of-domain input vector 232 are in the same cluster, 'same cluster', otherwise 'different cluster' is an algorithm for learning the process of the present invention. In this case, in the present invention, through the contrast learning 250 , it is possible to learn why an incorrect answer is an incorrect answer as well as why the correct answer according to the cluster is the correct answer.

The functional dialogue filter system according to an embodiment of the present invention through the process learned as described above, when the input sentence 210 is given, it is converted using the language model 220 and the language vector x and the auto-encoder 240 ), the restored vector x' is compared, and if x and x' record more than a certain value, the sentence is determined as an out-of-domain sentence input.

3 is a diagram to explain an example of discriminating a sentence according to a specific domain using the functional conversation filter system according to an embodiment of the present invention.

Referring to FIG. 3 , the functional dialogue filter system according to an embodiment of the present invention refers to an interface that communicates through a user's natural language utterance, and is characterized by processing specific domain data allocated according to the purpose of development. . For example, the present invention may be designed to process only food-related categories such as dishes, food, restaurants, and ingredients, such as food orders and food recommendation requests, when processing a conversation for a food domain, and in the movie domain In the case of processing a conversation about a movie, only categories related to movies such as movie schedule reading and theater reservation request, movie theaters, reservations, ratings, and reviews can be processed.

In addition, looking at the example of FIG. 3 , inscriptions such as 'Fool' or 'Melong' are classified as sentence input other than the food domain or movie domain, and other domains such as 'How is the weather now?' and 'What is the popular chart?' Request sentences may also be classified as non-domain sentences so that they may not be included in the conversation for a specific domain.

The functional dialogue filter system according to the embodiment of the present invention is learned to process only the specified domain function, and by determining the input sentence input according to the learning as an in-domain sentence or an out-of-domain sentence, it cannot be accepted into the conversation. Classify sentences to share broad and deep conversations about specific domains.

4 is a flowchart illustrating an operation of a functional dialog filter method according to an embodiment of the present invention.

The method of FIG. 4 is performed by a functional dialog filter system according to the embodiment of the present invention shown in FIG. 1 .

Referring to FIG. 4 , in step 410, an input sentence is received. In this case, the input sentence may be any one of a declarative sentence, an interrogative sentence, a command sentence, a request sentence, an exclamation sentence, and a promise sentence, and may be a sentence form in which a plurality of words are combined.

In this case, in step 410, not only the input sentences of text and morphemes are received based on the user's voice or the conversational data input by the user's chatting, but also the conversational data such as emoticons or stickers used by the user may be received. have.

In step 420, a language vector for an input sentence processed by a language model that converts an input into a vector, and a restored vector restored by an autoencoder that receives an input vector in a domain among language vectors By comparison, the input sentence is determined as an in-domain or out-of-domain sentence. Here, step 420 may be a combination of a language model that converts a sentence into a vector and an auto encoder that reduces and restores a language vector processed through the language model.

In this case, the language model refers to a model that maps sentences to vectors, and the autoencoder refers to a vector mapped by the language model after reducing the vector mapped to the original vector. Furthermore, the auto-encoder is learned only from input vectors within the domain and only reconstructs the input vectors within the domain, but cannot reconstruct the input vectors outside the domain.

Accordingly, in step 420 according to an embodiment of the present invention, the language vector processed by the language model and the restored vector restored by the auto-encoder are compared. , otherwise it can be identified as a sentence within the domain.

More specifically, in step 420, if the difference between the vector values between the language vector and the reconstructed vector is greater than or equal to a preset numerical value, it is determined that the similarity between the two vectors is low, and it is determined as a sentence outside the domain. If it is less than the preset numerical value, the similarity between the two vectors is high. It can be determined as a sentence within the domain by judging. For example, step 420 compares the language vector x converted by the language model with respect to the input sentence and the restored vector x' restored by the auto-encoder, and when the language vector x and the restored vector x' record a certain number or more The sentence may be determined as a sentence input outside the domain, otherwise it may be determined as a sentence input within the domain.

In step 430, the auto-encoder may be trained by extracting only an input vector within a domain from among the language vectors.

Step 430 can train the auto-encoder by extracting and using only an input vector (In-Domain) from among language vectors for reconstruction loss and contrast loss of the auto-encoder that restores the reconstructed vector after reducing the language vector, The auto-encoder is trained only with input vectors within the domain and can only restore input vectors within the domain. In addition, in operation 430, a contrastive task may be performed so that the language model classifies and processes intra-domain input and out-of-domain input. The contrast learning is an algorithm that learns 'same cluster', otherwise 'different cluster' if they are in matching clusters.

Accordingly, step 430 may perform a contrast learning algorithm for learning the same cluster or a different cluster by using an input vector within a domain and an input vector outside the domain among language vectors.

However, although the step 430 of learning the auto-encoder in FIG. 4 is shown to be performed after the step 420, it is not limited thereto, and the step 430 is located between the steps 410 and 420 to train the auto-encoder.

The system or apparatus described above may be implemented as a hardware component, a software component, and/or a combination of the hardware component and the software component. For example, devices and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA). , a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions, may be implemented using one or more general purpose or special purpose computers. The processing device may execute an operating system (OS) and one or more software applications executed on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, although one processing device is sometimes described as being used, one of ordinary skill in the art will recognize that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that can include For example, the processing device may include a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

Software may comprise a computer program, code, instructions, or a combination of one or more thereof, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or apparatus, to be interpreted by or to provide instructions or data to the processing device. , or may be permanently or temporarily embody in a transmitted signal wave. The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with reference to the limited embodiments and drawings, various modifications and variations are possible from the above description by those skilled in the art. For example, the described techniques are performed in an order different from the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (12)

a receiver for receiving an input sentence; and
A language vector for the input sentence processed by a language model that converts an input into a vector is compared with a reconstructed vector restored by an auto-encoder that receives an input vector in the domain from among the language vectors in the domain for the input sentence (In-Domain) or out-of-domain (Out-of-Domain) sentence discrimination unit
A feature dialog filter system that includes. According to claim 1,
The sentence discrimination unit
The functional dialogue filter system, characterized in that the language model for converting a sentence into a vector and the auto-encoder for reducing and restoring the language vector processed through the language model are combined. 3. The method of claim 2,
The sentence discrimination unit
The language vector processed with the language model is compared with the restored vector restored by the auto-encoder, and if the language vector and the restored vector represent more than a preset numerical value, the input sentence is determined as a sentence outside the domain, otherwise If not, to determine the sentence within the domain, a function dialog filter system. According to claim 1,
A control unit for learning the auto-encoder by extracting only an input vector within the domain from among the language vectors
Features dialog filter system further comprising. 5. The method of claim 4,
the control unit
After reducing the language vector, training through the reconstruction loss and contrast loss of the auto-encoder to restore the restored vector,
The auto encoder
Functional dialogue filter system, characterized in that only the input vector within the domain is learned and only the input vector within the domain is restored. 5. The method of claim 4,
the control unit
A functional dialog filter system, characterized in that the language model performs a contrastive task in order to classify and process intra-domain input and out-of-domain input. 7. The method of claim 6,
the control unit
A functional dialog filter system for performing a contrast learning algorithm for learning the same cluster or a different cluster by using the input vector within the domain and the input vector outside the domain among the language vectors. receiving an input sentence; and
A language vector for the input sentence processed by a language model that converts an input into a vector is compared with a reconstructed vector restored by an auto-encoder that receives an input vector in the domain from among the language vectors in the domain for the input sentence Step to determine (In-Domain) or Out-of-Domain sentences
A feature dialog filter method comprising a. 9. The method of claim 8,
The determining step is
The language vector processed with the language model is compared with the restored vector restored by the auto-encoder, and if the language vector and the restored vector represent more than a preset numerical value, the input sentence is determined as a sentence outside the domain, otherwise Otherwise, it is determined as a sentence within the domain, a function dialog filter method. 9. The method of claim 8,
Learning the auto-encoder by extracting only the input vector within the domain from among the language vectors
A function dialog filter method further comprising a. 11. The method of claim 10,
The learning step
Functional dialogue filter method, characterized in that the language model performs a contrast learning (contrast task) to classify and process intra-domain input and out-of-domain input. 12. The method of claim 11,
The learning step
A function dialog filter method for performing a contrast learning algorithm for learning the same cluster or a different cluster by using the input vector within the domain and the input vector outside the domain among the language vectors.

Images