KR20200083404A - Method, system and computer program for artificial intelligence answer - Google Patents

Abstract

According to an embodiment of the present invention, provided is a system for artificial intelligence query and answer which comprises: a user query receiving unit for receiving a user query from a user terminal; a first query expansion unit for analyzing a user query to generate a question template and determining whether the user query matches the generated question template; a second query expansion unit for generating a similar question template by using natural language processing and deep learning models when the user query and the generated question template do not match; a training data construction unit for generating training data for training the second query expansion unit by using a neural machine translation (NMT) engine; and a query response unit for transmitting a user query result derived through the first or second query expansion unit to the user terminal. Accordingly, a natural language-based sentence can be accurately understood and a search result suitable for the intent can be provided.

Description

Artificial Intelligence Question and Answer System, Method and Computer Program {METHOD, SYSTEM AND COMPUTER PROGRAM FOR ARTIFICIAL INTELLIGENCE ANSWER}

The present invention relates to an artificial intelligence query response system, method, and computer program, and more specifically, constructs and paraphrases learning data using an NMT engine to accurately understand natural language-based sentences and provide search results suitable for intentions. Artificial intelligence question and answer system, method and computer program for learning engine.

In order to implement various application services that deal with data expressed in natural language, it is necessary to understand and engineer linguistic knowledge, language-specific structural knowledge, and complex qualities of the language, so it is an entry barrier to work such as adding a new language or domain. This exists.

In particular, the traditional NLU (Natural Language Understanding) method has a characteristic that strongly relies on the characteristics hand-crafted. For this reason, it takes a lot of time to extract features, and there are limitations that cannot cope with various cases such as new patterns, typos, and spelling errors.

To solve this problem, a deep learning-based NLU processing method has been recently considered. The deep learning-based NLU method is a method of automatically learning features from data, and has the advantage of being able to process a wider range of context information than before. Through this, even in a new word or typo that has not been previously learned, it can be robust compared to a traditional rule/statistical based NLU, thereby partially compensating for the disadvantages of the existing traditional NLU.

On the other hand, with the expansion of the spread of smart machines with voice-recognition speakers, and the development of artificial intelligence technology, the information retrieval method is based on the existing keyword input, the existing search method that checks the document list, the natural language-based sentence input, and the specific response form Search trends are changing.

KR 10-1851787 B1

An object of the present invention is to accurately understand a natural language-based sentence and provide a search result suitable for an intention.

Another object of the present invention is to build learning data using an NMT engine and to improve the accuracy of a search by learning a paraphrase engine.

According to an embodiment of the present invention, a user query receiver for receiving a user query from a user terminal; A first query expansion unit that analyzes a user query to generate a question template, and determines whether the user query and the generated question template match; A second query expansion unit that generates a similar question template using natural language processing and a deep learning model when the user query and the generated question template do not match; A training data construction unit that generates training data for training the second query expansion unit using a Neural Machine Translation (NMT) engine; And a query response unit that delivers a user query result derived through the first query expansion unit or the second query expansion unit to the user terminal. An artificial intelligence query response system is provided.

In the present invention, the question template and the similar question template may be a semantic triple-based question template consisting of an entity, an attribute, and an instant answer.

In the present invention, the learning data construction unit uses the NMT engine to translate a first sentence in Korean into a specific foreign language, and translates the first sentence translated into a specific foreign language back into Korean to obtain a second sentence, The generated second sentence can be constructed as learning data.

In the present invention, the second query expansion unit, a natural language expansion module for processing the user query in natural language; And

A paraphrase engine that generates a similar question template through paraphrasing the natural language processed user query; It may include.

In the present invention, when the user query and the generated question template do not match, an immediate answer corresponding to the generated question template may be provided to the user terminal.

According to an embodiment of the present invention, a user query receiving step of receiving a user query from a user terminal; A first query expansion step of analyzing a user query to generate a question template and determining whether the user query and the generated question template match; A second query expansion step of generating a similar question template using natural language processing and a deep learning model when the user query and the generated question template do not match; A learning data construction step of generating learning data for training the second query expansion unit using a Neural Machine Translation (NMT) engine; And a query response step of transmitting a user query result derived through the first query expansion step or the second query expansion step to the user terminal. An artificial intelligence query answering method is provided.

In the present invention, the question template and the similar question template may be a semantic triple-based question template consisting of an entity, an attribute, and an instant answer.

In the present invention, in the step of constructing the learning data, the first sentence in Korean is translated into a specific foreign language using the NMT engine, and the first sentence translated into a specific foreign language is translated into Korean again to obtain a second sentence. , The generated second sentence can be constructed as learning data.

In the present invention, the second query expansion step includes: a natural language expansion module for processing the user query in natural language; And a paraphrase engine that generates a similar question template through paraphrasing the natural language-processed user query. It may include.

In the present invention, when the user query and the generated question template do not match, an immediate answer corresponding to the generated question template may be provided to the user terminal.

According to an embodiment of the present invention, a user query receiver for receiving a user query from a user terminal; A query expansion unit generating a similar question template of the user query using a paraphrase engine; Includes; a learning data construction unit for generating training data for training the query expansion unit using a NMT (Neural Machine Translation) engine, wherein the learning data construction unit is the first Korean language using the NMT engine An artificial intelligence query response system is provided that translates a sentence into a specific foreign language, translates a first sentence translated into a specific foreign language back into Korean, obtains a second sentence, and builds the generated second sentence into learning data.

According to an embodiment of the present invention, a user query receiver for receiving a user query from a user terminal; A query expansion unit generating a similar question template of the user query using a paraphrase engine; Includes; a learning data construction unit for generating training data for training the query expansion unit using a NMT (Neural Machine Translation) engine, wherein the learning data construction unit is the first Korean language using the NMT engine An artificial intelligence query response system is provided that translates a sentence into a specific foreign language, translates a first sentence translated into a specific foreign language back into Korean, obtains a second sentence, and builds the generated second sentence into learning data.

In the present invention, the similar question template may be a semantic triple-based question template consisting of an entity, an attribute, and an instant answer.

According to an embodiment of the present invention, a user query receiving step of receiving a user query from a user terminal; A query expansion step of generating a similar question template of the user query using a paraphrase engine; And a learning data construction step of generating training data for training the query expansion unit using a NMT (Neural Machine Translation) engine, wherein the training data construction step comprises: It may be an artificial intelligence query answering method in which one sentence is translated into a specific foreign language, the first sentence translated into a particular foreign language is translated into Korean again to obtain a second sentence, and the generated second sentence is constructed as learning data. .

In the present invention, the similar question template may be a semantic triple-based question template consisting of an entity, an attribute, and an instant answer.

According to an embodiment of the present invention, a user query receiver for receiving a user query from a user terminal; A query expansion unit generating a similar question template of the user query using a paraphrase engine; Includes; learning data construction unit for generating training data for training the query expansion unit using a NMT (Neural Machine Translation) engine; the learning data construction unit, the user log data to the neural network based NMT engine NMT engine management unit for translation and re-translation to generate learning data; And learning to store the training data generated by the NMT engine management unit, to train a paraphrase model that can be applied to the paraphrase engine using the generated training data, and to test and verify the paraphrase model. Data management department; It may include.

According to the present invention, it is possible to accurately understand a sentence based on a natural language and provide search results suited to the intention.

According to the present invention, it is possible to automatically construct a large amount of accurate learning data using an existing NMT engine.

1 is a diagram illustrating an example of a network environment according to an embodiment of the present invention.
2 is a block diagram illustrating an internal configuration of a user terminal and a server in an embodiment of the present invention.
3 is for explaining the semantic triple-based search results.
4 shows an example of performing semantic triple-based search.
5 illustrates an internal configuration of a processor according to an embodiment of the present invention.
6 is a time series diagram of an artificial intelligence query response method according to an embodiment of the present invention.
7 is a view showing the overall structure of an artificial intelligence query response system according to an embodiment of the present invention.
8 is a diagram for explaining building a training data and a paraphrase model according to an embodiment of the present invention.

For a detailed description of the present invention, which will be described later, reference is made to the accompanying drawings that illustrate, by way of example, specific embodiments in which the present invention may be practiced. These embodiments are described in detail enough to enable those skilled in the art to practice the present invention. It should be understood that the various embodiments of the invention are different, but need not be mutually exclusive. For example, specific shapes, structures, and characteristics described in this specification may be implemented by changing from one embodiment to another without departing from the spirit and scope of the present invention. In addition, it should be understood that the position or arrangement of individual components within each embodiment may be changed without departing from the spirit and scope of the present invention. Therefore, the detailed description to be described later is not intended to be taken in a limiting sense, and the scope of the present invention should be taken to cover the scope claimed by the claims of the claims and all equivalents thereto. In the drawings, similar reference numerals denote the same or similar components throughout several aspects.

1 is a diagram illustrating an example of a network environment according to an embodiment of the present invention.

The network environment of FIG. 1 shows an example including a plurality of user terminals 110, 120, 130, 140, a server 150 and a network 160. 1 is not limited to the number of user terminals or the number of servers as shown in FIG. 1 as an example for explaining the present invention.

The plurality of user terminals 110, 120, 130, and 140 may be fixed terminals or mobile terminals implemented as computer devices. For example, a plurality of user terminals (110, 120, 130, 140), a smart phone (smart phone), mobile phones, navigation, computers, notebooks, digital broadcasting terminals, PDA (Personal Digital Assistants), PMP (Portable Multimedia Player) ), tablet PC, etc. For example, the user terminal 1 110 may communicate with other user terminals 120, 130, 140 and/or the server 150 through the network 160 using a wireless or wired communication method.

The communication method is not limited, and a communication method using a communication network (for example, a mobile communication network, a wired Internet, a wireless Internet, and a broadcast network) that the network 160 may include may include short-range wireless communication between devices. For example, the network 160 includes a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), and a broadband network (BBN). , Any one or more of the networks such as the Internet. In addition, the network 160 may include any one or more of a network topology including a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree or a hierarchical network, etc. It is not limited.

The server 150 is a computer device or a plurality of computer devices that communicate with a plurality of user terminals 110, 120, 130, 140 through a network 160 to provide commands, codes, files, contents, services, and the like. Can be implemented.

In one example, the server 150 may provide a file for installing the application to the user terminal 1 110 accessed through the network 160. In this case, the user terminal 1 110 may install an application using a file provided from the server 150. In addition, the service provided by the server 150 is accessed by accessing the server 150 under the control of an operating system (OS) included in the user terminal 1 110 and at least one program (eg, a browser or an installed application). Content can be provided. For example, when the user terminal 1 110 transmits the content viewing through the network 160 to the server 150 under the control of the application, the server 150 unique response using the semantic triple-based knowledge expansion system It can be transmitted to the user terminal 1 (110), the user terminal 1 (110) can display a unique instant response under the control of the application. As another example, the server 150 may establish a communication session for data transmission and reception, and route data transmission and reception between a plurality of user terminals 110, 120, 130, and 140 through the established communication session.

2 is a block diagram illustrating an internal configuration of a user terminal and a server in an embodiment of the present invention.

2 illustrates an internal configuration of the user terminal 1 110 as an example for one user terminal and the server 150 as an example for one server. Other user terminals 120, 130, 140 may also have the same or similar internal configuration.

The user terminal 1 110 and the server 150 may include memories 211 and 221, processors 212 and 222, communication modules 213 and 223, and input/output interfaces 214 and 224. The memory 211, 221 is a computer-readable recording medium, and may include a non-permanent mass storage device such as random access memory (RAM), read only memory (ROM), and a disk drive. In addition, an operating system and at least one program code (for example, a code for a browser or an application described above, which is installed and driven in the user terminal 1 110) may be stored in the memories 211 and 221. These software components can be loaded from a computer-readable recording medium separate from the memories 211 and 221 using a drive mechanism. Such a separate computer-readable recording medium may include a computer-readable recording medium such as a floppy drive, disk, tape, DVD/CD-ROM drive, and memory card. In other embodiments, software components may be loaded into memory 211 and 221 through communication modules 213 and 223 rather than a computer-readable recording medium. For example, at least one program is a program (an example) in which a file distribution system (for example, the server 150 described above) for distributing installation files of developers or applications is installed by files provided through the network 160. It can be loaded into the memory (211, 221) based on the above-described application).

Processors 212 and 222 may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input/output operations. Instructions may be provided to processors 212 and 222 by memory 211 and 221 or communication modules 213 and 223. For example, the processors 212 and 222 may be configured to execute instructions received according to program codes stored in a recording device such as the memories 211 and 221.

The communication modules 213 and 223 may provide a function for the user terminal 1 110 and the server 150 to communicate with each other through the network 160, and other user terminals (for example, the user terminal 2 120). Alternatively, a function for communicating with another server (eg, the server 150) may be provided. In one example, the request generated by the processor 212 of the user terminal 1 110 according to the program code stored in the recording device such as the memory 211 is controlled through the network 160 under the control of the communication module 213 server ( 150). Conversely, control signals, commands, contents, files, etc. provided under the control of the processor 222 of the server 150 are communicated through the communication module 223 and the network 160 to the communication module of the user terminal 1 110 ( 213) may be received by the user terminal 1 (110). For example, control signals or commands of the server 150 received through the communication module 213 may be transmitted to the processor 212 or the memory 211, and the user terminal 1 110 may be used for content or files. It may be stored as a storage medium that may further include.

The input/output interfaces 214 and 224 may be means for interfacing with the input/output device 215. For example, the input device may include a device such as a keyboard or mouse, and the output device may include a device such as a display for displaying a communication session of an application. As another example, the input/output interface 214 may be a means for an interface with a device in which functions for input and output are integrated into one, such as a touch screen. As a more specific example, the processor 212 of the user terminal 1 110 is configured using data provided by the server 150 or the user terminal 2 120 in processing a command of a computer program loaded in the memory 211. The service screen or content to be displayed may be displayed on the display through the input/output interface 214.

In addition, in other embodiments, the user terminal 1 110 and the server 150 may include more components than those of FIG. 2. However, there is no need to clearly show most prior art components. For example, the user terminal 1 110 is implemented to include at least a portion of the input/output device 215 described above, or other configuration such as a transceiver, global positioning system (GPS) module, camera, various sensors, database, etc. It may further include elements.

The artificial intelligence query response method and system of the present invention may be implemented by the server 150, and in more detail, the artificial intelligence query response method may be implemented by an instruction processed by the processor 222 of the server 150.

The problem to be solved by the present invention is to provide a system for responding to various queries of a user by providing an artificial intelligence question-and-answer method, apparatus, and program.

Due to the emergence of AI-based smart machines led by artificial intelligence speakers, QA (Question Answering) search, which is different from the existing portal search, has appeared, and information search and input means are not touch, keyword input, etc. As it changed, the need to understand natural language-based sentences increased, unlike the keyword-based search of the existing portal search.

For this reason, artificial intelligence-based question-and-answer methods, devices, and programs that can cope with various cases such as new patterns, typos, and spelling errors are needed to accurately understand natural language-based sentences and deliver search results that match their intentions.

In addition to the QA search described above, the present invention has characteristics applicable to all systems that grasp the user's intention and provide desired results, and thus can be applied in various forms. For example, if Slot Filling is used instead of the instant answer provided as the correct answer in Knowledge QA, it can be applied in a manner that provides necessary information as an API that provides specific functions according to the user's intention. Through this, the present invention can be said to be a technology with high utilization, which can be used for various ranges such as home IoT, smart toy/home robot, and connected car. Therefore, in the following specification, the QA search method is mainly described, but the present invention is not necessarily limited thereto, and is applicable to all applicable systems.

First, before explaining the present invention in full detail, the differences between the artificial intelligence query response method of the present invention and the existing search engine will be described. The artificial intelligence query response system according to an embodiment of the present invention may provide a unique instant answer using a semantic triple-based question template. The artificial intelligence query response method of the present invention may have a difference from the existing search engine in that the search result is provided in a unique instant answer, that is, in the form of an immediate answer, rather than in the form of a document. In addition, the artificial intelligence query response method of the present invention can construct learning data for providing semantic triple-based search results.

3 is for explaining the semantic triple-based search results.

Referring to FIG. 3, an existing search engine (As-Is, Searh) has an input method as a keyword, provides a document list as a search result, and the search platform operates on a PC or mobile.

In contrast, in the artificial intelligence query response method (To-Be, Question-Answering) of the present invention, the input method is a natural language-based sentence, and a specific response, ie, an instant unique response, can be provided as a search result, and the platform is a PC. Or it is not limited to mobile and can be implemented anywhere.

In more detail, the artificial intelligence query response method of the present invention enables a user to search information as naturally as a user asks a question by allowing a natural search engine-based sentence to be input while a conventional search engine inputs a keyword. In addition, the artificial intelligence query response method of the present invention provides a specific response as a search result, thereby reducing the inconvenience of a user having to search for a search result directly from a document list provided by an existing search engine and providing an optimal search result. have. In addition, the artificial intelligence query response method of the present invention is not limited to a PC or mobile as a platform, and has the advantage of being able to instantly search for information anywhere on a smart machine basis.

4 shows an example of performing semantic triple-based search.

The knowledge DB 400 shown in FIG. 4 is a special type of knowledge base database that stores data in the form of semantic triples simulating real users' queries, and it is a unique instant answer without a separate reasoning process. ). Knowledge DB 400 has the form of entity 432-attribute 434-instant answer 438. In the embodiment of the present invention described below, the knowledge DB 400 may be a database that exists inside or outside the server 150 and communicates with the processor 222 to provide data.

Figure 4 is a "What is the height of Baekdusan?" When receiving a user query (410), by analyzing the user query (420),'Baekdusan' and'height' after extracting the key words as a subject to ask for Baekdusan The height can be analyzed as a question. Accordingly, data with entity = "Baekdusan" and attribute = "height" is searched, and the instant answer of the corresponding item is determined as a result value, and the corresponding answer is 2,744m to the user (450). As described above, the knowledge DB 400 may search for an optimal correct answer and provide an optimal answer without a separate reasoning process. Hereinafter, the artificial intelligence query response method and system based on the semantic triple as described in FIGS. 3 and 4 will be described in more detail.

5 illustrates an internal configuration of a processor according to an embodiment of the present invention.

The processor 212 may include a web browser or application capable of receiving and outputting a web page from online. The configuration of the semantic triple-based knowledge expansion system according to an embodiment of the present invention in the processor 212 is a query receiving unit 510, a first query expansion unit 520, a second query expansion as shown in FIG. It may include a unit 530, a learning data construction unit 540 and a query response unit 550. In addition, the second query expansion unit 530 includes a natural language expansion module 531 and a paraphrase engine 532, and the training data construction unit 540 includes an NMT engine management unit 541, a learning data management unit 542, and A model distribution unit 543 may be included. According to an embodiment of the present invention, components of the processor 212 may be selectively included or excluded from the processor 212. Further, according to an embodiment, the components of the processor 212 may be separated or merged to express the function of the processor 212.

Here, the components of the processor 212 are the processor 212 according to an instruction provided by the program code stored in the user terminal 1 110 (eg, an instruction provided by a web browser driven by the user terminal 1 110). It may be expressions of different functions (different functions) of the processor 212 performed by.

The processor 212 and the components of the processor 212 may control the user terminal 1 110 to perform steps S1 to S5 included in the artificial intelligence query response method of FIG. 4. For example, the processor 212 and components of the processor 212 may be implemented to execute instructions according to code of an operating system included in the memory 211 and code of at least one program.

FIG. 6 is a time series diagram of an artificial intelligence query response method according to an embodiment of the present invention, and FIGS. 7a and 7b are diagrams showing the overall structure of an artificial intelligence query response system according to an embodiment of the present invention. More specifically, FIG. 7A shows the operation of the query response system when the user query analysis is successful, and FIG. 7B shows the operation of the query response system when the user query analysis is unsuccessful. Hereinafter, the present invention will be described with reference to FIGS. 6, 7A, and 7B together.

First, the query receiving unit 510 receives a user's query from the user terminal 110 (S61). In more detail, the query receiving unit 510 receives the user query (S71), and transmits the received query to the first query expansion unit 520 (S72). The form of the user's query may be received in various forms such as voice and text. The query receiving unit 510 may convert the received user's query into an appropriate form through an appropriate conversion process.

According to an embodiment of the present invention, the user may input a natural language-based query to the user terminal 100, which is an AI-based smart machine to which the artificial intelligence query response method, apparatus, and program of the present invention are applied. The smart machine may include not only existing smart phones and computers, but also artificial intelligence speakers, connected cars, home IoT, AI appliances, personal assistants, home robots/smart toys, chatbot applications, and intranets.

Next, the first query expansion unit 520 analyzes the user query to determine whether the entity and attribute are recognizable (S62). As a result of user query analysis, if the entity and attribute are recognizable, a question template is generated to perform the first query extension, and it is checked whether the user query and the generated question template match (S64). In more detail, the first query expansion unit 520 analyzes the user's query, checks the answer information for the query from the knowledge DB 400, finds answer information in the knowledge DB 400, and finds a number of similar queries. Create a question template and compare the user query with the question template to perform a primary search.

In more detail, the first query extension unit 520 may generate a question template matching the user query based on the result of analyzing the user query. When a user query is received, the first query expansion unit 520 analyzes the user query, generates a question template based on the analyzed result, and performs the first query expansion. At this time, the first query expansion unit 520 may generate a question template from the knowledge DB 590 described above. Hereinafter, a specific configuration in which the first query expansion unit 520 generates a question template will be described.

First, the first query extension unit 520 may analyze a user query using a natural language processing (NLP) engine to analyze a user query in the semantic triple format as described above. In this case, entities and attributes are found in the user query using techniques such as morpheme analysis. Since ordinary users ask questions in the form of entity + attribute, the sentences are sequentially analyzed to find candidate entities and attribute groups in the user query.

In an embodiment of the present invention, the first query expansion unit 520 generates a content of the corresponding entity and attribute as a question template when there is only one entity and attribute criteria when analyzing a user query. For example,'Which Overwatch heroes are active in London?' In the case of an entity, if an entity is analyzed as an'overwatch hero' and the attribute is not detected, a question template can be created by utilizing all the attributes that have an'overwatch hero' as an entity in the knowledge DB.

Next, the first query expansion unit 520 may generate a question template from the knowledge DB 400 based on the result of query analysis. Specifically, the first query extension unit 520 searches the entity and attribute candidate groups for each category matching the user query in a search form, and generates a question template based on the entity synonyms and attribute synonyms. At this time, the question template may have not only entity and attribute, but also instant answer as additional information. This may provide an instant answer to the user when the user query to be described later and the question template created by the first query expansion unit 520 match (S65 ). If an instant answer exists, the corresponding question template may be generated as training data for training a second query expansion step described later (S68).

Next, the first query expansion unit 520 compares the generated question template and the user query, and checks whether the result matches (S73). If it is determined that the question template generated by the first query expansion unit 520 is the same as the user query, an instant answer previously generated from the question template is provided as an answer (S65). The first query expansion unit 520 may compare the user template with the question template generated by removing meaningless letters or words from the user query. For example,'What is the elevation of Mt. If there is a question that can be eliminated through the natural language processing,'righteousness' and'silver' that are not related to meaning expression can be removed. It is determined whether the user query provided with meaningless letters or words exactly matches the generated question template. If it is determined that the generated query template and the user query are not the same, the user query may be extended using the second query expansion unit 530.

Next, the second query extension unit 530 generates a similar question template when the generated question template and the user query are inconsistent. In more detail, the second query expansion unit 530 compares the results by expanding the second question template by generating a semantic-like question template using a natural language processing and a deep learning model (S66). That is, when the question template generated by the first query expansion unit 520 and the result of the user query matching do not exist, the question template previously generated using the second query expansion unit 530 which is a similar query engine. Based on the query expansion, similar question templates can be additionally generated (S74).

Alternatively, if it is determined in step S72 that the user query analysis of the first query expansion unit 520 is not possible to recognize the entity and attribute, the second query expansion unit 530 uses the previously derived user query results as a similar question template. To expand the query and compare the results (S63). That is, the user query and the previously generated question template are verified using a similar query engine (S78). That is, when the user query analysis does not find both the entity and attribute, the paraphrase engine of the second query extension compares the user query and the system log answer based on similarity and performs a search based on the answers from the existing system log. It can be, and the corresponding step is S63 and S78.

The semantic similar question generation method using natural language processing and a deep learning model is used as a method for generating the similar question template of the second query expansion unit 530, and the results can be compared by expanding the semantic similar question template. As illustrated in FIG. 3, the second query expansion unit 530 may include a natural language expansion module 531 and a paraphrase engine 532. That is, the second query expansion unit 530 proceeds with the predicate expansion through the natural language expansion module 531, and grasps the similarity between the user query and the question template through the deep learning-based paraphrase engine 532 to determine the user's intention. You can provide the correct answer.

First, the natural language expansion module 531 of the second query expansion unit 530 may generate a similar question template by providing various patterns for queries of a specific subject using a natural language processing method. In one embodiment,'where is [Person] born?' The question is,'Where was [Person] born?','Where was [Person] born?' It has the same meaning. The second query expansion unit 530 may generate a similar question template by extending the question template of the entity-attribute combination created from the query template through a natural language processing method as in the example above.

In more detail, the natural language processing method of the natural language expansion module 531 may be implemented in a manner of expanding in a pattern form according to a specific attribute by using a separately constructed DB of similar quality. For example, in the case of the'birth place' attribute, the second query expansion unit 530 asks,'Where were you born?','Where were you born?','Where were you born?' Similar question templates can be generated by extending question templates through various predicate extensions such as.

In addition, the paraphrase engine 532 of the second query expansion unit 530 generates a semantic similar question template of the user query through paraphrase. In addition, the paraphrase engine 532 compares the user query and the similar question template to check the result (S75). In more detail, the second query extension unit 530 compares the similarity between the user query, the question template, and the similar question template through a deep learning-based paraphrase engine by combining the existing generated question template and the expanded similar question template. The similarity comparison is conducted in two stages, and the details are as follows.

As a first step, first, the paraphrase engine 532 measures the similarity between the user query, the existing query template generated by the first query expansion unit 520, and the similar question template, and is higher than the total number of the question template and the similar question template. N candidates are selected. At this time, the number of the top N can be changed according to the manager page or the statistics-based feedback program. The paraphrase engine 532 compares the user query with the existing question templates and Top N similar question templates, and returns one question template and similarity that are considered to be the most similar.

In the second step, the paraphrase engine 532 determines whether the Top 1 question template selected based on the similarity has the same meaning as the user query. In the case of judgment criteria, the manager initially selects them arbitrarily, but can then be adjusted based on statistics through actual result feedback. For example, even if the initial similarity is selected as 90%, if the answer is correct even though it is more than 85% by checking the actual correct answer derivation history, the answer is automatically changed by changing the similarity from 90% to 85% automatically Coverage can be expanded. If the similarity is less than a certain threshold, a message indicating that there is no search result may be output.

On the other hand, when the above-described step is completed, the second query expansion unit 530 includes a user query, a question template selected as Top 1, a question template selected as Top N, a search time and step, and a search system. Information can be stored in a separate DB.

Next, the query response unit 550 transmits the user query results derived through the first query expansion unit 520 and the second query expansion unit 530 to the user terminal 110 (S65). The user query result is delivered to the AI-based smart machine and delivered to the interface that fits the characteristics. 'Answer'. Detailed information such as'time' and'device' is stored as a system log and can be used for future paraphrase model management.

Next, the query response unit 550 delivers the derived user query result to the user terminal (S67).

In addition, the training data construction unit 540 according to an embodiment of the present invention uses the derived user query result to train the training data for training the second query expansion unit 530 (NMT). ) Can be generated using the engine (S68). That is, the second query expansion unit 530 may perform learning using the training data generated by the training data construction unit 540. In particular, the paraphrase engine 532 uses a method of using a semantic similar question using a deep learning model, and for this, it is necessary to train the corresponding deep learning with rich learning data. Accordingly, the training data construction unit 540 may perform training of the paraphrase engine and generate training data for training.

In more detail, according to an embodiment of the present invention, in the case of the deep learning-based paraphrase engine 532, the model does not specify the structure of the model in advance, and a model is generated through training to minimize operator intervention, and to create a complex and deep structure. It has the feature of being more accurate than the existing method. However, there is a problem in that a large amount of learning data of tens of thousands or more is required in order to produce a performance that can replace human tasks. Accordingly, according to an embodiment of the present invention, the artificial intelligence query response device proposes a method for automatically constructing learning data.

First, the training data construction unit 540 generates training data for paraphrase. In more detail, the learning data construction unit 540 constructs training data capable of learning the artificial intelligence response method of the present invention so that the above-described second query expansion unit 530 can form a semantic similar question template. To this end, the training data construction unit 540 may include an NMT engine management unit 541, a training data management unit 542, and a model distribution unit 543.

In the present invention, the second query expansion unit 330 uses the NMT engine management unit 541 for using a plurality of neural machine translation (NMT) engines to build training data in order to continuously secure the quality of the constructed paraphrase engine. Including, learning data management unit 542 using a statistical-based separate management program for managing the translation quality of the NMT engine. In addition, the learning data construction and quality management including a series of processes, including a model distribution unit 543 that progresses learning with the generated learning data and distributes and applies a paraphrase model, collectively constitutes the present invention. Can be.

8 is a diagram for explaining building a training data and a paraphrase model according to an embodiment of the present invention.

First, the learning data construction unit 540 sets the actual query of the user as raw data to generate the training data, and transmits user log data based on the actual query to the NMT engine management unit 541 (S81). At this time, the actual query can use the data stored in the server's Log DB. The training data construction unit 540 transmits the corresponding data to the NMT engine management unit 541, and prepares to generate training data.

Next, the NMT engine management unit 541 uses a plurality of external NMT engines based on a neural network, and translates user log data or user queries into other languages, and then re-translates them into Korean to generate learning data (S82). . According to an embodiment of the present invention, the NMT engine management unit 541 may translate the first sentence written in Korean into a specific foreign language, and translate the first sentence translated into a specific foreign language back into Korean to obtain a second sentence. have. That is, the NMT engine management unit 541 may utilize the NMT engine so that the paraphrase engine 532 can collect similar natural language expressions as learning data for the same query or sentence.

Since the NMT engine used in the present invention performs translation in a neural network manner through learning, not based on predetermined patterns and rules, if the first sentence is translated into a foreign language and then translated into Korean again, the same or similar meaning to the first sentence You can obtain natural language sentences with different expressions. In addition, since the external NMT translation engine uses different neural network rules and different learning data, when the same sentence is translated into a specific foreign language and then translated into Korean again, natural language sentences having similar meanings but different expressions are additionally acquired. can do.

In addition, when the first sentence in Korean is translated into the first foreign language, the first foreign language is translated into the second foreign language, and then the second foreign language is translated into Korean, the natural language sentence having similar meaning but expressed in another expression Can be obtained additionally.

The learning data formed in the above-described manner is transmitted to the learning data management unit 542 together with actual user queries and external NMT information used, translation steps and languages used for translation. Specifically, since the generated learning data is generated based on an actual user query, the two may be matched and transmitted to the learning data management unit 542 in the form of [real user query-generated learning data]. If the user query and generated learning data match, the corresponding information is not transmitted. In addition, related information such as a creation date and time, an NMT model, and a translation language may also be transmitted to the learning data management unit 542. Meanwhile, the language used for translation, the translation step, and the like may be automatically adjusted in the learning data management unit 542 based on the actual paraphrase engine learning result.

Next, the learning data management unit 542 may store the learning data generated by the NMT engine management unit 541, progress the learning of the actual paraphrase model, and test and verify (S83). In more detail, the learning data management unit 542 stores the learning data generated by the NMT engine management unit 541 and uses the generated learning data to train a paraphrase model that can be applied to the paraphrase engine. Test and verify the paraphrase model. The paraphrase learning processor illustrated in FIG. 8 actually refers to a processor that learns a paraphrase engine, and may mean a function included in the learning data construction unit 540.

Specifically, the learning data management unit 542 is a learning data generated by the NMT engine management unit 541, generates and trains a paraphrase model under various conditions, conducts quality evaluation based on the established test set, and has achieved excellent results. The language used for the NMT model and the translation is continuously used, and the language used for the NMT model and the translation also serves to manage the quality of learning data, such as reducing or excluding the frequency of use. In addition, the NMT engine management unit 541 may perform a quality evaluation of the corresponding NMT engine through verification using a paraphrase engine to reduce or exclude the weight of the NMT engine that generated low-quality learning data.

More specifically, the learning data management unit 542 may configure the learning data as a pair of an actual user query and a query generated by the NMT.

In addition, the training data management unit 542 classifies the training data generated by the NMT engine management unit 541 into a certain rule, and secures a certain quantity for each step for accurate paraphrase engine training and quality comparison. As an example, based on the same user query, a certain amount of learning data translated into Korean → English → Korean using the Google NMT engine and Korean → English → Korean using the Naver NMT engine are secured, The quantity secured may be the same for each engine.

In addition, the training data management unit 542 trains a parametric model of a certain number or more for each NMT engine use, translation language level, and type, and through a test set that establishes the accuracy of the paraphrase model trained for each engine and language. Compare. At this time, the test set consists of real user queries and test queries that are not trained in the paraphrase model, and is evaluated based on whether the actual user query is correctly derived by putting the test query into the paraphrase model.

In addition, the learning data management unit 542 sums the results of each NMT model and translation language level and type according to the result of the given test set. According to the result, the NMT engine management automatically feedbacks which engine is used more and which translation method is mainly used for each NMT and is reflected in the learning data generation process. At this time, the learning data management unit 542 may determine the amount of future learning data generation based on the performance evaluation. The formula representing the performance evaluation is {(evaluation result)-(basic model performance)}/(basic model performance), and the learning data management unit 542 adjusts the total amount of the training data quantity based on the result of the formula.

For example, if the results shown in [Table 1] below were produced. Google NMT generates 20% more Korean → English → Korean-based learning data, and at the same time creates more Naver NMT Korean → English → Japanese → Korean-based learning data at 20%. Through this, based on the translation language order and quality for each translation engine, better learning data can be automatically generated.

Translation model Google NMT Google NMT Naver NMT Naver NMT Translation language English English->Japanese English English->Japanese Quantity of training data 10,000 10,000 10,000 10,000 Basic model performance 50 50 50 50 Evaluation results 60 50 50 60 Performance evaluation 20% - - 20%

Next, the model distribution unit 543 distributes the deep learning-based paraphrase model learned based on the training data so that it can be actually used and generates an ensemble of the paraphrase model (S84). The model distribution unit 543 bundles a plurality of paraphrase models in order to improve performance when performing an actual operation, and enables them to be used in a paraphrase engine in an ensemble form.

Finally, the learning data construction unit 540 regularly applies the ensemble-type paraphrase engine to the service (S85), so that the latest high-quality engine can be applied to the service at all times.

The embodiment according to the present invention described above may be implemented in the form of a computer program that can be executed through various components on a computer, and such a computer program can be recorded on a computer-readable medium. In this case, the medium may continuously store a program executable on a computer or may be stored for execution or download. In addition, the medium may be various recording means or storage means in the form of a combination of single or several hardware, and is not limited to a medium directly connected to a computer system, but may be distributed on a network. Examples of the medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floptical disks, And program instructions including ROM, RAM, flash memory, and the like. In addition, examples of other media may include an application store for distributing applications or a recording medium or storage medium managed by a site, server, or the like that supplies or distributes various software.

In the above, the present invention has been described by specific matters such as specific components and limited examples and drawings, but it is provided to help a more comprehensive understanding of the present invention, and the present invention is not limited to the above embodiments, Those skilled in the art to which the invention pertains may seek various modifications and changes from these descriptions.

Accordingly, the spirit of the present invention is not limited to the above-described embodiments, and should not be determined, and the scope of the spirit of the present invention as well as the claims to be described later, as well as all ranges that are equivalent to or equivalently changed from the claims Would belong to

Claims (16)

A user query receiving unit that receives a user query from a user terminal;
A query extension unit generating a question template by analyzing a user query and generating a similar question template using a natural language processing and a deep learning model; And
A learning data construction unit that generates learning data for training the query expansion unit;
Artificial intelligence question and answer system comprising a. According to claim 1,
The question template and the similar question template are semantic triple-based question templates consisting of entities, attributes, and instant answers. According to claim 1,
The learning data construction unit,
Artificial intelligence query response system that translates the first sentence in Korean into a specific foreign language, translates the first sentence translated into a specific foreign language back into Korean, acquires the second sentence, and builds the generated second sentence into learning data . According to claim 1,
The query expansion unit,
A natural language expansion module that processes the user query in natural language; And
A paraphrase engine that generates a similar question template through paraphrasing the natural language processed user query; Including, artificial intelligence question and answer system. According to claim 1,
When the user query and the generated question template do not match, the artificial intelligence query response system provides an immediate answer corresponding to the generated question template to the user terminal. A user query receiving step of receiving a user query from a user terminal;
A query expansion step of generating a question template by analyzing a user query and generating a similar question template using a natural language processing and a deep learning model; And
A training data construction step of generating training data for training the second query expansion unit;
Artificial intelligence query answering method comprising a. The method of claim 6,
The question template and the similar question template are semantic triple-based question templates consisting of entities, attributes, and instant answers. The method of claim 6,
The learning data construction step,
Artificial intelligence query response method for translating the first sentence in Korean into a specific foreign language, translating the first sentence translated into a specific foreign language back into Korean, obtaining a second sentence, and constructing the generated second sentence as learning data . The method of claim 6,
The second query expansion step,
Processing the user query in natural language; And
Generating a similar question template through paraphrasing the natural language processed user query; Including, artificial intelligence question and answer method. The method of claim 6,
If the user query and the generated question template do not match, providing an immediate answer corresponding to the generated question template to the user terminal, artificial intelligence query response method. A user query receiving unit that receives a user query from a user terminal;
A query expansion unit generating a similar question template of the user query using a paraphrase engine;
It includes; a learning data construction unit for generating training data for training the query expansion unit using an NMT (Neural Machine Translation) engine.
The learning data construction unit,
Using the NMT engine, the first sentence in Korean is translated into a specific foreign language, the first sentence translated into a specific foreign language is translated back into Korean to obtain a second sentence, and the generated second sentence is constructed as learning data. ,
Artificial intelligence question and answer system. A user query receiving step of receiving a user query from a user terminal;
A query expansion step of generating a similar question template of the user query using a paraphrase engine;
A training data construction step of generating training data for training the query extension using a neural machine translation (NMT) engine; Including,
The learning data construction step,
Using the NMT engine, the first sentence in Korean is translated into a specific foreign language, the first sentence translated into a specific foreign language is translated back into Korean to obtain a second sentence, and the generated second sentence is constructed as learning data. ,
Artificial intelligence question and answer system comprising a. The method of claim 12,
The similar question template is a semantic triple (semantic triple) based question template consisting of an entity (attribute) and attributes (instant answer), artificial intelligence query response system. A user query receiving step of receiving a user query from a user terminal;
A query expansion step of generating a similar question template of the user query using a paraphrase engine;
It includes; learning data construction step of generating training data for training the query expansion unit using an NMT (Neural Machine Translation) engine;
The learning data construction step,
Using the NMT engine, the first sentence in Korean is translated into a specific foreign language, the first sentence translated into a specific foreign language is translated into Korean again to obtain a second sentence, and the generated second sentence is constructed as learning data. ,
Artificial intelligence query answering method comprising a. The method of claim 14,
The similar question template is a semantic triple based question template consisting of an entity, an attribute, and an instant answer. A user query receiving unit that receives a user query from a user terminal;
A query expansion unit generating a similar question template of the user query using a paraphrase engine;
It includes; a learning data construction unit for generating training data for training the query expansion unit using an NMT (Neural Machine Translation) engine.
The learning data construction unit,
An NMT engine management unit that translates and retranslates user log data into the neural network-based NMT engine to generate learning data; And
Learning data for storing learning data generated by the NMT engine management unit, learning a paraphrase model that can be applied to the paraphrase engine using the generated learning data, and testing and verifying the paraphrase model Management; Including, artificial intelligence question and answer system.

Images