KR102261199B1 - Method, system and computer program for artificial intelligence answer - Google Patents

Abstract

According to an embodiment of the present invention, a user query receiving unit for receiving a user query from a user terminal; a first query extension unit that generates a question template by analyzing a user query, and determines whether the user query and the generated question template match; a second query extension unit for generating a similar question template using a natural language processing and deep learning model when the user query and the generated question template do not match; a training data construction unit generating training data for training the second query extension unit using a Neural Machine Translation (NMT) engine; and a query response unit transmitting a user query result derived through the first query expansion unit or the second query expansion unit to the user terminal. There is provided an artificial intelligence question and answer system comprising a.

Description

AI Q&A SYSTEM, METHOD AND COMPUTER PROGRAM {METHOD, SYSTEM AND COMPUTER PROGRAM FOR ARTIFICIAL INTELLIGENCE ANSWER}

The present invention relates to an artificial intelligence question-and-answer system, method, and computer program, and more particularly, to construct and paraphrase learning data using an NMT engine in order to accurately understand natural language-based sentences and provide search results that fit the intention. It relates to an artificial intelligence question-and-answer system, method and computer program learning engine.

In order to implement various application services that deal with data expressed in natural language, it is necessary to understand and engineer linguistic knowledge, structural knowledge for each language, and the complex qualities of a language. Therefore, there are barriers to entry to work such as adding a new language or domain. this exists

In particular, the traditional NLU (Natural Language Understanding) method strongly relies on hand-crafted features. For this reason, it takes a lot of time to extract the features, and there is a limit in that it cannot cope with various cases such as new patterns, typos, and spelling errors.

To solve this problem, a deep learning-based NLU processing method is being considered recently. The deep learning-based NLU method is a method that automatically learns features from data, and has the advantage of being able to process a wider range of contextual information than before. Through this, even new words or typos that have not been learned before are robust compared to traditional rule/statistics-based NLU, and some disadvantages of the existing traditional NLU can be supplemented.

On the other hand, with the expansion of smart machines led by voice recognition speakers and the development of artificial intelligence technology, the information retrieval method has changed from the existing search method based on keyword input and checking the document list to natural language based sentence input and specific response form. Trends in search are changing.

KR 10-1851787 B1

An object of the present invention is to accurately understand natural language-based sentences and provide search results that match the intention.

Another object of the present invention is to increase the accuracy of a search by constructing learning data using an NMT engine and learning a paraphrase engine.

According to an embodiment of the present invention, a user query receiving unit for receiving a user query from a user terminal; a first query extension unit that generates a question template by analyzing a user query, and determines whether the user query and the generated question template match; a second query extension unit for generating a similar question template using a natural language processing and deep learning model when the user query and the generated question template do not match; a training data construction unit generating training data for training the second query extension unit using a Neural Machine Translation (NMT) engine; and a query response unit transmitting a user query result derived through the first query expansion unit or the second query expansion unit to the user terminal. There is provided an artificial intelligence question and answer system comprising a.

In the present invention, the question template and the similar question template may be a semantic triple-based question template consisting of an entity, an attribute, and an instant answer.

In the present invention, the learning data construction unit, using the NMT engine to translate a first sentence in Korean into a specific foreign language, and to obtain a second sentence by translating the first sentence translated into the specific foreign language back into Korean, The generated second sentence may be constructed as learning data.

In the present invention, the second query expansion unit includes: a natural language expansion module for natural language processing of the user query; and

a paraphrasing engine that generates a similar question template through paraphrasing of the natural language-processed user query; may include.

In the present invention, when the user query and the generated question template do not match, an immediate answer corresponding to the generated question template may be provided to the user terminal.

According to an embodiment of the present invention, a user query receiving step of receiving a user query from a user terminal; a first query expansion step of generating a question template by analyzing a user query, and determining whether the user query and the generated question template match; a second query expansion step of generating a similar question template using a natural language processing and deep learning model when the user query and the generated question template do not match; a training data construction step of generating training data for training the second query extension unit using a Neural Machine Translation (NMT) engine; and a query response step of delivering a user query result derived through the first query expansion step or the second query expansion step to the user terminal; There is provided an artificial intelligence question and answer method comprising a.

In the present invention, the question template and the similar question template may be a semantic triple-based question template consisting of an entity, an attribute, and an instant answer.

In the present invention, the step of constructing the learning data includes translating a first sentence in Korean into a specific foreign language using the NMT engine, and translating the first sentence translated into a specific foreign language back into Korean to obtain a second sentence, , the generated second sentence may be constructed as learning data.

In the present invention, the second query expansion step comprises: a natural language expansion module for natural language processing of the user query; and a paraphrasing engine that generates a similar question template by paraphrasing the natural language-processed user query. may include.

In the present invention, when the user query and the generated question template do not match, an immediate answer corresponding to the generated question template may be provided to the user terminal.

According to an embodiment of the present invention, a user query receiving unit for receiving a user query from a user terminal; a query extension unit for generating a similar question template of the user query using a paraphrase engine; and a training data construction unit that generates training data for training the query expansion unit using a Neural Machine Translation (NMT) engine, wherein the training data construction unit includes a first Korean language using the NMT engine. An artificial intelligence question and answer system is provided that translates a sentence into a specific foreign language, translates a first sentence translated into a specific foreign language back into Korean, obtains a second sentence, and builds the generated second sentence as learning data.

According to an embodiment of the present invention, a user query receiving unit for receiving a user query from a user terminal; a query extension unit for generating a similar question template of the user query using a paraphrase engine; and a training data construction unit that generates training data for training the query expansion unit using a Neural Machine Translation (NMT) engine, wherein the training data construction unit includes a first Korean language using the NMT engine. An artificial intelligence question and answer system is provided that translates a sentence into a specific foreign language, translates a first sentence translated into a specific foreign language back into Korean, obtains a second sentence, and builds the generated second sentence as learning data.

In the present invention, the similar question template may be a semantic triple-based question template consisting of an entity, an attribute, and an instant answer.

According to an embodiment of the present invention, a user query receiving step of receiving a user query from a user terminal; a query expansion step of generating a similar question template of the user query using a paraphrase engine; and a training data construction step of generating training data for training the query expansion unit using a Neural Machine Translation (NMT) engine, wherein the training data construction step is performed in Korean using the NMT engine. It may be an artificial intelligence question answering method in which one sentence is translated into a specific foreign language, the first sentence translated into the specific foreign language is translated back into Korean to obtain a second sentence, and the generated second sentence is constructed as learning data. .

In the present invention, the similar question template may be a semantic triple-based question template consisting of an entity, an attribute, and an instant answer.

According to an embodiment of the present invention, a user query receiving unit for receiving a user query from a user terminal; a query extension unit for generating a similar question template of the user query using a paraphrase engine; and a training data construction unit that generates training data for training the query expansion unit using a Neural Machine Translation (NMT) engine, wherein the training data construction unit includes, in the neural network-based NMT engine, user log data NMT engine management unit for generating learning data by translating and re-translating; and learning to store the learning data generated by the NMT engine management unit, and to learn a paraphrase model that can be applied to the paraphrase engine using the generated learning data, and to test and verify the paraphrase model. data management department; may include.

According to the present invention, it is possible to accurately understand a natural language-based sentence and provide a search result suitable for an intention.

According to the present invention, it is possible to automatically construct a large amount of accurate learning data using the existing NMT engine.

1 is a diagram illustrating an example of a network environment according to an embodiment of the present invention.
2 is a block diagram illustrating the internal configuration of a user terminal and a server according to an embodiment of the present invention.
3 is for explaining a search result based on a semantic triple.
4 shows an example of performing a search based on a semantic triple.
5 shows an internal configuration of a processor according to an embodiment of the present invention.
6 is a time-series diagram illustrating an artificial intelligence question answering method according to an embodiment of the present invention.
7 is a diagram illustrating the overall structure of an artificial intelligence question and answer system according to an embodiment of the present invention.
8 is a diagram for explaining the construction of training data and a paraphrase model according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS [0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS [0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS [0023] Reference is made to the accompanying drawings, which show by way of illustration specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the present invention. It should be understood that the various embodiments of the present invention are different but need not be mutually exclusive. For example, certain shapes, structures, and characteristics described herein may be implemented with changes from one embodiment to another without departing from the spirit and scope of the present invention. In addition, it should be understood that the location or arrangement of individual components within each embodiment may be changed without departing from the spirit and scope of the present invention. Accordingly, the following detailed description is not to be taken in a limiting sense, and the scope of the present invention should be taken as encompassing the scope of the claims and all equivalents thereto. In the drawings, like reference numerals refer to the same or similar elements throughout the various aspects.

1 is a diagram illustrating an example of a network environment according to an embodiment of the present invention.

The network environment of FIG. 1 shows an example including a plurality of user terminals 110 , 120 , 130 , 140 , a server 150 , and a network 160 . 1 is an example for the description of the invention, and the number of user terminals or the number of servers is not limited as in FIG. 1 .

The plurality of user terminals 110 , 120 , 130 , and 140 may be a fixed terminal implemented as a computer device or a mobile terminal. Examples of the plurality of user terminals 110 , 120 , 130 , and 140 include a smart phone, a mobile phone, a navigation device, a computer, a notebook computer, a digital broadcasting terminal, a personal digital assistant (PDA), and a portable multimedia player (PMP). ), and tablet PCs. For example, the user terminal 11 10 may communicate with other user terminals 120 , 130 , 140 and/or the server 150 through the network 160 using a wireless or wired communication method.

The communication method is not limited, and not only a communication method using a communication network (eg, a mobile communication network, a wired Internet, a wireless Internet, a broadcasting network) that the network 160 may include, but also short-range wireless communication between devices may be included. For example, the network 160 may include a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), and a broadband network (BBN). , the Internet, and the like. In addition, the network 160 may include any one or more of a network topology including a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree or a hierarchical network, etc. not limited

The server 150 is a computer device or a plurality of computer devices that communicates with a plurality of user terminals 110 , 120 , 130 , 140 and the network 160 to provide commands, codes, files, contents, services, etc. can be implemented.

For example, the server 150 may provide a file for installing an application to the user terminal 1110 connected through the network 160 . In this case, the user terminal 1110 may install the application using the file provided from the server 150 . In addition, by accessing the server 150 under the control of an operating system (OS) and at least one program (eg, a browser or an installed application) included in the user terminal 1110, the service provided by the server 150 or content can be provided. For example, when the user terminal 1 110 transmits content viewing to the server 150 through the network 160 under the control of the application, the server 150 provides a unique instant response using a semantic triple-based knowledge expansion system. may be transmitted to the user terminal 1110, and the user terminal 1110 may display a unique instant response according to the control of the application. As another example, the server 150 may establish a communication session for data transmission/reception, and route data transmission/reception between the plurality of user terminals 110 , 120 , 130 , and 140 through the established communication session.

2 is a block diagram illustrating the internal configuration of a user terminal and a server according to an embodiment of the present invention.

In FIG. 2 , the internal configuration of the user terminal 1110 as an example of one user terminal and the server 150 as an example of one server will be described. Other user terminals 120 , 130 , and 140 may also have the same or similar internal configuration.

The user terminal 1110 and the server 150 may include memories 211 and 221 , processors 212 and 222 , communication modules 213 and 223 , and input/output interfaces 214 and 224 . The memories 211 and 221 are computer-readable recording media and may include random access memory (RAM), read only memory (ROM), and permanent mass storage devices such as disk drives. In addition, an operating system and at least one program code (eg, a code for a browser installed and driven in the user terminal 1110 and the above-described application) may be stored in the memories 211 and 221 . These software components may be loaded from a computer-readable recording medium separate from the memories 211 and 221 using a drive mechanism. The separate computer-readable recording medium may include a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD/CD-ROM drive, and a memory card. In another embodiment, the software components may be loaded into the memories 211 and 221 through the communication modules 213 and 223 instead of a computer-readable recording medium. For example, the at least one program is a program installed by files provided through the network 160 by a file distribution system (eg, the above-described server 150 ) that distributes the installation files of developers or applications (eg, the program installed). may be loaded into the memories 211 and 221 based on the above-described application).

The processors 212 and 222 may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input/output operations. The instructions may be provided to the processors 212 and 222 by the memories 211 and 221 or the communication modules 213 and 223 . For example, the processors 212 and 222 may be configured to execute received instructions according to program codes stored in a recording device such as the memories 211 and 221 .

The communication modules 213 and 223 may provide a function for the user terminal 11 10 and the server 150 to communicate with each other through the network 160, and another user terminal (eg, the user terminal 2 120). Alternatively, a function for communicating with another server (eg, server 150 ) may be provided. For example, a request generated by the processor 212 of the user terminal 1110 according to a program code stored in a recording device such as the memory 211 is transmitted to the server ( 150) can be transferred. Conversely, a control signal, command, content, file, etc. provided under the control of the processor 222 of the server 150 passes through the communication module 223 and the network 160 to the communication module ( 213 may be received by the user terminal 1110 . For example, a control signal or command of the server 150 received through the communication module 213 may be transmitted to the processor 212 or the memory 211 , and contents or files may be transmitted to the user terminal 1110 by the user terminal 1110 . It may be stored as a storage medium that may further include.

The input/output interfaces 214 and 224 may be means for interfacing with the input/output device 215 . For example, the input device may include a device such as a keyboard or mouse, and the output device may include a device such as a display for displaying a communication session of an application. As another example, the input/output interface 214 may be a means for an interface with a device in which functions for input and output are integrated into one, such as a touch screen. As a more specific example, the processor 212 of the user terminal 1110 is configured using data provided by the server 150 or the user terminal 2 120 in processing the command of the computer program loaded in the memory 211 . A service screen or content to be used may be displayed on the display through the input/output interface 214 .

Also, in other embodiments, the user terminal 1110 and the server 150 may include more components than those of FIG. 2 . However, there is no need to clearly show most of the prior art components. For example, the user terminal 1110 is implemented to include at least a part of the above-described input/output device 215 or other components such as a transceiver, a global positioning system (GPS) module, a camera, various sensors, and a database. It may include more elements.

The artificial intelligence question answering method and system of the present invention may be implemented by the server 150 , and in more detail, the artificial intelligence question answering method may be implemented by a command processed by the processor 222 of the server 150 .

The problem to be solved by the present invention is to construct a system capable of answering various queries from users by providing an artificial intelligence question answering method, apparatus, and program.

Due to the rise of AI-based smart machines led by artificial intelligence speakers, QA (Question Answering) search, which is different from existing portal search, is emerging, and information search and input means are voice, not touch or keyword input. , unlike the keyword-based search of the existing portal search, the need to understand natural language-based sentences has increased.

For this reason, in order to accurately understand natural language-based sentences and deliver search results that fit the intent, there is a need for an AI-based question-and-answer method, device and program that can cope with various cases such as new patterns, typos, and spelling errors.

In addition to the above-described QA search, the present invention has characteristics applicable to all systems that identify user intentions and provide desired results, and thus can be applied in various forms. For example, if the slot filling method is used instead of the instant answer provided as an answer in knowledge QA, it can be applied as an API that provides a specific function according to the user's intention. Through this, the present invention can be said to be a technology with a high application range that can be used in various fields such as home IoT, smart toy/home robot, and connected car. Therefore, in the following specification, the QA search method will be mainly described, but the present invention is not necessarily limited thereto, and can be applied to all applicable systems.

First, before describing the present invention in earnest, let's look at the differences between the artificial intelligence question answering method of the present invention and the existing search engine. The artificial intelligence question and answer system according to an embodiment of the present invention may provide a unique instant answer by using a question template based on a semantic triple. The AI query answering method of the present invention may be different from the existing search engines in that the search results are not provided in the form of a document, but in the form of a unique instant answer, that is, an immediate answer. In addition, the artificial intelligence question and answer method of the present invention can construct learning data for providing a semantic triple-based search result.

3 is for explaining a search result based on a semantic triple.

Referring to FIG. 3 , an existing search engine (As-Is, Searh) uses a keyword as an input method, provides a document list as a search result, and a search platform operates on a PC or mobile device.

In contrast, in the artificial intelligence question-answering method (To-Be, Question-Answering) of the present invention, the input method is a natural language-based sentence, and a specific response, that is, an instant unique response, can be provided as a search result, and the platform is PC Alternatively, it may be implemented anywhere without being limited to mobile.

In more detail, the artificial intelligence question and answer method of the present invention enables a natural language-based sentence to be input as opposed to a keyword input by an existing search engine, so that a user can naturally search for information as if asking a human question. In addition, the artificial intelligence question and answer method of the present invention provides a specific answer as a search result, thereby reducing the inconvenience of a user having to find a search result directly in a document list provided by an existing search engine and providing an optimal search result. have. In addition, the artificial intelligence question and answer method of the present invention is not limited to PC or mobile as a platform, but has the advantage of being able to immediately search for information anywhere based on a smart machine.

4 shows an example of performing a search based on a semantic triple.

The knowledge DB 400 shown in FIG. 4 is a special type of knowledge base database that stores data in the form of a semantic triple that simulates queries of real users, and a unique instant answer without a separate reasoning process. ) can be searched. The knowledge DB 400 has the form of entity (432)-attribute (434)-instant answer (438). In an embodiment of the present invention to be described later, the knowledge DB 400 may be a database that exists inside or outside the server 150 and can communicate with the processor 222 to provide data.

4 shows that when a user query 410 of “What is the height of Mt. Baekdu?” is received, the user query is analyzed (420), key words such as 'Mt. Baekdu' and 'height' are extracted, and then Mt. Baekdu is a target to ask. The height can be analyzed to the intent of the question. Accordingly, the data of entity = “Mt. Baekdu” and attribute = “height” is searched, the instant answer of the corresponding item is determined as the result value, and the corresponding answer 2,744m is provided to the user (450). The knowledge DB 400 as described above may provide an optimal answer without a separate reasoning process to search for an optimal answer. Hereinafter, the artificial intelligence question answering method and system of the present invention based on the semantic triple as described with reference to FIGS. 3 and 4 will be described in more detail.

5 shows an internal configuration of a processor according to an embodiment of the present invention.

The processor 212 may include a web browser or an application capable of outputting a web page provided online. As shown in FIG. 3 , the configuration of a semantic triple-based knowledge expansion system according to an embodiment of the present invention in the processor 212 includes a query receiver 510 , a first query expansion 520 , and a second query expansion. It may include a unit 530 , a learning data building unit 540 , and a question answering unit 550 . In addition, the second query expansion unit 530 includes a natural language expansion module 531 and a paraphrase engine 532 , and the learning data construction unit 540 includes an NMT engine management unit 541 , a learning data management unit 542 and A model distribution unit 543 may be included. According to an embodiment of the present invention, components of the processor 212 may be selectively included in or excluded from the processor 212 . In addition, according to an embodiment, the components of the processor 212 may be separated or combined to express the functions of the processor 212 .

Here, the components of the processor 212 are the processor 212 according to a command provided by the program code stored in the user terminal 1110 (eg, a command provided by a web browser driven in the user terminal 1110 ). may be representations of different functions of the processor 212 performed by

The processor 212 and the components of the processor 212 may control the user terminal 11 10 to perform the steps S1 to S5 included in the artificial intelligence question answering method of FIG. 4 . For example, the processor 212 and the components of the processor 212 may be implemented to execute instructions according to the code of the operating system included in the memory 211 and the code of at least one program.

6 is a view showing an artificial intelligence question answering method according to an embodiment of the present invention in time series, and FIGS. 7A and 7B are diagrams showing the overall structure of an artificial intelligence question answering system according to an embodiment of the present invention. In more detail, FIG. 7A illustrates the operation of the question answering system when user query analysis is successful, and FIG. 7B illustrates the operation of the question answering system when user query parsing is not successful. Hereinafter, the present invention will be described with reference to FIGS. 6, 7A and 7B together.

First, the query receiving unit 510 receives a user's query from the user terminal 110 (S61). In more detail, the query receiving unit 510 receives a user query ( S71 ), and transmits the received query to the first query expansion unit 520 ( S72 ). The user's query form may be received in various forms such as voice and text. The query receiver 510 may convert the received user's query into a suitable form through an appropriate conversion process.

According to an embodiment of the present invention, a user may input a natural language-based query to the user terminal 100, which is an AI-based smart machine to which the artificial intelligence question-answering method, apparatus, and program of the present invention are applied. The smart machine may include not only existing smartphones and computers, but also artificial intelligence speakers, connected cars, home IoT, AI appliances, personal assistants, home robots/smart toys, chatbot applications, and intranets.

Next, the first query extension unit 520 analyzes the user query and checks whether an entity and an attribute are recognizable ( S62 ). If the entity and attribute are recognizable as a result of the user query analysis, the first query extension is performed by creating a question template, and it is checked whether the user query and the generated question template match (S64). In more detail, the first query expansion unit 520 analyzes the user's query, checks answer information to the query from the knowledge DB 400 , finds answer information in the knowledge DB 400 , and finds a plurality of answers similar to the query. A first search is performed by creating a question template and comparing whether the user query and the question template match.

In more detail, the first query extension 520 may generate a question template corresponding to the user query based on a result of analyzing the user query. When a user's query is received, the first query expansion unit 520 analyzes the user's query, generates a question template based on the analysis result, and performs the first query expansion. In this case, the first query extension 520 may generate a question template from the above-described knowledge DB 590 . Hereinafter, a detailed configuration in which the first query extension unit 520 generates a question template will be described.

First, the first query expansion unit 520 may analyze the user query using a Natural Language Processing (NLP) engine in order to analyze the user query in the above-described semantic triple format. At this time, the entity and attribute are found by using techniques such as morphological analysis in the user query. Since normal users ask questions in the form of entity + attribute, they sequentially analyze sentences to find entity and attribute candidates in the user query.

In an embodiment of the present invention, when analyzing a user query, when there is only one entity and attribute as reference, the first query extension unit 520 generates content corresponding to the corresponding entity and attribute as a question template. For example, 'Which of the Overwatch heroes is mainly active in London?' In the case of , if the entity is analyzed as an 'Overwatch hero' and no attribute is detected, a question template can be created using all attributes that have 'Overwatch hero' as an entity in the knowledge DB.

Next, the first query extension unit 520 may generate a question template from the knowledge DB 400 based on the query word analysis result. Specifically, the first query expansion unit 520 searches for entity and attribute candidates in a search form for each category matching the user query, and generates a question template based on the entity synonym and attribute synonym. In this case, the question template may have not only entities and attributes, but also instant answers as additional information. This may provide an instant answer to the user when a user query, which will be described later, matches the question template created by the first query extension unit 520 ( S65 ). If an instant answer exists, the corresponding question template may be generated as training data for training the second query expansion step to be described later (S68).

Next, the first query extension unit 520 compares the generated question template with the user query, and checks the result of whether they match ( S73 ). If it is determined that the question template generated by the first query extension 520 is the same as the user query, an instant answer previously generated from the question template is provided as an answer (S65). The first query extension unit 520 may compare the user query with a question template generated by removing meaningless characters or words from the user query. For example, 'What is the elevation of Mt. Geumgang?' ',' 'of' and 'is' that are not related to semantic expression can be removed through natural language processing. It is determined whether the user query provided with meaningless characters or words exactly matches the generated question template. If it is determined that the generated question template and the user query are not the same, the user query may be expanded using the second query extension unit 530 .

Next, when the generated question template and the user query do not match, the second query extension unit 530 generates a similar question template. In more detail, the second query extension unit 530 generates a semantic-like question template by utilizing a natural language processing and deep learning model, thereby expanding the question template second and comparing the results (S66). That is, when there is no result that matches the user query with the question template generated by the first query extension 520 , the question template previously created using the second query extension 530 , which is a similar query engine, is used. Based on the query expansion, a similar question template may be additionally generated (S74).

Alternatively, if it is determined in step S72 that the user query analysis result of the first query expansion unit 520 cannot recognize the entity and attribute, the second query expansion unit 530 converts the derived user query result into a similar question template. to expand the query and compare the results (S63). That is, the results of the user query and the pre-generated question template are checked by using the similar query engine (S78). That is, if neither entity nor attribute is found during user query analysis, the search is performed by comparing the similarity between the user query and the system log answer in the paraphrase engine of the second query extension based on the answers from the existing system log. This can be done, and the corresponding steps are steps S63 and S78.

A method of generating a similar question template of the second query extension unit 530 uses a method of generating a semantic similar question using natural language processing and a deep learning model, so that the result can be compared by extending the semantic similar question template. 3 , the second query expansion unit 530 may include a natural language expansion module 531 and a paraphrase engine 532 . That is, the second query expansion unit 530 expands the predicate through the natural language expansion module 531, and recognizes the similarity between the user query and the question template through the deep learning-based paraphrase engine 532 to respond to the user's intention. I can give you the right answer.

First, the natural language extension module 531 of the second query extension unit 530 may generate a similar question template by providing various patterns for a query on a specific topic using a natural language processing method. In one embodiment, 'Where is [Person]'s birthplace?' The question is, 'Where was [Person] born?', 'Where was [Person] born?' have the same meaning as The second query extension unit 530 may generate a similar question template by extending the question template of the entity-attribute combination created from the query template through a natural language processing method as in the above example.

In more detail, the natural language processing method of the natural language extension module 531 may be implemented by using a separately constructed similarity DB to expand it in a pattern format according to a specific attribute. For example, in the case of the 'Birthplace' attribute, the second query expansion unit 530 may include 'Where were you born?', 'Where were you born?', and 'Where are you born?' A similar question template can be created by extending the question template through various predicate extensions such as

In addition, the paraphrase engine 532 of the second query extension unit 530 generates a semantic similarity question template of the user query through paraphrasing. Also, the paraphrase engine 532 compares the user query with the similar question template and confirms the result ( S75 ). In more detail, the second query extension unit 530 compares the similarity between the user query, the question template, and the similar question template through the deep learning-based paraphrase engine by combining the previously created question template and the extended similar question template. The similarity comparison is performed in two steps, and the details are as follows.

As a first step, first, the paraphrase engine 532 measures the similarity between the user query and the existing question templates and similar question templates generated by the first query extension unit 520, and ranks higher in the total number of question templates and similar question templates. N candidates are selected. In this case, the number of the top N items may be changed according to the admin page or a statistics-based feedback program. The paraphrase engine 532 compares the user's query with the existing question templates and Top N similar question templates, and returns one question template considered to be most similar and the degree of similarity.

In a second step, the paraphrase engine 532 determines whether the selected Top 1 question template is ultimately the same as the user query based on the similarity. The decision criteria are initially randomly selected by the manager, but can be adjusted based on statistics through feedback on actual results. For example, even if the initial similarity is selected as 90%, the answer is automatically changed by changing the similarity from 90% to 85% You can expand your coverage. If the degree of similarity is less than a predetermined reference value, a message indicating that there is no search result may be output.

On the other hand, when the above-described step is completed, the second query expansion unit 530 provides a system including the user query, including the top 1 question templates, the top N question templates, the search time and steps, and the search speed. Information can be stored in a separate DB.

Next, the query response unit 550 transmits the user query result derived through the first query expansion unit 520 and the second query expansion unit 530 to the user terminal 110 (S65). The user query result is transmitted to the AI-based smart machine and transmitted to an interface suitable for the characteristics, along with the 'user query'. 'Answered or not'. Detailed information such as 'time' and 'device' is stored as a system log and can be used for paraphrase model management in the future.

Next, the query response unit 550 transmits the derived user query result to the user terminal (S67).

Also, the training data construction unit 540 according to an embodiment of the present invention converts training data for training the second query extension unit 530 using the derived user query result to NMT (Neural Machine Translation). ) can be generated using the engine (S68). That is, the second query extension unit 530 may perform learning using the learning data generated by the learning data construction unit 540 . In particular, the paraphrase engine 532 uses a method of using a semantic similarity question using a deep learning model, and for this, the deep learning needs to be trained with rich learning data. Accordingly, the learning data construction unit 540 may perform training of the paraphrase engine, and may generate training data for learning.

In more detail, according to an embodiment of the present invention, in the case of the deep learning-based paraphrase engine 532, the operator's intervention is minimized by making a model through learning without specifying the structure of the model in advance, and a complex and deep structure is created. It is characterized by higher accuracy than the existing method. However, there is a problem that a large amount of learning data of tens of thousands or more is required to produce performance that can replace human work. Accordingly, according to an embodiment of the present invention, an artificial intelligence question answering apparatus provides a method of automatically constructing learning data.

First, the training data construction unit 540 generates training data for paraphrasing. In more detail, the training data building unit 540 constructs training data capable of learning the AI response method of the present invention so that the above-described second query expansion unit 530 can form a semantic similarity question template. To this end, the training data construction unit 540 may include an NMT engine management unit 541 , a training data management unit 542 , and a model distribution unit 543 .

The second query expansion unit 330 is an NMT engine management unit 541 for using a plurality of NMT (Neural Machine Translation) engines to construct learning data in order to continuously secure the quality of the built paraphrase engine in the present invention. and may include a learning data management unit 542 that uses a separate statistical management program for managing the translation quality of the corresponding NMT engine. In addition, the learning data construction and quality management including a series of processes, including a model distribution unit 543 that conducts learning with the generated learning data, distributes and applies a paraphrase model, and quality control as a whole is the configuration of the present invention. can be

8 is a view for explaining the construction of training data and a paraphrase model according to an embodiment of the present invention.

First, the learning data construction unit 540 sets the user's actual query as raw data in order to create the learning data, and transmits the actual query-based user log data to the NMT engine management unit 541 (S81). In this case, the actual query can use the data stored in the log DB of the server. The learning data construction unit 540 transmits the corresponding data to the NMT engine management unit 541, and prepares for generating the learning data.

Next, the NMT engine management unit 541 uses a plurality of neural network-based external NMT engines to translate user log data or user queries into other languages and then re-translates them into Korean to generate learning data (S82). . According to an embodiment of the present invention, the NMT engine management unit 541 may obtain a second sentence by translating a first sentence written in Korean into a specific foreign language, and re-translating the first sentence translated into a specific foreign language into Korean. have. That is, the NMT engine management unit 541 may utilize the NMT engine so that the paraphrase engine 532 may collect natural language expressions similar to these for the same query or sentence as learning data.

The NMT engine used in the present invention performs translation in a neural network method through learning, not based on preset patterns and rules, so that when the first sentence is translated into a foreign language and then translated back into Korean, the same meaning as or similar to the first sentence It is possible to obtain natural language sentences with different expressions. In addition, since the external NMT translation engine uses different neural network rules and different learning data, when the same sentence is translated into a specific foreign language and then translated back to Korean, natural language sentences with similar meaning but different expressions are additionally obtained can do.

In addition, when a first Korean sentence is translated into a first foreign language, the first foreign language is again translated into a second foreign language, and the second foreign language is translated into Korean, natural language sentences with similar meanings but expressed in different expressions are used. Additional can be obtained.

The learning data formed in the above-described manner is transmitted to the learning data management unit 542 together with the actual user query and used external NMT information, the translation step, the language used for the translation, and the like. Specifically, since the generated learning data is generated based on an actual user query, it may be transmitted to the learning data management unit 542 in the form of [real user query - generated learning data] by matching both. If the user query and the generated learning data match, the corresponding information is not transmitted. In addition, related information such as a creation date and time, an NMT model, and a translation language may also be transmitted to the learning data management unit 542 . Meanwhile, a language used for translation, a translation step, and the like may be automatically adjusted based on the actual paraphrase engine learning result in the learning data management unit 542 .

Next, the learning data management unit 542 may store the training data generated by the NMT engine management unit 541 , and may perform training of the actual paraphrase model, test and verify ( S83 ). In more detail, the learning data management unit 542 stores the learning data generated by the NMT engine management unit 541, and uses the generated learning data to learn a paraphrase model that can be applied to the paraphrase engine, and the Test and validate the paraphrase model. The paraphrase learning processor shown in FIG. 8 may mean a processor that actually learns a paraphrase engine, and may mean a function included in the learning data building unit 540 .

Specifically, the learning data management unit 542 generates and trains a paraphrase model under various conditions with the learning data generated by the NMT engine management unit 541, and performs quality evaluation based on a pre-established test set to achieve excellent results. The language used for the NMT model and translation is continuously used, and the language used for the non-NMT model and translation also plays a role in managing the quality of the learning data, such as reducing the frequency of use or excluding it altogether. In addition, the NMT engine management unit 541 may reduce or exclude the weight of the NMT engine that generated low-quality learning data by performing quality evaluation of the corresponding NMT engine through verification using the paraphrase engine.

More specifically, the learning data management unit 542 may configure the learning data by pairing an actual user query and a query generated in NMT.

In addition, the learning data management unit 542 classifies the learning data generated by the NMT engine management unit 541 into a certain rule, and secures a certain quantity at each stage for accurate paraphrase engine training and quality comparison. For example, based on the same user query, the amount of learning data translated into Korean → English → Korean using the Google NMT engine and the learning data translated into Korean → English → Korean using the Naver NMT engine are constantly secured. The secured quantity may be the same for each engine.

In addition, the learning data management unit 542 trains a certain number of paraphrase models for each NMT engine use, each translation language level and type, and establishes the accuracy of the paraphrase model trained for each engine and language through a pre-established test set. Compare. At this time, the test set consists of real user queries and test queries that are not trained in the paraphrase model, and evaluates based on whether the actual user queries are correctly derived by putting the test queries into the paraphrase model.

In addition, the learning data management unit 542 sums up the results of each NMT model and the translation language stage and type according to the result of the given test set. According to the result, the NMT engine management automatically feeds back which engine is used more and which translation method is mainly used for each NMT, and it is reflected in the learning data generation process. In this case, the learning data management unit 542 may determine the quantity of future learning data generated based on the performance evaluation. The equation representing the performance evaluation is {(evaluation result)-(basic model performance)}/(basic model performance), and the learning data management unit 542 adjusts the total amount of the training data quantity based on the result of the equation.

As an example, if the results shown in [Table 1] below were obtained. It generates 20% more learning data based on Google NMT’s Korean → English → Korean language, and at the same time generates 20% more learning data based on Naver NMT’s Korean → English → Japanese → Korean language. Through this, better learning data can be automatically generated based on the order and quality of the translation languages for each translation engine.

translation model Google NMT Google NMT Naver NMT Naver NMT translation language English English -> Japanese English English -> Japanese training data quantity 10,000 10,000 10,000 10,000 Base model performance 50 50 50 50 Evaluation results 60 50 50 60 performance evaluation 20% - - 20%

Next, the model distribution unit 543 distributes the deep learning-based paraphrase model learned based on the training data so that it can be actually used and creates an ensemble of the paraphrase model (S84). The model distribution unit 543 bundles a plurality of paraphrase models in order to improve performance when performing an actual task, so that they can be used in the paraphrase engine in an ensemble form.

Finally, the learning data construction unit 540 regularly applies the ensemble-type paraphrase engine to the service (S85), so that the latest high-quality engine can always be applied to the service.

The embodiment according to the present invention described above may be implemented in the form of a computer program that can be executed through various components on a computer, and such a computer program may be recorded in a computer-readable medium. In this case, the medium may continue to store a program executable by a computer, or may be stored for execution or download. In addition, the medium may be various recording means or storage means in the form of a single or several hardware combined, it is not limited to a medium directly connected to any computer system, and may exist distributed on a network. Examples of the medium include a hard disk, a magnetic medium such as a floppy disk and a magnetic tape, an optical recording medium such as CD-ROM and DVD, a magneto-optical medium such as a floppy disk, and those configured to store program instructions, including ROM, RAM, flash memory, and the like. In addition, examples of other media may include recording media or storage media managed by an app store that distributes applications, sites that supply or distribute various other software, and servers.

In the above, the present invention has been described with reference to specific matters such as specific components and limited embodiments and drawings, but these are provided to help a more general understanding of the present invention, and the present invention is not limited to the above embodiments, and the present invention is not limited to the present invention. Those of ordinary skill in the art to which the invention pertains can make various modifications and changes from these descriptions.

Therefore, the spirit of the present invention should not be limited to the above-described embodiments, and the scope of the spirit of the present invention is not limited to the scope of the scope of the present invention. will be said to belong to

Claims (16)

a user query receiver for receiving a user query from the user terminal;
A question template is generated by analyzing a user query, a similar question template is generated using a natural language processing and deep learning model, a template having a high degree of similarity to the user query among the question template and the similar question template is determined, and the determined template and a query expansion unit for determining whether the user query has the same meaning; and
a training data construction unit generating training data for training the query extension unit;
Artificial intelligence question and answer system comprising a. The method of claim 1,
wherein the question template and the similar question template are a semantic triple-based question template comprising an entity, an attribute, and an instant answer. The method of claim 1,
The learning data construction unit,
An artificial intelligence question and answer system that translates a first sentence in Korean into a specific foreign language, translates the first sentence translated into a specific foreign language back into Korean to obtain a second sentence, and builds the generated second sentence as learning data . The method of claim 1,
The query extension is
a natural language extension module for natural language processing of the user query; and
a paraphrasing engine that generates a similar question template by paraphrasing the natural language-processed user query; Including, artificial intelligence question and answer system. The method of claim 1,
When the user query and the generated question template do not match, providing an immediate answer corresponding to the generated question template to the user terminal. a user query receiving step of receiving a user query from a user terminal;
A question template is generated by analyzing a user query, a similar question template is generated using a natural language processing and deep learning model, a template having a high degree of similarity to the user query among the question template and the similar question template is determined, and the determined template and a query expansion step of determining whether the user query has the same meaning; and
a training data construction step of generating training data for training the deep learning model;
Artificial intelligence question and answer method comprising a. 7. The method of claim 6,
wherein the question template and the similar question template are a semantic triple-based question template comprising an entity, an attribute, and an instant answer. 7. The method of claim 6,
The step of constructing the learning data is,
Artificial intelligence question answering method, in which a first sentence in Korean is translated into a specific foreign language, the first sentence translated into a specific foreign language is translated back into Korean to obtain a second sentence, and the generated second sentence is constructed as learning data . 7. The method of claim 6,
The query expansion step is
natural language processing of the user query; and
generating a similar question template by paraphrasing the natural language-processed user query; Including, artificial intelligence question and answer method. 7. The method of claim 6,
When the user query and the generated question template do not match, providing an immediate answer corresponding to the generated question template to the user terminal. a user query receiver for receiving a user query from the user terminal;
a query extension unit for generating a similar question template of the user query using a paraphrase engine;
a training data construction unit that generates training data for training the query extension unit using a Neural Machine Translation (NMT) engine;
The learning data construction unit,
Translating a first sentence in Korean into a specific foreign language using the NMT engine, obtaining a second sentence by translating the first sentence translated into a specific foreign language back into Korean, and constructing the generated second sentence as learning data ,
Artificial intelligence question and answer system. a user query receiving step of receiving a user query from a user terminal;
a query expansion step of generating a similar question template of the user query using a paraphrase engine;
a training data construction step of generating training data for training the query extension unit using a Neural Machine Translation (NMT) engine; including,
The step of constructing the learning data is,
Translating a first sentence in Korean into a specific foreign language using the NMT engine, obtaining a second sentence by translating the first sentence translated into a specific foreign language back into Korean, and constructing the generated second sentence as learning data ,
Artificial intelligence question and answer system comprising a. 13. The method of claim 12,
The similar question template is a question template based on a semantic triple consisting of an entity, an attribute, and an instant answer. a user query receiving step of receiving a user query from a user terminal;
a query expansion step of generating a similar question template of the user query using a paraphrase engine;
a training data construction step of generating training data for training the query extension unit using a Neural Machine Translation (NMT) engine;
The step of constructing the learning data is,
Translating a first sentence in Korean into a specific foreign language using the NMT engine, obtaining a second sentence by translating the first sentence translated into a specific foreign language back into Korean, and constructing the generated second sentence as learning data ,
Artificial intelligence question and answer method comprising a. 15. The method of claim 14,
The similar question template is a question template based on a semantic triple consisting of an entity, an attribute, and an instant answer. a user query receiver for receiving a user query from the user terminal;
a query extension unit for generating a similar question template of the user query using a paraphrase engine;
a training data construction unit that generates training data for training the query extension unit using a Neural Machine Translation (NMT) engine;
The learning data construction unit,
NMT engine management unit for generating learning data by translating and re-translating user log data with the NMT engine based on the neural network; and
Learning data for storing the learning data generated by the NMT engine management unit, learning a paraphrase model that can be applied to the paraphrase engine using the generated learning data, and testing and verifying the paraphrase model Management; Including, artificial intelligence question and answer system.

Images