KR102334583B1 - A method and apparatus for question-answering on educational contents in interactive query system - Google Patents

Abstract

The present specification provides a method for answering a question performed by an electronic device, the method comprising: generating an embedding vector for each part included in content using a neural network; generating an embedding vector for a user query using the neural network; Disclosed is a method for answering a question including determining a portion of the content having an embedding vector value most similar to an embedding vector for the user query as a response to the user query.

Description

A question-answering method and question-answering system for answering learners' questions about educational contents in an interactive questioning system {A method and apparatus for question-answering on educational contents in interactive query system}

The present invention relates to a method for presenting a user with a response to a user query.

During the learning process, learners experience the inconvenience of having to search through books or the Internet for information they do not know. Given that a lot of time is spent on such searches, learners often spend more time searching for content they don't know than learning the educational content. This process is one factor that makes the learner lose interest in the learning process.

In order to reduce the time to search for unknown content, private tutoring, a service that asks an instructor or the Internet for unknown content and waits for an answer exists. However, such a service has problems in that real-time performance is poor in that a person directly provides a response, and the cost of the service is also excessive.

In addition, there is a problem in that search efficiency cannot be guaranteed because the method of constructing a search formula is different for each user in a method of searching for information using the Internet.

In order to solve this problem, methods for searching a labeled text document database are proposed in Korean Patent Application Laid-Open No. 10-2011-0134909 or the like.

However, the above methods also have the inconvenience of using a separate search engine to search for specific information.

The present specification provides a method for effectively presenting a learner's query regarding educational content in an interactive query system.

In order to solve the above problem, a question answering method performed by an electronic device according to an embodiment of the present invention includes generating an embedding vector for each part included in content using a neural network; generating an embedding vector for a user query using the neural network; and determining, as a response to the user query, a portion of the content having an embedding vector value most similar to an embedding vector for the user query.

An embedding vector generated for the content is generated for each phrase included in the content, and the part having the embedding vector value most similar to the embedding vector for the user query is for each phrase included in the content. can be decided.

The content includes sequentially classified learning data according to a curriculum, and an embedding vector for each part may be determined based on a classification order of the educational content.

The embedding vector for the user query may be determined based on a user query input before the user query.

The user query is inputted in time series, and the embedding vector for the current user query is calculated by calculating the vector value of the token constituting the previous user query input before the current user query and the vector value of the token constituting the current user query. can be calculated.

In addition, in order to solve the above problem, a neural network training method performed by an electronic device according to an embodiment of the present invention includes generating a first embedding vector of a neural network by receiving a training query as an input; generating a second embedding vector of the neural network by inputting a portion of the content corresponding to the response to the training query; and modifying the parameters of the neural network until a similarity between the first embedding vector and the second embedding vector exceeds a preset value.

The content includes sequentially classified learning data according to a curriculum, the second embedding vector is generated for a phrase included in the content, and the second embedding vector is in the order of classification of the educational content. Based on the determination, the first embedding vector may be generated based on the entire part of the training query. The second embedding vector may be generated by calculating a dense vector and a sparse vector.

The second embedding vector may include a vector of start tokens constituting the sphere and a vector of end tokens constituting the sphere.

The second embedding vector may further include a parameter indicating consistency between the start token and the end token.

In addition, in order to solve the above problem, a method for answering a question performed by an electronic device according to an embodiment of the present invention includes: determining a query type of a user query; generating a first response that is a search result based on the vector similarity of the query when the query type is the first type; generating a second response that is a search result based on the text similarity of the query when the query type is the second type; and determining a response to the query according to the reliability evaluation results of the first response and the second response, and generating the first response includes using an embedding vector for each part included in the content with a neural network. generating using; generating an embedding vector for a user query using the neural network; and determining, as a response to the user query, a portion of the content having an embedding vector value most similar to an embedding vector for the user query.

The first type may be a type in which the query type corresponds to the six-fold rule, and the second type may be a type in which the query type does not correspond to the six-fold rule.

In addition, an electronic device according to an embodiment of the present invention for solving the above problem includes: a processor; and a memory for storing instructions executed by the processor, wherein the processor generates an embedding vector for each part included in the content using a neural network, and generates an embedding vector for a user query using the neural network and determining a portion of the content having an embedding vector value most similar to an embedding vector for the user query as a response to the user query.

In addition, the computer-readable recording medium on which a computer program executed by a computer is recorded according to an embodiment of the present invention for solving the above problem is to generate an embedding vector for each part included in the content using a neural network. step; generating an embedding vector for a user query using the neural network; and determining, as a response to the user query, a portion of the content having an embedding vector value most similar to an embedding vector for the user query.

In addition, a computer program executed by a computer according to an embodiment of the present invention for solving the above problem includes generating an embedding vector for each part included in the content using a neural network; generating an embedding vector for a user query using the neural network; and determining, as a response to the user query, a portion of the content having an embedding vector value most similar to an embedding vector for the user query.

The present specification provides a method for effectively presenting a learner's query regarding educational content in an interactive query system.

1 is a block diagram illustrating an education system according to an embodiment of the present invention.
2 is a block diagram illustrating the configuration of a terminal and a server according to an embodiment of the present invention.
3 is a block diagram illustrating the configuration of a server according to an embodiment of the present invention.
4 is a block diagram illustrating an example in which a server presents a response to a user query according to an embodiment of the present invention.
5 is a flowchart illustrating a method for answering a question by a server according to an embodiment of the present invention.
6 is a flowchart illustrating a method of generating a response to a user query by using a neural network according to an embodiment of the present invention.
7 is a flowchart illustrating a process of training a response search neural network for performing a question and answer according to an embodiment of the present invention.

The following is merely illustrative of the principles of the invention. Therefore, those skilled in the art can devise various devices that, although not explicitly described or shown herein, embody the principles of the invention and are included in the spirit and scope of the invention. In addition, it should be understood that all conditional terms and examples listed herein are, in principle, expressly intended only for the purpose of understanding the inventive concept and are not limited to the specifically enumerated embodiments and states as such. .

The above-described objects, features and advantages will become more apparent through the following detailed description in relation to the accompanying drawings, and accordingly, those of ordinary skill in the art to which the invention pertains will be able to easily practice the technical idea of the invention. .

In the specification and claims, terms such as "first," "second," "third," and "fourth," are used, if any, to distinguish between like elements and, if not necessarily, in a specific sequence. or to describe the order of occurrence. It will be understood that the terms so used are interchangeable under appropriate circumstances to enable the embodiments of the invention described herein to operate, for example, in sequences other than those shown or described herein. Likewise, where methods are described herein as comprising a series of steps, the order of those steps presented herein is not necessarily the order in which those steps may be performed, and any described steps may be omitted and/or Any other steps not described may be added to the method.

Also, terms such as "left", "right", "front", "behind", "top", "bottom", "above", "below" in the specification and claims are used for descriptive purposes, It is not necessarily intended to describe an invariant relative position. It will be understood that the terms so used are interchangeable under appropriate circumstances to enable the embodiments of the invention described herein to operate otherwise than, for example, as shown or described herein. As used herein, the term “connected” is defined as being directly or indirectly connected in an electrical or non-electrical manner. Objects described herein as being "adjacent" to one another may be in physical contact with one another, in proximity to one another, or in the same general scope or area as appropriate for the context in which the phrase is used. The presence of the phrase “in one embodiment” herein refers to the same, but not necessarily, embodiment.

In addition, in the specification and claims, references to 'connected', 'connecting', 'fastened', 'fastening', 'coupled', 'coupled', etc., and various variations of these expressions, refer to other elements directly It is used in the sense of being connected or indirectly connected through other components.

In addition, the suffixes "module" and "part" for components used in this specification are given or used in consideration of ease of writing the specification, and do not have a meaning or role distinct from each other by themselves.

In addition, the terms used herein are for the purpose of describing the embodiments and are not intended to limit the present invention. In this specification, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, 'comprise' and/or 'comprising' means that a referenced component, step, operation and/or element is the presence of one or more other components, steps, operations and/or elements. or addition is not excluded.

In addition, in the description of the invention, if it is determined that a detailed description of a known technology related to the invention may unnecessarily obscure the gist of the invention, the detailed description thereof will be omitted. Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram illustrating an education system according to an embodiment of the present invention. The terminal 100 according to an embodiment transmits a query input from a learner to the server 200 and receives a response to the query from the server 200 . The terminal 100 provides an answer to the user's query to the user based on the query response received from the server 200 .

The terminal 100 may be a general-purpose terminal such as a generally used smart phone, tablet, or computer. The terminal 100 according to an embodiment of the present invention may include a processor 1 , a memory 2 , a communication unit 3 , a user interface 4 , and a database 5 as shown in FIG. 2 . The memory 2 may store a program necessary for the terminal 100 in order to perform the question answering method according to an embodiment of the present invention. For example, the memory 2 may store an application program that obtains a user's input, generates a query according to the user's input, and transmits it to the server 200 . In addition, the memory 2 may store learning data necessary for the education of the learner according to the curriculum.

The processor 1 processes data according to a program stored in the memory 2 to control the communication unit 3, the user interface 4, and the database 5 to perform the question answering method according to an embodiment of the present invention. can

The server 200 according to an embodiment of the present invention may also include a processor 1 , a memory 2 , a communication unit 3 , a user interface 4 , and a database 5 as shown in FIG. 2 . The memory 2 may store a program necessary for the server 200 to perform the question answering method according to an embodiment of the present invention. For example, the memory 2 may store a program for obtaining a user query from the terminal 100 , generating a response according to it, and transmitting it to the terminal 100 . Examples of such programs include a QA engine program that generates responses to user queries, a chatbot engine program that provides user queries and responses to users in a chatting format, and a training content management program that provides responses to user queries. An educational content management program, an interface module program for exchanging data with an external network, etc. may be included. The processor 1 of the server 200 processes data according to the program stored in the memory 2 to control the communication unit 3, the user interface 4, and the database 5, thereby making a query according to an embodiment of the present invention. The response method can be performed.

Meanwhile, the server 200 according to an embodiment of the present invention may be configured in a module unit as shown in FIG. 3 . The server 200 may be composed of a QA engine 210 , a chatbot engine 220 , a database 230 , and an interface module 240 according to functional units.

The QA engine 210 may train and manage a neural network in order to extract a response corresponding to a user query from a large-scale document based on the neural network model. The QA engine can select and operate an appropriate neural network model for performing a large-scale neighbor search.

The QA engine may generate an embedding vector for the text of the target content in a phrase unit in order to extract a response corresponding to a query from the target content. The QA engine may index target content in a phrase unit or document unit in order to extract a response corresponding to a query from the target content. The QA engine may generate an embedding vector for a user query. The QA engine can train the neural network so that the embedding vector for the user query and the embedding vector of the sphere of the target content that can be selected as a response to the user query have a similarity (i.e. cosine similarity) higher than a set standard.

In addition, the QA engine 210 according to an embodiment may generate an embedding vector of a phrase constituting the educational content based on the context of the educational content by using a BERT (Bidirectional Encoder Representations from Transformers) model. Also, the QA engine 210 may generate an embedding vector for the user query based on the context of the conversation history before the user query using the BERT model. Accordingly, the QA engine 210 may provide a continuous response based on the user's continuous interactive question-and-answer history.

The chatbot engine 220 analyzes the user input received from the terminal 100 and transmits an answer to the user input to the terminal 100 . The chatbot engine 220 may perform a basic natural language processing process for identifying a user input composed of natural language. For example, the chatbot engine 220 may acquire a token by performing pre-processing of removing research or stopwords from a user input composed of natural language. And by using the given token, keywords constituting the query and the type of query can be recognized.

The chatbot engine 220 may understand the context of the conversation by analyzing a conversation formed according to a user query input from the user and a response transmitted by the chatbot engine 220 to the terminal 100 . The chatbot engine 220 may identify the context of the conversation based on the user's previous question and generate a response based on the context. For example, the chatbot engine 220 may send a conversation history to the QA engine 210 , and may transmit a response generated by the QA engine 210 to the terminal 100 .

The chatbot engine 220 may generate a first response based on a neural network by utilizing the QA engine 210 to generate a response corresponding to a user query included in the user input. Also, the chatbot engine 220 may generate a second response based on the rule base based on a pair of questions and answers of the knowledge base stored in the database 230 . The chatbot engine 220 may provide the terminal 100 by merging the first response and the second response in order to generate an appropriate response to the user's query. In an embodiment, the chatbot engine 220 may select a response having a higher reliability as an answer to the user query from among the first response and the second response and transmit it to the terminal 100 . In an embodiment, the database 230 may store reliability evaluation results of the first response and the second response to a specific user query. The chatbot engine 220 may select a response having a high reliability evaluation value and transmit it to the terminal 100 .

In one embodiment, the chatbot engine 220 provides the first response and the second response to the user's query to the user through the terminal 100 to evaluate the reliability of the first response and the second response, and then Reliability evaluation results may be received. To this end, the chatbot engine 220 may present a part of the document including the response to the user as the basis for the response. Furthermore, as shown in FIG. 4 , the chatbot engine 220 may transmit the response and the reliability score of the corresponding response together with a portion of the document in which the response is expressed to the terminal 100 .

The chatbot engine 220 may store the reliability of the first response and the second response to the user's query based on the user's reliability evaluation result. The chatbot engine 220 may then select a response based on a pre-stored reliability evaluation value for the received user query (which may be a query from another user).

The database 230 stores content for the server 200 to provide a response corresponding to a user's query to the terminal 100 . The content corresponding to the user's query may be composed of QA content selected based on the QA engine and knowledge base-based content selected by the query-answer correspondence table.

The interface module 240 may be composed of a user interface module for obtaining an input from the server manager for controlling the server 200 or providing information to the server manager, and a system interface module for exchanging data with another platform or system. can

The user interface may include a keyboard, a mouse, a touch screen, a speaker, etc. for receiving an input from a user, and may include a monitor, a speaker, a vibration module, etc. for providing information to the user.

The system interface may be configured as a program for performing processing for exchanging data by accessing a chatbot platform or a voice recognition platform of another system. For example, the system interface may perform interfacing so that an external platform such as KakaoTalk, Google Home & Assistant may be treated as the terminal 100 to perform a query and answer according to the present invention.

In addition, the system interface may perform interfacing so that content can be updated in real time from an external web-based database. For example, the system interface may perform interfacing to obtain a search result for a specific search term from Wikipedia or the like.

Hereinafter, a question answering method according to an embodiment of the present invention will be described with reference to a flowchart. 5 is a flowchart illustrating a method for answering a question by the server 200 according to an embodiment of the present invention. First, the server 200 according to an embodiment may analyze the query received from the terminal 100 to determine the type of the query ( S510 ). The server 200 may be configured to further include an intention classifier to analyze the type of user query.

The server 200 may determine the type of the user query based on the keywords included in the user query by using the intention classifier. For example, the intent classifier uses keywords such as 'who', 'when', 'where', 'what', 'how', and 'why', which are keywords corresponding to the six-fold principle, in the user query. may be determined as the first type of query to which a response should be provided through a neural network.

For example, if a user query is “Who is Admiral Yi Sun-sin?” or “Where was Admiral Yi Sun-sin born?” In the case of a query such as , it can be determined as the first type query.

On the other hand, the intention classifier may determine the user's answer as the second type of query to be provided based on the rule base when the keyword corresponding to the six-fold principle is not included in the user's query. For example, the intent classifier is included in the user query based on the query-answer table of the knowledge base when a token such as 'what is', which is a keyword for definition rather than a keyword corresponding to the six-fold principle, is included in the user query. It can be determined as a rule-base-based second type query that provides the search result for the object name that has been searched for as a response. For example, if the user query is a query such as “What is a turtle ship?”, it may be determined as the second type of query.

Next, when the user query is the first type of query, the server 200 may generate a response using the neural network generated by the QA engine ( S520 ). The server 200 may generate an embedding vector of a user query using the QA engine, and then search for an answer based on the determination of the similarity of the embedding vector. This will be described in more detail later with reference to FIG. 6 .

Next, when the user query is the second type of query, the server 200 may generate a response based on the rule base ( S530 ). For example, when the user query is the second type of query, the server 200 may generate a response to the user query according to the correspondence relationship of the query-answer table of the knowledge base. In an embodiment, the correspondence of the query-answer table of the knowledge base is “IF (keyword) return response;” can be of the form.

Next, the server 200 provides this to the terminal 100 after the reliability evaluation of the generated response (S540). In an embodiment, the server 200 may not accept the response generated by using the single-layer simple filter as a response when an unusable stopword such as profanity is included in the response. In another embodiment, when the reliability between the generated response and the keyword of the user's query is less than a certain level, the server 200 may not accept it as the response. The reliability between the generated response and the keyword of the user's query may be previously stored in the database.

Meanwhile, in some embodiments, the server 200 may generate both a response using a neural network and a response based on a rule base without determining the type of the user's query. Furthermore, in response search using a neural network, the server 200 may generate a plurality of responses in the order of a high degree of similarity between an embedding vector between a query and a phrase. In addition, a plurality of responses may be generated in the rule base response search. In this case, the server 200 may perform a reliability evaluation for each response, and may provide a response selected according to the evaluation result to the terminal 100 . When a plurality of responses to a user's query are generated, the server 200 may provide a preset number of responses to the terminal 100 in order of high relevance to the user's query among them. As described above, the reliability between the generated response and the keyword of the user's query may be previously stored in the database.

In addition, according to an embodiment, information of educational subjects or tutors corresponding to user queries and responses may be matched and stored in a database. In this case, the server 200 may select a highly relevant response by evaluating the relevance of the user's query and response according to the type of tutor or education subject, and provide it as a response to the user.

Hereinafter, a method ( S520 ) of generating a response using a neural network will be described in more detail with reference to FIG. 6 . 6 is a flowchart illustrating a method of generating a response to a user query using a neural network.

First, the server 200 obtains data for generating educational content (S610). In an embodiment, the data may be educational data. The server 200 may acquire data for generating educational contents by accessing a large-scale web-based database such as Wikipedia. Alternatively, the server 200 may acquire education data from an education office or an external server that provides educational content.

Next, the server 200 generates educational content (S620). The server 200 may generate educational content by classifying pre-obtained education data into items according to the table of contents, based on the table of contents of the educational content input from the user. Alternatively, the server 200 may generate educational content by receiving the content of the curriculum input from the user in units of paragraphs, classifying them by content, and storing them. When the data is a table or a figure, the server 200 may present a request message to the manager through a user interface to request the manager to input a description thereof as text.

The server 200 may classify and record the question-and-answer items generated in the conversation history with the terminal 100 according to the contents of the educational content. For example, the question “What is the hypothesis of Mencius who felt skeptical about Jeong Do-jeon?” in FIG. 4 and “It is [Seong Seon-seol],” which is a conversation history generated in response to this question, can be newly added to the related table of contents of educational contents. .

In an embodiment, the conversation history may be generated by a plurality of users. For example, when the chatbot engine 220 participates in a group chat room, conversation details may be generated by a plurality of users. In this case, the question-answer items generated in the group chat room may be classified and recorded according to the contents of the educational content.

Also, when the item the user has inquired about does not exist in the table of contents of the educational content, a table of contents for the corresponding item may be created in the educational content. For example, an education table of contents for each subject related to the item queried by the user may be automatically incorporated into the education content. To this end, the server 200 may store an educational table of contents for each subject.

In addition, when the item the user has inquired about does not exist in the table of contents of the educational content, the table of contents for the corresponding query item in the educational content of other users who have a similar educational tendency to the user may be automatically incorporated into the table of contents of the user's educational content. . To this end, the server 200 may match and store education propensity information for each user.

Next, the server 200 learns the neural network (S630). The server 200 may learn the neural network using a Korean data set such as KorQuAD 1.0. The server learns the neural network so that phrases existing in user queries and educational contents can be expressed as embedding vectors similar to each other. For example, a phrase could be a noun phrase expressed like “fresh vegetables”.

A neural network model according to an embodiment of the present invention encodes large text spans and uses a dense vector and a sparse vector together to provide an effect of obtaining an answer in real time by performing a nearest neighbor search. can be used to create an embedding vector. Dense vectors are effective for expressing syntactic and semantic signals, and sparse vectors are efficient for encoding precise lexical information.

The server 200 may generate an embedding vector for an independent phrase by connecting the sparse vector to the dense vector. The neural network model according to an embodiment of the present invention may express a sphere span as a dense vector and a sparse vector in the corpus of the generated educational content, respectively. In the educational content x, the embedding vector of each span (i, j) may be expressed as follows.

[Equation 1]

는 교육 컨텐츠에서 스팬(i, j)에 대한 밀집 벡터를 나타내고,

는 교육 컨텐츠에서 스팬(i, j)에 대한 희소 벡터를 나타낸다.here,

represents the dense vector for the span (i, j) in the educational content,

denotes a sparse vector for span (i, j) in educational content.

Next, the server 200 generates an embedding vector for each phrase of the educational content using the learned neural network (S640). Since the server 200 generates an embedding vector for each sphere in advance, when a user query is input, the vector similarity can be compared only with respect to the previously generated vector. For example, when the maximum span length (ie all possible spans from all training content) is J, the server 200 pre- _{prescribes text span embeddings (x i:j ) for all i,j pairs where ji<J.} It may be calculated and stored in the database as a phrase index.

In an embodiment, the server 200 may generate an embedding vector composed of a dense vector and a sparse vector using a neural network. A dense vector is generated using a dense vector model. The dense vector model can encode syntactic or semantic information of a phrase depending on the context. The dense vector d _i:j is a scalar value measuring the consistency of the vector of tokens corresponding to the starting point of the sphere, the vector of tokens corresponding to the ending point, and the vector of starting tokens and ending tokens (hereafter, coherency scalars). ) can be composed of

Since the vector value corresponding to each token is generated according to the context, the coherency scalar provides the effect that a sphere composed of tokens having different contexts is not selected as a response. For example, in the query “What foods help growth?”, “Eggs help growth. Excessive exercise is not recommended. Milk is rich in calcium, so it is helpful for bone health.” If the start and end vectors are modeled respectively, 'egg' and 'milk' can be selected as the start and end tokens. However, as the vector values for 'egg' and 'milk' have different contexts due to the token constituting the sentence “It is not desirable to exercise excessively”, the coherency scalar value has a low value.

To obtain this construction of dense vectors, we can utilize context-based representations such as BERT-large, which are pre-trained on large corpus such as Wikipedia. BERT (Bidirectional Encoder Representations from Transformers) is a sequence of document tokens x=x ₁ , … , _{a sequence of vectors corresponding to x n} (ie a matrix) H=[h ₁ ; … ; h _n ]. Here, n is the length of the input sequence, d is the size of the hidden stage, and [;] is the vertical concatenation. A detailed description of BERT is provided in the paper BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding by Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.

An example will be described in more detail. In an embodiment, the token encoding may be set to be divided into _{four vectors h i} =[h ¹ _i, h ² _i, h ³ _i, h ⁴ _{i ].} Here, [,] means column-wise connection. In this case, a dense start vector a _i may be obtained ^{from h 1} _i , and a dense start vector b _j may be obtained ^{from h 2} _{j .} Finally, ^{a coherency scalar c k} _i,j can be obtained from the dot product of ^{h 3} _{i and} h ⁴ _{j .} In this case, the dot product results show higher values for highly consistent spheres. Accordingly, the dense vector can be expressed as follows.

[Equation 2]

Accordingly, the embedding vector for each phrase included in the educational content may be determined based on the classification order of the educational content or the arrangement of phrases listed in the educational content.

Meanwhile, a sparse vector may be generated using a sparse vector model. The server 200 may use a term frequency based encoding to obtain a ^{sparse vector s k} _{i:j for each sphere.} In an embodiment, the server 200 may generate a sparse vector by applying an n-gram-based TF-IDF (Term Frequency - Inverse Document Frequency) to the educational content.

Meanwhile, when data is added from an external database such as Wikipedia, the server 200 may generate an embedding vector for the added data in a phrase unit.

Next, when receiving the user query, the server 200 generates an embedding vector for the user query ( S650 ). The server 200 may determine an embedding vector for the user query based on the user query input before the user query. For example, a user query is input in time series, and the embedding vector for the current user query is a vector value of a token constituting a previous user query input before the current user query and a vector value of a token constituting the current user query can be calculated by calculating

에 임베딩하여 사용자 질의에 대한 임베딩 벡터를 생성할 수 있다.For example, the server 200 executes the user query in the same vector space in a similar manner to the previously generated embedding vector for the sphere of educational content.

By embedding in , it is possible to generate an embedding vector for a user query.

The embedding vector for the user query may be generated for each phrase constituting the user query, but may be generated for all tokens constituting the user query. For example, an embedding vector generated as a result of applying a neural network to a tokenized query after preprocessing excluding stopwords and investigations from the user query may be generated for the user query.

To obtain a dense query vector, we can use a special token ([CLS] for BERT) that is appended to the token constituting the query. For example, the query q is [CLS], q ₁ , … , can be expressed as _{q s .} Accordingly, the server 200 can model dense query embedding differently from dense embedding in the phrase index even though all parameters of the BERT encoder are shared. That is, the dense query vector can be expressed as follows.

[Equation 3]

Here, h' ₁ ¹ is the encoding corresponding to the (first) special token, and other parameters can be obtained in a similar manner. In order to obtain the sparse query vector s', the TF-IDF embedding model may be applied to the entire query as described above.

Next, the server 200 searches for a phrase of the educational content having an embedding vector most similar to the embedding vector of the user's query, and generates the searched phrase as a response (S660). The server 200 sets the i, j index indicating the position of the sphere in the educational content expressing the response to the query, the user query embedding vector 'q' and the embedding vector 'x _i:j ' for each sphere of the educational content. It can be obtained by calculating the maximum dot product between

[Equation 4]

Hereinafter, a training process of the neural network will be described in more detail with reference to FIG. 7 . 7 is a flowchart illustrating a process of training a response search neural network for performing a question and answer according to an embodiment of the present invention.

First, the server 200 generates an embedding vector for a text phrase (noun phrase, etc.), and generates an embedding vector for a user query (S710).

Next, the server 200 trains the neural network so that the phrase of the educational content and the embedding vector of the user query matching it have similar values ( S620 ). The similarity may be determined according to calculating the cosine similarity of two embedding vectors and whether a result value is higher than a preset reference value. The server 200 may determine that the value of the cosine similarity is similar to 1, and is dissimilar as it is closer to -1.

In an embodiment, the server 200 may perform forward propagation to determine a phrase of educational content and an embedding vector of a user query matching the phrase. And the server 200 uses a loss function to compare the cosine similarity of the embedding vector of the user query matching the phrase of the educational content with the reference value to determine the error value for determining the value of the parameter constituting the neural network. can decide The server 200 may adjust the parameters of the neural network model using backward propagation so that the error between the cosine similarity value and the reference value is reduced. As an optimizer used for backpropagation, a batch gradient descent method, a stochastic gradient descent method, or the like may be used.

Meanwhile, the server 200 may apply early termination or dropout to prevent overfitting. In an embodiment, the server 200 trains an epoch before the accuracy is lowered if it is found that the accuracy of the verification data is continuously lowered while monitoring the error value between the predicted value and the actual value of the verification data. Early termination can be implemented by setting it as an epoch. Alternatively, the server 200 may prevent overfitting by performing dropout that is not used for learning on nodes selected based on a certain criterion. Accordingly, the server 200 may prevent the neural network from changing dependently on a specific node or combination of nodes.

The question answering method according to the embodiment described above may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured according to the embodiment, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - a hardware platform diet management server 10 specially configured to store and execute magneto-optical media, and program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.

Each of the drawings referenced in the description of the previous embodiment is merely an embodiment shown for convenience of description, and items, contents, and images of information displayed on each screen may be modified and displayed in various forms.

Although the present invention has been described with reference to an embodiment shown in the drawings, this is merely exemplary, and those of ordinary skill in the art will understand that various modifications and equivalent other embodiments are possible therefrom. Accordingly, the true technical protection scope of the present invention should be determined by the technical spirit of the appended claims.

Claims (15)

A method for answering a question performed by an electronic device, the method comprising:
generating an embedding vector for each part included in the content using a neural network;
generating an embedding vector for a user query using the neural network; and
determining, as a response to the user query, a portion of the content having an embedding vector value most similar to an embedding vector for the user query,
The content is classified and stored in a predetermined input unit based on the table of contents of the educational content,
The embedding vector for each part is determined based on the table of contents of the educational content and the predetermined input unit,
A question answering method in which a response to the user's query is determined based on a degree of relevance of a tutor to the user's query.
The method of claim 1,
An embedding vector generated for the content is generated for each phrase included in the content,
A portion having an embedding vector value most similar to the embedding vector for the user query is determined for each phrase included in the content.
delete The method of claim 1,
An embedding vector for the user query is determined based on a user query input before the user query.
5. The method of claim 4,
The user query is inputted in time series,
The embedding vector for the current user query is calculated by calculating a vector value of a token constituting a previous user query input before the current user query and a vector value of a token constituting the current user query.
delete delete delete delete delete A question answering method performed by an electronic device, the method comprising:
determining a query type of a user query;
generating a first response that is a search result based on the vector similarity of the query when the query type is the first type;
generating a second response that is a search result based on the text similarity of the query when the query type is the second type; and
determining a response to the query according to the reliability evaluation results of the first response and the second response;
Generating the first response comprises:
generating an embedding vector for each part included in the content using a neural network;
generating an embedding vector for a user query using the neural network; and
determining, as a response to the user query, a portion of the content having an embedding vector value most similar to an embedding vector for the user query,
The content is classified and stored in a predetermined input unit based on the table of contents of the educational content,
The embedding vector for each part is determined based on the table of contents of the educational content and the predetermined input unit,
A question answering method in which a response to the user's query is determined based on a degree of relevance of a tutor to the user's query.
12. The method of claim 11,
In the first type, the query type is a type corresponding to the six-fold principle, and the second type is a type in which the query type does not correspond to the six-fold principle.
processor; and
a memory for storing instructions executed by the processor;
The processor is
An embedding vector for each part included in the content is generated using a neural network,
An embedding vector for a user query is generated using the neural network,
determining a portion of the content having an embedding vector value most similar to an embedding vector for the user query as a response to the user query,
The content is classified and stored in a predetermined input unit based on the table of contents of the educational content,
The embedding vector for each part is determined based on the table of contents of the educational content and the predetermined input unit,
The electronic device in which the response to the user's query is determined based on a degree of relevance of a tutor to the user's query.
A computer-readable recording medium in which a computer program is recorded, which is executed by a computer for executing the question-and-answer method of claim 1 .
A computer program, stored in a computer-readable storage medium, that executes the question and answer method of claim 1 .

Images