KR20230077588A - Method of classifying intention of various question and searching answers of financial domain based on financial term language model and system impelemting thereof - Google Patents

Abstract

The present invention relates to a method and system for classifying intentions for various queries in a financial domain and retrieving answers based on a financial terminology model, and in relation to a financial terminology language model according to an embodiment of the present invention. A method of classifying intentions and searching for answers to various queries includes extracting, by a processor of the system, an entity name based on unsupervised learning using first input data, and storing the entity name extracted by the processor in a database of the system. and a processor performing tokenization by applying the object name stored in the database to the preprocessed result of the first input data or the second input data, and a language model including a dictionary of financial terms based on the result of the tokenization by the processor It includes the step of constructing and storing in a database.

Description

METHOD OF CLASSIFYING INTENTION OF VARIOUS QUESTION AND SEARCHING ANSWERS OF FINANCIAL DOMAIN BASED ON FINANCIAL TERM LANGUAGE MODEL AND SYSTEM IMPELEMTING THEREOF }

The present invention relates to a method and system for classifying intentions for various queries in the financial domain and retrieving answers based on a financial term language model.

Unlike the general knowledge area, the area that provides information related to finance requires skills to process specialized terms and abbreviations. In particular, since various articles and reports provide high-quality financial information, it is necessary to effectively provide financial information based on them.

Conventionally, technology for providing knowledge information using chatbots or providing query responses for knowledge information searches for knowledge based on morphemes or word patterns, so it could not provide answers to new queries that are not in the training data. However, there was a limitation in providing only limited information to the database.

Accordingly, a method of processing information based on terms specific to finance and classifying the intention of a question to provide an answer is requested.

This specification intends to provide information by in-depth analysis of input financial text by applying a methodology that introduces a machine learning-based classification model and a machine reading comprehension model using application domain-specific pre-learning language model learning and the corresponding model.

This specification aims to build an artificial intelligence financial expert system applying text reasoning technology.

This specification seeks to search for an answer in the document by classifying the intent of the question and extracting related real-time financial text information.

The problem to be solved by the present invention is not limited to the above-mentioned problems, and other problems not mentioned herein will be clearly understood by those skilled in the art from the following description.

A method for classifying intentions for various queries in the financial domain and searching for answers based on a financial terminology model according to an embodiment of the present invention is a method in which a processor of a system uses first input data to provide an unsupervised learning-based entity name. Extracting and storing the extracted object name in the database of the system by the processor, and performing tokenization by the processor by applying the object name stored in the database to the preprocessed result of the first input data or the second input data. and constructing, by the processor, a language model including a dictionary of financial terms based on the tokenized result and storing it in a database.

A system for classifying intentions for various queries in the financial domain and searching for answers based on a financial terminology model according to an embodiment of the present invention is generated by a processor processing information using input data and an input database and the processor A database that stores information.

In the system for classifying intentions for various queries in the financial domain and searching for answers based on a financial term language model according to an embodiment of the present invention, a processor extracts an unsupervised learning-based entity name using first input data. and stores the extracted object name in the database, and the processor performs tokenization by applying the object name stored in the database to the preprocessed result of the first input data or the second input data, and the processor performs tokenization on the tokenized result Based on this, a language model including a dictionary of financial terms is built and stored in the database.

When the embodiment of the present invention is implemented, information is provided by in-depth analysis of input financial text by applying a methodology that introduces a machine learning-based classification model and a machine reading comprehension model using application domain-specific pre-learning language model learning and the corresponding model. can

When implementing an embodiment of the present invention, an artificial intelligence financial expert system to which text inference technology is applied may be constructed.

When implementing an embodiment of the present invention, the intention of the question is classified, related real-time financial text information is extracted, and an answer can be searched for in the document.

The effects provided by the present invention are not limited to the effects mentioned above, and other effects not mentioned herein will be clearly understood by those skilled in the art from the description below.

1 is a flowchart illustrating a question answer search system using supervised learning and machine reading comprehension methodologies according to an embodiment of the present invention.
2 is a detailed flowchart of an entity name extraction step based on unsupervised learning according to an embodiment of the present invention.
3 is a detailed flowchart of embedding financial words and sentences for transfer learning according to an embodiment of the present invention.
4 is a detailed flowchart of a specific method of performing the supervised learning-based classification model learning and using step (S300) according to an embodiment of the present invention.
5 is a detailed flowchart illustrating a step of learning and using a machine reading comprehension model according to an embodiment of the present invention.
6 is a detailed flowchart of a machine reading comprehension training set generating module according to an embodiment of the present invention.
7 is a detailed flowchart of object name-based document filtering and title embedding comparison according to an embodiment of the present invention.
8 is a diagram showing the configuration of an artificial intelligence financial expert system according to an embodiment of the present invention.
9 is a diagram showing the integrated processes of S100, S200, S500, and S600 according to an embodiment of the present invention.
10 is a diagram showing a process of performing the processes of S300 and S400 by applying an embedding model and learning data according to an embodiment of the present invention.
11 is a diagram illustrating a supervised learning classification model according to an embodiment of the present invention and an explanation of inputs and outputs used in machine learning.
12 is a diagram illustrating input and output used when learning a machine reading comprehension model according to an embodiment of the present invention.
13 is an overall configuration diagram of a chatbot system including the main embodiments of the invention described in the present invention.
14 is a diagram showing a process of automatically extracting financial information for language processing in the financial domain according to an embodiment of the present invention.
15 is a diagram illustrating a process of classifying intentions for various queries in the financial domain and searching for answers based on a financial term language model according to an embodiment of the present invention.
16 is a diagram showing in detail a process of performing retorquenizing according to an embodiment of the present invention.
17 is a diagram showing an example of performing financial term-based retalking according to an embodiment of the present invention.

Advantages and features of the present invention, and methods of achieving them, will become clear with reference to the detailed description of the following embodiments taken in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and will be implemented in various forms different from each other, only these embodiments make the disclosure of the present invention complete, and common knowledge in the art to which the present invention pertains. It is provided to completely inform the person who has the scope of the invention, and the present invention is only defined by the scope of the claims. Like reference numbers designate like elements throughout the specification.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The detailed description of the present invention refers to the accompanying drawings which illustrate, by way of illustration, specific embodiments in which the present invention may be practiced. These embodiments are described in sufficient detail to enable any person skilled in the art to practice the present invention. It should be understood that the various embodiments of the present invention are different, but need not be mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be implemented in relation to one embodiment to another without departing from the spirit and scope of the invention. Additionally, it should be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the invention.

Accordingly, the detailed description set forth below is not to be taken in a limiting sense, and the scope of the present invention, if properly described, is limited only by the appended claims, along with all equivalents as claimed by those claims. Like reference numbers in the drawings indicate the same or similar function throughout the various aspects.

Below, we look at an artificial intelligence financial expert system based on deep understanding of text for real-time investment advisory chatbot service.

In one embodiment, a domain-specific pre-learning model is built through financial knowledge base and corpus learning, and artificial intelligence investment advisory services are provided by applying deep text understanding technology based on intent classification technology and machine reading comprehension technology to which the model is applied. Let's take a look at how to do it and the device for it.

A dialog system for providing information in a specific field consists of a process of designing and tracking a conversation state by extracting a query sentence from a text input by a user and analyzing a pattern in order to define it as a query for knowledge base search.

In addition, the scale and functional scope of the system may be limited to suit the purpose of the service in the field to provide information. In addition, it is possible to provide an answer to a query entered by a user by retaining prepared knowledge in the corresponding field in the knowledge DB.

Conventional technologies use a method of constructing a knowledge base by including a predefined user-expected query, searching for a matching query within the knowledge base, and providing an answer.

However, since questions are generally not entered only within a predefined query, there is a high possibility that an answer may not be provided or an incorrect answer may be provided to a query that is not predefined.

This is due to the use of a word-level entity name query. In the method of using a word-by-word query, there is a limit to grasping the meaning of other words for which a relation called a synonym is not defined in advance. In addition, in the case of a word not in the dictionary, an Out-Of-Vocabulary (OOV) problem occurs, which is highly likely to cause a problem in the morphological analysis process.

In addition, the query retrieval method using the rulebase has limitations in that it is difficult to provide an answer if knowledge matching the query is not in the knowledge base, and that knowledge bases such as similar query patterns must be continuously added.

Although it is possible to continuously manage the knowledge base for unanswered queries, the knowledge base for searching queries is limited and has limitations in that it is difficult to include real-time updated information in answers. In addition, there are limitations in the quality and quantity of the knowledge building process, as well as modeling or maintaining the knowledge base of a goal-oriented conversation system with large scale and variability, such as knowledge that appears infrequently and knowledge that is added or changed over time. There is a limit to what to do.

Applied in a method of providing appropriate information or answers based on a financial knowledge base such as a thesaurus or ontology with additional knowledge such as pre-built synonyms, top and bottom words, and graphs by analyzing morphemes or keywords for user input There are limitations in constructing additional knowledge according to domains, and there are problems with interoperability with external knowledge bases.

Therefore, in the present specification, in order to overcome the limitations of query search and knowledge base construction, knowledge with high similarity is retrieved within knowledge by expanding the query through a linguistic analysis methodology of user input.

In addition, real-time investment advisory chatbot service that overcomes linguistic qualities such as morphemes and query patterns and knowledge-based conversation systems and knowledge building problems and performance limitations of chatbot services, and provides real-time customized financial information such as external economic indicators and statistical data. It is necessary to develop

To this end, in this specification, a methodology that introduces a machine learning-based classification model and a machine reading comprehension model using application domain-specific pretraining language model learning and the corresponding model is applied. When the embodiments of the present specification are applied, an artificial intelligence financial expert system to which an input financial text is deeply understood and text inference technology is applied is constructed. Through this, it categorizes the intention, extracts related real-time financial text information, and searches for answers in the document.

According to an embodiment of the present invention, unsupervised learning-based object name extraction step; Embedding words and sentences for transfer learning; Learning and using a supervised learning-based intent classification model; Learning and using machine reading comprehension models; generating a learning set for machine reading comprehension; and entity name-based document filtering and subject embedding comparison.

According to an embodiment, the step of extracting entity names based on unsupervised learning may include preprocessing a corpus of sentences; Secondary/middle/final separation and capitalization of the alphabet; Internal word separation step; may include.

According to an embodiment, the step of embedding words and sentences for transfer learning may include a preprocessing step of corpus sentences; applying an unsupervised learning-based tokenizer; A step of building a pre-learning language model; may be included.

According to an embodiment, the step of learning and using a supervised learning-based intention classification model includes the step of tokenizing a financial-related question input; Input question embedding step; Building a supervised learning-based classification model; may include.

According to an embodiment, the step of learning and using the machine reading comprehension model may include preprocessing the machine reading comprehension input data; input question/document embedding step; It may include; building a machine reading comprehension model.

According to an embodiment, the generating of the machine reading comprehension training set includes generating an additional feature; Data structure conversion step; may include.

According to an embodiment, the entity name-based document filtering and title embedding comparison step may include entity name-based document filtering step; It may include; embedding comparison step between titles/intentions of financial documents.

Hereinafter, a question answer search method using a supervised learning and machine reading comprehension methodology according to an embodiment of the present invention will be described with reference to the accompanying drawings.

1 is a flowchart illustrating a question answer search system using supervised learning and machine reading comprehension methodologies according to an embodiment of the present invention.

1 is a representative diagram of the present invention according to an embodiment, and the present invention presents a technique for classifying intentions and searching for answers in response to various queries occurring in the financial domain as an embodiment. unsupervised learning-based object name extraction step (S100); Embedding words and sentences for transfer learning (S200); Learning and using a supervised learning-based intention classification model (S300); machine reading comprehension model learning and use step (S400); Creating a learning set for machine reading comprehension (S500); and entity name-based document filtering and title embedding comparison step (S600).

Referring to FIG. 1, a method of learning a domain/corpus-specific tokenizer model through morpheme analysis and unsupervised learning was used for the entity name dictionary construction part. In the process of embedding (S200) into a language model based on the extracted entity name (S100), an unsupervised learning model is used, and the intention is classified based on the supervised learning model using this as an input.

The entity name and classification intention extracted through the entity name dictionary and supervised learning model are used for entity name-based document filtering and title embedding comparison to extract related financial documents, and the financial documents are used as input information for the machine reading comprehension model to extract answers. do. In addition, through a separate machine reading comprehension model training set generation module (S500), data is constructed in a structure capable of machine reading comprehension learning.

The financial-specific raw corpus is used in the process of unsupervised learning-based object name extraction (S100) and word and sentence embedding for transfer learning (S200). In addition, the result of learning and using the supervised learning-based intention classification model (S300) receiving the financial query set and the result of S200 generates an intention classification result and an answer.

On the other hand, the finance-specific raw corpus is input to the machine reading comprehension learning set generation module (S500) after being combined with finance-related questions/answers and finance-specific documents. In addition, question-related documents are generated by object name-based document filtering and title embedding comparison (S600) in a large-scale business database. S500 and the question-related documents are input again to the machine reading comprehension model learning and use (S400) process to generate an answer extraction result.

The processes of S100 to S600 in the process of FIG. 1 will be described in detail. Each process is performed by the processor 1500 of the system 1000 to be described later.

2 is a detailed flowchart of an entity name extraction step based on unsupervised learning according to an embodiment of the present invention. 2 shows a specific method of performing step S100.

Referring to FIG. 2 , the steps of preprocessing the corpus sentences (S110), separating the initial/middle/final letters and capitalizing the alphabet (S120), and separating internal words (S130) according to an exemplary embodiment may be included. there is. In another embodiment, when the entity name dictionary is composed of lowercase letters, the alphabet lowercase step may be performed.

Referring to FIG. 2, in the unsupervised learning-based tokenizer learning method applied to this invention, the processor 1500 learns frequently combined and used syllable (letter) patterns in a given corpus to automatically learn the boundaries of compound words and object names. .

This method is called span detection. To apply this method, corpus (news articles, official documents, etc.) of a specific domain can be simply collected. In addition, according to another embodiment, a model may be constructed using the representation of learning data itself.

The biggest advantage of the tokenizer built through this learning method is that it can automatically build index word knowledge, and in the case of adding a new index word, it can be extracted by adding only the sentence using the index word. In addition, since intermediate results are generated in a form similar to existing index word knowledge, it is relatively easy to combine with existing index word knowledge.

In addition, the processor 1500 may independently apply learning to existing index word knowledge or morpheme analyzers. Since the processor 1500 performs tokenization in processing compound nouns, postpositions, and suffixes, it is possible to process and analyze inputs regardless of spacing, and results do not deteriorate depending on the performance of morpheme analysis.

In addition, since the processor 1500 applies a subword separation algorithm to learn a similar expression pattern as internal information even within a token, learning can be performed even when there is no index word knowledge (Out-of-Vocaburary). In addition, the morpheme analyzer Without using , the processor 1500 can improve learning and index word extraction performance, and can tokenize about 15,000 sentences per second.

3 is a detailed flowchart of embedding financial words and sentences for transfer learning according to an embodiment of the present invention. This is a detailed flowchart of a specific execution method of the word and sentence embedding step (S200) for intention classification and machine reading comprehension. The embodiment of FIG. 3 shows a process of pre-processing a financial corpus and learning a pre-learning language model by applying a tokenizer including a main entity name dictionary and an entity name specific to an application domain.

Referring to FIG. 3 , a corpus sentence preprocessing step (S210), an unsupervised learning-based tokenizer application step (S220), and a pretrained language model construction step (S230) may be included according to an exemplary embodiment.

In the sentence preprocessing step (S210) according to an exemplary embodiment, the processor 1500 performs a preprocessing process while analyzing input data. In the case of alphabet input, it is possible to batch process half-width capital letters, and the processor 1500 performs an operation of separating sentences from paragraphs of a certain length or more for better sentence embedding.

In addition, OOV (Out of Vocabulary) problems are prevented in advance by correcting spaces and misspellings. Through this, the processor 1500 can improve not only the learning performance in embedding, but also the classification accuracy in the step of learning and using a supervised learning-based classification model (S300).

Using the process of FIGS. 2 and 3, a dictionary of financial terms can be provided. It is based on its own morpheme analyzer and dictionary, and can also embed financial terms using retalking. In performing corpus tokenization in FIGS. 2 and 3 , retalking is applied. This is processed so that financial terms or key words are not separated by prioritizing the financial terminology dictionary.

4 is a detailed flowchart illustrating a step of learning and using a supervised learning-based intent classification model according to an embodiment of the present invention.

4 is a detailed flowchart of a specific method of performing the supervised learning-based classification model learning and using step (S300) according to an embodiment of the present invention. The embodiment of FIG. 4 implements a financial intent classification model by learning an intent classification model through a financial domain query and answer defined based on a learned pre-learning language model, and derives an intention identified from a new real-time financial query input. show the process

Referring to FIG. 4 , according to an exemplary embodiment, tokenizing input questions related to finance (S310), embedding input questions (S320), learning a classification model based on supervised learning (S330); An input question intention classification step (S340) may be included.

In the tokenizing financial-related input question step (S310) according to an exemplary embodiment, the processor 1500 tokenizes the input question based on the entity name dictionary extracted in the unsupervised learning-based entity name extraction step (S100). perform the jing

In the step of embedding an input question (S320) according to an exemplary embodiment, the processor 1500 may bring embedding information for an input sentence based on the language model learned in the step of constructing the pretrained language model (S230). Through this, the input sentence is converted into a real vector form that can be operated, and input information used in the supervised learning classification model and the machine reading comprehension model can be generated.

In the supervised learning-based classification model learning step (S330) according to an exemplary embodiment, the processor 1500 undergoes a process of supervising the sentence embedding result and intention converted in the input question embedding step (S320). Through this process, a financial quality intention classification model can be constructed.

In the input question intent classification step (S340) according to an exemplary embodiment, the processor 1500 creates an input vector through a process of embedding an input sentence, and calculates a probability for each intent with a supervised intent classification model, so that the most reliable Classification results can be predicted in order of high intent. If the result does not exceed a certain level of reliability in the process, the machine reading comprehension model training and use step (S400) is used to search for documents related to the question, and the answer to the question is retrieved from the document to improve the response rate of the chatbot system can do.

5 is a detailed flowchart illustrating a step of learning and using a machine reading comprehension model according to an embodiment of the present invention.

5 is a detailed flowchart of a specific execution method of learning and using machine reading comprehension (S400) according to an embodiment of the present invention. The embodiment of FIG. 5 implements a financial machine reading comprehension model by learning a machine reading comprehension model after performing preprocessing of converting financial documents and questions into an input form for machine reading model learning, and then searching for and extracting answers from new financial documents. show the process

Referring to FIG. 5 , preprocessing of input data for machine reading comprehension (S410), embedding an input question/document (S420), and learning a machine reading comprehension model (S430) according to an exemplary embodiment; An answer search step (S440) may be included.

In the machine reading comprehension input data pre-processing step S410 according to an exemplary embodiment, the processor 1500 converts the input question and related document into a form usable as input information of the question/document embedding step S420. In this process, the questions and document contents are collected as one data based on the separator.

In the input question/document embedding step (S420) according to an exemplary embodiment, the processor 1500, in the same manner as the input question embedding step (S320), builds the pre-learning language model based on the language model learned in the step (S230). Input information used in the machine reading comprehension model can be generated by importing the embedding information for the input question/document.

In the machine reading comprehension model learning step (S430) according to an exemplary embodiment, the processor 1500 undergoes a process of supervising the embedding result and intention converted in the question/document embedding step (S420). Through this process, it is possible to build a machine reading comprehension model that searches for answers to queries in financial documents.

In the answer search step (S440) according to an exemplary embodiment, the processor 1500 may create an input vector through a process of embedding an input question/document and predict a search answer result using the learned financial document machine reading comprehension model.

6 is a detailed flowchart of a machine reading comprehension training set generating module according to an embodiment of the present invention.

6 is a detailed flowchart of a method of performing the machine reading comprehension training set generation module (S500) of FIG. 1 according to an embodiment of the present invention. The embodiment of FIG. 6 generates a machine reading comprehension learning set by receiving a financial document and a question/answer, adding new features such as answer position and length, and converting them into an input form for machine reading comprehension model learning.

In the additional feature generation step (S510) according to an exemplary embodiment, the processor 1500 performs a necessary conversion from the corresponding financial document to a form usable as input information in the machine reading comprehension model learning step (S430) through financial questions and answers and financial documents. Performs the process of additionally generating the location of the answer and the length of the answer as features.

According to an exemplary embodiment, in the data structure conversion step (S520), the processor 1500 machine-reads the data generated in the additional feature generation step (S510) and uses it as input information in the model learning step (S430). to do the conversion. As a result, the processor 1500 may generate learning data.

7 is a detailed flowchart of object name-based document filtering and title embedding comparison according to an embodiment of the present invention.

FIG. 7 is a detailed flowchart of a specific method of performing the entity name-based document filtering and title embedding comparison step (S600) of FIG. 1 according to an embodiment of the present invention. The embodiment of FIG. 7 extracts the main entity name for the input query, first filters the financial document including the entity name, compares the title of the document and the embedding of the query, and extracts the most relevant document. show the process

In the entity name-based document filtering step (S610) according to an exemplary embodiment, the processor 1500 primarily filters by determining whether the document title includes the principal entity name based on the principal entity name included in the input question. Do it.

In the embedding comparison between financial document titles and questions (S620) according to an exemplary embodiment, the processor 1500 converts the document title and question primarily filtered in the object name-based document filtering step (S610) to the pre-learning language model. Based on the language model learned in the construction step (S230), embedding information for the input question/document is fetched and compared, and a document having the highest similarity and relation to the question is extracted. As a result of the execution of FIG. 7 , only financial documents with high relevance may be extracted from a plurality of candidate financial documents.

8 is a diagram showing the configuration of an artificial intelligence financial expert system according to an embodiment of the present invention. The system 1000 builds a financial specialized learning model using the input data 1100 and the input database 1200, and calculates a result 1300 through deep text understanding based on intent classification and machine reading comprehension based on this. . The processor 1500 performs the above processes, and the database 1900 stores the entity dictionary, learning data, embedding model, and the like calculated in the above process.

The input data 1100 includes, for example, a finance-specific raw corpus, a finance query answer, a finance-related question, a financial document, and the like.

An example of the input database 1200 is a large-scale business database, a financial query set, a financial machine reading comprehension set, and the like.

The system of FIG. 8 implements a machine learning-based chatbot system for in-depth understanding of real-time financial text and utilization of an external knowledge base, and implements the processes presented in the embodiments of FIGS. 1 to 7 described above.

The system of FIG. 8 collects large-scale business data for implementing AI analyst support services for language processing in the financial domain, and builds base knowledge for data analysis and natural language processing in the financial domain into a dictionary of financial terms.

The system of FIG. 8 develops unstructured financial information summary technology and analyzes causal relationships based on natural language processing in order to extract financial-specific causal relationships from unstructured data.

The system of FIG. 8 builds a learning model for real-time text understanding of financial information in order to build a pre-learning language model for the financial domain, and advances the learning model for real-time financial information support. In addition, the pre-learning language model of the system of FIG. 8 can be lightened according to the detailed area of the financial domain, and the application domain-specific vocabulary knowledge can be refined.

The system of FIG. 8 may include a semantic-based (positive/negative, importance, highlight) analysis module to process financial terminology based on semantics.

The system of FIG. 8 may include an open-source based machine reading comprehension engine for real-time text understanding in order to perform classification of answers and intentions specifically for question and answer based on a financial domain pre-learning model. In addition, Q&A and machine reading comprehension can be advanced to support investment information at the level of financial experts.

9 is a diagram showing the integrated processes of S100, S200, S500, and S600 according to an embodiment of the present invention. The processor 1500 generates an embedding model by performing the processes of S100 and S200. Then, the processor 1500 generates learning data by performing the process of S500.

In addition, the processor 1500 performs document filtering (S610) and embedding comparison (S620) using the entity name dictionary and large-scale business database in step S600 to finally extract documents having a high similarity to the question.

Large-scale business data is the result of collecting domestic and foreign news, policy data, public announcement data, technical data, research reports, etc., and these are unstructured data as an example.

10 is a diagram showing a process of performing the processes of S300 and S400 by applying an embedding model and learning data according to an embodiment of the present invention.

The process of FIGS. 5 and 6 will be integrated and reviewed. S310 and S410 are processes of preprocessing an input finance-related question (or, in the case of S410, a finance-related question input together with a financial document or an extracted document of FIG. 8) using a financial query set or a financial machine reading comprehension training set and learning data. (S11).

The processor 1500 applies the embedding model to the processed result of S11 to embed the input question or document (S320, S420) (S12). Based on this, the processor 1500 may build a supervised learning-based classification model (S330) to generate a financial query intent classification model (S13). Alternatively, the processor 1500 may perform machine reading comprehension model learning (S430) to generate a financial document machine reading comprehension model (S13).

As a result, the processor 1500 classifies the intention of the input question as in S14 (S340) and generates an intention classification result and an answer.

Alternatively, the processor 1500 searches for an answer to the input finance-related question as in S14 and generates an answer extraction result.

When the above-described embodiment is applied, a technique can be applied to increase the embedding effect of financial terms by applying retalking.

For example, by preparing financial terms in advance and processing them based on its own morpheme analyzer and dictionary, it is possible to solve a problem in which specific terms disappear or are separated in the process of processing financial terms.

In addition, retalking is applied in the process of tokenizing the corpus. That is, it is possible to prioritize the financial terminology dictionary so that financial terms or key words are not separated.

In addition, when the above-described embodiment is applied, a correct answer search technology using an external real-time database can be applied to increase the response rate.

When searching in the existing query knowledge database, the following process may be followed for a query having a low probability threshold using an external real-time crawling database. That is, similar documents are searched in an external real-time crawling database. Then, a query is queried to the related document through a machine reading comprehension model, and the answer with the highest probability is found and adopted as a response.

The processor 1500 may include a module that performs learning, creates a model, or performs filtering, comparison, classification, and the like based thereon.

The embodiment of FIG. 10 may further include a summary description of unstructured financial information and a causal relationship analysis based on natural language processing. The system 1000 of FIG. 8 may generate summary data for each market condition and each stock item through a corpus of finance-related news articles. In addition, the system 1000 of FIG. 8 may analyze context/semantic role-based cross-reference relationships and construct a causal knowledge graph.

As a result, the system 1000 of FIG. 8 may perform semantic analysis (positive/negative, importance, highlight)-based conversation processing, generate graph knowledge-based causal analysis results, and provide answers to questions.

Meanwhile, the system 1000 of FIG. 8 may further include an open source based machine reading comprehension engine. This calculates a query-response based on an open pre-learning language model and provides a response corresponding to the query.

In order to improve performance of question-answering and machine reading comprehension, the system 1000 of FIG. 8 may further include a financial domain-specific corpus-based question-answering knowledge automatic extraction module. In addition, the system 1000 of FIG. 8 may further include a close-domain question-answering engine based on machine reading comprehension using a pre-learning language model specific to an application domain.

11 is a diagram illustrating a supervised learning classification model according to an embodiment of the present invention and an explanation of inputs and outputs used in machine learning. 11 shows the types and forms of user input information.

11 is an explanatory diagram of inputs and outputs used when learning a supervised learning-based classification model according to an embodiment in the supervised learning-based classification model learning and using step (S300), showing the type and form of user input information. .

12 is a diagram illustrating input and output used when learning a machine reading comprehension model according to an embodiment of the present invention. 12 shows what types and forms of input information are.

12 is an explanatory diagram of inputs and outputs used when learning a machine reading comprehension model according to an embodiment in the machine reading comprehension model learning and use step (S400), showing the types and forms of input and output information.

13 is an overall configuration diagram of a chatbot system including the main embodiments of the invention described in the present invention.

13 is an overall configuration diagram of a question answer search system using supervised learning and machine reading comprehension methodologies according to a main embodiment of the present invention.

FIG. 13 is a diagram showing detailed components of the processor 1500 performing each of the processes of S100 to S600 described above. These are the domain/corpus-specific tokenizer learning model (1510) and the financial-specific pre-learning model (1520), which are preprocessing tasks.

In addition, the financial text intent classification model 1530 of the processor 1500 may classify the intent of the question and output an answer to the question.

In addition, the related financial document extraction module 1560 of the processor 1500 inputs the extracted document, which is the result of the work performed by the document filtering module S610 and the embedding comparison module S620, to the machine reading comprehension model 1540. The machine reading comprehension model S430 outputs an answer to the question.

The final models that output answers are the machine reading comprehension model 1540 and the financial text intent classification model 1530.

14 is a diagram showing a process of automatically extracting financial information for language processing in the financial domain according to an embodiment of the present invention. It can be applied by specializing in language processing in the financial domain. Corpus data includes analyst research reports, economic market reports, and economic articles. The processor 1500 performs morpheme analysis and builds a dictionary of keywords and compound words. In order to advance the dictionary, knowledge of synonyms is built and new entity names are registered. Synonyms can be treated as synonyms on keywords with different expressions of the same meaning. When registering a new entity name, words related to each other can be combined and registered.

In addition, the system 1000 may build a learning model for real-time financial text understanding. A financial domain-specific pre-learning language model can be constructed and application task performance can be compared using the open pre-learning language model. The system 1000 may combine domain-specific corpus data and a pre-learning model specific to the financial domain and perform encoder training on the language model. In particular, the corpus data includes data generated from phone counseling or chatting counseling, and the system 1000 may pre-process them to learn sentences.

15 is a diagram illustrating a process of classifying intentions for various queries in the financial domain and searching for answers based on a financial term language model according to an embodiment of the present invention.

The foregoing embodiments are summarized as follows.

The processor 1500 of the system 1000 extracts an entity name based on unsupervised learning using the first input data (S710).

The entity name extracted by the processor 1500 is stored in the database 1900 of the system 1000 (S720).

The processor 1500 performs tokenization by applying the object name stored in the database 1900 to the preprocessed result of the first input data or the second input data (S730). The first input data and the second input data are corpus data input with a time difference as an example. The corpus data is finance-specific raw corpus data as an example. Alternatively, as an embodiment, the first input data is corpus data and the second input data is question data.

In extracting entity names, the processor 1500 preprocesses sentences included in the corpus to separate them into initial consonants, neutral consonants, and final consonants, and then converts alphabets included in the sentences into uppercase or lowercase letters to separate words. Also, in the process of separating words, words included in the entity name dictionary and the financial terminology dictionary may be treated as a single word, and retalking may be performed.

The processor 1500 builds a language model including a dictionary of financial terms based on the tokenized result and stores it in the database (S740).

Thereafter, the processor 1500 generates training data using a result of extracting one or more financial documents by applying entity name dictionary to a plurality of candidate financial documents and filtering the documents, and the financial query answer document extracted by the processor 1500 from the corpus. It can be done (S750).

Meanwhile, after receiving question data as input data, the processor 1500 may classify intentions or generate answers corresponding to the question data (S760).

In more detail, in the step of extracting the entity name or performing tokenization, the processor 1500 processes the word included in the entity name dictionary or financial term dictionary stored in the database 1900 as one word without separating it. Retonizing can be performed. In addition, if there is an empty space such as a space (blank) between these words or a symbol is included, the processor 1500 corrects and processes it.

For example, if the word “Samjeonjuju” is included in the entity name dictionary or the financial terminology dictionary, and if a space is included, such as “Samsung stock price” or “Samsung stock price” in the corpus, the processor 1500 selects the word Retalking can be performed to treat it as a single word.

Also, the processor 1500 may sequentially process input data. For example, an embodiment in which there is a plurality of corpus data and both the first and second input data are finance-specific raw corpus data will be described. The processor 1500 processes the first input data to add a word to the entity name dictionary or the financial term dictionary, and then processes the new second input data to identify a specific word. And, if the identified word does not exist in the entity name dictionary or financial term dictionary, it is added.

That is, the processor 1500 may repeatedly perform a process of identifying words not included in the entity dictionary or financial term dictionary from the corpus and adding them to the entity dictionary or financial term dictionary.

Also, the processor 1500 may perform steps S510 and S520 of FIG. 9 . For example, processor 1500 extracts one or more financial documents by applying entity name dictionaries to a number of candidate financial documents to filter the documents. Also, the processor 1500 generates learning data by using the financial question answer document extracted from the corpus.

The financial query answer document is a document including a financial query and a corresponding financial answer.

The processor 1500 may create a feature according to the position or length of the answer included in the financial query answer document, which is reflected in the training data.

Then, after receiving question data, the processor 1500 may classify intentions or generate answers.

As an embodiment, the processor 1500 may apply the entity name dictionary, language model, and learning data to question data to classify and output question intentions of the question data.

As another embodiment, the processor 1500 may generate and output an answer of the question data by applying the entity name dictionary, the language model, and the learning data to the question data. In this case, the processor 1500 may classify the intention of the question and generate an answer of the question data using the result.

16 is a diagram showing in detail a process of performing retorquenizing according to an embodiment of the present invention.

16 shows the process S730 of FIG. 15 in detail.

An object name dictionary 1910 may be generated from the original financial patent corpus through S110/S120/S130. At this time, the entity name dictionary 1910 is based on a corpus.

In addition, a result of applying the tokenizer based on unsupervised learning (S220) and a tokenizing result for an input sentence or document may be generated (S731 and S732).

In addition, morpheme analysis POS (Part-Of-Speech) can be performed on the input sentence or document (S731, S735). The result is calculated (S736).

As a result, key words related to finance are tokenized to be identifiable as words without being separated.

17 is a diagram showing an example of performing financial term-based retalking according to an embodiment of the present invention.

17 shows the results of S732 and S736 of FIG. 16 .

As in S731a, the sentence "I lost my OTP card, but is it possible to increase the transfer limit in the Internet banking app?" is entered. At this time, the result of performing tokenization by applying a dictionary through internal word separation is the same as S732a.

Examining S732a in detail, the system applies an internal dictionary to check and tokenize each word included in the input sentence as in S731a. For example, the words included in the dictionary are "God, Han, OTP, card, lost, transfer, limit, upward, to, internet, banking, app, from, possible", and these words are tokenized It is a word.

As a result of applying the dictionary (1910a), S732a produces a tokenized result such as "I have lost # a new OTP # card, but if I want to increase the transfer # limit, # is it possible in the Internet #banking #app #?" will be. As a result of applying the aforementioned dictionary, "#Han" shows a separate result. Similarly, S732a shows the result of separating words that are meaningfully grouped together in finance, such as "OTP #card", "transfer #limit", and "Internet #banking".

To prevent financial terms from being split into individual words, the system performs financial term-first tokenization on the tokenized words using the financial term dictionary 1920a. As a result, as in S736a, words such as "Shinhan", "OTP Card", "transfer limit", and "Internet banking" become one word so that they are not separated into individual letters.

On the other hand, the system performs a process of adding the corresponding object names to the dictionary in order to construct the token id again for words such as "Shinhan", "OTP Card", "transfer limit", and "Internet banking" (S739). As a process, the words included in the financial terminology dictionary are continuously added to the dictionary, and the financial terms can be meaningfully composed of one word without being separated, and accordingly, the system can provide information suitable for the input sentence or document. there is.

The system or a processor constituting the system may periodically or in real time perform an operation of including proper nouns of financial institutions, card names, promotion names, and financial service names in the financial terminology dictionary. For example, the system provides documents or financial services (various financial services such as insurance, securities, and funds) containing new promotional information about financial services periodically or in real time from each financial institution or financial institutions that have entered into a transaction contract. Including) receives a document promoting the introduction or service launch, or a publicity document for a new card name.

In addition, the system or the processor constituting the system includes words that are included in the title of the received document, classified as keywords, or that do not appear in the entity dictionary but appear repeatedly, in the dictionary of financial terms.

As a result, since new financial terms can be updated in the financial terminology dictionary in real time or periodically, accurate answers to questions or consultations related to the future can be generated.

In addition, since the above-described received documents include explanations of related financial terms, the system or a processor constituting the system parses the received documents so that the contents corresponding to the answers or explanations can be used as answers later.

In addition, while maintaining a dictionary of financial terms, the system continuously manages words limited to a specific financial institution, so that when services are provided to other financial institutions, the words are determined as general words.

For example, as shown in Table 1, a code number is allocated and stored for each financial institution in the financial terminology dictionary.

word Code K bank 00123 K Bank Plus Insurance 00123 C bank 00321 C Bank Happy Fund 00321

In addition, when generating an answer or classifying an intention in relation to K bank, the system processes words having a code value of “00123” as financial terms. In addition, when generating an answer or classifying an intention related to Bank K, the system treats words having a code value of “00321” related to Bank C as non-financial terms.

As a result, the system can generate an answer or classify intentions specific to the service provided by the financial institution in relation to K bank.

Similarly, when generating an answer or classifying an intention in relation to Bank C, words having a code value of “00321” are treated as financial terms. In addition, when generating an answer or classifying an intention in relation to Bank C, the system treats words having a code value of “00123” in relation to Bank K as non-financial terms.

As a result, the system can generate an answer or classify intent when it relates to Bank C, specific to the services provided by that financial institution.

As in the above-described embodiment, by including the relevant financial institution code in the financial terminology dictionary, it is possible to provide a service specialized to a financial institution.

In summary, in the case of answer generation/intention classification related to K bank, the system reflects the financial terms uniquely used by K bank, so accuracy can be increased. The same goes for Bank C.

In addition, when a plurality of financial institutions use the same financial terms, the system does not process the words as financial terms of a specific financial institution, but processes them as general financial terms. For example, in Table 1, if the word "Happy Fund" is used by a number of financial institutions, the system can delete the financial institution code for the word so that the word "Happy Fund" itself is treated as a general financial term. .

In the case of implementing the above-described embodiment, it is possible to supplement the limitations of building a knowledge base of a conventional destination-oriented conversation system and chatbot system and the performance limitations of a search-based question-answering system. Recently, chatbot services using destination-oriented conversation systems and question and answer systems such as customer consultation, employee chatbots, and RPA services have increased in various application domains such as public and distribution, and there are limitations and difficulties in building and managing knowledge databases according to application domains. Therefore, when an embodiment of the present invention is applied or applied, a service capable of classifying intentions for various queries in the financial domain and searching for answers can be provided.

The system 1000 of the present invention includes in the database 1900 a basic dictionary to be applied to analyze a query word, object name, etc., in analyzing unstructured text input. The processor 1500 applies an unsupervised learning-based lexical segmentation (tokenizer) learning method, and through this, it is possible to optimally provide an entity dictionary having the characteristics of a query/index word dictionary specialized for an application domain.

The system 1000 of the present invention additionally collects the financial domain corpus and builds a language model specialized for the financial sector through pre-learning language model learning, minimizing related (financial/investment) complex preprocessing or knowledge building work, and An input processing procedure for common use in various services can be provided. In addition, the system 1000 of the present invention enables performance optimization by providing processes such as deep text understanding and machine reading comprehension, such as question classification and text reasoning.

In addition, the system 1000 of the present invention may classify the intention of the customer's query by applying a supervised learning-based intention classification model. By building an intent classification model and implementing an additional learning process for misclassified/unclassified cases, we constructed a system that can improve the performance of an existing classification model with minimal effort.

In addition, the system 1000 of the present invention applies text reasoning technology based on semantic and intrinsic information to input, and adds a function to search for similar clauses or documents even when an answer cannot be found in the intent classification model, thereby providing knowledge The response rate for answers to queries not defined in the database can be improved.

In addition, the system 1000 of the present invention searches related documents in an external knowledge base in real time through accurate document index and rapid search, or automatically extracts answers to undefined questions from documents and provides them as a result of a question and answer. can do.

Methods according to embodiments of the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the medium may be those specially designed and configured for the present invention or those known and usable to those skilled in computer software.

As described above, although the present invention has been described by the limited embodiments and drawings, the present invention is not limited to the above embodiments, and those skilled in the art in the field to which the present invention belongs can make various modifications and variations from these descriptions. this is possible It should be understood that the foregoing embodiments are illustrative in all respects and not restrictive, and the scope of the present invention will be indicated by the following claims rather than the foregoing detailed description. And it should be construed that all changes and transformable forms derived from the meaning and scope of the claims, as well as their equivalent concepts, are included in the scope of the present invention.

Therefore, the scope of the present invention should not be limited to the described embodiments and should not be defined, and should be defined by the following claims and equivalents to these claims.

1000: system 1500: processor
1900: database

Claims (12)

extracting, by a processor of the system, an entity name based on unsupervised learning using first input data;
storing, by the processor, the extracted object name in a database of the system;
performing tokenization by the processor by applying the entity name stored in the database to the preprocessed result of the first input data or the second input data; and
The processor builds a language model including a dictionary of financial terms based on the tokenized result and stores it in the database, and the step of extracting the entity name or the step of performing the tokenizing
The processor determines the intent for various queries in the financial domain based on the financial terminology language model, including the step of processing, by the processor, a word included in the entity name dictionary stored in the database or the financial terminology dictionary as a single word without separating it. How to categorize and search for answers.
According to claim 1,
The processor classifies intentions for various queries in the financial domain based on a financial term language model, further comprising the step of including a proper noun of a financial institution, a card name, a promotion name, and a financial service name in the financial term dictionary, and how to search for answers.
According to claim 1,
The first and second input data are financial-specific raw corpus data,
The processor identifies a word not included in the entity dictionary or the financial term dictionary from the corpus and adds it to the entity dictionary or the financial term dictionary, based on the financial term language model. How to categorize intent and retrieve answers to various queries.
According to claim 3,
Generating, by the processor, learning data using a result of extracting one or more financial documents by filtering documents by applying the entity name dictionary to a plurality of candidate financial documents and, by the processor, a financial query answer document extracted from the corpus. A method of classifying intentions and retrieving answers to various queries in the financial domain based on a financial terminology model, comprising:
According to claim 4,
The financial query answer document is a document including a financial query and a corresponding financial answer,
The processor classifies intentions for various queries in the financial domain based on a financial term language model, further comprising generating a feature according to the position of the answer or the length of the answer included in the financial query answer document, and How to search for answers.
According to claim 4,
receiving, by the processor, question data; and
The processor applies the entity name dictionary, the language model, and the learning data to the question data to classify and output the intent of the question of the question data, based on the financial terminology language model. How to categorize intent and retrieve answers to various queries.
According to claim 4,
receiving, by the processor, question data; and
The processor generates and outputs an answer to the question data by applying the entity name dictionary, the language model, and the learning data to the question data, wherein various queries in the financial domain are made based on the financial terminology language model. How to categorize intentions and search for answers.
According to claim 1,
The first and second input data are financial-specific raw corpus,
The step of extracting the object name is
The processor pre-processes the sentences included in the corpus to separate them into initial consonants, neutral consonants, and final consonants, and then converts alphabets included in the sentences into uppercase letters or lowercase letters to separate words, further comprising a step of separating words. A method for classifying intentions and retrieving answers for various queries in the financial domain based on
a processor that processes information using input data and an input database; and
A database for storing information generated by the processor,
The processor extracts an unsupervised learning-based entity name using the first input data, and stores the extracted entity name in the database;
The processor performs tokenization by applying the entity name stored in the database to the preprocessed result of the first input data or the second input data,
Based on the tokenized result, the processor builds a language model including a financial terminology dictionary and stores it in the database, and the processor does not separate the entity name dictionary stored in the database or words included in the financial terminology dictionary. A system for classifying intentions for various queries in the financial domain and retrieving answers based on a language model of financial terms that is processed as a single word.
According to claim 9,
The processor classifies intentions for various queries in the financial domain based on a financial terminology language model that includes proper nouns of financial institutions, card names, promotion names, and financial service names in the financial terminology dictionary, and searches for answers. system.
According to claim 9,
The first and second input data are financial-specific raw corpus data,
The processor identifies words not included in the entity dictionary or the financial term dictionary from the corpus and adds them to the entity dictionary or the financial term dictionary, based on a financial terminology language model for various queries in the financial domain. A system that categorizes intent and retrieves answers.
According to claim 11,
The processor generates learning data using a result of extracting one or more financial documents by filtering documents by applying the entity name dictionary to a plurality of candidate financial documents and a financial query answer document extracted by the processor from the corpus. A system for classifying intentions and retrieving answers for various queries in the financial domain based on language models.

Images