CN117875292A

CN117875292A - Financial knowledge intelligent question-answering method, system, terminal equipment and storage medium

Info

Publication number: CN117875292A
Application number: CN202311834301.XA
Authority: CN
Inventors: 赵洋; 朱丽; 陈龙; 包荣鑫; 迟伟明; 王童萱; 林晓绿
Original assignee: Shenzhen Valueonline Technology Co ltd
Current assignee: Shenzhen Valueonline Technology Co ltd
Priority date: 2023-12-27
Filing date: 2023-12-27
Publication date: 2024-04-12

Abstract

The application is applicable to the technical field of artificial intelligence, and provides a financial knowledge intelligent question-answering method, a system, terminal equipment and a storage medium, wherein the method comprises the following steps: acquiring a problem input by a user; constructing an initial prompt according to the problems and a preset prompt template, wherein the initial prompt comprises a reference tool list, and the preset prompt template is constructed based on the gold fusion rule field; and inputting the initial prompt into a large language model, guiding the large language model to generate a target answer result of the problem based on a target reference tool in the reference tool list. The method and the device are applicable to various complex scenes in the financial field, fine adjustment or model fusion of a large language model is not needed, labor cost is greatly reduced, and accuracy of financial knowledge question-answering can be improved.

Description

Financial knowledge intelligent question-answering method, system, terminal equipment and storage medium

Technical Field

The application relates to the technical field of artificial intelligence, in particular to a financial knowledge intelligent question-answering method, a system, terminal equipment and a storage medium.

Background

With the rapid development of natural language technology, the solution based on common problems has been gradually intelligent, and more enterprises utilize a question-answering system to solve various problems of online users without any person. Currently, the question-answering system commonly employs a large language model (Large Language Model, LLM).

However, to implement knowledge questions and answers in a specific domain using LLM requires fine tuning of LLM or knowledge embedding. Whether fine tuning is carried out on LLM or knowledge embedding is adopted, the cost is high, the labor is relatively dependent, and the accuracy of financial knowledge question-answering is low for complex multiple scenes in the field of gold fusion rules.

Disclosure of Invention

The embodiment of the application provides a financial knowledge intelligent question-answering method, a system, terminal equipment and a storage medium, which can improve the accuracy of knowledge question-answering in the field of gold fusion rules and reduce the cost of question-answering implementation.

In a first aspect, an embodiment of the present application provides a method for intelligent question-answering of financial knowledge, including:

acquiring a problem input by a user;

constructing an initial prompt according to the problems and a preset prompt template, wherein the initial prompt comprises a reference tool list, and the preset prompt template is constructed based on the gold fusion rule field;

And inputting the initial prompt into a large language model, guiding the large language model to generate a target answer result of the problem based on a target reference tool in the reference tool list.

In a possible implementation manner of the first aspect, the inputting the initial prompt into a large language model, guiding the large language model to generate a target answer result of the question based on a target reference tool in the reference tool list, includes:

inputting the initial prompt into a large language model, and acquiring an initial answer of the large language model based on the initial prompt;

guiding the large language model to extract key elements of the initial answer, and determining a target reference tool to be used;

and guiding the large language model to generate a target answer result of the problem based on the extracted key elements and the target reference tool.

In a possible implementation manner of the first aspect, the directing the large language model to generate a target answer result of the question based on the extracted key element and the target reference tool includes:

updating the initial prompt according to the key elements and the target reference tool to obtain a target prompt;

And guiding the large language model to carry out intention recognition according to the intention recognition turns and the target prompt, and determining a target answer result of the problem.

In a possible implementation manner of the first aspect, the guiding the large language model to perform intent recognition according to the intent recognition round and the target prompt, determining a target answer result of the question includes:

if the configured intention recognition turns are 1, the target prompt is used for guiding the large language model to perform intention recognition;

updating the target prompt for 1 time according to the result of intention recognition, guiding the large language model to carry out reasoning reply based on the updated target prompt, and determining the result of the reasoning reply as the target reply result of the problem.

if the configured intention recognition turns are more than 1 time, updating the target prompt of the current turn according to the intention recognition turns and the intention recognition result of each turn of the large language model to obtain the target prompt of the next turn;

And guiding the large language model to carry out reasoning reply by utilizing the target prompt of the last round, and determining the result of the reasoning reply as the target reply result of the problem.

when more than one target reference tools are used, the target prompt is input to the large language model, the large language model is guided to respectively carry out reasoning reply based on each target reference tool in the target prompt, and the reasoning reply results corresponding to all the target reference tools are spliced to generate target reply results of the problem.

In a second aspect, an embodiment of the present application provides a financial knowledge intelligent question-answering system, including:

a question acquisition unit for acquiring a question input by a user;

the processing reply unit is used for constructing an initial prompt according to the problem and a preset prompt template, wherein the initial prompt comprises a reference tool list, and the preset prompt template is constructed based on the gold fusion rule field; and the method is also used for inputting the initial prompt into a large language model, guiding the large language model to generate a target answer result of the problem based on a target reference tool in the reference tool list.

In a third aspect, an embodiment of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, where the processor implements the intelligent question-answering method for financial knowledge according to the first aspect when executing the computer program.

In a fourth aspect, embodiments of the present application provide a computer readable storage medium storing a computer program, where the computer program is executed by a processor to implement the intelligent question-answering method according to the first aspect.

In a fifth aspect, embodiments of the present application provide a computer program product, which when run on a terminal device, causes the terminal device to perform the financial knowledge intelligent question-answering method according to the first aspect described above.

In the embodiment of the application, the problem input by the user is acquired, the initial prompt is constructed according to the problem and the preset prompt template constructed based on the gold fusion rule field, the initial prompt is input into the large language model, and the large language model is guided to generate the target answer result of the problem based on the target reference tool in the reference tool list in the initial prompt. According to the method and the device, the target reference tool is identified and determined through guiding the large language model, and the reply result is generated based on the target reference tool, so that the method and the device can be applied to various complex scenes in the financial field, fine adjustment or model fusion on the large language model is not needed, labor cost is greatly reduced, and accuracy of financial knowledge question-answering is improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required for the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flowchart of an implementation of a method for intelligent question-answering of financial knowledge provided in an embodiment of the present application;

fig. 2 is a flowchart of a specific implementation of step S103 in the intelligent question-answering method of financial knowledge provided in the embodiment of the present application;

FIG. 3 is a flowchart of a specific implementation of determining a target reply result in the intelligent question-answering method of financial knowledge provided in an embodiment of the present application;

FIG. 4 is a flowchart of a specific implementation of determining a target reply result according to intent recognition turns and a target prompt in the intelligent question-answering method of financial knowledge provided in the embodiment of the present application;

FIG. 5 is a flowchart of another embodiment of determining a target reply result according to intent recognition turns and a target prompt in the intelligent question-answering method of financial knowledge according to the embodiment of the present application;

FIG. 6 is a block diagram of a financial knowledge intelligent question-answering system provided by an embodiment of the present application;

fig. 7 is a schematic diagram of a terminal device provided in an embodiment of the present application.

Detailed Description

In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system configurations, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

It should be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It should also be understood that the term "and/or" as used in this specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.

As used in this specification and the appended claims, the term "if" may be interpreted as "when..once" or "in response to a determination" or "in response to detection" depending on the context. Similarly, the phrase "if a determination" or "if a [ described condition or event ] is detected" may be interpreted in the context of meaning "upon determination" or "in response to determination" or "upon detection of a [ described condition or event ]" or "in response to detection of a [ described condition or event ]".

In addition, in the description of the present application and the appended claims, the terms "first," "second," "third," and the like are used merely to distinguish between descriptions and are not to be construed as indicating or implying relative importance.

Reference in the specification to "one embodiment" or "some embodiments" or the like means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," and the like in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments" unless expressly specified otherwise. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless expressly specified otherwise.

The LLM technology is a natural language processing technology based on artificial intelligence, and can automatically learn language patterns and semantic information, so as to generate natural and smooth text. The core of LLM technology is a pre-trained neural network model that can process large amounts of text data from which language rules and semantic information are learned, thereby generating natural and fluent text. The large language model can process various natural language tasks, such as text classification, question-answering, dialogue and the like, and is an important path to artificial intelligence. Existing large language models such as ChatGPT, religion all provide interfaces for external calls. However, LLM technology also presents some challenges and limitations. First, it requires a large amount of data to train to achieve high accuracy and fluency. Secondly, large language models cannot be networked and cannot acquire up-to-date information and knowledge.

At present, LLM development and application are mainly divided into two directions: one way is to fine tune the model, adjust the parameters and node values of the original LLM, so that it obtains more updated knowledge, and is more suitable for vertical field application; the other mode is realized based on a knowledge embedding mode, namely, external knowledge is vectorized, then the query of the user is vectorized in the same method, finally, the knowledge required by the LLM for answering the user questions is inquired based on a vector retrieval mode, and the user questions are answered based on the LLM.

The first fine tuning approach described above suffers from two problems: firstly, huge calculation force is needed, and the method is difficult for common small and medium enterprises to bear; secondly, the data needing manual labeling: since fine tuning requires three steps: supervised Fine Tuning (SFT), feedback Model (Reward Model), and human feedback learning (RLHF), each step requires labeling data in a specific format, and thus the cost of acquiring these data simultaneously is quite high. In conclusion, the application landing feasibility of a general enterprise through a fine tuning model is very low. The second knowledge embedding method has the following problems: firstly, for application scenes with more complicated or knowledge fields, the accuracy is low by directly searching vectors, and a plurality of models are needed to cooperate to approach the purpose. Secondly, the search term is single, so that the search accuracy is not high.

Whether fine tuning is carried out on LLM or knowledge embedding is adopted, the cost is high, the labor is relatively dependent, and the accuracy of financial knowledge question-answering is low for complex multiple scenes in the field of gold fusion rules.

In order to solve the above problems, embodiments of the present application provide a method, a system, a terminal device and a storage medium for intelligent question answering of financial knowledge, which are specifically described below.

The intelligent question-answering method of the financial knowledge can be applied to various intelligent devices or servers which need to execute the knowledge question-answering in the financial field, and the intelligent terminal can comprise a mobile phone, a tablet computer, a wearable device, a notebook computer, a desktop computer and the like.

Fig. 1 shows a flow of implementing the intelligent question-answering method of financial knowledge provided in the embodiment of the present application, where the flow of the method includes steps S101 to S103. The specific implementation principle of each step is as follows:

step S101: the problem of user input is obtained.

Generally, the user may input any question, and in this embodiment, the question entered by the user is related to the gold fusion gauge field.

In the embodiment of the present application, the problem input by the user may be in any form, such as audio or text, and the present application does not limit the form of the input problem.

Step S102: and constructing an initial prompt according to the problems and a preset prompt template, wherein the initial prompt comprises a reference tool list, and the preset prompt template is constructed based on the gold fusion rule field.

In this embodiment, a prompt template in the field of gold fusion gauge is constructed in advance. The preset prompt template comprises date, reference tool, history conversation, questions, format requirements to be followed by the large language model answer and other parameter items. Based on the question entered by the user, an initial prompt is constructed. The initial prompt includes a date, a reference tool list, a history session, and a question entered by the user, wherein the reference tool list includes a reference tool name and description. The reference tool is an auxiliary tool in the field of gold fusion rule, has professional property, and covers related reference tools in various aspects such as rules, rule violations, company correlation, financial indexes and the like in a reference tool list.

The initial prompt is the first-round prompt, and in the initial prompt, the history session is null.

In this embodiment, the prompt is an artificial intelligent prompt, which is a method for guiding or exciting a large language model to complete a specific task by using natural language. In the embodiment of the application, the reference tool in the gold fusion rule field is contained in the initial prompt based on the proxy mechanism, so that the large language model is combined with the reasoning process to think about the accuracy of the reference tool, and the accuracy of the large language model on the financial knowledge question-answering processing is improved.

Illustratively, the initial prompt is as follows:

today is { today }. You ask questions, which is a compliant question-answering product developed by group B information technology stock company. Brief description of the drawings questions are answered professionally. Meets the requirements in the questions as much as possible, and answers are in Chinese.

[ reference tool List ]

[ History Session ]

Intent and thinking { thoughts }

[ problem ]

Wherein, { today } is the date of the day, { chat_history } is a history session, the format refers to the history session format, { thick } is the content of the thinking extracted by the first round of LLM questions, and { query } is the user question.

Step S103: and inputting the initial prompt into a large language model, guiding the large language model to generate a target answer result of the problem based on a target reference tool in the reference tool list.

The large language model in the embodiment of the application refers to a deep learning model trained by using a large amount of text data, and can generate natural language text or understand the meaning of the language text. The constructed initial prompt is input to the large language model, the large language model is guided to intelligently generate target answer results of the questions input by the user based on the target reference tools in the reference tool list, the method is applicable to various complex scenes in the financial field, fine adjustment or model fusion of the large language model is not needed, labor cost and annotation data cost can be reduced, and accuracy of financial knowledge questions and answers can be effectively improved.

As a possible implementation manner of the present application, fig. 2 shows a specific implementation flow of step S103 in the financial knowledge intelligent question-answering method provided in the embodiment of the present application, which is described in detail below:

s1031: and inputting the initial prompt into a large language model, and acquiring an initial answer of the large language model based on the initial prompt.

The large language model carries out initial reply based on the initial prompt, and the initial reply needs to follow the format requirement in the initial prompt.

In this embodiment, the initial answer includes the question to be answered by the large language model, whether the reference tool needs to be used, the name of the target reference tool to be used, and the initial thinking of the large language model based on the target reference tool.

S1032: and guiding the large language model to extract key elements of the initial answer, and determining a target reference tool to be used.

S1033: and guiding the large language model to generate a target answer result of the problem based on the extracted key elements and the target reference tool.

In this embodiment, the large language model rewrites the problem by extracting the key elements, and performs search reasoning by combining the extracted key elements with the target reference tool, so that accuracy of intention recognition can be improved, and when the large language model automatically determines that the reasoning response result meets the preset condition, a target response result which is finally fed back to the user is generated.

As a possible implementation manner of the present application, fig. 3 shows a specific implementation flow of guiding the large language model to generate a target answer result of the question based on the extracted key element and the target reference tool in the financial knowledge intelligent question answering method provided by the embodiment of the present application, which is described in detail as follows:

A1: and updating the initial prompt according to the key elements and the target reference tool to obtain a target prompt.

A2: and guiding the large language model to carry out intention recognition according to the intention recognition turns and the target prompt, and determining a target answer result of the problem.

The embodiment can realize the intention recognition of the multi-round agent mechanism, and the intention recognition round can be set according to the actual application requirement. The update times of the target prompt are less than the intended recognition turns.

As a possible implementation manner of the present application, fig. 4 shows a specific implementation flow of determining a target answer result of the question by guiding the large language model to perform intent recognition according to intent recognition turns and the target prompt in the financial knowledge intelligent question answering method provided by the embodiment of the present application, which is described in detail as follows:

b1: if the configuration intention recognition turns are 1, the intention recognition is conducted by using the target prompt to guide the large language model.

B2: updating the target prompt for 1 time according to the result of intention recognition, guiding the large language model to carry out reasoning reply based on the updated target prompt, and determining the result of the reasoning reply as the target reply result of the problem.

For example, when considering time efficiency, the maximum answer round is set to 2 times, wherein the intention recognition round is 1 time, the initial prompt is updated according to the initial answer, the target prompt is obtained, the target prompt guides the large language model to perform intention recognition, the target prompt is reconstructed and updated according to the result of intention recognition, the updated target prompt is used for guiding the large language model to perform reasoning answer, and the result of reasoning answer is determined as the target answer result.

As a possible implementation manner of the present application, fig. 5 shows a specific implementation flow of determining a target answer result of the question by guiding the large language model to perform intent recognition according to intent recognition turns and the target prompt in the financial knowledge intelligent question answering method provided by the embodiment of the present application, which is described in detail as follows:

c1: if the configured intention recognition turns are more than 1 time, updating the target prompt of the current turn according to the intention recognition turns and the intention recognition result of each turn of the large language model to obtain the target prompt of the next turn. The update times of the target prompt are 1 time less than the intended recognition turns.

C2: and guiding the large language model to carry out reasoning reply by utilizing the target prompt of the last round, and determining the result of the reasoning reply as the target reply result of the problem.

For example, when accuracy is considered, setting the maximum answer round to be 3 times, wherein the intention recognition round is 2 times, updating the initial prompt according to the initial answer to obtain a first round of target prompt, carrying out 1 st intention recognition on the first round of target prompt guide large language model, carrying out reconstruction update on the target prompt according to the result of intention recognition to obtain a second round of target prompt, carrying out 2 nd intention recognition on the second round of target prompt guide large language model, carrying out reconstruction update on the second round of target prompt according to the result of 2 nd intention recognition to obtain a third round of target prompt, carrying out 3 rd intention recognition on the third round of target prompt guide large language model, and determining the intention recognition result of the round as a target answer result.

As a possible implementation manner of the method, the initial prompt is updated according to the key element and the target reference tool to obtain a target prompt; when more than one target reference tools are used, the target prompt is input to the large language model, the large language model is guided to respectively carry out reasoning reply based on each target reference tool in the target prompt, and the reasoning reply results corresponding to all the target reference tools are spliced to generate target reply results of the problem.

In this embodiment, without limiting the number of target reference tools selected by the large language model per round, a plurality of target reference tools may theoretically be selected simultaneously. And if the large language model selects a plurality of target reference tools, splicing and summarizing the retrieval results corresponding to each target reference tool to obtain a final target reply result.

In this embodiment, the large language model generates a target reply result in the case where the end condition is satisfied. The end condition may be: the non-initial answer and tool list is empty and the retrieved material is empty, or the maximum number of loop setups is reached.

The reference tool is a pre-built tool related to the financial domain. In this embodiment, interfaces corresponding to the target reference tool are encapsulated, and the reference tool is a key element or a rewritten problem. The purpose of the reference tool is to assist the large language model in answering user questions. The method of constructing the reference tool will be briefly described below by taking a "company-related" reference tool as an example.

The "company related" reference tool first disassembles the recently disclosed periodic reports, such as annual, semi-annual and quaternary reports, etc., of all marketplaces on the whole market: the text portion is broken down into sentences or paragraphs in units of three-level titles (generally, the number of words is greater than 100 words, broken down into a plurality of small paragraphs, and if the number of words is less than 100 words, the words are combined into one paragraph); the form portion forms text based on a particular template, e.g., converts the contents of the form into several records.

Project	2023 1-6 months	2022 month 1-6
			Revenue of business	111,047	115,853
Profit margin	49,105	52,173

Illustratively, the contents in the above table are converted into 4 records: 1. the business income of 2023, 1-6 months is 111047 million yuan; 2. the business income of 2022, 1-6 months is 115853 million yuan; 3. the sum of profit amounts from 1 to 6 months in 2023 is 49105 million yuan; 4. the sum of the profit amounts from 1 to 6 months 2022 is 52173 million yuan.

Through the steps, the recent periodic report of the all-market companies can be disassembled into a plurality of records. Then, a vector model such as m3e, text2vec or OpenAI Embedding service is adopted to convert the disassembled records into vectors with high dimensionality, wherein the normal dimensionality is 768, 1536 and the like. These vectors may be stored in a vector database, such as Milvus, elastic search, etc., for quick retrieval.

When the large language model determines to use a 'company related' reference tool, firstly, preprocessing a user problem, wherein the preprocessing comprises removing stop words, inertial substitution and the like, vectorizing the problem based on the vector model, carrying out similarity calculation with vectors in a vector database, and finally screening TOP N records with the nearest similarity and returning the TOP N records as target reply results. The similarity calculation method can calculate semantic similarity between two texts by adopting a cosine similarity algorithm.

Taking an application scenario as an example, the financial knowledge intelligent question-answering method based on the large language model in this embodiment is described as follows:

acquiring a user input problem: honored Dong Bi you are asking 2023 how the business income of group A is, very thank you, and hope that the business of the company is rising.

The construction hint 1 is as follows:

today 2023-10-08. In a given format, you can answer the following questions using these reference tools 1-7:

1. company-related: when you need to know about the related content of the A-stock marketing company;

input requirements: the input should be a search query;

2. legislation related: when you need to know the legal and legal related content;

input requirements: the input should be a search query;

3. violation cases: when you need to know the relevant information of each supervision organization and the self-discipline punishment company and punishment content;

input requirements: the input should be a search query;

4. financial analysis: when the financial statement data of a company, the liability statement, the cash flow statement and the profit statement are required to be inquired, and the financial situation is judged;

input requirements: the input should be a search query;

Group b related: when you need to know B group company, value consultation, company products (easy to do, know easy, value platform, IPO version, finance version, etc.), board of board Su Mei, etc.;

input requirements: the input should be a search query;

6. the bing_search must be searched: can be used when answering the current event;

input requirements: the input should be a search query;

7. wikipedia: can be used when you need to answer general questions about people, places, companies, facts, historical events, or other topics;

input requirements: the input should be a search query;

the following format is required for reply:

# Previous conversation history historical session, supplementing the ambiguous or incomplete information of the problem;

# Question i need questions to answer;

# thoughts i need do something to answer the question;

# Do I need to use a tool? One of the answers in Yes, no;

the # Action is "one or two of company related, regulation related, violation case, financial analysis, B group related, bing_search, wikipedia, no." the best one you need to use is given as much as possible;

Action Input, # Action Input, input required by the selection tool;

# observing the results returned by the selection tool;

.. (this thinking/action input/observation can be repeated N times);

# Thought i now know the final answer;

# Final Answer of original input question;

starting answer, please make a step-by-step reasoning according to the designated format before giving the answer;

# history session Previous conversation history:

[]

the # Question is Dong Bi you in respect, please ask 2023 how the business income of group A is very thank you, and hope that the business of the company is rising.

##Thought:

The above is the constructed prompt 1. The prompt 1 is input into the LLM model, and the response 1 of LLM is obtained as follows:

i need to use financial analysis tools to query the revenues of group a 2023.

##Do I need to use a tool？:Yes；

# Action, finance analysis;

the business income of group A in 2023;

# Observation, [ A group 2023 years of business income is XX hundred million Yuan ];

# Thought i now know the revenue of group a for 2023;

the revenue of # Final Answer: A group 2023 was XX hundred million yuan.

Based on the above results, the tool was known as "financial analysis", and the input was "business income of group a 2023".

Extracting key elements from the answer 1, wherein the key elements 1 are extracted as follows:

thinking: i need to use financial analysis tools to query the revenues of group a for 2023;

selection tool: financial analysis;

keyword: the business income of group A in 2023;

search results: financial analysis the investor in respect was in good charge, and according to corporate data, the incomes of group A in 2023 were 388,996.33 ten thousand yuan.

According to the extracted key element 1, updating the prompt 1 to obtain a prompt 2, and adding the following contents on the basis of the prompt 1 in the prompt 2:

# thoughts i need to use financial analysis tools to query the revenues of group a for 2023.

LLM reply 1:

# Action, financial analysis

Revenue of group A2023 # Action Input

Financial analysis of # Observation the investor in respect was good, and the incomes of group A in 2023 were 388,996.33 ten thousand yuan according to the corporate data.

##Thought:

Inputting the prompt 2 into the LLM, guiding the LLM to continuously reason and reply, and obtaining the LL reply 2 as follows:

i now know the incomes of group a for 2023.

# Final Answer, the investor in respect, is in good charge, and the incomes of group A in 2023 are 388,996.33 ten thousand yuan according to the company data.

At this time, no selection tool and search result are available, and key element 2 is extracted:

thinking: i now know the incomes of group a for 2023.

Selection tool: []

Keyword:

search results: []

And (5) when the end condition is met, returning the final result to the user: the investor in respect is good, and the business income of the A group in 2023 is 388,996.33 ten thousand yuan according to the company data.

From the above, in the embodiment of the present application, by acquiring the problem input by the user, according to the problem and the preset prompt template constructed based on the gold fusion rule field, constructing an initial prompt, inputting the initial prompt into a large language model, and guiding the large language model to generate a target reply result of the problem based on a target reference tool in a reference tool list in the initial prompt. According to the method and the device, the target reference tool is identified and determined through guiding the large language model, and the reply result is generated based on the target reference tool, so that the method and the device can be applied to various complex scenes in the financial field, fine adjustment or model fusion on the large language model is not needed, labor cost is greatly reduced, and accuracy of financial knowledge question-answering is improved.

It should be understood that the sequence number of each step in the foregoing embodiment does not mean that the execution sequence of each process should be determined by the function and the internal logic of each process, and should not limit the implementation process of the embodiment of the present application in any way.

Corresponding to the method for intelligent question-answering of financial knowledge described in the above embodiments, fig. 6 shows a block diagram of the system for intelligent question-answering of financial knowledge provided in the embodiment of the present application, and for convenience of explanation, only the portions relevant to the embodiments of the present application are shown.

Referring to fig. 6, the financial knowledge intelligent question-answering system includes: a question acquisition unit 61, a process reply unit 62, wherein:

a question acquisition unit 61 for acquiring a question inputted by a user;

a processing reply unit 62, configured to construct an initial prompt according to the question and a preset prompt template, where the initial prompt includes a reference tool list, and the preset prompt template is constructed based on a gold fusion rule field; and the method is also used for inputting the initial prompt into a large language model, guiding the large language model to generate a target answer result of the problem based on a target reference tool in the reference tool list.

As a possible embodiment of the present application, the process reply unit 62 includes:

the initial answer module is used for inputting the initial prompt into a large language model and acquiring an initial answer of the large language model based on the initial prompt;

the first guiding module is used for guiding the large language model to extract key elements of the initial answer and determining a target reference tool to be used;

And the second guiding module is used for guiding the large language model to generate a target answer result of the problem based on the extracted key elements and the target reference tool.

As a possible implementation manner of the present application, the second guiding module includes:

the prompt updating sub-module is used for updating the initial prompt according to the key elements and the target reference tool to obtain a target prompt;

and the intention recognition sub-module is used for guiding the large language model to carry out intention recognition according to the intention recognition turns and the target prompt, and determining a target answer result of the problem.

As a possible implementation manner of the present application, the intention recognition submodule is specifically configured to:

As a possible implementation manner of the present application, the intention recognition sub-module is further specifically configured to:

and the reply determination submodule is used for inputting the target prompt language into the large language model when more than one target reference tool is used, guiding the large language model to respectively carry out reasoning reply based on each target reference tool in the target prompt language, and generating target reply results of the problem after the reasoning reply results corresponding to all the target reference tools are spliced.

It should be noted that, because the content of information interaction and execution process between the above systems/units is based on the same concept as the method embodiment of the present application, specific functions and technical effects thereof may be referred to in the method embodiment section, and will not be described herein again.

Embodiments of the present application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the financial knowledge intelligent question-answering methods as represented in fig. 1 to 5.

The embodiment of the application also provides a terminal device, which comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the steps of any one of the financial knowledge intelligent question-answering methods shown in fig. 1 to 5 are realized when the processor executes the computer program.

The embodiments of the present application also provide a computer program product which, when run on a terminal device, causes the terminal device to perform the steps of implementing any one of the financial knowledge intelligent question-answering methods as represented in fig. 1 to 5.

Fig. 7 is a schematic diagram of a terminal device according to an embodiment of the present application. As shown in fig. 7, the terminal device 7 of this embodiment includes: a processor 70, a memory 71, and a computer program 72 stored in the memory 71 and executable on the processor 70. The processor 70, when executing the computer program 72, implements the steps of the various embodiments of the financial knowledge intelligent question-answering method described above, such as steps S101 to S103 shown in fig. 1. Alternatively, the processor 70, when executing the computer program 72, performs the functions of the modules/units of the system embodiments described above, such as the functions of the units 61-62 shown in fig. 6.

By way of example, the computer program 72 may be partitioned into one or more modules/units that are stored in the memory 71 and executed by the processor 70 to complete the present application. The one or more modules/units may be a series of computer readable instruction segments capable of performing the specified functions describing the execution of the computer program 72 in the terminal device 7.

The terminal device 7 may include, but is not limited to, a processor 70, a memory 71. It will be appreciated by those skilled in the art that fig. 7 is merely an example of the terminal device 7 and does not constitute a limitation of the terminal device 7, and may include more or less components than illustrated, or may combine certain components, or different components, e.g. the terminal device 7 may further include input-output devices, network access devices, buses, etc.

The processor 70 may be a central processing unit (Central Processing Unit, CPU), but may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSPs), application specific integrated circuits (Application Specific Integrated Circuit, ASICs), off-the-shelf programmable gate arrays (Field-Programmable Gate Array, FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory 71 may be an internal storage unit of the terminal device 7, such as a hard disk or a memory of the terminal device 7. The memory 71 may be an external storage device of the terminal device 7, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card) or the like, which are provided on the terminal device 7. Further, the memory 71 may also include both an internal storage unit and an external storage device of the terminal device 7. The memory 71 is used for storing the computer program as well as other programs and data required by the terminal device. The memory 71 may also be used for temporarily storing data that has been output or is to be output.

It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the system is divided into different functional units or modules to perform all or part of the above-described functions. The functional units and modules in the embodiment may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit, where the integrated units may be implemented in a form of hardware or a form of a software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working process of the units and modules in the above system may refer to the corresponding process in the foregoing method embodiment, which is not described herein again.

The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the present application implements all or part of the flow of the method of the above embodiments, and may be implemented by a computer program to instruct related hardware, where the computer program may be stored in a computer readable storage medium, where the computer program, when executed by a processor, may implement the steps of each of the method embodiments described above. Wherein the computer program comprises computer program code which may be in source code form, object code form, executable file or some intermediate form etc. The computer readable medium may include at least: any entity or system capable of carrying computer program code to a system/terminal device, recording medium, computer Memory, read-Only Memory (ROM), random access Memory (RAM, random Access Memory), electrical carrier signals, telecommunications signals, and software distribution media. Such as a U-disk, removable hard disk, magnetic or optical disk, etc. In some jurisdictions, computer readable media may not be electrical carrier signals and telecommunications signals in accordance with legislation and patent practice.

In the foregoing embodiments, the descriptions of the embodiments are emphasized, and in part, not described or illustrated in any particular embodiment, reference is made to the related descriptions of other embodiments.

The above embodiments are only for illustrating the technical solution of the present application, and are not limiting; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application, and are intended to be included in the scope of the present application.

Claims

1. The intelligent question-answering method for the financial knowledge is characterized by comprising the following steps of:

acquiring a problem input by a user;

2. The method of claim 1, wherein the inputting the initial prompt into a large language model directs the large language model to generate a target answer result for the question based on a target reference tool in the reference tool list, comprising:

3. The method of claim 2, wherein the directing the large language model to generate a target answer result for the question based on the extracted key elements and the target reference tool comprises:

4. The method of claim 3, wherein the directing the large language model to perform intent recognition based on intent recognition turns and the target prompt, determining a target answer result for the question, comprises:

5. The method of claim 3, wherein the directing the large language model to perform intent recognition based on intent recognition turns and the target prompt, determining a target answer result for the question, comprises:

6. The method of claim 2, wherein the directing the large language model to generate a target answer result for the question based on the extracted key elements and the target reference tool comprises:

7. A financial knowledge intelligent question-answering system, comprising:

a question acquisition unit for acquiring a question input by a user;

8. The system of claim 7, wherein the process reply unit comprises:

9. A terminal device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the financial knowledge intelligent question-answering method according to any one of claims 1 to 6 when the computer program is executed by the processor.

10. A computer readable storage medium storing a computer program, wherein the computer program when executed by a processor implements the financial knowledge intelligent question-answering method according to any one of claims 1 to 6.