CN118394890A - Knowledge retrieval enhancement generation method and system based on large language model - Google Patents
Knowledge retrieval enhancement generation method and system based on large language model Download PDFInfo
- Publication number
- CN118394890A CN118394890A CN202410853156.8A CN202410853156A CN118394890A CN 118394890 A CN118394890 A CN 118394890A CN 202410853156 A CN202410853156 A CN 202410853156A CN 118394890 A CN118394890 A CN 118394890A
- Authority
- CN
- China
- Prior art keywords
- target
- query
- text
- document
- language model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 46
- 239000012634 fragment Substances 0.000 claims abstract description 35
- 238000012163 sequencing technique Methods 0.000 claims abstract description 12
- 239000013598 vector Substances 0.000 claims description 49
- 238000012545 processing Methods 0.000 claims description 22
- 238000004590 computer program Methods 0.000 claims description 15
- 238000006243 chemical reaction Methods 0.000 claims description 8
- 238000005457 optimization Methods 0.000 claims description 8
- 238000000605 extraction Methods 0.000 claims description 7
- 238000003860 storage Methods 0.000 claims description 5
- 230000004927 fusion Effects 0.000 claims description 4
- 238000003062 neural network model Methods 0.000 claims description 4
- 238000005516 engineering process Methods 0.000 abstract description 6
- 238000010586 diagram Methods 0.000 description 23
- 230000008569 process Effects 0.000 description 11
- 238000004364 calculation method Methods 0.000 description 7
- 238000012937 correction Methods 0.000 description 6
- 230000007246 mechanism Effects 0.000 description 5
- 230000004048 modification Effects 0.000 description 5
- 238000012986 modification Methods 0.000 description 5
- 238000012549 training Methods 0.000 description 5
- 239000003623 enhancer Substances 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000010276 construction Methods 0.000 description 3
- 239000000284 extract Substances 0.000 description 3
- 230000004075 alteration Effects 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000003058 natural language processing Methods 0.000 description 2
- 238000011176 pooling Methods 0.000 description 2
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000004898 kneading Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a knowledge retrieval enhancement generation method and a system based on a large language model, wherein the knowledge retrieval enhancement generation method comprises the following steps: converting the query question text into a target query question; extracting key prompt words of a target query problem; searching a document set to be matched in a knowledge base according to a target query problem based on a document search enhancement model; obtaining target documents based on the correlation degree of each document to be matched and the target query problem; matching the target document based on the key prompt words to obtain a target text segment; sequencing the target text fragments to obtain a sequenced text fragment sequence; and inputting the target query questions and the ordered text fragment sequences into the text generation enhancement model as an input set to obtain an answer text output by the text generation enhancement model. The application combines the document retrieval and language generation technology, can provide more accurate and comprehensive answers, and simultaneously uses a large-scale language model, so that the generated answer text is more natural and smooth.
Description
Technical Field
The application relates to the technical field of computers, in particular to a knowledge retrieval enhancement generation method and system based on a large language model.
Background
Knowledge retrieval enhancement generation is a technical framework combining retrieval technology and a generation model, and aims to improve performance of Natural Language Processing (NLP) tasks, in particular to tasks such as open domain dialogue, question-answering systems, text generation and the like. The core idea of knowledge retrieval enhancement generation is to enhance the generation capacity of a model by utilizing a large amount of external knowledge or information, so that the problems of insufficient knowledge coverage, poor long tail phenomenon processing and the like possibly faced by the traditional end-to-end generation model are overcome.
In the prior art, knowledge retrieval enhancement generation mainly comprises a retriever, a pre-training language model and a generator, wherein the retriever is used for retrieving related information from a large-scale document library, a knowledge base or an embedded index of the pre-training language model; the pre-trained language model provides a powerful semantic understanding basis. Through pre-training, models learn rich language representations that can be used to calculate the similarity between documents or sentences, thereby selecting the most relevant candidate information during the retrieval phase; the generator generates a consistent, accurate response or answer from the retrieved knowledge, rather than just copying the retrieved segments.
The main problem of the current constraint knowledge retrieval enhancement generation is that the retriever can not accurately retrieve the most relevant documents and text blocks only by calculating the similarity of the documents or text blocks, and the text blocks with extremely high similarity are uniformly input into a pre-trained large language model and a generator for processing, so that the complexity of the system and the consumption of computing resources are doubled. Therefore, how to search a large number of documents and text blocks with high similarity and associated documents and text blocks is a key for further improving the problems of illusion, long tail phenomenon and the like of the pre-training large language model.
Disclosure of Invention
The application provides a knowledge retrieval enhancement generation method and a system based on a large language model, which are used for providing more accurate and comprehensive answers and enabling generated answer texts to be more natural and smooth.
The technical scheme for solving the technical problems is as follows:
A knowledge retrieval enhancement generation method based on a large language model comprises the following steps:
Converting the received query question text into a target query question;
splitting the target query problem, and extracting key prompt words of the target query problem;
searching a document set to be matched in a user private knowledge base according to the target query problem based on a document search enhancement model; the document set to be matched comprises at least two documents to be matched;
Obtaining a target document based on the correlation degree of each document to be matched and the target query problem; the correlation degree of the target document and the target query problem is greater than a first threshold, and the target document comprises at least two text segments;
Matching the target document based on the key prompt words to obtain target text fragments in the target document; the matching degree of the target text segment and the key prompt word is larger than a second threshold;
Sequencing the target text fragments to obtain a sequenced text fragment sequence;
Inputting the target query questions and the ordered text fragment sequences into a text generation enhancement model as an input set to obtain answer texts output by the text generation enhancement model; the text generation enhancement model is constructed based on a large language model.
The application also provides a knowledge retrieval enhancement generation system based on the large language model, which comprises the following steps:
the conversion module is used for converting the received query question text into a target query question;
the keyword extraction module is used for splitting the target query problem and extracting the keyword prompt words of the target query problem;
The document retrieval module is used for retrieving a document set to be matched from a user private knowledge base according to the target query problem based on a document retrieval enhancement model; the document set to be matched comprises at least two documents to be matched;
The document matching module is used for obtaining target documents based on the correlation degree of each document to be matched and the target query problem; the correlation degree of the target document and the target query problem is greater than a first threshold, and the target document comprises at least two text segments;
The text segment matching module is used for matching the target document based on the key prompt words to obtain a target text segment in the target document; the matching degree of the target text segment and the key prompt word is larger than a second threshold;
The text segment sequencing module is used for sequencing the target text segments to obtain a sequenced text segment sequence;
the answer text generation module is used for inputting the target query questions and the ordered text fragment sequences into a text generation enhancement model as an input set to obtain answer texts output by the text generation enhancement model; the text generation enhancement model is constructed based on a large language model.
The application also provides electronic equipment, which comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the processor realizes the knowledge retrieval enhancement generation method based on the large language model when executing the program.
The present application also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the knowledge retrieval enhancement generation method based on a large language model of any one of the above.
The application also provides a computer product, wherein a computer program is stored on the computer product, and the computer program realizes the knowledge retrieval enhancement generation method based on the large language model when being executed by a processor.
The embodiment of the application converts the query question text into the target query question; extracting key prompt words of a target query problem; searching a document set to be matched in a knowledge base according to a target query problem based on a document search enhancement model; obtaining target documents based on the correlation degree of each document to be matched and the target query problem; matching the target document based on the key prompt words to obtain a target text segment; sequencing the target text fragments to obtain a sequenced text fragment sequence; the target query questions and the ordered text fragment sequences are used as input sets to be input into the text generation enhancement model, and answer texts output by the text generation enhancement model are obtained, so that more accurate and comprehensive answers can be provided by combining document retrieval and language generation technologies, and meanwhile, a large-scale language model is used, so that the generated answer texts are more natural and smooth.
Drawings
FIG. 1 is a flow chart of a knowledge retrieval enhancement generation method based on a large language model provided by the application;
FIG. 2 is a schematic diagram of an execution flow of a knowledge retrieval enhancement generation method according to an embodiment of the present application;
FIG. 3 is a block diagram of a knowledge retrieval enhancer provided by an embodiment of the application;
fig. 4 is a schematic diagram of a text feature similarity calculation method according to an embodiment of the present application;
FIG. 5 is a block diagram of a problem alert word disassembler provided by an embodiment of the present application;
FIG. 6 is a schematic diagram of a method for calculating similarity of problem-solving words according to an embodiment of the present application;
FIG. 7 is a block diagram of a text generation enhancer provided by an embodiment of the present application;
FIG. 8 is a block diagram of a knowledge retrieval enhancement generation system based on a large language model provided by the present application;
Fig. 9 is a schematic structural diagram of an electronic device provided by the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
In the description of the present application, the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more of the described features. In the description of the present application, the meaning of "a plurality" is two or more, unless explicitly defined otherwise.
In the description of the present application, the term "for example" is used to mean "serving as an example, instance, or illustration. Any embodiment described as "for example" in this disclosure is not necessarily to be construed as preferred or advantageous over other embodiments. The following description is presented to enable any person skilled in the art to make and use the application. In the following description, details are set forth for purposes of explanation. It will be apparent to one of ordinary skill in the art that the present application may be practiced without these specific details. In other instances, well-known structures and processes have not been described in detail so as not to obscure the description of the application with unnecessary detail. Thus, the present application is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.
Referring to fig. 1, fig. 1 is a flowchart of a knowledge retrieval enhancement generation method based on a large language model according to an embodiment of the present application, and specifically includes steps 10 to 70:
step 10, converting the received query question text into a target query question;
step 20, splitting the target query problem, and extracting key prompt words of the target query problem;
step 30, searching a document set to be matched in a user private knowledge base according to a target query problem based on a document search enhancement model; the document set to be matched comprises at least two documents to be matched;
Step 40, obtaining target documents based on the correlation degree of each document to be matched and the target query problem; the correlation degree of the target document and the target query problem is larger than a first threshold value, and the target document comprises at least two text fragments;
Step 50, matching the target document based on the key prompt words to obtain a target text segment in the target document; the matching degree of the target text segment and the key prompt word is larger than a second threshold;
step 60, sequencing the target text fragments to obtain a sequenced text fragment sequence;
Step 70, inputting the target query questions and the ordered text fragment sequences as input sets into a text generation enhancement model to obtain answer texts output by the text generation enhancement model; the text generation enhancement model is built based on a large language model.
Optionally, referring to fig. 2, fig. 2 is a schematic flow chart of an execution flow of the knowledge retrieval enhancement generation method according to an embodiment of the present application, in an Input stage, a query question text (Input) of a user is received, and the query question text is converted into an explicit question. I.e., target query Question (Input Question).
Further, in the document retrieval stage, relevant documents are retrieved from a private knowledge base of a user according to a target query problem (Input Question) by using a document retrieval enhancement model, and a document set is to be matched.
Further, according to the correlation degree of each document to be matched and the target query problem, a target document is obtained, wherein the correlation degree of the target document and the target query problem is larger than a first threshold, the target document comprises at least two sections of text fragments, and the first threshold is set according to practice.
Further, the system splits the target query problem through the problem prompt word splitting model, and extracts the key prompt words of the target query problem.
Further, matching is carried out according to the key prompt words and the target document to obtain target text fragments in the target document, the matching degree of the target text fragments and the key prompt words is larger than a second threshold value, namely the text fragments in the most relevant document are further screened out, and the second threshold value is set according to the actual situation.
Further, the target text segments are ordered to obtain an ordered text segment sequence, so that the text segment most relevant to the key prompt word of the target query problem is determined.
Further, in the generation stage, the target query questions and the ordered text fragment sequences are input into a text generation enhancement model as an input set, answer texts output by the text generation enhancement model are obtained and provided for a user, and the text generation enhancement model is constructed based on a large language model.
The embodiment of the application converts the query question text into the target query question; extracting key prompt words of a target query problem; searching a document set to be matched in a knowledge base according to a target query problem based on a document search enhancement model; obtaining target documents based on the correlation degree of each document to be matched and the target query problem; matching the target document based on the key prompt words to obtain a target text segment; sequencing the target text fragments to obtain a sequenced text fragment sequence; the target query questions and the ordered text fragment sequences are used as input sets to be input into the text generation enhancement model, and answer texts output by the text generation enhancement model are obtained, so that more accurate and comprehensive answers can be provided by combining document retrieval and language generation technologies, and meanwhile, a large-scale language model is used, so that the generated answer texts are more natural and smooth.
Further, referring to fig. 3, fig. 3 is a block diagram of a knowledge retrieval enhancer provided by an embodiment of the present application, for a document retrieval enhancement model:
optionally, each document to be matched is processed according to the document retrieval enhancement model, so that document content in each document to be matched is converted into a first document content feature vector, that is, in a document embedding stage, for each document in the document library, the document retrieval enhancement model performs one-time embedding operation to convert the document content into a vector form.
Optionally, feature extraction is performed on the first document content feature vector of each document to be matched according to a preset neural network model to obtain a second document content feature vector of each document to be matched, that is, in a document feature extraction stage, after the document is embedded, the document retrieval enhancement model uses a Convolutional Neural Network (CNN) to further process the document vector, and more useful feature vectors are extracted.
Optionally, the target query question is processed according to the document retrieval enhancement model, so as to convert the target query question into a first query question feature vector, namely a question embedding stage, and the input target query question passes through an embedding layer (Embedding) to convert text information of the target query question into a vector form. This process typically involves Word embedding (Word Embedding) techniques, such as Word2Vec or GloVe.
Optionally, feature extraction is performed on the first query problem feature vector according to the neural network model to obtain a second query problem feature vector of the target query problem, that is, after the problem is embedded, the model may perform feature aggregation operation, which may include pooling (Pooling), attention mechanism (Attention), and other methods, so as to extract the most important feature vector from the problem vector.
Optionally, determining the document set to be matched according to the second document content feature vector and the second query problem feature vector specifically includes: calculating similarity values of second document content feature vectors and second query problem feature vectors of all the documents to be matched based on a preset model; collecting documents to be matched corresponding to the target similarity value to obtain a document set to be matched; the target similarity value is a similarity value which is larger than or equal to a preset threshold value, the preset model is a multi-scale fusion feedforward network, namely the multi-scale fusion feedforward network compares the feature vector of the problem with the feature vector of each document in the document library, and the similarity between the feature vector and the document library is calculated.
Further, referring to fig. 4, fig. 4 is a schematic diagram of a text feature similarity calculation method provided by an embodiment of the present application, and a Multi-scale fusion feed forward Network (MSFN) is a Network structure for feature enhancement. It consists of two connected paths: a cue word feature calculation path and a document feature vector calculation path. The cue word feature calculation path extracts detail information by performing conventional convolution on a specific input feature channel, and generates feature vectors with the same size as the original input feature vectors. The document feature vector computation path captures more global information using a larger convolution kernel, generating feature vectors of the same size as the hint word feature computation path. The feature vectors of the two paths gain complementary knowledge through information exchange, improving features at different scales in the network.
The MSFN model aims at extracting document information according to the prompt words of the user problems, and comprises two paths of prompt word feature calculation and document vector feature. The document vector computation path uses a conventional 3 x3 depth convolution to extract detailed information, generating feature vectors of the same size as the original input feature vector. On the other hand, the hint word feature computation path captures more global information using a larger 5 x5 convolution kernel, generating feature vectors of the same size as the document vector computation. In order to integrate the prompt word feature and the document feature information, feature vectors of the two branches are fused. The design of a multi-scale fused feed forward network allows features to be enhanced on multiple scales, which is a key advantage of MSFNs over other architectures. By exchanging information between the hint word features and the document vector features, the MSFN can improve the features on different scales and enhance the document retrieval enhancement effect.
Further, referring to fig. 5, fig. 5 is a structural diagram of a problem prompt word disassembler provided in an embodiment of the present application, for a problem prompt word disassembler model:
The problem prompt words are disassembled into a plurality of key words by means of an NFA mechanism according to the characteristics between the calculation and extraction prompt words and the document context.
Further, referring to fig. 6, fig. 6 is a schematic diagram of a method for calculating similarity of problem resolution words provided by the embodiment of the present application, NFA is a feature diagram of linear frequency aware attention mechanism processing feature vectors xΣrχ { c×h×w } through two paths, where R { c×h×w } represents input feature diagrams, C is a channel number (feature depth), and H and W are the number of words in a word vector matrix and vector dimensions, respectively. The first path converts the X-generated Query (Query) from X through a series of convolution operations (including 1X1 and 3X 3 depth-separable convolutions) by a 1X1 and 3X 3 depth-separable convolutions layer, the Query QεR { 4CxH2 } meaning that the Query vector is 4 times extended in the channel dimension, keeping the spatial dimension unchanged. The second path performs discrete wavelet transform on X, decomposes it into four frequency domains, and generates keys (Key) and values (Value) through two sets of 1×1 and 3×3 convolution layers, the keys K ε R {16C×H/2×W/2}, the values V ε R {16C×H/2×W/2} indicate a 16-fold expansion in the channel dimension while the spatial dimensions are halved, reflecting the compression of high frequency information. At the same time, the local context downsampling feature Xc is inverse wavelet transformed (IDWT), according to wavelet theory, each detail of the original input X can be preserved. The query, key and value are rescaled to be ˆ Q εR {4C×HW }, ˆ K εR { HW×4C } and ˆ V εR { HW×4C } for matrix multiplication. The introduction of the process by which DWT-convolution-IDWT is implemented in a nonlinear frequency-aware attention mechanism is to capture features at different frequency levels and enhance the modeling capabilities of the model for local contexts and global structures. Through the mechanism, the model can more effectively integrate spatial information with different scales, and stronger local context modeling and expanded receptive fields are realized.
Further, referring to fig. 7, fig. 7 is a block diagram of a text generation enhancer provided in an embodiment of the present application, and for a text generation enhancement model:
questions and contexts: one problem (Q) is combined with multiple contexts (C1, C2, CN). The question represents the goal or intent of the generation, while the context provides the relevant information needed for the generation. The result of this step is the generation of a series of Question-prompter-context pairs (question+context 1, question+context 2,...
Encoder (Encoder): the question-prompting word-context pairs are processed multiple times to extract deeper information.
Circulation connection (Recurrent Connection): allowing the model to trace back and utilize the previous output information during the generation process.
Large language model text generation: these question-context pairs, after encoder processing, are input into a large language model, which is responsible for generating the final answer. This answer is generated based on all the information provided and the context, and should be able to answer the question that was originally presented.
In one embodiment, after inputting the target query question and the ordered text segment sequence as an input set into the text generation enhancement model to obtain the answer text output by the text generation enhancement model, the method further includes:
And acquiring a database to be queried and query problem text input by a user aiming at the database.
In the step, the knowledge retrieval enhancement generation system based on the large language model can comprise a database module, a prompt word engineering module, a knowledge graph construction module, a large language model module, an output module and an error correction module. The database module can be connected with the database to be queried, namely can be connected to the database to be queried, and queries the database by executing query sentences to obtain query results. The prompt word engineering module is mainly used for constructing the prompt word engineering of the large language model. The knowledge graph construction module is mainly used for reading tables, field information and the like in the database to construct the knowledge graph of the database. The large language model module mainly generates query sentences according to text input by a user and prompt word engineering. The output module is mainly used for returning the query statement generated by the large language model module and the query result of the database module. The error correction module is mainly used for correcting the error query statement through manual work (i.e. professional staff), namely, correcting the error query statement generated by the large language model.
Based on the description of the knowledge retrieval enhancement generation system based on the large language model, when a certain database is actually required to be queried, the certain database can be recorded as a database to be queried, the database to be queried can be connected through a database module, then the database to be queried can be obtained, and meanwhile, the related information of the database can be obtained. The type of the database to be queried can be determined according to actual situations, and can be a Mysql database, an sql database, an Oracle database and the like. The relevant information of the database to be queried may include relevant information of the database itself and relevant information of a data table in the database, wherein the relevant information of the database itself may include, for example, a database name, a database type, etc., and the relevant information of the data table may include, for example, table name, table size, table structure definition, etc. of the data table.
In addition, the knowledge retrieval enhancement generation system based on the large language model can further comprise an acquisition module, and the acquisition module can receive query question text input by a user on a display interface of the knowledge retrieval enhancement generation system or platform based on the large language model. The query question text is mainly aimed at the question of asking or inquiring or searching the database to be queried, and the database to be queried is connected through the database module, so that the text content of the corresponding database can be reduced in the query question text, namely the query question text can be a relatively simple text, such as 'please help me query mathematical achievements of all students', wherein the query question text does not need to comprise contents such as query sentences, and the technical threshold of users using the database to query or search can be reduced.
Meanwhile, the acquisition module can acquire the database to be queried and the related information of the database to be queried from the database module.
And constructing prompt word engineering according to the related information of the database.
In this step, after the related information of the database is obtained, a prompt word project can be constructed through the related information of the database, and the prompt word is mainly used for guiding the subsequent large language model to generate a more accurate query sentence which is more in line with the actual situation according to the prompt word project.
When the term project is constructed by the relevant information of the database, for example, the relevant information of the database may be added to the term project, or constraint conditions for constraining the generated query sentence, which are further determined by the relevant information of the database, may be added, for example, the format of the query sentence generated by constraint, the number of query sentences generated by constraint, or the dependency relationship of the data table in the database, which is further determined by the relevant information of the database, may be added to the term project. In a word, the prompt word engineering can be constructed through the related information of the database.
Inputting the query problem text and the prompt word engineering into a large language model, converting the text into query sentences, and determining target query sentences corresponding to the query problem text; the prompt word engineering is used for guiding the large language model to generate query sentences according to the prompt word engineering.
Among them, the advent of large language models (or called large models) has brought about many revolutionary changes, which have surprisingly achieved effects in intelligent question-answering, natural language understanding, image recognition, etc. by using more parameters or more complex neural networks, so that the large language models are also used in the database query in this embodiment to reduce the technological threshold of users for database queries.
The large language model in this embodiment, large Language Model, is called LLM model for short. The type of the large language model can be specifically set according to practical situations, and can be Qwen series models, llama2 series models and the like. The large language model is mainly used for converting texts into query sentences, wherein the query sentences can be sql query sentences, and then the large language model can be used for converting natural language texts in the field of databases into structured query sentences sql which can be executed in relational databases. In addition, the large language model may be trained in advance, and the training process of the large language model is not described here.
After obtaining the query question text (input) input by the user and the prompt word engineering (prompt) constructed by the related information of the database, the query question text and the prompt word engineering may be input into a large language model (or a large language model module), where the large language model may convert the query question text into a query sentence according to the information included in the prompt word engineering, and generate a target query sentence corresponding to the query question text, where the target query sentence may be, for example, an sql query sentence/sql sentence.
It should be noted that, the target query term is generally a query term for the text of the query question, and the performance of the target query term is generally optimal, such as the fastest query efficiency or the highest query accuracy.
And inquiring in the database according to the target inquiry statement to determine an inquiry result.
In this step, after the target query sentence corresponding to the query question text is obtained through the large language model module, the target query sentence may be transmitted to the output module for output, so that the user may learn the currently generated target query sentence. Meanwhile, the large language model module can also input the target query statement into the database module, and the target query statement is executed in the connected database to be queried through the database module so as to query corresponding contents in the target query statement in the database, thereby obtaining a query result.
The query results can comprise query failure and query success, and if the query fails, the database module can also transmit the query results of the query failure to the output module for output, so that a user can timely acquire the query failure and then perform the next error correction operation. If the query is successful, the query result can also comprise the queried data, and the same database module can acquire the queried data and transmit the queried data to the output module for output, so that a user can acquire the queried data in time for further data processing and other operations.
Although the database query is described as an example in the above embodiments, the database may be substantially searched (similar to the query), added, deleted, modified, and the like. The specific implementation of these operations is similar to that of the present embodiment, and will not be described in detail here.
In the embodiment, a database to be queried and query question text input by a user aiming at the database are obtained, prompt word engineering is constructed according to related information of the database, then the query question and the prompt word engineering are input into a large language model for conversion processing from text to query sentence, a target query sentence corresponding to the query question text is determined, and then query is performed in the database according to the target query sentence to determine a query result; the prompt word engineering is used for guiding the large language model to generate query sentences according to the prompt word engineering. In the method, a user can obtain a query sentence and query a database only by inputting a query problem text, and the query problem text is a relatively simple text, so that the user does not need to know the writing technology of the query sentence of the database, and the technical threshold of the user for querying the database can be reduced; meanwhile, the method can realize the inquiry of the database without writing database inquiry sentences by a professional database operator, so that the workload of the professional database operator can be reduced, and the working efficiency is improved; furthermore, the prompt word engineering which can be constructed through the related information of the database is added in the process of generating the query statement by the large language model, and the large language model can be guided to generate the query statement which is more accurate and meets the requirements through the prompt word engineering, so that the accuracy of a query result obtained when the database is queried through the query statement can be improved.
In one embodiment, the method may further include the steps of:
If the query result is that the query fails, carrying out error correction processing on the target query statement to determine a new target query statement; and re-inquiring in the database according to the target inquiry statement to determine a new inquiry result.
In this step, if the post-query fails when the target query statement generated by the large language model queries the database, it indicates that the target query statement generated by the large language model may have an error, and then the target query statement may be output to a professional operating on the database.
The output module may then input the new target query statement into the database module, and re-execute the new target query statement in the connected database via the database module to obtain a new query result. The new query result is generally successful, and if the query is failed, the error correction process can be continued until the query is successful.
In this embodiment, by correcting the error query statement generated by the large language model, the success rate of querying the database can be improved. In addition, because the query statement generated by the large language model is not greatly wrong even if the error is generated, the correct query statement can be obtained by only carrying out a small amount of modification during the error correction processing, so that a professional is not required to write all the query statement, thereby reducing the workload of the professional and improving the working efficiency.
The following examples illustrate specific processes for constructing a prompt word engineering for a large language model from relevant information in a database.
And acquiring database names of the databases and table structure definition information of data tables in the databases.
In this step, the database module may be connected to the database to be queried, and ensure that the database is connected normally. After the database module is normally connected to the database to be queried, the database module can read the name of the database (recorded as the name of the database), then the number of data tables specifically included in the database, table structure definition information of the data tables and the like can be obtained, and the information is used for constructing prompt word engineering of a large language model. The general data table is composed of a table name, fields in the table and records of the table, wherein the table structure definition information of the data table is information such as a file name of the definition data table, which fields are contained in the data table, a field name, a field type, a width and the like of each field.
And constructing prompt word engineering according to the database name and the table structure definition information.
In this step, after obtaining the database name of the connection and the table structure definition information of the data table, the database module may directly use the two parts of information as the prompt word engineering, or may also add other information to the prompt word engineering.
Taking the construction of the hint word engineering by database names, table structure definition information, and other information as an example, as an alternative embodiment, this step may include the steps of:
determining triple information for each data table according to the data table and table structure definition information of each data table; the triplet information comprises entities, relations and dependency relations among the entities.
In this step, after obtaining the table structure definition information of each data table, information such as a table name, a field in the table, a field name, a field type, and a width of each field in each data table may be obtained, and then a triplet may be established for each data table, and the triplet may be a triplet of entity-relationship-entity. Assuming that a certain student score data table includes field names such as student name, number, score and the like, triples such as score-belonging to student name or score-belonging to number and the like can be constructed.
By constructing the triples for each data table, the dependency relationship or the association relationship between the fields in each data table can be obtained, so that the subsequent large language model can accurately understand the data table and the fields in the data table, and more accurate query sentences can be generated.
And constructing a knowledge graph corresponding to the database according to the triplet information of each data table and the dependency relationship among the data tables.
In this step, after obtaining each data table of the database, triples between each data table may be constructed to represent the association relationship or the dependency relationship between each data table, and then the knowledge graph of the database may be generated through the dependency relationship between each data table and the triples information of each data table. That is, the knowledge graph can learn the dependency relationship between the data tables in the database and the dependency relationship between the fields in each data table, so that the follow-up large language model can accurately understand the dependency relationship between the data tables and the dependency relationship between the fields in the tables, and a more accurate query statement is generated.
And constructing prompt word engineering according to the database name, the table structure definition information and the knowledge graph.
In this step, as an alternative embodiment, the database name, the table structure definition information and the knowledge graph may be used to determine the prior information of the prompt word engineering; constructing constraint conditions of prompt word engineering according to the table structure definition information and the knowledge graph; the constraint condition is used for guiding the large language model to generate a query sentence based on the table structure definition information and the knowledge graph.
That is, database names, table structure definition information (may also be referred to as table structure definition), knowledge maps (may also be referred to as knowledge map relationships), and the like of the database to be queried may be used as prior information of the prompt word engineering.
The constraint conditions are set based on table structure definition information and a knowledge graph, and are mainly set for the number of query sentences generated by a large language model, the format of the generated query sentences, how to generate the query sentences, how to optimize the generated query sentences, and the like.
By combining the prior information with the constraint conditions, a constructed prompt word project (i.e., prompt) can be obtained as follows:
prompt={
database name { db_name }
Table structure definition { table_info }
Knowledge graph relationship: { knowledgegraph }
Constraint conditions:
1. The user intention is fully understood, the dependency relationship between the table and the table is understood according to the knowledge graph relationship { knowledgegraph }, the structure in the table is defined and a sql statement with correct grammar is created by using a given table structure definition, and if the sql is not needed, the user question is directly answered.
2. The query is limited to a maximum of { top_k } results unless the user specifies this { top_k } in the question.
3. The sql statement can only be generated using the table provided in the table structure definition information, and if the sql statement cannot be generated from the table provided in the provided table structure definition information, please say: "cannot generate sql statement according to the provided table structure", prohibit self-exertion, and make random kneading.
4. Please check the correctness of the sql statement, and optimize the performance if it is correct.
5. Please output as follows:
{xxx，xxx，xxx
}
The "understanding the table and the dependency relationship between the structures in the table" in the constraint condition 1 refers to understanding the dependency relationship between the data tables in the database, and understanding the dependency relationship between the fields in each data table, etc., where the constraint condition 1 and the 3 rd point constraint condition are used to indicate how the large language model generates the query statement. In constraint 2, the term "query is limited to the maximum { top_k } results" refers to a general default output set number of sql sentences unless the user specifies the { top_k } result in the question, and if the user specifies the number of generated sql sentences, the specified number of sql sentences may be generated according to the user's requirement, that is, the number of query sentences generated by the large language model is indicated. The constraint condition 4 refers to that when the large language model initially generates a plurality of accurate query sentences for a query question text, a query sentence with optimal performance can be selected for output through optimizing performance, namely, how the large language model optimizes the generated query sentence is indicated. The constraint 5 above specifies the format of the query statement generated, i.e., instructs the large language model to output the sql statement in such a format.
In this embodiment, the prompt word engineering can be constructed by the database name of the database to be queried and the table structure definition information of the data table, so that the large language model can be guided to better understand the database and the table structure thereof, and more accurate query sentences can be generated. In addition, when the prompt word engineering is constructed, the triplet can be constructed through the table structure definition information of the data table so as to construct a knowledge graph aiming at the database, so that a large language model can be convenient to fully and accurately understand specific dependency relations between the data table and the table and specific dependency relations between fields in the table, and the correctness of the generated query statement is improved. Furthermore, constraint conditions aiming at table structure definition information and knowledge graph can be added when the prompt word engineering is constructed, so that a large language model can be instructed to generate an accurate query statement meeting the requirements of users according to the constraint conditions, the condition that the large language model randomly generates the query statement under certain conditions is avoided, and the accuracy and normalization of the generated query statement are further improved.
As mentioned above, the term engineering includes constraint conditions for generating the query sentence by the large language model, the constraint conditions include contents for optimizing the query sentence, and the following embodiments describe the process of optimizing the query sentence by the large language model.
And inputting the query problem text and the prompt word engineering into a large language model, and performing conversion processing from the text to the query sentences to generate a plurality of candidate query sentences.
In this step, after the query question text and the constructed prompt word engineering are obtained, the query question text and the prompt word engineering may be both input into a large language model to perform conversion processing from text to query sentences, so as to generate a plurality of query sentences, which are all recorded as candidate query sentences. The plurality of candidate query sentences are not output in the large language model first, but are subjected to subsequent performance optimization processing.
Performing performance optimization processing on a plurality of candidate query sentences according to constraint conditions in prompt word engineering, and determining target query sentences corresponding to query problem texts; the constraint conditions include constraint conditions for performance optimization under the condition that a plurality of candidate query sentences are correct.
In this step, a term-prompting project constructed for the database is added to the large language model, and the constraint condition of the term-prompting project includes the constraint condition of performing performance optimization under the condition that a plurality of candidate query sentences are correct, namely the 4 th constraint condition. The large language model can firstly judge whether each candidate query sentence is accurate or not based on the constraint condition, if so, the performance parameters of each candidate query sentence are obtained, then the performance parameters of each candidate query sentence are compared, and one candidate query sentence with optimal performance is selected as a target query sentence and output.
The performance parameters here may be, for example, query speed, query time, etc. of the candidate query statement. By selecting the candidate query statement with the optimal performance as the final target query statement, the efficiency and accuracy of the subsequent database query through the target query statement can be improved.
In this embodiment, performance optimization processing is performed on a plurality of candidate query sentences initially generated by a large language model through constraint conditions of performance optimization in prompt word engineering, so as to obtain a target query sentence with optimal performance, and thus the efficiency and accuracy of subsequent database query through the target query sentence can be improved.
The knowledge retrieval enhancement generation system based on the large language model provided by the application is described below, and the knowledge retrieval enhancement generation system based on the large language model described below and the knowledge retrieval enhancement generation method based on the large language model described above can be correspondingly referred to each other. FIG. 8 is a block diagram of a knowledge retrieval enhancement generation system based on a large language model according to an embodiment of the present application, as shown in FIG. 8, the knowledge retrieval enhancement generation system based on a large language model according to an embodiment of the present application includes:
the conversion module 801 is configured to convert the received query question text into a target query question;
a keyword extraction module 802, configured to split the target query question, and extract a keyword of the target query question;
The document retrieval module 803 is configured to retrieve a document set to be matched from a private knowledge base of a user according to the target query problem based on a document retrieval enhancement model; the document set to be matched comprises at least two documents to be matched;
A document matching module 804, configured to obtain a target document based on a degree of correlation between each document to be matched and the target query question; the correlation degree of the target document and the target query problem is greater than a first threshold, and the target document comprises at least two text segments;
A text segment matching module 805, configured to match the target document based on the keyword, and obtain a target text segment in the target document; the matching degree of the target text segment and the key prompt word is larger than a second threshold;
A text segment ordering module 806, configured to order the target text segments to obtain an ordered text segment sequence;
an answer text generation module 807, configured to input the target query question and the ordered text segment sequence as an input set into a text generation enhancement model, to obtain an answer text output by the text generation enhancement model; the text generation enhancement model is constructed based on a large language model.
The embodiment of the application converts the query question text into the target query question; extracting key prompt words of a target query problem; searching a document set to be matched in a knowledge base according to a target query problem based on a document search enhancement model; obtaining target documents based on the correlation degree of each document to be matched and the target query problem; matching the target document based on the key prompt words to obtain a target text segment; sequencing the target text fragments to obtain a sequenced text fragment sequence; the target query questions and the ordered text fragment sequences are used as input sets to be input into the text generation enhancement model, and answer texts output by the text generation enhancement model are obtained, so that more accurate and comprehensive answers can be provided by combining document retrieval and language generation technologies, and meanwhile, a large-scale language model is used, so that the generated answer texts are more natural and smooth.
Referring to fig. 9, fig. 9 is a schematic diagram of an embodiment of an electronic device according to the present application. As shown in fig. 9, an embodiment of the present application provides an electronic device 900, including a memory 910, a processor 920, and a computer program 911 stored on the memory 910 and executable on the processor 920, wherein the processor 920 executes the computer program 911 to implement the following steps:
Converting the received query question text into a target query question;
splitting the target query problem, and extracting key prompt words of the target query problem;
searching a document set to be matched in a user private knowledge base according to the target query problem based on a document search enhancement model; the document set to be matched comprises at least two documents to be matched;
Obtaining a target document based on the correlation degree of each document to be matched and the target query problem; the correlation degree of the target document and the target query problem is greater than a first threshold, and the target document comprises at least two text segments;
Matching the target document based on the key prompt words to obtain target text fragments in the target document; the matching degree of the target text segment and the key prompt word is larger than a second threshold;
Sequencing the target text fragments to obtain a sequenced text fragment sequence;
Inputting the target query questions and the ordered text fragment sequences into a text generation enhancement model as an input set to obtain answer texts output by the text generation enhancement model; the text generation enhancement model is constructed based on a large language model.
In the foregoing embodiments, the descriptions of the embodiments are focused on, and for those portions of one embodiment that are not described in detail, reference may be made to the related descriptions of other embodiments.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a system for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the application.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present application without departing from the spirit or scope of the application. Thus, it is intended that the present application also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
Claims (10)
1. The knowledge retrieval enhancement generation method based on the large language model is characterized by comprising the following steps of:
Converting the received query question text into a target query question;
splitting the target query problem, and extracting key prompt words of the target query problem;
searching a document set to be matched in a user private knowledge base according to the target query problem based on a document search enhancement model; the document set to be matched comprises at least two documents to be matched;
Obtaining a target document based on the correlation degree of each document to be matched and the target query problem; the correlation degree of the target document and the target query problem is greater than a first threshold, and the target document comprises at least two text segments;
Matching the target document based on the key prompt words to obtain target text fragments in the target document; the matching degree of the target text segment and the key prompt word is larger than a second threshold;
Sequencing the target text fragments to obtain a sequenced text fragment sequence;
Inputting the target query questions and the ordered text fragment sequences into a text generation enhancement model as an input set to obtain answer texts output by the text generation enhancement model; the text generation enhancement model is constructed based on a large language model.
2. The large language model-based knowledge retrieval enhancement generation method according to claim 1, wherein the document retrieval enhancement model retrieves a document set to be matched from a user private knowledge base according to the target query problem, comprising:
Processing each document to be matched based on a document retrieval enhancement model so as to convert document contents in each document to be matched into a first document content feature vector;
Extracting features of the first document content feature vectors of the documents to be matched based on a preset neural network model to obtain second document content feature vectors of the documents to be matched;
Processing the target query question based on the document retrieval enhancement model to convert the target query question into a first query question feature vector;
extracting features of the first query problem feature vector based on the neural network model to obtain a second query problem feature vector of the target query problem;
And determining the document set to be matched based on the second document content feature vector and the second query question feature vector.
3. The large language model based knowledge retrieval enhancement generation method according to claim 2, wherein the determining the set of documents to be matched based on the second document content feature vector and the second query question feature vector comprises:
Calculating similarity values of second document content feature vectors and the second query problem feature vectors of the documents to be matched based on a preset model;
collecting documents to be matched corresponding to the target similarity value to obtain a document set to be matched; the target similarity value is a similarity value which is larger than or equal to a preset threshold value.
4. The knowledge retrieval enhancement generation method based on a large language model according to claim 3, wherein the preset model is a multi-scale fusion feedforward network.
5. The large language model based knowledge retrieval enhancement generation method according to any one of claims 1 to 4, further comprising:
acquiring a database to be queried and query problem text input by a user aiming at the database;
constructing a prompt word project according to the related information of the database;
Inputting the query problem text and the prompt word engineering into a large language model, converting the text into query sentences, and determining target query sentences corresponding to the query problem text; the prompt word engineering is used for guiding the large language model to generate a query sentence according to the prompt word engineering;
and inquiring in the database according to the target inquiry statement to determine an inquiry result.
6. The large language model based knowledge retrieval enhancement generation method according to claim 5, wherein the constructing a hint word project according to the related information of the database comprises:
acquiring database names of the database and table structure definition information of a data table in the database;
determining triple information for each data table according to the data table and table structure definition information of each data table;
Constructing a knowledge graph corresponding to the database according to the triplet information of each data table and the dependency relationship among the data tables;
and constructing prompt word engineering according to the database name, the table structure definition information and the knowledge graph.
7. The large language model based knowledge retrieval enhancement generation method according to claim 6, wherein the constructing a hint word project from the database name, the table structure definition information, and the knowledge graph includes:
determining prior information of the prompt word engineering by the database name, the table structure definition information and the knowledge graph;
Constructing constraint conditions of the prompt word engineering according to the table structure definition information and the knowledge graph; the constraint condition is used for guiding the large language model to generate a query statement based on the table structure definition information and the knowledge graph;
Inputting the query question text and the prompt word engineering into a large language model, performing conversion processing from the text to a query sentence, and determining a target query sentence corresponding to the query question text, wherein the method comprises the following steps:
Inputting the query problem text and the prompt word engineering into a large language model, and performing conversion processing from the text to query sentences to generate a plurality of candidate query sentences;
Performing performance optimization processing on the candidate query sentences according to constraint conditions in the prompt word engineering, and determining target query sentences corresponding to the query problem text;
And the constraint conditions comprise constraint conditions for performance optimization under the condition that the candidate query sentences are correct.
8. A knowledge retrieval enhancement generation system based on a large language model, comprising:
the conversion module is used for converting the received query question text into a target query question;
the keyword extraction module is used for splitting the target query problem and extracting the keyword prompt words of the target query problem;
The document retrieval module is used for retrieving a document set to be matched from a user private knowledge base according to the target query problem based on a document retrieval enhancement model; the document set to be matched comprises at least two documents to be matched;
The document matching module is used for obtaining target documents based on the correlation degree of each document to be matched and the target query problem; the correlation degree of the target document and the target query problem is greater than a first threshold, and the target document comprises at least two text segments;
The text segment matching module is used for matching the target document based on the key prompt words to obtain a target text segment in the target document; the matching degree of the target text segment and the key prompt word is larger than a second threshold;
The text segment sequencing module is used for sequencing the target text segments to obtain a sequenced text segment sequence;
the answer text generation module is used for inputting the target query questions and the ordered text fragment sequences into a text generation enhancement model as an input set to obtain answer texts output by the text generation enhancement model; the text generation enhancement model is constructed based on a large language model.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the large language model based knowledge retrieval enhancement generation method of any one of claims 1 to 7 when the program is executed.
10. A non-transitory computer-readable storage medium, having stored thereon a computer program which, when executed by a processor, implements the large language model based knowledge retrieval enhancement generation method of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410853156.8A CN118394890A (en) | 2024-06-28 | 2024-06-28 | Knowledge retrieval enhancement generation method and system based on large language model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410853156.8A CN118394890A (en) | 2024-06-28 | 2024-06-28 | Knowledge retrieval enhancement generation method and system based on large language model |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN118394890A true CN118394890A (en) | 2024-07-26 |
Family
ID=91985134
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410853156.8A Pending CN118394890A (en) | 2024-06-28 | 2024-06-28 | Knowledge retrieval enhancement generation method and system based on large language model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118394890A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180060422A1 (en) * | 2016-08-25 | 2018-03-01 | Lakeside Software, Inc. | Method and apparatus for natural language query in a workspace analytics system |
| US20230040095A1 (en) * | 2021-10-28 | 2023-02-09 | Beijing Baidu Netcom Science Technology Co., Ltd. | Method for pre-training model, device, and storage medium |
| CN117290492A (en) * | 2023-11-27 | 2023-12-26 | 深圳市灵智数字科技有限公司 | Knowledge base question-answering method and device, electronic equipment and storage medium |
| CN117573821A (en) * | 2023-11-10 | 2024-02-20 | 科大讯飞股份有限公司 | Knowledge question-answering method, device, equipment and storage medium |
| CN117573846A (en) * | 2024-01-16 | 2024-02-20 | 宏景科技股份有限公司 | Output optimization method of large language model |
| CN118132512A (en) * | 2024-01-25 | 2024-06-04 | 北京有竹居网络技术有限公司 | Method, apparatus, device and readable medium for document query |
-
2024
- 2024-06-28 CN CN202410853156.8A patent/CN118394890A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180060422A1 (en) * | 2016-08-25 | 2018-03-01 | Lakeside Software, Inc. | Method and apparatus for natural language query in a workspace analytics system |
| US20230040095A1 (en) * | 2021-10-28 | 2023-02-09 | Beijing Baidu Netcom Science Technology Co., Ltd. | Method for pre-training model, device, and storage medium |
| CN117573821A (en) * | 2023-11-10 | 2024-02-20 | 科大讯飞股份有限公司 | Knowledge question-answering method, device, equipment and storage medium |
| CN117290492A (en) * | 2023-11-27 | 2023-12-26 | 深圳市灵智数字科技有限公司 | Knowledge base question-answering method and device, electronic equipment and storage medium |
| CN117573846A (en) * | 2024-01-16 | 2024-02-20 | 宏景科技股份有限公司 | Output optimization method of large language model |
| CN118132512A (en) * | 2024-01-25 | 2024-06-04 | 北京有竹居网络技术有限公司 | Method, apparatus, device and readable medium for document query |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109271505B (en) | Question-answering system implementation method based on question-answer pairs | |
| CN109325040B (en) | FAQ question-answer library generalization method, device and equipment | |
| CN111488467B (en) | Construction method and device of geographical knowledge graph, storage medium and computer equipment | |
| CN111524593B (en) | Medical question-answering method and system based on context language model and knowledge embedding | |
| CN116701431A (en) | Data retrieval method and system based on large language model | |
| EP4187400A1 (en) | Method for converting natural language query to sql and device thereof | |
| CN117370580A (en) | Knowledge-graph-based large language model enhanced dual-carbon field service method | |
| CN115470338B (en) | Multi-scenario intelligent question answering method and system based on multi-path recall | |
| CN114897163A (en) | Pre-training model data processing method, electronic device and computer storage medium | |
| CN113742446A (en) | Knowledge graph question-answering method and system based on path sorting | |
| CN111553138B (en) | Auxiliary writing method and device for standardizing content structure document | |
| CN114997181A (en) | Intelligent question-answering method and system based on user feedback correction | |
| CN117807202A (en) | Knowledge graph enhanced large language model reasoning trademark law intelligent question-answering method | |
| CN111309930A (en) | Medical knowledge graph entity alignment method based on representation learning | |
| CN112860916B (en) | Movie-television-oriented multi-level knowledge map generation method | |
| CN117708282A (en) | Knowledge question-answering method and system based on large language model | |
| CN117009599A (en) | Data retrieval method and device, processor and electronic equipment | |
| CN117112743A (en) | Method, system and storage medium for evaluating answers of text automatic generation questions | |
| CN111737413A (en) | Feedback model information retrieval method, system and medium based on concept net semantics | |
| CN118394890A (en) | Knowledge retrieval enhancement generation method and system based on large language model | |
| CN114328823A (en) | Database natural language query method and device, electronic equipment and storage medium | |
| CN118170894B (en) | Knowledge graph question-answering method, knowledge graph question-answering device and storage medium | |
| Gatti et al. | Composite Sketch+ Text Queries for Retrieving Objects with Elusive Names and Complex Interactions | |
| Zhang et al. | Retrieval-Based Question Answering Based on Emotion-Aware Graph Attention Network | |
| CN118069815B (en) | Large language model feedback information generation method and device, electronic equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination |