CN117216226A - Knowledge positioning method, device, storage medium and equipment - Google Patents
Knowledge positioning method, device, storage medium and equipment Download PDFInfo
- Publication number
- CN117216226A CN117216226A CN202311386064.5A CN202311386064A CN117216226A CN 117216226 A CN117216226 A CN 117216226A CN 202311386064 A CN202311386064 A CN 202311386064A CN 117216226 A CN117216226 A CN 117216226A
- Authority
- CN
- China
- Prior art keywords
- text
- target
- positioning
- model
- prompt
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 54
- 238000003860 storage Methods 0.000 title claims abstract description 13
- 238000004458 analytical method Methods 0.000 claims abstract description 11
- 238000012795 verification Methods 0.000 claims description 76
- 238000012549 training Methods 0.000 claims description 23
- 230000006870 function Effects 0.000 claims description 12
- 238000010606 normalization Methods 0.000 claims description 11
- 238000000605 extraction Methods 0.000 description 9
- 230000008569 process Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 6
- 239000012634 fragment Substances 0.000 description 6
- 230000009471 action Effects 0.000 description 3
- 238000005259 measurement Methods 0.000 description 3
- 238000004590 computer program Methods 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 239000013598 vector Substances 0.000 description 2
- 230000002457 bidirectional effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000005728 strengthening Methods 0.000 description 1
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a knowledge positioning method, a knowledge positioning device, a knowledge positioning storage medium and knowledge positioning equipment, wherein the knowledge positioning method comprises the following steps: firstly, acquiring a target problem text to be replied; and performing information retrieval and information analysis by utilizing a target problem text based on a preset information search engine to obtain a target text related to the target problem text, then utilizing the target text and the target problem text to construct a positioning prompt instruction prompt, inputting the positioning prompt instruction prompt into a pre-constructed knowledge positioning model to obtain text information related to the target problem text output by the knowledge positioning model, and then inputting the target problem text and the text information related to the target problem text into a general generation model to obtain reply content aiming at the target problem text output by the general generation model. Therefore, the application utilizes the gradual generation of each character in the reply content of the knowledge positioning model and the general generation model which are constructed in advance, not only can improve the accuracy of the positioning result, but also can generate more accurate problem reply.
Description
Technical Field
The present application relates to the field of natural language processing technologies, and in particular, to a knowledge positioning method, apparatus, storage medium, and device.
Background
Knowledge localization is typically one of the essential basic capabilities in large language model (large language model, LLM) knowledge plug-in application scenarios. The knowledge positioning aims at filtering and screening the knowledge, retaining information closely related to user inquiry, and increasing the quality of introduced knowledge, thereby ensuring the application effect of the knowledge plug-in.
At present, the main objective of the knowledge positioning task is to extract information related to user query in the existing text, and the information positioning task is similar to the text similarity task and accords with the paradigm of the information extraction task, so that two positioning modes are generally adopted when knowledge positioning is performed currently: one is to split the existing text into a plurality of text fragments, use the text fragments and user query to make similarity measurement tasks, select the text fragments with high similarity as the result of knowledge positioning. And the other is to use the user query and the existing text as model data, use the knowledge positioning result as the result of the information extraction task and use the information extraction model to extract the related information. But both of these positioning methods are not accurate enough.
Disclosure of Invention
The embodiment of the application mainly aims to provide a knowledge positioning method, a knowledge positioning device, a storage medium and knowledge positioning equipment, which can improve the accuracy of a positioning result when knowledge positioning is performed.
The embodiment of the application provides a knowledge positioning method, which comprises the following steps:
acquiring a target problem text to be replied; information retrieval and information analysis are carried out on the basis of a preset information search engine by utilizing the target question text, so that a target text related to the target question text is obtained;
constructing a positioning prompt instruction prompt by using the target text and the target problem text, inputting the positioning prompt instruction prompt into a pre-constructed knowledge positioning model, and obtaining text information which is output by the knowledge positioning model and is related to the target problem text;
and inputting the target question text and text information related to the target question text into a pre-constructed general generation model to obtain the reply content which is output by the general generation model and aims at the target question text.
In one possible implementation manner, the knowledge positioning model is a knowledge positioning plug-in which is introduced in advance for a large language model; the method for constructing a positioning prompt instruction prompt by using the target text and the target question text, inputting the positioning prompt instruction prompt into a pre-constructed knowledge positioning model to obtain text information which is output by the knowledge positioning model and is related to the target question text comprises the following steps:
and constructing a positioning prompt instruction prompt by using the target text and the target question text, and inputting the positioning prompt instruction prompt into a large language model so that the large language model can position text information related to the target question text by using knowledge stored by a knowledge positioning plug-in.
In one possible implementation, the generic generation model is a large language model; the large language model includes an N-layer decoder; n is a positive integer greater than 0; the decoder includes a multi-headed attention portion, a fully connected layer portion, and a layer normalization portion.
In a possible implementation manner, the inputting the target question text and the text information related to the target question text into a pre-built general generation model to obtain the reply content for the target question text output by the general generation model includes:
and inputting the target question text and text information related to the target question text layer by layer into a multi-head attention part, a full-connection layer part and a layer normalization part in an N-layer decoder of the large language model to obtain the reply content which is output by the large language model and aims at the target question text.
In a possible implementation manner, the general generation model is constructed as follows:
acquiring a sample question text, and constructing a sample prompt instruction prompt by using the sample question text;
inputting the sample prompt instruction prompt into an initial general generation model to obtain sample reply prediction content corresponding to the sample question text;
and training an initial general generation model by utilizing sample reply prediction content corresponding to the sample question text and a preset loss function to generate the general generation model.
In a possible implementation manner, the constructing a sample prompt instruction prompt by using the sample question text includes:
and distinguishing the sample question text from the reference text by using a preset mark, and constructing a sample prompt instruction prompt.
In a possible implementation manner, the method further includes:
acquiring a verification problem text, and constructing a verification prompt instruction prompt by using the verification problem text;
inputting the verification prompt instruction prompt into the general generation model to obtain a verification reply prediction result of the verification question text;
and when the real answer result of the verification answer prediction result of the verification question text and the real answer result corresponding to the verification question text do not meet a preset verification condition, the verification question text is taken as the sample question text again, and the general generation model is updated.
The embodiment of the application also provides a knowledge positioning device, which comprises:
the first acquisition unit is used for acquiring a target question text to be replied; information retrieval and information analysis are carried out on the basis of a preset information search engine by utilizing the target question text, so that a target text related to the target question text is obtained;
the first input unit is used for constructing a positioning prompt instruction prompt by utilizing the target text and the target problem text, inputting the positioning prompt instruction prompt into a pre-constructed knowledge positioning model, and obtaining text information which is output by the knowledge positioning model and is related to the target problem text;
and the second input unit is used for inputting the target question text and the text information related to the target question text into a pre-constructed general generation model to obtain the reply content which is output by the general generation model and aims at the target question text.
In one possible implementation manner, the knowledge positioning model is a knowledge positioning plug-in which is introduced in advance for a large language model; the first input unit is specifically configured to:
and constructing a positioning prompt instruction prompt by using the target text and the target question text, and inputting the positioning prompt instruction prompt into a large language model so that the large language model can position text information related to the target question text by using knowledge stored by a knowledge positioning plug-in.
In one possible implementation, the generic generation model is a large language model; the large language model includes an N-layer decoder; n is a positive integer greater than 0; the decoder includes a multi-headed attention portion, a fully connected layer portion, and a layer normalization portion.
In a possible implementation manner, the second input unit is specifically configured to:
and inputting the target question text and text information related to the target question text layer by layer into a multi-head attention part, a full-connection layer part and a layer normalization part in an N-layer decoder of the large language model to obtain the reply content which is output by the large language model and aims at the target question text.
In a possible implementation manner, the apparatus further includes:
the second acquisition unit is used for acquiring a sample question text and constructing a sample prompt instruction prompt by utilizing the sample question text;
the third input unit is used for inputting the sample prompt instruction prompt into an initial general generation model to obtain sample reply prediction content corresponding to the sample question text;
and the training unit is used for training the initial general generation model by utilizing the sample reply prediction content corresponding to the sample question text and the preset loss function to generate the general generation model.
In a possible implementation manner, the second obtaining unit is specifically configured to:
and distinguishing the sample question text from the reference text by using a preset mark, and constructing a sample prompt instruction prompt.
In a possible implementation manner, the apparatus further includes:
the third acquisition unit is used for acquiring a verification problem text and constructing a verification prompt instruction prompt by utilizing the verification problem text;
the fourth input unit is used for inputting the verification prompt instruction prompt into the general generation model to obtain a verification reply prediction result of the verification question text;
and the updating unit is used for updating the general generation model by taking the verification question text as the sample question text again when the verification reply prediction result of the verification question text and the real reply result corresponding to the verification question text do not meet the preset verification condition.
The embodiment of the application also provides knowledge positioning equipment, which comprises: a processor, memory, system bus;
the processor and the memory are connected through the system bus;
the memory is for storing one or more programs, the one or more programs comprising instructions, which when executed by the processor, cause the processor to perform any of the implementations of the knowledge positioning method described above.
The embodiment of the application also provides a computer readable storage medium, wherein the computer readable storage medium stores instructions which, when run on terminal equipment, cause the terminal equipment to execute any implementation mode of the knowledge positioning method.
The embodiment of the application also provides a computer program product which, when run on terminal equipment, causes the terminal equipment to execute any implementation mode of the knowledge positioning method.
The embodiment of the application provides a knowledge positioning method, a knowledge positioning device, a knowledge positioning storage medium and knowledge positioning equipment, which are used for firstly acquiring a target problem text to be replied; and performing information retrieval and information analysis by utilizing a target problem text based on a preset information search engine to obtain a target text related to the target problem text, then utilizing the target text and the target problem text to construct a positioning prompt instruction prompt, inputting the positioning prompt instruction prompt into a pre-constructed knowledge positioning model to obtain text information related to the target problem text output by the knowledge positioning model, and then inputting the target problem text and the text information related to the target problem text into a general generation model to obtain reply content aiming at the target problem text output by the general generation model. Therefore, the application utilizes the gradual generation of each character in the reply content of the knowledge positioning model and the general generation model which are constructed in advance, not only can improve the accuracy of the positioning result, but also can generate more accurate problem reply.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow chart of a knowledge positioning method according to an embodiment of the present application;
FIG. 2 is a schematic process diagram of reply content to a target question text output by a generic generation model according to an embodiment of the present application;
FIG. 3 is an exemplary diagram of a sample hint instruction prompt provided by an embodiment of the present application;
FIG. 4 is an exemplary diagram of sample reply prediction content provided by an embodiment of the present application;
FIG. 5 is an exemplary diagram of training data provided by an embodiment of the present application;
fig. 6 is a schematic diagram of a knowledge positioning apparatus according to an embodiment of the present application.
Detailed Description
Existing knowledge positioning methods typically include two types:
one is to split the existing text into a plurality of text fragments, use the text fragments and user query to make similarity measurement tasks, select the text fragments with high similarity as the result of knowledge positioning. The scheme of text splitting and similarity measurement needs to split the existing text preferentially, the splitting strategy is difficult to control, the extracted text semantics are easy to be incoherent due to too fine splitting, too coarse splitting and fusion are used for introducing too much irrelevant information, and the method is contrary to the original purpose of knowledge positioning.
And the other is to use the user query and the existing text as model data, use the knowledge positioning result as the result of the information extraction task and use the information extraction model to extract the related information. However, the information extraction scheme generally can only extract continuous texts, and cannot achieve the purpose of extracting key information of the target text. For example, after the user's query is input into the information extraction model constructed based on the BERT (Bidirectional Encoder Representations for Transformers) model, the model can only have the start position and end position of the content related to the user's query in the text. In addition, the length of the existing text is generally not limited, and the information extraction model with thousands of characters is generally insufficient.
In addition, the existing two knowledge positioning methods can not utilize the positioned knowledge to form natural language which is easier to be understood by the user, such as related knowledge abstract, and the like, so that not only is the positioning result inaccurate, but also a reply result provided for the user is inaccurate.
In order to solve the defects, the application provides a knowledge positioning method, which comprises the steps of firstly obtaining a target problem text to be replied; and performing information retrieval and information analysis by utilizing a target problem text based on a preset information search engine to obtain a target text related to the target problem text, then utilizing the target text and the target problem text to construct a positioning prompt instruction prompt, inputting the positioning prompt instruction prompt into a pre-constructed knowledge positioning model to obtain text information related to the target problem text output by the knowledge positioning model, and then inputting the target problem text and the text information related to the target problem text into a general generation model to obtain reply content aiming at the target problem text output by the general generation model. Therefore, the application utilizes the gradual generation of each character in the reply content of the knowledge positioning model and the general generation model which are constructed in advance, not only can improve the accuracy of the positioning result, but also can generate more accurate problem reply.
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments of the present application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
First embodiment
Referring to fig. 1, a flow chart of a knowledge positioning method provided in this embodiment includes the following steps:
s101: acquiring a target problem text to be replied; and performing information retrieval and information analysis by utilizing the target question text based on a preset information search engine to obtain a target text related to the target question text.
In this embodiment, a question text that any one of intelligent interaction software or equipment such as LLM needs to be replied to be inputted by a user is defined as a target question text to be replied to, and the present embodiment does not limit the language type of the target question text, for example, the target question text may be a chinese text or an english text. The content of the target question text is not limited in this embodiment either, that is, the target question text may be a question related to a daily conversation of a person, or may be a question text related to various fields such as education, medical treatment, and the like.
It can be understood that the target question text may be a sentence text, which is a set of words, and after the target question text to be replied is obtained, information retrieval and information analysis may be performed based on a preset information search engine (specific content is not limited and may be selected according to actual conditions and experience values) to obtain a target text related to the target question text. For example, the general information search engine may be used to search information related to the target problem (including but not limited to web pages, databases, documents, etc.), and obtain the target text after the information is parsed, for executing the subsequent step S102.
S102: and constructing a positioning prompt instruction prompt by using the target text and the target problem text, inputting the positioning prompt instruction prompt into a pre-constructed knowledge positioning model, and obtaining text information which is output by the knowledge positioning model and is related to the target problem text.
In this embodiment, after the target text related to the target problem text is obtained through step S101, a positioning prompt instruction prompt may be further constructed by using the target text and the target problem text by using the existing or future existing method for constructing a prompt, and the positioning prompt instruction prompt may be input into a knowledge positioning model constructed in advance, so as to obtain text information related to the target problem text output by the knowledge positioning model, as shown in fig. 2, for executing the subsequent step S103.
It should be noted that, for a target problem, a plurality of target texts related to the target problem are generally retrieved, so that text information (such as the valid text shown in fig. 2) related to the target problem text can be generated by using the pre-built knowledge positioning model.
Specifically, the specific composition of the knowledge positioning model is not limited enough, the knowledge positioning model can be selected and set according to actual conditions and experience values, and an alternative implementation manner is that the knowledge positioning model can be set as a knowledge positioning plug-in which is introduced in advance into a large language model LLM, so that after a positioning prompt instruction prompt is constructed by utilizing a target text and a target problem text, the knowledge positioning plug-in can be input into the large language model LLM, and the large language model can position text information related to the target problem text by utilizing knowledge stored by the knowledge positioning plug-in.
S103: and inputting the target question text and text information related to the target question text into the general generation model to obtain reply contents, which are output by the general generation model, of the target question text.
In this embodiment, after obtaining the text information related to the target question text output by the knowledge positioning model in step S102, in order to improve the accuracy of the positioning result and the reply result, the target question text and the text information related to the target question text may be further input into a pre-constructed general generation model, so as to obtain more accurate reply content for the target question text output by the general generation model, as shown in fig. 2.
In the application, in order to improve the accuracy of the positioning result and the reply result, a general generation model is pre-constructed, the specific network composition structure of the general generation model is not limited, and the general generation model can be selected and set according to actual conditions. A preferred implementation is that the pre-built generic generative model includes, but is not limited to, layer decoders comprising N (N is a positive integer greater than 0), wherein each decoder may comprise, but is not limited to, a multi-head attention portion, a fully connected layer portion, a layer normalization portion. The multi-head attention part is used for strengthening the association between words and simultaneously can prevent the problem of forgetting information when the sentences are too long. The layer normalization part is used for normalizing vectors of the spread network and guaranteeing stability of distribution of each dimension. The fully connected layer portion is used to dimensionally change the result vector of multi-head attention while being able to learn complex feature representations naturally.
It should be further noted that, since the prediction process of the generic generation model constructed by the present application starts from the first character, the next character is generated successively until the generation of the entire reply sentence is completed or the total sentence length exceeds the limit length. Therefore, the problem of text splitting can be avoided, and meanwhile, the problem that only continuous texts can be extracted in the existing information extraction scheme can be solved in the form of generating the texts according to probability. And further improves the positioning accuracy. Thus, after the target question text and the text information related to the target question text are input into the multi-head attention part, the full-connection layer part and the layer normalization part in the N-layer decoder of the large language model layer by layer, more accurate reply content of the target question text output by the large language model can be obtained.
Next, the present embodiment will describe a process for constructing a generic generative model, where an alternative implementation manner may specifically include: firstly, a sample question text is obtained, and a sample prompt instruction prompt is constructed by using the sample question text. And inputting a sample prompt instruction prompt into the initial general generation model to obtain sample reply predicted content corresponding to the sample question text, and training the initial general generation model by utilizing the sample reply predicted content corresponding to the sample question text and a preset loss function to generate the general generation model.
Specifically, in the present embodiment, a large amount of preparation work is required in advance to construct a generic generation model, and first, a large amount of training data is required to be constructed. It can be appreciated that the training data construction generally requires a relatively large amount of time and effort, and in order to save manpower and accelerate efficiency, the application generates training data related to tasks by designing related prompt instructions promtt and using a large language model LLM in the initial stage of the training data construction. The large language model LLM is a generation type large model based on multi-field non-supervision corpus training, has good semantic understanding and text generation capacity, even exceeds the artificial effect in part of fields, and is an effective tool for improving productivity.
It should be noted that, when the large language model LLM is used, attention needs to be paid to the design of prompt instruction prompt, and reasonable prompt can better excite the capability of the large language model LLM, so as to obtain higher data quality. The general standard of the template contains Instructions, context, input data, output indicator, and the different task types may have a small difference, but basically within these ranges.
To better meet the paradigm requirements of the knowledge positioning task, adjustments to the standard campt are required. Since the input text of the knowledge location task contains two parts, one is the user's query question, or the specified question, and the other is the reference text. In order to ensure that the sample prompt instruction of the knowledge positioning task has definite meanings, the two parts need to have definite limits, therefore, the application adopts preset marks (specific content is not limited and can be set according to actual conditions and experience) to distinguish sample problem texts from reference texts so as to construct the sample prompt instruction, and the sample prompt is shown in fig. 3.
Then, the constructed sample prompting instruction prompt is used for submitting a large language model LLM interface, so that sample reply predicted contents (contents related to the specified problem in the reference text can be extracted by a reply result) generated according to the prompt, which are output by the large language model LLM, can be obtained, and training data (including the sample prompting instruction prompt, the sample reply predicted contents and the like) are constructed as shown in fig. 4.
It should be noted that, the sample prompt instruction prompt constructed by the above manner can construct or construct training data with relatively high quality, but is not fully capable of meeting the requirement of actual training, and in order to improve the accuracy of the model, the training data can be manually checked, screened, modified and the like, so as to ensure the availability and accuracy of the training data.
Next, to meet the requirements of generic generative model training, the constructed training data may be processed into a unified long text form stitched by prompt, SEP, answer, as shown in fig. 5. The SEP is a special symbol for representing the separator, the specific content is not limited, and other preset symbols can be replaced to define two parts of the sample and the answer. Because the knowledge positioning task only concerns the accuracy of character prediction in the answer, only characters in the answer part participate in the evaluation of the quality of the general generation model. In each step of prediction process of the general generation model, the model predicts a probability value for each character in the provided word list, and the character with the highest probability value is the next character predicted by the model. The evaluation criteria of the model are also determined by the actual prediction probability of the actual next character, and the larger the actual probability is, the higher the prediction accuracy is.
When the model is trained, in order to achieve the purpose of optimizing the model, the following preset loss function pair can be adopted to train the model in multiple rounds, and the model parameters are updated according to the value of the preset loss function obtained by each round of training until preset conditions are met, for example, the value of the preset loss function is minimum and basically unchanged, the updating of the model parameters is stopped, the training of the general generation model is completed, and a trained general generation model is generated. The specific calculation formula of the preset loss function is as follows:
wherein θ represents a parameter of the model; x represents the input representing the model; y is<t represents all character sequences before; y is t Representing the predicted next character; the function P (|. Cndot.) represents a probability function. The goal of model training is to maximize the true next character prediction probability, i.e., minimize the preset loss function L.
On the basis, further, the generated general generation model can be verified by using the verification problem text. The specific verification process may include the following steps (1) - (3):
step (1): and acquiring a verification problem text, and constructing a verification prompt instruction prompt by using the verification problem text.
In this embodiment, in order to implement verification on the generic generation model, it is first necessary to obtain a verification question text, and use the verification question text to verify the question text, construct a verification prompt instruction prompt, and may continue to execute the subsequent step (2).
Step (2): inputting a verification prompt instruction prompt into a general generation model to obtain a verification reply prediction result of the verification question text.
After the verification problem text is obtained and the verification prompt instruction prompt is constructed in the step (1), the verification prompt instruction prompt can be further input into a general generation model to obtain a verification reply prediction result of the verification problem text, and the verification reply prediction result is used for executing the subsequent step (3).
Step (3): and when the real answer result of the verification answer prediction result of the verification question text and the verification question text do not meet the preset verification condition, the verification question text is taken as the sample question text again, and the general generation model is updated.
After obtaining the verification reply prediction result of the verification question text through the step (2), if the verification reply prediction result of the verification question text and the real reply result corresponding to the verification question text do not meet the preset verification condition (specific content is not limited), if the verification reply prediction result and the real reply result are inconsistent, or the similarity of the verification question text and the real reply result does not meet the preset similarity threshold, the verification question text can be used as the sample question text again, and the general generation model is updated.
In summary, in the knowledge positioning method provided in this embodiment, a target problem text to be replied is first obtained; and performing information retrieval and information analysis by utilizing a target problem text based on a preset information search engine to obtain a target text related to the target problem text, then utilizing the target text and the target problem text to construct a positioning prompt instruction prompt, inputting the positioning prompt instruction prompt into a pre-constructed knowledge positioning model to obtain text information related to the target problem text output by the knowledge positioning model, and then inputting the target problem text and the text information related to the target problem text into a general generation model to obtain reply content aiming at the target problem text output by the general generation model. Therefore, the application utilizes the gradual generation of each character in the reply content of the knowledge positioning model and the general generation model which are constructed in advance, not only can improve the accuracy of the positioning result, but also can generate more accurate problem reply.
Second embodiment
The present embodiment will be described with reference to a knowledge positioning apparatus, and for related content, reference is made to the above-mentioned method embodiment.
Referring to fig. 6, a schematic diagram of a knowledge positioning apparatus according to the present embodiment is provided, and the apparatus 600 includes:
a first obtaining unit 601, configured to obtain a target question text to be replied; information retrieval and information analysis are carried out on the basis of a preset information search engine by utilizing the target question text, so that a target text related to the target question text is obtained;
a first input unit 602, configured to construct a positioning prompt instruction prompt by using the target text and the target question text, input the positioning prompt instruction prompt to a knowledge positioning model constructed in advance, and obtain text information related to the target question text output by the knowledge positioning model;
and a second input unit 603, configured to input the target question text and text information related to the target question text into a pre-built general generation model, so as to obtain reply content, output by the general generation model, for the target question text.
In one implementation manner of this embodiment, the knowledge positioning model is a knowledge positioning plug-in that is introduced in advance by a large language model; the first input unit 602 is specifically configured to:
and constructing a positioning prompt instruction prompt by using the target text and the target question text, and inputting the positioning prompt instruction prompt into a large language model so that the large language model can position text information related to the target question text by using knowledge stored by a knowledge positioning plug-in.
In one implementation manner of this embodiment, the generic generation model is a large language model; the large language model includes an N-layer decoder; n is a positive integer greater than 0; the decoder includes a multi-headed attention portion, a fully connected layer portion, and a layer normalization portion.
In one implementation manner of this embodiment, the second input unit 603 is specifically configured to:
and inputting the target question text and text information related to the target question text layer by layer into a multi-head attention part, a full-connection layer part and a layer normalization part in an N-layer decoder of the large language model to obtain the reply content which is output by the large language model and aims at the target question text.
In one implementation of this embodiment, the apparatus further includes:
the second acquisition unit is used for acquiring a sample question text and constructing a sample prompt instruction prompt by utilizing the sample question text;
the third input unit is used for inputting the sample prompt instruction prompt into an initial general generation model to obtain sample reply prediction content corresponding to the sample question text;
and the training unit is used for training the initial general generation model by utilizing the sample reply prediction content corresponding to the sample question text and the preset loss function to generate the general generation model.
In one implementation manner of this embodiment, the second obtaining unit is specifically configured to:
and distinguishing the sample question text from the reference text by using a preset mark, and constructing a sample prompt instruction prompt.
In one implementation of this embodiment, the apparatus further includes:
the third acquisition unit is used for acquiring a verification problem text and constructing a verification prompt instruction prompt by utilizing the verification problem text;
the fourth input unit is used for inputting the verification prompt instruction prompt into the general generation model to obtain a verification reply prediction result of the verification question text;
and the updating unit is used for updating the general generation model by taking the verification question text as the sample question text again when the verification reply prediction result of the verification question text and the real reply result corresponding to the verification question text do not meet the preset verification condition.
Further, the embodiment of the application also provides knowledge positioning equipment, which comprises: a processor, memory, system bus;
the processor and the memory are connected through the system bus;
the memory is for storing one or more programs, the one or more programs comprising instructions, which when executed by the processor, cause the processor to perform any of the implementations of the knowledge positioning method described above.
Further, the embodiment of the application also provides a computer readable storage medium, wherein the computer readable storage medium stores instructions, and when the instructions run on the terminal equipment, the terminal equipment is caused to execute any implementation method of the knowledge positioning method.
Further, the embodiment of the application also provides a computer program product, which when being run on the terminal equipment, causes the terminal equipment to execute any implementation method of the knowledge positioning method.
From the above description of embodiments, it will be apparent to those skilled in the art that all or part of the steps of the above described example methods may be implemented in software plus necessary general purpose hardware platforms. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network communication device such as a media gateway, etc.) to execute the method described in the embodiments or some parts of the embodiments of the present application.
It should be noted that, in the present description, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different manner from other embodiments, and identical and similar parts between the embodiments are all enough to refer to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section.
It is further noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A knowledge positioning method, comprising:
acquiring a target problem text to be replied; information retrieval and information analysis are carried out on the basis of a preset information search engine by utilizing the target question text, so that a target text related to the target question text is obtained;
constructing a positioning prompt instruction prompt by using the target text and the target problem text, inputting the positioning prompt instruction prompt into a pre-constructed knowledge positioning model, and obtaining text information which is output by the knowledge positioning model and is related to the target problem text;
and inputting the target question text and text information related to the target question text into a pre-constructed general generation model to obtain the reply content which is output by the general generation model and aims at the target question text.
2. The method of claim 1, wherein the knowledge location model is a knowledge location plug-in that is pre-introduced to a large language model; the method for constructing a positioning prompt instruction prompt by using the target text and the target question text, inputting the positioning prompt instruction prompt into a pre-constructed knowledge positioning model to obtain text information which is output by the knowledge positioning model and is related to the target question text comprises the following steps:
and constructing a positioning prompt instruction prompt by using the target text and the target question text, and inputting the positioning prompt instruction prompt into a large language model so that the large language model can position text information related to the target question text by using knowledge stored by a knowledge positioning plug-in.
3. The method of claim 1, wherein the generic generative model is a large language model; the large language model includes an N-layer decoder; n is a positive integer greater than 0; the decoder includes a multi-headed attention portion, a fully connected layer portion, and a layer normalization portion.
4. The method according to claim 3, wherein inputting the target question text and the text information related thereto into a pre-constructed generic generation model to obtain the reply content for the target question text output by the generic generation model includes:
and inputting the target question text and text information related to the target question text layer by layer into a multi-head attention part, a full-connection layer part and a layer normalization part in an N-layer decoder of the large language model to obtain the reply content which is output by the large language model and aims at the target question text.
5. A method according to claim 3, wherein the generic generative model is constructed as follows:
acquiring a sample question text, and constructing a sample prompt instruction prompt by using the sample question text;
inputting the sample prompt instruction prompt into an initial general generation model to obtain sample reply prediction content corresponding to the sample question text;
and training an initial general generation model by utilizing sample reply prediction content corresponding to the sample question text and a preset loss function to generate the general generation model.
6. The method of claim 5, wherein constructing a sample hint instruction prompt using the sample question text comprises:
and distinguishing the sample question text from the reference text by using a preset mark, and constructing a sample prompt instruction prompt.
7. The method of claim 5, wherein the method further comprises:
acquiring a verification problem text, and constructing a verification prompt instruction prompt by using the verification problem text;
inputting the verification prompt instruction prompt into the general generation model to obtain a verification reply prediction result of the verification question text;
and when the real answer result of the verification answer prediction result of the verification question text and the real answer result corresponding to the verification question text do not meet a preset verification condition, the verification question text is taken as the sample question text again, and the general generation model is updated.
8. A knowledge positioning apparatus, comprising:
the first acquisition unit is used for acquiring a target question text to be replied; information retrieval and information analysis are carried out on the basis of a preset information search engine by utilizing the target question text, so that a target text related to the target question text is obtained;
the first input unit is used for constructing a positioning prompt instruction prompt by utilizing the target text and the target problem text, inputting the positioning prompt instruction prompt into a pre-constructed knowledge positioning model, and obtaining text information which is output by the knowledge positioning model and is related to the target problem text;
and the second input unit is used for inputting the target question text and the text information related to the target question text into a pre-constructed general generation model to obtain the reply content which is output by the general generation model and aims at the target question text.
9. A knowledge positioning apparatus, comprising: a processor, memory, system bus;
the processor and the memory are connected through the system bus;
the memory is for storing one or more programs, the one or more programs comprising instructions, which when executed by the processor, cause the processor to perform the method of any of claims 1-7.
10. A computer readable storage medium, characterized in that the computer readable storage medium has stored therein instructions, which when run on a terminal device, cause the terminal device to perform the method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311386064.5A CN117216226A (en) | 2023-10-23 | 2023-10-23 | Knowledge positioning method, device, storage medium and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311386064.5A CN117216226A (en) | 2023-10-23 | 2023-10-23 | Knowledge positioning method, device, storage medium and equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117216226A true CN117216226A (en) | 2023-12-12 |
Family
ID=89046394
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311386064.5A Pending CN117216226A (en) | 2023-10-23 | 2023-10-23 | Knowledge positioning method, device, storage medium and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117216226A (en) |
-
2023
- 2023-10-23 CN CN202311386064.5A patent/CN117216226A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116628172B (en) | Dialogue method for multi-strategy fusion in government service field based on knowledge graph | |
| CN108304372B (en) | Entity extraction method and device, computer equipment and storage medium | |
| CN110110062B (en) | Machine intelligent question and answer method and device and electronic equipment | |
| EP3096246A1 (en) | Method, system and storage medium for realizing intelligent answering of questions | |
| CN112035599B (en) | Query method and device based on vertical search, computer equipment and storage medium | |
| CN110287482B (en) | Semi-automatic participle corpus labeling training device | |
| CN110175585B (en) | Automatic correcting system and method for simple answer questions | |
| CN114357127A (en) | Intelligent question-answering method based on machine reading understanding and common question-answering model | |
| CN116501306A (en) | Method for generating interface document code based on natural language description | |
| CN111831810A (en) | Intelligent question and answer method, device, equipment and storage medium | |
| CN112115252A (en) | Intelligent auxiliary writing processing method and device, electronic equipment and storage medium | |
| CN114266256A (en) | Method and system for extracting new words in field | |
| CN115905487A (en) | Document question and answer method, system, electronic equipment and storage medium | |
| CN117370190A (en) | Test case generation method and device, electronic equipment and storage medium | |
| CN101114282A (en) | Participle processing method and equipment | |
| CN111062832A (en) | Auxiliary analysis method and device for intelligently providing patent answer and debate opinions | |
| CN116186259A (en) | Session cue scoring method, device, equipment and storage medium | |
| CN117216226A (en) | Knowledge positioning method, device, storage medium and equipment | |
| CN115238705A (en) | Semantic analysis result reordering method and system | |
| CN114417010A (en) | Knowledge graph construction method and device for real-time workflow and storage medium | |
| CN114238595A (en) | Metallurgical knowledge question-answering method and system based on knowledge graph | |
| CN114239555A (en) | Training method of keyword extraction model and related device | |
| CN112905747A (en) | Professional system archive question-answering robot system based on semantic analysis technology | |
| CN117909494B (en) | Abstract consistency assessment model training method and device | |
| CN116842128B (en) | Text relation extraction method and device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |