CN117370580A - Knowledge-graph-based large language model enhanced dual-carbon field service method - Google Patents
Knowledge-graph-based large language model enhanced dual-carbon field service method Download PDFInfo
- Publication number
- CN117370580A CN117370580A CN202311435057.XA CN202311435057A CN117370580A CN 117370580 A CN117370580 A CN 117370580A CN 202311435057 A CN202311435057 A CN 202311435057A CN 117370580 A CN117370580 A CN 117370580A
- Authority
- CN
- China
- Prior art keywords
- carbon
- language model
- knowledge
- double
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 229910052799 carbon Inorganic materials 0.000 title claims abstract description 132
- 238000000034 method Methods 0.000 title claims abstract description 50
- QVFWZNCVPCJQOP-UHFFFAOYSA-N chloralodol Chemical compound CC(O)(C)CC(C)OC(O)C(Cl)(Cl)Cl QVFWZNCVPCJQOP-UHFFFAOYSA-N 0.000 claims abstract description 34
- 238000006386 neutralization reaction Methods 0.000 claims abstract description 32
- OKTJSMMVPCPJKN-UHFFFAOYSA-N Carbon Chemical compound [C] OKTJSMMVPCPJKN-UHFFFAOYSA-N 0.000 claims abstract description 24
- 230000002708 enhancing effect Effects 0.000 claims abstract description 9
- 238000000605 extraction Methods 0.000 claims abstract description 6
- 238000012549 training Methods 0.000 claims description 23
- 239000011159 matrix material Substances 0.000 claims description 20
- 238000012545 processing Methods 0.000 claims description 7
- 239000000463 material Substances 0.000 claims description 3
- 238000011084 recovery Methods 0.000 abstract description 2
- 238000010276 construction Methods 0.000 description 6
- 238000011156 evaluation Methods 0.000 description 5
- 238000013519 translation Methods 0.000 description 5
- 230000014616 translation Effects 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 238000012360 testing method Methods 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000005431 greenhouse gas Substances 0.000 description 2
- 230000003014 reinforcing effect Effects 0.000 description 2
- 239000013598 vector Substances 0.000 description 2
- 235000002198 Annona diversifolia Nutrition 0.000 description 1
- 241000282842 Lama glama Species 0.000 description 1
- CBENFWSGALASAD-UHFFFAOYSA-N Ozone Chemical compound [O-][O+]=O CBENFWSGALASAD-UHFFFAOYSA-N 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000001939 inductive effect Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 238000009966 trimming Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
- G06F16/367—Ontology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/332—Query formulation
- G06F16/3329—Natural language query formulation or dialogue systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
- G06F16/3344—Query execution using natural language analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/166—Editing, e.g. inserting or deleting
- G06F40/186—Templates
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
- G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
- G06F40/295—Named entity recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/02—Knowledge representation; Symbolic representation
- G06N5/022—Knowledge engineering; Knowledge acquisition
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Artificial Intelligence (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Mathematical Physics (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- General Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Animal Behavior & Ethology (AREA)
- Human Computer Interaction (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a service method for enhancing a large language model double-carbon field based on a knowledge graph, and relates to the technical field of double-carbon. The method is based on a two-carbon field question-answer data set, and adopts a LoRA method to finely adjust a large language model, so that the extraction capability of the model on carbon-to-peak carbon neutralization field keywords is improved; constructing a double-carbon knowledge graph as a local knowledge base to provide knowledge in the double-carbon field for the model, taking the knowledge as the context of the problem, allowing a large language model to learn, and designing and prompting an engineering auxiliary model to generate a reply. The invention enables the large language model to learn domain knowledge by fine tuning the large language model, enables the large language model to answer questions according to the context, effectively utilizes the powerful performance of the large language model, builds a knowledge service model in a specific domain, and realizes the intelligent recovery in the carbon-to-carbon neutralization domain.
Description
Technical Field
The invention relates to the technical field of double carbon, in particular to a service method for reinforcing a large language model double carbon field based on a knowledge graph.
Background
The increase of greenhouse gases causes a series of problems such as greenhouse effect, ozone layer cavity and the like, so that climate change causes unprecedented scale influence on the global scope. Currently, global consensus is reached, climate change is actively dealt with, and emission reduction of greenhouse gases mainly comprising CO2 is promoted. Currently, nearly 2/3 of the world's countries define carbon neutralization targets, and 80% of economics promise to achieve carbon neutralization positively.
In recent years, scholars at home and abroad have made many researches on application of knowledge maps in the carbon-to-carbon neutralization field, for example, knowledge extraction is performed on carbon trade triples based on a BiLSTM-CRF model, a knowledge map construction method oriented to the carbon trade field is provided, a knowledge map of a carbon market is constructed based on BERT-CRF, and a Neo4j database is used for storage and analysis, so that a map construction method in the carbon market field is provided. However, at present, the knowledge graph research in the carbon-to-carbon neutralization field is still in a starting stage, and no matter the knowledge arrangement in the carbon-to-carbon neutralization field or the knowledge service question-and-answer construction based on the two-carbon business is still required to be continuously developed.
In recent years, with the continuous development of large-scale language models, the prospects of human-computer interaction and intelligence are rapidly changing. The open language models such as the GPT large model of OpenAI and the LLaMA large model of Meta are continuously updated and surpassed, and the ChatGPT of OpenAI is paid attention to once being pushed out. In 2023, the advent of GPT-4 re-ignited the hot tide of artificial intelligence development. Large language models have become essential building blocks for intelligent question-answering and generating applications. Based on the large model, the intelligent question-answering system is able to better understand human language and generate consistent, logical answers.
Knowledge Graph (knowledgegraph) was first proposed by Google in 2012, and with the continuous development of artificial intelligence and semantic networks, knowledge Graph is an important data processing technology and data representation format, and research in recent years is continuously advancing, and the Knowledge Graph shows unique advantages in understanding and applying Knowledge in complex fields. The knowledge graph is essentially a semantic network, nodes of the knowledge graph represent entities, edges of the knowledge graph represent semantic relations among the entities, the knowledge graph is formed by connection between the entities and the relations, and data are stored by taking an entity-relation-entity triplet as a basic composition unit. Compared with the traditional data storage method, the knowledge graph can enable users to analyze the relation between data well, has semantic deducing capability, and is widely applied to the fields of medical treatment, electronic commerce, finance and the like based on knowledge questions and answers, knowledge searching, data analysis and the like.
The method and the device for training the large language model based on the knowledge graph in the Chinese patent CN116662577A are as follows: firstly, financial knowledge graph is built by taking financial history entities and history events as nodes and financial history relations as edges, then financial events used for training are arranged, related events are derived based on the knowledge graph, event sets are converted into event vectors, vector matrixes are built, similarity of the events is calculated, reasoning target events are determined, data are processed, training samples are obtained, and the training samples are utilized to finely tune a pre-trained large language model, so that a financial prediction model is obtained. The method only carries out fine tuning training on the large language model, but the fine tuned large model still has the illusion problem, and the knowledge graph is not introduced as a knowledge base to enhance the field reliability of the large language model.
The method and the device for question answering based on the knowledge graph of the Chinese patent CN116303970A are as follows: extracting an entity from a question of a user, determining a topic of a knowledge graph according to the entity, determining the similarity of the entity mention according to the entity, and determining candidate similarity according to the topic entity, the knowledge graph and the question of the user. And pushing an answer to the user according to the entity mentioned phase velocity and the candidate similarity. The method lacks complete supplement and humanized design of answers to users, and the answers are matched only according to questions of the users.
Although the large language model is excellent in the fields such as information retrieval, the use of the large language model directly in a specific field may result in the occurrence of a case where an answer or an answer error is created.
Disclosure of Invention
Aiming at the defects of the prior art, the technical problem to be solved by the invention is to provide a dual-carbon field service method for enhancing a large language model based on a knowledge graph, which is used for fine tuning the large language model, learning field knowledge by the large model, and enabling the large language model to answer questions according to the context, so that the powerful performance of the large language model is effectively utilized, a knowledge service model in a specific field is constructed, and the intelligent recovery in the carbon-to-carbon peak neutralization field is realized.
In order to solve the technical problems, the invention adopts the following technical scheme:
a method for enhancing the service of a large language model in a double-carbon field based on a knowledge graph is based on a double-carbon field question-answer data set, and adopts a LoRA method to finely tune the large language model, so that the extraction capability of the model on keywords in the carbon-peak-carbon neutralization field is improved; constructing a double-carbon knowledge graph as a local knowledge base to provide knowledge in the double-carbon field for the model, taking the knowledge as the context of the problem, allowing a large language model to learn, and designing and prompting an engineering auxiliary model to generate a reply.
Further, the detail steps of fine tuning the large language model by using the LoRA method are as follows:
step 1: the literature in the double-carbon field is collated and generalized, and question-answer pairs in the double-carbon field are constructed;
step 2: processing the question-answer pairs into a format conforming to LoRA fine tuning to form a two-carbon field question-answer data set;
step 3: and (3) amplifying the double-carbon field question-answer data set to the LoRA fine tuning interface position of the large language model, and carrying out LoRA fine tuning of the large language model.
Further, the method for fine tuning LoRA comprises the following steps: adding a new path beside the original pre-training large model, wherein the middle dimension is r, and simulating the intrinsic rank by multiplying the dimension-reducing matrix A and the dimension-increasing matrix B; the matrix A is initialized by random Gaussian distribution, the matrix B is initialized by matrix 0, only the matrix A, B is trained during training, and BA and original parameters of the pre-training large model are overlapped and output during output.
Further, the training data set constructed when the large language model is subjected to fine tuning comprises 3 types, namely, extracting task questions and answers for entity relations, named entity recognition task questions and answers and double-carbon business knowledge questions and answers, and constructing the data set by referring to the data format of the LoRA fine tuning large model.
Further, after the large language model is finely adjusted, the knowledge in the carbon-to-carbon neutralization field is arranged, a double-carbon knowledge graph is constructed, after a user inputs a problem, keywords are firstly extracted from the large language model, then search matching is carried out on the double-carbon knowledge graph based on the keywords, then a search result is combined with a large model prompt template to be used as the upper part of the user problem to be spliced with the user problem, the spliced large model is transmitted, and finally a reply of the large model is obtained.
Further, the specific implementation steps of the two-carbon knowledge graph are as follows:
step 4: combing the data materials of the double-carbon key guidelines, policies, double-carbon field related papers and enterprise carbon emission, constructing a double-carbon knowledge graph data set, and transmitting the data set to a neo4j database to construct a double-carbon knowledge graph;
step 5: acquiring a user problem, and extracting keywords from the user problem by using a large language model;
step 6: based on the extracted user problem keywords, carrying out query matching in a double-carbon knowledge graph to obtain a query result triplet;
step 7: constructing a prompt project, combining the query result with a large model prompt template, and splicing the user problem with the user problem as the user problem;
step 8: inputting the prompt engineering into the large language model subjected to LoRA fine tuning to obtain a result of the large language model;
step 9: and processing the result of the large language model, and extracting the user problem replies.
Further, in the step 4, the two-carbon knowledge-graph dataset is constructed in a format of entity 1-entity 2-entity relationship-entity 1 attribute-entity 2 attribute.
The beneficial effects of adopting above-mentioned technical scheme to produce lie in: according to the dual-carbon field service method based on the knowledge graph enhanced large language model, fine adjustment is performed on the large model, knowledge of the large model on field knowledge is improved, meanwhile, the capability of the large language model for extracting keywords is enhanced, then key fact knowledge is provided in a mode that the dual-carbon knowledge graph is used as a local database, and the large model is assisted in providing accurate field answers. The large language model is used for two rounds, and the excellent capability of the large model in the NLP field is fully utilized. The invention constructs a double-carbon knowledge graph and a double-carbon question-answer data set, combines the knowledge graph with a large language model, provides knowledge service in the carbon-to-carbon neutralization field, can realize the answer to accurate numbers, for example, the enterprise carbon emission takes the double-carbon knowledge graph as a local database, and enables the large language model to learn by providing accurate knowledge, thereby realizing the accurate answer to the carbon-to-carbon neutralization field; the reply result of the large language model can be changed by changing the knowledge graph, repeated training of the large language model can be avoided, and resources are saved greatly; and the excellent capability of the large language model is combined with a knowledge graph in the carbon-to-carbon neutralization field to construct a knowledge service in the carbon-to-carbon neutralization field, so as to assist the carbon-to-carbon neutralization construction.
Drawings
FIG. 1 is a schematic diagram of a LoRA fine tuning model provided by an embodiment of the present invention;
FIG. 2 is a schematic diagram of a two-carbon knowledge graph according to an embodiment of the present invention;
FIG. 3 is a flowchart of a method for enhancing a large language model by using a knowledge graph according to an embodiment of the present invention;
fig. 4 is a flowchart of an example of a knowledge graph enhancement large language model according to an embodiment of the present invention.
Detailed Description
The following describes in further detail the embodiments of the present invention with reference to the drawings and examples. The following examples are illustrative of the invention and are not intended to limit the scope of the invention.
The method of the embodiment adopts the LoRA method to finely adjust the large language model, improves the extraction capability of the model on keywords in the carbon-to-carbon neutralization field, constructs a double-carbon knowledge graph as a local knowledge base to provide knowledge in the double-carbon field for the model, learns the large language model by taking the knowledge as the context of the problem, designs and prompts an engineering auxiliary model to generate a reply, and enhances the service in the double-carbon field of the large language model by the knowledge graph, thereby realizing the intelligent reply in the carbon-to-carbon neutralization field.
The specific implementation steps of the LoRA fine tuning large language model are as follows:
step 1: and (3) sorting and summarizing documents in the double-carbon field, and constructing question-answer pairs in the double-carbon field.
Step 2: and processing the question-answer pairs into a format conforming to the LoRA fine tuning to form a two-carbon field question-answer data set.
Step 3: and (3) amplifying the double-carbon field question-answer data set to the LoRA fine tuning interface position of the large language model, and carrying out LoRA fine tuning of the large language model.
According to the method, a new path is added beside an original pre-trained large model, the middle dimension is r, and the eigen rank is simulated by multiplying a dimension-reducing matrix A and a dimension-increasing matrix B. The matrix A is initialized by random Gaussian distribution, the matrix B is initialized by matrix 0, only the matrix A, B is trained during training, and the BA and the original parameters of the pre-training large model are overlapped and output during output, as shown in figure 1. The LoRA tuning method has the advantage that tasks can be efficiently switched by replacing the dimension matrix A with the dimension matrix B. Meanwhile, the LoRA fine tuning method allows the weight combination of the LoRA fine tuning method and the pre-training large model during deployment, and compared with a fully fine-tuned model, the LoRA fine tuning method is constructed without introducing inference delay, and the principle of the LoRA fine tuning method is shown in figure 1.
In this embodiment, the training data set constructed by fine tuning is divided into 3 types, the data set includes an entity relation extraction task question-answer, a named entity recognition task question-answer and a two-carbon business knowledge question-answer, 1500 pieces of data are obtained altogether, the data is constructed by referring to the data format of the LoRA fine tuning large model, and examples of the data set are shown in table 1.
Table 1 example of the construction of the LoRA fine-tuning large language model data
The experimental environment of this embodiment is configured to: GPU, T4; and (3) video memory: 15GB; python version 3.9.17; cuda version 11.3. The large model dimension trim parameter settings are shown in table 2.
Table 2 large language model trim parameter settings
| Parameters (parameters) | Chinese meaning | Parameter value |
| per_device_train_batch_size | Designating training batch size for each training device | 2 |
| per_device_eval_batch_size | Specifying an evaluation lot size for each evaluation device | 2 |
| max_steps | Specifying a maximum number of steps for training | 5000 |
| save_steps | Specifying how many steps to protectModel for storing once | 1000 |
| learning_rate | Learning rate | 1e-3 |
And fixing the trimmed parameters on the large language model, selecting 100 test sets to evaluate and compare the trimmed large language model with the non-trimmed large language model, namely BLEU-4, and measuring the similarity between sentences generated by the machine translation system and the reference translations. ROUGE-1: ROUGE-1 measures the overlap of individual words between a generated abstract or translation and a reference text. ROUGE-2: ROUGE-2 measures the overlap of two consecutive phrases between a generated abstract or translation and a reference text. ROUGE-L: the ROUGE-L measures the length of the Longest Common Subsequence (LCS) between the generated abstract or translation and the reference text, and the test results are shown in Table 3.
Table 3 comparison of the lorea fine tuning model evaluation
| Evaluation index | Original large language model | Fine tuning large language model |
| BLEU-4 | 15.8764 | 81.1775 |
| ROUGE-1 | 33.4601 | 88.5555 |
| ROUGE-2 | 18.7778 | 87.0452 |
| ROUGE-L | 26.0457 | 88.1565 |
By comparing the evaluation indexes of the original large language model and the trimmed large language model aiming at the carbon-to-carbon neutralization field test set, the answer effect of the double-carbon training data set on the carbon-to-carbon neutralization field is greatly improved after the large language model is trimmed, and the feasibility of the LoRA trimming large model is proved.
Although the large language model has been subjected to LoRA fine tuning on the constructed knowledge training set, the large language model has the illusion problem, forgetting the fact or knowledge which has been trained, so that incorrect answers are returned, the reliability of the large model is reduced, the application of the large language model in some carbon peak carbon neutralization fields is affected, for example, the application of inquiring the carbon emission of a certain enterprise for a certain year is affected, and the large language model can return incorrect values, so that the analysis of the carbon emission of the enterprise is affected.
In order to solve the above problems, the present embodiment provides a method of constructing a carbon-to-peak carbon neutralization domain knowledge graph and integrating into a large language model to enhance answers to the large language model. The knowledge graph has wide application, can store a large amount of knowledge in the form of triples, can provide accurate knowledge for users, and can update new knowledge.
Because the entity in the carbon-to-carbon neutralization field is complex, and the entity and entity relation are difficult to extract, the embodiment manually sorts the important data in the carbon-to-carbon neutralization field, including the important data such as the construction guidelines of the carbon-to-carbon neutralization standard system, the carbon emission amount calculation method and the like, and setting policies, contents, systems, standards, methods, enterprises and data as main types by inducing key knowledge in the enterprise carbon emission real data obtained from the carbon detection network, and arranging a double-carbon data set by taking an entity-entity relationship-entity type as a format. Based on the double-carbon data set, a py2neo database is called through python to connect with a neo4j database, a database statement is constructed, a knowledge graph is automatically constructed in the neo4j database, the constructed double-carbon knowledge graph comprises 256 entities and 17 entity relations, the double-carbon knowledge graph is shown in fig. 2, and the fig. 2 comprises nodes and entity relations with partial entity attributes of standards, contents and policies.
Firstly, carrying out LoRA fine tuning on a large language model, deepening the recognition of the model on the double-carbon field, simultaneously improving the recognition capability of a named entity of the model, then arranging the knowledge in the carbon-to-carbon neutralization field, constructing a double-carbon knowledge graph, after a user inputs a problem, firstly extracting keywords from the large language model, then carrying out search matching on the double-carbon knowledge graph based on the keywords, then combining the search result with a large model prompt template, splicing the search result with the user problem as the user problem, transmitting the result to the large model, finally obtaining the reply of the large model, and specifically constructing a method for reinforcing the large language model by the knowledge graph, wherein the specific implementation steps are as follows:
step 4: and combing the data materials of the double-carbon key guidelines, policies, double-carbon field related papers and enterprise carbon emission, constructing a double-carbon knowledge graph data set in a format of entity 1-entity 2-entity relationship-entity 1 attribute-entity 2 attribute, and transmitting the data to a neo4j database to construct a double-carbon knowledge graph.
Step 5: and acquiring the user problem, and extracting keywords from the user problem by using a large language model.
Step 6: and carrying out query matching in the double-carbon knowledge graph based on the extracted user problem keywords to obtain a query result triplet.
Step 7: and constructing a prompt project, combining the query result with the large model prompt template, and splicing the user problem with the user problem as the user problem.
Step 8: inputting the prompt engineering into the large language model to obtain the result of the large language model.
Step 9: and processing the result of the large language model, and extracting the user problem replies.
In this embodiment, the tuned large language model is named ChatGLM-6B-32k-lora, and although the answer of the trimmed large language model to the dual-carbon field is greatly improved, the situation that the enterprise is still under the condition of building and inaccurate positioning still occurs for the answer of accurate numbers, such as enterprise carbon emission, so that the dual-carbon knowledge graph is taken as a local database, and the large language model is learned by providing accurate knowledge, thereby realizing the accurate answer to the carbon-peak carbon neutralization field, and an example of enhancing the large language model by the dual-carbon knowledge graph is shown in fig. 4.
A carbon-peak-carbon neutralization knowledge service system is constructed based on Vue3 and flash technology, and a method for enhancing a large language model by using a double-carbon knowledge graph is applied to the carbon-peak-carbon neutralization knowledge service system, so that knowledge question and answer in the field of carbon-peak-carbon neutralization is realized.
In order to realize knowledge question and answer for the carbon-to-peak carbon neutralization field by deploying the dual-carbon knowledge graph and the large language model into the question and answer system, the embodiment inquires about the original large language model, the trimmed large language model and the design of some representative questions based on the knowledge graph enhanced large language model, and compares answers of the three methods, thereby proving the feasibility of the knowledge graph enhanced large language model, and test results are shown in table 4.
TABLE 4 question-answer analysis
The result shows that the effect of directly using the large language model for output is not ideal and has an excessive difference from the actual answer. After LoRA fine tuning is carried out on the model, the output result starts to be close to a correct answer, but compared with an actual answer, larger deviation still exists, the result output by the method for enhancing the large language model based on the knowledge graph quite correctly answers the questions, the effect is quite similar to the actual answer, and the result proves the feasibility of the method for enhancing the large language model based on the knowledge graph.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the corresponding technical solutions, which are defined by the scope of the appended claims.
Claims (7)
1. A method for enhancing service in a large language model double-carbon field based on a knowledge graph is characterized by comprising the following steps: the method is based on a two-carbon field question-answer data set, and adopts a LoRA method to finely adjust a large language model, so that the extraction capability of the model on carbon-peak-carbon neutralization field keywords is improved; constructing a double-carbon knowledge graph as a local knowledge base to provide knowledge in the double-carbon field for the model, taking the knowledge as the context of the problem, allowing a large language model to learn, and designing and prompting an engineering auxiliary model to generate a reply.
2. The knowledge-based large language model dual-carbon domain service method based on the knowledge graph enhancement as claimed in claim 1, wherein the method comprises the following steps: the detail steps of fine tuning the large language model by adopting the LoRA method are as follows:
step 1: the literature in the double-carbon field is collated and generalized, and question-answer pairs in the double-carbon field are constructed;
step 2: processing the question-answer pairs into a format conforming to LoRA fine tuning to form a two-carbon field question-answer data set;
step 3: and (3) amplifying the double-carbon field question-answer data set to the LoRA fine tuning interface position of the large language model, and carrying out LoRA fine tuning of the large language model.
3. The knowledge-based large language model dual-carbon domain service method based on the knowledge graph enhancement as claimed in claim 2, wherein the method is characterized by comprising the following steps: the method for fine tuning LoRA comprises the following steps: adding a new path beside the original pre-training large model, wherein the middle dimension is r, and simulating the intrinsic rank by multiplying the dimension-reducing matrix A and the dimension-increasing matrix B; the matrix A is initialized by random Gaussian distribution, the matrix B is initialized by matrix 0, only the matrix A, B is trained during training, and BA and original parameters of the pre-training large model are overlapped and output during output.
4. The knowledge-based enhanced large language model dual-carbon domain service method according to claim 3, wherein the method comprises the following steps: the training data set constructed when the large language model is subjected to fine tuning comprises 3 types, namely, extracting task questions and answers for entity relations, named entity recognition task questions and answers and double-carbon business knowledge questions and answers, and constructing the data set by referring to the data format of the LoRA fine tuning large model.
5. The knowledge-based large language model dual-carbon domain service method based on the knowledge graph enhancement as claimed in claim 1, wherein the method comprises the following steps: after the large language model is finely adjusted, the knowledge in the carbon-to-carbon neutralization field is arranged, a double-carbon knowledge graph is constructed, after a user inputs a problem, keywords are firstly extracted from the large language model, then search matching is carried out on the double-carbon knowledge graph based on the keywords, then a search result is combined with a large model prompt template, the search result is spliced with the user problem as the user problem, the spliced result is transmitted to the large model, and finally a reply of the large model is obtained.
6. The knowledge-based large language model dual-carbon domain service method based on the knowledge graph enhancement of claim 5, wherein the method comprises the following steps: the specific implementation steps of the double-carbon knowledge graph are as follows:
step 4: combing the data materials of the double-carbon key guidelines, policies, double-carbon field related papers and enterprise carbon emission, constructing a double-carbon knowledge graph data set, and transmitting the data set to a neo4j database to construct a double-carbon knowledge graph;
step 5: acquiring a user problem, and extracting keywords from the user problem by using a large language model;
step 6: based on the extracted user problem keywords, carrying out query matching in a double-carbon knowledge graph to obtain a query result triplet;
step 7: constructing a prompt project, combining the query result with a large model prompt template, and splicing the user problem with the user problem as the user problem;
step 8: inputting the prompt engineering into the large language model subjected to LoRA fine tuning to obtain a result of the large language model;
step 9: and processing the result of the large language model, and extracting the user problem replies.
7. The knowledge-based large language model dual-carbon domain service method based on the knowledge graph enhancement of claim 6, wherein the method comprises the following steps: in the step 4, the two-carbon knowledge graph dataset is constructed in a format of entity 1-entity 2-entity relationship-entity 1 attribute-entity 2 attribute.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311435057.XA CN117370580A (en) | 2023-11-01 | 2023-11-01 | Knowledge-graph-based large language model enhanced dual-carbon field service method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311435057.XA CN117370580A (en) | 2023-11-01 | 2023-11-01 | Knowledge-graph-based large language model enhanced dual-carbon field service method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117370580A true CN117370580A (en) | 2024-01-09 |
Family
ID=89390868
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311435057.XA Pending CN117370580A (en) | 2023-11-01 | 2023-11-01 | Knowledge-graph-based large language model enhanced dual-carbon field service method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117370580A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117573843A (en) * | 2024-01-15 | 2024-02-20 | 图灵人工智能研究院（南京）有限公司 | Knowledge calibration and retrieval enhancement-based medical auxiliary question-answering method and system |
| CN117708601A (en) * | 2024-02-06 | 2024-03-15 | 智慧眼科技股份有限公司 | Similarity calculation model training method, device, equipment and storage medium |
-
2023
- 2023-11-01 CN CN202311435057.XA patent/CN117370580A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117573843A (en) * | 2024-01-15 | 2024-02-20 | 图灵人工智能研究院（南京）有限公司 | Knowledge calibration and retrieval enhancement-based medical auxiliary question-answering method and system |
| CN117573843B (en) * | 2024-01-15 | 2024-04-02 | 图灵人工智能研究院（南京）有限公司 | Knowledge calibration and retrieval enhancement-based medical auxiliary question-answering method and system |
| CN117708601A (en) * | 2024-02-06 | 2024-03-15 | 智慧眼科技股份有限公司 | Similarity calculation model training method, device, equipment and storage medium |
| CN117708601B (en) * | 2024-02-06 | 2024-04-26 | 智慧眼科技股份有限公司 | Similarity calculation model training method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109271505B (en) | Question-answering system implementation method based on question-answer pairs | |
| CN109635083B (en) | Document retrieval method for searching topic type query in TED (tele) lecture | |
| CN109325040B (en) | FAQ question-answer library generalization method, device and equipment | |
| CN117370580A (en) | Knowledge-graph-based large language model enhanced dual-carbon field service method | |
| CN117009490A (en) | Training method and device for generating large language model based on knowledge base feedback | |
| CN111897944B (en) | Knowledge graph question-answering system based on semantic space sharing | |
| CN116127095A (en) | Question-answering method combining sequence model and knowledge graph | |
| Sovrano et al. | Legal knowledge extraction for knowledge graph based question-answering | |
| CN115599899B (en) | Intelligent question-answering method, system, equipment and medium based on aircraft knowledge graph | |
| CN116150335A (en) | Text semantic retrieval method under military scene | |
| CN117033721A (en) | Legal consultation report generation system and method based on legal knowledge graph | |
| CN116932776A (en) | Knowledge graph-based large model knowledge updating method and device | |
| CN115422323A (en) | Intelligent intelligence question-answering method based on knowledge graph | |
| CN115858750A (en) | Power grid technical standard intelligent question-answering method and system based on natural language processing | |
| CN117271792A (en) | Method for constructing enterprise domain knowledge base based on large model | |
| CN116561264A (en) | Knowledge graph-based intelligent question-answering system construction method | |
| CN116992007A (en) | Limiting question-answering system based on question intention understanding | |
| CN111666374A (en) | Method for integrating additional knowledge information into deep language model | |
| CN116932717A (en) | Knowledge base retrieval method and device based on large language model self-evaluation and self-feedback | |
| CN114579605B (en) | Table question-answer data processing method, electronic equipment and computer storage medium | |
| CN115964468A (en) | Rural information intelligent question-answering method and device based on multilevel template matching | |
| CN113326361B (en) | Knowledge question-answering method and system based on automobile industry map and electronic equipment | |
| CN115238705A (en) | Semantic analysis result reordering method and system | |
| CN115186073A (en) | Open domain table text question-answering method based on hybrid retrieval | |
| CN115810422B (en) | Internet intelligent automatic diagnosis and treatment response system based on AI technology |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |