CN117688150A - Method, device, processor and storage medium for aligning large language model with knowledge graph intent based on trigger word position coding - Google Patents

Method, device, processor and storage medium for aligning large language model with knowledge graph intent based on trigger word position coding Download PDF

Info

Publication number
CN117688150A
CN117688150A CN202311576591.2A CN202311576591A CN117688150A CN 117688150 A CN117688150 A CN 117688150A CN 202311576591 A CN202311576591 A CN 202311576591A CN 117688150 A CN117688150 A CN 117688150A
Authority
CN
China
Prior art keywords
knowledge
intention
trigger word
language model
knowledge graph
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311576591.2A
Other languages
Chinese (zh)
Inventor
孙亚茹
杨莹
王永剑
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Third Research Institute of the Ministry of Public Security
Original Assignee
Third Research Institute of the Ministry of Public Security
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Third Research Institute of the Ministry of Public Security filed Critical Third Research Institute of the Ministry of Public Security
Priority to CN202311576591.2A priority Critical patent/CN117688150A/en
Publication of CN117688150A publication Critical patent/CN117688150A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33—Querying
    • G06F16/332—Query formulation
    • G06F16/3329—Natural language query formulation or dialogue systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33—Querying
    • G06F16/3331—Query processing
    • G06F16/334—Query execution
    • G06F16/3344—Query execution using natural language analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/35—Clustering; Classification
    • G06F16/353—Clustering; Classification into predefined classes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
    • G06F16/367—Ontology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/30—Semantic analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/042—Knowledge-based neural networks; Logical representations of neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0499—Feedforward networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/02—Knowledge representation; Symbolic representation
    • G06N5/022—Knowledge engineering; Knowledge acquisition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/04—Inference or reasoning models
    • G06N5/041—Abduction

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Probability & Statistics with Applications (AREA)
  • Animal Behavior & Ethology (AREA)
  • Human Computer Interaction (AREA)
  • Machine Translation (AREA)

Abstract

The invention relates to a method for realizing the intention alignment processing of a large language model and a knowledge graph based on trigger word position coding, which comprises the following steps: acquiring user intention; decomposing the intention; performing instruction mapping; retrieving knowledge from a local knowledge graph base through the generated KG instruction; carrying out knowledge aggregation; the final answer is generated using LLM. The invention also relates to a device, a processor and a storage medium for realizing the alignment processing of the large language model and the knowledge graph intention based on the trigger word position coding. The method, the device, the processor and the computer readable storage medium thereof for realizing the alignment processing of the large language model and the intention of the knowledge graph based on the position code of the trigger word utilize the deep neural network model to realize the alignment of the intention of the user and the instruction, integrate the position information of the trigger word and ensure the accuracy of the query statement. The method utilizes the deep learning model to complete interaction between LLM and local KG in a time-saving and labor-saving manner, effectively generates information required by user intention, and has great innovation.

Description

Method, device, processor and storage medium for aligning large language model with knowledge graph intent based on trigger word position coding
Technical Field
The invention relates to the technical field of natural language processing, in particular to the field of deep learning models based on intention decomposition and alignment, and specifically relates to a method, a device, a processor and a computer readable storage medium for realizing intention alignment processing of a large-scale language model and a knowledge graph based on trigger word position coding.
Background
The LLM uses the deep learning network to learn from massive training data, achieves the effect of knowledge emergence under the addition of model parameters, has the understanding capability on text knowledge, has good reformation performance on a large language model, and can be well customized into an LLM method suitable for the field by introducing proprietary field knowledge and a post-processing algorithm. However, in constructing a vertical domain LLM, some problems with LLM itself are: first, there is a limitation in the results of model output. In some cases, the training data source may be biased or incomplete, resulting in a hypothesis or conclusion that the result is inconsistent with reality. And in statistical mode, the result will have meaningless results without contextual meaning; secondly, model instantaneity is poor. Model training data is large in scale, long in time consumption and slow in updating, cannot provide exact evidence of latest and verifiable information, and may generate inaccurate or outdated reactions; thirdly, the calculation force dependence is strong. During model training, the training parameters are increased at an exponential rate, resulting in a dramatic increase in the dependence of the model on computational power.
KG is widely used in domain knowledge engineering, and represents integrated data information in a structured form to provide accurate and interpretable explicit knowledge for downstream tasks. With the appearance of new knowledge, iteration can be updated in time under the condition of less hardware dependence so as to meet the requirements of existing knowledge query, retrieval and the like. Because KG has the characteristics of definite and accurate knowledge, quick updating and small calculation force resource, how to combine LLM and KG to create a strong intelligent application model of knowledge in the field, and the intelligent application model becomes a hot spot for research in the field of knowledge engineering.
Questions and answers to user interactions relate to the technical field of text semantic understanding and classification. Traditional text classification algorithms focus much on linear expressions of text, such as support vector machine models that use lexicons or n-gram word vectors as inputs. Recent years of research have shown that nonlinear models can effectively capture text context information and can produce more accurate predictions than linear models. In particular, the information is aggregated over discrete knowledge graph results. The convolutional neural network model is a typical nonlinear model that converts local features of data into low-dimensional vectors and retains information related to tasks. This efficient mapping method performs better on short text than the sequence model.
The graph convolution neural network acquires the characteristic information of the data area by adopting a mode of adjacent information aggregation. As the number of coding layers increases, the target-related positioning information is gradually lost. Text regions may express more complex concepts, and such learning by extracting region feature information with step size limitation may ignore mission critical information. In addition, the coupled connections between the network layers may increase redundancy of the model.
The attention mechanism is a method for effectively focusing on key information in model input data. The attention model not only pays attention to the characteristic information in the training process, but also effectively adjusts the parameters of the neural network aiming at different characteristics, so that more hidden characteristic information related to tasks can be mined.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides a method, a device, a processor and a computer readable storage medium thereof for realizing the purpose of aligning a large language model with a knowledge graph based on trigger word position coding, wherein the method, the device, the processor and the computer readable storage medium have high timeliness, high accuracy and wide application range.
In order to achieve the above object, the method, the device, the processor and the computer readable storage medium thereof for realizing the alignment processing of the large language model and the knowledge graph intention based on the trigger word position coding according to the invention are as follows:
the method for realizing the alignment processing of the large language model and the intention of the knowledge graph based on the trigger word position coding is mainly characterized by comprising the following steps:
(1) Acquiring user intention, arranging user questions into a serialized text embedded model, and judging the intention of the user questions through semantic coding analysis;
(2) Decomposing the intention, and selecting different processing modes according to the intention type;
(3) Performing instruction mapping, performing guide scoring strategy on the combination of trigger words during instruction coding, and decoding to generate KG instructions;
(4) Retrieving knowledge from a local knowledge graph base through the generated KG instruction;
(5) Carrying out knowledge aggregation;
(6) The final answer is generated using LLM.
Preferably, the step (1) specifically includes the following steps:
(1.1) word segmentation of text using an external tool;
(1.2) invoking a pre-trained word vector library to perform sequence mapping;
(1.3) encoding the sequence through a multi-layer graph neural network to refine the intent information;
and (1.4) carrying out softmax classification through a layer of feedforward neural network to obtain the intended category information.
Preferably, the step (2) specifically includes the following steps:
and adopting a general LLM answer to the intention questions not belonging to the field, further decomposing the intention questions belonging to the field, and adopting field trigger words to decompose the complex questions into LLM instructions with finer granularity.
Preferably, the step (3) specifically includes the following steps:
(2.1) cutting the instruction into words, performing sequence mapping through a pre-trained word vector library, and recording trigger words;
(2.2) coding calculation is carried out on the sequence data by adopting a multi-layer graph neural network, trigger word position coding information is added during calculation, and first-layer coding information is added;
and (2.3) constructing a trigger word attention matrix, and adjusting the performance of the model by adopting a guide scoring strategy.
Preferably, the step (5) specifically includes:
and inputting the retrieved knowledge and the user questions to the LLM, and calling related tool convergence information by the LLM triggering instruction.
The device for realizing the alignment processing of the large language model and the knowledge graph intention based on the trigger word position coding is mainly characterized by comprising the following components:
a processor configured to execute computer-executable instructions;
and the memory stores one or more computer executable instructions which, when executed by the processor, implement the steps of the method for realizing the alignment processing of the large language model and the intent of the knowledge graph based on the trigger word position coding.
The processor for realizing the large language model and knowledge graph intent alignment based on the trigger word position coding is mainly characterized in that the processor is configured to execute computer executable instructions, and when the computer executable instructions are executed by the processor, the steps of the method for realizing the large language model and knowledge graph intent alignment based on the trigger word position coding are realized.
The computer readable storage medium is mainly characterized in that the computer program is stored thereon, and the computer program can be executed by a processor to realize the steps of the method for realizing the alignment processing of the large language model and the intention of the knowledge graph based on the trigger word position coding.
The method, the device, the processor and the computer readable storage medium for realizing the intent alignment processing of the large language model and the knowledge graph based on the trigger word position coding can supplement LLM according to local knowledge to realize the accurate question-answering of domain knowledge. The problems of deviation, long training time, slow knowledge updating, strong hardware dependence and the like of a model result are solved. On the one hand, from the user intention, the intention belongs to the field, the field range is defined, the intention task is divided in finer granularity, the LLM instruction is called better, and accurate answers are generated in a converging mode. On the other hand, the knowledge base needs accurate query sentences for searching, and the generation of KG instructions is particularly important. And starting from the instruction semantics and the position coding of the query trigger words, generating the KG instruction by adopting a guide scoring strategy tuning model. The invention uses the deep neural network model to align the user intention and the instruction so as to better retrieve the answer from the knowledge base. And the text semantic coding is performed, and meanwhile, the position information of the trigger words is fused, so that the accuracy of the query statement is ensured. The method utilizes the deep learning model to complete interaction between LLM and local KG in a time-saving and labor-saving manner, effectively generates information required by user intention, and has great innovation.
Drawings
Fig. 1 is a schematic structural diagram of an apparatus for implementing the alignment process of a large language model and a knowledge graph intent based on trigger word position coding according to the present invention.
FIG. 2 is a flow chart of a method for realizing intent alignment processing of a large language model and a knowledge graph based on trigger word position coding.
Detailed Description
In order to more clearly describe the technical contents of the present invention, a further description will be made below in connection with specific embodiments.
The method for realizing the intent alignment processing of the large language model and the knowledge graph based on the trigger word position coding comprises the following steps:
(1) Acquiring user intention, arranging user questions into a serialized text embedded model, and judging the intention of the user questions through semantic coding analysis;
(2) Decomposing the intention, and selecting different processing modes according to the intention type;
(3) Performing instruction mapping, performing guide scoring strategy on the combination of trigger words during instruction coding, and decoding to generate KG instructions;
(4) Retrieving knowledge from a local knowledge graph base through the generated KG instruction;
(5) Carrying out knowledge aggregation;
(6) The final answer is generated using LLM.
As a preferred embodiment of the present invention, the step (1) specifically includes the steps of:
(1.1) word segmentation of text using an external tool;
(1.2) invoking a pre-trained word vector library to perform sequence mapping;
(1.3) encoding the sequence through a multi-layer graph neural network to refine the intent information;
and (1.4) carrying out softmax classification through a layer of feedforward neural network to obtain the intended category information.
As a preferred embodiment of the present invention, the step (2) specifically includes the following steps:
and adopting a general LLM answer to the intention questions not belonging to the field, further decomposing the intention questions belonging to the field, and adopting field trigger words to decompose the complex questions into LLM instructions with finer granularity.
As a preferred embodiment of the present invention, the step (3) specifically includes the following steps:
(2.1) cutting the instruction into words, performing sequence mapping through a pre-trained word vector library, and recording trigger words;
(2.2) coding calculation is carried out on the sequence data by adopting a multi-layer graph neural network, trigger word position coding information is added during calculation, and first-layer coding information is added;
and (2.3) constructing a trigger word attention matrix, and adjusting the performance of the model by adopting a guide scoring strategy.
As a preferred embodiment of the present invention, the step (5) specifically includes:
and inputting the retrieved knowledge and the user questions to the LLM, and calling related tool convergence information by the LLM triggering instruction.
The device for realizing the alignment processing of the large language model and the knowledge graph intention based on the trigger word position coding comprises:
a processor configured to execute computer-executable instructions;
and the memory stores one or more computer executable instructions which, when executed by the processor, implement the steps of the method for realizing the alignment processing of the large language model and the intent of the knowledge graph based on the trigger word position coding.
The processor for realizing the large language model and the knowledge graph intent alignment based on the trigger word position code is configured to execute computer executable instructions, and when the computer executable instructions are executed by the processor, the steps of the method for realizing the large language model and the knowledge graph intent alignment based on the trigger word position code are realized.
The computer readable storage medium of the present invention has a computer program stored thereon, the computer program being executable by a processor to perform the steps of the method for implementing the large language model and the knowledge graph intent alignment based on trigger word position coding.
In the specific implementation mode of the invention, the method for aligning the large language model and the knowledge graph intention based on the trigger word position coding is protected. Aiming at the problems of knowledge limitation, timeliness, calculation force dependence and the like in the prior art, the invention provides a large language model and knowledge graph intention alignment method based on a deep learning model.
According to the deep learning model-based large language model and knowledge graph intent alignment method, a user problem is segmented into subtasks from the field dimension, the subtasks are decomposed into different LLM instructions, the sentence semantic is encoded through a multi-layer graph neural network, positioning encoding information of trigger words is fused during encoding, and the mapping performance of the model to the instructions is reversely optimized by combining a scoring guidance strategy, so that an accurate KG instruction is generated. Finally, the knowledge is searched from the local knowledge graph base through the generated knowledge query statement, and after the knowledge result is returned, the knowledge result and the user question are input into the LLM together for analysis, so that a final answer is generated.
The method comprises the following steps:
step one: the user intent is obtained. And (3) arranging the user questions into a serialized text embedded model, and judging the intention of the user questions through semantic coding analysis. First, text is segmented using an external tool. Then, invoking a pre-trained word vector library to perform sequence mapping. The sequence is then encoded by a multi-layer graph neural network, refining the intent information. Finally, a layer of feedforward neural network is used for carrying out softmax classification to obtain the intended category information.
Step two: and (5) intention decomposition. Different processing modes are selected according to the intention category. Generic LLM answers are employed for intent questions not belonging to the domain. Further decomposition is performed on the intent questions belonging to the field, and the complex questions are decomposed into finer granularity LLM instructions. The resolution of complex questions adopts domain trigger word resolution, for example, trigger words in the social domain include personnel, relationships among personnel, personnel attributes and the like.
Step three: instruction mapping. And when the instruction is encoded, a guide scoring strategy is combined with the trigger word, and the KG instruction is decoded and generated. Firstly, cutting a word by an instruction, performing sequence mapping through a pre-trained word vector library, and recording a trigger word; then, coding calculation is carried out on the sequence data by adopting a multi-layer graph neural network, and trigger word position coding information is added during calculation; in addition, in order not to lose the most initial information, the information of the first layer coding is added in each layer of calculation; then, constructing a trigger word attention matrix, and adjusting the performance of the model by adopting a guide scoring strategy to enable the model to generate the nearest KG instruction.
Step four: and (5) knowledge retrieval. And retrieving knowledge from the local knowledge graph base through the generated KG instruction.
Step five: and (5) knowledge gathering. And the retrieved knowledge and the user problem are input into the LLM together, and the LLM triggers an instruction to call related tool convergence information. The user questions are open and LLM answers are a broad collection of information. And the retrieval knowledge and the questions are input to the LLM together, so that the answer range is reduced, and the accuracy of information generated by the LLM is improved.
Step six: an answer is generated. The final answer is generated by LLM and can be presented to the customer in text or file form.
Taking multi-language mixed short text for distinguishing Chinese and English as an example, the multi-language mixed short text classification method of the invention comprises the following steps:
1. and judging the intention of the user. First, text data is segmented. For example, the user problem is as follows: please provide the latest week's a event information and generate public opinion report. The result after the segmentation is: { 'please provide' last 'week' the 'A' event 'information' and 'generate' public opinion 'report'. Next, the word after segmentation is embedded as X= { X by word2vec 1 ,x 2 ,...,x n }. Then, the sequence is encoded through the L-layer graph neural network in the following way,
wherein W is l Is a weight matrix, b l Is the vector of the deviation and,is x j Inputs on the layer 1 neural network. Then, by->Converting word vectors into sentence vectors, d is the dimension of the vectors, outputting text category characteristics by text information characteristics through a layer of feedforward neural network FFNN (& gt), and predicting probability distribution of multiple categories of the text by adopting softmax & lt & gt>
P＝softmax(FFNN(f(h L )))……(2)
Wherein h is L Representing the output of the last L layer. Finally, by probability distributionThe category of the output intention is calculated, for example, P corresponds to two categories: domain, non-domain. If P {0.99,0.01}, the value of the corresponding "domain" category label is the largest, i.e. the model judges that the result of the intention category of the user is the "domain" knowledge question-answer.
2. And (5) intention decomposition. And extracting trigger words in the problem and dividing the trigger words into subtasks. The trigger words { ' information collection ' public opinion report ' week ' }, task set { ' A event information collection ' report template tool in week ' call ' }, corresponding LLM instruction generates ' } for { ' query the A event information of the last week '.
3. Instruction mapping. And (3) word segmentation is carried out on the instruction, after word segmentation, the word2vec is embedded into an N-layer graph neural network, the information K of the trigger word is fused during encoding, and a scoring matrix Q is created through the trigger word attention matrix K, so that an accurate KG instruction is generated. For example, the trigger words of 'inquiring for the a event information of the last week' are 'event information' and 'week'. The KG instruction generated is 'Match (n: event { event_name: "A" }, time: time1-time 2) - - (p: person) - - (d: location) return n'. The mapping objective function is calculated as follows,
J(θ)＝E[Q 1:n |h l ,θ]＝∑ y∈Y GCN(h l )·K(h l |y)……(3)
wherein E is the desired value, Y is the target value, Y is the target value set, Q 1:n Is the prize value for the complete sequence,is the characteristic value inside the data after the graph convolution.
4. And (5) knowledge retrieval. And searching the local knowledge by adopting the generated KG instruction, and searching out people and places related to the event within a week.
And (5) knowledge gathering. The user questions and the knowledge retrieval result are input into the LLM together, a trigger instruction is sent, the corresponding Agent is called to output a line text of the public opinion report in combination with the knowledge retrieval result, and the line text can be displayed to a client in the form of text or file.
The specific implementation manner of this embodiment may be referred to the related description in the foregoing embodiment, which is not repeated herein.
It is to be understood that the same or similar parts in the above embodiments may be referred to each other, and that in some embodiments, the same or similar parts in other embodiments may be referred to.
It should be noted that in the description of the present invention, the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. Furthermore, in the description of the present invention, unless otherwise indicated, the meaning of "plurality" means at least two.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and further implementations are included within the scope of the preferred embodiment of the present invention in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present invention.
It is to be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution device. For example, if implemented in hardware, as in another embodiment, may be implemented using any one or combination of the following techniques, as is well known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.
Those of ordinary skill in the art will appreciate that all or part of the steps carried out in the method of the above embodiments may be implemented by a program to instruct related hardware, and the corresponding program may be stored in a computer readable storage medium, where the program when executed includes one or a combination of the steps of the method embodiments.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing module, or each unit may exist alone physically, or two or more units may be integrated in one module. The integrated modules may be implemented in hardware or in software functional modules. The integrated modules may also be stored in a computer readable storage medium if implemented as software functional modules and sold or used as a stand-alone product.
The above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, or the like.
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
The method, the device, the processor and the computer readable storage medium for realizing the intent alignment processing of the large language model and the knowledge graph based on the trigger word position coding can supplement LLM according to local knowledge to realize the accurate question-answering of domain knowledge. The problems of deviation, long training time, slow knowledge updating, strong hardware dependence and the like of a model result are solved. On the one hand, from the user intention, the intention belongs to the field, the field range is defined, the intention task is divided in finer granularity, the LLM instruction is called better, and accurate answers are generated in a converging mode. On the other hand, the knowledge base needs accurate query sentences for searching, and the generation of KG instructions is particularly important. And starting from the instruction semantics and the position coding of the query trigger words, generating the KG instruction by adopting a guide scoring strategy tuning model. The invention uses the deep neural network model to align the user intention and the instruction so as to better retrieve the answer from the knowledge base. And the text semantic coding is performed, and meanwhile, the position information of the trigger words is fused, so that the accuracy of the query statement is ensured. The method utilizes the deep learning model to complete interaction between LLM and local KG in a time-saving and labor-saving manner, effectively generates information required by user intention, and has great innovation.
In this specification, the invention has been described with reference to specific embodiments thereof. It will be apparent, however, that various modifications and changes may be made without departing from the spirit and scope of the invention. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.

Claims (8)

1. A method for realizing the intent alignment processing of a large language model and a knowledge graph based on trigger word position coding is characterized by comprising the following steps:
(1) Acquiring user intention, arranging user questions into a serialized text embedded model, and judging the intention of the user questions through semantic coding analysis;
(2) Decomposing the intention, and selecting different processing modes according to the intention type;
(3) Performing instruction mapping, performing guide scoring strategy on the combination of trigger words during instruction coding, and decoding to generate KG instructions;
(4) Retrieving knowledge from a local knowledge graph base through the generated KG instruction;
(5) Carrying out knowledge aggregation;
(6) The final answer is generated using LLM.
2. The method for implementing the alignment processing of the large language model and the knowledge graph intent based on the trigger word position coding according to claim 1, wherein the step (1) specifically comprises the following steps:
(1.1) word segmentation of text using an external tool;
(1.2) invoking a pre-trained word vector library to perform sequence mapping;
(1.3) encoding the sequence through a multi-layer graph neural network to refine the intent information;
and (1.4) carrying out softmax classification through a layer of feedforward neural network to obtain the intended category information.
3. The method for implementing the alignment processing of the large language model and the knowledge graph intent based on the trigger word position coding according to claim 1, wherein the step (2) specifically comprises the following steps:
and adopting a general LLM answer to the intention questions not belonging to the field, further decomposing the intention questions belonging to the field, and adopting field trigger words to decompose the complex questions into LLM instructions with finer granularity.
4. The method for implementing the alignment processing of the large language model and the knowledge graph intent based on the trigger word position coding according to claim 1, wherein the step (3) specifically comprises the following steps:
(2.1) cutting the instruction into words, performing sequence mapping through a pre-trained word vector library, and recording trigger words;
(2.2) coding calculation is carried out on the sequence data by adopting a multi-layer graph neural network, trigger word position coding information is added during calculation, and first-layer coding information is added;
and (2.3) constructing a trigger word attention matrix, and adjusting the performance of the model by adopting a guide scoring strategy.
5. The method for implementing the alignment processing of the large language model and the knowledge graph intent based on the trigger word position coding according to claim 1, wherein the step (5) is specifically:
and inputting the retrieved knowledge and the user questions to the LLM, and calling related tool convergence information by the LLM triggering instruction.
6. An apparatus for implementing trigger word position coding based large language model and knowledge graph intent alignment processing, said apparatus comprising:
a processor configured to execute computer-executable instructions;
a memory storing one or more computer-executable instructions which, when executed by the processor, perform the steps of the method for performing large language model and knowledge-graph intent alignment processing based on trigger word position encoding of any one of claims 1 to 5.
7. A processor for implementing trigger word position encoding based large language model and knowledge graph intent alignment processing, wherein the processor is configured to execute computer executable instructions that, when executed by the processor, implement the steps of the trigger word position encoding based large language model and knowledge graph intent alignment processing method of any one of claims 1 to 5.
8. A computer readable storage medium having stored thereon a computer program executable by a processor to perform the steps of the method of any one of claims 1 to 5 for achieving large language model and knowledge-graph intent alignment based on trigger word position encoding.
CN202311576591.2A 2023-11-23 2023-11-23 Method, device, processor and storage medium for aligning large language model with knowledge graph intent based on trigger word position coding Pending CN117688150A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311576591.2A CN117688150A (en) 2023-11-23 2023-11-23 Method, device, processor and storage medium for aligning large language model with knowledge graph intent based on trigger word position coding

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311576591.2A CN117688150A (en) 2023-11-23 2023-11-23 Method, device, processor and storage medium for aligning large language model with knowledge graph intent based on trigger word position coding

Publications (1)

Publication Number Publication Date
CN117688150A true CN117688150A (en) 2024-03-12

Family

ID=90136243

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311576591.2A Pending CN117688150A (en) 2023-11-23 2023-11-23 Method, device, processor and storage medium for aligning large language model with knowledge graph intent based on trigger word position coding

Country Status (1)

Country Link
CN (1) CN117688150A (en)

Similar Documents

Publication Publication Date Title
Yu et al. FlowSense: A natural language interface for visual data exploration within a dataflow system
CN110364234B (en) Intelligent storage, analysis and retrieval system and method for electronic medical records
US8818795B1 (en) Method and system for using natural language techniques to process inputs
CN111143576A (en) Event-oriented dynamic knowledge graph construction method and device
CN109885698A (en) A kind of knowledge mapping construction method and device, electronic equipment
CN109829052A (en) A kind of open dialogue method and system based on human-computer interaction
CN112800170A (en) Question matching method and device and question reply method and device
Tyagi et al. Demystifying the role of natural language processing (NLP) in smart city applications: background, motivation, recent advances, and future research directions
EP3598436A1 (en) Structuring and grouping of voice queries
US20230153522A1 (en) Image captioning
CN114341865A (en) Progressive concatenation for real-time conversations
CN116077942B (en) Method for realizing interactive content recommendation
CN112528654A (en) Natural language processing method and device and electronic equipment
CN112579733A (en) Rule matching method, rule matching device, storage medium and electronic equipment
CN113988071A (en) Intelligent dialogue method and device based on financial knowledge graph and electronic equipment
Bai et al. Sentiment extraction from unstructured text using tabu search-enhanced markov blanket
CN113282729A (en) Question-answering method and device based on knowledge graph
CN118132719A (en) Intelligent dialogue method and system based on natural language processing
Paria et al. A neural architecture mimicking humans end-to-end for natural language inference
CN114896387A (en) Military intelligence analysis visualization method and device and computer readable storage medium
CN111738008B (en) Entity identification method, device and equipment based on multilayer model and storage medium
CN114842982B (en) Knowledge expression method, device and system for medical information system
Anisha et al. Text to sql query conversion using deep learning: A comparative analysis
CN117688150A (en) Method, device, processor and storage medium for aligning large language model with knowledge graph intent based on trigger word position coding
CN113934450B (en) Method, apparatus, computer device and medium for generating annotation information

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination