CN115270805A - Semantic information extraction method of service resources - Google Patents
Semantic information extraction method of service resources Download PDFInfo
- Publication number
- CN115270805A CN115270805A CN202110483937.9A CN202110483937A CN115270805A CN 115270805 A CN115270805 A CN 115270805A CN 202110483937 A CN202110483937 A CN 202110483937A CN 115270805 A CN115270805 A CN 115270805A
- Authority
- CN
- China
- Prior art keywords
- model
- service resources
- extracting
- neural network
- deep neural
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000000605 extraction Methods 0.000 title claims abstract description 19
- 238000003062 neural network model Methods 0.000 claims abstract description 19
- 238000012549 training Methods 0.000 claims abstract description 14
- 230000007246 mechanism Effects 0.000 claims abstract description 13
- 238000000034 method Methods 0.000 claims description 16
- 230000008859 change Effects 0.000 claims description 7
- 230000004913 activation Effects 0.000 claims description 6
- 230000005012 migration Effects 0.000 claims description 6
- 238000013508 migration Methods 0.000 claims description 6
- 230000008569 process Effects 0.000 claims description 5
- 230000003068 static effect Effects 0.000 claims description 4
- 210000002569 neuron Anatomy 0.000 claims description 3
- 238000011176 pooling Methods 0.000 claims description 3
- 238000004088 simulation Methods 0.000 claims description 3
- 239000000284 extract Substances 0.000 abstract description 2
- 238000004519 manufacturing process Methods 0.000 abstract description 2
- 238000004458 analytical method Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000006399 behavior Effects 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 230000008092 positive effect Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
- G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
- G06F40/295—Named entity recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Machine Translation (AREA)
Abstract
The invention discloses a semantic information extraction method of service resources, which extracts named entity information based on a BERT model; constructing a deep neural network model by combining an attention mechanism, taking an embedded vector output by a BERT model as input, and extracting a semantic relation by adopting the deep neural network model; when a deep neural network model is constructed, different attentions are generated for different characteristics by combining an attentive force mechanism; the invention solves the problem of semantic information extraction by using an end-to-end model, does not depend on the characteristics of manual manufacture or an external NLP tool, has high model training speed, and avoids repeated learning because most parameters are trained in advance.
Description
Technical Field
The invention belongs to the technical field of natural language understanding, and particularly relates to a semantic information extraction method for service resources.
Background
With the continuous advance of office informatization and living networking, typical online service resources such as patent services, legal services, policy services, scientific services, research and consultation are rapidly increasing and continuously accumulating.
The method provides new requirements and challenges for accurately and efficiently retrieving the service resources which meet the real intention of the user; meanwhile, the classification management of service resources is too simple at present, the subtle semantic difference of services cannot be reflected, and a set of effective semantic analysis description system is lacked.
At present, a large number of emerging methods are integrated into semantic representation models of texts, for example, various pre-training models such as BERT and GPT, and feature extraction methods such as RNN, LSTM/CNN and transformer are used to improve the semantic representation models, so that the models themselves become complicated. However, for the service resource function description of such special natural languages, the above models are directly used, and for fine-grained and refined semantic analysis and poor and strong matching humanity, the problems that the computation precision of the semantic similarity of Chinese vocabularies is not high, the expression information of the sentence representation method is incomplete and incomplete, the real functional semantics of resources cannot be matched, and the like exist.
Disclosure of Invention
The invention aims to provide a semantic information extraction method of service resources, which can be used for simultaneously extracting named entity information and semantic relations.
The invention is realized by adopting the following technical scheme:
a semantic information extraction method of service resources is provided, which comprises the following steps: named entity information is extracted based on a BERT model; constructing a deep neural network model by combining an attention mechanism, taking an embedded vector output by a BERT model as input, and extracting a semantic relation by adopting the deep neural network model; when the deep neural network model is constructed, different attentions are generated for different features by combining an attentiveness mechanism.
Further, for static service resources, attention weight is obtained through an activation unit; the activation unit carries out element subtraction operation on the input embedded vector, then the embedded vector is connected with the original embedded vector and then sent to the full connection layer, and attention weight is obtained through the output layer of the single neuron; aiming at the dynamic service resources, extracting a semantic relation by combining a service relation migration network; the service relationship migration network negatively samples the user behavior of the embedded vector, the change rule is mined through the change process of the GRU network simulation relationship, and the attention score is added on the basis of the update gate of the GRU network to obtain the changed embedded representation.
Further, the method further comprises: outputting the embedded representation of the extracted semantic relation in a form of a triple; a triple includes concepts, parent concepts and child concepts.
Further, the triplets are converted into three line paragraphs, one concept name per line.
Further, in the training step of extracting the named entity information, in the first stage of training, the contribution of the NER module to the total loss is weighted.
Further, the embedded vector output by the BERT model is sent into the deep neural network model after the average pooling operation.
Compared with the prior art, the invention has the advantages and positive effects that: the semantic information extraction method of the service resource provided by the invention extracts named entity information based on a BERT model; constructing a deep neural network model by combining an attention mechanism, taking an embedded vector output by a BERT model as input, and extracting a semantic relation by adopting the deep neural network model; when a deep neural network model is constructed, different attentions are generated for different characteristics by combining an attentive force mechanism; the invention solves the problem of semantic information extraction by using an end-to-end model, does not depend on the characteristics of manual manufacture or an external NLP tool, has high model training speed, and avoids repeated learning because most parameters are trained in advance.
Other features and advantages of the present invention will become more apparent from the detailed description of the embodiments of the present invention when taken in conjunction with the accompanying drawings.
Drawings
Fig. 1 is a flowchart of a semantic information extraction method for service resources according to the present invention.
Detailed Description
The following describes embodiments of the present invention in further detail with reference to the accompanying drawings.
The semantic information extraction method of the service resource, as shown in fig. 1, includes the following steps:
step S11: named entity information is extracted based on the BERT model.
For a given input sequence S and tok tokens of N Chinese characters, a vector sequence is generated by a pre-trained BERT-based model and then is sent to a feed-forward neural network for classification. The output size of the layer is the task of NLP (non-line segment) of named entity extraction, relation extraction and the like when the training number is output.
Fine-tuning the weight of the BERT model in the training process, and training the whole model in an end-to-end mode; specifically, in the first stage of training, the contribution of the NER module to the total loss is weighted.
Step S12: and constructing a deep neural network model by combining an attention mechanism.
Constructing a deep neural network model by combining an attention mechanism, adopting different models and methods aiming at different application scenes, and learning the embedded representation of the context relationship by using the deep neural network combined with the attention mechanism when the relationship among static service resources is stable; when the service resource relation is dynamically changed, a time sequence sub-network based on GRU units is added to capture the relation change rule.
Step S13: and taking the embedded vector output by the BERT model as input, and extracting the semantic relation by adopting a deep neural network model.
In constructing the deep neural network model, different attentions are generated for different features in conjunction with an attentiveness mechanism.
Wherein, aiming at the static service resource, the attention weight is obtained by the activation unit; the activation unit carries out element subtraction operation on the input embedded vector, then the embedded vector is connected with the original embedded vector and then sent to the full connection layer, and attention weight is obtained through the output layer of the single neuron; and the embedded vector output by the BERT model is sent into the deep neural network model after average pooling operation.
Aiming at dynamic service resources, extracting a semantic relation by combining a service relation migration network; the service relationship migration network negatively samples the user behavior of the embedded vector, the change rule is mined through the change process of the GRU network simulation relationship, and the attention score is added on the basis of the update gate of the GRU network to obtain the changed embedded representation.
And S14, outputting the extracted embedded representation of the semantic relation in a form of a triple.
A triple includes a concept, a parent concept, and a child concept.
For a focus concept, assuming it has parent concepts and child concepts, a triple may be converted into three lines of paragraphs, one concept name per line, each paragraph separated from another by an empty line.
The overall training process for the relationship extraction model of the invention is as follows:
the method comprises the steps of firstly dividing a service resource data set, preprocessing a training set, then passing through a BERT network to obtain vector representation of resources, then training a relationship extraction and classification network, giving weights of different characteristics by using an attention mechanism as a classification basis, storing a model into a model base after verification is carried out by using a test set, and issuing extraction for service resource relationship.
It should be noted that the above description is not intended to limit the present invention, and the present invention is not limited to the above examples, and those skilled in the art can easily understand the present invention.
Claims (6)
1. A semantic information extraction method for service resources is characterized by comprising the following steps:
named entity information is extracted based on a BERT model;
constructing a deep neural network model by combining an attention mechanism, taking an embedded vector output by a BERT model as input, and extracting a semantic relation by adopting the deep neural network model; when the deep neural network model is constructed, different attentions are generated for different features by combining an attentiveness mechanism.
2. The method for extracting semantic information of service resources according to claim 1,
aiming at the static service resources, the attention weight is obtained through an activation unit; the activation unit performs element subtraction operation on the input embedded vector, connects the embedded vector with the original embedded vector, sends the embedded vector to the full-connection layer, and obtains attention weight through the output layer of the single neuron;
aiming at the dynamic service resources, extracting a semantic relation by combining a service relation migration network; the service relationship migration network negatively samples the user behavior of the embedded vector, the change rule is mined through the change process of the GRU network simulation relationship, and the attention score is added on the basis of the update gate of the GRU network to obtain the changed embedded representation.
3. The method for extracting semantic information of service resources according to claim 1, further comprising:
outputting the embedded representation of the extracted semantic relation in a form of a triple; a triple includes a concept, a parent concept, and a child concept.
4. The method of extracting semantic information of a service resource of claim 3, wherein the triplets are converted into three rows of paragraphs, one concept name for each row.
5. The method of semantic information extraction of service resources of claim 1, characterized in that in the training step of extracting named entity information, in the first stage of training, the contribution of the NER module to the total loss is weighted.
6. The method for extracting semantic information of service resources according to claim 1, wherein the embedded vector output by the BERT model is fed into the deep neural network model after the average pooling operation.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110483937.9A CN115270805A (en) | 2021-04-30 | 2021-04-30 | Semantic information extraction method of service resources |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110483937.9A CN115270805A (en) | 2021-04-30 | 2021-04-30 | Semantic information extraction method of service resources |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115270805A true CN115270805A (en) | 2022-11-01 |
Family
ID=83744740
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110483937.9A Withdrawn CN115270805A (en) | 2021-04-30 | 2021-04-30 | Semantic information extraction method of service resources |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115270805A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115994668A (en) * | 2023-02-16 | 2023-04-21 | 浙江非线数联科技股份有限公司 | Intelligent community resource management system |
-
2021
- 2021-04-30 CN CN202110483937.9A patent/CN115270805A/en not_active Withdrawn
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115994668A (en) * | 2023-02-16 | 2023-04-21 | 浙江非线数联科技股份有限公司 | Intelligent community resource management system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109241255B (en) | Intention identification method based on deep learning | |
| CN111143576A (en) | Event-oriented dynamic knowledge graph construction method and device | |
| CN111368996A (en) | Retraining projection network capable of delivering natural language representation | |
| CN113239700A (en) | Text semantic matching device, system, method and storage medium for improving BERT | |
| CN109885670A (en) | A kind of interaction attention coding sentiment analysis method towards topic text | |
| CN109472024A (en) | A kind of file classification method based on bidirectional circulating attention neural network | |
| WO2019046463A1 (en) | System and method for defining dialog intents and building zero-shot intent recognition models | |
| CN113127624B (en) | Question-answer model training method and device | |
| CN109271493A (en) | A kind of language text processing method, device and storage medium | |
| WO2023284716A1 (en) | Neural network searching method and related device | |
| CN112749556B (en) | Multi-language model training method and device, storage medium and electronic equipment | |
| Shakhovska et al. | An Approach for a Next‐Word Prediction for Ukrainian Language | |
| Fung et al. | Empathetic dialog systems | |
| CN111985243A (en) | Emotion model training method, emotion analysis device and storage medium | |
| CN110472245A (en) | A kind of multiple labeling emotional intensity prediction technique based on stratification convolutional neural networks | |
| CN117251057A (en) | AIGC-based method and system for constructing AI number wisdom | |
| CN115062123A (en) | Knowledge base question-answer pair generation method of conversation generation system | |
| Bharathi et al. | Machine Learning Based Approach for Sentiment Analysis on Multilingual Code Mixing Text. | |
| US20220414344A1 (en) | Method and system for generating an intent classifier | |
| CN113486143A (en) | User portrait generation method based on multi-level text representation and model fusion | |
| CN115270805A (en) | Semantic information extraction method of service resources | |
| KR102297480B1 (en) | System and method for structured-paraphrasing the unstructured query or request sentence | |
| CN112347786A (en) | Artificial intelligence scoring training method and device | |
| CN111368524A (en) | Microblog viewpoint sentence recognition method based on self-attention bidirectional GRU and SVM | |
| US10706086B1 (en) | Collaborative-filtering based user simulation for dialog systems |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20221101 |
|
| WW01 | Invention patent application withdrawn after publication |