CN115146080A - Method and device for constructing knowledge graph - Google Patents
Method and device for constructing knowledge graph Download PDFInfo
- Publication number
- CN115146080A CN115146080A CN202211005380.9A CN202211005380A CN115146080A CN 115146080 A CN115146080 A CN 115146080A CN 202211005380 A CN202211005380 A CN 202211005380A CN 115146080 A CN115146080 A CN 115146080A
- Authority
- CN
- China
- Prior art keywords
- data set
- structured
- training
- recognition model
- named entity
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
- G06F16/367—Ontology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/31—Indexing; Data structures therefor; Storage structures
- G06F16/313—Selection or weighting of terms for indexing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
- G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
- G06F40/295—Named entity recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Artificial Intelligence (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Animal Behavior & Ethology (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a method and a device for constructing a knowledge graph, which comprise the following steps: acquiring a structured data set, a semi-structured data set and an unstructured data set of a target field; training a named entity recognition model by utilizing a structured data set, a remote supervision data set and a sample selector; converting the unstructured data set into a first converted structured data set by using the trained named entity recognition model; and constructing a knowledge graph of the target field according to the structured data set, the semi-structured data set and the first conversion structured data set. Therefore, the method and the device introduce the remote supervision data set, comprehensively utilize the sample selector and the named entity recognition model, enable the named entity recognition model to learn a large amount of high-quality training data, convert the unstructured data set into the structured data set by utilizing the trained named entity recognition model, greatly enrich the data for constructing the knowledge graph, and further construct the knowledge graph containing rich information.
Description
Technical Field
The present application relates to the field of knowledge graph technology, and in particular, to a method and an apparatus for constructing a knowledge graph.
Background
With the development of science and technology, the knowledge graph is playing an increasingly important role in the aspects of analyzing and processing large-scale data and information mining as a novel knowledge structural representation method. A knowledge graph is a structured semantic knowledge base that describes concepts and their interrelationships in the physical world in symbolic form. Meanwhile, the knowledge graph is also a basic technology of downstream services such as intelligent question answering, relation prediction, semantic search, intelligent recommendation and the like.
In some specific areas with strong verticality, the existing canonical structured data is few, so that only a general knowledge graph containing a small amount of information can be constructed. For example, in the field of the sea battlefield, various data and information related to weaponry, weather conditions, battlefield situation and the like are needed, and in the existing knowledge graph, because the structured data of the specification of the existing field of the sea battlefield is less, the existing knowledge graph has the problems of information loss, low precision and the like, for example, most of the existing knowledge graph only contains weapon names and brief descriptions, but the needed information is more than that, and the information such as production dates, manufacturers and the like does not exist in most of the existing knowledge graph. Therefore, how to construct a knowledge graph containing rich information becomes a problem to be urgently solved.
Disclosure of Invention
Based on the above problems, the application provides a method and a device for constructing a knowledge graph, which can be used for constructing a domain-specific knowledge graph containing rich information and having strong verticality.
The embodiment of the application discloses the following technical scheme:
in one aspect, an embodiment of the present application provides a method for constructing a knowledge graph, including:
acquiring a structured data set, a semi-structured data set and an unstructured data set of a target field;
training a named entity recognition model using the structured dataset, a remote supervisory dataset, and a sample selector;
converting the unstructured dataset into a first converted structured dataset by using the trained named entity recognition model;
constructing a knowledge graph of a target domain from the structured dataset, the semi-structured dataset, and the first transformed structured dataset.
Optionally, the training the named entity recognition model by using the structured data set, the remote supervision data set, and the sample selector includes:
carrying out preliminary training on a named entity recognition model by utilizing the structured data set;
mixing the structured data set and the remote monitoring data set to obtain a mixed data set;
and training the preliminarily trained named entity recognition model by utilizing the mixed data set and the sample selector.
Optionally, the training, by using the mixed data set and the sample selector, the named entity recognition model after the preliminary training includes:
randomly extracting a group of target data in the mixed data set, and judging whether the target data can be used as a training sample by using a sample selector;
and if the target data can be used as a training sample, training the named entity recognition model by using the target data.
Optionally, the method further includes:
feeding back model representations of the named entity recognition model to the sample selector;
the sample selector is updated based on the model representation.
Optionally, the constructing a knowledge graph of a target domain according to the structured data set, the semi-structured data set, and the first transformed structured data set includes:
processing the semi-structured data to obtain a second conversion structured data set;
constructing a knowledge graph from the structured data set, the first transformed structured data set, and the second transformed structured data set.
Optionally, the constructing a knowledge graph according to the structured data set, the first transformed structured data set, and the second transformed structured data set includes:
performing entity alignment and entity disambiguation on the structured data set, the first transformed structured data set, and the second transformed structured data set to obtain a total structured data set;
and constructing a knowledge graph of the target field according to the total structured data set.
Optionally, the named entity recognition model includes:
the Bi-LSTM network and the BERT model of the CRF layer are added.
On the other hand, the embodiment of the present application further provides an apparatus for constructing a knowledge graph, including:
the acquisition module is used for acquiring a structured data set, a semi-structured data set and an unstructured data set of a target field;
a training module for training a named entity recognition model using the structured dataset, a remote surveillance dataset, and a sample selector;
a conversion module for converting the unstructured dataset into a first converted structured dataset using the trained named entity recognition model;
a construction module to construct a knowledge-graph of a target domain from the structured dataset, the semi-structured dataset, and the first transformed structured dataset.
Optionally, the training module includes:
the preliminary training module is used for carrying out preliminary training on the named entity recognition model by utilizing the structured data set;
the mixing module is used for mixing the structured data set and the remote monitoring data set to obtain a mixed data set;
and the subsequent training module is used for training the primarily trained named entity recognition model by utilizing the mixed data set and the sample selector.
Optionally, the subsequent training module includes:
the judging module is used for randomly extracting a group of target data in the mixed data set and judging whether the target data can be used as a training sample or not by using the sample selector;
and the target training module is used for training the named entity recognition model by utilizing the target data.
Compared with the prior art, the method has the following beneficial effects:
the application provides a method for constructing a knowledge graph, which comprises the following steps: acquiring a structured data set, a semi-structured data set and an unstructured data set of a target field; training a named entity recognition model using the structured dataset, a remote surveillance dataset, and a sample selector; converting the unstructured dataset into a first converted structured dataset by using the trained named entity recognition model; constructing a knowledge graph of a target domain from the structured dataset, the semi-structured dataset, and the first transformed structured dataset. According to the method and the device, a large amount of data can be obtained to train the named entity recognition model by introducing the remote supervision data set, and the sample selector and the named entity recognition model are comprehensively utilized, so that the named entity recognition model can independently learn high-quality training data, an unstructured data set can be converted into a structured data set by using the trained named entity recognition model, and then data for constructing the knowledge graph is enriched, and a specific field knowledge graph containing rich information and high in verticality can be constructed.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive exercise.
FIG. 1 is a flow chart of a method for constructing a knowledge graph according to an embodiment of the present application;
FIG. 2 is a flowchart of a method for training a named entity recognition model according to an embodiment of the present disclosure;
fig. 3 is a schematic structural diagram of an apparatus for constructing a knowledge graph according to an embodiment of the present application.
Detailed Description
As described above, the currently constructed knowledge graph has problems of information loss and the like for the field with strong verticality.
Through research, the inventor invents a method and a device for constructing a knowledge graph, and can construct the knowledge graph which contains rich information and has strong verticality and a specific field.
In order to make the technical solutions of the present application better understood, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Method embodiment
Referring to fig. 1, the figure is a flowchart of a method for constructing a knowledge graph according to an embodiment of the present application, where the method for constructing a knowledge graph includes:
s101, acquiring a structured data set, a semi-structured data set and an unstructured data set of a target field.
It should be noted that the target field may be a specific field with strong verticality, such as a field of a sea battlefield. The normalized structured data set can be used for constructing a training set, a verification set and a test set required by model training, manual marking work can be added in the process of obtaining the structured data set so as to improve the quality of model training, and the structured data set can be obtained through databases of desensitization data with open sources released by the official. The semi-structured data set is a data set with a certain uniform structure, and can be converted into a structured data set through reasonable processing, such as regularized extraction and other operations, and the semi-structured data set can be obtained through webpage crawling of encyclopedia and the like. The unstructured data set is composed of data which are not provided with or provided with a small amount of unified structures, contains massive text data and can be obtained through websites and books in a target field.
And S102, training a named entity recognition model by utilizing the structured data set, the remote supervision data set and the sample selector.
In the embodiments provided in the present application, the named entity recognition model may be a BERT (Bidirectional Encoder retrieval from transformations) model added with a Bi-directional Long Short-Term Memory (Bi-LSTM) network and a Conditional Random Field (CRF) layer, and the remote supervision data set may include data labeled in a large batch using heuristic rules. Since a large amount of noise may exist in data labeled in a large batch by using a heuristic rule, a constructed sample selector is adopted in the embodiment of the application, and the sample selector can autonomously learn and select high-quality training data.
S103, converting the unstructured data set into a first conversion structured data set by using the trained named entity recognition model.
In the embodiment of the present application, the named entity recognition model can be enabled to convert the unstructured data into the structured data through the training in step S102, so that the trained named entity recognition model can be used to convert the unstructured data set into the structured data set, and the structured data set is named as a first converted structured data set, and the conversion method may be to extract the unstructured data set as input data of the named entity recognition model.
And S104, constructing a knowledge graph of the target field according to the structured data set, the semi-structured data set and the first conversion structured data set.
Specifically, the semi-structured data set may be processed to be converted into a structured data set, and the processing manner may be, for example, regularization extraction, and entity alignment and entity disambiguation are performed on the original structured data set, the structured data set converted from the semi-structured data set, and the structured data set converted from the unstructured data set, so as to obtain a total structured data set, and a knowledge graph of the target field is constructed according to the total structured data set.
According to the method for constructing the knowledge graph, a structured data set, a semi-structured data set and an unstructured data set in a target field are obtained; training a named entity recognition model using the structured dataset, a remote surveillance dataset, and a sample selector; converting the unstructured dataset into a first converted structured dataset by using the trained named entity recognition model; constructing a knowledge graph of a target domain from the structured dataset, the semi-structured dataset, and the first transformed structured dataset. According to the method and the device, a large amount of data can be obtained to train the named entity recognition model by introducing the remote supervision data set, and the sample selector and the named entity recognition model are comprehensively utilized, so that the named entity recognition model can independently learn high-quality training data, an unstructured data set can be converted into a structured data set by using the trained named entity recognition model, data for constructing the knowledge graph are enriched, and the specific-field knowledge graph containing rich information and high in verticality can be constructed.
Referring to fig. 2, which is a flowchart of a method for training a named entity recognition model according to an embodiment of the present application, the method for training the named entity recognition model includes:
s201, utilizing the structured data set to carry out preliminary training on a named entity recognition model.
It should be noted that, because the data in the structured data set is normalized data, the preliminary training of the named entity recognition model by using the structured data set can make the named entity recognition model have a good foundation, and can effectively improve the training efficiency.
S202, mixing the structured data set and the remote monitoring data set to obtain a mixed data set.
S203, randomly extracting a group of target data in the mixed data set, and judging whether the target data can be used as a training sample by using a sample selector.
It should be noted that, in the embodiments provided in the present application, as an example, the sample selector may use a reinforcement learning method based on a policy. Firstly, a subset of a data set is selected as a packet, and a sample selector selects data with correct labels from the labeled data in the packet, so that the selected data with correct labels can be used as high-quality data to train a named entity recognition model.
And S204, if the target data can be used as a training sample, training the named entity recognition model by using the target data.
It should be noted that, the named entity recognition model may adopt a structure of a BERT plus CRF layer, and the training process is to train and encode the labeled data using a pre-trained BERT model to obtain an accurate semantic representation of the character, and to perform state transition constraint on the encoded structure by the CRF layer.
Specifically, in the embodiments provided in the present application, a BIO labeling manner may be adopted for labeling. The method comprises the steps that codes are obtained through a BERT layer, then coding information is input into a Bi-directional long-short time memory network layer Bi-LSTM, the Bi-directional long-short time memory network layer Bi-LSTM comprises a forward long-short time memory network LSTM and a reverse long-short time memory network LSTM, context information of sentences is mainly learned, and the output of the Bi-directional long-short time memory network layer is the probability that each word in data is different named entities; introducing a conditional random field model (CRF) layer for learning a transfer rule before adjacent entity labels of a data center, wherein the CRF layer receives an input sequence X and outputs a target sequence Y, and two adjacent nodes meet the following formula:
P(Y i |X，Y1，Y2，…Yn)＝p(Y i |X，Y i-1 ,Y i+1 )
wherein, the lower corner of Y is marked as the serial number of Y, and the given input sequence X is solved by using CRF, so that the probability formula of Y can be obtained as follows:
because adjacent state sequences have a defined relationship and depend on observation sequence data, the state feature s and the transition feature t are substituted into the following formula:
wherein Z (X) is a normalization function, t k For transfer of state functions, λ k Is t k Weight of (1), s l Is a characteristic function of the state, u l Is as s l K is the number of transition state functions, l is the number of state feature functions, t k And s l Is 0 or 1, in t k For example, the formula is as follows:
s205, feeding back the model performance of the named entity recognition model to the sample selector.
Specifically, in the embodiments provided in the present application, as an example, there is a standard structured data set and a remote supervision data set, the standard structured data set and the remote supervision data set are combined into a mixed data set, a subset is randomly selected from the mixed data set, for each data in the subset, the sample selector needs to perform a decision to decide whether to select the data, and after each data in the subset is determined to be completed, a return of the current policy of the sample selector on the subset is returned, and the return is used to update the sample selector.
Use state S t Representing the current data, and its tag sequence, representing the state as a vector S t Vector S t May consist of a policy network and rewards.
A policy network: the agent obtains a decision value a according to the current strategy t Wherein a is t E {0,1},0 indicates that the data will not be selected, and 1 indicates that the data will be selected. The policy network can be represented by the following formula.
A Θ (s t ,a t )＝a t σ(W*S t +b)+(1-a t )(1-σ(W*S t +b))
Wherein A is Θ (s t ,a t ) Representing a policy network, a t Indicates whether the selector selects the t-th sample, S t Is the state vector of t, and W and b are parameters of the multilayer perceptron MLB for processing the state vector, and then using the sigmoid σ function.
The reward may be used to evaluate the ability of the current sample selector to predict each data tag, and when the model completes all selections in the current subset, an average reward for that subset will be obtained. The reward can be calculated by the following formula.
Wherein, r is the return,
the structured data set is then used to construct,
for remote supervision of data sets, p (z | x) j ) And p (y | x) k ) Conditional probability, x, of tag sequence calculated for CRF layer j For data candidates in the remote supervision data set, x k For candidate data in a structured dataset, z for a remote supervised datasetSpecific data, y is specific data in the structured dataset.
In the embodiment provided by the present application, the policy parameter of the sample selector may be updated by reporting a value and solving a gradient, for example, the policy parameter may be updated by the following formula:
wherein, theta 1 Is the set of all parameters after the sample selector is updated, theta is the set of all parameters before the sample selector is updated, alpha is the learning rate, r (a) t ) Is the average return value of the return signals,
for remote supervision of data set A Θ (s t ,a t ) A network of policies is represented that is,
the gradient operator of the policy network for the parameter Θ is represented.
S206, updating the sample selector based on the model representation.
In the embodiment provided by the present application, as an example, a joint training mode may be adopted to iteratively update parameters of the sample selector and the named entity recognition model, specifically, select a part of data from the remote supervision data set, mix the part of data with the structured data to obtain a mixed data set, learn parameters of the named entity recognition model from the mixed data set, and provide a feedback to the sample selector to optimize the effect of the sample selector.
According to the embodiment provided by the application, the named entity recognition model is preliminarily trained by utilizing the structured data set, the sample selector is utilized to judge and select the mixed data set obtained after the structured data set and the remote supervision data set are mixed, so that the named entity recognition model can be trained according to a large amount of high-quality data, the sample selector is updated by utilizing the model expression of the trained named entity recognition model, so that the sample selector can adjust strategies according to the model expression of the named entity recognition model, and the tag sequence of the named entity recognition depends on distribution, so that the probability of occurrence of an illegal sequence in tag sequence prediction can be effectively reduced by adopting the CRF, and the performance of the named entity recognition model is higher.
Device embodiment
Referring to fig. 3, the diagram is a schematic structural diagram of an apparatus for constructing a knowledge graph according to an embodiment of the present application, where the apparatus for constructing a knowledge graph includes: the device comprises an acquisition module 301, a training module 302, a conversion module 303 and a construction module 304.
The obtaining module 301 is configured to obtain a structured data set, a semi-structured data set, and an unstructured data set in a target field.
A training module 302 for training a named entity recognition model using the structured dataset, the remote supervisory dataset, and the sample selector.
A converting module 303, configured to convert the unstructured dataset into a first converted structured dataset by using the trained named entity recognition model.
A construction module 304 for constructing a knowledge graph of the target domain from the structured data set, the semi-structured data set, and the first transformed structured data set.
Optionally, the training module 302 includes:
the preliminary training module is used for processing and training the named entity recognition model by utilizing the structured data set;
the mixing module is used for mixing the structured data set and the remote supervision data set to obtain a mixed data set;
and the subsequent training module is used for training the primarily trained named entity recognition model by utilizing the mixed data set and the sample selector.
Optionally, the subsequent training module includes:
the judging module is used for randomly extracting a group of target data in the mixed data set and judging whether the target data can be used as a training sample by using a sample selector;
and the target training module is used for training the named entity recognition model by utilizing the target data.
Optionally, the apparatus further comprises:
and a feedback module. For feeding back a model representation of the named entity recognition model to the sample selector;
an update module to update the sample selector based on the model representation.
Optionally, the building block 304 includes:
the processing module is used for processing the semi-structured data to obtain a second conversion structured data set;
a transformation building module to build a knowledge graph from the structured data set, the first transformed structured data set, and the second transformed structured data set.
Optionally, the conversion building block includes:
an alignment disambiguation module for performing entity alignment and entity disambiguation the structured data set, the first transformed structured data set, and the second transformed structured data set to obtain a total structured data set;
and the total construction module is used for constructing the knowledge graph of the target field according to the total structured data set.
The device for constructing the knowledge graph, provided by the embodiment of the application, utilizes an acquisition module to acquire a structured data set, a semi-structured data set and an unstructured data set of a target field, utilizes a training module to train a named entity recognition model by utilizing the structured data set, a remote supervision data set and a sample selector, utilizes a trained named entity recognition module to convert the unstructured data set into a first converted structured data set by utilizing a conversion module, and utilizes a construction module to construct the knowledge graph of the target field according to the structured data set, the semi-structured data set and the first converted structured data set. According to the method and the device, a large amount of data can be obtained to train the named entity recognition model by introducing the remote supervision data set, and the sample selector and the named entity recognition model are comprehensively utilized, so that the named entity recognition model can independently learn high-quality training data, an unstructured data set can be converted into a structured data set by using the trained named entity recognition model, data for constructing the knowledge graph are enriched, and the specific-field knowledge graph containing rich information and high in verticality can be constructed.
It should be noted that, in the present specification, all the embodiments are described in a progressive manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, the apparatus and system embodiments, because they are substantially similar to the method embodiments, are described in a relatively simple manner, and reference may be made to some of the descriptions of the method embodiments for related points. The above-described embodiments of the apparatus and system are merely illustrative, and the units described as separate parts may or may not be physically separate, and the parts suggested as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. One of ordinary skill in the art can understand and implement without inventive effort.
The above description is only one specific embodiment of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present application should be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (10)
1. A method of constructing a knowledge graph, comprising:
acquiring a structured data set, a semi-structured data set and an unstructured data set of a target field;
training a named entity recognition model using the structured dataset, a remote supervisory dataset, and a sample selector;
converting the unstructured dataset into a first converted structured dataset by using the trained named entity recognition model;
constructing a knowledge graph of a target domain from the structured dataset, the semi-structured dataset, and the first transformed structured dataset.
2. The method of claim 1, wherein training a named entity recognition model using the structured dataset, a remote supervised dataset, and a sample selector comprises:
carrying out preliminary training on a named entity recognition model by utilizing the structured data set;
mixing the structured data set and the remote supervision data set to obtain a mixed data set;
and training the preliminarily trained named entity recognition model by utilizing the mixed data set and the sample selector.
3. The method of claim 2, wherein training the preliminarily trained named entity recognition model using the mixed dataset and sample selector comprises:
randomly extracting a group of target data in the mixed data set, and judging whether the target data can be used as a training sample by using a sample selector;
and if the target data can be used as a training sample, training the named entity recognition model by using the target data.
4. The method of claim 3, further comprising:
feeding back a model representation of the named entity recognition model to the sample selector;
updating the sample selector based on the model representation.
5. The method of claim 1, wherein constructing a knowledge graph of a target domain from the structured dataset, the semi-structured dataset, and the first transformed structured dataset comprises:
processing the semi-structured data to obtain a second conversion structured data set;
constructing a knowledge graph from the structured data set, the first transformed structured data set, and the second transformed structured data set.
6. The method of claim 5, wherein constructing a knowledge graph from the structured data set, the first transformed structured data set, and the second transformed structured data set comprises:
performing entity alignment and entity disambiguation on the structured data set, the first transformed structured data set, and the second transformed structured data set to obtain a total structured data set;
and constructing a knowledge graph of the target field according to the total structured data set.
7. The method of claim 1, wherein the named entity recognition model comprises:
the BERT model of the Bi-LSTM network and the CRF layer is added.
8. An apparatus for constructing a knowledge graph, comprising:
the acquisition module is used for acquiring a structured data set, a semi-structured data set and an unstructured data set of a target field;
a training module for training a named entity recognition model using the structured dataset, a remote surveillance dataset, and a sample selector;
a conversion module for converting the unstructured dataset into a first converted structured dataset using the trained named entity recognition model;
a construction module to construct a knowledge graph of a target domain from the structured dataset, the semi-structured dataset, and the first transformed structured dataset.
9. The apparatus of claim 8, wherein the training module comprises:
the preliminary training module is used for carrying out preliminary training on the named entity recognition model by utilizing the structured data set;
the mixing module is used for mixing the structured data set and the remote monitoring data set to obtain a mixed data set;
and the subsequent training module is used for training the primarily trained named entity recognition model by utilizing the mixed data set and the sample selector.
10. The apparatus of claim 9, wherein the subsequent training module comprises:
the judging module is used for randomly extracting a group of target data in the mixed data set and judging whether the target data can be used as a training sample by using a sample selector;
and the target training module is used for training the named entity recognition model by utilizing the target data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211005380.9A CN115146080A (en) | 2022-08-22 | 2022-08-22 | Method and device for constructing knowledge graph |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211005380.9A CN115146080A (en) | 2022-08-22 | 2022-08-22 | Method and device for constructing knowledge graph |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115146080A true CN115146080A (en) | 2022-10-04 |
Family
ID=83415481
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211005380.9A Pending CN115146080A (en) | 2022-08-22 | 2022-08-22 | Method and device for constructing knowledge graph |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115146080A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115600601A (en) * | 2022-11-08 | 2023-01-13 | 税友软件集团股份有限公司（Cn） | Method, device, equipment and medium for constructing tax law knowledge base |
-
2022
- 2022-08-22 CN CN202211005380.9A patent/CN115146080A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115600601A (en) * | 2022-11-08 | 2023-01-13 | 税友软件集团股份有限公司（Cn） | Method, device, equipment and medium for constructing tax law knowledge base |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109992782B (en) | Legal document named entity identification method and device and computer equipment | |
| CN114020862B (en) | Search type intelligent question-answering system and method for coal mine safety regulations | |
| CN112818691A (en) | Named entity recognition model training method and device | |
| CN104598611B (en) | The method and system being ranked up to search entry | |
| CN111985239A (en) | Entity identification method and device, electronic equipment and storage medium | |
| CN113780002A (en) | Knowledge reasoning method and device based on graph representation learning and deep reinforcement learning | |
| CN111709244A (en) | Deep learning method for identifying causal relationship of contradictory dispute events | |
| CN115688879A (en) | Intelligent customer service voice processing system and method based on knowledge graph | |
| CN109918477A (en) | A kind of distributed search resources bank selection method based on variation self-encoding encoder | |
| CN115146080A (en) | Method and device for constructing knowledge graph | |
| CN115905487A (en) | Document question and answer method, system, electronic equipment and storage medium | |
| CN114356990A (en) | Base named entity recognition system and method based on transfer learning | |
| CN117094325B (en) | Named entity identification method in rice pest field | |
| CN112035629B (en) | Method for implementing question-answer model based on symbolized knowledge and neural network | |
| CN116244484B (en) | Federal cross-modal retrieval method and system for unbalanced data | |
| CN110390050B (en) | Software development question-answer information automatic acquisition method based on deep semantic understanding | |
| Li et al. | Fitcnn: A cloud-assisted lightweight convolutional neural network framework for mobile devices | |
| CN116401353A (en) | Safe multi-hop question-answering method and system combining internal knowledge patterns and external knowledge patterns | |
| CN116992880A (en) | Building name identification method, device, electronic equipment and storage medium | |
| CN115906846A (en) | Document-level named entity identification method based on double-graph hierarchical feature fusion | |
| CN116522165A (en) | Public opinion text matching system and method based on twin structure | |
| US20230168989A1 (en) | BUSINESS LANGUAGE PROCESSING USING LoQoS AND rb-LSTM | |
| CN115934905A (en) | Document question and answer method, device and system, electronic equipment and storage medium | |
| CN115905458A (en) | Event extraction method based on machine reading understanding model | |
| CN113051353A (en) | Attention mechanism-based knowledge graph path reachability prediction method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |