WO2022174496A1

WO2022174496A1 - Data annotation method and apparatus based on generative model, and device and storage medium

Info

Publication number: WO2022174496A1
Application number: PCT/CN2021/083758
Authority: WO
Inventors: 李薿; 陈曦; 崔艳; 庄伯金; 王少军
Original assignee: 平安科技（深圳）有限公司
Priority date: 2021-02-20
Filing date: 2021-03-30
Publication date: 2022-08-25
Also published as: CN112860919A

Abstract

A data annotation method and apparatus based on a generative model, and a device and a storage medium, which relate to the technical field of artificial intelligence and can be applied to the field of natural language processing. The method comprises: acquiring text to be annotated, and performing splitting, word segmentation and merging processing on said text, in order to obtain a target phrase; annotating the target phrase on the basis of multiple preset annotation rules to obtain a label sample; then acquiring a sample annotation probability, for the target phrase, of the label sample, iteratively updating, on the basis of the sample annotation probability, initial parameters generated by a generative model, in order to obtain a trained generative model, and outputting annotation accuracy by means of the trained generative model; and then determining a target label sample according to the annotation accuracy. The present invention also relates to blockchain technology, and text to be annotated is stored in a blockchain. Data is annotated on the basis of multiple preset rules, and a label sample with the highest data annotation accuracy is selected according to a generative model, thereby facilitating the improvement of data annotation accuracy.

Description

Data annotation method, device, device and storage medium based on generative model

This application claims the priority of the Chinese patent application filed on February 20, 2021 with the application number 202110193454.5 and the title of the invention is "Generation Model-Based Data Labeling Method, Device, Equipment and Storage Medium", the entire contents of which are Incorporated herein by reference.

technical field

The present application relates to the technical field of artificial intelligence, and in particular, to a data labeling method, apparatus, device and storage medium based on a generative model.

Background technique

With the increasingly prominent role of knowledge graphs in various vertical fields, how to label large-scale unlabeled data is the focus of the current knowledge graph field.

Although the accuracy rate of named entity recognition for labeled data has reached more than 99%, the time period for manual construction of text labeled data in different fields is extremely long. Moreover, labeled data in different fields are not completely universal. Differences in business scenarios, target users, and product definitions directly make it difficult for the text field to have large-scale labeled data that can be applied to various fields. Therefore, how to improve the efficiency of labeling large-scale data has become a difficult problem.

In view of the above problems, the existing solution is to obtain the word sequence corresponding to the original text, transform and map the word sequence to obtain the entity labeling vector, and count the number of preset entity information in the entity labeling vector, so as to realize the data analysis. However, the inventor found that this kind of labeling method is obtained by transforming and mapping word vectors, which is easy to cause errors in data labeling, resulting in low accuracy of data labeling for large-scale data. There is an urgent need for a method that can improve the accuracy of data labeling.

SUMMARY OF THE INVENTION

The purpose of the embodiments of the present application is to propose a data labeling method, apparatus, device and storage medium based on a generative model, so as to improve the accuracy of data labeling.

In order to solve the above technical problems, the embodiment of the present application provides a data labeling method based on a generative model, including:

Obtain the text to be marked, and split the text to be marked to obtain a split statement;

By performing word segmentation processing on the split sentence, a target word segmentation is obtained, and the target word segmentation is combined to obtain a target phrase;

Obtaining a plurality of preset labeling rules, and labeling the target phrase respectively through the plurality of the predefined labeling rules, to obtain a label sample corresponding to each of the predefined rules;

Obtain the sample labeling probability of the target phrase by the label sample corresponding to each of the preset labeling rules, and obtain the initial parameters of the generation model according to the sample labeling probability and the label sample;

Iteratively update the initial parameters of the generative model through the sample labeling probability to obtain a trained generative model, and output the labeling accuracy rate corresponding to the label sample through the trained generative model;

The label sample with the highest labeling accuracy is selected as the target label sample.

In order to solve the above technical problems, an embodiment of the present application provides a data labeling device based on a generative model, including:

A to-be-labeled text splitting module, configured to obtain the to-be-labeled text, and to split the to-be-labeled text to obtain a split statement;

a target phrase acquisition module, used to obtain a target word segmentation by performing word segmentation processing on the split statement, and merging the target word segmentation to obtain a target phrase;

The label sample generation module is configured to obtain a plurality of preset labeling rules, and respectively label the target phrase through the plurality of the predefined labeling rules, to obtain label samples corresponding to each of the predefined rules;

The initial parameter generation module is used to obtain the sample labeling probability of the target phrase by the label sample corresponding to each of the preset labeling rules, and obtain the initial parameters of the generation model according to the sample labeling probability and the label sample ;

The labeling accuracy output module is used to iteratively update the initial parameters of the generative model through the sample labeling probability to obtain a trained generative model, and output the labels corresponding to the label samples through the trained generative model Accuracy;

The label sample selection module is configured to select the label sample with the highest label accuracy rate as the target label sample.

In order to solve the above technical problems, a technical solution adopted in the present application is to provide a computer device, including a memory and a processor, wherein the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions Implement the following steps:

In order to solve the above technical problems, a technical solution adopted in this application is: a computer-readable storage medium, where the computer-readable instructions are executed by a processor to implement the following steps:

Embodiments of the present application provide a method, apparatus, device, and storage medium for data labeling based on a generative model. In the embodiment of the present application, the data is labeled by a variety of preset rules, and the label sample with the highest data labeling accuracy is selected according to the generation model, which is beneficial to improve the data labeling accuracy.

Description of drawings

In order to illustrate the solutions in the present application more clearly, the following will briefly introduce the accompanying drawings used in the description of the embodiments of the present application. For those of ordinary skill, other drawings can also be obtained from these drawings without any creative effort.

1 is a schematic diagram of an application environment of a method for labeling data based on a generative model provided by an embodiment of the present application;

FIG. 2 is an implementation flowchart of a method for data labeling based on a generative model provided according to an embodiment of the present application;

3 is a flow chart of an implementation of a sub-process in the method for labeling data based on a generative model provided by an embodiment of the present application;

Fig. 4 is another realization flow chart of the sub-process in the data labeling method based on the generative model provided by the embodiment of the present application;

Fig. 5 is another realization flow chart of the sub-process in the data labeling method based on the generative model provided by the embodiment of the present application;

6 is another implementation flowchart of the sub-process in the method for labeling data based on a generative model provided by an embodiment of the present application;

7 is another implementation flow chart of the sub-process in the data labeling method based on the generative model provided by the embodiment of the present application;

FIG. 8 is another implementation flowchart of the sub-process in the data labeling method based on the generative model provided by the embodiment of the present application;

9 is a schematic diagram of a data labeling device based on a generative model provided by an embodiment of the present application;

FIG. 10 is a schematic diagram of a computer device provided by an embodiment of the present application.

Detailed ways

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field of this application; the terms used herein in the specification of the application are for the purpose of describing specific embodiments only It is not intended to limit the application; the terms "comprising" and "having" and any variations thereof in the description and claims of this application and the above description of the drawings are intended to cover non-exclusive inclusion. The terms "first", "second" and the like in the description and claims of the present application or the above drawings are used to distinguish different objects, rather than to describe a specific order.

Reference herein to an "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor a separate or alternative embodiment that is mutually exclusive of other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

In order to make those skilled in the art better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings.

The present application will be described in detail below with reference to the accompanying drawings and embodiments.

Referring to FIG. 1 , the system architecture 100 may include

terminal devices

101 , 102 , and 103 , a network 104 and a server 105 . The network 104 is a medium used to provide a communication link between the

terminal devices

101 , 102 , 103 and the server 105 . The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.

The user can use the

terminal devices

101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages and the like. Various communication client applications may be installed on the

terminal devices

101 , 102 and 103 , such as web browser applications, search applications, instant communication tools, and the like.

The

terminal devices

101, 102, 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

The server 105 may be a server that provides various services, such as a background server that provides support for the pages displayed on the

terminal devices

101 , 102 , and 103 .

It should be noted that the data labeling method based on the generative model provided in the embodiment of the present application is generally executed by the server, and accordingly, the data labeling apparatus based on the generative model is generally configured in the server.

It should be understood that the numbers of terminal devices, networks and servers in FIG. 1 are merely illustrative. There can be any number of terminal devices, networks and servers according to implementation needs.

Referring to FIG. 2, FIG. 2 shows a specific implementation manner of a data labeling method based on a generative model.

It should be noted that, if there is substantially the same result, the method of the present application is not limited to the flow sequence shown in FIG. 2, and the method includes the following steps:

S1: Obtain the text to be marked, and split the text to be marked to obtain a split sentence.

Specifically, after the server obtains the text to be marked, it will preprocess it, such as data cleaning, and then split the text to be marked into paragraphs, sentences, etc. sub-sentence. Among them, the text to be labeled is the data that needs to be labeled, so as to generate text with labeled labels.

S2: By performing word segmentation on the split sentence, the target word segmentation is obtained, and the target word segmentation is combined to obtain the target phrase.

Specifically, in the above steps, the text to be marked has been split into split sentences, and the split sentences exist in the form of short sentences. The segmented sentences are processed by word segmentation to generate each target word segment, and then the part of speech is marked according to the part of speech of the target word segment, and the target word segment is merged according to the method of dependency syntax analysis to generate the target phrase.

It should be noted that the preset word segmentation tools include but are not limited to: stuttering word segmentation, NLPIR word segmentation system, SnowNLP, etc. Preferably, stuttering word segmentation is used to segment the split sentence to obtain the target word segmentation. The stuttering word segmentation has the ability to cut the sentence most accurately, which is suitable for text analysis, and it scans all the words that can be formed into words in the sentence, and the speed is relatively fast.

Among them, dependency syntax analysis was first proposed by the French linguist L.Tesniere. It analyzes the sentence into a dependency syntax tree, and describes the dependency relationship between each word, that is, it points out the syntactic collocation between words, which is related to semantics. In the embodiment of the present application, the target word segmentation is merged by means of dependency syntax analysis.

S3: Acquire a variety of preset labeling rules, and label the target phrase respectively through the multiple preset labeling rules, and obtain a label sample corresponding to each preset rule.

Specifically, in the embodiment of the present application, after the text to be marked is split, segmented and merged, the target phrase is marked with various marking rules, and then the accuracy rate of data marking with various rules is determined by the generation model, so as to select The label sample with the highest accuracy rate is used to complete the labeling of the data. Therefore, the server obtains multiple preset tagging rules, and then tags the target phrase with corresponding tags according to each preset tagging rule, so that the target phrase generates a tag sample corresponding to each preset rule.

It should be noted that a variety of preset labeling rules include, but are not limited to: regular recognition, remote matching knowledge base recognition, and external data matching methods. Among them, the regular recognition refers to matching the corresponding labeling rules by presetting different SQL query statements, so as to realize different rules to label the target phrase. The remote matching knowledge base means that the target phrase is annotated by matching it with the knowledge base of the peripheral device one by one. The method of matching external data refers to matching the target phrase with the external data provided by, for example, a crowdsourcing platform, so as to complete the labeling of the target phrase. Preferably, by using a variety of different labeling rules to label the target phrase, the accuracy of data labeling in various ways can be screened, thereby improving the accuracy of data labeling.

S4: Obtain the sample labeling probability of the target phrase by the label sample corresponding to each preset labeling rule, and obtain the initial parameters of the generation model according to the sample labeling probability and the label sample.

Specifically, the sample labeling probability refers to the coverage rate of the target phrase by the sample label obtained by using the preset labeling rule, and the parameters of the generation model can be iteratively updated subsequently. And since each preset tagging rule has different sample tagging probabilities for different target phrases, it is necessary to first obtain the sample tagging probability corresponding to each preset tagging rule. The server also obtains initial estimated parameters of the generative model after initializing the sample labeling probability and labeling samples, that is, obtains the initial parameters of the generative model.

Among them, a generative model refers to a model that can randomly generate observed data, especially given some implicit parameters. Generative models assign a joint probability distribution to observations and labeled data sequences. In the embodiment of the present application, the implicit parameter corresponds to the true label of the target phrase of the present application, the observed value corresponds to the sample labeling probability of the present application, and the labeled data sequence corresponds to the label sample of the present application; therefore, according to the implicit parameter, that is, the true The data label is randomly generated by the observation data model, which can determine the labeling probability of each preset labeling rule for the target phrase.

S5: Iteratively update the initial parameters of the generative model through the sample labeling probability to obtain a trained generative model, and output the labeling accuracy rate corresponding to the label sample through the trained generative model.

Specifically, the initial parameters of the generative model are fitted by the sample label probabilities, and the sample label probabilities are back-propagated back to iteratively update the initial parameters by means of stochastic gradient descent, so that the sample label probabilities are different and close to the parameters of the generative model. , so as to obtain a trained generative model. Using the parameters of the trained generative model to estimate the probability of label samples, and after weighted average processing, the labeling accuracy rate of label samples under each preset rule is obtained.

Among them, iterative update refers to fitting the initial parameters of the generative model through the sample label probability, and using stochastic gradient descent to backpropagate the sample label probability back to iteratively calculate the initial parameters, so that the sample label probabilities are different and close to the generation parameters of the model.

S6: Select the label sample with the highest labeling accuracy as the target label sample.

Specifically, through the above steps, the labeling accuracy rate of the label samples under each preset labeling rule has been obtained, so the label sample with the highest labeling accuracy rate is selected as the target label sample, so as to try multiple labeling rules for the target phrase. Labeling, selecting the label sample with the highest accuracy rate is conducive to improving the accuracy of data labeling.

In this embodiment, after splitting, segmenting and merging the acquired text to be labeled, a target phrase is obtained, which facilitates subsequent data labeling of the text to be labeled according to the target phrase; and then obtains a variety of preset labeling rules, and Label the target phrases through a variety of preset labeling rules, obtain the label samples corresponding to each preset labeling rule, and then obtain the sample labeling probability of the target phrase by the label samples corresponding to each predefined labeling rule, and then obtain the sample labeling probability of the target phrase according to the sample Label the probability and label samples to obtain the initial parameters of the generative model, and then iteratively update the initial parameters of the generative model through the sample labeling probability to obtain a trained generative model, and output the labeling accuracy rate corresponding to the label sample through the trained generative model , select the label sample with the highest labeling accuracy as the target label sample, realize the labeling of data through a variety of preset rules, and select the label sample with the highest data labeling accuracy according to the generation model, which is beneficial to improve the accuracy of data labeling.

Please refer to FIG. 3. FIG. 3 shows a specific implementation of step S4. In step S4, the sample labeling probability of the target phrase corresponding to the label sample corresponding to each preset labeling rule is obtained, and the sample labeling probability and label sample are obtained according to the sample labeling probability and label sample. , the specific implementation process of obtaining the initial parameters of the generated model is described in detail as follows:

S41: Calculate the coverage rate of the target phrase by the label samples corresponding to each preset labeling rule, and use the coverage rate as the sample labeling probability.

Specifically, in order to subsequently train the generative model so that the sample labeling probability is close to the parameters of the generative model, the sample labeling probability needs to be obtained first. Therefore, the coverage rate of the target phrase by the label samples corresponding to each preset labeling rule is calculated, and the coverage rate is used as the sample labeling probability. In the embodiment of the present application, the coverage rate is obtained by calculating the coverage degree of the target phrase by the tag sample.

In a specific embodiment, when the target phrase is marked by means of remote matching knowledge base recognition, because the knowledge base of the peripheral device may not match the target word segmentation in the target phrase one by one, these target word segmentation cannot be matched. Marking in this way causes the target phrase to fail to be marked; if the target word segmentation in the target phrase and the knowledge base of the peripheral can match one by one, the target phrase is marked successfully. Divide the successfully labeled samples of target phrases by the total number of target phrases, and the result is the coverage rate of the target phrase by the way of remote matching knowledge base recognition, and the coverage rate is taken as the sample labeling probability. For example, if the number of successfully tagged target phrases is 9,000, and the total number of target phrases is 10,000, the probability of sample tagging is 90%.

S42: Initialize the sample label probability and the label sample to obtain the initial parameters of the generative model.

Specifically, the initialization process refers to assigning estimated parameter values to the initial parameters of the generative model according to the sample label probability and label samples, so as to obtain the initial parameters of the generative model.

In this implementation, by calculating the coverage rate of the target phrase by the label samples corresponding to each preset labeling rule, and using the coverage rate as the sample labeling probability, and then initializing the sample label probability and label samples, the generated model is obtained. The initial parameters are used to obtain the sample labeling probability and the initial parameters of the generation model, which is convenient for subsequent training of the generation model, thereby improving the accuracy of data labeling.

Please refer to FIG. 4. FIG. 4 shows a specific implementation of step S5. In step S5, the initial parameters of the generative model are iteratively updated through the sample labeling probability to obtain a trained generative model. The specific implementation process of the labeling accuracy corresponding to the output label sample is described in detail as follows:

S51: Use the difference between the parameters of the generated model and the sample labeling probability as the optimization feature value.

Specifically, in the embodiment of the present application, the parameters of the generative model are iteratively updated, so that the parameters of the generative model are constantly approaching the sample labeling probability, so the difference between the parameters of the generative model and the sample labeling probability is used as the optimization feature value, and the evaluation It optimizes the eigenvalues and judges the training degree of the generated model.

Specifically, when the amount of data reaches a certain scale, the target phrase is labeled based on a variety of preset labeling rules, and the generated model is trained. The estimation of the true label of the target phrase based on the generative model is better than that of the sample label. Random guessing; and because the parameters of the generative model are used to estimate the accuracy of the label samples, and the sample label probability is calculated by the coverage of the total target phrase volume by the number of target phrases successfully labeled; so when the generative model's The closer the parameter is to the sample labeling probability, that is, the smaller the optimized eigenvalue, the closer the generative model is to the completion of training. For example, if the initial parameter of the generative model is 0.4 and the sample label probability is 0.92, the optimal eigenvalue is 0.52. After continuous iterative updating, the optimal eigenvalue gradually becomes smaller, and when the optimal eigenvalue becomes 0.01, the If the parameters are already close to the sample label probability, the iterative update ends.

S52 : Back-propagating the sample labeling probability by means of stochastic gradient descent to iteratively update the initial parameters, wherein, each iterative update results in new parameters of the generation model and changes in the optimized eigenvalues.

Specifically, by adopting the stochastic gradient descent method, the sample labeling probability is back-propagated to iteratively update the initial parameters, and a new parameter will be obtained for each update calculation to generate the model, and the new parameter and the sample label will be obtained. Probabilistic difference calculation can be used to obtain new optimized eigenvalues. Among them, since the optimized eigenvalues are calculated by the difference between the parameters of the generative model and the sample labeling probability, and after each iteration update, the parameters of the generative model will change, so each iterative update makes the optimized eigenvalues occur. Change.

Among them, the gradient descent method is a kind of iterative method, which can be used to solve the least squares problem. Gradient Descent is one of the most commonly used methods when solving model parameters of machine learning algorithms, i.e. unconstrained optimization problems. When solving the minimum value of the loss function, the gradient descent method can be used to iteratively solve the problem step by step to obtain the minimized loss function and model parameter values. Conversely, if you need to find the maximum value of the loss function, you need to use gradient ascent to iterate. In machine learning, two gradient descent methods have been developed based on the basic gradient descent method, namely stochastic gradient descent and batch gradient descent. In the embodiment of the present application, the method of stochastic gradient descent is used to back-propagate the sample labeling probability to iteratively update the initial parameters.

Among them, the back-propagation algorithm is a learning algorithm suitable for multi-layer neuron networks, which is based on the gradient descent method. The input-output relationship of the back-propagation network is essentially a mapping relationship: the function completed by a back-propagation neural network with n input and m output is the continuity from n-dimensional Euclidean space to a finite field in m-dimensional Euclidean space. mapping, which is highly nonlinear.

In a specific embodiment, the sample labeling probability is input into the input layer of the neural network, passes through the hidden layer, and finally reaches the output layer and outputs the result. This process is a forward propagation process. However, due to the error between the output result of the neural network and the actual result, the error between the estimated value and the actual value is calculated, which is also the optimized eigenvalue, and the optimized eigenvalue is back-propagated from the output layer to the hidden layer until Propagated to the input layer; in the process of backpropagation, the value of the sample labeling probability is adjusted according to the random drop of the optimized eigenvalue, so that the optimized eigenvalue is reduced. The above steps are iterated until the optimized feature value reaches a preset threshold.

S53: When the optimized feature value reaches the preset threshold, stop the iterative update to obtain a trained generative model.

Specifically, when the optimized feature value reaches the preset threshold, it indicates that the parameters of the generative model are very close to the sample labeling probability, and at this time, the updating of the parameters of the generative model is stopped, thereby obtaining a trained generative model.

It should be noted that the preset threshold is set according to the actual situation, and is not limited here. In a specific embodiment, the preset threshold value is 0.01.

S54: Output the labeling accuracy corresponding to the label sample through the trained generative model.

Specifically, the above steps have generated a trained generative model, and then perform probability estimation on the label samples through the trained generative model, and output the labeling accuracy rate corresponding to the label samples.

In this embodiment, the difference between the parameters of the generated model and the sample labeling probability is used as the optimization feature value, and the sample labeling probability is back-propagated by means of stochastic gradient descent to iteratively update the initial parameters. When the value reaches the preset threshold, the iterative update is stopped to obtain a trained generative model. The trained generative model outputs the labeling accuracy rate corresponding to the label sample, so as to train the generative model and output the labeling accuracy rate corresponding to the label sample. It is beneficial to improve the accuracy of data annotation.

Please refer to FIG. 5. FIG. 5 shows a specific implementation of step S54. In step S54, the specific implementation process of outputting the labeling accuracy rate corresponding to the label sample through the trained generation model is described in detail as follows:

S541: Perform probability estimation on the label samples by using the current parameters of the trained generative model to obtain the basic probability.

Specifically, the probability estimation is performed on the label samples through the current parameters to obtain the basic probability, which is convenient for further processing of the basic probability in the future, so as to obtain the final labeling accuracy rate. The current parameter refers to the parameters of the generated model obtained by iterative update when the optimized feature value reaches the preset threshold.

Specifically, a generative model refers to a model that can randomly generate observational data, especially given some implicit parameters. Generative models assign a joint probability distribution to observations and labeled data sequences. In the embodiment of the present application, the implicit parameter corresponds to the true label of the target phrase of the present application, the observed value corresponds to the sample labeling probability of the present application, and the labeled data sequence corresponds to the label sample of the present application; therefore, according to the implicit parameter, that is, the true The data label is randomly generated, and the model of the observation data is composed of its current parameters, which can judge the probability estimation of each preset labeling rule on the label sample, so as to obtain the basic probability.

S542: Perform weighted average processing on the basic probability to obtain the labeling accuracy rate corresponding to the labeling sample.

Specifically, by performing a weighted average process on the basic probability, the labeling accuracy is made more accurate.

In this embodiment, the probability estimation is performed on the label samples through the current parameters of the trained generation model to obtain the basic probability, and the weighted average processing is performed on the basic probability to obtain the labeling accuracy rate corresponding to the labeling sample, so that the generated labeling accuracy rate is more accurate , so as to improve the accuracy of data annotation.

Please refer to FIG. 6. FIG. 6 shows a specific implementation of step S1. In step S1, the text to be marked is obtained, and the text to be marked is split to obtain the specific implementation process of the split sentence, which is described in detail as follows:

S11: Acquire the text to be labeled, and preprocess the text to be labeled to obtain basic text.

Specifically, the preprocessing includes data cleaning of the text to be annotated. Among them, data cleaning refers to the process of re-examining and verifying data, with the purpose of removing duplicate information, correcting existing errors, and providing data consistency.

S12: Obtain the text delimiter contained in the basic text by means of regular matching.

S13: Split the basic text by the text separator to obtain a split statement.

Specifically, a regular matching method is used to obtain the text separator contained in the basic text, which is used to segment the text in subsequent steps.

Optionally, the text delimiters include format delimiters and punctuation delimiters.

The format delimiter refers to the delimiter that is divided according to the text encoding type or the text structure. The basic text is split according to the encoding type of the text or the structure of the text through the format separator.

The punctuation separator refers to the separator that divides the text according to the punctuation characters. Quickly split basic text with punctuation separators.

In this embodiment, the basic text is obtained by acquiring the text to be marked and preprocessing the text to be marked, and the text separator contained in the basic text is obtained by regular matching, and the basic text is split by the text separator, The split sentences are obtained, which is convenient for subsequent generation of target phrases, and is conducive to subsequent labeling of corresponding labels.

Please refer to FIG. 7. FIG. 7 shows a specific implementation manner after step S6. This embodiment includes:

S61: Obtain the storage path of the text to be marked as the target storage path;

S62: Map the target label sample to the target storage path by using a preset data mapping method.

Specifically, in order to trace the source of the data, it is convenient to query the target label sample corresponding to the text to be labeled, and the target label sample and the to-be-labeled file are stored in the same path.

The preset data mapping methods include but are not limited to: hand-coded (Hand-coded) and visual operation (Graphical manual). Manual coding is to define data correspondences directly with programming languages such as XSLT, JAVA, and C++; visualization operations usually allow users to draw a line between data items to define the correspondence between data items. In a specific embodiment, the target label samples are mapped into the target storage path through a visualization operation.

Please refer to Fig. 8. Fig. 8 shows the specific implementation process of merging the target word segmentation to obtain the target phrase, which is described in detail as follows:

S2A: Through part-of-speech tagging, part-of-speech tagging is performed on the target word to obtain part-of-speech segmentation.

Among them, part-of-speech tagging, also known as grammar tagging or part-of-speech disambiguation, is a text data processing technology in corpus linguistics that labels the parts of speech of words in a corpus according to their meaning and contextual content. Part-of-speech tagging can be done manually or by a specific algorithm. Using machine learning methods to achieve part-of-speech tagging is the research content of natural language processing. Common part-of-speech tagging algorithms include Hidden Markov Models, Conditional Random Fields, etc. In the embodiment of the present application, the target word segmentation is marked with the part of speech by means of part of speech tagging to obtain the part of speech segmentation.

S2B: According to the method of dependency syntax analysis, the parts of speech that conform to the consistency rules are combined to obtain the target phrase.

Among them, the consistency rule is to use the subject-verb-object (SBV) relationship, and mark the corresponding words. For example, "I eat apples" is marked as (I, Subject), (eat, Predict), (apple, Object), the extracted part-of-speech segmentation corresponds to the component of speech, and the part-of-speech segmentation that conforms to the consistency rules is merged, Get the target phrase.

In this embodiment, the part-of-speech tagging is used for the target word segmentation to obtain the part-of-speech segmentation, and according to the method of dependency syntax analysis, the part-of-speech segmentation that conforms to the consistency rule is combined to obtain the target phrase, and the target word segmentation is realized. Combined to facilitate subsequent data annotation.

It should be emphasized that, in order to further ensure the privacy and security of the above-mentioned text to be marked, the above-mentioned text to be marked may also be stored in a node of a blockchain.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing relevant hardware through computer-readable instructions of a computer, and the computer-readable instructions of the computer can be stored in a computer-readable In the storage medium, the computer-readable instructions, when executed, may include the processes of the foregoing method embodiments. Wherein, the aforementioned storage medium may be a non-volatile storage medium such as a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM), or a random access memory (Random Access Memory, RAM) or the like.

Please refer to FIG. 9. As an implementation of the method shown in FIG. 2, the present application provides an embodiment of a data labeling device based on a generative model. The device embodiment corresponds to the method embodiment shown in FIG. 2. Specifically, the device can be applied to various electronic devices.

As shown in FIG. 9 , the data labeling device based on the generation model in this embodiment includes: a text to be labelled splitting module 71 , a target phrase acquisition module 72 , a label sample generation module 73 , an initial parameter generation module 74 , and a labeling accuracy rate output module 75 and label sample selection module 76, wherein:

The to-be-labeled text splitting module 71 is used to obtain the to-be-labeled text, and to split the to-be-labeled text to obtain a split statement;

The target phrase acquisition module 72 is used to obtain the target word segmentation by performing word segmentation processing on the split sentence, and merge the target word segmentation to obtain the target phrase;

The label sample generation module 73 is configured to obtain multiple preset labeling rules, and respectively label the target phrase through the multiple preset labeling rules to obtain label samples corresponding to each preset rule;

The initial parameter generation module 74 is used to obtain the sample labeling probability of the target phrase by the label sample corresponding to each preset labeling rule, and obtain the initial parameters of the generation model according to the sample labeling probability and the label sample;

The labeling accuracy rate output module 75 is configured to iteratively update the initial parameters of the generative model through the sample labeling probability to obtain a trained generative model, and output the labeling accuracy rate corresponding to the label sample through the trained generative model;

The label sample selection module 76 is configured to select the label sample with the highest label accuracy rate as the target label sample.

Further, the initial parameter generation module 74 includes:

The sample labeling probability obtaining unit is used to calculate the coverage rate of the target phrase by the label sample corresponding to each preset labeling rule, and use the coverage rate as the sample labeling probability;

The initialization processing unit is used to initialize the sample label probability and the label sample to obtain the initial parameters of the generative model.

Further, the labeling accuracy output module 75 includes:

The optimization eigenvalue definition unit is used to use the difference between the parameters of the generated model and the sample labeling probability as the optimization eigenvalue;

The iterative update unit is used to back-propagate the sample labeling probability by means of stochastic gradient descent to iteratively update the initial parameters, in which, each iterative update obtains new parameters of the generative model and changes in the optimized eigenvalues ;

The iterative update stop unit is used to stop the iterative update when the optimized feature value reaches a preset threshold to obtain a trained generative model;

The labeling accuracy rate obtaining unit is used to output the labeling accuracy rate corresponding to the label sample through the trained generation model.

Further, the labeling accuracy obtaining unit includes:

The basic probability acquisition subunit is used to estimate the probability of the label sample through the current parameters of the trained generative model to obtain the basic probability;

The basic probability processing sub-unit is used to perform weighted average processing on the basic probability to obtain the labeling accuracy corresponding to the label sample.

Further, the to-be-labeled text splitting module 71 includes:

The basic text generation unit is used to obtain the text to be marked, and preprocess the text to be marked to obtain the basic text;

The text separator obtaining unit is used to obtain the text separator contained in the basic text by means of regular matching;

The split statement generation unit is used to split the basic text by the text separator to obtain the split statement.

Further, after the label sample selection module 76, the data labeling device based on the generative model also includes:

The target storage path obtaining module is used to obtain the storage path of the text to be marked as the target storage path;

The data mapping module is used to map the target label samples to the target storage path through a preset data mapping method.

Further, the target phrase acquisition module 72 also includes:

The part-of-speech and word-segmentation generating unit is used to perform part-of-speech tagging on the target word by means of part-of-speech tagging to obtain part-of-speech segmentation;

The target phrase generation unit is used to combine the part-of-speech segmentations that conform to the consistency rules according to the method of dependency syntax analysis to obtain the target phrase.

To solve the above technical problems, the embodiments of the present application also provide computer equipment. Please refer to FIG. 10 for details. FIG. 10 is a block diagram of the basic structure of a computer device according to this embodiment.

The computer device 8 includes a memory 81 , a processor 82 , and a network interface 83 that are connected to each other through a system bus. It should be pointed out that the figure only shows the computer device 8 with three components, the memory 81, the processor 82, and the network interface 83, but it should be understood that it is not required to implement all the components shown, and alternative implementations are possible. More or fewer components. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculation and/or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, special-purpose Integrated circuit (Application Specific Integrated Circuit, ASIC), programmable gate array (Field-Programmable Gate Array, FPGA), digital processor (Digital Signal Processor, DSP), embedded equipment, etc.

A computer device includes a memory and a processor, the memory stores computer-readable instructions, and the processor implements the following steps when executing the computer-readable instructions:

By segmenting the split sentences, the target word segmentation is obtained, and the target word segmentation is merged to obtain the target phrase;

Obtaining a variety of preset labeling rules, and labeling the target phrase respectively through the multiple preset labeling rules, to obtain a label sample corresponding to each preset rule;

Obtain the sample labeling probability of the target phrase by the label sample corresponding to each preset labeling rule, and obtain the initial parameters of the generation model according to the sample labeling probability and label sample;

Iteratively update the initial parameters of the generative model through the sample labeling probability to obtain a trained generative model, and output the labeling accuracy corresponding to the label sample through the trained generative model;

Select the label sample with the highest label accuracy as the target label sample.

The computer equipment may be a desktop computer, a notebook computer, a palmtop computer, and a cloud server and other computing equipment. Computer devices can interact with users through keyboards, mice, remote controls, touchpads, or voice-activated devices.

The memory 81 includes at least one type of readable storage medium, the computer-readable storage medium may be non-volatile or volatile, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (eg , SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), programmable read only memory (PROM) ), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 81 may be an internal storage unit of the computer device 8 , such as a hard disk or memory of the computer device 8 . In other embodiments, the memory 81 may also be an external storage device of the computer device 8, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, flash memory card (Flash Card), etc. Of course, the memory 81 may also include both the internal storage unit of the computer device 8 and its external storage device. In this embodiment, the memory 81 is generally used to store the operating system and various application software installed on the computer device 8 , such as computer-readable instructions of the data labeling method based on the generation model, and the like. In addition, the memory 81 can also be used to temporarily store various types of data that have been output or will be output.

The processor 82 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip in some embodiments. The processor 82 is typically used to control the overall operation of the computer device 8 . In this embodiment, the processor 82 is configured to run the computer-readable instructions stored in the memory 81 or process the data, for example, run the computer-readable instructions of the above-mentioned generative model-based data labeling method, so as to realize the data labeling method based on the generative model. various embodiments.

The network interface 83 may comprise a wireless network interface or a wired network interface, and the network interface 83 is typically used to establish a communication connection between the computer device 8 and other electronic devices.

The present application also provides another embodiment, that is, to provide a computer-readable storage medium, where the computer-readable storage medium stores computer-readable instructions of a computer, and the computer-readable instructions of the computer can be executed by at least one processor to Cause at least one processor to perform the following steps:

From the description of the above embodiments, those skilled in the art can clearly understand that the method of the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is better implementation. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product in essence or in a part that contributes to the prior art, and the computer software product is stored in a storage medium (such as ROM/RAM, magnetic disk, CD-ROM), including several instructions to make a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) execute the methods of the various embodiments of the present application.

The blockchain referred to in this application is a new application mode of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanism, and encryption algorithm. Blockchain, essentially a decentralized database, is a series of data blocks associated with cryptographic methods. Each data block contains a batch of network transaction information to verify its Validity of information (anti-counterfeiting) and generation of the next block. The blockchain can include the underlying platform of the blockchain, the platform product service layer, and the application service layer.

Obviously, the above-described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The accompanying drawings show the preferred embodiments of the present application, but do not limit the scope of the patent of the present application. This application may be embodied in many different forms, rather these embodiments are provided so that a thorough and complete understanding of the disclosure of this application is provided. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or perform equivalent replacements for some of the technical features. . Any equivalent structure made by using the contents of the description and drawings of the present application, which is directly or indirectly used in other related technical fields, is also within the scope of protection of the patent of the present application.

Claims

A data labeling method based on a generative model, comprising:

Obtain the text to be marked, and split the text to be marked to obtain a split statement;

By performing word segmentation processing on the split sentence, a target word segmentation is obtained, and the target word segmentation is combined to obtain a target phrase;

Obtaining a plurality of preset labeling rules, and labeling the target phrase respectively through the plurality of the predefined labeling rules, to obtain a label sample corresponding to each of the predefined rules;

Obtain the sample labeling probability of the target phrase by the label sample corresponding to each of the preset labeling rules, and obtain the initial parameters of the generation model according to the sample labeling probability and the label sample;

Iteratively update the initial parameters of the generative model through the sample labeling probability to obtain a trained generative model, and output the labeling accuracy rate corresponding to the label sample through the trained generative model;

The label sample with the highest labeling accuracy is selected as the target label sample.
The data labeling method based on a generative model according to claim 1, wherein the obtaining a label sample corresponding to each of the preset labeling rules is a sample labeling probability of the target phrase, and according to the sample labeling probability and the label samples, the initial parameters of the generated model include:

Calculate the coverage rate of each label sample corresponding to the preset labeling rule to the target phrase, and use the coverage rate as the sample labeling probability;

The sample label probability and the label sample are initialized to obtain the initial parameters of the generative model.
The data labeling method based on a generative model according to claim 1, wherein the initial parameters of the generative model are iteratively updated by using the sample labeling probability to obtain a trained generative model, and the trained The labeling accuracy rate corresponding to the labeling sample output by the generative model includes:

Taking the difference between the parameters of the generation model and the sample labeling probability as the optimization feature value;

Using the stochastic gradient descent method, the sample labeling probability is back-propagated to iteratively update the initial parameters, wherein, new parameters of the generative model and the optimization feature are obtained each time the iterative update is performed. value changes;

When the optimized feature value reaches a preset threshold, stop the iterative update to obtain the trained generative model;

The labeling accuracy rate corresponding to the label sample is output through the trained generation model.
The data labeling method based on a generative model according to claim 3, wherein the outputting the labeling accuracy rate corresponding to the label sample by the trained generative model comprises:

Perform probability estimation on the label sample by using the current parameter of the trained generation model to obtain a basic probability, wherein the current parameter refers to the parameter obtained by the iterative update when the optimized feature value reaches a preset threshold;

A weighted average process is performed on the basic probability to obtain the labeling accuracy rate corresponding to the labeling sample.
The data labeling method based on a generative model according to claim 1, wherein the obtaining the text to be labelled and splitting the text to be labelled to obtain a split statement comprises:

obtaining the text to be marked, and preprocessing the text to be marked to obtain basic text;

Obtain the text delimiter contained in the basic text by means of regular matching;

The basic text is split by the text separator to obtain the split statement.
The data labeling method based on a generative model according to claim 1, wherein after selecting the label sample with the highest label accuracy rate as the target label sample, the method further comprises:

Obtain the storage path of the text to be marked as the target storage path;

The target label sample is mapped to the target storage path by a preset data mapping method.
The data labeling method based on a generative model according to claim 1, wherein the merging the target word segmentation to obtain the target phrase comprises:

By means of part-of-speech tagging, part-of-speech tagging is performed on the target word segmentation to obtain part-of-speech segmentation;

According to the method of dependency syntax analysis, the part-of-speech segmentations that conform to the consistency rule are combined to obtain the target phrase.
A data labeling device based on a generative model, comprising:

A to-be-labeled text splitting module, configured to obtain the to-be-labeled text, and to split the to-be-labeled text to obtain a split statement;

a target phrase acquisition module, used to obtain a target word segmentation by performing word segmentation processing on the split statement, and merging the target word segmentation to obtain a target phrase;

The label sample generation module is configured to obtain a plurality of preset labeling rules, and respectively label the target phrase through the plurality of the predefined labeling rules, to obtain label samples corresponding to each of the predefined rules;

The initial parameter generation module is used to obtain the sample labeling probability of the target phrase by the label sample corresponding to each of the preset labeling rules, and obtain the initial parameters of the generation model according to the sample labeling probability and the label sample ;

The labeling accuracy output module is used to iteratively update the initial parameters of the generative model through the sample labeling probability to obtain a trained generative model, and output the labels corresponding to the label samples through the trained generative model Accuracy;

The label sample selection module is configured to select the label sample with the highest label accuracy rate as the target label sample.
A computer device, comprising a memory and a processor, wherein computer-readable instructions are stored in the memory, wherein the processor implements the following steps when executing the computer-readable instructions:

Obtain the text to be marked, and split the text to be marked to obtain a split statement;

By performing word segmentation processing on the split sentence, a target word segmentation is obtained, and the target word segmentation is combined to obtain a target phrase;

Obtaining a plurality of preset labeling rules, and labeling the target phrase respectively through the plurality of the predefined labeling rules, to obtain a label sample corresponding to each of the predefined rules;

Obtain the sample labeling probability of the target phrase by the label sample corresponding to each of the preset labeling rules, and obtain the initial parameters of the generation model according to the sample labeling probability and the label sample;

Iteratively update the initial parameters of the generative model through the sample labeling probability to obtain a trained generative model, and output the labeling accuracy rate corresponding to the label sample through the trained generative model;

The label sample with the highest labeling accuracy is selected as the target label sample.
The computer device according to claim 9, wherein the obtaining the sample labeling probability of the target phrase by the label sample corresponding to each of the preset labeling rules, and according to the sample labeling probability and the label sample , the initial parameters of the generated model include:

Calculate the coverage rate of each label sample corresponding to the preset labeling rule to the target phrase, and use the coverage rate as the sample labeling probability;

The sample label probability and the label sample are initialized to obtain the initial parameters of the generative model.
The computer device according to claim 9, wherein the obtaining the sample labeling probability of the target phrase by the label sample corresponding to each of the preset labeling rules, and according to the sample labeling probability and the label sample , the initial parameters of the generated model include:

The iteratively updating the initial parameters of the generation model through the sample labeling probability to obtain a trained generation model, and outputting the labeling accuracy rate corresponding to the label sample through the trained generation model includes:

Taking the difference between the parameters of the generation model and the sample labeling probability as the optimization feature value;

Using the stochastic gradient descent method, the sample labeling probability is back-propagated to iteratively update the initial parameters, wherein, new parameters of the generative model and the optimization feature are obtained each time the iterative update is performed. value changes;

When the optimized feature value reaches a preset threshold, stop the iterative update to obtain the trained generative model;

The labeling accuracy rate corresponding to the label sample is output through the trained generation model.
The computer device according to claim 11, wherein the outputting the labeling accuracy rate corresponding to the labeling sample by the trained generative model comprises:

Perform probability estimation on the label sample by using the current parameter of the trained generation model to obtain a basic probability, wherein the current parameter refers to the parameter obtained by the iterative update when the optimized feature value reaches a preset threshold;

A weighted average process is performed on the basic probability to obtain the labeling accuracy rate corresponding to the labeling sample.
The computer device according to claim 9, wherein the acquiring the text to be marked and splitting the text to be marked, and obtaining the split statement comprises:

obtaining the text to be marked, and preprocessing the text to be marked to obtain basic text;

Obtain the text delimiter contained in the basic text by means of regular matching;

The basic text is split by the text separator to obtain the split statement.
The computer device according to claim 9, wherein after selecting the label sample with the highest label accuracy rate as the target label sample, the method further comprises:

Obtain the storage path of the text to be marked as the target storage path;

The target label sample is mapped to the target storage path by a preset data mapping method.
The computer device according to claim 9, wherein the combining the target word segmentation to obtain the target phrase comprises:

By means of part-of-speech tagging, part-of-speech tagging is performed on the target word segmentation to obtain part-of-speech segmentation;

According to the method of dependency syntax analysis, the part-of-speech segmentations that conform to the consistency rule are combined to obtain the target phrase.
A computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions that, when executed by a processor, cause the processor to perform the following steps:

Obtain the text to be marked, and split the text to be marked to obtain a split statement;

By performing word segmentation processing on the split sentence, a target word segmentation is obtained, and the target word segmentation is combined to obtain a target phrase;

Obtaining a plurality of preset labeling rules, and labeling the target phrase respectively through the plurality of the predefined labeling rules, to obtain a label sample corresponding to each of the predefined rules;

Obtain the sample labeling probability of the target phrase by the label sample corresponding to each of the preset labeling rules, and obtain the initial parameters of the generation model according to the sample labeling probability and the label sample;

Iteratively update the initial parameters of the generative model through the sample labeling probability to obtain a trained generative model, and output the labeling accuracy rate corresponding to the label sample through the trained generative model;

The label sample with the highest labeling accuracy is selected as the target label sample.
The computer-readable storage medium according to claim 16, wherein the obtaining the sample labeling probability of the target phrase by the label sample corresponding to each of the preset labeling rules, and according to the sample labeling probability and the sample labeling probability According to the label sample, the initial parameters of the generated model include:

Calculate the coverage rate of each label sample corresponding to the preset labeling rule to the target phrase, and use the coverage rate as the sample labeling probability;

The sample label probability and the label sample are initialized to obtain the initial parameters of the generative model.
The computer-readable storage medium according to claim 16, wherein the obtaining the sample labeling probability of the target phrase by the label sample corresponding to each of the preset labeling rules, and according to the sample labeling probability and the sample labeling probability According to the label sample, the initial parameters of the generated model include:

The iteratively updating the initial parameters of the generation model through the sample labeling probability to obtain a trained generation model, and outputting the labeling accuracy rate corresponding to the label sample through the trained generation model includes:

Taking the difference between the parameters of the generation model and the sample labeling probability as the optimization feature value;

Using the stochastic gradient descent method, the sample labeling probability is back-propagated to iteratively update the initial parameters, wherein, new parameters of the generative model and the optimization feature are obtained each time the iterative update is performed. value changes;

When the optimized feature value reaches a preset threshold, stop the iterative update to obtain the trained generative model;

The labeling accuracy rate corresponding to the label sample is output through the trained generation model.
The computer-readable storage medium according to claim 18, wherein the outputting the labeling accuracy rate corresponding to the labeling sample by the trained generative model comprises:

Perform probability estimation on the label sample by using the current parameter of the trained generation model to obtain a basic probability, wherein the current parameter refers to the parameter obtained by the iterative update when the optimized feature value reaches a preset threshold;

A weighted average process is performed on the basic probability to obtain the labeling accuracy rate corresponding to the labeling sample.
The computer-readable storage medium according to claim 16, wherein the acquiring the text to be marked and splitting the text to be marked, and obtaining the split sentence comprises:

obtaining the text to be marked, and preprocessing the text to be marked to obtain basic text;

Obtain the text delimiter contained in the basic text by means of regular matching;

The basic text is split by the text separator to obtain the split statement.