CN112487817A - Named entity recognition model training method, sample labeling method, device and equipment - Google Patents
Named entity recognition model training method, sample labeling method, device and equipment Download PDFInfo
- Publication number
- CN112487817A CN112487817A CN202011481841.0A CN202011481841A CN112487817A CN 112487817 A CN112487817 A CN 112487817A CN 202011481841 A CN202011481841 A CN 202011481841A CN 112487817 A CN112487817 A CN 112487817A
- Authority
- CN
- China
- Prior art keywords
- named entity
- entity recognition
- sample
- recognition result
- text
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012549 training Methods 0.000 title claims abstract description 167
- 238000000034 method Methods 0.000 title claims abstract description 62
- 238000002372 labelling Methods 0.000 title abstract description 21
- 238000012545 processing Methods 0.000 claims abstract description 16
- 238000004590 computer program Methods 0.000 claims description 18
- 238000004891 communication Methods 0.000 description 14
- 238000010586 diagram Methods 0.000 description 10
- 230000008569 process Effects 0.000 description 9
- 239000003086 colorant Substances 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000003058 natural language processing Methods 0.000 description 2
- 238000013519 translation Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 235000019800 disodium phosphate Nutrition 0.000 description 1
- 238000009432 framing Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
- G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
- G06F40/295—Named entity recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/332—Query formulation
- G06F16/3329—Natural language query formulation or dialogue systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Evolutionary Biology (AREA)
- Databases & Information Systems (AREA)
- Machine Translation (AREA)
Abstract
The application provides a named entity recognition model training method, a sample labeling method, a device and equipment, and belongs to the technical field of character recognition. The method comprises the following steps: receiving a session text sent by terminal equipment; carrying out named entity recognition processing on the session text by using a named entity recognition model to obtain a named entity recognition result of the session text, and sending the named entity recognition result to the terminal equipment; if the adjusted recognition result sent by the terminal equipment is received, updating the adjusted recognition result and the session text to a training set of the named entity recognition model by taking the adjusted recognition result and the session text as new samples to obtain an updated training set; and training the named entity recognition model based on the updated training set. The text recognition cost can be reduced.
Description
Technical Field
The application relates to the technical field of character recognition, in particular to a named entity recognition model training method, a sample labeling device and equipment.
Background
People often use communication tools to interact with each other during work, such as: chat tools, and the like. The communication words obtained by the communication tools can be used as linguistic data in the natural language processing field. In the field of natural language processing, such as information extraction, question and answer system, syntax analysis, machine translation, etc., Named Entity Recognition (NER) can be performed on the unstructured characters such as the above-mentioned communication characters, so as to extract various Named entities. The method for extracting the named entities by using the named entity recognition model is a commonly used means. The named entity recognition model needs to be trained in advance.
At present, the adopted recognition method mainly adopts the type of linguistic data to be recognized to train and label the existing named entity recognition model, and then recognizes the characters corresponding to the type of linguistic data. However, when different types of characters need to be recognized, the training corpus needs to be reset to train the model, and a large amount of cost needs to be consumed in the training process, which results in high recognition cost.
Disclosure of Invention
The application aims to provide a named entity recognition model training method, a sample labeling method, a device and equipment, which can reduce text recognition cost.
The embodiment of the application is realized as follows:
in one aspect of the embodiments of the present application, a method for training a named entity recognition model is provided, where the method includes:
receiving a session text sent by terminal equipment;
carrying out named entity recognition processing on the session text by using a named entity recognition model to obtain a named entity recognition result of the session text, and sending the named entity recognition result to the terminal equipment;
if the adjusted recognition result sent by the terminal equipment is received, updating the adjusted recognition result and the session text to a training set of the named entity recognition model by taking the adjusted recognition result and the session text as new samples to obtain an updated training set;
and training the named entity recognition model based on the updated training set.
Optionally, training the named entity recognition model based on the updated training set includes:
determining the sample weight of each sample in the updated training set;
and training the named entity recognition model based on the sample weight of each sample.
Optionally, determining the sample weight of each sample in the updated training set includes:
and determining the sample weight of each sample according to the time sequence of each sample.
Optionally, determining the sample weight of each sample in the updated training set includes:
and determining the sample weight of each sample according to the sample category of each sample.
Optionally, training the named entity recognition model based on the updated training set includes:
judging whether to retrain the named entity recognition model or not according to a preset updating strategy;
and if so, training the named entity recognition model based on the updated training set.
In another aspect of the embodiments of the present application, a sample labeling method is provided, where the method includes:
acquiring a session text and sending the session text to a server;
receiving a named entity recognition result of the session text sent by the server;
displaying the named entity recognition result by using a preset style;
and if the adjusted recognition result aiming at the named entity recognition result input by the user is received, sending the adjusted recognition result to the server.
Optionally, the preset pattern includes: underline, preset background color.
In another aspect of the embodiments of the present application, a named entity recognition model training apparatus is provided, where the apparatus includes: the system comprises a text receiving module, a text recognition module, an adjustment recognition module and a model training module;
the text receiving module is used for receiving a session text sent by the terminal equipment;
the text recognition module is used for carrying out named entity recognition processing on the session text by using the named entity recognition model to obtain a named entity recognition result of the session text and sending the named entity recognition result to the terminal equipment;
the adjustment identification module is used for updating the adjusted identification result and the session text as new samples to a training set of the named entity identification model to obtain an updated training set if the adjusted identification result sent by the terminal equipment is received;
and the model training module is used for training the named entity recognition model based on the updated training set.
Optionally, the model training module is specifically configured to determine a sample weight of each sample in the updated training set; and training the named entity recognition model based on the sample weight of each sample.
Optionally, the model training module is specifically configured to determine a sample weight of each sample according to a time sequence of each sample.
Optionally, the model training module is specifically configured to determine a sample weight of each sample according to a sample category of each sample.
Optionally, the model training module is specifically configured to determine whether to retrain the named entity recognition model according to a preset update strategy; and if so, training the named entity recognition model based on the updated training set.
In another aspect of the embodiments of the present application, there is provided a sample labeling apparatus, including: the system comprises a text sending module, a result receiving module, a result display module and a result adjusting module;
the text sending module is used for acquiring the session text and sending the session text to the server;
the result receiving module is used for receiving the named entity recognition result of the session text sent by the server;
the result display module is used for displaying the named entity recognition result by using a preset style;
and the result adjusting module is used for sending the adjusted recognition result to the server if the adjusted recognition result aiming at the named entity recognition result input by the user is received.
Optionally, in the apparatus, the preset pattern includes: underline, preset background color.
In another aspect of the embodiments of the present application, a server is provided, which includes: the first memory and the first processor are used for storing a computer program which can run on the first processor, and when the first processor executes the computer program, the steps of the named entity recognition model training method are realized.
In another aspect of the embodiments of the present application, a terminal device is provided, including: the second processor executes the computer program to realize the steps of the sample labeling method.
In another aspect of the embodiments of the present application, a storage medium is provided, where a computer program is stored on the storage medium, and when the computer program is executed by a processor, the steps of the named entity recognition model training method and the sample labeling method are implemented.
The beneficial effects of the embodiment of the application include:
in the named entity recognition model training method, the sample labeling method, the device and the equipment provided by the embodiment of the application, a session text sent by terminal equipment can be received; carrying out named entity recognition processing on the session text by using a named entity recognition model to obtain a named entity recognition result of the session text, and sending the named entity recognition result to the terminal equipment; if the adjusted recognition result sent by the terminal equipment is received, updating the adjusted recognition result and the session text to a training set of the named entity recognition model by taking the adjusted recognition result and the session text as new samples to obtain an updated training set; and training the named entity recognition model based on the updated training set. The adjusted recognition result and the session text are used as new samples to be updated to the training set of the named entity recognition model, the updated training set can be obtained, the named entity recognition model can be trained based on the updated training set, further, the named entity recognition model does not need to be trained by adopting additional training samples, self-iterative training of the model is achieved, in addition, the named entity recognition model can be trained based on adjustment of sample weights of all samples, different application scenes can be further adapted, new training corpora do not need to be added again to serve as training samples, accordingly, the cost of text recognition can be reduced, and manpower and expense resources are saved. In addition, the accuracy and timeliness of model identification can be improved by continuously updating the training samples.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained from the drawings without inventive effort.
Fig. 1 is a schematic view of an application scenario of a named entity recognition model training method and a sample labeling method according to an embodiment of the present application;
fig. 2 is a first flowchart of a named entity recognition model training method according to an embodiment of the present disclosure;
fig. 3 is a schematic flowchart illustrating a second method for training a named entity recognition model according to an embodiment of the present disclosure;
fig. 4 is a third schematic flowchart of a named entity recognition model training method provided in the embodiment of the present application;
fig. 5 is a schematic flowchart of a sample labeling method according to an embodiment of the present application;
fig. 6 is an interaction diagram of a server and a terminal device according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of a named entity recognition model training device according to an embodiment of the present disclosure;
FIG. 8 is a schematic structural diagram of a sample labeling apparatus according to an embodiment of the present application;
fig. 9 is a schematic structural diagram of a server according to an embodiment of the present application;
fig. 10 is a schematic structural diagram of a terminal device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
In the description of the present application, it is noted that the terms "first", "second", "third", and the like are used merely for distinguishing between descriptions and are not intended to indicate or imply relative importance.
In order to facilitate understanding of the scheme provided by the embodiment of the present application, a scenario applied in the embodiment of the present application is now described, in the embodiment of the present application, the named entity recognition model training method is applied to a server, the sample labeling method is applied to a terminal device, and the server is in communication connection with the terminal device, where the server may be a cloud server or a local server, and is not limited herein; the terminal device may be a computer, a mobile phone, a tablet computer, or a dedicated electronic device, and is not limited herein.
Fig. 1 is a schematic view of an application scenario of a named entity recognition model training method and a sample labeling method provided in an embodiment of the present application, please refer to fig. 1, where the application scenario includes: the server 100 and the terminal device 200 are connected in communication, the communication connection may be a wired communication connection or a wireless communication connection, and fig. 1 illustrates a wired communication connection as an example. The specific interactive information may include: conversation text, named entity recognition result and adjusted recognition result.
The following specifically explains a specific implementation process of the named entity recognition model training method provided in the embodiment of the present application.
Fig. 2 is a first schematic flow chart of a named entity recognition model training method provided in an embodiment of the present application, please refer to fig. 2, where the method includes:
s110: and receiving the session text sent by the terminal equipment.
Alternatively, the conversation text may be a text file including text information, and specifically may be a text message sent through chat software or office software (e.g., enterprise WeChat), for example. The session text may be sent by a terminal device communicatively coupled to the server.
Alternatively, the session text may be sent directly to the server by the terminal device, or may be sent correspondingly by the terminal device in response to a request instruction sent by the server, which is not limited herein.
S120: and carrying out named entity identification processing on the session text by using the named entity identification model to obtain a named entity identification result of the session text, and sending the named entity identification result to the terminal equipment.
Optionally, a Named Entity Recognition model (NER) may be a model for performing information extraction, syntax analysis, machine translation, and other processing on a text, and specifically may recognize three categories in the text: entity class, time class, number class, and seven minor classes: and identifying the name, the organization name, the place name, the time, the date, the currency, the percentage and the like to obtain a corresponding identification result, wherein the identification result is the category of one word or one section of sentence obtained by identifying the named entity identification model in the conversation text. The named entity recognition model may be a convolutional neural network model or a deep neural network model, and the named entity recognition model may implement the above functions, which is not limited herein.
For example: the text content is 'Zhang Sanming day to play in Shanghai', and the 'Zhang San' can be recognized as 'name of a person' through the named entity recognition model; "tomorrow" is identified as "time"; "Shanghai" is identified as "place name" and the like. The categories such as 'name', 'time', 'place name' and the like and the text content corresponding to the categories are the recognition results of the named entity recognition model. The named entity recognition result includes a certain word or a sentence in the conversation text and a category corresponding to the word or the sentence, that is, "zhang san" corresponds to "name of person" is one of the recognition results, and other recognition results can be obtained correspondingly.
Optionally, after obtaining the corresponding recognition result through the named entity recognition model, the recognition result may be sent to a terminal device in communication connection with the server.
S130: and if the adjusted recognition result sent by the terminal equipment is received, updating the adjusted recognition result and the session text to a training set of the named entity recognition model by taking the adjusted recognition result and the session text as new samples to obtain an updated training set.
Optionally, the training set of the named entity recognition model may be used to perform model training on the named entity recognition model, and the training set may be updated by using the adjusted recognition result and the session text as new samples, that is, adding new samples to the training set. If the recognition result of the named entity recognition model is wrong, the terminal device can perform corresponding adjustment processing based on the recognition result of the named entity recognition model and can send the adjusted recognition result to the server, and if the server receives the adjusted recognition result sent by the terminal device, the training set can be updated to obtain an updated training set.
S140: and training the named entity recognition model based on the updated training set.
Optionally, the updated training set includes a new sample, and the new sample is obtained based on the adjusted recognition result and the session text, so that performing model training on the named entity recognition model based on the updated training set can improve accuracy of the named entity recognition model in recognizing the session text corresponding to the adjusted recognition result.
In the training method for the named entity recognition model provided by the embodiment of the application, a session text sent by terminal equipment can be received; carrying out named entity recognition processing on the session text by using a named entity recognition model to obtain a named entity recognition result of the session text, and sending the named entity recognition result to the terminal equipment; if the adjusted recognition result sent by the terminal equipment is received, updating the adjusted recognition result and the session text to a training set of the named entity recognition model by taking the adjusted recognition result and the session text as new samples to obtain an updated training set; and training the named entity recognition model based on the updated training set. The adjusted recognition result and the session text are used as new samples to be updated to the training set of the named entity recognition model, the updated training set can be obtained, the named entity recognition model can be trained based on the updated training set, and then the named entity recognition model does not need to be trained by adopting extra training samples, so that the self-iterative training of the model is realized, correspondingly, the text recognition cost can be reduced, and the manpower and expense resources are saved.
The following explains a specific implementation process of training the named entity recognition model provided in the embodiment of the present application.
Fig. 3 is a flowchart illustrating a second method for training a named entity recognition model according to an embodiment of the present application, and referring to fig. 3, the training of the named entity recognition model based on an updated training set includes:
s210: and determining the sample weight of each sample in the updated training set.
Optionally, after the updated training set is obtained, a sample weight of each sample or each sample in the training set may be determined, and the sample weight may be set according to actual requirements.
The sample weight may be a quantity characterizing the importance degree of each sample, and the greater the sample weight is, the higher the importance degree of the sample in the training set is, that is, the higher the priority in performing training recognition is.
S220: and training the named entity recognition model based on the sample weight of each sample.
Optionally, after the sample weights of the samples are determined, corresponding model training may be performed on the named entity recognition model according to the sample weights of each sample or each sample, specifically, a large amount of training may be performed on samples with high sample weights, and a small amount of training may be performed on samples with low sample weights.
In one possible embodiment, determining the sample weight of each sample in the updated training set includes:
and determining the sample weight of each sample according to the time sequence of each sample.
Optionally, the time sequence may be a sequence in which each sample is input into the training set, the sample weight of each sample input into the training set first may be set to be larger, or the sample weight of each sample input into the training set later may be set to be larger, and the time sequence may be specifically set according to an actual work requirement, which is not limited herein.
For example, in the morning and afternoon of a certain day, the terminal device sends a plurality of adjusted recognition results, and accordingly, new samples corresponding to the adjusted recognition results and the corresponding session texts may be determined and sequentially input into the training set according to the time sequence, and when performing weight setting on the samples in the training set, the weight setting of the samples input in the morning may be larger, and the weight setting of the samples input in the afternoon may be smaller.
In another possible embodiment, determining the sample weight of each sample in the updated training set includes:
and determining the sample weight of each sample according to the sample category of each sample.
Optionally, the sample category may be three or seven categories in the recognition result of the named entity recognition model in the foregoing description, or may be some categories that are reset by the user according to the user's own needs, for example: divided into different categories according to the size of the sample, etc.
For example, when the named entity recognition model is required to recognize the "person name", the samples including the "person name" may be classified into one type, the samples not including the "person name" may be classified into another type, the sample weight of each sample including the "person name" may be set to be larger, and the sample weight of each sample not including the "person name" may be set to be smaller. Accordingly, if other types of session texts need to be identified, the classification mode may be modified correspondingly and the sample weight may be changed correspondingly, and the specific classification mode may be set according to the specific identification work performed by using the named entity identification model, which is not limited herein.
In the embodiment of the application, the named entity recognition model can be trained based on the adjustment of the sample weight of each sample, so that the method can adapt to different application scenes, a new training corpus is not required to be added as a training sample, only the divided classes are required to be adjusted, and the samples of different classes are endowed with different sample weights for model training, so that the cost of the training sample is saved. In addition, the accuracy and timeliness of model identification can be improved by continuously updating the training samples.
The following explains yet another specific implementation procedure for training the named entity recognition model provided in the embodiment of the present application.
Fig. 4 is a third schematic flowchart of a method for training a named entity recognition model according to an embodiment of the present application, and please refer to fig. 4, the method for training a named entity recognition model based on an updated training set includes:
s310: and judging whether to retrain the named entity recognition model or not according to a preset updating strategy.
Optionally, the preset update policy may be: the number of the accumulated update samples, the ratio of the number of the accumulated update samples to the total sample amount, and the like may be set according to actual requirements, and are not limited herein.
Alternatively, the condition may be met according to a preset update policy, for example: and if the conditions are met, retraining the named entity recognition model can be determined, and then the named entity recognition model is updated.
If yes, S320: and training the named entity recognition model based on the updated training set.
Optionally, if it is determined that the named entity recognition model is retrained, the named entity recognition model may be subjected to model training according to the updated training set, and it should be noted that the process of performing model training on the named entity recognition model is a training process of a neural network, which is not described herein again.
Optionally, if the condition is not met, the named entity recognition model may not be updated, and the existing named entity recognition model may be kept unchanged.
The following explains a specific implementation process of the sample labeling method provided in the embodiment of the present application.
Fig. 5 is a schematic flowchart of a sample labeling method according to an embodiment of the present application, please refer to fig. 5, in which the method includes:
s410: and acquiring the session text and sending the session text to the server.
Optionally, the session text may be obtained from an application program in the terminal device, and the session text may be sent to a server in communication connection with the terminal device to perform named entity identification, where the specific identification process is already explained in the foregoing S120, and details are not repeated here.
Optionally, the session text may be extracted from the recorded content of the history message in the application software, or may be input when the user performs chat or message interaction through the application software in the terminal device in real time.
S420: and receiving a named entity recognition result of the session text sent by the server.
Alternatively, after the server performs named entity recognition on the session text, the recognition result may be sent to the terminal device, and the terminal device may receive the named entity recognition result of the session text. The named entity recognition result includes a certain word or a sentence in the conversation text and a category corresponding to the word or the sentence.
S430: and displaying the named entity recognition result by using a preset style.
Optionally, different preset styles may be set for different categories, and a term or a sentence in the conversation text corresponding to the category may be highlighted by using the different preset styles, so as to achieve an effect of displaying the named entity recognition result.
For example: if the session text is "tomorrow is one-tenth-month-day", and the named entity recognition result is that "one-tenth-month-day" corresponds to "time", a type of time can be preset, and then "one-tenth-month-day" in the session text can be displayed as the preset type.
S440: and if the adjusted recognition result aiming at the named entity recognition result input by the user is received, sending the adjusted recognition result to the server.
Optionally, after the named entity recognition result is displayed, the user may determine according to the displayed content to determine whether the recognition result meets the user's requirement, and if so, the recognition result is a normal recognition result, and the recognition result may be output.
If the user's requirement is not met, the user can input the adjusted recognition result aiming at the named entity recognition result, namely, the named entity recognition result can be modified and the recognition result meeting the user requirement is modified to be used as the adjusted recognition result.
For example: the session text is ' I need climb to a mountain at seaside ', the named entity recognition result is ' Shanghai ' corresponding to ' place ', the user judges that the ' Shanghai ' in the session text is not a place according to actual semantics, the named entity recognition result can be modified in a mode of inputting an adjusted recognition result aiming at the named entity recognition result, the adjusted recognition result can be ' the ' seaside mountain ' corresponding to ' place ', then the adjusted recognition result is sent to the server, and the server can establish a new sample based on the adjusted recognition result to perform model training on the named entity recognition model again.
Optionally, the preset pattern includes: underline, preset background color.
The underlining can be marking a straight line or a curve below the words or sentences corresponding to the categories of the named entity recognition results in the conversation text; the preset background color may be different colors set on the background of the words or sentences corresponding to the category of the named entity recognition result in the conversation text. Alternatively, different categories may be characterized by different underline shapes or categories of preset background colors.
Optionally, in addition to the preset style, the words may be displayed in various manners such as bolding, framing, enlarging, adjusting the font, and the like, and the difference between the words or phrases and other words or phrases in the conversation text may be highlighted, which is not limited herein.
For example, "person name" may be set as a downward straight line, "place" may be set as the background color of blue, and "time" may be set as the background color of red. For the conversation text ' zhang san tomorrow go to play in the Shanghai ' and the recognition result corresponding to the text, a straight line can be drawn below two characters of zhang san ' and the text; setting the background color of two characters of tomorrow as red; the background colors of the two characters of Shanghai are set to be blue, so that the different types of highlighting are realized.
Fig. 6 is an interaction diagram of a server and a terminal device according to an embodiment of the present application, please refer to fig. 6, where fig. 6 is an execution method for the server and the terminal device as a whole, and the specific execution method is as follows:
s510: the terminal equipment acquires the session text and sends the session text to the server.
S520: and the server uses the named entity recognition model to perform named entity recognition processing on the session text to obtain a named entity recognition result of the session text, and sends the named entity recognition result to the terminal equipment.
S530: and the terminal equipment displays the named entity recognition result by using a preset style.
S540: and if the terminal equipment receives the adjusted recognition result aiming at the named entity recognition result input by the user, the adjusted recognition result is sent to the server.
S550: and the server updates the adjusted recognition result and the session text as new samples to a training set of the named entity recognition model to obtain an updated training set.
S560: the server trains the named entity recognition model based on the updated training set.
The specific implementation process executed by the method of S510-S560 is already explained in the foregoing S110-S140 and S410-S440, and is not repeated herein.
The following describes apparatuses, devices, and storage media for executing the named entity recognition model training method and the sample labeling method provided in the present application, and specific implementation processes and technical effects thereof are referred to above and will not be described further below.
Fig. 7 is a schematic structural diagram of a named entity recognition model training device according to an embodiment of the present application, please refer to fig. 7, where the device includes: a text receiving module 110, a text recognition module 120, an adjustment recognition module 130, and a model training module 140;
a text receiving module 110, configured to receive a session text sent by a terminal device;
the text recognition module 120 is configured to perform named entity recognition processing on the session text by using a named entity recognition model to obtain a named entity recognition result of the session text, and send the named entity recognition result to the terminal device;
the adjustment recognition module 130 is configured to, if an adjusted recognition result sent by the terminal device is received, update the adjusted recognition result and the session text as new samples to a training set of the named entity recognition model to obtain an updated training set;
and a model training module 140, configured to train the named entity recognition model based on the updated training set.
Optionally, the model training module 140 is specifically configured to determine a sample weight of each sample in the updated training set; and training the named entity recognition model based on the sample weight of each sample.
Optionally, the model training module 140 is specifically configured to determine a sample weight of each sample according to a time sequence of each sample.
Optionally, the model training module 140 is specifically configured to determine a sample weight of each sample according to a sample category of each sample.
Optionally, the model training module 140 is specifically configured to determine whether to retrain the named entity recognition model according to a preset update strategy; and if so, training the named entity recognition model based on the updated training set.
Fig. 8 is a schematic structural diagram of a sample labeling apparatus according to an embodiment of the present application, please refer to fig. 8, which includes: a text sending module 210, a result receiving module 220, a result displaying module 230, and a result adjusting module 240;
the text sending module 210 is configured to obtain a session text and send the session text to the server;
a result receiving module 220, configured to receive a named entity identification result of the session text sent by the server;
a result display module 230 for displaying the named entity recognition result using a preset style;
and the result adjusting module 240 is configured to, if an adjusted recognition result for the named entity recognition result input by the user is received, send the adjusted recognition result to the server.
Optionally, in the apparatus, the preset pattern includes: underline, preset background color.
The above-mentioned apparatus is used for executing the method provided by the foregoing embodiment, and the implementation principle and technical effect are similar, which are not described herein again.
These above modules may be one or more integrated circuits configured to implement the above methods, such as: one or more Application Specific Integrated Circuits (ASICs), or one or more microprocessors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs), among others. For another example, when one of the above modules is implemented in the form of a Processing element scheduler code, the Processing element may be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).
Fig. 9 is a schematic structural diagram of a server according to an embodiment of the present application, please refer to fig. 9, where the server includes: the named entity recognition model training method comprises a first memory 310 and a first processor 320, wherein a computer program capable of running on the first processor 320 is stored in the first memory 310, and when the computer program is executed by the first processor 320, the steps of the named entity recognition model training method are realized.
Fig. 10 is a schematic structural diagram of a terminal device according to an embodiment of the present application, and referring to fig. 10, the terminal device includes: a second memory 410 and a second processor 420, wherein the second memory 410 stores a computer program operable on the second processor 420, and the second processor 420 executes the computer program to implement the steps of the sample labeling method.
In another aspect of the embodiments of the present application, a storage medium is further provided, where the storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the named entity recognition model training method and the sample labeling method are implemented.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, a division of a unit is merely a logical division, and an actual implementation may have another division, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
The integrated unit implemented in the form of a software functional unit may be stored in a computer readable storage medium. The software functional unit is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device) or a processor (processor) to execute some steps of the methods according to the embodiments of the present invention. And the aforementioned storage medium includes: a U disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (12)
1. A named entity recognition model training method, characterized in that the method comprises:
receiving a session text sent by terminal equipment;
using a named entity recognition model to perform named entity recognition processing on the session text to obtain a named entity recognition result of the session text, and sending the named entity recognition result to the terminal equipment;
if the adjusted recognition result sent by the terminal equipment is received, updating the adjusted recognition result and the session text to a training set of the named entity recognition model by taking the adjusted recognition result and the session text as new samples to obtain an updated training set;
training the named entity recognition model based on the updated training set.
2. The method of claim 1, wherein the training the named entity recognition model based on the updated training set comprises:
determining sample weights of the samples in the updated training set;
and training the named entity recognition model based on the sample weight of each sample.
3. The method of claim 2, wherein determining the sample weight for each sample in the updated training set comprises:
and determining the sample weight of each sample according to the time sequence of each sample.
4. The method of claim 2, wherein determining the sample weight for each sample in the updated training set comprises:
and determining the sample weight of each sample according to the sample category of each sample.
5. The method of any one of claims 1-4, wherein the training the named entity recognition model based on the updated training set comprises:
judging whether to retrain the named entity recognition model or not according to a preset updating strategy;
and if so, training the named entity recognition model based on the updated training set.
6. A method for annotating a sample, the method comprising:
acquiring a session text and sending the session text to a server;
receiving a named entity recognition result of the session text sent by the server;
displaying the named entity recognition result by using a preset style;
and if an adjusted recognition result aiming at the named entity recognition result input by the user is received, sending the adjusted recognition result to the server.
7. The method of claim 6, wherein the preset pattern comprises: underline, preset background color.
8. An apparatus for training a named entity recognition model, the apparatus comprising: the system comprises a text receiving module, a text recognition module, an adjustment recognition module and a model training module;
the text receiving module is used for receiving a session text sent by the terminal equipment;
the text recognition module is used for carrying out named entity recognition processing on the session text by using a named entity recognition model to obtain a named entity recognition result of the session text and sending the named entity recognition result to the terminal equipment;
the adjustment recognition module is used for updating the adjusted recognition result and the session text as new samples to a training set of the named entity recognition model to obtain an updated training set if the adjusted recognition result sent by the terminal equipment is received;
and the model training module is used for training the named entity recognition model based on the updated training set.
9. A sample annotation device, said device comprising: the system comprises a text sending module, a result receiving module, a result display module and a result adjusting module;
the text sending module is used for acquiring a session text and sending the session text to a server;
the result receiving module is used for receiving the named entity recognition result of the session text sent by the server;
the result display module is used for displaying the named entity recognition result by using a preset style;
and the result adjusting module is used for sending the adjusted recognition result to the server if the adjusted recognition result aiming at the named entity recognition result input by the user is received.
10. A server, comprising: a first memory in which a computer program is stored, the computer program being executable on the first processor, the first processor implementing the steps of the method of any of the preceding claims 1 to 5 when executing the computer program.
11. A terminal device, comprising: a second memory in which a computer program is stored, the computer program being executable on the second processor, and a second processor, the second processor implementing the steps of the method as claimed in claim 6 or 7 when executing the computer program.
12. A storage medium, characterized in that the storage medium has stored thereon a computer program which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011481841.0A CN112487817A (en) | 2020-12-14 | 2020-12-14 | Named entity recognition model training method, sample labeling method, device and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011481841.0A CN112487817A (en) | 2020-12-14 | 2020-12-14 | Named entity recognition model training method, sample labeling method, device and equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112487817A true CN112487817A (en) | 2021-03-12 |
Family
ID=74917257
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011481841.0A Pending CN112487817A (en) | 2020-12-14 | 2020-12-14 | Named entity recognition model training method, sample labeling method, device and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112487817A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113239196A (en) * | 2021-05-12 | 2021-08-10 | 同方知网数字出版技术股份有限公司 | Entity classification model training and predicting method based on digital humanity |
| CN114609925A (en) * | 2022-01-14 | 2022-06-10 | 中国科学院自动化研究所 | Training method of underwater exploration strategy model and underwater exploration method of bionic machine fish |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110276075A (en) * | 2019-06-21 | 2019-09-24 | 腾讯科技（深圳）有限公司 | Model training method, name entity recognition method, device, equipment and medium |
| CN110704633A (en) * | 2019-09-04 | 2020-01-17 | 平安科技（深圳）有限公司 | Named entity recognition method and device, computer equipment and storage medium |
| WO2020232861A1 (en) * | 2019-05-20 | 2020-11-26 | 平安科技（深圳）有限公司 | Named entity recognition method, electronic device and storage medium |
| CN112069302A (en) * | 2020-09-15 | 2020-12-11 | 腾讯科技（深圳）有限公司 | Training method of conversation intention recognition model, conversation intention recognition method and device |
-
2020
- 2020-12-14 CN CN202011481841.0A patent/CN112487817A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020232861A1 (en) * | 2019-05-20 | 2020-11-26 | 平安科技（深圳）有限公司 | Named entity recognition method, electronic device and storage medium |
| CN110276075A (en) * | 2019-06-21 | 2019-09-24 | 腾讯科技（深圳）有限公司 | Model training method, name entity recognition method, device, equipment and medium |
| CN110704633A (en) * | 2019-09-04 | 2020-01-17 | 平安科技（深圳）有限公司 | Named entity recognition method and device, computer equipment and storage medium |
| CN112069302A (en) * | 2020-09-15 | 2020-12-11 | 腾讯科技（深圳）有限公司 | Training method of conversation intention recognition model, conversation intention recognition method and device |
Non-Patent Citations (1)
| Title |
|---|
| 张春祥，高雪瑶著: "基于短语评价的翻译知识获取", vol. 978, 29 February 2012, 哈尔滨：哈尔滨工业大学出版社, pages: 93 - 96 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113239196A (en) * | 2021-05-12 | 2021-08-10 | 同方知网数字出版技术股份有限公司 | Entity classification model training and predicting method based on digital humanity |
| CN114609925A (en) * | 2022-01-14 | 2022-06-10 | 中国科学院自动化研究所 | Training method of underwater exploration strategy model and underwater exploration method of bionic machine fish |
| CN114609925B (en) * | 2022-01-14 | 2022-12-06 | 中国科学院自动化研究所 | Training method of underwater exploration strategy model and underwater exploration method of bionic machine fish |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107609092B (en) | Intelligent response method and device | |
| CN110597952A (en) | Information processing method, server, and computer storage medium | |
| CN105657129A (en) | Call information obtaining method and device | |
| CN111966796B (en) | Question and answer pair extraction method, device and equipment and readable storage medium | |
| CN114547274B (en) | Multi-turn question and answer method, device and equipment | |
| CN114757176A (en) | Method for obtaining target intention recognition model and intention recognition method | |
| CN110018823B (en) | Processing method and system, and generating method and system of interactive application program | |
| CN112487817A (en) | Named entity recognition model training method, sample labeling method, device and equipment | |
| CN109492221A (en) | Information reply method based on semantic analysis and wearable device | |
| CN110489747A (en) | A kind of image processing method, device, storage medium and electronic equipment | |
| CN111222837A (en) | Intelligent interviewing method, system, equipment and computer storage medium | |
| CN111737424A (en) | Question matching method, device, equipment and storage medium | |
| CN115238688B (en) | Method, device, equipment and storage medium for analyzing association relation of electronic information data | |
| CN114549241A (en) | Contract examination method, device, system and computer readable storage medium | |
| CN114969326A (en) | Classification model training and semantic classification method, device, equipment and medium | |
| CN113935331A (en) | Abnormal semantic truncation detection method, device, equipment and medium | |
| CN113918698A (en) | Customer service processing system and device | |
| CN111199208A (en) | Head portrait gender identification method and system based on deep learning framework | |
| RU2688758C1 (en) | Method and system for arranging dialogue with user in user-friendly channel | |
| CN110110777A (en) | Image processing method and training method and device, medium and calculating equipment | |
| CN109919657A (en) | User demand information acquisition method and device, storage medium and voice equipment | |
| CN109582971B (en) | Correction method and correction system based on syntactic analysis | |
| CN113505293A (en) | Information pushing method and device, electronic equipment and storage medium | |
| CN108304362B (en) | Clause detection method and device | |
| CN112183097B (en) | Entity recall method and related device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |