CN112560462B - Event extraction service generation method, device, server and medium - Google Patents
Event extraction service generation method, device, server and medium Download PDFInfo
- Publication number
- CN112560462B CN112560462B CN202011463891.6A CN202011463891A CN112560462B CN 112560462 B CN112560462 B CN 112560462B CN 202011463891 A CN202011463891 A CN 202011463891A CN 112560462 B CN112560462 B CN 112560462B
- Authority
- CN
- China
- Prior art keywords
- event
- annotation
- client
- control
- request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000605 extraction Methods 0.000 title claims abstract description 163
- 238000000034 method Methods 0.000 title claims abstract description 62
- 238000002372 labelling Methods 0.000 claims abstract description 107
- 238000012549 training Methods 0.000 claims abstract description 106
- 238000003860 storage Methods 0.000 claims description 16
- 230000004044 response Effects 0.000 claims description 15
- 238000012545 processing Methods 0.000 claims description 10
- 238000012795 verification Methods 0.000 claims description 6
- 230000001960 triggered effect Effects 0.000 claims description 4
- 238000013135 deep learning Methods 0.000 abstract description 3
- 238000003058 natural language processing Methods 0.000 abstract description 2
- 230000008569 process Effects 0.000 description 12
- 238000004590 computer program Methods 0.000 description 11
- 238000010586 diagram Methods 0.000 description 10
- 238000004891 communication Methods 0.000 description 8
- 230000008093 supporting effect Effects 0.000 description 7
- 238000011161 development Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000003993 interaction Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The application discloses a method, a device, a server and a medium for generating event extraction service, which relate to the fields of natural language processing, cloud computing, deep learning and knowledge graph, and specifically comprise the following steps: receiving a transmitted labeling request, and labeling the training sample according to event labeling content in the labeling request; training the event extraction model by adopting the marked training sample, and distributing an event extraction service port for calling the event extraction model; and sending the event extraction service port to the client. Therefore, after the labeling request sent by the client is received, the training sample is labeled according to the event labeling content in the labeling request, so that the user labels the training sample at the client according to the personalized demand, and personalized event extraction service is obtained.
Description
Technical Field
The application discloses a generation method, a device, a server and a storage medium of event extraction service, relates to the technical field of deep learning, and particularly relates to the technical field of natural language processing, cloud computing and knowledge graph.
Background
In recent years, with the rise of digitization in various fields, various industries store a large number of files on a network. Most of the massive data are structured or semi-structured data, and a user is difficult to obtain information needed by the user, so that the structured information needs to be extracted from the massive data.
Event extraction refers to identifying a specific type of event and determining and extracting relevant information. The requirements of different practitioners in different industries on the types and attributes of the extracted events are different, the types of texts or information to be extracted are also different, and the general extraction service cannot meet the event extraction requirements of all industries. Therefore, there is an urgent need for a customizable set of event extraction services to meet the needs of different industries for event extraction.
Disclosure of Invention
The application provides a method, a device, a server and a storage medium for generating event extraction services.
According to an aspect of the present application, there is provided a method for generating an event extraction service, including:
receiving a labeling request sent by a client;
marking the training sample according to event marking content in the marking request;
Training an event extraction model by adopting the marked training sample, and distributing an event extraction service port for calling the event extraction model;
and sending the event extraction service port to the client.
As a possible implementation manner of an aspect of the present application, before the receiving the labeling request sent by the client, the method further includes:
receiving a configuration request sent by the client; configuring a labeling control corresponding to each event labeling content according to a plurality of event labeling contents indicated by the configuration request;
and sending each annotation control to the client so that the client displays each annotation control on a sample annotation page, and generating the corresponding annotation request in response to control triggering operation of each annotation control.
As another possible implementation manner of an aspect of the present application, the plurality of event labeling contents include at least one event type and at least one event attribute contained in each of the event types;
the configuring the annotation control corresponding to each event annotation content according to the plurality of event annotation contents indicated by the configuration request comprises the following steps:
Configuring corresponding first annotation controls according to the at least one event type;
and configuring a second annotation control associated with each first annotation control according to at least one event attribute contained in each event type.
As another possible implementation manner of an aspect of the present application, the labeling request is used to instruct the first labeling control and the second labeling control triggered by the control triggering operation, and the labeled training sample;
labeling the training sample according to the event labeling content in the labeling request comprises the following steps:
verifying the association relation between the first annotation control indicated by the annotation request and the second annotation control indicated by the annotation request;
and under the condition that the association relation exists through verification, marking the training sample indicated by the marking request by adopting the event type corresponding to the first marking control indicated by the marking request and the event attribute corresponding to the second marking control indicated by the marking request.
As another possible implementation manner of an aspect of the present application, the method further includes:
transmitting a data set to the client;
or receiving the data set uploaded by the client;
Wherein the data set comprises a plurality of training samples which are not marked by the client.
As another possible implementation manner of an aspect of the present application, each training sample included in the data set has corresponding history labeling information.
According to another aspect of the present application, there is provided a generation apparatus of an event extraction service, including:
the receiving module is used for receiving the labeling request sent by the client;
the marking module is used for marking the training samples according to event marking contents in the marking request;
the training module is used for training the event extraction model by adopting the marked training sample and distributing an event extraction service port for calling the event extraction model;
and the port sending module is used for sending the event extraction service port to the client.
According to another aspect of the present application, there is provided a server including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of generating an event extraction service as set forth in the above embodiments.
According to another aspect of the present application, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the generation method of the event extraction service described in the above embodiment.
According to another aspect of the present application, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the method of generating an event extraction service described in the above embodiments.
It should be understood that the description of this section is not intended to identify key or critical features of the embodiments of the application or to delineate the scope of the application. Other features of the present application will become apparent from the description that follows.
Drawings
The drawings are for better understanding of the present solution and do not constitute a limitation of the present application. Wherein:
fig. 1 is a flow chart of a method for generating an event extraction service according to an embodiment of the present application;
fig. 2 is a flow chart of another method for generating an event extraction service according to an embodiment of the present application;
fig. 3 is a flowchart of a method for generating an event extraction service according to another embodiment of the present application;
FIG. 4 provides a flow diagram of a full flow development of an event extraction service;
FIG. 5 is a block diagram illustrating a development system of an event extraction service according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of a generating device of an event extraction service according to an embodiment of the present application;
fig. 7 is a block diagram of a server for implementing a method of generating an event extraction service according to an embodiment of the present application.
Detailed Description
Exemplary embodiments of the present application are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present application to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
The following describes a method, an apparatus, a device, and a storage medium for generating an event extraction service according to embodiments of the present application with reference to the accompanying drawings.
Fig. 1 is a flow chart of a method for generating an event extraction service according to an embodiment of the present application.
The embodiment of the application is exemplified by the fact that the generating method of the event extraction service is configured in the generating device of the event extraction service, and the generating device of the event extraction service can be applied to any type of server, so that the server can execute the generating function of the event extraction service.
The server may be a cloud server or other types of servers.
As shown in fig. 1, the method for generating the event extraction service may include the following steps:
and step 101, receiving a labeling request sent by a client.
The labeling request can be sent by the client in response to a triggering operation of the user. The triggering operation of the user may be clicking, sliding, touching, or the like of the user, which is not limited herein.
In the embodiment of the application, the sample annotation page of the client can display the annotation control, and the user can operate the annotation control displayed on the sample annotation page by the client to generate the annotation request. And the client responds to the user triggering operation to send a labeling request to the server, and the server receives the labeling request sent by the client.
And 102, marking the training sample according to event marking content in the marking request.
Event annotation refers to the annotated content of an event, e.g., event type, event trigger word, event attribute, etc.
The event trigger word refers to a word that most represents occurrence of an event in an event, and is an important feature for determining the type of the event, and is generally a verb or noun.
ACE2015 defines 8 event types and 33 seed event types, each with its own set of potential participant roles.
It can be understood that the training samples are stored in the server, and after the labeling request sent by the client is received, the training samples indicated by the labeling request can be labeled according to event labeling content in the labeling request.
It should be noted that, the training samples in the present application may be unlabeled raw data, and may also be data extracted for labeled events. The marked training samples are marked by adopting event marking content indicated by marking requests, so that the marking quality of the marked training samples can be optimized, and the condition of marking errors is avoided.
And step 103, training the event extraction model by adopting the marked training sample, and distributing an event extraction service port for calling the event extraction model.
The event extraction refers to a technology of extracting events of interest to a user from text data describing event information and identifying event types and event attributes. And the event extraction model is used for extracting the events which are interested by the user from the text data describing the event information and presenting the events in a structured form.
The event extraction model in the application has stronger generalization capability, and can meet the event extraction requirements of different scenes after training according to training samples of different scenes.
In the embodiment of the application, after the training sample is marked, the marked training sample can be used for training the event extraction model.
As a possible implementation manner, vectorization processing can be performed on the labeled training samples, feature extraction is performed on the vectorized training samples, and then the event extraction model is trained through a deep learning algorithm based on the extracted features and labeled labels.
It should be noted that the training method for the event extraction model is described as an example, which is not limited in this application.
In the embodiment of the application, the marked training sample is adopted to train the event extraction model to produce the model capable of carrying out event extraction according to event configuration. Further, the server may allocate an event extraction service port for invoking the event extraction model for the user to use the event extraction service in the actual production process.
The service port may be a logical port for differentiating services in a logical sense. Such as a service port in the TCP/IP protocol, the port number ranges from 0 to 65535, such as 80 ports for browsing web services, 21 ports for FTP services, etc.
And 104, sending an event extraction service port to the client.
In this embodiment of the present application, after the server allocates an event extraction service port for invoking the event extraction model, the server sends the event extraction service port to the client, so that the user may use the event extraction model after inputting the event extraction service port into the client in actual use
According to the method for generating the event extraction service, after the labeling request sent by the client is received, the training samples are labeled according to the event labeling content in the labeling request, so that a user can label the training samples at the client according to personalized requirements, personalized event extraction service is obtained, and the deployment process of the event extraction service is simplified.
On the basis of the above embodiment, before the client sends the annotation request to the server, the annotation request needs to be generated first, and the detailed description will be given below with reference to fig. 2, where fig. 2 is a flow chart of another method for generating the event extraction service according to the embodiment of the present application.
As shown in fig. 2, the method for generating the event extraction service may include the following steps:
step 201, a configuration request sent by a client is received.
The client is used for creating the project for the event extraction service according to the requirement for the event extraction scene by the user. The client may include a configuration center through which a user may configure settings and configuration management of events. For example, the user can add, delete, modify, etc. event annotation content meeting different event extraction scenarios at the client. Wherein an event, a manifestation of information, is defined as the objective fact that a specific person, thing, interact at a specific time and a specific place, generally at sentence level.
In this embodiment of the present application, after a user logs in registration information at a client, the user may configure event annotation content at the client according to its own event extraction scenario requirement, so as to generate a configuration request for configuring multiple event annotation contents at the client. Further, the client transmits a configuration request to the server, so that the server receives the configuration request transmitted by the client in response to the user configuration operation.
Step 202, configuring label controls corresponding to the event label contents according to the plurality of event label contents indicated by the configuration request.
The annotation control is generated by the server according to a configuration request sent by the client and is used for indicating the client to display the control for configuring event annotation content on the sample annotation page. The sample labeling page is a page displayed on the client side and used for labeling event labeling contents by a user.
In this embodiment of the present application, after receiving a configuration request sent by a client in response to a user configuration operation and used for configuring a plurality of event annotation contents, a server may configure an annotation control corresponding to each event annotation content according to the plurality of event annotation contents indicated by the configuration request.
And 203, sending each labeling control to the client so that the client displays each labeling control on the sample labeling page, and generating a corresponding labeling request in response to control triggering operation of each labeling control.
In the embodiment of the application, after the server completes configuration of the labeling control corresponding to each event labeling content, each labeling control can be sent to the client. After receiving the labeling controls sent by the server, the client can display the labeling controls on a sample labeling page of the client.
In the embodiment of the application, after the sample annotation page of the client displays each annotation control, the user can perform control triggering operation on the sample annotation page of the client according to actual extraction requirements, so that the client responds to the control triggering operation of the user on each annotation control to generate a corresponding annotation request.
The triggering operation of the control may be clicking, sliding, touching, etc. of the user, which is not limited herein.
And step 204, receiving a labeling request sent by the client.
And 205, labeling the training sample according to event labeling content in the labeling request.
And 206, training the event extraction model by using the labeled training sample, and distributing an event extraction service port for calling the event extraction model.
Step 207, send event extraction service port to client.
It should be noted that, the implementation process of the steps 204 to 207 may be referred to the implementation process of the steps 101 to 104 in the above embodiment, and will not be described herein.
According to the method for generating the event extraction service, before a server receives a labeling request sent by a client, a configuration request sent by the client is received, labeling controls corresponding to all event labeling contents are configured according to a plurality of event labeling contents indicated by the configuration request, all the labeling controls are sent to the client, so that the client displays all the labeling controls on a sample labeling page, and corresponding labeling requests are generated in response to control triggering operations of all the labeling controls. Therefore, the corresponding annotation request is generated according to the event annotation content in different extraction scenes, and the aim of personalized customization of event extraction service is fulfilled.
In an actual application scene, the event annotation content can comprise at least one event type and at least one event attribute contained in each event type, and when the annotation control corresponding to each event annotation content is configured, the annotation control corresponding to the at least one event type and the at least one event attribute can be configured, so that event extraction service output under different extraction scenes is satisfied. Fig. 3 is a flowchart of a method for generating an event extraction service according to another embodiment of the present application.
As shown in fig. 3, the method for generating the event extraction service may include the following steps:
step 301, a configuration request sent by a client is received.
It should be noted that, the implementation process of step 301 may refer to the implementation process of step 201 in the above embodiment, which is not described herein.
In one possible scenario, the server may send the data set to the client after receiving the configuration request sent by the client. The data set comprises a plurality of training samples which are not marked by the client, and each training sample has corresponding historical marking information.
It can be understood that the server side stores training samples which are not marked by the client side, and a plurality of training samples which are not marked by the client side can be sent to the client side, so that a user optimizes marking quality of the training samples with history marking information at the client side.
In another possible scenario, the server may also receive the data set uploaded by the client after receiving a configuration request sent by the client in response to a user configuration operation. The data set comprises a plurality of training samples which are not marked by the client, and each training sample has corresponding historical marking information.
It can be appreciated that the server may receive training samples sent by the client and including a plurality of non-client-side labeled training samples, so as to label the plurality of non-client-side labeled training samples according to actual scene requirements.
Therefore, the training samples with the history labeling information contained in the data set are remarked, so that the labeling quality of the labeled training samples is optimized, the event extraction model is trained by adopting the remarked training samples, and the accuracy of the event extraction model is improved.
In the present application, the data set may be acquired through a capturing manner on a website, or may be acquired through a capturing manner in an event database, or may be acquired in an event map, or may be acquired in other manners according to an actual application scenario, which is not limited herein. The event map is a heterogeneous map formed by a plurality of events, and comprises both events and attribute information of an entity.
Step 302, configuring corresponding first annotation controls according to at least one event type.
In the embodiment of the application, the event annotation content can comprise at least one event type and at least one event attribute contained in each event type.
It is understood that an event type has a corresponding at least one event attribute. For example, assuming that the event type is wedding, the corresponding event attributes may include time, place, and so forth.
In this embodiment of the present application, after receiving a configuration request sent by a client in response to a user configuration operation, the server may configure each first annotation control corresponding to at least one event type according to at least one event type.
Step 303, configuring a second annotation control associated with each first annotation control according to at least one event attribute contained in each event type.
In this embodiment of the present application, after receiving a configuration request sent by a client in response to a user configuration operation, the second annotation control associated with each first annotation control may be configured according to at least one event attribute included in each event type.
Therefore, according to at least one event type and at least one event attribute contained in each event type, each corresponding first annotation control and each second annotation control associated with each first annotation control are configured, so that the client side displays the first annotation control and the second annotation control on the sample annotation page, and a user can trigger the first annotation control and the second annotation control displayed on the client side to generate an annotation request.
And step 304, sending each annotation control to the client so that the client displays each annotation control on the sample annotation page, and generating a corresponding annotation request in response to control triggering operation of each annotation control.
In the embodiment of the application, the server configures each corresponding first annotation control and each corresponding second annotation control associated with each first annotation control according to at least one event type and at least one event attribute contained in each event type, and then sends each first annotation control and each second annotation control associated with each first annotation control to the client.
After receiving the first annotation controls and the second annotation controls associated with the first annotation controls, the client can display the first annotation controls and the second annotation controls on the sample annotation page.
The user can operate each first annotation control and each second annotation control displayed on the sample annotation page of the client, so that the client responds to control triggering operation of each annotation control by the user to generate a corresponding annotation request, and the annotation request is sent to the server.
Step 305, verifying the association relationship between the first annotation control indicated by the annotation request and the second annotation control indicated by the annotation request.
In this embodiment of the present application, after receiving a labeling request sent by a client in response to a control triggering operation, before labeling a training sample indicated by the labeling request, in order to avoid a false triggering operation of a user on the control by the client, the server may further verify an association relationship between a first labeling control indicated by the labeling request and a second labeling control indicated by the labeling request, so as to determine whether an association relationship exists between the first labeling control indicated by the labeling request and the second labeling control indicated by the labeling request.
Under a possible condition, if it is determined that the event attribute of the second annotation control indicated by the annotation request does not conform to the event type of the first annotation control, if the event type of the first annotation control is wedding and the event attribute of the second annotation control is seismic grade, it can be determined that no association exists between the first annotation control indicated by the annotation request and the second annotation control indicated by the annotation request.
In another possible case, if it is determined that the event attribute of the second annotation control indicated by the annotation request accords with the event type of the first annotation control, if the event type of the first annotation control is wedding and the event attribute of the second annotation control is time, it may be determined that an association relationship exists between the first annotation control indicated by the annotation request and the second annotation control indicated by the annotation request.
And 306, marking a training sample indicated by the marking request by adopting an event type corresponding to a first marking control indicated by the marking request and an event attribute corresponding to a second marking control indicated by the marking request under the condition that the association relation is checked and determined.
In the embodiment of the application, the association relationship between the first annotation control indicated by the annotation request and the second annotation control indicated by the annotation request is checked, and the association relationship between the first annotation control indicated by the annotation request and the second annotation control is determined. Further, the training samples indicated by the annotation request are annotated by adopting event types corresponding to the first annotation control indicated by the annotation request and event attributes corresponding to the second annotation control indicated by the annotation request.
Step 307, training the event extraction model by using the labeled training sample, and allocating an event extraction service port for calling the event extraction model.
Step 308, send event extraction service port to client.
It should be noted that, the implementation process of step 307 and step 308 may be referred to the implementation process of steps 103 and 104 in the above embodiment, which is not described herein.
According to the method for generating the event extraction service, after receiving a configuration request sent by a client for configuring a plurality of event annotation contents, indication information of each first annotation control is configured according to at least one event type, and second annotation controls associated with each first annotation control are configured according to at least one event attribute contained in each event type, so that the client displays the first annotation control and the second annotation control on a sample annotation page, receives the annotation request sent by the client, verifies the first annotation control indicated by the annotation request, and annotates a training sample indicated by the annotation request when an association relationship exists between the first annotation control indicated by the annotation request and the second annotation control indicated by the annotation request. Therefore, the situation that the marked event attribute is irrelevant to the event type when the training sample is marked is avoided, and the marked training sample is adopted to train the event extraction model, so that the accuracy of the model is improved.
As an example, as shown in fig. 4, fig. 4 provides a flow diagram of a full flow development of an event extraction service.
As shown in fig. 4, the method for generating the event extraction service may include the steps of:
Step 401, receiving a labeling request sent by a client.
And the client responds to the user triggering operation to send a labeling request to the server, and the server receives the labeling request sent by the client.
In the embodiment of the application, the user can formulate the event configuration according to the actual event extraction requirement. For example, event types to be extracted and event attributes contained in the event types in the actual application scene can be set. The number of event types and event attributes is not limited herein.
In step 402, the user uploads the data set meeting the target scene to the server according to the actual event extraction requirement, so that the server receives the data set under the target scene.
The data set includes a plurality of training samples, and each training sample may be labeled or unlabeled original data, which is not limited herein.
Step 403, judging whether to label the training sample according to the actual event extraction requirement.
In step 404, labeling the training samples is determined.
In one possible case, if it is determined that each training sample included in the data set is not labeled at the client, the training sample needs to be labeled.
As a possible implementation manner, a user can configure a plurality of event annotation contents through a client, after a configuration request is generated, the client sends the configuration request to a server, and the server sends indication information of a corresponding annotation control to the client according to the configured plurality of event annotation contents, so that the client displays the annotation control on a sample annotation page according to the received indication information of the annotation control. The server receives a labeling request sent by the client in response to the control triggering operation, and labels the training sample indicated by the labeling request by adopting event labeling content corresponding to the labeling control indicated by the labeling request.
Step 405, training the event extraction model by using the training sample without labeling the training sample.
In one possible scenario, each training sample included in the dataset is a training sample labeled by the client, and at this time, the event extraction model can be trained using the training sample without labeling each training sample.
After training the event extraction model by using the training sample, the server can allocate an event extraction service port for calling the event extraction model and send the event extraction service port to the client, so that the user can call the extraction service port at the client according to the requirement to perform event extraction by using the trained event extraction model.
In the application, the generated event extraction service may also perform iteration of the event extraction service, where an iteration process of the event extraction service refers to iteration that a user may perform event configuration according to own needs, iteration of a data set, iteration of an event extraction model, event extraction service management, and so on. The steps in the iteration process of the event extraction service can be performed out of order, and the iteration process can be determined according to requirements. The iteration refers to adjusting the event extraction service to adapt to the continuously changing requirements.
As an example, the present application also proposes a system for developing an event extraction service.
Fig. 5 is a block diagram of a development system of an event extraction service according to an embodiment of the present application.
As shown in fig. 5, the structural block diagram of the development system of the event extraction service may include: configuration center, data center, training center and service center.
The configuration center supports the user to perform configuration setting and configuration management. Supporting a user to set event types needing to be extracted and attributes contained in each event type under a target application scene; and operations such as adding, deleting, modifying, inquiring and the like of different version configurations by a user are supported.
The data center supports users to upload data, produce labeling data and manage data. Supporting a user to upload a data set corresponding to a target scene, wherein the data set can be a marked event extraction data set or unmarked original data; supporting a user to directly perform data annotation work on a client, performing original data annotation or annotation quality optimization of existing annotation data, and finally generating event annotation data for training an event extraction model; the support user manages all uploaded data sets.
The training center supports event extraction model training and management. After a user selects a specific marked data set and corresponding event configuration, training an event extraction model in a target scene, and embedding the trained event extraction model in the general field to support training the event extraction model under a small number of marked samples; the user is supported to manage all the event extraction models which are completed or are being trained, such as deleting operation and checking configuration information corresponding to the models, and the event extraction models which are completed to be trained can be controlled to produce actual event extraction services.
The service center supports verification, online trial, service output and service management of event extraction services. Supporting effect verification and comparison of a plurality of event extraction services; supporting online trial of existing event extraction services; supporting a user to apply for formal event extraction service and outputting an event extraction service port; and supporting event extraction services of the formal output of the user off line.
In order to implement the above embodiment, the present application proposes a generation apparatus of an event extraction service.
Fig. 6 is a schematic structural diagram of a generating device of an event extraction service according to an embodiment of the present application.
As shown in fig. 6, the generating device 600 of the event extraction service may include: a receiving module 610, a labeling module 620, a training module 630, and a port transmitting module 640.
The receiving module 610 is configured to receive an annotation request sent by a client.
The labeling module 620 is configured to label the training sample according to the event labeling content in the labeling request.
The training module 630 is configured to train the event extraction model by using the labeled training sample, and allocate an event extraction service port for calling the event extraction model.
The port sending module 640 is configured to send the event extraction service port to the client.
As a possible case, the generating device 600 of the event extraction service may further include: the control sending module is used for receiving a configuration request sent by the client; configuring annotation controls corresponding to the event annotation contents according to the event annotation contents indicated by the configuration request; and sending each annotation control to the client so that the client displays each annotation control on the sample annotation page, and generating a corresponding annotation request in response to control triggering operation of each annotation control.
As another possible scenario, the plurality of event annotation content comprises at least one event type and at least one event attribute contained by each event type; the control sending module may be further specifically configured to: configuring corresponding first annotation controls according to at least one event type; and configuring a second annotation control associated with each first annotation control according to at least one event attribute contained in each event type.
As another possible case, a labeling request is used for indicating a first labeling control and a second labeling control triggered by a control triggering operation, and a labeled training sample; the labeling module 630 may be further specifically configured to: verifying the association relationship between a first annotation control indicated by the annotation request and a second annotation control indicated by the annotation request; and under the condition that the association relation exists through verification, marking the training sample indicated by the marking request by adopting the event type corresponding to the first marking control indicated by the marking request and the event attribute corresponding to the second marking control indicated by the marking request.
As another possible case, the generating device 600 of the event extraction service may further include:
The processing module is used for sending the data set to the client;
or receiving the data set uploaded by the client;
wherein the data set comprises a plurality of training samples which are not marked by clients.
As another possibility, each training sample contained within the dataset has corresponding historical annotation information.
According to the event extraction service generation device, after the annotation request sent by the client is received, the training samples are annotated according to the event annotation content in the annotation request, so that a user can annotate the training samples at the client according to personalized requirements, personalized event extraction services are obtained, and the deployment process of the event extraction services is simplified.
In order to implement the foregoing embodiment, the present application further proposes a server, which may include: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of generating an event extraction service in the above embodiments.
To achieve the above embodiments, the present application also proposes a non-transitory computer-readable storage medium storing computer instructions.
A non-transitory computer-readable storage medium storing computer instructions according to an embodiment of the present application, where the computer instructions are configured to cause the computer to execute the method for generating an event extraction service described in the foregoing embodiment.
In order to implement the above embodiments, the present application also proposes a computer program product comprising a computer program which, when executed by a processor, implements the method for generating an event extraction service described in the above embodiments.
According to embodiments of the present application, a server and a readable storage medium are also provided.
As shown in fig. 7, a block diagram of a server for implementing a method for generating an event extraction service according to an embodiment of the present application is shown. Servers are intended to represent various forms of digital computers, such as laptops, desktops, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The server may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the application described and/or claimed herein.
As shown in fig. 7, the apparatus 700 includes a computing unit 701 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 702 or a computer program loaded from a storage unit 708 into a Random Access Memory (RAM) 703. In the RAM703, various programs and data required for the operation of the device 700 may also be stored. The computing unit 701, the ROM702, and the RAM703 are connected to each other through a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.
Various components in device 700 are connected to I/O interface 705, including: an input unit 706 such as a keyboard, a mouse, etc.; an output unit 707 such as various types of displays, speakers, and the like; a storage unit 708 such as a magnetic disk, an optical disk, or the like; and a communication unit 709 such as a network card, modem, wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.
The computing unit 701 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 701 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above. Such as a method of generating an event extraction service, in some embodiments, the applet packet processing may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as the storage unit 708. In some embodiments, part or all of the computer program may be loaded and/or installed onto device 700 via ROM702 and/or communication unit 709. When the computer program is loaded into the RAM703 and executed by the computing unit 701, one or more steps of the above-described generation method of the event extraction service may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the method of generating the event extraction service in any other suitable manner (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for carrying out methods of the present application may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this application, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), the internet, and blockchain networks.
The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service ("Virtual Private Server" or simply "VPS") are overcome. The server may also be a server of a distributed system or a server that incorporates a blockchain.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present application may be performed in parallel, sequentially, or in a different order, provided that the desired results of the technical solutions disclosed in the present application can be achieved, and are not limited herein.
The above embodiments do not limit the scope of the application. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application are intended to be included within the scope of the present application.
Claims (12)
1. A method of generating an event extraction service, comprising:
receiving a labeling request sent by a client;
marking the training sample according to event marking content in the marking request;
training an event extraction model by adopting the marked training sample, and distributing an event extraction service port for calling the event extraction model;
sending the event extraction service port to the client;
Before receiving the labeling request sent by the client, the method further comprises the following steps:
receiving a configuration request sent by the client;
configuring a labeling control corresponding to each event labeling content according to a plurality of event labeling contents indicated by the configuration request;
and sending each annotation control to the client so that the client displays each annotation control on a sample annotation page, and generating the corresponding annotation request in response to control triggering operation of each annotation control.
2. The generating method according to claim 1, wherein the plurality of event annotation contents include at least one event type and at least one event attribute contained in each of the event types;
the configuring the annotation control corresponding to each event annotation content according to the plurality of event annotation contents indicated by the configuration request comprises the following steps:
configuring corresponding first annotation controls according to the at least one event type;
and configuring a second annotation control associated with each first annotation control according to at least one event attribute contained in each event type.
3. The generating method according to claim 2, wherein the annotation request is used for indicating a first annotation control and a second annotation control triggered by a control triggering operation, and an annotated training sample;
Labeling the training sample according to the event labeling content in the labeling request comprises the following steps:
verifying the association relation between the first annotation control indicated by the annotation request and the second annotation control indicated by the annotation request;
and under the condition that the association relation exists through verification, marking the training sample indicated by the marking request by adopting the event type corresponding to the first marking control indicated by the marking request and the event attribute corresponding to the second marking control indicated by the marking request.
4. A method of generating according to any one of claims 1-3, wherein the method further comprises:
transmitting a data set to the client;
or receiving the data set uploaded by the client;
wherein the data set comprises a plurality of training samples which are not marked by the client.
5. The generation method according to claim 4, wherein each of the training samples contained in the dataset has corresponding history labeling information.
6. An event extraction service generation apparatus, comprising:
the receiving module is used for receiving the labeling request sent by the client;
the marking module is used for marking the training samples according to event marking contents in the marking request;
The training module is used for training the event extraction model by adopting the marked training sample and distributing an event extraction service port for calling the event extraction model;
the port sending module is used for sending the event extraction service port to the client;
the apparatus further comprises:
the control sending module is used for receiving the configuration request sent by the client; configuring a labeling control corresponding to each event labeling content according to a plurality of event labeling contents indicated by the configuration request; and sending each annotation control to the client so that the client displays each annotation control on a sample annotation page, and generating the corresponding annotation request in response to control triggering operation of each annotation control.
7. The generating device of claim 6, wherein the plurality of event annotation content comprises at least one event type and at least one event attribute contained by each of the event types; the control sending module is specifically configured to:
configuring corresponding first annotation controls according to the at least one event type;
and configuring a second annotation control associated with each first annotation control according to at least one event attribute contained in each event type.
8. The generating device of claim 7, wherein the annotation request is used for indicating a first annotation control and a second annotation control triggered by a control triggering operation, and an annotated training sample; the labeling module is specifically configured to:
verifying the association relation between the first annotation control indicated by the annotation request and the second annotation control indicated by the annotation request;
and under the condition that the association relation exists through verification, marking the training sample indicated by the marking request by adopting the event type corresponding to the first marking control indicated by the marking request and the event attribute corresponding to the second marking control indicated by the marking request.
9. The generating device according to any of claims 6-8, wherein the device further comprises:
the processing module is used for sending a data set to the client;
or receiving the data set uploaded by the client;
wherein the data set comprises a plurality of training samples which are not marked by the client.
10. The generating device of claim 9, wherein each of the training samples contained within the dataset has corresponding historical labeling information.
11. A server, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of generating an event extraction service of any one of claims 1-5.
12. A non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the method of generating the event extraction service of any of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011463891.6A CN112560462B (en) | 2020-12-11 | 2020-12-11 | Event extraction service generation method, device, server and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011463891.6A CN112560462B (en) | 2020-12-11 | 2020-12-11 | Event extraction service generation method, device, server and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112560462A CN112560462A (en) | 2021-03-26 |
| CN112560462B true CN112560462B (en) | 2023-08-01 |
Family
ID=75063116
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011463891.6A Active CN112560462B (en) | 2020-12-11 | 2020-12-11 | Event extraction service generation method, device, server and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112560462B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109460795A (en) * | 2018-12-17 | 2019-03-12 | 北京三快在线科技有限公司 | Classifier training method, apparatus, electronic equipment and computer-readable medium |
| WO2019137444A1 (en) * | 2018-01-12 | 2019-07-18 | 第四范式（北京）技术有限公司 | Method and system for executing feature engineering for use in machine learning |
| CN110717017A (en) * | 2019-10-17 | 2020-01-21 | 腾讯科技（深圳）有限公司 | Method for processing corpus |
| CN111159415A (en) * | 2020-04-02 | 2020-05-15 | 成都数联铭品科技有限公司 | Sequence labeling method and system, and event element extraction method and system |
| CN111414482A (en) * | 2020-03-20 | 2020-07-14 | 北京百度网讯科技有限公司 | Event argument extraction method and device and electronic equipment |
| CN111966890A (en) * | 2020-06-30 | 2020-11-20 | 北京百度网讯科技有限公司 | Text-based event pushing method and device, electronic equipment and storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10742498B2 (en) * | 2016-06-22 | 2020-08-11 | Amazon Technologies, Inc. | Application migration system |
| US20200134103A1 (en) * | 2018-10-26 | 2020-04-30 | Ca, Inc. | Visualization-dashboard narration using text summarization |
-
2020
- 2020-12-11 CN CN202011463891.6A patent/CN112560462B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019137444A1 (en) * | 2018-01-12 | 2019-07-18 | 第四范式（北京）技术有限公司 | Method and system for executing feature engineering for use in machine learning |
| CN109460795A (en) * | 2018-12-17 | 2019-03-12 | 北京三快在线科技有限公司 | Classifier training method, apparatus, electronic equipment and computer-readable medium |
| CN110717017A (en) * | 2019-10-17 | 2020-01-21 | 腾讯科技（深圳）有限公司 | Method for processing corpus |
| CN111414482A (en) * | 2020-03-20 | 2020-07-14 | 北京百度网讯科技有限公司 | Event argument extraction method and device and electronic equipment |
| CN111159415A (en) * | 2020-04-02 | 2020-05-15 | 成都数联铭品科技有限公司 | Sequence labeling method and system, and event element extraction method and system |
| CN111966890A (en) * | 2020-06-30 | 2020-11-20 | 北京百度网讯科技有限公司 | Text-based event pushing method and device, electronic equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| 基于条件随机场的专利术语抽取;刘辉;刘耀;;数字图书馆论坛(第12期);全文 * |
| 开放信息抽取技术的现状研究;刘振;张智雄;;情报杂志(第11期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112560462A (en) | 2021-03-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9015657B2 (en) | Systems and methods for developing and delivering platform adaptive web and native application content | |
| WO2020108063A1 (en) | Feature word determining method, apparatus, and server | |
| JP2021534493A (en) | Techniques for building knowledge graphs within a limited knowledge domain | |
| WO2021023149A1 (en) | Method and apparatus for dynamically returning message | |
| KR20210090576A (en) | A method, an apparatus, an electronic device, a storage medium and a program for controlling quality | |
| CN112527281B (en) | Operator upgrading method and device based on artificial intelligence, electronic equipment and medium | |
| CN113568626B (en) | Dynamic packaging and application package opening method and device and electronic equipment | |
| US9824403B2 (en) | Measuring problems from social media discussions | |
| CN113626223A (en) | Interface calling method and device | |
| JP7309811B2 (en) | Data annotation method, apparatus, electronics and storage medium | |
| CN116069319A (en) | Form rendering method and device, electronic equipment and storage medium | |
| CN109840072B (en) | Information processing method and device | |
| CN117608738A (en) | Browser interaction method, device, equipment, readable storage medium and product | |
| CN112560462B (en) | Event extraction service generation method, device, server and medium | |
| CN113138760B (en) | Page generation method and device, electronic equipment and medium | |
| CN115687717A (en) | Method, device and equipment for acquiring hook expression and computer readable storage medium | |
| CN113360672B (en) | Method, apparatus, device, medium and product for generating knowledge graph | |
| CN113590447B (en) | Buried point processing method and device | |
| CN114281964A (en) | Method and device for determining conversation skill service, electronic equipment and storage medium | |
| CN113362111A (en) | Content sending method and device and electronic equipment | |
| CN113239296B (en) | Method, device, equipment and medium for displaying small program | |
| CN117874211B (en) | Intelligent question-answering method, system, medium and electronic equipment based on SAAS software | |
| CN114492456B (en) | Text generation method, model training method, device, electronic equipment and medium | |
| CN113408300B (en) | Model training method, brand word recognition device and electronic equipment | |
| CN117591145A (en) | Updating method and device of interface document, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |