CN110378378B - Event retrieval method and device, computer equipment and storage medium - Google Patents
Event retrieval method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN110378378B CN110378378B CN201910520888.4A CN201910520888A CN110378378B CN 110378378 B CN110378378 B CN 110378378B CN 201910520888 A CN201910520888 A CN 201910520888A CN 110378378 B CN110378378 B CN 110378378B
- Authority
- CN
- China
- Prior art keywords
- event
- vector
- description
- recognition model
- descriptions
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses an event retrieval method, an event retrieval device, computer equipment and a storage medium, wherein the method comprises the following steps: training to obtain an identification model for identifying whether two event descriptions correspond to the same event or not; forming an event vector extraction service based on the recognition model, the event vector representing an understanding of the recognition model for the event description; according to the event vector extraction service, respectively acquiring an event vector corresponding to the event description to be retrieved and an event vector corresponding to the event description of the event in the event library; and determining an event matched with the event description to be retrieved in the event library according to the acquired event vector. By applying the scheme of the invention, the event retrieval based on the event description can be realized, the accuracy of the retrieval result is improved, and the like.
Description
[ technical field ] A method for producing a semiconductor device
The present invention relates to computer application technologies, and in particular, to an event retrieval method and apparatus, a computer device, and a storage medium.
[ background of the invention ]
In some scenarios, it may be desirable to perform event retrieval, such as retrieving corresponding events (same events) from an event library based on an event description provided by a user or appearing in a text, for convenience of text understanding or information recommendation, etc.
An event is a special entity which exists objectively, but is different from a common entity, more complex and more diverse in description, and the description of the event of two identical events can be greatly different, while the description of the event of two different events can be very similar.
In view of the above problems, there is no better implementation method for retrieving events according to event descriptions.
[ summary of the invention ]
In view of the above, the invention provides an event retrieval method, an event retrieval device, a computer device and a storage medium.
The specific technical scheme is as follows:
an event retrieval method, comprising:
training to obtain an identification model for identifying whether the two event descriptions correspond to the same event;
forming an event vector extraction service based on the recognition model, the event vector representing an understanding of the recognition model for an event description;
according to the event vector extraction service, respectively acquiring an event vector corresponding to the event description to be retrieved and an event vector corresponding to the event description of the event in the event library;
and determining an event matched with the event description to be retrieved in the event library according to the acquired event vector.
According to a preferred embodiment of the present invention, the training to obtain a recognition model for recognizing whether two event descriptions correspond to the same event includes:
constructing a positive sample and a negative sample as training samples, wherein the positive sample comprises two event descriptions corresponding to the same event, and the negative sample comprises two event descriptions corresponding to different events;
and training according to the positive sample and the negative sample to obtain the recognition model.
According to a preferred embodiment of the present invention, the constructing the positive samples and the negative samples as training samples comprises:
extracting an event description from a specified data source;
and constructing the positive sample and the negative sample according to the extracted event description.
According to a preferred embodiment of the present invention, the recognition model includes: fine-tune model based on the converter bi-directional encoder feature BERT.
According to a preferred embodiment of the present invention, the event vector extraction service includes: when an event description is input into the recognition model, extracting a last layer sentence separator [ SEP ] vector of the event description in the recognition model as an event vector corresponding to the event description.
According to a preferred embodiment of the present invention, the determining, according to the obtained event vector, an event that is matched with the event description to be retrieved in the event library includes:
and determining event vectors matched with the event vectors corresponding to the event descriptions to be retrieved in the event vectors corresponding to the event descriptions of the events in the event library based on an approximate nearest neighbor ANN tool, and taking the events corresponding to the matched event vectors as the events matched with the event descriptions to be retrieved.
An event retrieval apparatus comprising: the system comprises a model training unit, a service generation unit and an event retrieval unit;
the model training unit is used for training to obtain an identification model for identifying whether two event descriptions correspond to the same event or not;
the service generation unit is used for forming an event vector extraction service based on the recognition model, and the event vector represents the understanding of the recognition model to the event description;
the event retrieval unit is used for respectively acquiring an event vector corresponding to the event description to be retrieved and an event vector corresponding to the event description of the event in the event library according to the event vector extraction service; and determining an event matched with the event description to be retrieved in the event library according to the acquired event vector.
According to a preferred embodiment of the present invention, the model training unit is further configured to construct a positive sample and a negative sample as training samples, where the positive sample includes two event descriptions corresponding to a same event, and the negative sample includes two event descriptions corresponding to different events, and the recognition model is obtained by training according to the positive sample and the negative sample.
According to a preferred embodiment of the present invention, the model training unit extracts the event description from a specified data source, and constructs the positive and negative examples according to the extracted event description.
According to a preferred embodiment of the present invention, the recognition model includes: fine-tune mode based on the converter bi-directional encoder feature BERT.
According to a preferred embodiment of the present invention, the event vector extraction service includes: when an event description is input into the recognition model, extracting a last-layer sentence separator [ SEP ] vector of the event description in the recognition model as an event vector corresponding to the event description.
According to a preferred embodiment of the present invention, the event retrieving unit determines, based on an approximate nearest neighbor ANN tool, an event vector that matches the event vector corresponding to the event description to be retrieved among event vectors corresponding to event descriptions of events in the event library, and takes an event corresponding to the matched event vector as an event that matches the event description to be retrieved.
A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method as described above when executing the program.
A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method as set forth above.
Based on the introduction, it can be seen that, according to the scheme of the present invention, an identification model for identifying whether two event descriptions correspond to the same event can be obtained through training, an event vector extraction service can be formed based on the identification model, the event vector represents the understanding of the identification model on the event descriptions, then, an event vector corresponding to the event description to be retrieved and an event vector corresponding to the event description of the event in the event library can be respectively obtained according to the event vector extraction service, and further, the event matching with the event description to be retrieved in the event library can be determined according to the obtained event vector, so that the event retrieval based on the event description is realized, and the event vector is used to represent the event characteristics, so that the accuracy of the retrieval result is improved, and the present invention is applicable to various different scenes and has wide applicability.
[ description of the drawings ]
Fig. 1 is a flowchart of a first embodiment of an event retrieval method according to the present invention.
Fig. 2 is a schematic diagram of a sample labeling method according to the present invention.
Fig. 3 is a schematic diagram of a BERT-based network model structure according to the present invention.
Fig. 4 is a flowchart of a second embodiment of the event retrieval method according to the present invention.
Fig. 5 is a schematic structural diagram of an event retrieval device according to an embodiment of the present invention.
FIG. 6 illustrates a block diagram of an exemplary computer system/server 12 suitable for use in implementing embodiments of the present invention.
[ detailed description ] A
In order to make the technical scheme of the invention more clear and understood, the scheme of the invention is further explained by referring to the attached drawings and embodiments.
It is to be understood that the embodiments described are only a few embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In addition, it should be understood that the term "and/or" herein is only one kind of association relationship describing an associated object, and means that three relationships may exist, for example, a and/or B may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship.
Fig. 1 is a flowchart of a first embodiment of an event retrieval method according to the present invention. As shown in fig. 1, the following detailed implementation is included.
In 101, a recognition model for recognizing whether two event descriptions (also referred to as event sentences and the like) correspond to the same event is obtained through training.
In 102, an event vector extraction service is formed based on the recognition model, the event vector representing an understanding of the recognition model for the event description.
In 103, according to the event vector extraction service, an event vector corresponding to the event description to be retrieved and an event vector corresponding to the event description of the event in the event library are respectively obtained.
In 104, according to the obtained event vector, an event matching the event description to be retrieved in the event library is determined.
In order to implement the solution described in this embodiment, a recognition model needs to be obtained through training first, and the recognition model can be used to recognize whether two input event descriptions correspond to the same event. That is, the recognition model needs to have the ability to distinguish event descriptions, i.e., it needs to be able to give a lower probability of "same event" for two event descriptions that are similar but correspond to different events, and it needs to be able to give a higher probability of "same event" for two event descriptions that are different but correspond to the same event. The recognition model is trained to have the understanding capability of the recognition model on the event description so as to extract the understanding of the recognition model on the event description later.
Accordingly, a positive sample and a negative sample can be constructed as training samples, wherein the positive sample can contain two event descriptions corresponding to the same event, the negative sample can contain two event descriptions corresponding to different events, and the recognition model can be obtained by training according to the constructed positive sample and the constructed negative sample.
For example, the event description may be extracted from a specific data source, and the positive and negative examples may be constructed according to the extracted event description.
For example, the following steps are carried out:
the crawler tool can be used for capturing news/information from a specified site, text clustering can be performed to form a plurality of information clusters, each information cluster can contain a plurality of information of the same type, and each information can be arranged in a title (title) list form.
Based on the above processing, sample labeling can be performed. Fig. 2 is a schematic diagram of a sample labeling manner according to the present invention, as shown in fig. 2, according to an instruction received from a user, different titles corresponding to the same event can be marked with the same color, for example, the titles in the first and second rows correspond to event 1, and can be marked with the same color, the titles in the third and fourth rows correspond to event 2, and can be marked with another color, etc., in short, it is ensured that different events have different colors, and the same event has the same color.
Based on the sample labels, a plurality of event description pairs can be constructed, such as:
the first event description pair: (singer, hey two-dimensional Massa);
the second event description pair: (singer, crazy singing);
the two event description marks in the first event description pair are of the same color and correspond to the same event, and can be used as constructed positive samples, and the two event description marks in the second event description pair are of different colors and correspond to different events, and can be used as constructed negative samples.
In the above manner, a large number of positive and negative samples can be constructed.
And training to obtain the identification model according to the constructed positive sample and the negative sample. The recognition model may be a neural network model, and preferably, the recognition model may be a fine-tune (fine-tune) model based on Bidirectional Encoder features (BERTs) of the converter. The input is two event descriptions, and the output is the recognition result of whether the events corresponding to the two event descriptions are the same event.
An event vector extraction service may be formed based on the trained recognition model. Specifically, when the recognition model is a BERT-based fine-tune model, the formed event vector extraction service may refer to: when an event description is input into the recognition model, extracting a last layer sentence separator [ SEP ] vector of the event description in the recognition model as an event vector corresponding to the event description, and obtaining an understanding vector of the event description.
Fig. 3 is a schematic diagram of a BERT-based network model structure according to the present invention. As shown in fig. 3, after training, the model already has the capability of identifying whether the events corresponding to the two event descriptions are the same, which is obtained based on understanding of the event descriptions, and the vector extraction is to extract understanding of the model for the event descriptions. According to analysis, the vector used by the model for task calculation is the vector of the last layer of the classification symbol [ CLS ], obviously, the vector contains information about whether the information is the same or not, the information is obtained on the basis of understanding of two event descriptions, the [ SEP ] vector is a vector of each event description, the [ SEP ] vectors of two event descriptions of each layer are used for calculating [ CLS ] of the next layer, and the [ SEP ] vector contains understanding of the model for each event description, so that understanding of the model for the event descriptions can be obtained by extracting the [ SEP ] vector.
The above process is served to form a service, namely an event vector extraction service.
After the event vector extraction service is formed, the existing Approximate Nearest Neighbor (ANN) tool can be combined to provide high concurrency retrieval capability aiming at the event, and the ANN tool can provide rapid vector retrieval capability.
Specifically, according to the event vector extraction service, an event vector corresponding to the event description to be retrieved and an event vector corresponding to the event description of the event in the event library can be respectively obtained, and the event matching the event description to be retrieved in the event library can be determined according to the obtained event vector, for example, the event vector matching the event vector corresponding to the event description to be retrieved in the event vectors corresponding to the event description of the event in the event library can be determined based on an ANN tool, and the event corresponding to the matched event vector is used as the event matching the event description to be retrieved.
And aiming at the event description of each event in the event library, respectively calling an event vector extraction service to obtain an event vector corresponding to each event description, and mapping each event into a high-dimensional space.
And aiming at the obtained event vector, vector space index construction can be carried out through an ANN tool, after the vector space index construction is completed, a rapid vector retrieval service can be provided, for example, an event description to be retrieved provided by a user is obtained, an event vector corresponding to the event description is obtained through an event vector extraction service, a plurality of event vectors which are closest to the event vector corresponding to the event description to be retrieved in the vector space index are obtained through the ANN tool, and the event/events corresponding to the event vectors are/are used as events matched with the event description to be retrieved.
The plurality of event vectors may be one event vector or a plurality of event vectors, where a single event vector may refer to an event vector with the highest similarity between the event vectors corresponding to the events in the event library and the event description to be retrieved, and where a plurality of event vectors may refer to event vectors ranked in the top N bits after the event vectors corresponding to the event description to be retrieved are ranked in descending order of similarity, and N is a positive integer greater than one.
Based on the above description, fig. 4 is a flowchart of a second embodiment of the event retrieval method according to the present invention. As shown in fig. 4, the following detailed implementation is included.
In 401, a positive sample and a negative sample are constructed as training samples, wherein the positive sample includes two event descriptions corresponding to the same event, and the negative sample includes two event descriptions corresponding to different events.
The event description may be extracted from a specified data source, and positive and negative examples may be constructed from the extracted event description.
In 402, a BERT-based fine-tune model is trained from the constructed positive and negative examples.
How to train to get a BERT-based fine-tune model is prior art.
In 403, an event vector extraction service is formed according to the BERT based fine-tune model.
The event vector extraction service may refer to: when an event description is input into the BERT-based fine-tune model, a [ SEP ] vector of the last layer of the event description in the BERT-based fine-tune model is extracted as an event vector corresponding to the event description.
At 404, an event vector corresponding to the event description of the event in the event repository is obtained according to the event vector extraction service.
In 405, according to the event vector extraction service, an event vector corresponding to the event description to be retrieved is obtained.
In 406, based on the ANN tool, an event vector matching the event vector corresponding to the event description to be retrieved is determined from the event vectors corresponding to the event descriptions of the events in the event library, and the event corresponding to the matched event vector is taken as the event matching the event description to be retrieved.
It should be noted that for simplicity of explanation, the foregoing method embodiments are described as a series of acts, but those skilled in the art will recognize that the present invention is not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the invention. Further, those skilled in the art will appreciate that the embodiments described in this specification are presently preferred and that no acts or modules are required by the invention.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In short, by adopting the scheme of the embodiment of the method, the event retrieval based on the event description can be realized, and the event vector is utilized to represent the event characteristics, so that the accuracy of the retrieval result can be improved, and the method can be suitable for various different scenes and has wide applicability.
The above is a description of embodiments of the method, and the embodiments of the apparatus are described below to further illustrate the aspects of the present invention.
Fig. 5 is a schematic structural diagram of an event retrieval device according to an embodiment of the present invention. As shown in fig. 5, includes: a model training unit 501, a service generation unit 502, and an event retrieval unit 503.
The model training unit 501 is configured to train to obtain a recognition model for recognizing whether two event descriptions correspond to the same event.
A service generation unit 502 for forming an event vector extraction service based on the recognition model, the event vector representing an understanding of the recognition model for the event description.
An event retrieving unit 503, configured to obtain, according to the event vector extraction service, an event vector corresponding to the event description to be retrieved and an event vector corresponding to the event description of the event in the event library, respectively; and determining an event matched with the event description to be retrieved in the event library according to the acquired event vector.
In order to train to obtain the recognition model, the model training unit 501 needs to first construct a positive sample and a negative sample as training samples, where the positive sample may include two event descriptions corresponding to the same event, and the negative sample may include two event descriptions corresponding to different events, and then the recognition model may be obtained according to the training of the positive sample and the negative sample.
How to construct the positive and negative examples is not limited, for example, the model training unit 501 may extract an event description from a specific data source, and construct the positive and negative examples according to the extracted event description.
According to the constructed positive samples and negative samples, an identification model can be trained, and the identification model can be a neural network model, preferably, the identification model can be a fine-tune model based on BERT. The input is two event descriptions, and the output is the recognition result of whether the events corresponding to the two event descriptions are the same event.
The service generation unit 502 may form an event vector extraction service based on the trained recognition model. Specifically, when the recognition model is a BERT-based fine-tune model, the formed event vector extraction service may refer to: when an event description is input into the recognition model, extracting a [ SEP ] vector of the last layer of the event description in the recognition model as an event vector corresponding to the event description, namely obtaining an understanding vector of the event description.
After the event vector extraction service is formed, high concurrent retrieval capability aiming at the event can be provided by combining the existing ANN tool, and the ANN tool can provide rapid vector retrieval capability.
Specifically, the event retrieving unit 503 may respectively obtain an event vector corresponding to the event description to be retrieved and an event vector corresponding to the event description of the event in the event library according to the event vector extraction service, and may determine the event in the event library that matches the event description to be retrieved according to the obtained event vector, for example, may determine the event vector that matches the event vector corresponding to the event description to be retrieved in the event vector corresponding to the event description of the event in the event library based on an ANN tool, and take the event corresponding to the matched event vector as the event that matches the event description to be retrieved.
For a specific work flow of the apparatus embodiment shown in fig. 5, reference is made to the related description in the foregoing method embodiment, and details are not repeated.
In short, by adopting the scheme of the embodiment of the device, the event retrieval based on the event description can be realized, and the event vector is utilized to represent the event characteristics, so that the accuracy of the retrieval result can be improved, and the device can be suitable for various different scenes and has wide applicability.
FIG. 6 illustrates a block diagram of an exemplary computer system/server 12 suitable for use in implementing embodiments of the present invention. The computer system/server 12 shown in FIG. 6 is only one example and should not be taken to limit the scope of use or functionality of embodiments of the present invention.
As shown in FIG. 6, computer system/server 12 is in the form of a general purpose computing device. The components of computer system/server 12 may include, but are not limited to: one or more processors (processing units) 16, a memory 28, and a bus 18 that connects the various system components, including the memory 28 and the processors 16.
Computer system/server 12 typically includes a variety of computer system readable media. Such media may be any available media that is accessible by computer system/server 12 and includes both volatile and nonvolatile media, removable and non-removable media.
The memory 28 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM) 30 and/or cache memory 32. The computer system/server 12 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 34 may be used to read from and write to non-removable, nonvolatile magnetic media (not shown in FIG. 6, and commonly referred to as a "hard drive"). Although not shown in FIG. 6, a magnetic disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 by one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention.
A program/utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination of which may comprise an implementation of a network environment. Program modules 42 generally carry out the functions and/or methodologies of the described embodiments of the invention.
Computer system/server 12 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), one or more devices that enable a user to interact with computer system/server 12, and/or any device (e.g., network card, modem, etc.) that enables computer system/server 12 to communicate with one or more other computing devices. Such communication may be through an input/output (I/O) interface 22. Also, the computer system/server 12 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN) and/or a public network, such as the Internet) via the network adapter 20. As shown in FIG. 6, network adapter 20 communicates with the other modules of computer system/server 12 via bus 18. It should be understood that although not shown in the figures, other hardware and/or software modules may be used in conjunction with computer system/server 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
The processor 16 executes various functional applications and data processing by executing programs stored in the memory 28, for example, implementing the methods in the embodiments shown in fig. 1 or fig. 4.
The invention also discloses a computer-readable storage medium, on which a computer program is stored, which program, when being executed by a processor, will carry out the method of the embodiments shown in fig. 1 or fig. 4.
Any combination of one or more computer-readable media may be employed. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus and method, etc., can be implemented in other ways. For example, the above-described device embodiments are merely illustrative, and for example, the division of the units is only one logical functional division, and other divisions may be realized in practice.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit may be implemented in the form of hardware, or in the form of hardware plus a software functional unit.
The integrated unit implemented in the form of a software functional unit may be stored in a computer readable storage medium. The software functional unit is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device) or a processor (processor) to execute some steps of the methods according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (8)
1. An event retrieval method, comprising:
training to obtain a recognition model for recognizing whether two event descriptions correspond to the same event, wherein the recognition model comprises the following steps: extracting event descriptions from a specified data source, clustering, constructing a positive sample and a negative sample serving as training samples for each obtained cluster based on color marks of a user, wherein the positive sample comprises two event descriptions corresponding to the same event, the negative sample comprises two event descriptions corresponding to different events, and training according to the positive sample and the negative sample to obtain the recognition model;
forming an event vector extraction service based on the recognition model, the event vector extraction service to: when an event description is input into the recognition model, extracting a last-layer sentence separator [ SEP ] vector of the event description in the recognition model as an event vector corresponding to the event description; the event vector represents an understanding of the recognition model for an event description;
according to the event vector extraction service, respectively acquiring an event vector corresponding to the event description to be retrieved and an event vector corresponding to the event description of the event in an event library;
and determining an event matched with the event description to be retrieved in the event library according to the acquired event vector.
2. The method of claim 1,
the recognition model comprises: fine-tune model based on the converter bi-directional encoder feature BERT.
3. The method of claim 1,
the determining, according to the obtained event vector, an event that is matched with the event description to be retrieved in the event library includes:
and determining event vectors matched with the event vectors corresponding to the event descriptions to be retrieved in the event vectors corresponding to the event descriptions of the events in the event library based on an approximate nearest neighbor ANN tool, and taking the events corresponding to the matched event vectors as the events matched with the event descriptions to be retrieved.
4. An event retrieval apparatus, comprising: the system comprises a model training unit, a service generating unit and an event retrieval unit;
the model training unit is used for training to obtain an identification model for identifying whether two event descriptions correspond to the same event, and comprises the following steps: extracting event descriptions from a specified data source, clustering, constructing a positive sample and a negative sample serving as training samples for each obtained cluster based on color marks of a user, wherein the positive sample comprises two event descriptions corresponding to the same event, the negative sample comprises two event descriptions corresponding to different events, and training according to the positive sample and the negative sample to obtain the recognition model;
the service generation unit is configured to form an event vector extraction service based on the recognition model, where the event vector extraction service is configured to: when an event description is input into the recognition model, extracting a last-layer sentence separator [ SEP ] vector of the event description in the recognition model as an event vector corresponding to the event description; the event vector represents an understanding of the recognition model for an event description;
the event retrieval unit is used for respectively acquiring an event vector corresponding to the event description to be retrieved and an event vector corresponding to the event description of the event in the event library according to the event vector extraction service; and determining an event matched with the event description to be retrieved in the event library according to the acquired event vector.
5. The apparatus of claim 4,
the recognition model includes: fine-tune mode based on the converter bi-directional encoder feature BERT.
6. The apparatus of claim 4,
the event retrieval unit determines an event vector matched with the event vector corresponding to the event description to be retrieved in the event vectors corresponding to the event descriptions of the events in the event library based on an approximate nearest neighbor ANN tool, and takes the event corresponding to the matched event vector as the event matched with the event description to be retrieved.
7. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method according to any of claims 1-3 when executing the program.
8. A computer-readable storage medium, on which a computer program is stored, which program, when being executed by a processor, carries out the method according to any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910520888.4A CN110378378B (en) | 2019-06-17 | 2019-06-17 | Event retrieval method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910520888.4A CN110378378B (en) | 2019-06-17 | 2019-06-17 | Event retrieval method and device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110378378A CN110378378A (en) | 2019-10-25 |
| CN110378378B true CN110378378B (en) | 2022-10-28 |
Family
ID=68248952
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910520888.4A Active CN110378378B (en) | 2019-06-17 | 2019-06-17 | Event retrieval method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110378378B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111259987B (en) * | 2020-02-20 | 2023-12-29 | 民生科技有限责任公司 | Method for extracting event main body by multi-model fusion based on BERT |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108021619A (en) * | 2017-11-13 | 2018-05-11 | 星潮闪耀移动网络科技（中国）有限公司 | A kind of event description object recommendation method and device |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104252488B (en) * | 2013-06-28 | 2017-12-22 | 华为技术有限公司 | The method and server of processing data |
| CN109299094A (en) * | 2018-09-18 | 2019-02-01 | 深圳壹账通智能科技有限公司 | Tables of data processing method, device, computer equipment and storage medium |
| CN109542956A (en) * | 2018-10-17 | 2019-03-29 | 深圳壹账通智能科技有限公司 | Report form generation method, device, computer equipment and storage medium |
-
2019
- 2019-06-17 CN CN201910520888.4A patent/CN110378378B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108021619A (en) * | 2017-11-13 | 2018-05-11 | 星潮闪耀移动网络科技（中国）有限公司 | A kind of event description object recommendation method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110378378A (en) | 2019-10-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108052577B (en) | Universal text content mining method, device, server and storage medium | |
| CN107992596B (en) | Text clustering method, text clustering device, server and storage medium | |
| CN107291828B (en) | Spoken language query analysis method and device based on artificial intelligence and storage medium | |
| CN107220235B (en) | Speech recognition error correction method and device based on artificial intelligence and storage medium | |
| CN107832662B (en) | Method and system for acquiring image annotation data | |
| CN108959257B (en) | Natural language parsing method, device, server and storage medium | |
| WO2021135469A1 (en) | Machine learning-based information extraction method, apparatus, computer device, and medium | |
| CN110276023B (en) | POI transition event discovery method, device, computing equipment and medium | |
| CN112015859A (en) | Text knowledge hierarchy extraction method and device, computer equipment and readable medium | |
| CN107908641B (en) | Method and system for acquiring image annotation data | |
| CN111738001B (en) | Training method of synonym recognition model, synonym determination method and equipment | |
| WO2021218028A1 (en) | Artificial intelligence-based interview content refining method, apparatus and device, and medium | |
| CN112507090B (en) | Method, apparatus, device and storage medium for outputting information | |
| CN113158656B (en) | Ironic content recognition method, ironic content recognition device, electronic device, and storage medium | |
| CN110377750B (en) | Comment generation method, comment generation device, comment generation model training device and storage medium | |
| CN110032734B (en) | Training method and device for similar meaning word expansion and generation of confrontation network model | |
| CN111144102A (en) | Method and device for identifying entity in statement and electronic equipment | |
| CN112052005A (en) | Interface processing method, device, equipment and storage medium | |
| CN107844531B (en) | Answer output method and device and computer equipment | |
| CN113096687B (en) | Audio and video processing method and device, computer equipment and storage medium | |
| CN113220854B (en) | Intelligent dialogue method and device for machine reading and understanding | |
| CN116402166B (en) | Training method and device of prediction model, electronic equipment and storage medium | |
| CN110378378B (en) | Event retrieval method and device, computer equipment and storage medium | |
| CN110362688B (en) | Test question labeling method, device and equipment and computer readable storage medium | |
| CN117011581A (en) | Image recognition method, medium, device and computing equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |