WO2020133039A1 - 对话语料中实体的识别方法、装置和计算机设备 - Google Patents
对话语料中实体的识别方法、装置和计算机设备 Download PDFInfo
- Publication number
- WO2020133039A1 WO2020133039A1 PCT/CN2018/124239 CN2018124239W WO2020133039A1 WO 2020133039 A1 WO2020133039 A1 WO 2020133039A1 CN 2018124239 W CN2018124239 W CN 2018124239W WO 2020133039 A1 WO2020133039 A1 WO 2020133039A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- entity
- corpus text
- corpus
- text
- recognition model
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
- G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
- G06F40/295—Named entity recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
- G06F40/284—Lexical analysis, e.g. tokenisation or collocates
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
Definitions
- the present invention relates to the field of machine learning technology, and in particular to a method, device, computer equipment, and storage medium for identifying entities in dialogue corpus.
- the existing method is to identify the entities in the text, and then understand the meaning of the text according to the identified entities.
- existing entity recognition models are usually trained based on the input word vectors to identify entities based on the input word information. This way leads to a low accuracy rate of the final identified entity.
- a method for identifying entities in a dialogue corpus includes:
- the text matrix is used as an input of an entity recognition model to obtain entities in the corpus text output by the entity recognition model.
- a device for identifying entities in dialogue corpus includes:
- the first obtaining module is used to obtain the corpus text of the entity to be identified
- the text segmentation module is used to segment the corpus text to obtain a segmentation result, and the segmentation result includes multiple words;
- a second obtaining module configured to obtain a word vector corresponding to each word in the word segmentation result, and combine the word vector corresponding to each word to obtain a text matrix corresponding to the corpus text;
- a third obtaining module is used to use the text matrix as an input of an entity recognition model to obtain entities in the corpus text output by the entity recognition model.
- a computer device includes a memory and a processor.
- the memory stores a computer program.
- the processor is caused to perform the following steps:
- the text matrix is used as an input of an entity recognition model to obtain entities in the corpus text output by the entity recognition model.
- a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor is caused to perform the following steps:
- the text matrix is used as an input of an entity recognition model to obtain entities in the corpus text output by the entity recognition model.
- the invention proposes a method, device and computer equipment for recognizing entities in a dialogue corpus.
- the corpus text of the entity to be recognized is obtained; at the same time, the corpus text is segmented to obtain a segmentation result, and the segmentation result contains multiple words ; Then obtain the word vector corresponding to each word in the word segmentation result, and combine the word vector corresponding to each word to obtain the text matrix corresponding to the corpus text; Finally, use the text matrix as the entity recognition model Input to obtain the entities in the corpus text output by the entity recognition model. Since the dialogue questions of robots are usually very short, they are typically short texts. Sometimes there may be only one word or one word in the sentence.
- the use of word vectors to identify entities can improve the accuracy of recognition compared to the use of word vectors. If the word vector is used for recognition, it is likely that the robot's entity has only one word, which leads to entity recognition failure. Further, because the number of commonly used Chinese characters is relatively determined, and the number of words will be different because of the combination of different Chinese characters, so the words The number of words is very large compared to the number of Chinese characters, and with the continuous development of online language, the number of words continues to expand, so compared to using word vectors to identify entities, using word vectors to predict the accuracy of entities The rate will be higher because it does not have the problem of finding new words.
- FIG. 1 is a schematic diagram of an implementation process of an entity recognition method in a dialogue corpus in an embodiment
- FIG. 2 is a schematic diagram of the BiLSTM+CRF model in an embodiment
- step 1022 is a schematic diagram of an implementation process of step 1022 in an embodiment
- FIG. 4 is a schematic diagram of an implementation process of an entity recognition method in a dialogue corpus in an embodiment
- FIG. 5 is a structural block diagram of an apparatus for identifying entities in a dialogue corpus in an embodiment
- FIG. 6 is a structural block diagram of a computer device in an embodiment.
- a method for identifying entities in a dialogue corpus is provided. This method is applied to the server.
- the server is a high-performance computer or a high-performance computer cluster.
- the method for identifying entities in the dialogue corpus includes the following steps:
- Step 102 Acquire the corpus text of the entity to be identified.
- the corpus text is a text containing one or more Chinese characters, and the corpus text may be text obtained through speech recognition.
- the corpus text is: I am going to eat.
- some processing needs to be performed on the original corpus text, such as removing stop words (punctuation marks), and then only the final corpus text of the entity to be recognized is obtained.
- Step S104 Segment the corpus text to obtain a segmentation result.
- the segmentation result includes multiple words.
- the corpus text "I'm going to eat” is used for word segmentation.
- the result of the word segmentation is: I, want, go, eat, eat.
- Step S106 Obtain a word vector corresponding to each word in the word segmentation result, and combine the word vector corresponding to each word to obtain a text matrix corresponding to the corpus text.
- the word vector is used to express a word by a vector.
- the word vector of different words can be obtained by training the word2vec model, for example, using the CBOW model or the Skip-Gram model.
- the word vector for the word “me” is [0.1 0.5 0.4]
- the word vector for the word “to” is [0.2 0.3 0.5]
- the word vector for the word “go” is [0.1 0.6 0.2]
- the word vector of the word "eat” is [0.4 0.3 0.2]
- the word vector of the word "rice” is [0.3 0.3 0.4]
- the padding mechanism is used to complete. For example, suppose the dimension of the preset text matrix is 6 ⁇ 3, and the dimension of the corpus text “I am going to eat” is 5 ⁇ 3, so the padding mechanism needs to be used to complete the text matrix:
- Step 108 Use the text matrix as an input of an entity recognition model to obtain entities in the corpus text output by the entity recognition model.
- the entity recognition model is a model capable of recognizing entities in the corpus text, for example, BiLSTM+CRF model. Among them, the entity refers to some keywords in the text. For example, the entity in the corpus text "I am going to eat” is "meal.”
- the BiLSTM+CRF model includes a forward LSTM layer, a backward LSTM layer, a BiLSTM output layer and a CRF entity labeling layer, first input each corpus text training sample in the corpus text training sample set into the BiLSTM+CRF model, and then The forward features of the corpus text training samples are mined through the forward LSTM layer, and the backward features of the corpus text training samples are mined through the backward LSTM layer.
- the method further includes: Step 1021: Obtain a corpus text training sample set.
- the corpus text training sample set includes multiple corpus text training samples.
- the corpus text training sample includes Colloquialized spoken corpus text training samples and associative corpus text training samples that semantically associate the spoken corpus text training samples; step 1022, training the entity recognition model according to the corpus text training sample set to obtain the Entity recognition model.
- the corpus text training sample set includes multiple corpus text training samples for training of entity recognition models. Specifically, multiple corpus text training samples in the corpus text training sample set are used for training of entity recognition models.
- the colloquial corpus text training sample set can be used to train the entity recognition model to improve the accuracy of the entity recognition model to recognize colloquial corpus text.
- the colloquial corpus text training sample set can be used to train the entity recognition model to improve the accuracy of the entity recognition model to recognize colloquial corpus text.
- associative corpus text training samples that perform semantic association on the spoken corpus text training samples.
- the content can include but is not limited to: synonymous associations, for example, "I am very angry", the association is "I am super angry”; rich tone auxiliary words, for example, "turn left”, association is "turn left and not OK”; Associate with polite terms, for example, "Turn you to the left”.
- the corpus text training samples in the corpus text training sample set can be obtained from various channels, such as instant messaging applications, live video applications, video viewing applications, news information applications, forums, and post bars.
- channels such as instant messaging applications, live video applications, video viewing applications, news information applications, forums, and post bars.
- a variety of channels can improve the accuracy of the entity recognition model. For example, with the development of the network, a large number of network terms have appeared, so you can choose from instant messaging applications, live video applications, video viewing applications, news information applications, forums and post bars Obtaining the training of these network terms to the entity recognition model enables the entity recognition model to have a higher recognition accuracy for these terms.
- instant messaging applications can include but are not limited to QQ and WeChat;
- the video live streaming application applications can include but are not limited to Betta live streaming and panda live streaming;
- the video viewing applications can include but are not limited to Tencent video and iQiyi;
- the news information application may include but is not limited to today's helmet and Weibo;
- the forum may include but is not limited to Tianya Forum;
- the post bar may include but not limited to Baidu post bar.
- training the entity recognition model according to the corpus text training sample set to obtain the entity recognition model includes:
- Step 1022A Perform word segmentation on each of the corpus text training samples in the corpus text training sample set to obtain a word segmentation result containing multiple words for each of the corpus text training samples.
- corpus texts in the corpus text training sample set I want to eat and I want to drink tea.
- the two corpus texts are segmented, and the word segmentation results are: "I, want, go, eat, eat” and " I want tea”.
- Step 1022B According to the word vector lookup table and the segmentation result of each of the corpus text training samples, a training text matrix corresponding to the corpus text training sample set is obtained.
- the word vector look-up table records the word identifier of each word and the word vector corresponding to the word identifier.
- the word vector look-up table may be as shown in Table 1. According to the result of word segmentation, determine the word to be searched, and then According to the word vector lookup table shown in Table 1, the word vector of each word in the corpus text is obtained, and finally the word vectors of the various corpus texts are combined to obtain the text matrix of the corresponding corpus text.
- Word mark Word vector I 110 [0.1 0.5 0.4] want 112 [0.2 0.3 0.5] eat 210 [0.4 0.3 0.2] rice 236 [0.3 0.3 0.4] drink 965 [0.7 0.2 0.1] tea 785 [0.7 0.3 0.2]
- Step 1022C Obtain a label corresponding to each word in each corpus text training sample to obtain a training text label matrix corresponding to the corpus text training sample set.
- the label is used to distinguish between entities and non-entities.
- the annotations are used to distinguish between entities and non-entities in corpus text training samples, as shown in Table 2. For example, if the corpus text training sample is "I'm angry", then the corpus text training sample is labeled "FFKJF", and it is converted to a computer-recognizable number "33203".
- the training text labeling matrix is A matrix containing numbers (computer processing recognizes numbers, not letters, so you need to convert alphabetic labels to numeric labels).
- Step 1022D the training text matrix is used as an input of an entity recognition model, and the corresponding training text labeling matrix is used as an output of the entity recognition model, and the entity recognition model is trained to obtain a target entity recognition model.
- step 108 the text matrix is used as an input of an entity recognition model, and obtaining entities in the corpus text output by the entity recognition model includes: using the text matrix as an entity recognition model To obtain the location distribution information of entities and non-entities in the corpus text; according to the location distribution information, obtain the entities in the corpus text.
- the output is the training text annotation matrix, which records the location distribution information of the entity and non-entity. Therefore, the text annotation matrix is also obtained at the time of recognition, for example .
- the text annotation matrix corresponding to the text matrix output by the entity recognition model is [3 3 3 2 0], and the text annotation matrix clearly records " Each word in "I'm going to eat” belongs to entity or non-entity, and the labeling matrix fully indicates the location distribution information of entity and non-entity. By obtaining the number of the corresponding position, we can clearly know whether the word corresponding to the number is Entity or non-entity.
- the sample types of the corpus text training samples include command type, emotion type, name type, and action type.
- the entity recognition model is performed according to the corpus text training sample set Training to obtain the entity recognition model, including: obtaining the training ratio of the command-type corpus text training sample, the emotion-type corpus text training sample, the name-type corpus text training sample, and the action-type corpus text training sample; according to the command-type corpus text Training ratios of the training samples, the emotional corpus text training samples, the name corpus text training samples, and the action corpus text training samples, and obtaining a corresponding number of corpus text training samples from the corpus text training sample set; Training the entity recognition model according to the obtained corresponding number of corpus text training samples to obtain the entity recognition model.
- the corpus of the command-type corpus text training sample is the corpus containing the spoken words of the entity, for example, "turn left", “turn right”;
- the corpus of the sentiment corpus text training sample is the corpus whose entity content is used to express emotions, for example , "I am a little bit angry", "I am very happy to chat with you”;
- the corpus of the name-type corpus text training sample is a corpus with entity content containing nouns.
- the nouns include but are not limited to names, names of places of historical interest and place names, for example, “Liu Dehua ", "Emei Mountain”; the corpus of the action-type corpus text training sample is a corpus with an entity containing action instructions, for example, "I want to eat", "I want to drink tea”.
- the command-type corpus text training samples the emotion-type corpus text training samples, the name-type corpus text training samples, and the action-type corpus text training samples
- their training ratios can be set to the same, for example, the different types of training ratios are all 60%.
- the number of command-type corpus text training samples, emotional-type corpus text training samples, name-type corpus text training samples, and action-type corpus text training samples in the corpus text training sample set are 100, 200, 300, and 200, respectively.
- the number of command corpus text training samples, sentiment corpus text training samples, name corpus text training samples, and action corpus text training samples that are sent to the entity recognition model for training is 60, 120.
- the training ratio can be determined according to the actual application scenario. For example, if a robot is used to execute commands, the training ratio of the command corpus text training samples can be set higher, for example, set to 100%, that is All the command-type corpus text training samples are sent to the entity recognition model for training.
- the corpus text of the entity to be recognized is first obtained; at the same time, the corpus text is segmented to obtain a segmentation result, and the segmentation result contains multiple words; then each of the segmentation results is obtained A word vector corresponding to each word, combining the word vector corresponding to each word to obtain a text matrix corresponding to the corpus text; and finally using the text matrix as an input of an entity recognition model to obtain the output of the entity recognition model
- the entity in the corpus text Since the dialogue questions of robots are usually very short, they are typically short texts. Sometimes there may be only one word or one word in the sentence. Therefore, the use of word vectors to identify entities can improve the accuracy of recognition compared to the use of word vectors.
- the robot's entity has only one word, which leads to entity recognition failure.
- the number of commonly used Chinese characters is relatively determined, and the number of words will be different because of the combination of different Chinese characters, so the words The number of words is very large compared to the number of Chinese characters, and with the continuous development of online language, the number of words continues to expand, so compared to using word vectors to identify entities, using word vectors to predict the accuracy of entities The rate will be higher because it does not have the problem of finding new words.
- step 108 the text matrix is used as an input of an entity recognition model, and after obtaining entities in the corpus text output by the entity recognition model, the method further includes:
- Step 109 Find whether the entity exists in the entity library.
- Step 110 If the entity exists in the entity library, the entity is a trusted entity.
- Step 111 If the entity does not exist in the entity library, the entity is a suspicious entity.
- the entity library is used to store entities. Here, it is mainly to judge the credibility of the acquired entity. If the entity obtained after identification exists in the preset entity library, the entity is considered to be a trusted entity. If the entity obtained after identification does not exist In the preset entity library, the entity is a suspicious entity, that is, the entity is likely to be a new entity. Further, after determining that the identification is a suspicious entity, it is necessary to further determine whether the entity is a new entity, if the entity If it is indeed a new entity, add it to the entity library.
- the entity library includes a command entity library, an emotional entity library, a name entity library, and an action entity library.
- entity library if the entity exists, the entity After being a trusted entity, it also includes:
- the entity type of the entity is determined according to the type of the entity library where the entity is located.
- the entity library is divided into a command entity library, an emotion entity library, a name entity library and an action entity library.
- the command entity library stores command entities, for example, "turn left”, and the emotion entity library Store emotional entities, for example, "happy”, store name entities in the name entity library, for example, “Liu Dehua”, and store action entities in the action entity library, for example, "meal”.
- the response templates may be similar. Therefore, different response templates are set for different types of entities, so that after determining the type of the entity, the response template corresponding to the type Match, find the response content corresponding to the entity, and set the response template for different types of entities, can greatly reduce the amount of matching, that is, when searching for the response content of the corpus text corresponding to the entity, only Searching in this type of reply template instead of searching in a large response template that contains multiple types can greatly improve search efficiency.
- an embodiment of the present invention provides an apparatus 500 for identifying entities in a dialogue corpus.
- the apparatus 500 includes:
- the first obtaining module 502 is used to obtain the corpus text of the entity to be identified
- the text segmentation module 504 is used to segment the corpus text to obtain a segmentation result, and the segmentation result includes multiple words;
- the second obtaining module 506 is configured to obtain a word vector corresponding to each word in the word segmentation result, and combine the word vector corresponding to each word to obtain a text matrix corresponding to the corpus text;
- the third obtaining module 508 is configured to use the text matrix as an input of an entity recognition model to obtain entities in the corpus text output by the entity recognition model.
- the device 500 further includes: a sample set acquisition module for acquiring a corpus text training sample set, the corpus text training sample set includes multiple corpus text training samples, and the corpus text training sample includes Colloquialized spoken corpus text training samples and associative corpus text training samples that semantically associate the spoken corpus text training samples; a model training module for training the entity recognition model according to the corpus text training sample set, Obtain the entity recognition model.
- the model training module includes: a training sample word segmentation module for segmenting each of the corpus text training samples in the corpus text training sample set to obtain each of the corpus text training samples A word segmentation result containing multiple words; a training text matrix acquisition module, used to obtain a training text matrix corresponding to the corpus text training sample set according to the word vector lookup table and the word segmentation results of each of the corpus text training samples; The labeling module is used to obtain a label corresponding to each word in each of the corpus text training samples to obtain a training text labeling matrix corresponding to the corpus text training sample set.
- the label is used to distinguish between entities and non-entities; target entities
- the model training module is used to take the training text matrix as the input of the entity recognition model, and use the corresponding training text annotation matrix as the output of the entity recognition model to train the entity recognition model to obtain the target entity recognition model.
- the sample types of the corpus text training samples include command type, emotion type, name type and action type
- the model training module includes: a training ratio acquisition module for acquiring command type corpus text training Training ratio of samples, sentiment-based corpus text training samples, name-based corpus text training samples, and action-based corpus text training samples; a proportional sample acquisition module for training based on the command-based corpus text training samples and the sentiment-based corpus text training The training ratio of the sample, the name-type corpus text training sample, and the action-type corpus text training sample, and obtaining a corresponding number of corpus text training samples from the corpus text training sample set; the proportional sample training module is used to Corresponding number of corpus text training samples are trained on the entity recognition model to obtain the entity recognition model.
- the device 500 further includes: an entity search module for searching whether the entity exists in an entity library; a trusted entity module for if the entity exists in the entity library, Then the entity is a trusted entity; an entity module may be used if the entity does not exist in the entity library, the entity is a suspicious entity.
- the entity library includes a command entity library, an emotional entity library, a name entity library, and an action entity library.
- the device 500 further includes: an entity type determination module, configured to The type of the entity library where the entity is located determines the entity type of the entity; the reply template acquisition module is used to acquire a reply template corresponding to the entity type, so as to find a reply result in the reply template.
- the third acquisition module 408 includes: a location distribution acquisition module, which is used to input the text matrix as an input of an entity recognition model to obtain the location distribution of entities and non-entities in the corpus text Information; location entity acquisition module, used to obtain the entity in the corpus text according to the location distribution information.
- FIG. 6 shows an internal structure diagram of a computer device in an embodiment.
- the computer device may specifically be a server.
- the computer device includes a processor, a memory, and a network interface connected by a system bus.
- the memory includes a non-volatile storage medium and an internal memory.
- the non-volatile storage medium of the computer device stores an operating system, and may also store a computer program.
- the processor may enable the processor to realize the entity recognition method in the dialog corpus.
- a computer program may also be stored in the internal memory.
- the processor may cause the processor to execute the method for identifying the entity in the dialog corpus.
- the network interface is used to communicate with the outside.
- FIG. 6 is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer equipment to which the solution of the present application is applied.
- the specific computer equipment may It includes more or fewer components than shown in the figure, or some components are combined, or have a different component arrangement.
- the method for identifying entities in the dialogue corpus provided by the present application may be implemented in the form of a computer program, and the computer program may run on the computer device shown in FIG. 6.
- the memory of the computer device may store various program templates constituting the identification device of the entities in the dialogue corpus. For example, the first acquisition module 502, the text segmentation module 504, the second acquisition module 506, and the third acquisition module 508.
- a computer device includes a memory and a processor.
- the memory stores a computer program.
- the processor is caused to perform the following steps: obtain a corpus text of an entity to be recognized;
- the corpus text is segmented to obtain a segmentation result, and the segmentation result includes multiple words;
- a word vector corresponding to each word in the segmentation result is obtained, and the word vector corresponding to each word is combined to obtain the A text matrix corresponding to a corpus text; using the text matrix as an input of an entity recognition model to obtain entities in the corpus text output by the entity recognition model.
- the above-mentioned computer program when executed by the processor, it is also used to perform the following steps: obtaining a corpus text training sample set, the corpus text training sample set includes multiple corpus text training samples, the corpus The text training samples include colloquialized spoken corpus text training samples and associative corpus text training samples that semantically associate the spoken corpus text training samples; the entity recognition model is trained according to the corpus text training sample set to obtain The entity recognition model is described.
- the training the entity recognition model according to the corpus text training sample set to obtain the entity recognition model includes: training each of the corpus texts in the corpus text training sample set The sample performs word segmentation to obtain a word segmentation result of each of the corpus text training samples that contains multiple words; according to the word vector lookup table and the word segmentation result of each of the corpus text training samples, a corresponding to the corpus text training sample set is obtained Training text matrix; obtain the annotation corresponding to each word in each of the corpus text training samples to obtain the training text annotation matrix corresponding to the corpus text training sample set.
- the annotations are used to distinguish between entities and non-entities;
- the training text matrix is used as the input of the entity recognition model, and the corresponding training text labeling matrix is used as the output of the entity recognition model to train the entity recognition model to obtain the target entity recognition model.
- the sample types of the corpus text training samples include command type, emotion type, name type and action type
- the entity recognition model is trained according to the corpus text training sample set to obtain
- the entity recognition model includes: obtaining the training ratio of the command corpus text training sample, the emotional corpus text training sample, the name corpus text training sample, and the action corpus text training sample; according to the command corpus text training sample, all The training proportion of the emotional corpus text training sample, the name corpus text training sample and the action corpus text training sample, and obtain a corresponding number of corpus text training samples from the corpus text training sample set; according to the obtained correspondence
- a number of corpus text training samples are used to train the entity recognition model to obtain the entity recognition model.
- the above-mentioned computer program when executed by the processor, it is also used to perform the following steps: go to an entity library to find whether the entity exists; if the entity exists in the entity library, then the The entity is a trusted entity; if the entity does not exist in the entity library, the entity is a suspicious entity.
- the above-mentioned computer program when executed by the processor, it is also used to perform the following steps: determining the entity type of the entity according to the type of the entity library where the entity is located; acquiring the entity type Reply template to find the answer result in the reply template.
- the text matrix is used as an input of an entity recognition model to obtain the corpus output by the entity recognition model
- the entities in the text include: using the text matrix as an input of an entity recognition model to obtain location distribution information of entities and non-entities in the corpus text; and obtaining entities in the corpus text according to the location distribution information .
- a computer-readable storage medium storing a computer program, which when executed by a processor, causes the processor to perform the following steps: obtain a corpus text of an entity to be recognized; segment the corpus text to obtain a word segmentation As a result, the word segmentation result contains multiple words; obtain the word vector corresponding to each word in the word segmentation result, and combine the word vector corresponding to each word to obtain the text matrix corresponding to the corpus text; The text matrix is used as an input of an entity recognition model to obtain entities in the corpus text output by the entity recognition model.
- the above-mentioned computer program when executed by the processor, it is also used to perform the following steps: obtaining a corpus text training sample set, the corpus text training sample set includes multiple corpus text training samples, The text training samples include colloquialized spoken corpus text training samples and associative corpus text training samples that semantically associate the spoken corpus text training samples; the entity recognition model is trained according to the corpus text training sample set to obtain The entity recognition model is described.
- the training the entity recognition model according to the corpus text training sample set to obtain the entity recognition model includes: training each of the corpus texts in the corpus text training sample set The sample performs word segmentation to obtain a word segmentation result of each of the corpus text training samples that contains multiple words; according to the word vector lookup table and the word segmentation result of each of the corpus text training samples, a corresponding to the corpus text training sample set is obtained Training text matrix; obtain the annotation corresponding to each word in each of the corpus text training samples to obtain the training text annotation matrix corresponding to the corpus text training sample set.
- the annotations are used to distinguish between entities and non-entities;
- the training text matrix is used as the input of the entity recognition model, and the corresponding training text labeling matrix is used as the output of the entity recognition model to train the entity recognition model to obtain the target entity recognition model.
- the sample types of the corpus text training samples include command type, emotion type, name type and action type
- the entity recognition model is trained according to the corpus text training sample set to obtain
- the entity recognition model includes: obtaining the training ratio of the command corpus text training sample, the emotional corpus text training sample, the name corpus text training sample, and the action corpus text training sample; according to the command corpus text training sample, all The training proportions of the emotional corpus text training samples, the name corpus text training samples and the action corpus text training samples, and obtain a corresponding number of corpus text training samples from the corpus text training sample set; according to the obtained correspondence
- a number of corpus text training samples are used to train the entity recognition model to obtain the entity recognition model.
- the above-mentioned computer program when executed by the processor, it is also used to perform the following steps: go to an entity library to find whether the entity exists; if the entity exists in the entity library, then the The entity is a trusted entity; if the entity does not exist in the entity library, the entity is a suspicious entity.
- the above-mentioned computer program when executed by the processor, it is also used to perform the following steps: determining the entity type of the entity according to the type of the entity library where the entity is located; acquiring the entity type Reply template to find the answer result in the reply template.
- the text matrix is used as an input of an entity recognition model to obtain the corpus output by the entity recognition model
- the entities in the text include: using the text matrix as an input of an entity recognition model to obtain location distribution information of entities and non-entities in the corpus text; and obtaining entities in the corpus text according to the location distribution information .
- the method for identifying entities in the dialogue corpus the device for identifying entities in the dialogue corpus, computer equipment and computer-readable storage media belong to the same inventive concept.
- the method for identifying entities in the dialogue corpus and the identification of entities in the dialogue corpus The contents involved in the apparatus, the computer equipment, and the computer-readable storage medium are mutually applicable.
- Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory.
- Volatile memory can include random access memory (RAM) or external cache memory.
- RAM random access memory
- DRAM dynamic RAM
- SDRAM synchronous DRAM
- DDRSDRAM double data rate SDRAM
- ESDRAM enhanced SDRAM
- SLDRAM synchronous chain (Synchlink) DRAM
- RDRAM direct RAM
- DRAM direct memory bus dynamic RAM
- RDRAM memory bus dynamic RAM
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Computing Systems (AREA)
- Machine Translation (AREA)
Abstract
一种对话语料中实体的识别方法、装置和计算机设备,包括:获取待识别实体的语料文本(S102);将所述语料文本进行分词,得到分词结果,所述分词结果中包含多个字(S104);获取所述分词结果中的每个字对应的字向量,将所述每个字对应的字向量进行组合得到所述语料文本对应的文本矩阵(S106);将所述文本矩阵作为实体识别模型的输入,获取所述实体识别模型输出的所述语料文本中的实体(S108)。通过上述方式,能够提高实体识别的准确率。
Description
本发明涉及机器学习技术领域,尤其涉及一种对话语料中实体的识别方法、装置、计算机设备及存储介质。
随着语音识别技术的发展,将语音识别成文本的瓶颈突破了,机器人对人表达的意思将更清楚,对话将更为简单。然而,在对语音进行语音识别后,得到的只是一串文本,机器人并不知晓该文本表达的含义。
为了理解文本含义,现有的方法是对文本中的实体进行识别,然后根据识别出的实体来理解文本表达的含义。但是,现有的实体识别模型通常是根据输入的词向量进行训练的,以根据输入的词语信息来识别实体,这样的方式导致最终识别的实体准确率低。
基于此,有必要针对上述问题,提出一种识别率高的对话语料中实体的识别方法、装置和计算机设备。
一种对话语料中实体的识别方法,所述方法包括:
获取待识别实体的语料文本;
将所述语料文本进行分词,得到分词结果,所述分词结果中包含多个字;
获取所述分词结果中的每个字对应的字向量,将所述每个字对应的字向量进行组合得到所述语料文本对应的文本矩阵;
将所述文本矩阵作为实体识别模型的输入,获取所述实体识别模型输出的所述语料文本中的实体。
一种对话语料中实体的识别装置,所述装置包括:
第一获取模块,用于获取待识别实体的语料文本;
文本分词模块,用于将所述语料文本进行分词,得到分词结果,所述分词结果中包含多个字;
第二获取模块,用于获取所述分词结果中的每个字对应的字向量,将所述每个字对应的字向量进行组合得到所述语料文本对应的文本矩阵;
第三获取模块,用于将所述文本矩阵作为实体识别模型的输入,获取所述实体识别模型输出的所述语料文本中的实体。
一种计算机设备,包括存储器和处理器,所述存储器存储有计算机程序,所述计算机程序被所述处理器执行时,使得所述处理器执行以下步骤:
获取待识别实体的语料文本;
将所述语料文本进行分词,得到分词结果,所述分词结果中包含多个字;
获取所述分词结果中的每个字对应的字向量,将所述每个字对应的字向量进行组合得到所述语料文本对应的文本矩阵;
将所述文本矩阵作为实体识别模型的输入,获取所述实体识别模型输出的所述语料文本中的实体。
一种计算机可读存储介质,存储有计算机程序,所述计算机程序被处理器执行时,使得所述处理器执行以下步骤:
获取待识别实体的语料文本;
将所述语料文本进行分词,得到分词结果,所述分词结果中包含多个字;
获取所述分词结果中的每个字对应的字向量,将所述每个字对应的字向量进行组合得到所述语料文本对应的文本矩阵;
将所述文本矩阵作为实体识别模型的输入,获取所述实体识别模型输出的所述语料文本中的实体。
实施本发明实施例,将具有如下效果:
本发明提出了一种对话语料中实体的识别方法、装置和计算机设备,首先获取待识别实体的语料文本;同时将所述语料文本进行分词,得到分词结果,所述分词结果中包含多个字;然后获取所述分词结果中的每个字对应的字向量,将所述每个字对应的字向量进行组合得到所述语料文本对应的文本矩阵;最后将所述文本矩阵作为实体识别模型的输入,获取所述实体识别模型输出的所述语料文本中的实体。由于机器人的对话问句通常都特别短,是典型的短文本,有时候句中可能就只有一个词或一个字,所以采用字向量对实体进行识别相对于采用词向量能够提高识别的准确率,因为若是采用词向量进行识别,很可能因为机器人的实体只有一个字导致实体识别失败,进一步的,由于常用的汉字的数量是比较确定的,而词语的数量会因为不同汉字的组合不同,所以词语的数量相对于汉字的数量是很大的,而且随着网络用语的不断发展,词语的数量还在继续扩大,所以相较于采用词向量的方式来识别实体,采用字向量来预测实体的准确率将更高,因为其不存在发现新词的问题。
为了更清楚地说明本发明实施例或现有技术中的技术方案,下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本发明的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。
其中:
图1为一个实施例中对话语料中实体的识别方法的实现流程示意图;
图2为一个实施例中BiLSTM+CRF模型的示意图;
图3为一个实施例中步骤1022的实现流程示意图;
图4为一个实施例中对话语料中实体的识别方法的实现流程示意图;
图5为一个实施例中对话语料中实体的识别装置的结构框图;
图6为一个实施例中计算机设备的结构框图。
下面将结合本发明实施例中的附图,对本发明实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例仅仅是本发明一部分实施例,而不是全部的实施例。基于本发明中的实施例,本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例,都属于本发明保护的范围。
如图1所示,在一个实施例中,提供了一种对话语料中实体的识别方法。该方法应用于服务器。所述服务器为高性能计算机或高性能计算机集群。该对话语料中实体的识别方法具体包括如下步骤:
步骤102,获取待识别实体的语料文本。
所述语料文本,为一个包含一个或者多个汉字的文本,所述语料文本可以是经过语音识别得到文本。例如,所述语料文本为:我要去吃饭。在经过语音识别获取到待识别实体的原始语料文本后,需要对原始的语料文本进行一些处理,比如去除停用词(标点符号),然后才得到最终的待识别实体的语料文本。
步骤S104,将所述语料文本进行分词,得到分词结果,所述分词结果中包含多个字。
例如,将语料文本“我要去吃饭”进行分词,得到的分词结果为:我,要,去,吃,饭。
步骤S106,获取所述分词结果中的每个字对应的字向量,将所述每个字对应的字向量进行组合得到所述语料文本对应的文本矩阵。
所述字向量,用于通过一个向量来表达一个字,可以通过训练word2vec模型获取不同字的字向量,例如,采用CBOW模型或者采用Skip-Gram模型。
对于分词结果中的每一个字,获取这些字的字向量。例如,字“我”的字向量为[0.1 0.5 0.4],字“要”的字向量为[0.2 0.3 0.5],字“去”的字向量为[0.1
0.6 0.2],字“吃”的字向量为[0.4 0.3 0.2],字“饭”的字向量为[0.3 0.3 0.4],然后将这些字的字向量进行组合,得到语料文本的文本矩阵:
需要说明的是,由于每个语料文本中包含的字的个数不一致,所以需要统一语料文本的文本矩阵的维度,对于不够预置的维度的,采用padding机制补齐。例如,假设预置的文本矩阵的维度是6×3,而语料文本“我要去吃饭”的维度是5×3,所以需要采用padding机制补齐,得到如下文本矩阵:
步骤108,将所述文本矩阵作为实体识别模型的输入,获取所述实体识别模型输出的所述语料文本中的实体。
所述实体识别模型,为能够识别语料文本中实体的模型,例如,BiLSTM+CRF模型。其中,所述实体,指文本中的一些关键词。例如,语料文本“我要去吃饭”中的实体为“吃饭”。
如图2所示,BiLSTM+CRF模型包括前向LSTM层、后向LSTM层,BiLSTM输出层和CRF实体标记层,首先将语料文本训练样本集中的各个语料文本训练样本输入BiLSTM+CRF模型,然后经过前向LSTM层对语料文本训练样本的前向特征进行挖掘,同时经过后向LSTM层对语料文本训练样本的后向特征进行挖掘,进一步的,将前向LSTM层和后向LSTM层的特征拼接起来,作为BiLSTM的特征输出,最后,将BiLSTM的输出作为CRF标记算法的输入,根据CRF层的输出结果得到最终的实体。
在本发明实施例中,为了获取到能够识别实体的实体识别模型,需要预先对模型进行训练,以得到训练好的实体识别模型,再用该训练好的实体识别模型对语料文本进行预测,所以,在步骤102所述获取待识别实体的语料文本之前,还包括:步骤1021,获取语料文本训练样本集,所述语料文本训练样本集包括多个语料文本训练样本,所述语料文本训练样本包括口语化的口语语料文本训练样本和对所述口语语料文本训练样本进行语义联想的联想语料文本训练样本;步骤1022,根据所述语料文本训练样本集对所述实体识别模型进行训练,得到所述实体识别模型。
所述语料文本训练样本集,包括多个语料文本训练样本,用于实体识别模型的训练,具体的,是将语料文本训练样本集中的多个语料文本训练样本用于实体识别模型的训练。
由于机器人的对话通常是比较口语化的,所以可以多采用口语化的语料文本训练样本集对实体识别模型进行训练,提高实体识别模型对口语化的语料文本的识别的准确率。同时,为了增大实体识别模型对某些句型或者说某些表达同一含义的语料文本的识别率,还需要得到对所述口语语料文本训练样本进行语义联想的联想语料文本训练样本,语义联想的内容可以包括但不限于:同义联想,例如,“我很生气”,联想为“我超生气”;丰富语气助词,例如,“向左转”,联想为“向左转行不行”;礼貌用语联想,例如,“麻烦你向左转”。
在本发明实施例中,语料文本训练样本集中的语料文本训练样本,可以从多种渠道获取,例如从即时通信应用、视频直播应用、视频观看应用、新闻资讯应用、论坛和贴吧获取,由于从多种渠道获取,能够提高实体识别模型的识别精度,例如,随着网络发展,出现了大量的网络用语,于是可以从即时通信应用、视频直播应用、视频观看应用、新闻资讯应用、论坛和贴吧获取到这些网络用语用语对实体识别模型的训练,使得实体识别模型能够对这些用语有更高的识别精度。
其中,即时通信应用可以包括但不限于QQ和微信;所述视频直播应用应用可以包括但不限于斗鱼直播和熊猫直播;所述视频观看应用可以包括但不限于腾讯视频和爱奇艺;所述新闻资讯应用可以包括但不限于今日头盔和微博;所述论坛可以包括但不限于天涯论坛;所述贴吧可以包括但不限于百度贴吧。
作为本发明的一种实施例,如图3所示,步骤1022所述根据所述语料文本训练样本集对所述实体识别模型进行训练,得到所述实体识别模型,包括:
步骤1022A,将所述语料文本训练样本集中的各个所述语料文本训练样本进行分词,得到每个所述语料文本训练样本的包含多个字的分词结果。
例如,语料文本训练样本集中有两个语料文本:我要去吃饭和我要喝茶,对这两个语料文本进行分词,得到分词结果为:“我,要,去,吃,饭”和“我,要,喝,茶”。
步骤1022B,根据字向量查找表和每个所述语料文本训练样本的分词结果,得到与所述语料文本训练样本集对应的训练文本矩阵。
所述字向量查找表,记载了每个字的字标识和与该字标识对应的字向量,例如,字向量查找表可以如表1所示,根据分词结果,确定需要查找的字,然后再根据表1所示的字向量查找表,得到语料文本中每个字的字向量,最后将各个语料文本的字向量进行组合,得到对应语料文本的文本矩阵。
表1
字 | 字标识 | 字向量 |
我 | 110 | [0.1 0.5 0.4] |
要 | 112 | [0.2 0.3 0.5] |
吃 | 210 | [0.4 0.3 0.2] |
饭 | 236 | [0.3 0.3 0.4] |
喝 | 965 | [0.7 0.2 0.1] |
茶 | 785 | [0.7 0.3 0.2] |
步骤1022C,获取每个所述语料文本训练样本中每个字对应的标注,得到所述语料文本训练样本集对应的训练文本标注矩阵,所述标注用于区分实体和非实体。
所述标注,用于区分语料文本训练样本中的实体和非实体,如表2所示。例如,语料文本训练样本为“我很生气啊”,则对该语料文本训练样本的标注为“FFKJF”,将其转换为计算机能够识别的数字为“33203”,所述训练文本标注矩阵即为包含数字的矩阵(计算机处理时是识别数字,而不是字母,所以需要将字母型标注转换为数字型标注)。
同样对上面的语料文本训练样本集:我要去吃饭和我要喝茶,得到我要去吃饭的标注矩阵为:[3 3 3 2 0],得到我要喝茶的标注矩阵为:[3 3 2 0 3],于是,将两个语料文本的标注矩阵进行组合,得到与语料文本训练样本集对应的训练文本标注矩阵为:
。
表2
实体开始 | 实体中间 | 实体结束 | 非实体 |
K | Z | J | F |
2 | 1 | 0 | 3 |
步骤1022D,将所述训练文本矩阵作为实体识别模型的输入,将对应的所述训练文本标注矩阵作为所述实体识别模型的输出,对所述实体识别模型进行训练,得到目标实体识别模型。
在本发明实施例中,步骤108所述将所述文本矩阵作为实体识别模型的输入,获取所述实体识别模型输出的所述语料文本中的实体,包括:将所述文本矩阵作为实体识别模型的输入,得到所述语料文本中的实体和非实体的位置分布信息;根据所述位置分布信息,得到所述语料文本中的实体。
由于上述对实体识别模型进行训练的时候,作为输出的是训练文本标注矩阵,该训练文本标注矩阵记载了实体和非实体的位置分布信息,因此,在识别的时候得到的也是文本标注矩阵,例如,将语料文本“我要去吃饭”作为实体识别模型的输入,得到实体识别模型输出的与该文本矩阵对应的文本标注矩阵为[3 3 3 2 0],该文本标注矩阵明确的记载了“我要去吃饭”中的每个字属于实体还是非实体,并且,该标注矩阵充分表明了实体和非实体的位置分布信息,通过获取对应位置的数字,能够清楚的知晓该数字对应的字是实体还是非实体。
作为本发明的一种实施例,所述语料文本训练样本的样本类型包括命令型、情感型、名字型和动作型,步骤1022所述根据所述语料文本训练样本集对所述实体识别模型进行训练,得到所述实体识别模型,包括:获取命令型语料文本训练样本、情感型语料文本训练样本、名字型语料文本训练样本和动作型语料文本训练样本的训练比例;根据所述命令型语料文本训练样本、所述情感型语料文本训练样本、所述名字型语料文本训练样本和所述动作型语料文本训练样本的训练比例,从所述语料文本训练样本集中获取对应数量的语料文本训练样本;根据获取的对应数量的语料文本训练样本,对所述实体识别模型进行训练,得到所述实体识别模型。
命令型语料文本训练样本的语料为实体内容包含命令口语的语料,例如,“向左转”,“向右转”;情感型语料文本训练样本的语料为实体内容用于表达情感的语料,例如,“我有点生气”,“和你聊天很开心”;名字型语料文本训练样本的语料为实体内容包含名词的语料,所述名词包括但不限于名字、名胜古迹名称和地名,例如,“刘德华”,“峨眉山”;动作型语料文本训练样本的语料为实体包含动作指示的语料,例如,“我要去吃饭”,“我要喝茶”。
对于命令型语料文本训练样本、情感型语料文本训练样本、名字型语料文本训练样本和动作型语料文本训练样本,他们的训练比例可以设置为相同,例如将不同类型的训练比例均为60%,假设语料文本训练样本集中命令型语料文本训练样本、情感型语料文本训练样本、名字型语料文本训练样本和动作型语料文本训练样本的个数分别为100个、200个、300个和200个,那么根据训练比例,最终送入实体识别模型进行训练的命令型语料文本训练样本、情感型语料文本训练样本、名字型语料文本训练样本和动作型语料文本训练样本个数为60个,120个,180个和120个;或者,也可以将他们训练比例设置为不同,例如,分别设置为60%,70%,40%和80%,则最终送入实体识别模型进行训练的命令型语料文本训练样本、情感型语料文本训练样本、名字型语料文本训练样本和动作型语料文本训练样本个数为60个,140个,120个和160个。具体的,可以根据实际的应用场景确定训练比例,例如,某一机器人是用于执行命令的机器人,那么可以将命令型语料文本训练样本的训练比例设置高一些,例如,设置为100%,即将全部的命令型语料文本训练样本送入实体识别模型进行训练。
上述对话语料中实体的识别方法,首先获取待识别实体的语料文本;同时将所述语料文本进行分词,得到分词结果,所述分词结果中包含多个字;然后获取所述分词结果中的每个字对应的字向量,将所述每个字对应的字向量进行组合得到所述语料文本对应的文本矩阵;最后将所述文本矩阵作为实体识别模型的输入,获取所述实体识别模型输出的所述语料文本中的实体。由于机器人的对话问句通常都特别短,是典型的短文本,有时候句中可能就只有一个词或一个字,所以采用字向量对实体进行识别相对于采用词向量能够提高识别的准确率,因为若是采用词向量进行识别,很可能因为机器人的实体只有一个字导致实体识别失败,进一步的,由于常用的汉字的数量是比较确定的,而词语的数量会因为不同汉字的组合不同,所以词语的数量相对于汉字的数量是很大的,而且随着网络用语的不断发展,词语的数量还在继续扩大,所以相较于采用词向量的方式来识别实体,采用字向量来预测实体的准确率将更高,因为其不存在发现新词的问题。
在本发明实施例中,如图4所示,在步骤108所述将所述文本矩阵作为实体识别模型的输入,获取所述实体识别模型输出的所述语料文本中的实体之后,还包括:
步骤109,到实体库中查找是否存在所述实体。
步骤110,若所述实体库中存在所述实体,则所述实体为可信实体。
步骤111,若所述实体库中不存在所述实体,则所述实体为可疑实体。
所述实体库,用于存储实体。在这里,主要是对获取的实体的可信度进行判断,如果经过识别之后得到的实体存在于预置的实体库中,则认为该实体是可信实体,如果经过识别之后得到的实体没有存在于预置的实体库中,则该实体为可疑实体,即该实体很可能是新实体,进一步的,在确定该识别是可疑实体之后,需要进一步的判断该实体是否是新实体,若该实体确实是新的实体,则将其加入实体库。
在本发明实施例中,所述实体库包括命令型实体库、情感型实体库、名字型实体库和动作型实体库,在所述若所述实体库中存在所述实体,则所述实体为可信实体之后,还包括:
根据所述实体所在的实体库的类型确定所述实体的实体类型。
获取与所述实体类型对应的答复模板,以在所述答复模板中查找答复结果。
在这里,将实体库分为命令型实体库、情感型实体库、名字型实体库和动作型实体库,命令型实体库中存储命令型实体,例如,“左转”,情感型实体库中存储情感型实体,例如,“高兴”,名字型实体库中存储名字型实体,例如,“刘德华”,动作型实体库中存储动作型实体,例如,“吃饭”。
在本发明实施例中,对于不同类型的实体,其答复模板可能比较相似,因此,为不同类型的实体设置不同的答复模板,以在判断该实体的类型之后,在与该类型对应的答复模板中进行匹配,找到与该实体对应的答复内容,通过给不同类型的实体设置答复模板,能够很大程度的缩小匹配的量,即在搜索与该实体对应的语料文本的答复内容的时候,只用在这个类型的答复模板中搜索,而不用在一个大的包含多种类型的答复模板中搜索,能够大大的提高搜索效率。
如图5所示,本发明实施例提供一种对话语料中实体的识别装置500,该装置500包括:
第一获取模块502,用于获取待识别实体的语料文本;
文本分词模块504,用于将所述语料文本进行分词,得到分词结果,所述分词结果中包含多个字;
第二获取模块506,用于获取所述分词结果中的每个字对应的字向量,将所述每个字对应的字向量进行组合得到所述语料文本对应的文本矩阵;
第三获取模块508,用于将所述文本矩阵作为实体识别模型的输入,获取所述实体识别模型输出的所述语料文本中的实体。
在其中一个实施例中,所述装置500还包括:样本集获取模块,用于获取语料文本训练样本集,所述语料文本训练样本集包括多个语料文本训练样本,所述语料文本训练样本包括口语化的口语语料文本训练样本和对所述口语语料文本训练样本进行语义联想的联想语料文本训练样本;模型训练模块,用于根据所述语料文本训练样本集对所述实体识别模型进行训练,得到所述实体识别模型。
在其中一个实施例中,所述模型训练模块,包括:训练样本分词模块,用于将所述语料文本训练样本集中的各个所述语料文本训练样本进行分词,得到每个所述语料文本训练样本的包含多个字的分词结果;训练文本矩阵获取模块,用于根据字向量查找表和每个所述语料文本训练样本的分词结果,得到与所述语料文本训练样本集对应的训练文本矩阵;标注模块,用于获取每个所述语料文本训练样本中每个字对应的标注,得到所述语料文本训练样本集对应的训练文本标注矩阵,所述标注用于区分实体和非实体;目标实体模型训练模块,用于将所述训练文本矩阵作为实体识别模型的输入,将对应的所述训练文本标注矩阵作为所述实体识别模型的输出,对所述实体识别模型进行训练,得到目标实体识别模型。
在其中一个实施例中,所述语料文本训练样本的样本类型包括命令型、情感型、名字型和动作型,所述模型训练模块,包括:训练比例获取模块,用于获取命令型语料文本训练样本、情感型语料文本训练样本、名字型语料文本训练样本和动作型语料文本训练样本的训练比例;比例样本获取模块,用于根据所述命令型语料文本训练样本、所述情感型语料文本训练样本、所述名字型语料文本训练样本和所述动作型语料文本训练样本的训练比例,从所述语料文本训练样本集中获取对应数量的语料文本训练样本;比例样本训练模块,用于根据获取的对应数量的语料文本训练样本,对所述实体识别模型进行训练,得到所述实体识别模型。
在其中一个实施例中,所述装置500,还包括:实体查找模块,用于到实体库中查找是否存在所述实体;可信实体模块,用于若所述实体库中存在所述实体,则所述实体为可信实体;可以实体模块,用于若所述实体库中不存在所述实体,则所述实体为可疑实体。
在其中一个实施例中,所述实体库包括命令型实体库、情感型实体库、名字型实体库和动作型实体库,所述装置500,还包括:实体类型确定模块,用于根据所述实体所在的实体库的类型确定所述实体的实体类型;答复模板获取模块,用于获取与所述实体类型对应的答复模板,以在所述答复模板中查找答复结果。
在其中一个实施例中,所述第三获取模块408,包括:位置分布获取模块,用于将所述文本矩阵作为实体识别模型的输入,得到所述语料文本中的实体和非实体的位置分布信息;位置实体获取模块,用于根据所述位置分布信息,得到所述语料文本中的实体。
图6示出了一个实施例中计算机设备的内部结构图。该计算机设备具体可以是服务器。如图6所示,该计算机设备包括通过系统总线连接的处理器、存储器和网络接口。其中,存储器包括非易失性存储介质和内存储器。该计算机设备的非易失性存储介质存储有操作系统,还可存储有计算机程序,该计算机程序被处理器执行时,可使得处理器实现对话语料中实体的识别方法。该内存储器中也可储存有计算机程序,该计算机程序被处理器执行时,可使得处理器执行对话语料中实体的识别方法。网络接口用于与外部进行通信。本领域技术人员可以理解,图6中示出的结构,仅仅是与本申请方案相关的部分结构的框图,并不构成对本申请方案所应用于其上的计算机设备的限定,具体的计算机设备可以包括比图中所示更多或更少的部件,或者组合某些部件,或者具有不同的部件布置。
在一个实施例中,本申请提供的对话语料中实体的识别方法可以实现为一种计算机程序的形式,计算机程序可在如图6所示的计算机设备上运行。计算机设备的存储器中可存储组成对话语料中实体的识别装置的各个程序模板。比如,第一获取模块502、文本分词模块504、第二获取模块506以及第三获取模块508。
一种计算机设备,包括存储器和处理器,所述存储器存储有计算机程序,所述计算机程序被所述处理器执行时,使得所述处理器执行如下步骤:获取待识别实体的语料文本;将所述语料文本进行分词,得到分词结果,所述分词结果中包含多个字;获取所述分词结果中的每个字对应的字向量,将所述每个字对应的字向量进行组合得到所述语料文本对应的文本矩阵;将所述文本矩阵作为实体识别模型的输入,获取所述实体识别模型输出的所述语料文本中的实体。
在其中一个实施例中,上述计算机程序被所述处理器执行时,还用于执行以下步骤:获取语料文本训练样本集,所述语料文本训练样本集包括多个语料文本训练样本,所述语料文本训练样本包括口语化的口语语料文本训练样本和对所述口语语料文本训练样本进行语义联想的联想语料文本训练样本;根据所述语料文本训练样本集对所述实体识别模型进行训练,得到所述实体识别模型。
在其中一个实施例中,所述根据所述语料文本训练样本集对所述实体识别模型进行训练,得到所述实体识别模型,包括:将所述语料文本训练样本集中的各个所述语料文本训练样本进行分词,得到每个所述语料文本训练样本的包含多个字的分词结果;根据字向量查找表和每个所述语料文本训练样本的分词结果,得到与所述语料文本训练样本集对应的训练文本矩阵;获取每个所述语料文本训练样本中每个字对应的标注,得到所述语料文本训练样本集对应的训练文本标注矩阵,所述标注用于区分实体和非实体;将所述训练文本矩阵作为实体识别模型的输入,将对应的所述训练文本标注矩阵作为所述实体识别模型的输出,对所述实体识别模型进行训练,得到目标实体识别模型。
在其中一个实施例中,所述语料文本训练样本的样本类型包括命令型、情感型、名字型和动作型,所述根据所述语料文本训练样本集对所述实体识别模型进行训练,得到所述实体识别模型,包括:获取命令型语料文本训练样本、情感型语料文本训练样本、名字型语料文本训练样本和动作型语料文本训练样本的训练比例;根据所述命令型语料文本训练样本、所述情感型语料文本训练样本、所述名字型语料文本训练样本和所述动作型语料文本训练样本的训练比例,从所述语料文本训练样本集中获取对应数量的语料文本训练样本;根据获取的对应数量的语料文本训练样本,对所述实体识别模型进行训练,得到所述实体识别模型。
在其中一个实施例中,上述计算机程序被所述处理器执行时,还用于执行以下步骤:到实体库中查找是否存在所述实体;若所述实体库中存在所述实体,则所述实体为可信实体;若所述实体库中不存在所述实体,则所述实体为可疑实体。
在其中一个实施例中,上述计算机程序被所述处理器执行时,还用于执行以下步骤:根据所述实体所在的实体库的类型确定所述实体的实体类型;获取与所述实体类型对应的答复模板,以在所述答复模板中查找答复结果。
在其中一个实施例中,上述计算机程序被所述处理器执行时,还用于执行以下步骤:所述将所述文本矩阵作为实体识别模型的输入,获取所述实体识别模型输出的所述语料文本中的实体,包括:将所述文本矩阵作为实体识别模型的输入,得到所述语料文本中的实体和非实体的位置分布信息;根据所述位置分布信息,得到所述语料文本中的实体。
一种计算机可读存储介质,存储有计算机程序,所述计算机程序被处理器执行时,使得所述处理器执行如下步骤:获取待识别实体的语料文本;将所述语料文本进行分词,得到分词结果,所述分词结果中包含多个字;获取所述分词结果中的每个字对应的字向量,将所述每个字对应的字向量进行组合得到所述语料文本对应的文本矩阵;将所述文本矩阵作为实体识别模型的输入,获取所述实体识别模型输出的所述语料文本中的实体。
在其中一个实施例中,上述计算机程序被所述处理器执行时,还用于执行以下步骤:获取语料文本训练样本集,所述语料文本训练样本集包括多个语料文本训练样本,所述语料文本训练样本包括口语化的口语语料文本训练样本和对所述口语语料文本训练样本进行语义联想的联想语料文本训练样本;根据所述语料文本训练样本集对所述实体识别模型进行训练,得到所述实体识别模型。
在其中一个实施例中,所述根据所述语料文本训练样本集对所述实体识别模型进行训练,得到所述实体识别模型,包括:将所述语料文本训练样本集中的各个所述语料文本训练样本进行分词,得到每个所述语料文本训练样本的包含多个字的分词结果;根据字向量查找表和每个所述语料文本训练样本的分词结果,得到与所述语料文本训练样本集对应的训练文本矩阵;获取每个所述语料文本训练样本中每个字对应的标注,得到所述语料文本训练样本集对应的训练文本标注矩阵,所述标注用于区分实体和非实体;将所述训练文本矩阵作为实体识别模型的输入,将对应的所述训练文本标注矩阵作为所述实体识别模型的输出,对所述实体识别模型进行训练,得到目标实体识别模型。
在其中一个实施例中,所述语料文本训练样本的样本类型包括命令型、情感型、名字型和动作型,所述根据所述语料文本训练样本集对所述实体识别模型进行训练,得到所述实体识别模型,包括:获取命令型语料文本训练样本、情感型语料文本训练样本、名字型语料文本训练样本和动作型语料文本训练样本的训练比例;根据所述命令型语料文本训练样本、所述情感型语料文本训练样本、所述名字型语料文本训练样本和所述动作型语料文本训练样本的训练比例,从所述语料文本训练样本集中获取对应数量的语料文本训练样本;根据获取的对应数量的语料文本训练样本,对所述实体识别模型进行训练,得到所述实体识别模型。
在其中一个实施例中,上述计算机程序被所述处理器执行时,还用于执行以下步骤:到实体库中查找是否存在所述实体;若所述实体库中存在所述实体,则所述实体为可信实体;若所述实体库中不存在所述实体,则所述实体为可疑实体。
在其中一个实施例中,上述计算机程序被所述处理器执行时,还用于执行以下步骤:根据所述实体所在的实体库的类型确定所述实体的实体类型;获取与所述实体类型对应的答复模板,以在所述答复模板中查找答复结果。
在其中一个实施例中,上述计算机程序被所述处理器执行时,还用于执行以下步骤:所述将所述文本矩阵作为实体识别模型的输入,获取所述实体识别模型输出的所述语料文本中的实体,包括:将所述文本矩阵作为实体识别模型的输入,得到所述语料文本中的实体和非实体的位置分布信息;根据所述位置分布信息,得到所述语料文本中的实体。
需要说明的是,上述对话语料中实体的识别方法、对话语料中实体的识别装置、计算机设备和计算机可读存储介质属于同一个发明构思,对话语料中实体的识别方法、对话语料中实体的识别装置、计算机设备和计算机可读存储介质中涉及的内容可相互适用。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程,是可以通过计算机程序来指令相关的硬件来完成,所述的程序可存储于一非易失性计算机可读取存储介质中,该程序在执行时,可包括如上述各方法的实施例的流程。其中,本申请所提供的各实施例中所使用的对存储器、存储、数据库或其它介质的任何引用,均可包括非易失性和/或易失性存储器。非易失性存储器可包括只读存储器(ROM)、可编程ROM(PROM)、电可编程ROM(EPROM)、电可擦除可编程ROM(EEPROM)或闪存。易失性存储器可包括随机存取存储器(RAM)或者外部高速缓冲存储器。作为说明而非局限,RAM以多种形式可得,诸如静态RAM(SRAM)、动态RAM(DRAM)、同步DRAM(SDRAM)、双数据率SDRAM(DDRSDRAM)、增强型SDRAM(ESDRAM)、同步链路(Synchlink) DRAM(SLDRAM)、存储器总线(Rambus)直接RAM(RDRAM)、直接存储器总线动态RAM(DRDRAM)、以及存储器总线动态RAM(RDRAM)等。
以上实施例的各技术特征可以进行任意的组合,为使描述简洁,未对上述实施例中的各个技术特征所有可能的组合都进行描述,然而,只要这些技术特征的组合不存在矛盾,都应当认为是本说明书记载的范围。
以上所述实施例仅表达了本申请的几种实施方式,其描述较为具体和详细,但并不能因此而理解为对本申请专利范围的限制。应当指出的是,对于本领域的普通技术人员来说,在不脱离本申请构思的前提下,还可以做出若干变形和改进,这些都属于本申请的保护范围。因此,本申请专利的保护范围应以所附权利要求为准。
Claims (10)
- 一种对话语料中实体的识别方法,其特征在于,所述方法包括:获取待识别实体的语料文本;将所述语料文本进行分词,得到分词结果,所述分词结果中包含多个字;获取所述分词结果中的每个字对应的字向量,将所述每个字对应的字向量进行组合得到所述语料文本对应的文本矩阵;将所述文本矩阵作为实体识别模型的输入,获取所述实体识别模型输出的所述语料文本中的实体。
- 如权利要求1所述的实体识别方法,其特征在于,在所述获取待识别实体的语料文本之前,还包括:获取语料文本训练样本集,所述语料文本训练样本集包括多个语料文本训练样本,所述语料文本训练样本包括口语化的口语语料文本训练样本和对所述口语语料文本训练样本进行语义联想的联想语料文本训练样本;根据所述语料文本训练样本集对所述实体识别模型进行训练,得到所述实体识别模型。
- 如权利要求2所述的方法,其特征在于,所述根据所述语料文本训练样本集对所述实体识别模型进行训练,得到所述实体识别模型,包括:将所述语料文本训练样本集中的各个所述语料文本训练样本进行分词,得到每个所述语料文本训练样本的包含多个字的分词结果;根据字向量查找表和每个所述语料文本训练样本的分词结果,得到与所述语料文本训练样本集对应的训练文本矩阵;获取每个所述语料文本训练样本中每个字对应的标注,得到所述语料文本训练样本集对应的训练文本标注矩阵,所述标注用于区分实体和非实体;将所述训练文本矩阵作为实体识别模型的输入,将对应的所述训练文本标注矩阵作为所述实体识别模型的输出,对所述实体识别模型进行训练,得到目标实体识别模型。
- 如权利要求2所述的方法,其特征在于,所述语料文本训练样本的样本类型包括命令型、情感型、名字型和动作型,所述根据所述语料文本训练样本集对所述实体识别模型进行训练,得到所述实体识别模型,包括:获取命令型语料文本训练样本、情感型语料文本训练样本、名字型语料文本训练样本和动作型语料文本训练样本的训练比例;根据所述命令型语料文本训练样本、所述情感型语料文本训练样本、所述名字型语料文本训练样本和所述动作型语料文本训练样本的训练比例,从所述语料文本训练样本集中获取对应数量的语料文本训练样本;根据获取的对应数量的语料文本训练样本,对所述实体识别模型进行训练,得到所述实体识别模型。
- 如权利要求1至4任一项所述的方法,其特征在于,在所述将所述文本矩阵作为实体识别模型的输入,获取所述实体识别模型输出的所述语料文本中的实体之后,还包括:到实体库中查找是否存在所述实体;若所述实体库中存在所述实体,则所述实体为可信实体;若所述实体库中不存在所述实体,则所述实体为可疑实体。
- 如权利要求5所述的方法,其特征在于,所述实体库包括命令型实体库、情感型实体库、名字型实体库和动作型实体库,在所述若所述实体库中存在所述实体,则所述实体为可信实体之后,还包括:根据所述实体所在的实体库的类型确定所述实体的实体类型;获取与所述实体类型对应的答复模板,以在所述答复模板中查找答复结果。
- 如权利要求1至4任一项所述的方法,其特征在于,所述将所述文本矩阵作为实体识别模型的输入,获取所述实体识别模型输出的所述语料文本中的实体,包括:将所述文本矩阵作为实体识别模型的输入,得到所述语料文本中的实体和非实体的位置分布信息;根据所述位置分布信息,得到所述语料文本中的实体。
- 一种对话语料中实体的识别装置,其特征在于,所述装置包括:第一获取模块,用于获取待识别实体的语料文本;文本分词模块,用于将所述语料文本进行分词,得到分词结果,所述分词结果中包含多个字;第二获取模块,用于获取所述分词结果中的每个字对应的字向量,将所述每个字对应的字向量进行组合得到所述语料文本对应的文本矩阵;第三获取模块,用于将所述文本矩阵作为实体识别模型的输入,获取所述实体识别模型输出的所述语料文本中的实体。
- 一种计算机设备,包括存储器和处理器,所述存储器存储有计算机程序,所述计算机程序被所述处理器执行时,使得所述处理器执行如权利要求1至7中任一项所述方法的步骤。
- 一种计算机可读存储介质,存储有计算机程序,所述计算机程序被处理器执行时,使得所述处理器执行如权利要求1至7中任一项所述方法的步骤。
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/CN2018/124239 WO2020133039A1 (zh) | 2018-12-27 | 2018-12-27 | 对话语料中实体的识别方法、装置和计算机设备 |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/CN2018/124239 WO2020133039A1 (zh) | 2018-12-27 | 2018-12-27 | 对话语料中实体的识别方法、装置和计算机设备 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2020133039A1 true WO2020133039A1 (zh) | 2020-07-02 |
Family
ID=71125878
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2018/124239 WO2020133039A1 (zh) | 2018-12-27 | 2018-12-27 | 对话语料中实体的识别方法、装置和计算机设备 |
Country Status (1)
Country | Link |
---|---|
WO (1) | WO2020133039A1 (zh) |
Cited By (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112115721A (zh) * | 2020-09-28 | 2020-12-22 | 青岛海信网络科技股份有限公司 | 一种命名实体识别方法及装置 |
CN112131357A (zh) * | 2020-08-21 | 2020-12-25 | 国网浙江省电力有限公司杭州供电公司 | 一种基于智能对话模型的用户意图识别方法及装置 |
CN112182243A (zh) * | 2020-09-27 | 2021-01-05 | 中国平安财产保险股份有限公司 | 基于实体识别模型构建知识图谱的方法、终端及存储介质 |
CN112287683A (zh) * | 2020-08-19 | 2021-01-29 | 北京沃东天骏信息技术有限公司 | 命名实体的识别方法和装置 |
CN112906381A (zh) * | 2021-02-02 | 2021-06-04 | 北京有竹居网络技术有限公司 | 对话归属的识别方法、装置、可读介质和电子设备 |
CN112906380A (zh) * | 2021-02-02 | 2021-06-04 | 北京有竹居网络技术有限公司 | 文本中角色的识别方法、装置、可读介质和电子设备 |
CN112966100A (zh) * | 2020-12-30 | 2021-06-15 | 北京明朝万达科技股份有限公司 | 一种数据分类分级模型的训练方法、装置及电子设备 |
CN113095085A (zh) * | 2021-03-30 | 2021-07-09 | 北京达佳互联信息技术有限公司 | 文本的情感识别方法、装置、电子设备和存储介质 |
CN113128196A (zh) * | 2021-05-19 | 2021-07-16 | 腾讯科技(深圳)有限公司 | 文本信息处理方法及其装置、存储介质 |
CN113239663A (zh) * | 2021-03-23 | 2021-08-10 | 国家计算机网络与信息安全管理中心 | 一种基于知网的多义词中文实体关系识别方法 |
CN113268452A (zh) * | 2021-05-25 | 2021-08-17 | 联仁健康医疗大数据科技股份有限公司 | 一种实体抽取的方法、装置、设备和存储介质 |
CN113591480A (zh) * | 2021-07-23 | 2021-11-02 | 深圳供电局有限公司 | 电力计量的命名实体识别方法、装置和计算机设备 |
WO2022213864A1 (zh) * | 2021-04-06 | 2022-10-13 | 华为云计算技术有限公司 | 一种语料标注方法、装置及相关设备 |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107391575A (zh) * | 2017-06-20 | 2017-11-24 | 浙江理工大学 | 一种基于词向量模型的隐式特征识别方法 |
US20170364503A1 (en) * | 2016-06-17 | 2017-12-21 | Abbyy Infopoisk Llc | Multi-stage recognition of named entities in natural language text based on morphological and semantic features |
CN108268447A (zh) * | 2018-01-22 | 2018-07-10 | 河海大学 | 一种藏文命名实体的标注方法 |
CN108460012A (zh) * | 2018-02-01 | 2018-08-28 | 哈尔滨理工大学 | 一种基于gru-crf的命名实体识别方法 |
-
2018
- 2018-12-27 WO PCT/CN2018/124239 patent/WO2020133039A1/zh active Application Filing
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20170364503A1 (en) * | 2016-06-17 | 2017-12-21 | Abbyy Infopoisk Llc | Multi-stage recognition of named entities in natural language text based on morphological and semantic features |
CN107391575A (zh) * | 2017-06-20 | 2017-11-24 | 浙江理工大学 | 一种基于词向量模型的隐式特征识别方法 |
CN108268447A (zh) * | 2018-01-22 | 2018-07-10 | 河海大学 | 一种藏文命名实体的标注方法 |
CN108460012A (zh) * | 2018-02-01 | 2018-08-28 | 哈尔滨理工大学 | 一种基于gru-crf的命名实体识别方法 |
Cited By (21)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112287683A (zh) * | 2020-08-19 | 2021-01-29 | 北京沃东天骏信息技术有限公司 | 命名实体的识别方法和装置 |
CN112131357A (zh) * | 2020-08-21 | 2020-12-25 | 国网浙江省电力有限公司杭州供电公司 | 一种基于智能对话模型的用户意图识别方法及装置 |
CN112182243A (zh) * | 2020-09-27 | 2021-01-05 | 中国平安财产保险股份有限公司 | 基于实体识别模型构建知识图谱的方法、终端及存储介质 |
CN112182243B (zh) * | 2020-09-27 | 2023-11-28 | 中国平安财产保险股份有限公司 | 基于实体识别模型构建知识图谱的方法、终端及存储介质 |
CN112115721A (zh) * | 2020-09-28 | 2020-12-22 | 青岛海信网络科技股份有限公司 | 一种命名实体识别方法及装置 |
CN112115721B (zh) * | 2020-09-28 | 2024-05-17 | 青岛海信网络科技股份有限公司 | 一种命名实体识别方法及装置 |
CN112966100A (zh) * | 2020-12-30 | 2021-06-15 | 北京明朝万达科技股份有限公司 | 一种数据分类分级模型的训练方法、装置及电子设备 |
CN112966100B (zh) * | 2020-12-30 | 2022-05-31 | 北京明朝万达科技股份有限公司 | 一种数据分类分级模型的训练方法、装置及电子设备 |
CN112906381A (zh) * | 2021-02-02 | 2021-06-04 | 北京有竹居网络技术有限公司 | 对话归属的识别方法、装置、可读介质和电子设备 |
CN112906381B (zh) * | 2021-02-02 | 2024-05-28 | 北京有竹居网络技术有限公司 | 对话归属的识别方法、装置、可读介质和电子设备 |
CN112906380A (zh) * | 2021-02-02 | 2021-06-04 | 北京有竹居网络技术有限公司 | 文本中角色的识别方法、装置、可读介质和电子设备 |
CN113239663A (zh) * | 2021-03-23 | 2021-08-10 | 国家计算机网络与信息安全管理中心 | 一种基于知网的多义词中文实体关系识别方法 |
CN113239663B (zh) * | 2021-03-23 | 2022-07-12 | 国家计算机网络与信息安全管理中心 | 一种基于知网的多义词中文实体关系识别方法 |
CN113095085A (zh) * | 2021-03-30 | 2021-07-09 | 北京达佳互联信息技术有限公司 | 文本的情感识别方法、装置、电子设备和存储介质 |
CN113095085B (zh) * | 2021-03-30 | 2024-04-19 | 北京达佳互联信息技术有限公司 | 文本的情感识别方法、装置、电子设备和存储介质 |
WO2022213864A1 (zh) * | 2021-04-06 | 2022-10-13 | 华为云计算技术有限公司 | 一种语料标注方法、装置及相关设备 |
CN113128196A (zh) * | 2021-05-19 | 2021-07-16 | 腾讯科技(深圳)有限公司 | 文本信息处理方法及其装置、存储介质 |
CN113268452A (zh) * | 2021-05-25 | 2021-08-17 | 联仁健康医疗大数据科技股份有限公司 | 一种实体抽取的方法、装置、设备和存储介质 |
CN113268452B (zh) * | 2021-05-25 | 2024-02-02 | 联仁健康医疗大数据科技股份有限公司 | 一种实体抽取的方法、装置、设备和存储介质 |
WO2023000725A1 (zh) * | 2021-07-23 | 2023-01-26 | 深圳供电局有限公司 | 电力计量的命名实体识别方法、装置和计算机设备 |
CN113591480A (zh) * | 2021-07-23 | 2021-11-02 | 深圳供电局有限公司 | 电力计量的命名实体识别方法、装置和计算机设备 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2020133039A1 (zh) | 对话语料中实体的识别方法、装置和计算机设备 | |
US11238232B2 (en) | Written-modality prosody subsystem in a natural language understanding (NLU) framework | |
US20210157984A1 (en) | Intelligent system that dynamically improves its knowledge and code-base for natural language understanding | |
WO2021042503A1 (zh) | 信息分类抽取方法、装置、计算机设备和存储介质 | |
JP6909832B2 (ja) | オーディオにおける重要語句を認識するための方法、装置、機器及び媒体 | |
CN109670163B (zh) | 信息识别方法、信息推荐方法、模板构建方法及计算设备 | |
CN107992585B (zh) | 通用标签挖掘方法、装置、服务器及介质 | |
US20200193217A1 (en) | Method for determining sentence similarity | |
US11567812B2 (en) | Utilizing a natural language model to determine a predicted activity event based on a series of sequential tokens | |
WO2021051866A1 (zh) | 判案结果确定方法、装置、设备及计算机可读存储介质 | |
WO2016197767A2 (zh) | 一种表情输入方法、装置、终端和计算机可读存储介质 | |
US20170308526A1 (en) | Compcuter Implemented machine translation apparatus and machine translation method | |
CN111191032B (zh) | 语料扩充方法、装置、计算机设备和存储介质 | |
US11429792B2 (en) | Creating and interacting with data records having semantic vectors and natural language expressions produced by a machine-trained model | |
WO2022179149A1 (zh) | 基于翻译记忆库的机器翻译方法及装置 | |
CN113343108B (zh) | 推荐信息处理方法、装置、设备及存储介质 | |
CN112016271A (zh) | 语言风格转换模型的训练方法、文本处理方法以及装置 | |
WO2022257452A1 (zh) | 表情回复方法、装置、设备及存储介质 | |
CN112632258A (zh) | 文本数据处理方法、装置、计算机设备和存储介质 | |
CN114449310A (zh) | 视频剪辑方法、装置、计算机设备及存储介质 | |
CN110633475A (zh) | 基于计算机场景的自然语言理解方法、装置、系统和存储介质 | |
CN111276149A (zh) | 语音识别方法、装置、设备及可读存储介质 | |
CN111859950A (zh) | 一种自动化生成讲稿的方法 | |
WO2021134416A1 (zh) | 文本转换方法、装置、计算机设备和计算机可读存储介质 | |
CN117112595A (zh) | 一种信息查询方法、装置、电子设备及存储介质 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 18945099 Country of ref document: EP Kind code of ref document: A1 |