CN111259126A - Method, Apparatus, Equipment and Storage Medium for Similarity Calculation Based on Word Features - Google Patents

Abstract

The invention relates to the field of artificial intelligence, and discloses a word feature-based similarity calculation method, device, equipment and storage medium, which are used to improve the calculation accuracy of text similarity values in specific business scenarios. The method of the invention includes: obtaining original question text; determining a target application scene according to the original question text and a preset application scene, and obtaining a target word segmentation standard corresponding to the target application scene and a plurality of similar question texts with similar semantics; Select any similar question text from the similar question texts as candidate question text, and extract the word features of the original question text and the word features of the candidate question text according to the target word segmentation standard; get the forward text similarity and reverse text similarity; generate Similarity matching score; the candidate similarity value with the largest value is determined as the target similarity value, and the candidate question text corresponding to the target similarity value is selected as the standard question text.

Description

Method, Apparatus, Equipment and Storage Medium for Similarity Calculation Based on Word Features

technical field

The present invention relates to the technical field of similarity matching, and in particular, to a method, apparatus, device and storage medium for calculating similarity based on word features.

Background technique

In the traditional customer service system or training system, it is often necessary to invest a lot of manpower and resources to respond to business requests, and the professionalism and proficiency of the staff are required to be high, the operating cost remains high, and the manpower retention rate is low. Therefore, in the process of intelligent transformation, the demand for intelligent question answering system is imminent.

At present, the mainstream way of building an intelligent question answering system in the industry is based on the retrieval method, that is, the most similar questions to the user's questions are recalled from the knowledge base of the question answering system, and the text similarity calculation is the core module in the retrieval type recall. The existing text similarity calculation methods mainly include deep learning model method, single-level literal matching calculation (such as keyword matching, edit distance, Jaccard similarity, etc.). Among them, the semantic representation method based on the deep learning model has a large demand for the sample size of the data, and iterates slowly for the newly added question database corpus, which is not easy to expand, while the single-level literal matching method essentially sets each token ( The weight of words/characters) is the same, which cannot effectively reflect the contribution of words at different levels in specific business scenarios, thus affecting the measurement of text similarity in specific business scenarios.

SUMMARY OF THE INVENTION

The invention provides a similarity calculation method, device, equipment and storage medium based on word features, which are used to reflect the type of words that need to be preferentially matched in a specific business scenario, reflect the semantic inclusion relationship of text, and improve the text similarity in the specific business scenario. The calculation accuracy of the value.

A first aspect of the embodiments of the present invention provides a method for calculating similarity based on word features, including: obtaining an original question text, where the original question text is used to instruct to find an answer corresponding to the original question text; The text and the preset application scene determine the target application scene, and obtain the target word segmentation standard corresponding to the target application scene and a plurality of similar question texts with similar semantics, and the preset application scene includes a plurality of preset candidate scenes; Select any one of the similar question texts with similar semantics as the candidate question text, and extract the word feature of the original question text and the word feature of the candidate question text according to the target word segmentation standard; The word features of the original question text and the word features of the candidate question text are calculated respectively to obtain the forward text similarity and the reverse text similarity; the forward text similarity and the reverse text similarity are calculated. Perform feature fusion to generate a similarity matching score, the similarity matching score is used to indicate the degree of similarity between the original question text and the candidate question text; the candidates corresponding to the multiple candidate question texts are similar The candidate similarity value with the largest numerical value is determined as the target similarity value, and the candidate question text corresponding to the target similarity value is selected as the standard question text.

Optionally, in the first implementation manner of the first aspect of the embodiment of the present invention, the target application scene is determined according to the original question text and the preset application scene, and the target word segmentation corresponding to the target application scene is obtained. standard and multiple similar question texts with similar semantics, the preset application scenarios include multiple preset candidate scenarios, including: selecting any one of the preset application scenarios as the target application according to the original question text Scenario, the preset application scenario includes multiple preset application scenarios; obtain the target word segmentation standard corresponding to the target application scenario; search for a similar question text semantically similar to the original question text in the target application scenario .

Optionally, in the second implementation manner of the first aspect of the embodiment of the present invention, selecting any one of the similar question texts with similar semantics as the candidate question text, and selecting the The word segmentation standard extracts the word features of the original question text and the word features of the candidate question text, including: selecting any one of the similar question texts with similar semantics as the candidate question text; based on the target The word segmentation standard performs word segmentation and named entity recognition on the original question text and the candidate question text, respectively, to obtain word features of the original question text and word features of the candidate question text.

Optionally, in a third implementation manner of the first aspect of the embodiment of the present invention, the original question text and the candidate question text are respectively subjected to word segmentation and named entity recognition based on the target word segmentation criteria, to obtain the result. Describe the word features of the original question text and the word features of the candidate question text, including: performing word segmentation on the original question text based on the target word segmentation standard to obtain a word segmentation result of the original question text; Perform word segmentation on the question text to obtain the word segmentation result of the candidate question text; perform named entity recognition on the word segmentation result of the original question text and the word segmentation result of the candidate question text respectively, and obtain the word features of the original question text and the words of the candidate question text. The word features of the original question text include the marked original words and the corresponding original word parts of speech, and the word features of the candidate question text include the marked candidate words and the corresponding candidate word parts of speech.

Optionally, in a fourth implementation manner of the first aspect of the embodiment of the present invention, the original question text and the candidate question text are respectively subjected to word segmentation and named entity recognition based on the target word segmentation criteria, to obtain the result. Describe the word features of the original question text and the word features of the candidate question text, including: performing word segmentation on the original question text based on the target word segmentation standard to obtain the word segmentation result of the original question text; obtaining the preset word segmentation of the candidate question text As a result, the preset word segmentation result is the result of performing offline word segmentation on the candidate question text in advance according to the target word segmentation standard; the word segmentation result of the original question text and the preset word segmentation result of the candidate question text are separately performed. Named entity recognition to obtain word features of the original question text and word features of the candidate question text, where the word features of the original question text include the labeled original words and the corresponding original word parts of speech, and the word features of the candidate question text include the labeled original words A good candidate word and the corresponding candidate word part-of-speech.

Optionally, in a fifth implementation manner of the first aspect of the embodiment of the present invention, the calculation is performed separately according to the word feature of the original question text and the word feature of the candidate question text to obtain the forward text similarity. and reverse text similarity, including:

其中A表示基准问题文本，B表示匹 配问题文本，L_A表示基准问题文本A的词语token个数，w_A,i表示基准问题文 本A中所有层次的token归一化后的权重，token_A,i表示基准问题文本对应下标 的token、token_B,j表示匹配问题文本对应下标的token，jaccard表示两个token的 相似度系数，

将候选问题文本确定为 基准问题文本，将原始问题文本确定为匹配问题文本，并基于预置匹配公式 计算得到反向文本相似度。The original question text is determined as the reference question text, the candidate question text is determined as the matching question text, and the forward text similarity is calculated based on the preset matching formula. The preset matching formula is

Among them, A represents the reference question text, B represents the matching question text, L _A represents the number of word tokens in the benchmark question text A, w _{A, i} represents the normalized weight of the tokens of all levels in the benchmark question text A, token _{A, i} represents the subscript token corresponding to the reference question text, token _{B, j} represents the subscript token corresponding to the matching question text, jaccard represents the similarity coefficient of the two tokens,

The candidate question text is determined as the reference question text, the original question text is determined as the matching question text, and the reverse text similarity is calculated based on the preset matching formula.

Optionally, in the sixth implementation manner of the first aspect of the embodiment of the present invention, the feature fusion is performed on the forward text similarity and the reverse text similarity to generate a similarity matching score, so The similarity matching score is used to indicate the degree of similarity between the original question text and the candidate question text, including: the forward text similarity and the reverse text similarity are fused by a preset formula, and the preset formula It is: score=w1*score (forward)+w2*score (reverse)+b, where b is a constant, and w1 and w2 are weight constants; the similarity matching score score is calculated, and the similarity matching score is The value indicates the degree of similarity between the original question text and the candidate question text.

A second aspect of the embodiments of the present invention provides an apparatus for calculating similarity based on word features, including: an acquiring unit, configured to acquire original question text, where the original question text is used to instruct to find an answer corresponding to the original question text A determination unit for determining a target application scene according to the original question text and a preset application scene, and acquiring the target word segmentation standard corresponding to the target application scene and a plurality of similar question texts with similar semantics, the preset application scene The application scene includes a plurality of preset candidate scenes; the selection and extraction unit is used to select any one of the similar question texts with similar semantics as the candidate question text, and extract the The word feature of the original question text and the word feature of the candidate question text; the computing unit is used to calculate respectively according to the word feature of the original question text and the word feature of the candidate question text to obtain the forward text similarity and Reverse text similarity; a generating unit for feature fusion of the forward text similarity and the reverse text similarity to generate a similarity matching score, and the similarity matching score is used to indicate the The similarity between the original question text and the candidate question text; the comparison selection unit is used to compare the candidate similarity values corresponding to the multiple candidate question texts, and determine the candidate similarity value with the largest numerical value as the target similarity degree value, and select the candidate question text corresponding to the target similarity value as the standard question text.

Optionally, in the first implementation manner of the second aspect of the embodiment of the present invention, the determining unit is specifically configured to: select any one of the application scenarios from the preset application scenarios as the target application scenario according to the original question text. The preset application scenario includes a plurality of preset application scenarios; the target word segmentation standard corresponding to the target application scenario is obtained; and the similar question text that is semantically similar to the original question text is searched for in the target application scenario.

Optionally, in a second implementation manner of the second aspect of the embodiment of the present invention, the selection and extraction unit includes: a selection module, configured to select any one of the similar question texts with similar semantics as a candidate. question text; a word segmentation recognition module for performing word segmentation and named entity recognition on the original question text and the candidate question text based on the target word segmentation criteria, respectively, to obtain word features of the original question text and the candidate question text. word features.

Optionally, in a third implementation manner of the second aspect of the embodiment of the present invention, the word segmentation recognition module is specifically configured to: perform word segmentation on the original question text based on the target word segmentation standard, and obtain a word segmentation result of the original question text; Perform word segmentation on the candidate question text based on the target word segmentation standard to obtain the word segmentation result of the candidate question text; perform named entity recognition on the word segmentation result of the original question text and the word segmentation result of the candidate question text respectively, and obtain the word segmentation result of the original question text. word features and word features of the candidate question text, the word features of the original question text include the labeled original words and the corresponding original word parts of speech, the word features of the candidate question text include the labeled candidate words and the corresponding candidate words part of speech.

Optionally, in the fourth implementation manner of the second aspect of the embodiment of the present invention, the word segmentation recognition is specifically further used to: perform word segmentation on the original question text based on the target word segmentation standard to obtain a word segmentation result of the original question text; Describe the preset word segmentation result of the candidate question text, wherein, the preset word segmentation result is the result of offline segmentation of the candidate question text in advance according to the target word segmentation standard; the word segmentation result of the original question text and the candidate question Named entity recognition is performed on the preset word segmentation results of the text to obtain the word features of the original question text and the word features of the candidate question text. The word features of the original question text include the marked original words and the corresponding original word parts of speech. The word features of the candidate question text include the labeled candidate words and the corresponding candidate word parts of speech.

Optionally, in a fifth implementation manner of the second aspect of the embodiment of the present invention, the computing unit is specifically configured to: determine the original question text as the reference question text, determine the candidate question text as the matching question text, and determine the matching question text based on the preset question text. The matching formula calculates the forward text similarity, and the preset matching formula is

其中A表示基准问题文本，B表示匹配问题文本，L_A表示基准问题文本A的词语token个数，w_A,i表示基准问题文 本A中所有层次的token归一化后的权重，token_A,i表示基准问题文本对应下标 的token、token_B,j表示匹配问题文本对应下标的token，jaccard表示两个token的 相似度系数，

将候选问题文本确定为 基准问题文本，将原始问题文本确定为匹配问题文本，并基于预置匹配公式 计算得到反向文本相似度。

Among them, A represents the reference question text, B represents the matching question text, L _A represents the number of word tokens in the benchmark question text A, w _{A, i} represents the normalized weight of the tokens of all levels in the benchmark question text A, token _{A, i} represents the subscript token corresponding to the reference question text, token _{B, j} represents the subscript token corresponding to the matching question text, jaccard represents the similarity coefficient of the two tokens,

The candidate question text is determined as the reference question text, the original question text is determined as the matching question text, and the reverse text similarity is calculated based on the preset matching formula.

Optionally, in the sixth implementation manner of the second aspect of the embodiment of the present invention, the generating unit is specifically configured to: fuse the forward text similarity and the reverse text similarity through a preset formula, and the preset formula is: score=w1*score (forward)+w2*score (reverse)+b, where b is a constant, and w1 and w2 are weight constants; the similarity matching score score is calculated, and the similarity matching score indicates The degree of similarity between the original question text and the candidate question text.

A third aspect of the embodiments of the present invention provides a word feature-based similarity computing device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor When the computer program is executed, the word feature-based similarity calculation method described in any of the above embodiments is implemented.

A fourth aspect of the embodiments of the present invention provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the system based on any of the foregoing embodiments is implemented. The steps of the similarity calculation method of word features.

In the technical solution provided by the embodiment of the present invention, the original question text is obtained, and the original question text is used to instruct to find the answer corresponding to the original question text; the target application scenario is determined according to the original question text and the preset application scenario, and obtain the target word segmentation standard corresponding to the target application scene and a plurality of similar question texts with similar semantics, and the preset application scene includes a plurality of preset candidate scenes; in the plurality of similar question texts with similar semantics Select any similar question text as candidate question text, and extract the word features of the original question text and the word features of the candidate question text according to the target word segmentation criteria; according to the word features of the original question text and the candidate The word features of the question text are calculated separately to obtain the forward text similarity and the reverse text similarity; the forward text similarity and the reverse text similarity are feature fusion to generate a similarity matching score, so The similarity matching score is used to indicate the similarity between the original question text and the candidate question text; the candidate similarity values corresponding to the multiple candidate question texts are compared, and the candidate similarity value with the largest value is compared. The value is determined as the target similarity value, and the candidate question text corresponding to the target similarity value is selected as the standard question text. In the embodiment of the present invention, the forward matching based on user questions and the reverse matching based on standard questions are used to calculate the forward text similarity and the reverse text similarity respectively, and the forward text similarity and the reverse text similarity are calculated. The final similarity calculation value is obtained by fusion, which reflects the type of words that need to be preferentially matched in a specific business scenario, reflects the semantic inclusion relationship of the text, and improves the calculation accuracy of the text similarity value in a specific business scenario.

Description of drawings

Fig. 1 is an embodiment schematic diagram of the similarity calculation method based on word feature in the embodiment of the present invention;

Fig. 2 is another embodiment schematic diagram of the similarity calculation method based on word feature in the embodiment of the present invention;

3 is a schematic diagram of an embodiment of a similarity computing device based on word features in an embodiment of the present invention;

4 is a schematic diagram of another embodiment of a similarity computing device based on word features in an embodiment of the present invention;

Fig. 5 is a schematic diagram of an embodiment of a similarity calculation device based on word features in an embodiment of the present invention.

Detailed ways

The invention provides a similarity calculation method, device, equipment and storage medium based on word features, which are used to reflect the type of words that need to be preferentially matched in a specific business scenario, reflect the semantic inclusion relationship of text, and improve the text similarity in the specific business scenario. The calculation accuracy of the value.

In order for those skilled in the art to better understand the solutions of the present invention, the embodiments of the present invention will be described below with reference to the accompanying drawings in the embodiments of the present invention.

The terms "first", "second", "third", "fourth", etc. (if any) in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It is to be understood that data so used may be interchanged under appropriate circumstances such that the embodiments described herein can be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those expressly listed steps or units, but may include other steps or units not expressly listed or inherent to these processes, methods, products or devices.

Referring to Fig. 1, a flowchart of a method for calculating similarity based on word features provided by an embodiment of the present invention specifically includes:

101. Obtain the original question text, where the original question text is used to instruct to find the answer corresponding to the original question text.

The server obtains the original question text, and the original question text is used to instruct to find the answer corresponding to the original question text.

It should be noted that, for an original question text, there may be multiple corresponding answers, for example, the original question text is: "What subjects are included in the physical examination?", the corresponding answer may include one or more of the following answers: "ENT Department", "Blood Routine", "Five Hepatitis B Tests", "Urine Routine Test", "ECG" or "B-ultrasound", etc., which are not specifically limited here.

It can be understood that the executive body of the present invention may be a similarity calculation device based on word features, and may also be a terminal or a server, which is not specifically limited here. The embodiment of the present invention is described by taking the server as the execution subject as an example.

102. Determine a target application scene according to the original question text and a preset application scene, and obtain a target word segmentation standard corresponding to the target application scene and a plurality of similar question texts with similar semantics. The preset application scene includes a plurality of preset candidate scenes .

Specifically, (1) according to the original question text, select any one of the application scenarios in the preset application scenarios as the target application scenario, and the preset application scenarios include a plurality of preset application scenarios;

(2) Obtain the target word segmentation standard corresponding to the target application scenario;

Different business scenarios have different word segmentation standards. Words in different business scenarios can be divided into multiple levels, and the weight of words at each level can be set. For example, in the practical application of Ping An Life's intelligent question answering system, the word segmentation criteria are six categories: insurance product entity, disease name entity, location name entity, occupation name entity, service operation keywords (such as "insurance", "claims") ), other words, and words in different categories have different weights.

(3) Find similar question texts that are semantically similar to the original question texts in the target application scenario.

Among them, there will be multiple similar question texts in the target application scenario. For example, in the life insurance intelligent question answering system, it may include: "Ping An Fu insurance", "Ping An Fu how to apply for insurance" and "Ping An Fu how to pay" and other questions, if The original question text is "I want to insure Ping An Fu", then both "Ping An Fu insurance" and "How to insure Ping An Fu" can be determined as similar standard question texts.

It should be noted that the same question may be expressed in different ways. For example, the original question text is: "What are the key points of pre-job training?" A similar question text with similar semantics to the original question text can be: "Pre-job training." The key content of training” or “What are the key points of pre-job training?”. For another example, the original question text is: "insurance for gastroenteritis", and the similar question text with similar semantics to the original question text is: "Can I insure for gastroenteritis?" or "insurance for acute gastroenteritis" or "Can I buy insurance for acute gastroenteritis? ?".

103. Select any one of the similar question texts with similar semantics as the candidate question text, and extract the word feature of the original question text and the word feature of the candidate question text according to the target word segmentation standard.

Specifically: the server selects any one of the similar question texts with similar semantics as the candidate question text; the server performs word segmentation and named entity recognition on the original question text and the candidate question text based on the target word segmentation criteria, respectively, to obtain the original question text. and the word features of the candidate question text.

The server performs word segmentation and named entity recognition on the original question text and the candidate question text based on the target word segmentation standard, respectively, and obtains the word features of the original question text and the word features of the candidate question text, which may include: Perform word segmentation to obtain the word segmentation result of the original question text; the server performs word segmentation on the candidate question text based on the target word segmentation standard, and obtains the word segmentation result of the candidate question text; the server names the entity result of the word segmentation result of the original question text and the word segmentation result of the candidate question text respectively. Identify and obtain the word features of the original question text and the word features of the candidate question text. The word features of the original question text include the marked original words and the corresponding original word parts of speech, and the word features of the candidate question text include the marked candidate words and corresponding original words. candidate word part-of-speech.

For example, for the original question text "Gastroenteritis insurance", select "Can gastroenteritis insurance be insured" as the candidate question text. It should be noted that, entity recognition, keyword recognition, etc. can be in the form of NER model or dictionary, which is not specifically limited here.

Alternatively, it may also include: the server performs word segmentation on the original question text based on the target word segmentation standard, and obtains a word segmentation result of the original question text; the server obtains a preset word segmentation result of the candidate question text, wherein the preset word segmentation result is based on the target word segmentation standard The result of offline word segmentation of the candidate question text; the server performs named entity recognition on the word segmentation result of the original question text and the preset word segmentation result of the candidate question text, respectively, to obtain the word features of the original question text and the word features of the candidate question text. The original question text The word features of , include the labeled original words and the corresponding original word parts of speech, and the word features of the candidate question text include the labeled candidate words and the corresponding candidate word parts of speech.

For example, the original question text is: "I want to insure Ping An Fu", and the candidate question text is: "Ping An Fu insurance", the original question text and the candidate question text are preprocessed, and some words can be used as stop words in practice. its removed. After the question text is preprocessed, the preset entity recognition model or keyword recognition model is used to obtain the word features of the original question text and the candidate question text at various levels. For example, the word features of the original question text are: (I, other words) (to, other words) (insurance, keywords) (Ping Anfu, insurance entity words). The word features of the candidate question text are: (Ping An Fu, insurance entity word) (insurance, keyword).

104. Calculate respectively according to the word feature of the original question text and the word feature of the candidate question text to obtain the forward text similarity and the reverse text similarity.

其中A表示基准问题文本，B表示 匹配问题文本，L_A表示基准问题文本A的词语token个数，w_A,i表示基准问题 文本A中所有层次的token归一化后的权重，token_A,i表示基准问题文本对应下 标的token、token_B,j表示匹配问题文本对应下标的token，jaccard表示两个token的 相似度系数，

服务器将候选问题文本 确定为基准问题文本，将原始问题文本确定为匹配问题文本，并基于预置匹 配公式计算得到反向文本相似度。The server determines the original question text as the reference question text, determines the candidate question text as the matching question text, and calculates the forward text similarity based on the preset matching formula. The preset matching formula is:

Among them, A represents the reference question text, B represents the matching question text, L _A represents the number of word tokens in the benchmark question text A, w _{A, i} represents the normalized weight of the tokens of all levels in the benchmark question text A, token _{A, i} represents the subscript token corresponding to the reference question text, token _{B, j} represents the subscript token corresponding to the matching question text, jaccard represents the similarity coefficient of the two tokens,

The server determines the candidate question text as the reference question text, determines the original question text as the matching question text, and calculates the reverse text similarity based on the preset matching formula.

For example, the original question text is: "I want to insure Ping An Fu", and the candidate question text is: "Ping An Fu insurance"; the word features of the original question text are: (I, other words) (want, other words) (insurance, Keywords) (Ping An Fu, insurance entity word); word features of the candidate question text are: (Ping An Fu, insurance entity word) (insurance, keyword). Assuming that the weight of entity words (including) is 3, the weight of keywords is 2, and the weight of other words is 1, then for the original question text A "I want to insure Ping An Fu" and the candidate question text B "Ping An Fu insurance" two The positive matching result of the text is: Score(positive)=1/(1+1+2+3)*max(jaccard(me, Ping Anfu), jaccard(me, insurance))+1/(1+1 +2+3)*max(jaccard(required, safe fortune), jaccard(required, insured))+2/(1+1+2+3)*max(jaccard(insured, safe fortune), jaccard(insured, Insurance))+3/(1+1+2+3)*max(jaccard(Ping An Fu, Ping An Fu), jaccard(Ping An Fu, Insurance))=1/8*0+1/8*0+2/ 8*1+3/8*1=5/8. The reverse matching result is: Score(reverse)=3/(3+2)*max(jaccard(Ping An Fu, I), jaccard( Ping An Fu, want), jaccard( Ping An Fu, insured), jaccard( Ping An Fu, insurance) , Ping An Fu))+2/(3+2)*max(jaccard(insurance, me), jaccard(insurance, to), jaccard(insurance, insurance), jaccard(insurance, Ping Anfu)) = 3/5+ 2/5=1.

105. Perform feature fusion on the forward text similarity and the reverse text similarity to generate a similarity matching score, and the similarity matching score is used to indicate the similarity between the original question text and the candidate question text.

The server performs feature fusion of the forward text similarity and the reverse text similarity to generate a similarity matching score, which is used to indicate the similarity between the original question text and the candidate question text. Specifically, the server fuses the forward text similarity and the reverse text similarity through a preset formula, and the preset formula is: score=w1*score(forward)+w2*score(reverse)+b, where, b is a constant, w1 and w2 are weight constants; the server calculates a similarity matching score score, and the similarity matching score indicates the degree of similarity between the original question text and the candidate question text.

For example, the original question text (user question) is: "Can stomach cancer be insured for Ping An Fu", the candidate question text 1 contained in the question database in the corresponding business scenario is: "Can gastric cancer be insured in Ping An Fu", and the candidate question text 2 is: "Can stomach cancer be insured with Ping An Fu and Love Full Score?" Assuming that the weights of feature fusion are: w1=0.6, w2=0.4, b=0, then it is calculated that the similarity matching score of candidate question text 1 is: score1 =0.6*(8/9)+0.4*(8/10)=0.853, the similarity matching score of candidate question text 2 is: score2=0.6*(8/9)+0.4*(8/14)=0.762 . Obviously, "Can gastric cancer be insured in Ping An Fu" and "Can gastric cancer be insured in Ping An Fu" are semantically equivalent, and "Can gastric cancer be insured in Ping An Fu" and "Can gastric cancer be insured in Ping An Fu and Love Full Score" are semantically different. Equivalent; therefore, for the life insurance intelligent question answering system, the similarity matching score of candidate question text 1 is greater than the similarity matching score of candidate question text 2. If there is only a positive match, the scores of the two candidate question texts are the same, both are 8/9, which is unreasonable, because the semantics of the candidate question text 2 contains the original question text, not the same as the original question text. price. At the same time, forward matching and reverse matching are considered, the text similarity matching algorithm is improved, and the problem of text semantic inclusion is optimized to a certain extent.

106. Compare the candidate similarity values corresponding to the multiple candidate question texts, determine the candidate similarity value with the largest value as the target similarity value, and select the candidate question text corresponding to the target similarity value as the standard question text.

The server compares the candidate similarity values corresponding to multiple candidate question texts, determines the candidate similarity value with the largest value as the target similarity value, and selects the candidate question text corresponding to the target similarity value as the standard question text.

For each original question text, there will be N candidate question texts. The server calculates the similarity between each candidate question text and the original question text, sorts them according to the similarity value, and finds the candidate question text with the highest similarity. and identified as the standard question text.

In the embodiment of the present invention, the forward matching based on user questions and the reverse matching based on standard questions are used to calculate the forward text similarity and the reverse text similarity respectively, and the forward text similarity and the reverse text similarity are calculated. The final similarity calculation value is obtained by fusion, which reflects the type of words that need to be preferentially matched in a specific business scenario, reflects the semantic inclusion relationship of the text, and improves the calculation accuracy of the text similarity value in a specific business scenario.

Please refer to Fig. 2, another flowchart of the similarity calculation method based on word feature provided by the embodiment of the present invention specifically includes:

201. Obtain the original question text, where the original question text is used to instruct to find the answer corresponding to the original question text.

The server obtains the original question text, and the original question text is used to instruct to find the answer corresponding to the original question text.

It should be noted that, for an original question text, there may be multiple corresponding answers, for example, the original question text is: "What subjects are included in the physical examination?", the corresponding answer may include one or more of the following answers: "ENT Department", "Blood Routine", "Five Hepatitis B Tests", "Urine Routine Test", "ECG" or "B-ultrasound", etc., which are not specifically limited here.

It can be understood that the executive body of the present invention may be a similarity calculation device based on word features, and may also be a terminal or a server, which is not specifically limited here. The embodiment of the present invention is described by taking the server as the execution subject as an example.

202. Determine a target application scene according to the original question text and a preset application scene, and obtain a target word segmentation standard corresponding to the target application scene and a plurality of similar question texts with similar semantics, and the preset application scene includes a plurality of preset candidate scenes .

Specifically, (1) according to the original question text, select any one of the application scenarios in the preset application scenarios as the target application scenario, and the preset application scenarios include a plurality of preset application scenarios;

(2) Obtain the target word segmentation standard corresponding to the target application scenario;

Different business scenarios have different word segmentation standards. Words in different business scenarios can be divided into multiple levels, and the weight of words at each level can be set. For example, in the practical application of Ping An Life's intelligent question answering system, the word segmentation criteria are six categories: insurance product entity, disease name entity, location name entity, occupation name entity, service operation keywords (such as "insurance", "claims") ), other words, and words in different categories have different weights.

(3) Find similar question texts that are semantically similar to the original question texts in the target application scenario.

Among them, there will be multiple similar question texts in the target application scenario. For example, in the life insurance intelligent question answering system, it may include: "Ping An Fu insurance", "Ping An Fu how to apply for insurance" and "Ping An Fu how to pay" and other questions, if The original question text is "I want to insure Ping An Fu", then both "Ping An Fu insurance" and "How to insure Ping An Fu" can be determined as similar standard question texts.

It should be noted that the same question may be expressed in different ways. For example, the original question text is: "What are the key points of pre-job training?" A similar question text with similar semantics to the original question text can be: "Pre-job training." The key content of training” or “What are the key points of pre-job training?”. For another example, the original question text is: "insurance for gastroenteritis", and the similar question text with similar semantics to the original question text is: "Can I insure for gastroenteritis?" or "insurance for acute gastroenteritis" or "Can I buy insurance for acute gastroenteritis? ?".

203. Select any one of the similar question texts with similar semantics as the candidate question text.

The server selects any one of the similar question texts with similar semantics as the candidate question text.

For example, for the original question text "insurance for gastroenteritis", similar question texts with similar semantics to the original question text are: "Can I insure for gastroenteritis" or "Insurance for acute gastroenteritis" or "Can I buy insurance for acute gastroenteritis" Is it possible?", select "Can I insure gastroenteritis" as the candidate question text.

204. Perform word segmentation and named entity recognition on the original question text and the candidate question text based on the target word segmentation standard, respectively, to obtain word features of the original question text and word features of the candidate question text.

Specifically, the server performs word segmentation on the original question text based on the target word segmentation standard to obtain a word segmentation result of the original question text; the server performs word segmentation on the candidate question text based on the target word segmentation standard to obtain the word segmentation result of the candidate question text; the server performs word segmentation on the original question text The result and the word segmentation result of the candidate question text are respectively subjected to named entity recognition, and the word features of the original question text and the word features of the candidate question text are obtained. The word features of the original question text include the marked original words and the corresponding original words. The word features of the text include the marked candidate words and the corresponding part of speech of the candidate words.

Or, the server performs word segmentation on the original question text based on the target word segmentation standard, and obtains the word segmentation result of the original question text; the server obtains the preset word segmentation result of the candidate question text, wherein the preset word segmentation result is based on the target word segmentation standard. The result of offline word segmentation; the server performs named entity recognition on the word segmentation result of the original question text and the preset word segmentation result of the candidate question text, respectively, and obtains the word features of the original question text and the word features of the candidate question text. The word features of the original question text include: The labeled original words and the corresponding original word parts of speech, and the word features of the candidate question text include the labeled candidate words and the corresponding candidate word parts of speech.

For example, the original question text is: "I want to insure Ping An Fu", and the candidate question text is: "Ping An Fu insurance", the original question text and the candidate question text are preprocessed, and some words can be used as stop words in practice. its removed. After the question text is preprocessed, the preset entity recognition model or keyword recognition model is used to obtain the word features of the original question text and the candidate question text at various levels. For example, the word features of the original question text are: (I, other words) (to, other words) (insurance, keywords) (Ping Anfu, insurance entity words). The word features of the candidate question text are: (Ping An Fu, insurance entity word) (insurance, keyword).

205. Calculate respectively according to the word feature of the original question text and the word feature of the candidate question text to obtain the forward text similarity and the reverse text similarity.

其中A表示基准问题文本，B表示 匹配问题文本，L_A表示基准问题文本A的词语token个数，w_A,i表示基准问题 文本A中所有层次的token归一化后的权重，token_A,i表示基准问题文本对应下 标的token、token_B,j表示匹配问题文本对应下标的token，jaccard表示两个token的 相似度系数，

服务器将候选问题文本 确定为基准问题文本，将原始问题文本确定为匹配问题文本，并基于预置匹 配公式计算得到反向文本相似度。The server determines the original question text as the reference question text, determines the candidate question text as the matching question text, and calculates the forward text similarity based on the preset matching formula. The preset matching formula is:

Among them, A represents the reference question text, B represents the matching question text, L _A represents the number of word tokens in the benchmark question text A, w _{A, i} represents the normalized weight of the tokens of all levels in the benchmark question text A, token _{A, i} represents the subscript token corresponding to the reference question text, token _{B, j} represents the subscript token corresponding to the matching question text, jaccard represents the similarity coefficient of the two tokens,

The server determines the candidate question text as the reference question text, determines the original question text as the matching question text, and calculates the reverse text similarity based on the preset matching formula.

For example, the original question text is: "I want to insure Ping An Fu", and the candidate question text is: "Ping An Fu insurance"; the word features of the original question text are: (I, other words) (want, other words) (insurance, Keywords) (Ping An Fu, insurance entity word); word features of the candidate question text are: (Ping An Fu, insurance entity word) (insurance, keyword). Assuming that the weight of entity words (including) is 3, the weight of keywords is 2, and the weight of other words is 1, then for the original question text A "I want to insure Ping An Fu" and the candidate question text B "Ping An Fu insurance" two The positive matching result of the text is: Score(positive)=1/(1+1+2+3)*max(jaccard(me, Ping Anfu), jaccard(me, insurance))+1/(1+1 +2+3)*max(jaccard(required, safe fortune), jaccard(required, insured))+2/(1+1+2+3)*max(jaccard(insured, safe fortune), jaccard(insured, Insurance))+3/(1+1+2+3)*max(jaccard(Ping An Fu, Ping An Fu), jaccard(Ping An Fu, Insurance))=1/8*0+1/8*0+2/ 8*1+3/8*1=5/8. The reverse matching result is: Score(reverse)=3/(3+2)*max(jaccard(Ping An Fu, I), jaccard( Ping An Fu, want), jaccard( Ping An Fu, insured), jaccard( Ping An Fu, insurance) , Ping An Fu))+2/(3+2)*max(jaccard(insurance, me), jaccard(insurance, to), jaccard(insurance, insurance), jaccard(insurance, Ping Anfu)) = 3/5+ 2/5=1.

206. Perform feature fusion on the forward text similarity and the reverse text similarity to generate a similarity matching score, where the similarity matching score is used to indicate the degree of similarity between the original question text and the candidate question text.

The server performs feature fusion of the forward text similarity and the reverse text similarity to generate a similarity matching score, which is used to indicate the similarity between the original question text and the candidate question text. Specifically, the server fuses the forward text similarity and the reverse text similarity through a preset formula, and the preset formula is: score=w1*score(forward)+w2*score(reverse)+b, where, b is a constant, w1 and w2 are weight constants; the server calculates a similarity matching score score, and the similarity matching score indicates the degree of similarity between the original question text and the candidate question text.

For example, the original question text (user question) is: "Can stomach cancer be insured for Ping An Fu", the candidate question text 1 contained in the question database in the corresponding business scenario is: "Can gastric cancer be insured in Ping An Fu", and the candidate question text 2 is: "Can stomach cancer be insured with Ping An Fu and Love Full Score?" Assuming that the weights of feature fusion are: w1=0.6, w2=0.4, b=0, then it is calculated that the similarity matching score of candidate question text 1 is: score1 =0.6*(8/9)+0.4*(8/10)=0.853, the similarity matching score of candidate question text 2 is: score2=0.6*(8/9)+0.4*(8/14)=0.762 . Obviously, "Can gastric cancer be insured in Ping An Fu" and "Can gastric cancer be insured in Ping An Fu" are semantically equivalent, and "Can gastric cancer be insured in Ping An Fu" and "Can gastric cancer be insured in Ping An Fu and Love Full Score" are semantically different. Equivalent; therefore, for the life insurance intelligent question answering system, the similarity matching score of candidate question text 1 is greater than the similarity matching score of candidate question text 2. If there is only a positive match, the scores of the two candidate question texts are the same, both are 8/9, which is unreasonable, because the semantics of the candidate question text 2 contains the original question text, not the same as the original question text. price. At the same time, forward matching and reverse matching are considered, the text similarity matching algorithm is improved, and the problem of text semantic inclusion is optimized to a certain extent.

207. Compare the candidate similarity values corresponding to the multiple candidate question texts, determine the candidate similarity value with the largest value as the target similarity value, and select the candidate question text corresponding to the target similarity value as the standard question text.

The server compares the candidate similarity values corresponding to multiple candidate question texts, determines the candidate similarity value with the largest value as the target similarity value, and selects the candidate question text corresponding to the target similarity value as the standard question text.

For each original question text, there will be N candidate question texts. The server calculates the similarity between each candidate question text and the original question text, sorts them according to the similarity value, and finds the candidate question text with the highest similarity. and identified as the standard question text.

In the embodiment of the present invention, the forward matching based on user questions and the reverse matching based on standard questions are used to calculate the forward text similarity and the reverse text similarity respectively, and the forward text similarity and the reverse text similarity are calculated. The final similarity calculation value is obtained by fusion, which reflects the type of words that need to be preferentially matched in a specific business scenario, reflects the semantic inclusion relationship of the text, and improves the calculation accuracy of the text similarity value in a specific business scenario.

The method for calculating similarity based on word features in the embodiment of the present invention has been described above. The following describes the device for calculating similarity based on word features in the embodiment of the present invention. Please refer to FIG. 3 . One embodiment of a degree computing device includes:

Obtaining unit 301, for obtaining original question text, and described original question text is used for instructing to find the answer corresponding to described original question text;

The determining unit 302 is configured to determine a target application scene according to the original question text and a preset application scene, and obtain a target word segmentation standard corresponding to the target application scene and a plurality of similar question texts with similar semantics. The application scenario includes multiple preset candidate scenarios;

The selection and extraction unit 303 is configured to select any one of the similar question texts with similar semantics as the candidate question text, and extract the word feature of the original question text and the candidate question text according to the target word segmentation standard word features of the question text;

The calculation unit 304 is used to calculate respectively according to the word feature of the original question text and the word feature of the candidate question text to obtain the forward text similarity and the reverse text similarity;

The generating unit 305 is configured to perform feature fusion between the forward text similarity and the reverse text similarity to generate a similarity matching score, and the similarity matching score is used to indicate that the original question text is the same as the original question text. Describe the degree of similarity between candidate question texts;

The comparison and selection unit 306 is configured to compare the candidate similarity values corresponding to the plurality of candidate question texts, determine the candidate similarity value with the largest numerical value as the target similarity value, and select the candidate corresponding to the target similarity value Question text as standard question text.

In the embodiment of the present invention, the forward matching based on user questions and the reverse matching based on standard questions are used to calculate the forward text similarity and the reverse text similarity respectively, and the forward text similarity and the reverse text similarity are calculated. The final similarity calculation value is obtained by fusion, which reflects the type of words that need to be preferentially matched in a specific business scenario, reflects the semantic inclusion relationship of the text, and improves the calculation accuracy of the text similarity value in a specific business scenario.

Referring to Fig. 4, another embodiment of the apparatus for calculating similarity based on word features in the embodiment of the present invention includes:

Obtaining unit 301, for obtaining original question text, and described original question text is used for instructing to find the answer corresponding to described original question text;

The determining unit 302 is configured to determine a target application scene according to the original question text and a preset application scene, and obtain a target word segmentation standard corresponding to the target application scene and a plurality of similar question texts with similar semantics. The application scenario includes multiple preset candidate scenarios;

The selection and extraction unit 303 is configured to select any one of the similar question texts with similar semantics as the candidate question text, and extract the word feature of the original question text and the candidate question text according to the target word segmentation standard word features of the question text;

The calculation unit 304 is used to calculate respectively according to the word feature of the original question text and the word feature of the candidate question text to obtain the forward text similarity and the reverse text similarity;

The generating unit 305 is configured to perform feature fusion between the forward text similarity and the reverse text similarity to generate a similarity matching score, and the similarity matching score is used to indicate that the original question text is the same as the original question text. Describe the degree of similarity between candidate question texts;

The comparison and selection unit 306 is configured to compare the candidate similarity values corresponding to the plurality of candidate question texts, determine the candidate similarity value with the largest numerical value as the target similarity value, and select the candidate corresponding to the target similarity value Question text as standard question text.

Optionally, the determining unit 302 is specifically configured to:

According to the original question text, any one of the preset application scenarios is selected as the target application scenario, and the preset application scenario includes a plurality of preset application scenarios; the target word segmentation standard corresponding to the target application scenario is obtained. ; Search for similar question texts that are semantically similar to the original question texts in the target application scenario.

Optionally, the selection extraction unit 303 includes:

The selection module 3031 is used to select any one of the similar question texts with similar semantics as the candidate question text;

A word segmentation recognition module 3032, configured to perform word segmentation and named entity recognition on the original question text and the candidate question text based on the target word segmentation criteria, respectively, to obtain word features of the original question text and words of the candidate question text feature.

Optionally, the word segmentation recognition module 3032 is specifically used for:

Perform word segmentation on the original question text based on the target word segmentation standard to obtain the word segmentation result of the original question text; perform word segmentation on the candidate question text based on the target word segmentation standard to obtain the word segmentation result of the candidate question text; The word segmentation result of the candidate question text and the word segmentation result of the candidate question text are respectively subjected to named entity recognition, and the word features of the original question text and the word features of the candidate question text are obtained. The word features of the original question text include the marked original words and corresponding The original word part of speech, the word features of the candidate question text include the marked candidate words and the corresponding candidate word parts of speech.

Optionally, the word segmentation recognition module 3032 is also specifically used for:

Perform word segmentation on the original question text based on the target word segmentation standard to obtain a word segmentation result of the original question text; obtain a preset word segmentation result of the candidate question text, wherein the preset word segmentation result is a pre-set word segmentation result according to the target word segmentation standard The result of offline word segmentation of the candidate question text; the named entity recognition is respectively performed on the word segmentation result of the original question text and the preset word segmentation result of the candidate question text, and the word features of the original question text and the word features of the candidate question text are obtained, The word features of the original question text include the marked original words and the corresponding original word parts of speech, and the word features of the candidate question text include the marked candidate words and the corresponding candidate word parts of speech.

Optionally, the computing unit 304 is specifically used for:

其中A表示基准问题文本，B表示匹 配问题文本，L_A表示基准问题文本A的词语token个数，w_A,i表示基准问题文 本A中所有层次的token归一化后的权重，token_A,i表示基准问题文本对应下标 的token、token_B,j表示匹配问题文本对应下标的token，jaccard表示两个token的 相似度系数，

将候选问题文本确定为 基准问题文本，将原始问题文本确定为匹配问题文本，并基于预置匹配公式 计算得到反向文本相似度。Determine the original question text as the reference question text, determine the candidate question text as the matching question text, and calculate the forward text similarity based on the preset matching formula. The preset matching formula is,

Among them, A represents the reference question text, B represents the matching question text, L _A represents the number of word tokens in the benchmark question text A, w _{A, i} represents the normalized weight of the tokens of all levels in the benchmark question text A, token _{A, i} represents the subscript token corresponding to the reference question text, token _{B, j} represents the subscript token corresponding to the matching question text, jaccard represents the similarity coefficient of the two tokens,

The candidate question text is determined as the reference question text, the original question text is determined as the matching question text, and the reverse text similarity is calculated based on the preset matching formula.

Optionally, the generating unit 305 is specifically used for:

The forward text similarity and reverse text similarity are fused by the preset formula. The preset formula is: score=w1*score(forward)+w2*score(reverse)+b, where b is a constant, w1 and w2 are weight constants; the similarity matching score score is calculated and obtained, and the similarity matching score indicates the degree of similarity between the original question text and the candidate question text.

In the embodiment of the present invention, the forward matching based on user questions and the reverse matching based on standard questions are used to calculate the forward text similarity and the reverse text similarity respectively, and the forward text similarity and the reverse text similarity are calculated. The final similarity calculation value is obtained by fusion, which reflects the type of words that need to be preferentially matched in a specific business scenario, reflects the semantic inclusion relationship of the text, and improves the calculation accuracy of the text similarity value in a specific business scenario.

Figures 3 to 4 above describe in detail the word feature-based similarity computing device in the embodiment of the present invention from the perspective of modular functional entities, and the following describes the word feature-based similarity computing device in the embodiment of the present invention from the perspective of hardware processing. Describe in detail.

FIG. 5 is a schematic structural diagram of a word feature-based similarity computing device provided by an embodiment of the present invention. The word feature-based similarity computing device 500 may vary greatly due to different configurations or performances, and may include one or more One or more central processing units (CPUs) 501 (eg, one or more processors) and memory 509, one or more storage media 508 (eg, one or more mass storage devices) that store applications 507 or data 506 ). Among them, the memory 509 and the storage medium 508 may be short-term storage or persistent storage. The program stored in the storage medium 508 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the word feature-based similarity computing device. Furthermore, the processor 501 may be configured to communicate with the storage medium 508 to execute a series of instruction operations in the storage medium 508 on the word feature-based similarity computing device 500.

The word feature-based similarity computing device 500 may also include one or more power supplies 502 , one or more wired or wireless network interfaces 503 , one or more input and output interfaces 504 , and/or, one or more operating systems 505 , such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art can understand that the structure of the word feature-based similarity computing device shown in FIG. 5 does not constitute a limitation to the word feature-based similarity computing device, and may include more or less components than those shown in the figure. Either some components are combined, or different component arrangements. The processor 501 may perform the functions of the acquisition unit 301, the determination unit 302, the selection extraction unit 303, the calculation unit 304, the generation unit 305, and the comparison selection unit 306 in the above embodiments.

Below in conjunction with Fig. 5, each constituent element of the similarity computing device based on word feature is specifically introduced:

The processor 501 is the control center of the word feature-based similarity calculation device, and can perform processing according to the set word feature-based similarity calculation method. The processor 501 uses various interfaces and lines to connect various parts of the entire word feature-based similarity computing device, and by running or executing the software programs and/or modules stored in the memory 509, and calling the data stored in the memory 509, Various functions and processing data of the similarity calculation device based on word features are performed, thereby improving the recall rate of keywords, improving the comprehensive score of each keyword, and improving the accuracy of keyword extraction. Both the storage medium 508 and the memory 509 are carriers for storing data. In this embodiment of the present invention, the storage medium 508 may refer to an internal memory with a small storage capacity but a fast speed, and the memory 509 may have a large storage capacity but a slow storage speed. of external memory.

The memory 509 can be used to store software programs and modules, and the processor 501 executes various functional applications and data processing of the word feature-based similarity computing device 500 by running the software programs and modules stored in the memory 509. The memory 509 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function (for example, calculation is performed according to the word feature of the original question text and the word feature of the candidate question text, respectively). , to obtain forward text similarity and reverse text similarity), etc.; the storage data area can store data (such as similarity matching scores, etc.) created according to the use of the similarity computing device based on word features. Additionally, memory 509 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. The program of the word feature-based similarity calculation method provided in the embodiment of the present invention and the received data stream are stored in the memory, and when needed, the processor 501 calls from the memory 509.

As mentioned above, the above embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand: The technical solutions described in the embodiments are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (10)

1. a similarity calculation method based on word feature, is characterized in that, comprises: Obtaining the original question text, the original question text is used for instructing to find the answer corresponding to the original question text; Determine a target application scenario according to the original question text and a preset application scenario, and obtain the target word segmentation standard corresponding to the target application scenario and a plurality of similar question texts with similar semantics, where the preset application scenario includes preset application scenarios. Multiple candidate scenarios; Selecting any one of the similar question texts with similar semantics as the candidate question text, and extracting the word feature of the original question text and the word feature of the candidate question text according to the target word segmentation standard; Calculate respectively according to the word feature of the original question text and the word feature of the candidate question text to obtain the forward text similarity and the reverse text similarity; The forward text similarity and the reverse text similarity are feature fusion to generate a similarity matching score, and the similarity matching score is used to indicate the difference between the original question text and the candidate question text. degree of similarity; The candidate similarity values corresponding to the plurality of candidate question texts are compared, the candidate similarity value with the largest numerical value is determined as the target similarity value, and the candidate question text corresponding to the target similarity value is selected as the standard question text. 2. The word feature-based similarity calculation method according to claim 1, wherein the target application scene is determined according to the original question text and a preset application scene, and the corresponding target application scene is obtained. The target word segmentation standard and multiple similar question texts with similar semantics, and the preset application scenarios include multiple preset candidate scenarios, including: According to the original question text, any one of the preset application scenarios is selected as the target application scenario, and the preset application scenario includes a plurality of preset application scenarios; Obtain the target word segmentation standard corresponding to the target application scenario; Search for similar question texts that are semantically similar to the original question texts in the target application scenario. 3. The word feature-based similarity calculation method according to claim 1, characterized in that, selecting any one similar question text from the plurality of semantically similar similar question texts as the candidate question text, and according to the Extract the word features of the original question text and the word features of the candidate question text using the target word segmentation criteria, including: selecting any one of the similar question texts with similar semantics as the candidate question text; Based on the target word segmentation criteria, word segmentation and named entity recognition are performed on the original question text and the candidate question text, respectively, to obtain word features of the original question text and word features of the candidate question text. 4. The similarity calculation method based on word features according to claim 3, wherein the original question text and the candidate question text are respectively subjected to word segmentation and named entity recognition based on the target word segmentation criteria, The word features of the original question text and the word features of the candidate question text are obtained, including: Perform word segmentation on the original question text based on the target word segmentation standard to obtain a word segmentation result of the original question text; Perform word segmentation on the candidate question text based on the target word segmentation standard to obtain a word segmentation result of the candidate question text; Perform named entity recognition on the word segmentation result of the original question text and the word segmentation result of the candidate question text, respectively, to obtain the word feature of the original question text and the word feature of the candidate question text. The word feature of the original question text includes the marked The original word and the corresponding original word part of speech, the word features of the candidate question text include the marked candidate word and the corresponding candidate word part of speech. 5. The similarity calculation method based on word features according to claim 3, wherein the original question text and the candidate question text are respectively subjected to word segmentation and named entity recognition based on the target word segmentation criteria, The word features of the original question text and the word features of the candidate question text are obtained, including: Perform word segmentation on the original question text based on the target word segmentation standard to obtain the word segmentation result of the original question text; Obtaining a preset word segmentation result of the candidate question text, wherein the preset word segmentation result is a result of performing offline word segmentation on the candidate question text in advance according to the target word segmentation standard; Perform named entity recognition on the word segmentation result of the original question text and the preset word segmentation result of the candidate question text, respectively, to obtain the word feature of the original question text and the word feature of the candidate question text, and the word feature of the original question text includes: The labeled original words and the corresponding original word parts of speech, and the word features of the candidate question text include the labeled candidate words and the corresponding candidate word parts of speech. 6. The word feature-based similarity calculation method according to claim 1, wherein the calculation is performed according to the word feature of the original question text and the word feature of the candidate question text to obtain a forward text Similarity and reverse text similarity, including: The original question text is determined as the reference question text, the candidate question text is determined as the matching question text, and the forward text similarity is calculated based on the preset matching formula. The preset matching formula is

Among them, A represents the reference question text, B represents the matching question text, L _A represents the number of word tokens in the benchmark question text A, w _{A, i} represents the normalized weight of the tokens of all levels in the benchmark question text A, token _{A, i} represents the subscript token corresponding to the reference question text, token _{B, j} represents the subscript token corresponding to the matching question text, jaccard represents the similarity coefficient of the two tokens,

The candidate question text is determined as the reference question text, the original question text is determined as the matching question text, and the reverse text similarity is calculated based on the preset matching formula. 7. The word feature-based similarity calculation method according to any one of claims 1-6, wherein the forward text similarity and the reverse text similarity are characterized by feature fusion, Generating a similarity matching score, the similarity matching score is used to indicate the degree of similarity between the original question text and the candidate question text, including: The forward text similarity and reverse text similarity are fused by the preset formula. The preset formula is: score=w1*score(forward)+w2*score(reverse)+b, where b is a constant, w1, w2 are weight constants; A similarity matching score score is calculated, and the similarity matching score indicates the degree of similarity between the original question text and the candidate question text. 8. A similarity computing device based on word features, comprising: an obtaining unit, used for obtaining the original question text, the original question text is used for instructing to find the answer corresponding to the original question text; a determining unit, configured to determine a target application scenario according to the original question text and a preset application scenario, and obtain a target word segmentation standard corresponding to the target application scenario and a plurality of similar question texts with similar semantics, and the preset application scenario The scene contains multiple preset candidate scenes; The selection and extraction unit is used to select any one of the similar question texts with similar semantics as the candidate question text, and extract the word feature of the original question text and the candidate question according to the target word segmentation standard word features of the text; a computing unit, configured to calculate respectively according to the word feature of the original question text and the word feature of the candidate question text to obtain the forward text similarity and the reverse text similarity; The generating unit is used for feature fusion of the forward text similarity and the reverse text similarity to generate a similarity matching score, and the similarity matching score is used to indicate that the original question text and the The degree of similarity between candidate question texts; A comparison and selection unit is used to compare the candidate similarity values corresponding to the multiple candidate question texts, determine the candidate similarity value with the largest numerical value as the target similarity value, and select the candidate question corresponding to the target similarity value Text as standard question text. 9. A word feature-based similarity computing device, comprising a memory, a processor, and a computer program stored on the memory and running on the processor, the processor executing the computer The program implements the word feature-based similarity calculation method according to any one of claims 1-7. 10. A computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the computer program according to any one of claims 1-7 is implemented. Similarity calculation method based on word features.

Images