CN111538894A

CN111538894A - Query feedback method and device, computer equipment and storage medium

Info

Publication number: CN111538894A
Application number: CN202010566762.3A
Authority: CN
Inventors: 赵瑞辉; 陆扩建; 赵博; 黄展鹏
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2020-06-19
Filing date: 2020-06-19
Publication date: 2020-08-14
Anticipated expiration: 2040-06-19
Also published as: CN111538894B

Abstract

The application relates to a query feedback method, a query feedback device, computer equipment and a storage medium. The method comprises the following steps: performing semantic feature recognition on the query statement to obtain a recognition result of the query statement; the recognition result comprises an original semantic entity and semantic information; acquiring at least two candidate semantic entities based on the recognition result of the query statement; screening a target semantic entity from at least two candidate semantic entities; sending a feedback result corresponding to the target semantic entity to the terminal; the scheme is based on an artificial intelligence technology, and the original semantic entities and the semantic information are combined to jointly query the candidate semantic entities, so that the sources of the subsequently determined target semantic entities are wider, the diversity of the target semantic entities is improved, the diversity of feedback results is improved, and the purpose of improving the feedback effect is achieved.

Description

Query feedback method and device, computer equipment and storage medium

Technical Field

The present application relates to the field of data retrieval technologies, and in particular, to a query feedback method, an apparatus, a computer device, and a storage medium.

Background

With the continuous development of search technology and its application, data query and result feedback based on natural sentences are the basic functions of each large search engine.

In the related art, a search engine generally implements data query and result feedback for natural sentences based on semantic entities. For example, for a query statement input by a user, an original semantic entity is identified from the query statement in a semantic identification mode, then a target semantic entity with high similarity is queried through the original semantic entity, and result feedback is performed.

However, the target semantic entities in the related art are obtained by querying the original semantic entities, so that the diversity of the queried target semantic entities is insufficient, the type of query results is single, and the query feedback effect is affected.

Disclosure of Invention

The embodiment of the application provides a query feedback method, a query feedback device, computer equipment and a storage medium, which can improve the diversity of queried target semantic entities, thereby improving the feedback effect for query statements.

In one aspect, a query feedback method is provided, and the method includes:

acquiring a query statement input in a terminal;

performing semantic feature recognition on the query statement to obtain a recognition result of the query statement; the recognition result comprises an original semantic entity and semantic information; the semantic information comprises at least one of a semantic tag and a sentence intent; the original semantic entities are semantic entities contained in corresponding sentences;

acquiring at least two candidate semantic entities based on the recognition result of the query statement;

screening a target semantic entity from the at least two candidate semantic entities based on the similarity between the at least two candidate semantic entities and a first original semantic entity contained in the recognition result of the query statement;

and sending a feedback result corresponding to the target semantic entity to the terminal.

In yet another aspect, a query feedback apparatus is provided, the apparatus comprising:

a query sentence acquisition module for acquiring a query sentence input in a terminal;

the semantic recognition module is used for carrying out semantic feature recognition on the query statement to obtain a recognition result of the query statement; the recognition result comprises an original semantic entity and semantic information; the semantic information comprises at least one of a semantic tag and a sentence intent; the original semantic entities are semantic entities contained in corresponding sentences;

a candidate entity obtaining module, configured to obtain at least two candidate semantic entities based on the recognition result of the query statement;

the target entity screening module is used for screening a target semantic entity from the at least two candidate semantic entities based on the similarity between the at least two candidate semantic entities and a first original semantic entity contained in the identification result of the query statement;

and the feedback module is used for sending a feedback result corresponding to the target semantic entity to the terminal.

In one possible implementation, the semantic recognition module is configured to,

processing the query statement through a semantic recognition model to obtain a recognition result output by the semantic recognition model;

the semantic recognition model is a multi-task learning model obtained by training query statement samples and sample labeling information corresponding to the query statement samples, and the sample labeling information is the same type of information as the recognition result.

In one possible implementation, the apparatus further includes:

the processing module is used for processing the query statement sample through the semantic recognition model in the semantic recognition module to obtain a prediction result of the query statement sample;

the loss calculation module is used for inputting the prediction result and the sample marking information into a loss function to obtain a loss function value;

the updating module is used for updating parameters in the semantic recognition model based on the loss function values;

wherein the loss function comprises a scaling coefficient, and the scaling coefficient is inversely related to a prediction probability, wherein the prediction probability is a probability that the semantic recognition model predicts that the query statement sample belongs to a positive sample or a negative sample.

In a possible implementation manner, in response to that the semantic information includes a first semantic tag, the candidate entity obtaining module is configured to query a first candidate semantic entity from the semantic ontology through the first semantic tag; the semantic ontology includes a correspondence between the first semantic tag and the first candidate semantic entity.

In one possible implementation, the apparatus further includes:

the first history identification module is used for carrying out semantic feature identification on the history query statement to obtain a third semantic tag of the history query statement and a second original semantic entity of the history query statement before the candidate entity acquisition module acquires at least two candidate semantic entities based on the identification result of the query statement;

and the relation establishing module is used for establishing the corresponding relation between the third semantic label and the second original semantic entity in the semantic ontology.

In one possible implementation manner, in response to the semantic information including a first sentence intention, the candidate entity obtaining module is configured to query an entity category corresponding to the first sentence intention; and querying a corresponding second candidate semantic entity through the entity category.

In one possible implementation manner, the candidate entity obtaining module is configured to,

acquiring the similarity of each semantic entity and the first original semantic entity in a knowledge graph; the knowledge graph comprises semantic entities and edges among the semantic entities; the edge is used for indicating the similarity between the two corresponding semantic entities;

and acquiring a third candidate semantic entity from each semantic entity based on the similarity of each semantic entity and the first original semantic entity in the knowledge graph.

In one possible implementation, the apparatus further includes:

a sub-similarity obtaining module, configured to obtain at least two seed similarities between a fourth candidate semantic entity and a first original semantic entity included in the recognition result of the query statement before the target entity screening module screens the target semantic entity from the at least two candidate semantic entities based on the similarities between the at least two candidate semantic entities and the first original semantic entity, respectively; the fourth candidate semantic entity is any one of the at least two candidate semantic entities;

a similarity obtaining module, configured to perform weighted average on at least two seed similarities between the fourth candidate semantic entity and the first original semantic entity, respectively, so as to obtain a similarity between the fourth candidate semantic entity and the first original semantic entity.

In a possible implementation manner, in response to that the at least two seed similarities include a co-occurrence relationship similarity, the sub-similarity obtaining module is configured to,

acquiring a first occurrence number and a second occurrence number; the first occurrence number is the occurrence number of the first original semantic entity in a query history; the second number of occurrences is a number of times that the first original semantic entity and the fourth candidate semantic entity collectively appear in the query history;

and acquiring the co-occurrence relation similarity between the fourth candidate semantic entity and the first original semantic entity based on the first occurrence number and the second occurrence number.

In a possible implementation manner, in response to that the at least two seed similarities include a vector similarity, the sub-similarity obtaining module is configured to,

obtaining respective word vectors of the fourth candidate semantic entity and the first original semantic entity;

and acquiring the vector similarity between the fourth candidate semantic entity and the first original semantic entity based on the respective word vectors of the fourth candidate semantic entity and the first original semantic entity.

In one possible implementation, the apparatus further includes:

the second history identification module is used for performing semantic feature identification on the history query sentence before the sub-similarity obtaining module obtains the word vectors of the fourth candidate semantic entity and the first original semantic entity, so as to obtain the identification result of the history query sentence;

the corpus construction module is used for constructing vector matrix training corpora based on the recognition result of the historical query statement;

the matrix training module is used for carrying out vector matrix training based on the vector matrix training corpus to obtain a word vector matrix;

the sub-similarity obtaining module is configured to obtain word vectors of the fourth candidate semantic entity and the first original semantic entity based on the word vector matrix.

In a possible implementation manner, in response to that the at least two seed similarities include a spectrum similarity, the sub-similarity obtaining module is configured to obtain a similarity of the fourth candidate semantic entity and the first original semantic entity in a knowledge graph as a graph similarity between the fourth candidate semantic entity and the first original semantic entity.

In yet another aspect, a computer device is provided, which includes a processor and a memory, where at least one instruction, at least one program, a set of codes, or a set of instructions is stored in the memory, and the at least one instruction, the at least one program, the set of codes, or the set of instructions is loaded and executed by the processor to implement the query feedback method described above.

In yet another aspect, a computer-readable storage medium is provided, having stored therein at least one instruction, at least one program, set of codes, or set of instructions that is loaded and executed by a processor to implement the query feedback method described above.

In yet another aspect, a computer program product is provided that includes computer instructions stored in a computer readable storage medium. The processor of the computer device may read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes and implements the query feedback method described above.

The technical scheme provided by the application can comprise the following beneficial effects:

after the query statement is obtained, when the semantic features of the query statement are identified, the identification result comprises at least one semantic information of a semantic label and a statement intention besides an original semantic entity in the query statement, and the candidate semantic entity is jointly queried by combining the original semantic entity and the semantic information, so that the source of the subsequently determined target semantic entity is wider, the diversity of the target semantic entity is improved, the diversity of the feedback result is improved, and the purpose of improving the feedback effect is achieved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.

FIG. 1 is a block diagram illustrating a query feedback system in accordance with an exemplary embodiment;

FIG. 2 is a flow diagram illustrating a query feedback method in accordance with an exemplary embodiment;

FIG. 3 is a schematic diagram of query feedback according to the embodiment shown in FIG. 2;

FIG. 4 is a flow diagram illustrating a query feedback method in accordance with an exemplary embodiment;

FIG. 5 is a schematic diagram of a BERT model according to the embodiment shown in FIG. 4;

FIG. 6 is a schematic illustration of a medical ontology mapping according to the embodiment shown in FIG. 4;

FIG. 7 is a schematic diagram of query feedback according to the embodiment shown in FIG. 4;

FIG. 8 is a block diagram of a target semantic entity determination according to the embodiment shown in FIG. 4;

FIG. 9 is a block diagram illustrating the structure of a query feedback device in accordance with an exemplary embodiment;

FIG. 10 is a block diagram illustrating a configuration of a computer device according to an example embodiment.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

According to the scheme, more diversified feedback can be realized based on artificial intelligence, and therefore the feedback effect of the query statement is improved.

Before describing the various embodiments shown herein, several concepts related to the present application will be described.

1) Artificial Intelligence (AI)

Artificial intelligence is a theory, method, technique and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use the knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technique of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a manner similar to human intelligence. Artificial intelligence is the research of the design principle and the realization method of various intelligent machines, so that the machines have the functions of perception, reasoning and decision making.

The artificial intelligence technology is a comprehensive subject and relates to the field of extensive technology, namely the technology of a hardware level and the technology of a software level. The artificial intelligence infrastructure generally includes technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and the like.

2) Natural Language Processing (NLP)

Natural language processing is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that enable efficient communication between humans and computers using natural language. Natural language processing is a science integrating linguistics, computer science and mathematics. Therefore, the research in this field will involve natural language, i.e. the language that people use everyday, so it is closely related to the research of linguistics. Natural language processing techniques typically include text processing, semantic understanding, machine translation, robotic question and answer, knowledge mapping, and the like.

3) Machine Learning (Machine Learning, ML)

Machine learning is a multi-field cross discipline, and relates to a plurality of disciplines such as probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and the like. The special research on how a computer simulates or realizes the learning behavior of human beings so as to acquire new knowledge or skills and reorganize the existing knowledge structure to continuously improve the performance of the computer. Machine learning is the core of artificial intelligence, is the fundamental approach for computers to have intelligence, and is applied to all fields of artificial intelligence. Machine learning and deep learning generally include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and teaching learning.

Fig. 1 is a schematic diagram illustrating a structure of a query feedback system according to an exemplary embodiment. The system comprises: server 120, and terminal 140.

The server 120 is a server, or includes a plurality of servers, or is a virtualization platform, or a cloud computing service center, and the like, which is not limited in the present application.

The terminal 140 may be a terminal device having an inquiry sentence input function and a network access function, for example, the terminal may be a mobile phone, a tablet computer, an electronic book reader, smart glasses, a smart watch, a smart tv, an MP3 player (Moving Picture Experts Group Audio Layer III, motion Picture Experts compression standard Audio Layer 3), an MP4 player (Moving Picture Experts Group Audio Layer IV, motion Picture Experts compression standard Audio Layer 4), a laptop computer, a desktop computer, and the like. The number of user terminals 140 is not limited.

Among them, the terminal 140 may have a client installed therein, and the client may be a video client, an instant messaging client, a browser client, an education client, and the like. The software type of the client is not limited in the embodiment of the application.

The terminal 140 and the server 120 are connected via a communication network. Optionally, the communication network is a wired network or a wireless network.

In this embodiment of the application, the terminal 140 may send the query statement to the server 120, and the server 120 feeds back the query result to the terminal 140 in real time according to the query statement; the server 120 may also recommend the relevant content to the terminal 140 based on the query statement.

Optionally, the system may further include a management device (not shown in fig. 1), which is connected to the server 120 through a communication network. Optionally, the communication network is a wired network or a wireless network.

Optionally, the wireless network or wired network described above uses standard communication techniques and/or protocols. The Network is typically the Internet, but may be any Network including, but not limited to, a Local Area Network (LAN), a Metropolitan Area Network (MAN), a Wide Area Network (WAN), a mobile, wireline or wireless Network, a private Network, or any combination of virtual private networks. In some embodiments, data exchanged over a network is represented using techniques and/or formats including Hypertext Mark-up Language (HTML), Extensible markup Language (XML), and the like. All or some of the links may also be encrypted using conventional encryption techniques such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet protocol Security (IPsec). In other embodiments, custom and/or dedicated data communication techniques may also be used in place of, or in addition to, the data communication techniques described above.

Please refer to fig. 2, which is a flowchart illustrating a query feedback method according to an exemplary embodiment. The method may be performed by a computer device, which may be a server, wherein the server may be the server 120 in the embodiment illustrated in fig. 1 described above. As shown in fig. 2, the flow of the query feedback method may include the following steps.

Step 21, the query statement input in the terminal is obtained.

In the embodiment of the application, the query statement is a query statement input by a user in real time in a terminal; alternatively, the query expression is a query expression that is input by the user in the history of the terminal.

In one possible implementation, the query statement is a natural language statement.

Step 22, performing semantic feature recognition on the query statement to obtain a recognition result of the query statement; the recognition result comprises an original semantic entity and semantic information; the semantic information includes at least one of a semantic tag and a sentence intent; the original semantic entity is the semantic entity contained in the corresponding sentence.

In the field of natural language processing, semantic entities are also referred to as entities (entities) or named entities (namedentities). Semantic entities can be realized by Named Entity Recognition (NER), wherein NER is also called Named Recognition and is a common task in natural language processing, and the range of use is very wide. Where semantic entities/named entities generally refer to entities with special meaning or strong reference in text, generally include names of people, places, organizations, time, proper nouns, and so on. The NER extracts the entities from the unstructured text and can identify more categories of entities according to business requirements, such as product names, models, prices, etc. Therefore, the concept of entity can be very wide, and any special text segment required by the service can be called an entity.

In the embodiment of the present application, the semantic entities included in the sentence include entities directly possessed in the sentence, and also include entities implicit in the sentence (i.e., entities that are not directly included in the sentence, but correspond to the semantics of the sentence).

In the embodiment of the application, when performing semantic feature recognition on a query statement, in addition to recognizing a semantic entity included in the statement through the NER, semantic information of the statement, including a semantic tag and a statement intention, is also recognized.

The semantic tag refers to a classification of semantics of a corresponding sentence. For example, if the sentence is "what I have got a cold and eat a happy fast wonder", the semantic meaning is to find the things that are eaten during the cold, and the semantic labels can be classified as "food" and "medicine".

The statement intent indicates the type of information that the corresponding statement is to query. For example, the sentence "i catch a cold and eat a good fast woollen article" is taken as an example, and the type of information to be inquired is "diet taboo".

And step 23, acquiring at least two candidate semantic entities based on the recognition result of the query statement.

In one possible implementation, the server combines the semantic information in the query statement to obtain the candidate semantic entities, thereby expanding the obtaining manner of the candidate semantic entities in addition to the original semantic entities.

And 24, screening a target semantic entity from the at least two candidate semantic entities based on the similarity between the at least two candidate semantic entities and the first original semantic entity contained in the identification result of the query statement.

And 25, sending a feedback result corresponding to the target semantic entity to the terminal.

In a possible implementation manner, the feedback result sent to the terminal is a query result fed back in real time based on the query statement.

In another possible implementation manner, the feedback result sent to the terminal is recommendation information subsequently pushed to the terminal based on the query statement.

For example, taking an example that the server obtains the candidate semantic entities through the original semantic entities and the semantic information in the query statement together, please refer to fig. 3, which shows a query feedback diagram according to an embodiment of the present application. As shown in fig. 3, the terminal sends a query sentence to the server, and the server identifies the query sentence (S31), and the obtained identification result includes the original semantic entity and the semantic information. The server acquires at least two candidate semantic entities by combining the original semantic entity and the semantic information (S32), and then calculates the similarity between the candidate semantic entities and the original semantic entity (S33) to obtain similarity information; and screening a target semantic entity from the candidate semantic entities according to the similarity information (S34), and finally sending a feedback result to the terminal according to the target semantic entity (S35).

In summary, in the scheme shown in the embodiment of the present application, when performing semantic feature recognition on a query statement after obtaining the query statement, a recognition result includes at least one semantic information of a semantic tag and a statement intention in addition to an original semantic entity in the query statement, and the original semantic entity and the semantic information are combined to jointly query a candidate semantic entity, so that a source of a subsequently determined target semantic entity is wider, thereby improving diversity of the target semantic entity, further improving diversity of a feedback result, and achieving a purpose of improving a feedback effect.

In the above embodiments of the present application, the semantic feature recognition may be implemented by a natural language processing technology in artificial intelligence, that is, the query sentence is recognized by a pre-trained semantic recognition model to obtain a recognition result.

In addition, the scheme shown in the above embodiments of the present application can be applied to query statement feedback in various fields, including the medical field, the e-commerce field, the game field, and the like. The following examples of the present application will be described by way of example in the medical field.

Please refer to fig. 4, which is a flowchart illustrating a method for query feedback according to an exemplary embodiment. The method may be performed by a computer device, which may be a server, wherein the server may be the server 120 in the embodiment illustrated in fig. 1 described above. As shown in fig. 4, the flow of the query feedback method may include the following steps.

Step 401, obtaining the query statement input in the terminal.

When the query statement is a statement sent by the terminal in real time, the server receives the query statement sent by the terminal.

And when the query statement is a statement sent by the terminal in history, the server queries the query statement corresponding to the terminal in a database.

The query statement corresponding to the terminal may refer to a query statement corresponding to a hardware address of the terminal, or may refer to a query statement corresponding to a user account logged in the terminal.

For example, after the terminal sends the query statement to the server, the server stores the query statement in the database in association with the hardware address of the terminal, or the server stores the query statement in the data in association with the user account registered in the terminal.

Step 402, the query statement is processed through the semantic recognition model, and a recognition result output by the semantic recognition model is obtained.

The recognition result comprises an original semantic entity and semantic information; the semantic information includes at least one of a semantic tag and a sentence intent; the original semantic entity is the semantic entity contained in the corresponding sentence.

The semantic recognition model is a multi-task learning model obtained by training a query statement sample and sample labeling information corresponding to the query statement sample, and the sample labeling information is the same type of information as the recognition result.

For query statement (query), in an exemplary scheme of the embodiment of the present application, a Bidirectional transformer model-Based Encoder (BERT) improved intent recognition, named entity recognition, and tag recognition Multi-task Learning model (Multi-task Learning) is proposed to simultaneously recognize an intent, an entity, and a tag of a query, please refer to fig. 5, which shows a schematic structural diagram of a BERT model according to the embodiment of the present application. The multi-task learning has the advantages that 3 tasks are trained simultaneously, model parameters are optimized to 3 global optimal parameters with minimum task loss (loss), and therefore the models respectively utilize mutual hidden information on the 3 tasks to improve the accuracy of each task.

In one possible implementation, the training process for the semantic recognition model may be as follows:

step a, processing the query statement sample through the semantic recognition model to obtain a prediction result of the query statement sample;

b, inputting the prediction result and the sample marking information into a loss function to obtain a loss function value;

and c, updating the parameters in the semantic recognition model based on the loss function values.

In one possible implementation, the multitask loss function (loss function) is designed as follows:

（1）

（2）

wherein, the above formula (1) is a general cross entropy loss function.

α, β, γ in the above formula (2) represent three parameters, the sum of which is 1, for measuring the importance of three tasks, i.e., the NER entity task, the TAG recognition task, and the INTENT recognition task, which are initialized at random and learned in training.

In one possible implementation, the loss function of each task in the above-mentioned multitask can be replaced by a respective focus loss (focal loss) function.

However, in natural language processing in many fields, there are problems that the distribution of samples is not uniform, and the label classification is inaccurate due to too many simple samples. For example, in natural language processing in the medical field, there may be two or more levels of labels in a pre-designed label system, which may result in uneven sample distribution among different labels and excessive simple sample occupation.

While the penalty function used by the multi-task learning equation (1) is the traditional cross entropy, for each sample, the cross entropy penalty is a term for them, regardless of whether the current sample is simple or complex. The main difficulties with the above problems are: how to judge whether the current sample is simple or complex, and how to design a loss function based on the final goal (boost F1 index).

In one possible implementation of the embodiment of the present application, the complexity of the current sample is determined by an active learning technique. For example, a scaling factor is designed in the loss function, and the scaling factor is inversely related to the prediction probability, which is the probability that the semantic recognition model predicts that the query sentence sample belongs to a positive sample or a negative sample. That is, by the scaling factor, the model can be made to pay less attention to correctly predicted samples and pay more attention to samples that have not been correctly predicted.

For example, for a single sample x, the following loss function is defined:

（3）

wherein p is₁Representing the probability that the model f predicts as a positive sample, for a simple sample found by the active learning process, i.e. p₁Approaching to 1 or 0, 1-p₁Acting as a scaling factor for (1-p)₁)*p₁In part, from the derivative point of view, once the model correctly classifies the sample (e.g., greater than 0.5), the formula (3) makes the model pay less attention to the sample, rather than encouraging the model to approach both endpoints of 0 or 1, as in cross entropy, which can avoid too many simple samples in the sample distribution, and the model learns more difficult samples in a targeted manner through active learning, and the PPL function shown in the formula (3) is from the soft form of F1 by adding a smoothing term(s) ((s))

) And the scaling coefficient is obtained, and the essence is to optimize the final F1 index of the model, thereby improving the F1 score of the final result of the model.

The model structure shown in fig. 5 is described by taking an example in which one multitask model executes three recognition tasks simultaneously. In other possible implementation manners, the server sets corresponding recognition models for the three recognition tasks respectively, and executes different recognition tasks through different models. Or the server sets one recognition model for two of the three recognition tasks and sets another recognition model for the other recognition task.

The training process of the semantic recognition model is executed by a server; or after the training process of the semantic recognition model is executed by model training equipment except the server, the trained semantic recognition model is provided for the server.

At step 403, at least two candidate semantic entities are obtained based on the recognition result of the query statement.

The method for acquiring at least two candidate semantic entities based on the recognition result of the query statement comprises at least two of the following methods:

the entity acquisition mode is as follows: responding to the semantic information containing a first semantic label, and inquiring a first candidate semantic entity from the semantic ontology through the first semantic label; the semantic ontology includes a corresponding relationship between the first semantic tag and the first candidate semantic entity.

Ontology is a special type of term set, has the characteristic of structuring, and is a formal expression of a set of concepts in a specific field and their interrelations.

Taking the medical field as an example, the server extracts the disease label from the search query, queries the entity corresponding to the disease label from the medical ontology, and takes the entity as an ontology recall result. For example, if the disease tag "cold" corresponds to an entity shown in table 1 below, it can be selected as a candidate semantic entity.

TABLE 1

And the entity acquisition mode II comprises the following steps: responding to the semantic information containing a first statement intention, and inquiring an entity category corresponding to the first statement intention; a second candidate semantic entity is queried by the entity category.

The embodiment of the application introduces the mapping dictionary of the intention and the entity category as follows:

intent_ner_mapping_dict=｛

"treatment/drug/regimen": [ "food", "medication", "check" ],

"diet is contraindicated": [ "food", "medication" ],

"doctor/hospital/department/institution": [ "department", "examination" ],

"symptom": [ "symptoms", "diseases", "drugs", "department", "examinations" ],

"inspection check": [ "department", "examination", "instrument" ],

"disease complex intent": [ "symptoms", "diseases", "examinations", "drugs" ],

"other medical intent": [ "symptoms", "diseases", "examinations" ],

"medical insurance/reimbursement/aid": [ "diseases", "examinations", "drugs" ],

"medical/industrial": [ "symptoms", "diseases", "examinations", "drugs" ],

｝

in the embodiment of the application, the entity categories required to be recalled for each type of intention can be configured in advance by manpower. The mapping dictionary improves the controllability of the scheme, for example, according to the diversity or accuracy of product characteristic preference, the required entity category can be increased or decreased appropriately. For the search query "what is good for a cold cough," which is intended to be "dietary contraindication," then the addition of the entity category "medication" will cause the recommended entities to include not only the food that is suitable for being eaten at the time of the cold cough, but also the medication that the cold cough should eat.

In another possible implementation manner, a label corresponding to each type of intention is preset in the server; responding to the semantic information containing a first statement intention, and inquiring a second semantic label corresponding to the first statement intention; and querying a second candidate semantic entity from the semantic ontology through the second semantic tag.

In a possible implementation manner, before obtaining at least two candidate semantic entities based on the recognition result of the query statement, the method further includes:

performing semantic feature recognition on the historical query statement to obtain a third semantic tag of the historical query statement and a second original semantic entity of the historical query statement; and establishing a corresponding relation between the third semantic label and the second original semantic entity in the semantic ontology.

In this embodiment of the application, the entity category in the mapping dictionary may be a tag in the semantic ontology, that is, the server searches for a tag corresponding to the semantic intent according to the mapping dictionary, and then searches for a corresponding candidate semantic entity from the semantic ontology through the tag.

Taking the application in the medical field as an example, the construction process of the medical ontology (i.e. the semantic ontology) may be as follows:

for the full amount of history search queries,

1) extracting disease labels according to a disease label system formulated by experts;

2) the entities contained by the query are obtained using a query understanding technique (i.e., the semantic recognition model described above).

Please refer to fig. 6, which illustrates a medical ontology mapping diagram according to an embodiment of the present application. As shown in fig. 6, the entity and the secondary disease label are mapped one by one (in case the query does not have a secondary label, the primary label is used), thereby constructing the medical ontology.

For example, for the query "allergy to mango", the extracted primary disease label is "surgical complex", the extracted secondary disease label is "allergy", and the identified entities include "mango" in the food category and "allergy" in the symptom category, and then the entity "mango" is linked to the concept "allergy" in the ontology by the food category, and the entity "allergy" is linked to the concept "allergy" in the ontology by the symptom category.

And the entity acquisition mode is three: acquiring the similarity of each semantic entity and the first original semantic entity in a knowledge graph; the knowledge graph spectrum comprises various semantic entities and edges among the semantic entities; the edge is used for indicating the similarity between the two corresponding semantic entities; and acquiring a third candidate semantic entity from each semantic entity based on the similarity of each semantic entity and the first original semantic entity in the knowledge graph.

Knowledge Graph (Knowledge Graph): a knowledge graph is a semantic network that exposes relationships between entities, with nodes representing entities and edges representing various semantic relationships between entities.

Taking the medical field as an example, on a medical knowledge graph (an open medical knowledge graph such as an OMAHA can be adopted, or a medical knowledge graph can be constructed semi-automatically by using a relation extraction technology in the knowledge graph in combination with services), node2vec technology in the graph embedding field (or a deep walking deepwater method can be used instead) is used in advance to obtain embedding of entity nodes, and cosine similarity between the entity nodes is calculated.

The step of obtaining the entity node embedding is as follows:

1) generating a plurality of continuous entity node sequences by Random Walk on the edge of the < entity, entity relation and entity >;

2) and taking the entity node sequence as a corpus, and training by using word2 vec.

In the application stage, the server uses the query understanding technology (namely the semantic recognition model) to obtain entities contained in the search query, links the entities to entity nodes in the knowledge graph through entity links, and selects a plurality of entities with higher node similarity from the medical knowledge graph as candidate semantic entities.

In step 404, the similarity between at least two candidate semantic entities and the first original semantic entity included in the recognition result of the query statement is obtained.

The process of obtaining the similarity between the candidate semantic entity and the first original semantic entity may be as follows:

acquiring at least two seed similarities between a fourth candidate semantic entity and the first original semantic entity; the fourth candidate semantic entity is any one of the at least two candidate semantic entities; and carrying out weighted average on at least two seed similarities between the fourth candidate semantic entity and the first original semantic entity respectively to obtain the similarity between the fourth candidate semantic entity and the first original semantic entity.

The manner of obtaining at least two seed similarities between the fourth candidate semantic entity and the first original semantic entity may be as follows:

a first sub-similarity obtaining mode: responding to the similarity of the co-occurrence relationship contained in the similarity of the at least two seeds, and acquiring a first occurrence frequency and a second occurrence frequency; the first occurrence is the occurrence of the first original semantic entity in the query history; the second number of occurrences is a number of common occurrences of the first original semantic entity and the fourth candidate semantic entity in the query history; and acquiring the co-occurrence relationship similarity between the fourth candidate semantic entity and the first original semantic entity based on the first occurrence number and the second occurrence number.

In a possible implementation manner, the server analyzes a history search query log, adds different entities appearing in the same query into a co-occurrence (co-occure) relationship, adds different entities searched in a session by the same user into the co-occure relationship, and calculates co-occure information. Formally, the notations e1, e2 denote the entities to be calculated, and the co-occurence statistical similarity feature of e2 with respect to e1 is calculated as follows:

（4）

wherein n is_e1Denotes the number of occurrences of e1 in the historical query, n_e2|e1Indicating the number of times e1 and e2 co-occur in the history query.

And a second sub-similarity obtaining mode: responding to the vector similarity contained in the at least two seed similarities, and acquiring the word vectors of the fourth candidate semantic entity and the first original semantic entity; and acquiring the vector similarity between the fourth candidate semantic entity and the first original semantic entity based on the respective word vectors of the fourth candidate semantic entity and the first original semantic entity.

Before obtaining the word vectors of the fourth candidate semantic entity and the first original semantic entity, the method further includes: performing semantic feature recognition on the historical query statement to obtain a recognition result of the historical query statement; constructing a vector matrix training corpus based on the recognition result of the historical query statement; performing vector matrix training based on the vector matrix training corpus to obtain a word vector matrix; the obtaining of the word vectors of the fourth candidate semantic entity and the first original semantic entity includes: and acquiring respective word vectors of the fourth candidate semantic entity and the first original semantic entity based on the word vector matrix.

For the < entity, candidate entity > pair, word embedding of the entity (namely, the original semantic entity) and the candidate entity (namely, the candidate semantic entity) is respectively obtained from the pre-training word vector matrix, and the cosine similarity of the two is calculated as the semantic similarity characteristic.

Taking the medical field as an example, the word vector matrix is obtained by the following pre-training method: and for each query of the full history search queries, splicing the query with a disease label, an intention and an entity of the query to serve as a pre-training corpus. For example, "how to get pregnant acute gastritis and how to get pregnant tag2 pregnant tag1 gynecological gestating tag2 gastritis/polyp tag1 gastrointestinal disease/disease complex intent/symptom/pregnancy/disease/acute gastritis". After the pre-training corpus is generated in this way, word2vec algorithm is used for pre-training to obtain the word vector of the entity. The method has the advantages that the contextual information of the entity is complemented by using the disease label and intention of the query, so that the semantic representation capability of the word vector of the entity is improved. In a possible implementation manner, the word vector pre-training technology based on word embedding may be replaced by using a BERT method for pre-training.

And a third sub-similarity obtaining mode: and responding to the at least two seed similarities including the spectrum similarity, and acquiring the similarity of the fourth candidate semantic entity and the first original semantic entity in the knowledge graph as the graph similarity between the fourth candidate semantic entity and the first original semantic entity.

The similarity characteristics (i.e. the graph similarity) between the entity nodes can be directly obtained by the part of the candidate semantic entities recalled from the knowledge graph. For candidate semantic entities not recalled from the knowledge-graph, similarity characteristics between the candidate semantic entities and the first original semantic entity can be inquired from the knowledge-graph.

In a possible implementation manner, after the server obtains the sub-similarities between the four candidate semantic entities and the first original semantic entity through at least two of the three manners, the sub-similarities corresponding to the at least two manners are weighted and averaged to obtain the fused similarity.

Step 405, based on the similarity between the at least two candidate semantic entities and the first original semantic entity included in the recognition result of the query statement, a target semantic entity is screened from the at least two candidate semantic entities.

In a possible implementation manner, after acquiring the similarity between at least two candidate semantic entities and a first original semantic entity included in the recognition result of the query statement, the server arranges the candidate semantic entities according to the sequence of the similarity from high to low, and determines the candidate semantic entities arranged at the top N bits as target semantic entities.

In another possible implementation, the server performs comprehensive ranking on at least two candidate semantic entities in combination with real-time popularity (popularity) of the entities. For example, the server adjusts the similarity between at least two candidate semantic entities and a first original semantic entity included in the recognition result of the query statement respectively according to the respective popularity of the at least two candidate semantic entities, then arranges the at least two candidate semantic entities in the sequence from high to low according to the adjusted similarity, and determines the candidate semantic entities arranged at the top N bits as the target semantic entities.

Step 406, sending a feedback result corresponding to the target semantic entity to the terminal.

The feedback result may be a query result fed back in real time to a query operation initiated by the terminal. For example, after the terminal receives an input operation of a query statement through a browser or an application program, a query request is sent to the server, the server extracts the query statement in the query request after obtaining the query request, and after the target semantic entity is determined through the scheme of the embodiment of the application, a feedback result is returned to the terminal, and the feedback result is a query result for the query statement, so that the terminal can display the query result in a search result of the browser or the application program.

Or the feedback result is a recommendation result fed back according to the query operation initiated by the terminal. For example, the terminal receives a query statement and initiates a query through a browser or an application program, and after a query result is displayed, the server subsequently obtains the query result again, and after a new target semantic entity is matched in different databases, obtains a recommendation result corresponding to the new target semantic entity, and feeds the recommendation result back to the terminal, and the terminal displays the recommendation result in a recommendation information display area (such as a webpage or an advertisement area in the application program) of the current interface.

In summary, for natural language understanding in the fields of medical treatment and the like, a pre-designed label system comprises at least two levels of labels, so that the label sample imbalance problem is caused, and the label classification is inaccurate due to excessive simple sample occupation. According to the embodiment of the application, the accuracy of three tasks of the module is improved simultaneously by combining multi-task training and a new loss function based on active learning.

In addition, the recommendation result is preferred due to the use of a single scheme, so that the problem that the diversity is insufficient and the product requirements in the recommendation field cannot be met is caused. According to the embodiment of the application, the recall strategy based on the medical ontology and the knowledge graph which are automatically pre-constructed is used in the recall stage, the diversity of the target semantic entities is improved, and then the diversity of feedback results is improved.

In addition, the embodiment of the application adds the mapping template of the intention and the entity category, and the recommended entity is controllable by indicating the specific entity category corresponding to the specific intention, so that the feedback effect is further improved.

Please refer to fig. 7, which illustrates a schematic diagram of query feedback according to the embodiment shown in fig. 4. As shown in fig. 7, the terminal sends the query sentence to the server, and the server instantly or subsequently identifies the query sentence (S71), and the obtained identification result includes the original semantic entity, the semantic tag and the semantic intent. The server combines the original semantic entity, the semantic label and the semantic intention to obtain at least two candidate semantic entities (S72); the method comprises the steps of obtaining a first candidate semantic entity by querying a semantic ontology through an original semantic entity, obtaining a second candidate semantic entity by querying a knowledge base through a semantic tag, querying a semantic category through semantic intent, and obtaining a third candidate semantic entity corresponding to the semantic category. Then, the server respectively calculates the co-occurrence relationship similarity, the vector similarity and the map similarity between the candidate semantic entity and the original semantic entity through three modes (S73), and combines the three seed similarities to comprehensively obtain the similarity between the candidate semantic entity and the original semantic entity (S74) to obtain similarity information; and then, according to the similarity information and the popularity information of the candidate semantic entities, screening target semantic entities from the candidate semantic entities (S75), and finally, sending a feedback result to the terminal according to the target semantic entities (S76).

Taking the medical field as an example, please refer to fig. 8, which shows a frame diagram of target semantic entity determination related to the embodiment shown in fig. 4. As shown in fig. 8, the flow of the scheme for determining the target semantic entity is as follows:

a medical ontology pre-construction phase.

S81, outputting entities, intentions and labels to queries (query) through a natural language understanding module according to the full medical search logs, and then automatically constructing a medical ontology and a knowledge graph;

the use stage is as follows:

s82, understanding the query input by the user in a medical natural language, and outputting an NER (original semantic entity), a label and an intention recognition result corresponding to the query;

s83, according to the original semantic entities, the intention and entity category mapping dictionary and the labels, respectively selecting candidate semantic entities from the medical ontology and the medical knowledge map;

s84, calculating the similarity between entities according to various characteristics, and further narrowing the range of the candidate semantic entities;

and S85, performing final sorting according to the popularity information to generate a target semantic entity.

Experiments show that when entity recommendation is performed on the query sampled randomly, the accuracy rate of the scheme reaches over 90%. For example, an example of the results of a related entity recommendation for query "what good fast is a cold cough to eat" is as follows:

the query of the user: the fast rate of what is eaten by cold and cough;

and (3) intention recognition result: the diet is contraindicated;

the recommended related entities: "Ganmaoling", "999", "heat-clearing and detoxicating", "Ganmaoyao", "compound paracetamol and amantadine hydrochloride", "Pudilan blue", "cough relieving", "bezoar detoxicating tablet", "defervescence patch" and "drinking".

As can be seen from the above results example, the recommended entities are all relevant and controllable to the user query.

Fig. 9 is a block diagram illustrating a structure of a query feedback device according to an exemplary embodiment. The query feedback device may implement all or part of the steps in the method provided by the embodiment shown in fig. 2 or fig. 4. The query feedback device may include:

a query sentence acquisition module 901, configured to acquire a query sentence input in a terminal;

a semantic recognition module 902, configured to perform semantic feature recognition on the query statement to obtain a recognition result of the query statement; the recognition result comprises an original semantic entity and semantic information; the semantic information comprises at least one of a semantic tag and a sentence intent; the original semantic entities are semantic entities contained in corresponding sentences;

a candidate entity obtaining module 903, configured to obtain at least two candidate semantic entities based on the recognition result of the query statement;

a target entity screening module 904, configured to screen a target semantic entity from the at least two candidate semantic entities based on similarity between the at least two candidate semantic entities and a first original semantic entity included in the recognition result of the query statement, respectively;

a feedback module 905, configured to send a feedback result corresponding to the target semantic entity to the terminal.

In one possible implementation, the semantic recognition module 902 is configured to,

In one possible implementation, the apparatus further includes:

a processing module, configured to process, at the semantic recognition module 902, the query statement sample through the semantic recognition model to obtain a prediction result of the query statement sample;

In a possible implementation manner, in response to that the semantic information includes a first semantic tag, the candidate entity obtaining module 903 is configured to query a first candidate semantic entity from a semantic ontology through the first semantic tag; the semantic ontology includes a correspondence between the first semantic tag and the first candidate semantic entity.

In one possible implementation, the apparatus further includes:

a first history identification module, configured to perform semantic feature identification on a history query statement to obtain a third semantic tag of the history query statement and a second original semantic entity of the history query statement before the candidate entity obtaining module 903 obtains at least two candidate semantic entities based on a recognition result of the query statement;

In a possible implementation manner, in response to the semantic information including a first sentence intent, the candidate entity obtaining module 903 is configured to query an entity category corresponding to the first sentence intent; and querying a corresponding second candidate semantic entity through the entity category.

In one possible implementation, the candidate entity obtaining module 903 is configured to,

In one possible implementation, the apparatus further includes:

a sub-similarity obtaining module, configured to obtain at least two seed similarities between a fourth candidate semantic entity and a first original semantic entity included in the recognition result of the query statement before the target entity screening module 904 screens the target semantic entity from the at least two candidate semantic entities based on the similarities between the at least two candidate semantic entities and the first original semantic entity, respectively; the fourth candidate semantic entity is any one of the at least two candidate semantic entities;

In one possible implementation, the apparatus further includes:

FIG. 10 is a block diagram illustrating a computer device according to an example embodiment. The computer device may be implemented as a server on the network side. The server may be the server 120 shown in fig. 1. The computer apparatus 1000 includes a central processing unit 1001, a system Memory 1004 including a Random Access Memory (RAM) 1002 and a Read-Only Memory (ROM) 1003, and a system bus 1005 connecting the system Memory 1004 and the central processing unit 1001. The computer device 1000 also includes a basic input/output system 1006, which facilitates the transfer of information between various components within the computer, and a mass storage device 1007, which stores an operating system 1013, application programs 1014, and other program modules 1015.

The mass storage device 1007 is connected to the central processing unit 1001 through a mass storage controller (not shown) connected to the system bus 1005. The mass storage device 1007 and its associated computer-readable media provide non-volatile storage for the computer device 1000. That is, the mass storage device 1007 may include a computer-readable medium (not shown) such as a hard disk or Compact disk Read-Only Memory (CD-ROM) drive.

Without loss of generality, the computer-readable media may comprise computer storage media and communication media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes RAM, ROM, flash memory or other solid state storage technology, CD-ROM, or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices. Of course, those skilled in the art will appreciate that the computer storage media is not limited to the foregoing. The system memory 1004 and mass storage device 1007 described above may be collectively referred to as memory.

The computer device 1000 may be connected to the internet or other network devices through a network interface unit 1011 connected to the system bus 1005.

The memory further includes one or more programs, the one or more programs are stored in the memory, and the cpu 1001 implements all or part of the steps of the method shown in fig. 2 or 4 by executing the one or more programs.

In an exemplary embodiment, there is also provided a non-transitory computer readable storage medium comprising instructions, such as a memory comprising computer programs (instructions), which are executable by a processor of a computer device to perform the methods illustrated in the various embodiments of the present application, the methods performed by a server or a user terminal. For example, the non-transitory computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

In an exemplary embodiment, a computer program product is also provided that includes computer instructions stored in a computer readable storage medium. The processor of the computer device may read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes and implements the query feedback method described above.

Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.

It will be understood that the present application is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims

1. A query feedback method, the method comprising:

acquiring a query statement input in a terminal;

2. The method of claim 1, wherein the performing semantic feature recognition on the query statement to obtain a recognition result comprises:

3. The method according to claim 2, wherein before performing semantic feature recognition on the query statement and obtaining a recognition result of the query statement, the method further comprises:

processing the query statement sample through the semantic recognition model to obtain a prediction result of the query statement sample;

inputting the prediction result and the sample marking information into a loss function to obtain a loss function value;

updating parameters in the semantic recognition model based on the loss function values;

4. The method of claim 1, wherein in response to the semantic information including a first semantic tag, the obtaining at least two candidate semantic entities based on the recognition result of the query statement comprises:

querying a first candidate semantic entity from a semantic ontology through the first semantic tag; the semantic ontology includes a correspondence between the first semantic tag and the first candidate semantic entity.

5. The method of claim 4, wherein before obtaining at least two candidate semantic entities based on the recognition result of the query statement, further comprising:

performing semantic feature recognition on a historical query statement to obtain a third semantic tag of the historical query statement and a second original semantic entity of the historical query statement;

and establishing a corresponding relation between the third semantic label and the second original semantic entity in the semantic ontology.

6. The method of claim 1, wherein in response to the semantic information including a first sentence intent, the obtaining at least two candidate semantic entities based on the recognition result of the query sentence comprises:

querying an entity category corresponding to the first statement intent;

and querying a corresponding second candidate semantic entity through the entity category.

7. The method of claim 1, wherein obtaining at least two candidate semantic entities based on the recognition result of the query statement comprises:

8. The method according to claim 1, wherein before the step of screening the target semantic entity from the at least two candidate semantic entities based on the similarity between the at least two candidate semantic entities and the first original semantic entity included in the recognition result of the query statement, the method further comprises:

acquiring at least two seed similarities between a fourth candidate semantic entity and the first original semantic entity; the fourth candidate semantic entity is any one of the at least two candidate semantic entities;

and carrying out weighted average on at least two seed similarities between the fourth candidate semantic entity and the first original semantic entity respectively to obtain the similarity between the fourth candidate semantic entity and the first original semantic entity.

9. The method of claim 8, wherein, in response to the at least two seed similarities including a co-occurrence similarity, the obtaining at least two seed similarities between a fourth candidate semantic entity and the first original semantic entity comprises:

10. The method of claim 8, wherein the obtaining at least two seed similarities between the fourth candidate semantic entity and the first original semantic entity in response to the at least two seed similarities including a vector similarity comprises:

11. The method according to claim 10, wherein before obtaining the word vectors of the fourth candidate semantic entity and the first original semantic entity, the method further comprises:

performing semantic feature recognition on historical query sentences to obtain recognition results of the historical query sentences;

constructing a vector matrix training corpus based on the recognition result of the historical query statement;

performing vector matrix training based on the vector matrix training corpus to obtain a word vector matrix;

the obtaining of the respective word vectors of the fourth candidate semantic entity and the first original semantic entity includes:

and acquiring respective word vectors of the fourth candidate semantic entity and the first original semantic entity based on the word vector matrix.

12. The method of claim 8, wherein the obtaining at least two seed similarities between a fourth candidate semantic entity and the first original semantic entity in response to the at least two seed similarities comprising a spectral similarity comprises:

and acquiring the similarity of the fourth candidate semantic entity and the first original semantic entity in a knowledge graph as the graph similarity between the fourth candidate semantic entity and the first original semantic entity.

13. An apparatus for query feedback, the apparatus comprising:

14. A computer device comprising a processor and a memory, the memory having stored therein at least one instruction, at least one program, a set of codes, or a set of instructions, the at least one instruction, the at least one program, the set of codes, or the set of instructions being loaded and executed by the processor to implement the query feedback method as claimed in any one of claims 1 to 12.

15. A computer readable storage medium having stored therein at least one instruction, at least one program, a set of codes, or a set of instructions, which is loaded and executed by a processor to implement the query feedback method as claimed in any one of claims 1 to 12.