CN111460264A

CN111460264A - Training method and device of semantic similarity matching model

Info

Publication number: CN111460264A
Application number: CN202010237646.7A
Authority: CN
Inventors: 邓哲宇; 王磊; 苑爱泉; 王宇昊; 何旺贵; 朱培源
Original assignee: Koukouxiangchuan Beijing Network Technology Co ltd
Current assignee: Koukouxiangchuan Beijing Network Technology Co ltd
Priority date: 2020-03-30
Filing date: 2020-03-30
Publication date: 2020-07-28
Anticipated expiration: 2040-03-30
Also published as: CN111460264B

Abstract

The invention discloses a training method and a device of a semantic similarity matching model, wherein the method comprises the following steps: obtaining a plurality of sample data from a historical search click log; inquiring category information of a category to which the search result object belongs aiming at any sample data, and generating a category vector of the category information, a title vector matrix of the title information and a search word vector of a search word; fusing according to the category vector and the title vector matrix to obtain an object vector of the search result object; determining object vectors and search word vectors corresponding to a plurality of sample data as matching input data, determining click execution data of a search result object as matching output data, training a neural network model, and constructing a semantic similarity matching model. According to the scheme of the invention, the category information of the category to which the search result object belongs is fused into the object vector for training, so that the object vector carries the category information, the accuracy of the training result can be further improved, and the method is favorable for carrying out accurate search response.

Description

Training method and device of semantic similarity matching model

Technical Field

The invention relates to the technical field of internet, in particular to a training method and a training device for a semantic similarity matching model.

Background

On each large internet platform, searching is the most direct way for a user to acquire information, and object information related to a search word can be acquired by inputting the search word, for example, store information of a hamburger store recalled by the platform can be acquired by inputting "hamburger" in a local life platform.

In the traditional object recall process, the object related to the object is recalled based on text matching, and accurate recall at a semantic level cannot be achieved. For example, inputting the search word "hamburger", only the store of "spicy hamburger" can be recalled, and the store of "spicy drumstick" cannot be recalled. Meanwhile, by using the recall method based on text matching, in some practical scenarios, the number of search results is too small because there are few or zero objects matched with the search terms, and thus the recall method based on semantic similarity matching is very important.

However, in the existing recall mode based on semantic similarity matching, only semantic similarity matching is carried out on the search terms and the object name or description of the object to be matched, and the matching result is not accurate enough; especially for the case where words contained in the name or description are not commonly used or do not reflect true semantics, matching accuracy is greatly reduced.

Disclosure of Invention

In view of the above, embodiments of the present invention are proposed to provide a training method and apparatus for a semantic similarity matching model, which overcome the above problems or at least partially solve the above problems.

According to an aspect of the embodiments of the present invention, there is provided a training method of a semantic similarity matching model, including:

obtaining a plurality of sample data according to a historical search click log, wherein any sample data comprises a search word and title information of a search result object corresponding to the search word;

for any sample data, inquiring the category information of the category to which the search result object belongs, and generating a category vector of the category information, a title vector matrix of the title information and a search word vector of a search word; fusing according to the category vector and the title vector matrix to obtain an object vector of the search result object;

determining object vectors and search word vectors corresponding to a plurality of sample data as matching input data, determining click execution data of search result objects in the plurality of sample data as matching output data, training a neural network model by using the matching input data and the matching output data, and constructing a semantic similarity matching model according to training results.

Optionally, the method further includes: carrying out splicing processing on search terms of a single user which are continuously searched for multiple times within a preset time period to form a long text training sample; and/or, aiming at any category, splicing the category information of the category and the title information of at least two objects belonging to the category to form a long text training sample; and inputting the long text training sample into a word vector calculation model to be trained to obtain a word vector table.

Optionally, the generating a category vector of the category information further includes: performing word segmentation processing on the category information, determining category word segmentation according to word segmentation results, and inquiring category word segmentation vectors of the category word segmentation according to the word vector table; if the category word segmentation vector is one, determining the category word segmentation vector as a category vector; and if the category word segmentation vectors are multiple, performing mean pooling on the multiple category word segmentation vectors to generate the category vectors of the category information.

Optionally, the obtaining the object vector of the search result object according to the fusion of the category vector and the title vector matrix further includes: calculating a first product of the category vector and each title participle vector forming a title vector matrix, and generating a title participle weight vector according to the first product; each element of the title participle weight vector is a participle weight of a title participle; and fusing the title word segmentation weight vector and the title vector matrix to obtain an object vector of the search result object.

Optionally, the obtaining of the object vector of the search result object according to the fusion of the title participle weight vector and the title vector matrix further includes: and calculating a second product of the title participle weight vector and a title vector matrix, and determining the second product as an object vector of the search result object.

Optionally, the determining click execution data of the search result object as matching output data further includes: when the search result object is an object clicked by a user, determining that the matching output data is first data; and when the search result object is an object which is not clicked by the user, determining that the matching output data is the second data.

Optionally, the method further includes: constructing a display position characteristic of the search result object, a distance characteristic of the search result object and/or a user activity characteristic of a search user; the training of the neural network model further comprises: and training the neural network model according to the display position characteristics, the distance characteristics and/or the user activity characteristics.

Optionally, the display position feature is a ranking position feature of the search result object in the search result; the distance feature is a spacing feature between the object position and the search position of the search result object; and/or the user liveness characteristic is a search click frequency characteristic of a search user.

According to another aspect of the embodiments of the present invention, there is provided a method for responding to a search request, including:

generating a search term vector of a real-time search term in response to the real-time search request;

calculating the similarity between the search word vector and the object vectors of a plurality of objects to be matched in an object library according to a semantic similarity matching model; the object vector is obtained by fusing a category vector of category information of a category to which an object to be matched belongs and a header vector matrix of header information of the object to be matched;

and screening out target similar objects from the plurality of objects to be matched according to the similarity calculation result, and sending the header information of the target similar objects to a request end.

Optionally, the semantic similarity matching model is obtained by training through any one of the above training methods of the semantic similarity matching model.

According to another aspect of the embodiments of the present invention, there is provided a training apparatus for a semantic similarity matching model, including:

the acquisition module is suitable for acquiring a plurality of sample data according to the historical search click log, wherein any sample data comprises a search word and the title information of a search result object corresponding to the search word;

the vector generation module is suitable for inquiring the category information of the category to which the search result object belongs aiming at any sample data, and generating a category vector of the category information, a title vector matrix of the title information and a search word vector of a search word;

the fusion module is suitable for obtaining an object vector of the search result object according to the fusion of the category vector and the title vector matrix;

the training module is suitable for determining object vectors and search word vectors corresponding to a plurality of sample data as matching input data, determining click execution data of search result objects in the plurality of sample data as matching output data, training a neural network model by using the matching input data and the matching output data, and constructing a semantic similarity matching model according to a training result.

Optionally, the apparatus further comprises: the word vector table generating module is suitable for splicing the search words searched continuously and repeatedly by a single user in a preset time period to form a long text training sample; and/or, aiming at any category, splicing the category information of the category and the title information of at least two objects belonging to the category to form a long text training sample; and inputting the long text training sample into a word vector calculation model to be trained to obtain a word vector table.

Optionally, the vector generation module is further adapted to: performing word segmentation processing on the category information, determining category word segmentation according to word segmentation results, and inquiring category word segmentation vectors of the category word segmentation according to the word vector table; if the category word segmentation vector is one, determining the category word segmentation vector as a category vector; and if the category word segmentation vectors are multiple, performing mean pooling on the multiple category word segmentation vectors to generate the category vectors of the category information.

Optionally, the fusion module is further adapted to: calculating a first product of the category vector and each title participle vector forming a title vector matrix, and generating a title participle weight vector according to the first product; each element of the title participle weight vector is a participle weight of a title participle; and fusing the title word segmentation weight vector and the title vector matrix to obtain an object vector of the search result object.

Optionally, the fusion module is further adapted to: and calculating a second product of the title participle weight vector and a title vector matrix, and determining the second product as an object vector of the search result object.

Optionally, the training module is further adapted to: when the search result object is an object clicked by a user, determining that the matching output data is first data; and when the search result object is an object which is not clicked by the user, determining that the matching output data is the second data.

Optionally, the apparatus further comprises: the error characteristic construction module is suitable for constructing the display position characteristic of the search result object, the distance characteristic of the search result object and/or the user activity characteristic of a search user; the training module further comprises: and training the neural network model according to the display position characteristics, the distance characteristics and/or the user activity characteristics.

According to still another aspect of the embodiments of the present invention, there is provided a device for responding to a search request, including:

the vector generation module is suitable for responding to the real-time search request and generating a search word vector of the real-time search word;

the matching module is suitable for calculating the similarity between the search word vector and the object vectors of a plurality of objects to be matched in the object library according to a semantic similarity matching model; the object vector is obtained by fusing a category vector of category information of a category to which an object to be matched belongs and a header vector matrix of header information of the object to be matched;

and the result returning module is suitable for screening out the target similar objects from the plurality of objects to be matched according to the similarity calculation result and sending the header information of the target similar objects to the request end.

Optionally, the semantic similarity matching model is obtained by training with a training device of any one of the above semantic similarity matching models.

According to still another aspect of an embodiment of the present invention, there is provided a computing device including: the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete mutual communication through the communication bus;

the memory is used for storing at least one executable instruction, and the executable instruction enables the processor to execute the operation corresponding to the training method of the semantic similarity matching model.

According to yet another aspect of embodiments of the present invention, there is provided a computing device including: the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete mutual communication through the communication bus;

the memory is used for storing at least one executable instruction, and the executable instruction enables the processor to execute the operation corresponding to the response method of the search request.

According to a further aspect of the embodiments of the present invention, there is provided a computer storage medium, in which at least one executable instruction is stored, where the executable instruction causes a processor to perform an operation corresponding to the training method of the semantic similarity matching model.

According to a further aspect of the embodiments of the present invention, there is provided a computer storage medium, in which at least one executable instruction is stored, and the executable instruction causes a processor to perform an operation corresponding to the response method of the search request.

According to the training method and device for the semantic similarity matching model, a plurality of sample data are obtained according to a historical search click log, wherein any sample data comprises a search word and title information of a search result object corresponding to the search word; for any sample data, an object vector is obtained by fusing a title vector matrix of the title information and a category vector of category information of a category to which the search result object belongs, so that the search result object is comprehensively represented by using the category information and the title information; and training by using the matching input data and the matching output data to obtain a semantic similarity matching model. Therefore, according to the scheme of the embodiment, the search result object is comprehensively represented by the category information and the title information, the title information and the category information of the search result object can be comprehensively matched with the search terms in the training process, the problem that the finally trained matching model is inaccurate due to the fact that the matching is only carried out according to the information such as name and/or description and the search terms is avoided, and the method and the device are further beneficial for matching accurate search results for the search request; particularly, the accuracy of the constructed matching model can be improved greatly under the condition that words in names and/or descriptions are not frequently used or real semantics cannot be reflected.

The foregoing description is only an overview of the technical solutions of the embodiments of the present invention, and the embodiments of the present invention can be implemented according to the content of the description in order to make the technical means of the embodiments of the present invention more clearly understood, and the detailed description of the embodiments of the present invention is provided below in order to make the foregoing and other objects, features, and advantages of the embodiments of the present invention more clearly understandable.

Drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the embodiments of the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:

FIG. 1 is a flow chart illustrating a training method of a semantic similarity matching model according to an embodiment of the present invention;

FIG. 2 is a flow chart illustrating a training method of a semantic similarity matching model according to another embodiment of the present invention;

FIG. 3 illustrates a process diagram of a training method of a semantic similarity matching model in one particular example;

FIG. 4 is a flow chart illustrating a method for responding to a search request according to another embodiment of the present invention;

FIG. 5 is a schematic structural diagram of a training apparatus for semantic similarity matching models according to an embodiment of the present invention;

fig. 6 is a schematic structural diagram illustrating a response device for a search request according to an embodiment of the present invention;

FIG. 7 is a schematic diagram illustrating a computing device provided by an embodiment of the invention;

fig. 8 is a schematic structural diagram of a computing device according to another embodiment of the present invention.

Detailed Description

Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

Before implementing the embodiments of the present invention, technical terms referred to hereinafter are explained in a unified manner:

O2O: the Online To Offline refers To a platform for Offline transaction by combining Offline business opportunities with the internet.

And (3) a new gating mechanism is fused into the CNN to reduce the gradient diffusion phenomenon, and compared with L STM, the Gated CNN is a network structure with a simpler model and higher convergence speed.

TRANSFORMER: the Transformer is a new framework proposed in the paper of Attention is all you needed, and proposes to directly learn the internal relationships of the source language and the target language by using an Attention mechanism.

Muti-head attribute: a multi-head attention mechanism.

DSSM (deep Structured semiconductor models): and (5) a deep semantic matching model. Massive Query and Title exposure logs in a search engine are clicked, the Query and the Title are expressed into low-dimensional semantic vectors through networks such as DNN (digital noise network) and the like, the distance between the two semantic vectors is calculated through cosine distance, and finally a semantic similarity matching model is trained. The model can be used for predicting semantic similarity of two sentences and obtaining low-dimensional semantic vector expression of a certain sentence.

Fig. 1 shows a flowchart of a training method of a semantic similarity matching model according to an embodiment of the present invention. The training method of the semantic similarity matching model provided by this embodiment may be executed by a computing device with corresponding computing capability, and the present embodiment does not limit the specific type of the computing device.

As shown in fig. 1, the method comprises the steps of:

step S110: and acquiring a plurality of sample data according to the historical search click log, wherein any sample data comprises a search word and the title information of a search result object corresponding to the search word.

The search click log records search words of historical search in the platform and click results of users corresponding to the search words. For example, record s in the search click log is as follows: the search term 111 indicates that the user clicks the objects with the information aaa, bbb, and ccc among the search results displayed by searching using the search term 111.

The search result object may be any physical object, service and/or virtual product available from the platform, and optionally, the search result object may be a store, a commodity, a service and/or internet data.

Specifically, a plurality of sample data are obtained according to records in a history search click log, and the plurality of sample data are used for training a semantic similarity matching model, wherein each sample data comprises a group of search word and title information forming information pairs, the search word is a search word of any one search record recorded in the history search click log, and the title information is semantic representation information of a search result object corresponding to the search word and can be an object name and/or object description. And the search result object can be any object clicked by the user in any search or any object not clicked by the user, any object clicked by the user can be determined through the user click result in the record of any search word, and any object not clicked by the user can be determined through excluding the object included in the user click result. Still by way of example, the search term 111 and the information aaa, the search term 111 and the information bbb, and/or the search term 111 and the information ccc may all be determined as positive sample data, while the search term 111 and any other title information (e.g., the title ddd) other than the above information may be determined as 1 negative sample data.

Step S120: and inquiring the category information of the category to which the search result object belongs aiming at any sample data, and generating a category vector of the category information, a title vector matrix of the title information and a search word vector of the search word.

Specifically, in an internet platform, objects in the platform are generally classified into a plurality of categories according to factors such as usage and functions. Any sample data comprises search words and title information of a search result object, and further by inquiring category information of a category to which the search result object belongs on the basis of the known title information of the search result object, more dimensional information of the search result object can be acquired.

Furthermore, a category vector of the category information and a search word vector of the search word are generated to represent the category information and the search word in a vector form, so that subsequent similarity calculation is facilitated. In this embodiment, a specific manner of generating the category vector and/or the search word vector is not limited, and optionally, the category vector and/or the search word vector may be generated by one-hot coding (one-hot coding), or may be obtained by generating a word segmentation vector based on a word vector table obtained by semantic mining, and then performing dimension reduction processing on the word segmentation vector.

And generating a title vector matrix of the title information, wherein the title vector matrix is composed of title participle vectors of the title participles. The word segmentation vector can also be generated by one-hot coding (one-hot coding) or by obtaining a word vector table based on semantic mining.

Step S130: and fusing the object vector of the search result object according to the category vector and the title vector matrix.

Specifically, the category vector and the title vector matrix are fused to determine the semantic weight of the title participle in the title information, and the title vector matrix is expressed by a low-dimensional semantic vector according to the semantic weight, namely, an object vector of a search result object is obtained by fusion, so that the object vector is a comprehensive expression result of the title participle vector and the semantic weight thereof. In this embodiment, the object vector of the search result object is represented by combining the category information through the fusion of the category vector and the header vector matrix, so that the object vector simultaneously contains the header information and the category information, and further, the training is facilitated to obtain a more accurate matching model.

Step S140: determining object vectors and search word vectors corresponding to a plurality of sample data as matching input data, determining click execution data of search result objects in the plurality of sample data as matching output data, training a neural network model by using the matching input data and the matching output data, and constructing a semantic similarity matching model according to training results.

The semantic similarity matching model is used for accurately predicting the matching degree between an input search word and an object to be matched, in the embodiment, the search word is represented by a search word vector, the object is represented by an object vector, and correspondingly, the object vector and the search word vector are used as matching input data in the process of model training; meanwhile, click execution data is determined as matching output data, wherein the click execution data refers to representation data of two execution conditions, namely click and non-click, respectively, if a user clicks the search result object, in the embodiment, the search result object is considered to be matched with the search word, namely, vector representations of the two are in a semantic approximate relationship; if the user does not click on the search result object, in this embodiment, the search result object is considered to be not matched with the search term, that is, the vector representations of the search result object and the search term do not have a semantic approximate relationship. Training is carried out based on the determined matching input data and the matching output data, and an accurate semantic similarity matching model can be constructed.

According to the training method of the semantic similarity matching model provided by the embodiment, a plurality of sample data are obtained according to a historical search click log, wherein any sample data comprises a search word and the title information of a search result object corresponding to the search word; for any sample data, an object vector is obtained by fusing a title vector matrix of the title information and a category vector of category information of a category to which the search result object belongs, so that the search result object is comprehensively represented by using the category information and the title information; and training by using the matching input data and the matching output data to obtain a semantic similarity matching model. Therefore, according to the scheme of the embodiment, the search result object is comprehensively represented by the category information and the title information, the title information and the category information of the search result object can be comprehensively matched with the search terms in the training process, the problem that the finally trained matching model is inaccurate due to the fact that the matching is only carried out according to the information such as name and/or description and the search terms is avoided, and the method and the device are further beneficial for matching accurate search results for the search request; particularly, the accuracy of the constructed matching model can be improved greatly under the condition that words in names and/or descriptions are not frequently used or real semantics cannot be reflected.

Fig. 2 is a flowchart illustrating a training method of a semantic similarity matching model according to another embodiment of the present invention. As shown in fig. 2, the method comprises the steps of:

step S210: training generates a word vector table.

In this embodiment, a word vector calculation model is obtained by mining semantic relations of word contexts, a word vector table is obtained by obtaining model parameters of the word vector calculation model, and different word vectors obtained by subsequently using the word vector table can reflect semantic similarity of different words.

Specifically, a long text training sample is formed according to historical user search and/or the corresponding relation between categories and objects, so that semantic relations among words are accurately mined according to the context relations of texts in the long text training sample through a neural network model, and a word vector table capable of accurately reflecting semantic similarity is obtained in a trainable mode.

In some optional embodiments, search terms of a single user searched continuously for multiple times within a preset time period are spliced to form a long text training sample, in practice, continuous multiple searches of the single user within a short time period usually correspond to the same search requirement, and based on the requirement, the search terms searched continuously for multiple times are spliced to obtain the long text training sample with semantic relation between the front and the back, so that the semantic relation of terms can be conveniently mined through search with interest in a short time. In these optional embodiments, after search terms of multiple continuous searches by a single user within a preset time period are obtained, it is determined whether the search terms of the multiple continuous searches are in a cross-category condition, if not, no search terms with different obvious interests exist in the search terms of the multiple continuous searches, and the search terms of the multiple continuous searches are spliced, for example, the search terms of the multiple continuous searches within a short time period are "milk tea", "pearl milk tea", "milk tea three brothers", and the interests of the multiple continuous searches are obviously similar. If so, discarding the obtained search terms of continuous multiple searches, for example, the search terms of continuous multiple searches in a short time are 'milk tea', 'beef', it is obvious that milk tea belongs to drinks, beef belongs to meat, and the milk tea and the beef belong to different interests, so that semantic mining errors are avoided, and splicing processing is not performed.

In other optional embodiments, for any category, the category information of the category and the header information of at least two objects belonging to the category are spliced to form a long text training sample, in practice, in the following cases, two objects belong to the same category, but the difference between the two header information is large, based on this, the header information of at least two objects of the same category and the category information of the category are spliced, and then the semantic relation of the header information of at least two objects can be established through the category information, so as to mine the semantic relation of words contained in the header information of at least two objects. The long text training sample can be formed by alternately splicing the category information and the title information of at least two objects, for example, the long text training sample is spliced in a format of 'store name 1' + 'category 1' + 'store name 2' to mine the similarity of the title information under the same category.

For example, a large number of long text training samples are obtained by performing a splicing process on a large number of search terms which are continuously searched for multiple times by a user in a preset time period and/or performing a splicing process on objects under a plurality of categories and categories, and the long text training samples are input into a word vector calculation model to be trained to obtain a word vector table. It should be noted that, in this embodiment, a specific model form of the word vector calculation model is not limited, and all the model forms that can perform context analysis on the input text to form the word vector table are included, alternatively, the word vector calculation model may be a word2vec model, and the model parameters of the word2vec model can be obtained as the word vector table after training by inputting the long text training sample into the word2vec model for self-learning.

In the above embodiment, short texts occurring in a search scene, such as search words, category information, title information, and the like, are spliced to obtain a long text training sample, and the long text training sample is input to the word vector calculation model for training, so that the limitation of the word vector calculation model on the semantic capturing capability of the short texts can be overcome, so that the word vector calculation model can sufficiently extract semantic relations among words from the long text training sample, and finally the obtained approximate relations among word vectors in the word vector table can more accurately reflect the semantic similarity among words.

Step S220: and acquiring a plurality of sample data according to the historical search click log, wherein any sample data comprises a search word and the title information of a search result object corresponding to the search word.

The search click log records search words of historical search in the platform and click results of users corresponding to the search words. And one sample data comprises search words of one time of search and title information of an object clicked by the corresponding search user or title information of an object not clicked by the corresponding search user, wherein the object not clicked by the user comprises an object which is shown in the search result but not clicked by the user or an object which is not shown in the search result and not clicked by the user.

Step S230: and inquiring the category information of the category to which the search result object belongs aiming at any sample data, and generating a category vector of the category information, a title vector matrix of the title information and a search word vector of the search word.

Specifically, the category information of the category to which the search result object belongs is queried, and the category information can limit the range of the search result object, so that the search result object can be more accurately represented through the title information and the category information. The category information, the title information and the search words are quantized and represented through the category vector, the title vector matrix and the search word vector so as to carry out matching training.

Furthermore, performing word segmentation processing on the category information, determining category word segmentation according to a word segmentation result, wherein the category information is generally formed by one or more category words; inquiring category word segmentation vectors of category word segmentation according to the word vector table; if the category word segmentation vector is one, determining the category word segmentation vector as a category vector; and if the category word segmentation vectors are multiple, performing mean pooling on the multiple category word segmentation vectors to generate the category vectors of the category information. The word vector table is obtained through training of a word vector calculation model, the similarity of semantics among category participles can be represented through the searched approximate relation among the category participle vectors, and the category vectors generated after mean pooling can be regarded as semantic vectors corresponding to the average semantics of a plurality of category participles.

And a title vector matrix and a search term vector can be generated by querying the term vector table, so that the generation standards of all vectors for matching degree calculation are consistent, and the matching degree between the search term and the object can be calculated more accurately. The method comprises the steps that in the process of generating a title vector matrix, word segmentation is carried out on title information, a title word segmentation vector of the title word segmentation is obtained by inquiring a word vector table, the title word segmentation vector forms a title vector matrix, and if only one title word is segmented, the title vector matrix only has one row; if there are multiple title tokens, then there are multiple rows in the title vector matrix. In the process of generating the search word vector, word segmentation processing is carried out on the search word, the search word vector of the search word is obtained by inquiring the word vector table, the search word vector is input into a network such as DNN or CNN, and the search word vector of the search word can be output. The search word vector can be obtained by inputting the search word segmentation vector into a CNN network which is fused into a gate control mechanism and outputting the search word vector, so that the gradient dispersion phenomenon is reduced, and the model convergence speed is accelerated.

In the process, the word vector is generated by inquiring the word vector table, so that the approximate relation among the word vectors of various types of words can reflect the semantic approximation degree among the words, the reasonable fusion of the category vector and the title vector matrix can be conveniently carried out subsequently according to the semantic approximation degree of the words corresponding to the vector, and the matching degree between the search word and the object can be conveniently and accurately calculated.

Step S240: and fusing the object vector of the search result object according to the category vector and the title vector matrix.

Specifically, the category vector and the title vector matrix are fused to obtain an object vector, so that the title information of the search result object and the category information of the belonged category are fused in the object vector, and the matching degree between the search word and the object is favorably and accurately calculated.

Further, the fusion may be performed by: calculating a first product of the category vector and each title participle vector forming the title vector matrix, and generating a title participle weight vector according to the first product; each element of the title participle weight vector is a participle weight of a title participle; and a first product of the category vector and each title participle vector can reflect semantic similarity between the category information and the title participle, and based on the semantic similarity, a participle weight of the title participle can be generated, wherein the participle weight of the title participle is positively correlated with a first product, optionally, a value of the first product or a preset multiple of the first product can be used as the participle weight of the title participle, in other words, the first product of the title participle vector and the category vector determines the weight of the title participle in an object vector representing the search result object, namely, a vector element m of the title participle weight vector_i＝d×(q·k_i) Wherein mi is an element value of the ith element of the title word segmentation weight vector, d is a preset multiple, q is a category vector, and ki is the title word segmentation vector of the ith title word segmentation. Then, the object vector of the search result object is obtained according to the fusion of the title participle weight vector and the title vector matrix, so as to obtain the search result objectThe information proportion of the title participles similar to the category information semantics in the object vector is enhanced, and the information proportion of the title participles far away from the category information semantics in the object vector is weakened, so that the object vector can accurately represent the real semantics of the object.

Furthermore, in the process of obtaining the object vector of the search result object according to the fusion of the title participle weight vector and the title vector matrix, a second product of the title participle weight vector and the title vector matrix is calculated, and the second product is determined as the object vector of the search result object. Multiplying the title word segmentation weight vector by the title vector matrix to obtain an object vector, wherein each element of the object vector obtained by fusion is the weighted average result of the word segmentation weight of each title word segmentation and the element value of the element in each title word segmentation vector, namely the vector element n of the object vector_j＝m₁×k_1j+m₂×k_2j+……m_i×k_ijWherein, nj is the jth element of the object vector, m1 and m2 … … mi are the segmentation weights of the 1 st and 2 … … i title segmentation, and k1j and k2j … … kij are the element values of the jth element in the 1 st and 2 … … i title segmentation vector, so as to accurately represent the real semantic meaning of the object.

It should be noted that the description of the fusion-acquired object vector is merely an explanation of the fusion principle, and a specific form for realizing the fusion principle is not limited to an actual implementation. In some specific embodiments, the above calculation may be implemented based on an improvement of a muti-head attribute mechanism (a multi-head attention mechanism), where in the original muti-head attribute mechanism, parameters Q, K, and V are vector matrices, and when the scheme is used in this embodiment, Q is no longer a vector matrix but a category vector, and K and V are header vector matrices, and then weights of each header participle vector in the header vector matrix may be adjusted according to the category vector, so as to more accurately represent a search result object. One specific calculation is as follows:

in the formula, Q is a category vector, K and V are both title vector matrixes, KT is the transposition of the title vector matrixes, and dk is the vector dimension of K.

In the embodiment, a category vector is generated firstly, then the participle weight of each title participle is determined according to the product of the category vector and the participle vector of different title participles in the title information, finally the participle weight of each title participle is utilized to perform fusion processing on the title participle vector in a title vector matrix to generate an object vector, so that the object can be comprehensively represented through the category information and the title information, and compared with a general concat (connecting two or more arrays), the method can adjust the weight of the information occupied by the title participle in the finally generated object vector according to the semantic approximation degree of the category information and the title participle, further better fuses the category information and the title information, and can more reasonably represent a search result object.

The process of the above fusion is illustrated below by way of a specific example: assuming that the title information of a search result object in sample data is a shop name of "one yoghurt cow" and the category information of the category to which the shop belongs is "sweet drink", after generating a category vector q, calculating the product of the title participles of "one", "sour" and "cow" respectively, such as title participles vectors k1, k2, k3 and the category vector q, so as to obtain semantic similarity between the category information and each title participle, and further endowing the title participles with high semantic similarity to the category information with higher participle weight, obviously endowing the title participles of "cow" and "sweet drink" with higher semantic similarity, endowing the title participle of "cow" with higher participle weight (such as 0.7), and endowing the title participle of "one" and "sour" with lower participle weight (such as 0.1 and 0.2), and obtaining the title participle weight vector m of (0.1,0.2, 0.7); then, when the title segmentation weight vector m2 is multiplied by a title vector matrix V formed by the title segmentation vectors of "one", "sour", and "cow" to [ k1, k2, k3], the fused object vector n is (0.1k11+0.2k21+0.7k31, 0.1k12+0.2k22+0.7k32, 0.1k13+0.2k23+0.7k33), where kij is an element value of the j-th element of the title segmentation vector of the i-th title segmentation.

Through the steps S230 to S240, an object vector of a search result object and a search word vector of a search word of each sample data can be obtained, and through the steps S230 to S240 for a plurality of sample data, an object vector and a search word vector corresponding to a plurality of sample data can be obtained.

Step S250: determining object vectors and search word vectors corresponding to a plurality of sample data as matching input data, determining click execution data of search result objects in the plurality of sample data as matching output data, training a neural network model by using the matching input data and the matching output data, and constructing a semantic similarity matching model according to training results.

Specifically, for each sample data, taking an object vector and a search word vector corresponding to the sample data as a set of matching input data of a neural network model, and taking click execution data of a search result object in the sample data by a user as matching output data of the neural network model corresponding to the set of matching input data, wherein when the search result object is an object clicked by the user, the matching output data is determined to be first data; and when the search result object is an object which is not clicked by the user, determining that the matching output data is the second data. For example, the matching output data of the user click and the user non-click are 1 and 0, respectively. Training the neural network model by using matching input data and matching output data corresponding to a plurality of sample data until the error between the training output data and the matching output data reaches an error target, and constructing by using model parameters of the neural network model when training is finished to obtain a semantic similarity matching model.

In the training process, when the search result object is an object clicked by a user, the search result object is considered to be matched with the search word, namely the vector representations of the search result object and the search word have semantic approximate relation; otherwise, the search result object is considered not to be matched with the search word, namely the vector representation of the search result object and the search word does not have semantic approximate relation, and training is carried out based on the semantic approximate relation. In some optional embodiments of the present invention, an error feature is further constructed, and training is performed in combination with the error feature, so as to improve the accuracy of a training result. Optionally, a display position feature of the search result object, a distance feature of the search result object, and/or a user activity feature of the search user are/is constructed, where the display position feature is a ranking position feature of the search result object in the search result, the distance feature is an interval feature between an object position and a search position of the search result object, and/or the user activity feature is a search click frequency feature of the search user. In practice, whether the user performs the click operation on the search result object is related to the display ranking of the search result object in the search result, the distance between the object position of the search result object and the search position, and/or the frequency of the click operation performed by the user, besides the matching degree between the search word and the search result object, and the probability that the search result object is clicked is increased when the object position of the search result object is closer to the search position and/or the frequency of the click operation performed by the user is higher, and is decreased when the object position of the search result object is closer to the search position. In these optional embodiments, the neural network model is trained according to the presentation bit feature, the distance feature and/or the user activity feature, wherein the presentation bit feature, the distance feature and/or the user activity feature are input into an error prediction network, an influence coefficient of the error feature on the click operation performed by the user is obtained through prediction, the matching output data is adjusted by using the influence coefficient, so that the correlation degree of the matching output data and the search word and the search result object is more consistent, for example, the search result object is an object clicked by the user, the matching output data is 1, if the search result object is arranged at the 1 st bit of the search result, the user is most likely to click the search result object because the search result object is presented at the 1 st bit, which indicates that the user clicks the search result object with a presentation bit error, and the influence coefficient of the presentation bit feature obtained through prediction by the error prediction network that the 1 st bit has the influence coefficient of 0.4 on the click operation performed by the user is obtained through prediction by the presentation bit feature Then the matching output data of the search result object may be adjusted to (1-0.4), and then trained with the adjusted matching output data as the output target of the training. In these alternative embodiments, by introducing the error feature, the interference of error factors such as the display position, the position and/or the user activity can be eliminated, and the accuracy of the training result can be improved.

To facilitate understanding of the present embodiment, the training method of the semantic similarity matching model is described below as a specific example. Fig. 3 is a process diagram illustrating a training method of a semantic similarity matching model in a specific example, in which a search result object is a store and title information is a store name. As shown in fig. 3, for the shop side, performing mean pooling on category participle vectors of category information of categories to which the shop belongs to obtain category vectors; performing fusion processing on the category vector and the shop name vector matrix by using a multi-head attention mechanism to obtain a shop vector; and for the search request, processing the search word segmentation vector through a CNN gate to obtain a word segmentation vector. Meanwhile, error characteristics (including display bit characteristics, distance characteristics and/or user activity characteristics) of the sample data are constructed, and error difference (namely influence coefficients) is obtained through error prediction network prediction. And continuously learning and training by using a neural network model to obtain the similarity score of the object vector and the search word vector, enabling the sum of the similarity score and the error score to approach to a matching score (namely matching output data), and finally training to obtain a semantic similarity matching model.

According to the training method of the semantic similarity matching model provided by the embodiment, a word vector table is generated by constructing the search word and the long text training sample of the category information and the title information and inputting the long text training sample into the word vector calculation model for training; generating a word segmentation vector by using the word vector table so that different word segmentation vectors can reflect the semantic similarity of different words, and fusing a category vector and a title vector matrix based on the semantic similarity to obtain an object vector so as to more accurately represent a search result object; and error characteristics are introduced to calculate error difference, so that training errors caused by directly using click execution data as a training output target are avoided, and the accuracy of a training result is improved by eliminating errors. Therefore, in the scheme of the embodiment, the defect of poor effect when the word vector calculation model trains the short text can be overcome by constructing the long text training sample and inputting the long text training sample into the word vector calculation model for training, so that the generated word vector table can accurately reflect semantic association among words; through the fusion of the category information and the title vector matrix, the word segmentation weight of different title word segments in the object vector can be adjusted, so that the rationality of the object vector obtained through fusion is improved, and the problem that a finally trained matching model is inaccurate due to the fact that the matching is only carried out according to information such as name and/or description and search words is avoided, and the method is further favorable for matching accurate search results for search requests; the matching output data is adjusted by introducing error features, so that the matching output data accords with the actual similarity, the influence of errors such as display positions is avoided, and the matching model obtained by training is more accurate.

Fig. 4 is a flowchart illustrating a method for responding to a search request according to another embodiment of the present invention. The method for responding to a search request provided by the embodiment can be applied to various internet platforms, for example, a local life service platform, an O2O platform, a takeaway platform, and the like. Moreover, the method for responding to the search request provided by the embodiment may be executed by a computing device with corresponding computing capability, and the embodiment does not limit the specific type of the computing device.

As shown in fig. 4, the method comprises the steps of:

step S410: in response to the real-time search request, a search term vector of real-time search terms is generated.

Step S420: calculating the similarity between the search word vector and the object vectors of a plurality of objects to be matched in the object library according to a semantic similarity matching model; the object vector is obtained by fusing a category vector of category information of a category to which the object to be matched belongs and a header vector matrix of header information of the object to be matched.

Step S430: and screening out target similar objects from the plurality of objects to be matched according to the similarity calculation result, and sending the header information of the target similar objects to the request end.

In this embodiment, the generation processes of the search term vector, the title vector matrix, the category vector, and the object vector can all refer to the corresponding generation processes in the sample data in the foregoing, and are not described herein again.

According to the response method of the search request provided by the embodiment, each object to be matched is represented by the object vector obtained by fusing the category vector and the title vector matrix, accordingly, in the process of performing similarity calculation by using the semantic similarity matching model, the title information and the category information can be integrated for measurement, and not only the title information is used for measurement, so that the matching accuracy can be improved, and particularly, the matching accuracy can be remarkably improved under the condition that the title information cannot sufficiently reflect the meaning of the object.

In an optional implementation manner, the semantic similarity matching model is obtained by training through the training method of the semantic similarity matching model of the embodiment corresponding to fig. 1 or fig. 2, and the object vector and the header vector matrix are fused according to the semantic approximation degree in the training process of the semantic similarity matching model, so that the semantic similarity matching model can be more accurately used for semantic similarity matching, and further, a search request can be accurately responded.

Fig. 5 is a schematic structural diagram illustrating a training apparatus for a semantic similarity matching model according to an embodiment of the present invention. As shown in fig. 5, the apparatus includes:

an obtaining module 510, adapted to obtain a plurality of sample data according to a history search click log, where any sample data includes a search word and title information of a search result object corresponding to the search word;

the vector generation module 520 is adapted to query the category information of the category to which the search result object belongs, and generate a category vector of the category information, a title vector matrix of the title information, and a search word vector of a search word, for any sample data;

a fusion module 530, adapted to obtain an object vector of the search result object according to the fusion of the category vector and the title vector matrix;

the training module 540 is adapted to determine object vectors and search term vectors corresponding to a plurality of sample data as matching input data, determine click execution data of search result objects in the plurality of sample data as matching output data, train the neural network model by using the matching input data and the matching output data, and construct a semantic similarity matching model according to a training result.

In an optional manner, the apparatus further comprises:

the word vector table generating module is suitable for splicing the search words searched continuously and repeatedly by a single user in a preset time period to form a long text training sample; and/or, aiming at any category, splicing the category information of the category and the title information of at least two objects belonging to the category to form a long text training sample; and inputting the long text training sample into a word vector calculation model to be trained to obtain a word vector table.

In an optional manner, the vector generation module is further adapted to:

performing word segmentation processing on the category information, determining category word segmentation according to word segmentation results, and inquiring category word segmentation vectors of the category word segmentation according to the word vector table;

if the category word segmentation vector is one, determining the category word segmentation vector as a category vector;

and if the category word segmentation vectors are multiple, performing mean pooling on the multiple category word segmentation vectors to generate the category vectors of the category information.

In an alternative approach, the fusion module is further adapted to:

calculating a first product of the category vector and each title participle vector forming a title vector matrix, and generating a title participle weight vector according to the first product; each element of the title participle weight vector is a participle weight of a title participle;

and fusing the title word segmentation weight vector and the title vector matrix to obtain an object vector of the search result object.

In an alternative approach, the fusion module is further adapted to:

and calculating a second product of the title participle weight vector and a title vector matrix, and determining the second product as an object vector of the search result object.

In an alternative, the training module is further adapted to:

when the search result object is an object clicked by a user, determining that the matching output data is first data;

and when the search result object is an object which is not clicked by the user, determining that the matching output data is the second data.

In an optional manner, the apparatus further comprises:

the error characteristic construction module is suitable for constructing the display position characteristic of the search result object, the distance characteristic of the search result object and/or the user activity characteristic of a search user;

the training module further comprises: and training the neural network model according to the display position characteristics, the distance characteristics and/or the user activity characteristics.

In an optional mode, the presentation position feature is a ranking position feature of the search result object in the search result; the distance feature is a spacing feature between the object position and the search position of the search result object; and/or the user liveness characteristic is a search click frequency characteristic of a search user.

Fig. 6 is a schematic structural diagram illustrating a response apparatus for a search request according to an embodiment of the present invention. As shown in fig. 6, the apparatus includes:

a vector generation module 610 adapted to generate a search term vector of real-time search terms in response to a real-time search request;

a matching module 620, adapted to calculate similarity between the search term vector and object vectors of a plurality of objects to be matched in an object library according to a semantic similarity matching model; the object vector is obtained by fusing a category vector of category information of a category to which an object to be matched belongs and a header vector matrix of header information of the object to be matched;

and the result returning module 630 is adapted to screen out a target similar object from the multiple objects to be matched according to the similarity calculation result, and send the header information of the target similar object to the requesting end.

In an optional manner, the semantic similarity matching model is obtained by training the training device of the semantic similarity matching model according to any one of the preceding items.

The embodiment of the invention provides a nonvolatile computer storage medium, wherein at least one executable instruction is stored in the computer storage medium, and the computer executable instruction can execute the training method of the semantic similarity matching model in any method embodiment.

Embodiments of the present invention provide a non-volatile computer storage medium, where at least one executable instruction is stored in the computer storage medium, and the computer executable instruction may execute a response method of a search request in any of the above method embodiments.

Fig. 7 is a schematic structural diagram of a computing device according to an embodiment of the present invention, and the specific embodiment of the present invention does not limit the specific implementation of the computing device.

As shown in fig. 7, the computing device may include: a processor (processor)702, a Communications Interface 704, a memory 706, and a communication bus 708.

Wherein: the processor 702, communication interface 704, and memory 706 communicate with each other via a communication bus 708. A communication interface 704 for communicating with network elements of other devices, such as clients or other servers. The processor 702, configured to execute the program 710, may specifically execute the relevant steps in the above-described embodiment of the training method for the semantic similarity matching model of the computing device.

In particular, the program 710 may include program code that includes computer operating instructions.

The processor 702 may be a central processing unit CPU, or an Application Specific Integrated Circuit (ASIC), or one or more Integrated circuits configured to implement an embodiment of the invention. The computing device includes one or more processors, which may be the same type of processor, such as one or more CPUs; or may be different types of processors such as one or more CPUs and one or more ASICs.

The memory 706 stores a program 710. The memory 706 may comprise high-speed RAM memory, and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.

The program 710 may specifically be used to cause the processor 702 to perform the following operations:

In an alternative, the program 710 causes the processor 702 to:

carrying out splicing processing on search terms of a single user which are continuously searched for multiple times within a preset time period to form a long text training sample; and/or, aiming at any category, splicing the category information of the category and the title information of at least two objects belonging to the category to form a long text training sample;

and inputting the long text training sample into a word vector calculation model to be trained to obtain a word vector table.

In an alternative, the program 710 causes the processor 702 to:

constructing a display position characteristic of the search result object, a distance characteristic of the search result object and/or a user activity characteristic of a search user;

and training the neural network model according to the display position characteristics, the distance characteristics and/or the user activity characteristics.

Fig. 8 is a schematic structural diagram of a computing device according to another embodiment of the present invention, and the specific embodiment of the present invention does not limit the specific implementation of the computing device.

As shown in fig. 8, the computing device may include: a processor (processor)802, a Communications Interface 804, a memory 806, and a communication bus 808.

Wherein: the processor 802, communication interface 804, and memory 806 communicate with one another via a communication bus 808. A communication interface 804 for communicating with network elements of other devices, such as clients or other servers. The processor 802, configured to execute the program 810, may specifically perform relevant steps in the above-described method embodiment for responding to a search request of a computing device.

In particular, the program 810 may include program code comprising computer operating instructions.

The processor 802 may be a central processing unit CPU, or an Application Specific Integrated Circuit (ASIC), or one or more Integrated circuits configured to implement an embodiment of the invention. The computing device includes one or more processors, which may be the same type of processor, such as one or more CPUs; or may be different types of processors such as one or more CPUs and one or more ASICs.

The memory 806 stores a program 810. The memory 806 may comprise high-speed RAM memory, and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.

The program 810 may be specifically configured to cause the processor 802 to perform the following operations:

In an alternative, the program 810 causes the processor 802 to:

the semantic similarity matching model is obtained by training through any one of the training methods of the semantic similarity matching model.

The algorithms or displays presented herein are not inherently related to any particular computer, virtual system, or other apparatus. Various general purpose systems may also be used with the teachings herein. The required structure for constructing such a system will be apparent from the description above. In addition, embodiments of the present invention are not directed to any particular programming language. It is appreciated that a variety of programming languages may be used to implement the teachings of embodiments of the present invention as described herein, and any descriptions of specific languages are provided above to disclose the best modes of embodiments of the invention.

In the description provided herein, numerous specific details are set forth. It is understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the embodiments of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: that is, the claimed embodiments of the invention require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.

Those skilled in the art will appreciate that the modules in the device in an embodiment may be adaptively changed and disposed in one or more devices different from the embodiment. The modules or units or components of the embodiments may be combined into one module or unit or component, and furthermore they may be divided into a plurality of sub-modules or sub-units or sub-components. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where at least some of such features and/or processes or elements are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

Furthermore, those skilled in the art will appreciate that while some embodiments herein include some features included in other embodiments, rather than other features, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments may be used in any combination.

The various component embodiments of the invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that a microprocessor or Digital Signal Processor (DSP) may be used in practice to implement some or all of the functionality of some or all of the components according to embodiments of the present invention. Embodiments of the invention may also be implemented as apparatus or device programs (e.g., computer programs and computer program products) for performing a portion or all of the methods described herein. Such programs implementing embodiments of the present invention may be stored on computer-readable media or may be in the form of one or more signals. Such a signal may be downloaded from an internet website or provided on a carrier signal or in any other form.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. Embodiments of the invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names. The steps in the above embodiments should not be construed as limiting the order of execution unless specified otherwise.

Claims

1. A training method of a semantic similarity matching model comprises the following steps:

2. The method of claim 1, wherein the method further comprises:

3. The method of claim 2, wherein the generating a category vector of category information further comprises:

4. A method of responding to a search request, comprising:

5. A training device of a semantic similarity matching model comprises:

6. A search request responding apparatus, comprising:

7. A computing device, comprising: the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete mutual communication through the communication bus;

the memory is used for storing at least one executable instruction, and the executable instruction causes the processor to execute the operation corresponding to the training method of the semantic similarity matching model according to any one of claims 1-3.

8. A computing device, comprising: the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete mutual communication through the communication bus;

the memory is used for storing at least one executable instruction, and the executable instruction causes the processor to execute the operation corresponding to the response method of the search request in claim 4.

9. A computer storage medium having stored therein at least one executable instruction for causing a processor to perform operations corresponding to the training method of the semantic similarity matching model according to any one of claims 1-3.

10. A computer storage medium having at least one executable instruction stored therein, the executable instruction causing a processor to perform operations corresponding to the method of responding to a search request according to claim 4.