CN113010771B

CN113010771B - Training method and device for personalized semantic vector model in search engine

Info

Publication number: CN113010771B
Application number: CN202110191195.2A
Authority: CN
Inventors: 陈咨尧; 陈强; 梁龙军
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2021-02-19
Filing date: 2021-02-19
Publication date: 2023-08-22
Anticipated expiration: 2041-02-19
Also published as: CN113010771A

Abstract

The embodiment of the application provides a training method and a training device for a personalized semantic vector model in a search engine, which relate to the technical field of application services of block chains, and comprise the following steps: acquiring a first query feature and M document features, wherein M is more than 0; converting the first query feature into a first query vector, and converting the M document features into M document vectors respectively; based on the first query vector and the M document vectors, training a personalized semantic vector model by taking a preset similarity difference value as a training target. According to the training method provided by the application, through the personalized semantic vector model, the semantic relevance of words and sentences input by a user can be considered, and further, the recommendation accuracy and the user experience of a search engine can be improved.

Description

Training method and device for personalized semantic vector model in search engine

Technical Field

The embodiment of the application relates to the technical field of application services of blockchains, in particular to a training method and device of a personalized semantic vector model in a search engine.

Background

At present, a search engine generally divides words and sentences input by a user, and then takes a document with a front similarity score in a document library in an inverted mode as a recommended document, so that the user can conveniently and quickly search for a required document. However, there is a possibility that recommended documents obtained by word segmentation and documents actually searched by the user come in and go out. For example, assuming that the words input by the user are "road motor vehicle traffic rules", in the above manner, the "road motor vehicle traffic rules" are first split into "roads", "motor vehicles" and "traffic rules", and then documents with these words hit respectively are selected in a document library in an inverted manner, for example, the documents "what traffic rules are violated by motor vehicles and speeding calculation on non-motor vehicles" are recommended preferentially, the documents "motor vehicles and non-motor vehicles in traffic rules" and the documents "motor vehicles and electric bicycle accidents" are recommended preferentially, etc.; typically, however, the user is a relevant document that wishes to recommend a specific traffic rule, such as the document "what traffic road travel rules are" or the document "what traffic road travel rules are", etc. It can be seen that although the actual recommended documents and the words and sentences input by the user are related, certain access exists between the actual recommended documents and the documents actually required by the user, and the user experience is reduced.

Disclosure of Invention

The embodiment of the application provides a training method and a training device for a personalized semantic vector model in a search engine, which can consider the semantic relevance of words and sentences input by a user through the personalized semantic vector model, and further can improve the recommendation accuracy and user experience of the search engine. For example, assuming that the words and sentences input by the user are "road motor vehicle traffic rules", the characteristics after word segmentation are semantically associated by using a personalized semantic vector model, so that the finally recommended document can be the document "which traffic road driving rules exist", and the like.

In one aspect, an embodiment of the present application provides a training method for a personalized semantic vector model in a search engine, including:

acquiring a first query feature and M document features, wherein M is more than 0;

converting the first query feature into a first query vector, and converting the M document features into M document vectors respectively;

based on the first query vector and the M document vectors, training a personalized semantic vector model by taking a preset similarity difference value as a training target;

the similarity difference is a difference between a similarity score between a positive example vector and a query vector and a similarity score between a negative example vector and the query vector, wherein the positive example vector is a vector converted from document features forming a positive example with the query features, and the negative example vector is a vector converted from document features forming a negative example with the query features.

On the other hand, the embodiment of the application provides a training device for a personalized semantic vector model in a search engine, which comprises the following components:

the acquisition unit is used for acquiring the first query feature and M document features, wherein M is more than 0;

the conversion unit is used for converting the first query feature into a first query vector and converting the M document features into M document vectors respectively;

the training unit is used for training a personalized semantic vector model by taking a preset similarity difference value as a training target based on the first query vector and the M document vectors;

In some implementations, the personalized semantic vector model includes a query-side encoder, a positive-side encoder, a negative-side encoder, and a scoring module; the conversion module is specifically used for: converting the first query feature into the first query vector using the query side encoder; converting the document features forming the positive example with the first query feature in the M document features into a first positive example vector in the M document vectors by using the positive example side encoder; converting the document features forming negative examples with the query features in the M document features into first negative example vectors in the M document vectors by using the negative example side encoder; wherein, this training unit is specifically used for: calculating a first difference value by using the scoring module; the first difference value is a difference value between a similarity score between the first positive example vector and the first query vector and a similarity score between the first negative example vector and the first query vector; based on the first difference, the personalized semantic vector model is trained by taking the similarity difference as a training target.

In some implementations, the negative side encoder includes a negative side text conversion module, a negative side non-text conversion module, and a negative side fusion module; the negative side encoder and the positive side encoder share parameters.

In some implementations, the query-side encoder and the positive-side encoder do not share parameters, and the query-side encoder and the negative-side encoder do not share parameters.

In some implementations, the query-side encoder includes a query-side text conversion module, a query-side non-text conversion module, and a query-side vector fusion module; the first query feature includes a first text feature and a first non-text feature; the conversion module is specifically used for: converting the first text feature into a first text vector by using the query-side text conversion module; converting the first non-text feature into a first non-text vector by using the query-side non-text conversion module; and fusing the first text vector and the first non-text vector by using the query side vector fusion module to obtain the first query vector.

In some implementations, the first non-text feature includes at least one of the following features of the user: age, gender, portrait, academic, language used and mobile phone system used.

In some implementations, the positive side encoder includes a positive side text conversion module, a positive side non-text conversion module, and a positive side fusion module; the document features forming a positive example with the first query feature in the M document features comprise a second text feature and a second non-text feature; the conversion module is specifically used for: converting the second text feature into a second text vector by using the positive example side text conversion module; converting the second non-text feature into a second non-text vector by using the positive side non-text conversion module; and fusing the second text vector and the second non-text vector by using the normal side vector fusion module to obtain the first normal vector.

In some implementations, the second non-textual feature includes at least one of: authority of document content, authority of document author and number of pieces of public numbers associated with the document.

In some implementations, the acquiring unit is specifically configured to:

acquiring X non-text features in training data, wherein X is more than 0; selecting N times from the X non-text features in a random selection mode to obtain N groups of non-text features; each set of non-text features in the N sets of non-text features includes Y non-text features, N > 1, X > Y > 0; the N groups of non-text features are used as N training sets of the personalized semantic vector model, and Y non-text features included in one group of non-text features in the N groups of non-text features are used as non-text features in the first query feature in one training set in the N training sets and document non-text features in the M document features.

In some implementations, the apparatus may further include a display unit for:

acquiring a query request through a query window, wherein the query request comprises a second query feature;

converting the second query feature into a second query vector using the personalized semantic vector model;

based on the second query vector, K document features with similarity scores being the front of the second query vector are selected in a document library, wherein K is more than 0;

and displaying the K document features under the query window according to the sequence of the similarity scores from large to small.

In another aspect, the present application provides an electronic device, including:

a processor adapted to implement computer instructions; the method comprises the steps of,

a computer readable storage medium storing computer instructions adapted to be loaded by a processor and to perform the above-described training method based on an actively learned image video quality assessment model.

In another aspect, embodiments of the present application provide a computer readable storage medium storing computer instructions that, when read and executed by a processor of a computer device, cause the computer device to perform a method of training a personalized semantic vector model in a search engine as described above.

In another aspect, embodiments of the present application provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium and executes the computer instructions to cause the computer device to perform the training method of the personalized semantic vector model in the search engine described above.

In the embodiment of the application, based on a first query vector and M document vectors, a preset similarity difference value is used as a training target to train a personalized semantic vector model; in the process of training the personalized semantic vector model, the semantic relevance of words and sentences input by a user is considered, and in the process of recommending the search engine, the recommendation accuracy and the user experience of the search engine can be improved. For example, assuming that the words and sentences input by the user are "road motor vehicle traffic rules", the characteristics after word segmentation are semantically associated by using a personalized semantic vector model, so that the finally recommended document can be the document "which traffic road driving rules exist", and the like.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic block diagram of a double tower model provided by an embodiment of the present application.

Fig. 2 is an example of a display interface provided by an embodiment of the present application.

FIG. 3 is a schematic block diagram of a system framework of a search engine provided by an embodiment of the present application.

Fig. 4 is a schematic flow chart of a training method of a personalized semantic vector model in a search engine according to an embodiment of the present application.

FIG. 5 is a schematic block diagram of a personalized semantic vector model provided by an embodiment of the present application.

Fig. 6 is a schematic block diagram of a training device for personalized semantic vector models in a search engine according to an embodiment of the present application.

Fig. 7 is a schematic block diagram of an electronic device provided by an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

The scheme provided by the application can relate to the technical field of block chains.

Blockchains are novel application modes of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanisms, encryption algorithms, and the like. The Blockchain (Blockchain), which is essentially a decentralised database, is a string of data blocks that are generated by cryptographic means in association, each data block containing a batch of information of network transactions for verifying the validity of the information (anti-counterfeiting) and generating the next block. The blockchain may include a blockchain underlying platform, a platform product services layer, and an application services layer.

The blockchain underlying platform may include processing modules for user management, basic services, smart contracts, operation monitoring, and the like. The user management module is responsible for identity information management of all blockchain participants, including maintenance of public and private key generation (account management), key management, maintenance of corresponding relation between the real identity of the user and the blockchain address (authority management) and the like, and under the condition of authorization, supervision and audit of transaction conditions of certain real identities, and provision of rule configuration (wind control audit) of risk control; the basic service module is deployed on all block chain node devices, is used for verifying the validity of a service request, recording the service request on a storage after the effective request is identified, for a new service request, the basic service firstly analyzes interface adaptation and authenticates the interface adaptation, encrypts service information (identification management) through an identification algorithm, and transmits the encrypted service information to a shared account book (network communication) in a complete and consistent manner, and records and stores the service information; the intelligent contract module is responsible for registering and issuing contracts, triggering contracts and executing contracts, a developer can define contract logic through a certain programming language, issue the contract logic to a blockchain (contract registering), invoke keys or other event triggering execution according to the logic of contract clauses to complete the contract logic, and simultaneously provide a function of registering contract upgrading; the operation monitoring module is mainly responsible for deployment in the product release process, modification of configuration, contract setting, cloud adaptation and visual output of real-time states in product operation, for example: alarms, monitoring network conditions, monitoring node device health status, etc.

The platform product service layer provides basic capabilities and implementation frameworks of typical applications, and developers can complete the blockchain implementation of business logic based on the basic capabilities and the characteristics of the superposition business. The application service layer provides the application service based on the block chain scheme to the business participants for use.

More specifically, the scheme provided by the embodiment of the application is suitable for the technical field of application service of the block chain.

In order to facilitate understanding of the aspects of the present application, the following description will refer to related terms.

Query (Query): refers to the problem of user search input.

Document (Document): the results returned by the pointer to the user input question are typically in the form of a document title or web page.

Click positive example: refers to a Query Document (Document) pair formed by a Query input by a user in search and a corresponding Document with click action.

The strong negative example is that the query input by the user in the search and the query document pair formed by the exposed but not clicked document or the query document pair which is similar to the document in terms of characters but has different essence meanings.

Weak negative example: refers to a query document pair consisting of a query entered by a user in a search and a randomly selected document.

Batch (Batch): the size of the amount of data fed per batch when training the model.

Conversion (transducer) model: for example, reference may be made to the models provided in the following articles: attention is not paid (Attention Is All You Need).

Bi-directional encoder representation (Bidirectional Encoder Representations fro m Transformers, BERT) model from conversion: is a language representation model that the BERT aims to pre-train a deep bi-directional representation by jointly adjusting the left and right contexts in all layers. Thus, only one extra output layer is needed to fine tune the pre-trained BERT to create the most advanced models for a wide range of tasks (e.g., answer questions and language inference tasks) without requiring extensive modification of the task-specific model structure. For example, reference may be made to the models in the following articles: bert deep bi-directional translation Pre-trained language understanding (Bert: pre-training of deep bidirectional transformers for language understanding).

Fig. 1 is a schematic block diagram of a dual tower model 100 provided by an embodiment of the present application.

As shown in fig. 1, the dual-tower model 100 may include a query-side parsing module 111, a query-side encoder 112, a document-side parsing module 121, a document-side encoder 122, and a similarity determination unit 130. Wherein the query side parsing module 111 processes the inputted problem as a query feature, the document side parsing module 121 processes the document as a document feature, the query side encoder 112 converts the obtained query feature into a query vector, the document side encoder 122 converts the obtained document feature into a document vector, and the similarity determining unit 130 obtains a similarity score of the query vector and the document vector based on the received query vector and the document vector. During the training phase, the dual tower model 100 may be trained using positive examples, strong negative examples, and weak negative examples. In the prediction stage, the distance between the query vector and the document vector may be calculated by using the twin-tower model 100, and then, whether the query vector and the document vector are similar may be determined by using the twin-tower model 100 based on the distance between the query vector and the document vector.

It should be noted that, in the embodiment of the present application, whether the query feature or the document feature is aimed at, it may include a Text (Text) feature and a non-Text feature, which is not limited in particular. For example, the non-text features may include a Social (Social) feature and an address (location) feature. Non-textual features may also be referred to as searchers and contexts (Searcher and context). Non-textual features for query features, which may be some features related to the user; for example, the social feature may be attribute information of the user, and the address feature may be an address of the user or a location of the electronic device. For non-text features in the document features, there may be some features related to the document; for example, the social feature may be social information of the document, such as authority of the document or authority of the document author; the address feature may be the source of the document, such as the public number where the public document is located.

In a specific implementation, after the double-tower model 100 divides the words and sentences input by the user, the documents with the front similarity scores in the document library are used as recommended documents in an inverted mode, so that the user can conveniently and quickly search for the required documents. However, there is a possibility that recommended documents obtained by word segmentation and documents actually searched by the user come in and go out. For example, assuming that the words input by the user are "road motor vehicle traffic rules", in the above manner, the "road motor vehicle traffic rules" are first split into "roads", "motor vehicles" and "traffic rules", and then documents with these words hit respectively are selected in a document library in an inverted manner, for example, the documents "what traffic rules are violated by motor vehicles and speeding calculation on non-motor vehicles" are recommended preferentially, the documents "motor vehicles and non-motor vehicles in traffic rules" and the documents "motor vehicles and electric bicycle accidents" are recommended preferentially, etc.; typically, however, the user is a relevant document that wishes to recommend a specific traffic rule, such as the document "what traffic road travel rules are" or the document "what traffic road travel rules are", etc. It can be seen that although the actual recommended documents and the words and sentences input by the user are related, certain access exists between the actual recommended documents and the documents actually required by the user, and the user experience is reduced.

In addition, in the stage of training the double-tower model 100, positive examples, strong negative examples, weak negative examples and non-text features (including portrait features and document features of a user) can be used for training the double-tower model 100, different non-text features correspond to different scenes, however, due to the limitation of training data, the double-tower model 100 cannot cover all actual scenes, and further, documents actually recommended by the double-tower model 100 and words and sentences input by the user are related, but have certain access to documents actually required by the user, and user experience is reduced.

The embodiment of the application provides a training method of a personalized semantic vector model in a search engine, which can consider the semantic relevance of words and sentences input by a user through the personalized semantic vector model, and further can improve the recommendation accuracy and user experience of the search engine. For example, assuming that the words and sentences input by the user are "road motor vehicle traffic rules", the characteristics after word segmentation are semantically associated by using a personalized semantic vector model, so that the finally recommended document can be the document "which traffic road driving rules exist", and the like.

Specifically, the personalized semantic vector model is mapped to a corresponding vector space by adopting a conversion (transducer) model, and the query non-text feature and the document non-text feature are mapped to the corresponding vector space by adopting respective full connection layers; the document features of the positive and negative examples are placed in a batch (batch) for parallel calculation, and the personalized semantic vector model is obtained through training, so that the expression of vectors generated by the personalized semantic vector model in mass recall can be improved.

In addition, in the data construction, various actual scenes are simulated in a sampling mode, so that the robustness of the personalized semantic vector model in actual use is improved.

It should be noted that the personalized semantic vector model may be used in, for example, an applet search engine, and may also be applied to a web page version search engine, which is not particularly limited in the embodiments of the present application. Fig. 2 is an example of a display interface provided by an embodiment of the present application, as shown in fig. 2, in the display interface 200, a query keyword or a query word (i.e. a query or a query feature in the present application) is input through a query window, a zoom image of the recommended document 1 may be displayed in a display area of the recommended document 1, a zoom image of the recommended document 2 may be displayed in a display area of the recommended document 2, a zoom image of the recommended document 3 may be displayed in a display area of the recommended document 3, and a zoom image of the recommended document 4 may be displayed in a display area of the recommended document 4, so that a user may click on a corresponding display area directly to enter a page on which the complete recommended document is displayed, thereby assisting the user to quickly find a target document.

Fig. 3 is a schematic block diagram of a system framework 300 of a search engine provided by an embodiment of the present application.

As shown in FIG. 3, the system framework 300 of the search engine includes a web Crawler (Crawler) generated data source 310, an offline vectorization task model 320 for vectorizing data in the data source, an Indexer 330 for the generated index library (Indexer), a vector recall module 330 for searching for documents with highest similarity, and an online vectorization task model 340 for vectorizing query features. In an implementation, first, portions of a web page are collected from the network, and then these raw materials are processed to form a data source 310; after the data in the data source 310 passes through the offline vectorization module 320 and the indexer 330, an index portion is formed and a portion waiting for the input of a query keyword; the vector recall module 330 may finally calculate the search system to provide the ranked query results according to the similarity between the document and the query feature, and provide the ranked query results to the user.

Web crawler: is a tool for acquiring network information, and specifically, from a pre-established list of global resource locators (Uniform Resource Locator, URLs), a breadth-first or depth-first strategy may be used to traverse the network and download documents. For example, after a web crawler accesses a web page, it analyzes it, extracts a new URL, adds it to the access list, which is a hyperlink queue or stack maintained in the system, and then recursively repeats the access until the queue or stack is empty. Whether the design of the web crawler reasonably directly influences the efficiency of the web crawler for accessing the network, so that the quality of a search database can be influenced, and in addition, the influence of the web crawler on the network and the accessed site must be considered when the web crawler is designed, so that the web crawler is the most main part for acquiring resources in the whole search engine, but the efficiency of the web crawler is not completely dependent on the design of a designer to a great extent, and the quality of the network also influences the crawling effect to a certain extent. The web crawlers work to store crawled web pages into a data source.

The index library is a huge database, and web pages grabbed by the web crawlers are put into the index library through the index established by the indexer. Of course, the search engine may index in a different manner. This index should include a positive list from page to index word, as well as an inverted list built for the necessary data items, such as keyword to web page inverted list, keyword site to web page inverted list, and so forth. Other search engines do not index all words of the entire web page file, and some only analyze the title or first few pieces of content of the web page file and then index, because these words at the title or first few pieces of content are already well characterized by the document. The words in the title are more important than the words in the ordinary paragraph, and the words have good characteristics in terms of semanteme, so that the expression strength of the words in the mathematical model is increased when the index is established, and the later vector recall module 340 can obviously match the words with the query words when the index is established. These tasks are performed by the indexer 330, and the end result of the tasks is an index library.

Vector recall module 340 is operable to respond to a user's retrieval request and track the user's retrieval behavior. When a user submits a request, vector recall module 340 obtains the data of the relevant documents and index words from the index, and then calculates the relevance of the web pages and query words in a broad sense according to the corresponding algorithm. And finally, sorting according to the correlation degree and outputting the result to a user. The user may respond to the results (e.g., access a web page in the results page) by obtaining the results web page, which is tracked and recorded by vector recall module 340. It is also important to process keywords before searching, which may include query expansion and query reconstruction, in particular. The former mainly uses synonyms or near-meaning words to expand keywords, for example, when the keywords searched by a user are computers, the processing system can return documents related to the synonyms such as computers, microcomputers and the like, and the accuracy of searching is improved. The latter refers to the appropriate modification of the keywords using the user's feedback information. Finally, it should be mentioned that the document ranking obtained by the search engine is no longer the result of the computation of a simple mathematical model, but rather the final ranking result obtained by integrating many other relevant factors. One of the most significant factors that may be considered here is the importance of the web page, although the algorithm for determining the importance of the web page is also incorporated into the generalized vector recall module 340.

The indexer 330 may run offline vector tasks periodically using the personalized semantic vector model and index periodically on the vector recall module, specifically, may index through various periods, such as daily updates in units of data of approximately one week, and weekly updates in units of data of approximately three months or longer, for example. Alternatively, in the present application, an index established in data of a unit of approximately one week may be referred to as a hot index, and an index established in data of a unit of approximately three months or more may be referred to as a cold index.

Wherein the offline vectorization task model 320 and the online vectorization task model 340 may be implemented by the personalized semantic vector model according to the present application. On an offline service side, the documents in the index library are all converted into low-dimensional document vectors through a personalized semantic vector model, so that unique identifications of the document vectors and the index library of the document vectors are formed; on the online service side, query features are real-time inferred by the personalized semantic vector model into low-dimensional query vectors, and K documents closest to the low-dimensional query vectors are recalled from an index library through a vector retrieval tool and used as input of a sorting flow of a search system. As an example, the vector retrieval tool may be a Faiss, which is a cluster and similarity search library for Facebook artificial intelligence (Facebook AI) team open source, providing efficient similarity searches and clusters for dense vectors, supporting billions level vector searches.

FIG. 4 is a schematic flow chart of a training method 400 for personalized semantic vector models in a search engine provided by an embodiment of the present application. It should be noted that, the scheme provided by the embodiment of the present application may be implemented by any electronic device having data processing capability. For example, the electronic device may be implemented as a server. The server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing cloud services, a cloud database, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, basic cloud computing services such as big data and an artificial intelligent platform, and the server may be directly or indirectly connected through a wired or wireless communication mode.

As shown in fig. 4, the method 400 may include:

s410, acquiring a first query feature and M document features, wherein M is more than 0; in a specific implementation, the personalized semantic vector model may be trained by a user searching for a first query feature and a first document feature in a behavior log.

S420, converting the first query feature into a first query vector, and converting the M document features into M document vectors respectively;

S430, training a personalized semantic vector model by taking a preset similarity difference value as a training target based on the first query vector and the M document vectors;

the similarity difference is a difference between a similarity score between a positive example vector and a query vector and a similarity score between a negative example vector and the query vector, wherein the positive example vector is a vector converted from document features forming a positive example with the query features, and the negative example vector is a vector converted from document features forming a negative example with the query features. For example, the cosine of the angle between the normal vector and the query vector may be used to represent the similarity score between the normal vector and the query vector. The cosine of the angle between the negative example vector and the query vector may be used to represent the similarity score between the negative example vector and the query vector.

Fig. 5 is a schematic block diagram of a personalized semantic vector model 500 provided by an embodiment of the present application.

As shown in fig. 5, the personalized semantic vector model 500 includes a query-side encoder (encod) 510, a positive-side encoder (Pos Doc encod) 520, a negative-side encoder (Neg Doc encod) 530, and a scoring (Score) module 540. The positive side encoder 520 is used for processing the encoder of the document feature of the positive example with the query feature, the negative side encoder 530 is used for processing the encoder of the document feature of the negative example with the query feature, and the scoring module 540 can also be called a pair scoring function (Pairwise Score Function) module for calculating the difference value of the two similarity scores. Based on this, the first query feature is converted into the first query vector using the query-side encoder 510; converting the document features forming the positive example with the first query feature in the M document features into a first positive example vector in the M document vectors by using the positive example side encoder 520; converting the document features forming a negative example with the query feature in the M document features into a first negative example vector in the M document vectors by using the negative example side encoder 530; calculating a first difference using the scoring module 540; the first difference value is a difference value between a similarity score between the first positive example vector and the first query vector and a similarity score between the first negative example vector and the first query vector; based on the first difference, the personalized semantic vector model 500 is trained with the similarity difference as a training target.

In some embodiments, as shown in fig. 5, the query-side encoder 510 includes a query-side text conversion module 511, a query-side non-text conversion module 512, and a query-side vector fusion module 513; the first query feature includes a first text feature and a first non-text feature; based on this, the first text feature is converted into a first text vector by the query-side text conversion module 511; converting the first non-text feature into a first non-text vector using the query-side non-text conversion module 512; and fusing the first text vector and the first non-text vector by using the query side vector fusion module 513 to obtain the first query vector. Optionally, the first non-text feature comprises at least one of the following features of the user: age, gender, portrait, academic, language used and mobile phone system used. Of course, the specific features included in the first non-textual feature described above are merely examples and should not be construed as limiting the application.

In some embodiments, the positive side encoder 520 includes a positive side text conversion module 521, a positive side non-text conversion module 522, and a positive side fusion module 523; the document features forming a positive example with the first query feature in the M document features comprise a second text feature and a second non-text feature; based on this, the second text feature is converted into a second text vector by the positive example side text conversion module 521; converting the second non-text feature to a second non-text vector using the positive side non-text conversion module 522; the positive side vector fusion module 523 is utilized to fuse the second text vector and the second non-text vector to obtain the first positive vector. Optionally, the second non-text feature includes at least one of: authority of document content, authority of document author and number of pieces of public numbers associated with the document. Of course, the specific features included in the second non-textual feature described above are merely examples and should not be construed as limiting the application. In one implementation, the negative side encoder 530 includes a negative side text conversion module, a negative side non-text conversion module, and a negative side fusion module; the negative side encoder 530 and the positive side encoder 520 share parameters. In one implementation, the query side encoder 510 and the positive side encoder 520 do not share parameters, and the query side encoder 510 and the negative side encoder 530 do not share parameters. In the embodiment of the present application, the negative side encoder 530 and the positive side encoder 520 are configured as shared parameters, and the query side encoder 510 and the positive side encoder 520, and the query side encoder 510 and the negative side encoder 530 are designed as unshared parameters, so that learning ability of the personalized semantic vector model 500 for semantic distinction between positive and negative examples can be ensured, and further, semantic recognition performance of the personalized semantic vector model 500 can be ensured.

Aiming at the personalized semantic vector model 500, the clicked document features are used for forming positive examples, the untrimmed document features are exposed, the document features recalled in a traditional mode are used for forming strong negative examples, the randomly sampled document features are used for forming weak negative examples, and the personalized semantic vector model 500 is trained by adopting the positive examples, the strong negative examples and the weak negative examples, so that the personalized semantic vector model 500 has semantic understanding capability.

In some embodiments, the S410 may include:

For example, the text conversion module according to the present application may employ an optimal conversion (transducer) module, and the non-text conversion module may employ a full connection layer module. The non-text features simulate non-text features included in the actual scene or missing non-text features in the actual scene in a sampled manner, such that the personalized semantic vector model 500 can cope with scenes that include various non-text features or scenes that include various combinations of non-text features. For example, assuming that a piece of training data has several tens of non-text query features and document features, the Rongguang randomly selects 25% of the features as a scene with some non-text features missing, and as a training set, repeats the above operation to sample the same piece of data several times to obtain multiple training sets, so that the personalized semantic vector model 500 can cover as many different scenes with some non-text features missing as possible.

In some embodiments, the method 400 may further comprise:

acquiring a query request through a query window, wherein the query request comprises a second query feature; converting the second query feature into a second query vector using the personalized semantic vector model; based on the second query vector, K document features with similarity scores being the front of the second query vector are selected in a document library, wherein K is more than 0; and displaying the K document features under the query window according to the sequence of the similarity scores from large to small.

For example, K document features with similarity scores earlier than the second query vector may be selected in the document library based on the second query vector by the vector recall module 340 shown in fig. 3, and the detailed implementation may refer to the description of fig. 3, and the description is omitted here for avoiding repetition. In addition, it should be noted that the value of K may be preset, or may be information carried in the query request, which is not limited in the embodiment of the present application.

It should be understood that the personalized semantic vector model 500 shown in FIG. 5 is only an example of the present application and should not be construed as limiting the present application. For example, in other alternative embodiments, the positive side encoder 520 and the negative side encoder 530 may be combined into one encoder, or the positive side encoder 520 and the negative side encoder 530 may respectively correspond to one scoring module, which is not particularly limited in the embodiment of the present application.

The preferred embodiments of the present application have been described in detail above with reference to the accompanying drawings, but the present application is not limited to the specific details of the above embodiments, and various simple modifications can be made to the technical solution of the present application within the scope of the technical concept of the present application, and all the simple modifications belong to the protection scope of the present application. For example, the specific features described in the above embodiments may be combined in any suitable manner, and in order to avoid unnecessary repetition, various possible combinations are not described further. As another example, any combination of the various embodiments of the present application may be made without departing from the spirit of the present application, which should also be regarded as the disclosure of the present application.

It should be further understood that, in the various method embodiments of the present application, the sequence numbers of the foregoing processes do not mean the order of execution, and the order of execution of the processes should be determined by the functions and internal logic of the processes, and should not constitute any limitation on the implementation process of the embodiments of the present application.

The method provided by the embodiment of the application is described above, and the device provided by the embodiment of the application is described below.

Fig. 6 is a schematic block diagram of a training apparatus 600 for personalized semantic vector models in a search engine provided by an embodiment of the present application.

As shown in fig. 6, the apparatus 600 may include:

an obtaining unit 610, configured to obtain a first query feature and M document features, where M > 0;

a conversion unit 620, configured to convert the first query feature into a first query vector, and convert the M document features into M document vectors, respectively;

the training unit 630 is configured to train a personalized semantic vector model based on the first query vector and the M document vectors, with a preset similarity difference as a training target;

In some embodiments, the personalized semantic vector model includes a query-side encoder, a positive-side encoder, a negative-side encoder, and a scoring module; the conversion module 620 is specifically configured to: converting the first query feature into the first query vector using the query side encoder; converting the document features forming the positive example with the first query feature in the M document features into a first positive example vector in the M document vectors by using the positive example side encoder; converting the document features forming negative examples with the query features in the M document features into first negative example vectors in the M document vectors by using the negative example side encoder; the training unit 630 is specifically configured to: calculating a first difference value by using the scoring module; the first difference value is a difference value between a similarity score between the first positive example vector and the first query vector and a similarity score between the first negative example vector and the first query vector; based on the first difference, the personalized semantic vector model is trained by taking the similarity difference as a training target.

In some embodiments, the negative side encoder includes a negative side text conversion module, a negative side non-text conversion module, and a negative side fusion module; the negative side encoder and the positive side encoder share parameters.

In some embodiments, the query-side encoder and the positive-side encoder do not share parameters, and the query-side encoder and the negative-side encoder do not share parameters.

In some embodiments, the query-side encoder includes a query-side text conversion module, a query-side non-text conversion module, and a query-side vector fusion module; the first query feature includes a first text feature and a first non-text feature; the conversion module 620 is specifically configured to: converting the first text feature into a first text vector by using the query-side text conversion module; converting the first non-text feature into a first non-text vector by using the query-side non-text conversion module; and fusing the first text vector and the first non-text vector by using the query side vector fusion module to obtain the first query vector.

In some embodiments, the first non-text feature comprises at least one of the following features of the user: age, gender, portrait, academic, language used and mobile phone system used.

In some embodiments, the positive side encoder includes a positive side text conversion module, a positive side non-text conversion module, and a positive side vector fusion module; the document features forming a positive example with the first query feature in the M document features comprise a second text feature and a second non-text feature; the conversion module 620 is specifically configured to: converting the second text feature into a second text vector by using the positive example side text conversion module; converting the second non-text feature into a second non-text vector by using the positive side non-text conversion module; and fusing the second text vector and the second non-text vector by using the normal side vector fusion module to obtain the first normal vector.

In some embodiments, the second non-textual feature includes at least one of: authority of document content, authority of document author and number of pieces of public numbers associated with the document.

In some embodiments, the obtaining unit 610 is specifically configured to:

In some embodiments, the apparatus 600 may further include a display unit for:

It should be understood that apparatus embodiments and method embodiments may correspond with each other and that similar descriptions may refer to the method embodiments. To avoid repetition, no further description is provided here. Specifically, the apparatus 600 may correspond to a corresponding main body in the method 400 for executing the embodiment of the present application, and each unit in the apparatus 600 is not described herein for brevity to implement a corresponding flow in the method 400.

It should also be understood that each unit in the video processing apparatus according to the embodiments of the present application may be formed by combining each unit into one or several other units, or some unit(s) thereof may be formed by splitting into a plurality of units with smaller functions, which may achieve the same operation without affecting the implementation of the technical effects of the embodiments of the present application. The above units are divided based on logic functions, and in practical applications, the functions of one unit may be implemented by a plurality of units, or the functions of a plurality of units may be implemented by one unit. In other embodiments of the present application, the video processing apparatus may also include other units, and in practical applications, these functions may also be implemented with assistance from other units, and may be implemented by cooperation of a plurality of units. According to another embodiment of the present application, the video processing apparatus according to the embodiment of the present application may be constructed by running a computer program (including program code) capable of executing steps involved in the respective methods on a general-purpose computing device of a general-purpose computer including a processing element such as a Central Processing Unit (CPU), a random access storage medium (RAM), a read only storage medium (ROM), and the like, and a storage element, and implementing the video processing method of the embodiment of the present application. The computer program may be recorded on a computer readable storage medium, and loaded into an electronic device and executed therein to implement a corresponding method of an embodiment of the present application.

In other words, the units referred to above may be implemented in hardware, or may be implemented by instructions in software, or may be implemented in a combination of hardware and software. Specifically, each step of the method embodiment in the embodiment of the present application may be implemented by an integrated logic circuit of hardware in a processor and/or an instruction in software form, and the steps of the method disclosed in connection with the embodiment of the present application may be directly implemented as a hardware decoding processor or implemented by a combination of hardware and software in the decoding processor. Alternatively, the software may reside in a well-established storage medium in the art such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, and the like. The storage medium is located in a memory, and the processor reads information in the memory, and in combination with hardware, performs the steps in the above method embodiments.

Fig. 7 is a schematic structural diagram of an electronic device 700 provided in an embodiment of the present application.

As shown in fig. 7, the electronic device 700 includes at least a processor 710 and a computer readable storage medium 720. Wherein the processor 710 and the computer-readable storage medium 720 may be connected by a bus or other means. The computer readable storage medium 720 is for storing a computer program 721, the computer program 721 comprising computer instructions, and the processor 710 is for executing the computer instructions stored by the computer readable storage medium 720. Processor 710 is a computing core and a control core of electronic device 700 that are adapted to implement one or more computer instructions, in particular to load and execute one or more computer instructions to implement a corresponding method flow or a corresponding function.

As an example, the processor 710 may also be referred to as a central processor (CentralProcessingUnit, CPU). Processor 710 may include, but is not limited to: a general purpose processor, digital signal processor (Digital Signal Processor, DSP), application specific integrated circuit (Application Specific Integrated Circuit, ASIC), field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like.

By way of example, computer readable storage medium 720 may be high speed RAM memory or Non-volatile memory (Non-VolatileMemorye), such as at least one magnetic disk memory; alternatively, it may be at least one computer-readable storage medium located remotely from the aforementioned processor 710. In particular, computer-readable storage media 720 include, but are not limited to: volatile memory and/or nonvolatile memory. The nonvolatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable EPROM (EEPROM), or a flash Memory. The volatile memory may be random access memory (Random Access Memory, RAM) which acts as an external cache. By way of example, and not limitation, many forms of RAM are available, such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (Double Data Rate SDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), and Direct memory bus RAM (DR RAM).

As shown in fig. 7, the electronic device 700 may also include a transceiver 730. The processor 710 may control the transceiver 730 to communicate with other devices, and in particular, may send information or data to other devices or receive information or data sent by other devices. Transceiver 730 may include a transmitter and a receiver. Transceiver 730 may further include antennas, the number of which may be one or more.

It should be noted that, the components in the electronic device 700 are connected through a bus system, where the bus system may include a power bus, a control bus, a status signal bus, and the like in addition to a data bus, which is not specifically limited in the embodiment of the present application.

In one implementation, the electronic device 700 may be the training apparatus 600 of the personalized semantic vector model in the search engine shown in FIG. 7; the computer readable storage medium 720 has stored therein computer instructions; computer instructions stored in computer-readable storage medium 720 are loaded and executed by processor 710 to implement the corresponding steps in the method embodiment shown in fig. 7; in particular, the computer instructions in the computer-readable storage medium 720 are loaded by the processor 710 and perform the corresponding steps, and for avoiding repetition, a detailed description is omitted here.

According to another aspect of the present application, the embodiment of the present application further provides a computer-readable storage medium (Memory), which is a Memory device in the electronic device 700, for storing programs and data. Such as computer readable storage medium 720. It is understood that the computer readable storage medium 720 herein may include a built-in storage medium in the electronic device 700, and may include an extended storage medium supported by the electronic device 700. The computer-readable storage medium provides storage space that stores an operating system of the electronic device 700. Also stored in this memory space are one or more computer instructions, which may be one or more computer programs 721 (including program code), adapted to be loaded and executed by the processor 710.

According to another aspect of the present application, there is provided a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. Such as a computer program 721. At this time, the electronic device 700 may be a computer, and the processor 710 reads the computer instructions from the computer-readable storage medium 720, and the processor 710 executes the computer instructions so that the computer performs the video processing methods provided in the above-mentioned various alternatives.

In other words, when implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, runs the processes of, or implements the functions of, embodiments of the present application, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, from one website, computer, server, or data center by wired (e.g., coaxial cable, fiber optic, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means.

Those of ordinary skill in the art will appreciate that the elements and process steps of the examples described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or as a combination of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

It should be noted that the above description is only a specific embodiment of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily think about the changes or substitutions within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method for training a personalized semantic vector model in a search engine, comprising:

the similarity difference value is a difference value between a similarity score between a positive example vector and a query vector and a similarity score between a negative example vector and the query vector, wherein the positive example vector is a vector converted from document features forming a positive example with the query features, and the negative example vector is a vector converted from document features forming a negative example with the query features;

Wherein the first query feature and the M document features each include a text feature and a non-text feature; the obtaining the first query feature and the M document features includes:

acquiring X non-text features in training data, wherein X is more than 0;

selecting N times from the X non-text features in a random selection mode to obtain N groups of non-text features; each set of non-text features in the N sets of non-text features includes Y non-text features, N > 1, X > Y > 0;

the N groups of non-text features are used as N training sets of the personalized semantic vector model, and Y non-text features included in one group of non-text features in the N groups of non-text features are used as non-text features in the first query feature and document non-text features in the M document features in one training set in the N training sets.

2. The method of claim 1, wherein the personalized semantic vector model comprises a query-side encoder, a positive-side encoder, a negative-side encoder, and a scoring module;

the converting the first query feature into a query vector and converting the M document features into M document vectors respectively includes:

Converting the first query feature into the first query vector using the query side encoder; converting the document features forming the positive examples with the first query features in the M document features into first positive example vectors in the M document vectors by using the positive example side encoder; converting the document features forming negative examples with the query features in the M document features into first negative example vectors in the M document vectors by using the negative example side encoder;

the training a personalized semantic vector model by taking a preset similarity difference value as a training target based on the query vector and the M document vectors comprises the following steps:

calculating a first difference value by using the scoring module; the first difference value is a difference value between a similarity score between the first positive example vector and the first query vector and a similarity score between the first negative example vector and the first query vector;

based on the first difference value, the similarity difference value is used as a training target, and the personalized semantic vector model is trained.

3. The method of claim 2, wherein the negative side encoder comprises a negative side text conversion module, a negative side non-text conversion module, and a negative side vector fusion module; the negative side encoder and the positive side encoder share parameters.

4. The method of claim 2, wherein the query-side encoder and the positive-side encoder do not share parameters, and wherein the query-side encoder and the negative-side encoder do not share parameters.

5. The method of claim 2, wherein the query-side encoder comprises a query-side text conversion module, a query-side non-text conversion module, and a query-side vector fusion module; the first query feature includes a first text feature and a first non-text feature;

wherein the converting the first query feature into a first query vector comprises:

converting the first text feature into a first text vector by using the query-side text conversion module; converting the first non-text feature into a first non-text vector by using the query side non-text conversion module; and fusing the first text vector and the first non-text vector by using the query side vector fusion module to obtain the first query vector.

6. The method of claim 5, wherein the first non-text feature comprises at least one of the following features of the user: age, gender, portrait, academic, language used and mobile phone system used.

7. The method of claim 2, wherein the case side encoder comprises a case side text conversion module, a case side non-text conversion module, and a case side vector fusion module; the document features forming a positive example with the first query feature in the M document features comprise a second text feature and a second non-text feature;

the converting the M document features into M document vectors respectively includes:

converting the second text feature into a second text vector by using the positive example side text conversion module; converting the second non-text feature into a second non-text vector by using the positive side non-text conversion module; and fusing the second text vector and the second non-text vector by using the positive side vector fusion module to obtain the first positive vector.

8. The method of claim 7, wherein the second non-textual feature comprises at least one of: authority of document content, authority of document author and number of pieces of public numbers associated with the document.

9. The method according to claim 1, wherein the method further comprises:

based on the second query vector, K document features with similarity scores higher than the similarity score of the second query vector are selected in a document library, wherein K is more than 0;

10. A training apparatus for a personalized semantic vector model in a search engine, comprising:

Wherein the first query feature and the M document features each include a text feature and a non-text feature; the acquisition unit is specifically configured to:

acquiring X non-text features in training data, wherein X is more than 0;

11. An electronic device, comprising:

a processor adapted to execute a computer program;

a computer readable storage medium having stored therein a computer program which, when executed by the processor, implements a method of training a personalized semantic vector model in a search engine according to any one of claims 1 to 9.

12. A computer readable storage medium storing a computer program for causing a computer to perform the training method of the personalized semantic vector model in a search engine according to any one of claims 1 to 9.