CN117271851A

CN117271851A - Vertical type searching method and device, searching system and storage medium

Info

Publication number: CN117271851A
Application number: CN202311569373.6A
Authority: CN
Inventors: 吴雪松
Original assignee: Beijing Xiaomi Mobile Software Co Ltd
Current assignee: Beijing Xiaomi Mobile Software Co Ltd
Priority date: 2023-11-22
Filing date: 2023-11-22
Publication date: 2023-12-22

Abstract

The method comprises the steps of carrying out text rewriting on an obtained user query phrase based on a preset limit text to obtain a dialogue query text, inputting the dialogue query text into a dialogue large model which is obtained through training in advance to obtain an output response result, matching the response result with a plurality of entities in a database to obtain a target entity, and determining a corresponding search result according to the target entity. In the embodiment of the disclosure, the data search in the vertical field is realized by using the large dialogue model, so that the semantic understanding capability and semantic characterization capability of the search engine are improved, the user intention is better understood, and especially the search for the general intention has better performance, the search effect is improved, and the generalization capability of the search engine is stronger, so that the cross-scene deployment in different vertical fields is realized conveniently.

Description

Vertical type searching method and device, searching system and storage medium

Technical Field

The disclosure relates to the technical field of data searching, in particular to a vertical type searching method and device, a searching system and a storage medium.

Background

The vertical search refers to searching information in a specific vertical field, and the capability of the vertical search is generally improved by combining data characteristics in the vertical field, for example, searching in fields such as commodities, music, videos, games and the like belongs to the vertical search.

Nowadays, with the development of Machine Learning (ML) technology, a search system in the related art more utilizes a deep Learning model such as Transformer, bert to realize vertical type search, but such a search system has poor generalization ability and poor search semantic understanding ability aiming at the general intention of a user, resulting in poor search effect.

Disclosure of Invention

In order to improve generalization capability and semantic understanding capability of a search system and further improve information search effects, the embodiment of the disclosure provides a vertical type search method and device, a search system and a storage medium.

In a first aspect, an embodiment of the present disclosure provides a method for searching for a vertical class, including:

acquiring a user query phrase, and carrying out text rewriting on the user query phrase based on a preset limit text to obtain a dialogue query text;

inputting the dialogue query text into a dialogue large model which is trained in advance, and obtaining a response result output by the dialogue large model;

matching a plurality of entities in a database based on the response result to obtain a target entity;

and determining search results corresponding to the user query phrase according to the target entity.

In some embodiments, the matching the response result in the plurality of entities in the database to obtain the target entity includes:

And carrying out text score matching with each entity included in the database based on the response result, and determining the entity with the highest text score as the target entity.

In some implementations, the determining, from the target entity, search results corresponding to the user query phrase includes:

determining a search log corresponding to the target entity in a history period based on the target entity, wherein the search log is used for recording an entity result actually clicked by a user under the condition that a user inquiry phrase is the target entity;

counting the clicking times of each entity result actually clicked by the user under the condition that the user inquiry phrase is the target entity based on the search log;

and sequencing all the entity results according to the clicking times from high to low, and determining the entity results with the preset quantity as the search results.

In some implementations, after the determining, according to the target entity, search results corresponding to the user query phrase, the method further includes:

and generating a result page according to the search result, and outputting and displaying the result page.

In some embodiments, the obtaining the user query phrase and performing text rewrite on the user query phrase based on a preset limit text to obtain a dialogue query text includes:

Performing text detection on the obtained user query phrase;

and responding to detection of a preset keyword from the user query phrase, and carrying out text rewriting on the user query phrase based on the preset limit text to obtain the dialogue query text.

In some embodiments, the process of pre-training the large dialog model includes:

acquiring a training data set, wherein the training data set comprises entity profile data, entity tag data and search click data pairs of a preset vertical field;

and generating dialogue sample data according to the training data set, and carrying out knowledge injection on the pre-trained dialogue large model by utilizing the dialogue sample data to obtain a trained dialogue large model.

In some implementations, the obtaining the user query phrase includes:

acquiring user input text information, and determining the user inquiry phrase according to the user input text information; or,

and acquiring user voice information, and carrying out text recognition on the user voice information to acquire the user query phrase.

In a second aspect, embodiments of the present disclosure provide a vertical search apparatus, including:

the text rewriting module is configured to acquire a user query phrase, and rewrite the text of the user query phrase based on a preset limit text to obtain a dialogue query text;

The dialogue model module is configured to input the dialogue query text into a dialogue large model which is trained in advance, and obtain a response result output by the dialogue large model;

the entity matching module is configured to obtain a target entity by matching among a plurality of entities in a database based on the response result;

and the result determining module is configured to determine search results corresponding to the user query phrase according to the target entity.

In some embodiments, the entity matching module is configured to:

In some embodiments, the result determination module is configured to:

In some embodiments, the vertical search device of the present disclosure further includes a result display module configured to:

In some embodiments, the text rewrite module is configured to:

performing text detection on the obtained user query phrase;

In some embodiments, the dialog model module is configured to:

In some embodiments, the text rewrite module is configured to:

In a third aspect, embodiments of the present disclosure provide a search system, including:

a processor; and

a memory storing computer instructions for causing a processor to perform the method according to any implementation of the first aspect.

In a fourth aspect, an embodiment of the present disclosure provides a storage medium storing computer instructions for causing a computer to perform the method according to any embodiment of the first aspect.

The vertical type searching method of the embodiment of the disclosure comprises the steps of carrying out text rewriting on an obtained user query phrase based on a preset limit text to obtain a dialogue query text, inputting the dialogue query text into a dialogue large model obtained through pre-training to obtain an output response result, matching the response result in a plurality of entities in a database to obtain a target entity, and determining a corresponding searching result according to the target entity. In the embodiment of the disclosure, the data search in the vertical field is realized by using the large dialogue model, so that the semantic understanding capability and semantic characterization capability of the search engine are improved, the user intention is better understood, and especially the search for the general intention has better performance, the search effect is improved, and the generalization capability of the search engine is stronger, so that the cross-scene deployment in different vertical fields is realized conveniently.

Drawings

In order to more clearly illustrate the embodiments of the present disclosure or the prior art, the drawings that are required in the detailed description or the prior art will be briefly described, it will be apparent that the drawings in the following description are some embodiments of the present disclosure, and other drawings may be obtained according to the drawings without inventive effort for a person of ordinary skill in the art.

Fig. 1 is a block diagram of a search system in the related art.

Fig. 2 is an architecture diagram of a search system in accordance with some embodiments of the present disclosure.

Fig. 3 is a flow chart of a method of vertical searching in accordance with some embodiments of the present disclosure.

Fig. 4 is a flow chart of a method of vertical searching in accordance with some embodiments of the present disclosure.

Fig. 5 is a flow chart of a method of vertical searching in accordance with some embodiments of the present disclosure.

Fig. 6 is a flow chart of a method of vertical searching in accordance with some embodiments of the present disclosure.

Fig. 7 is a block diagram of a vertical search device in accordance with some embodiments of the present disclosure.

Fig. 8 is a block diagram of a search system in accordance with some embodiments of the present disclosure.

Detailed Description

The following description of the embodiments of the present disclosure will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the described embodiments are some, but not all, of the embodiments of the present disclosure. All other embodiments, which can be made by one of ordinary skill in the art without inventive effort, based on the embodiments in this disclosure are intended to be within the scope of this disclosure. In addition, technical features related to different embodiments of the present disclosure described below may be combined with each other as long as they do not make a conflict with each other.

Search engines can be classified into broad searches and vertical searches according to the search content. The general search refers to searching general information, for example, a common search website belongs to the general search. The vertical search refers to searching information in a specific vertical field, and the capability of the vertical search is generally improved by combining data characteristics in the vertical field, for example, searching in fields such as commodities, music, videos, games and the like belongs to the vertical search.

In the field of data search, it is precisely understood that the user search intention is fundamental to improving the recall accuracy of results, and with the development of Machine Learning (ML), more and more NLP (Natural Language Processing ) models, such as DSSM (Deep Structured Semantic Model, deep-structure speech model), deepFM, transformer, bert, etc., are applied to the search field.

The software architecture of a conventional vertical-class search system is generally shown in fig. 1, and includes a semantic understanding module, a result recall module, a result ordering module, and a reordering module.

The semantic understanding module mainly performs intention recognition on a user query phrase (query) by a user and analyzes what meaning the text input by the user is. The recall module recalls the search results according to the query phrase input by the user. The result ordering module is mainly used for ordering the recall results according to a certain ordering rule to obtain search results. The reordering module reorders the search results mainly according to a certain manual intervention and presents the final search results to the user.

In the related art, each module of the search system is a deep neural network model, and the search system aiming at the vertical field can be obtained by modeling and jointly training each module by utilizing mass data of the vertical field.

However, in the related technical scheme, the traditional machine learning model is generally used for modeling and learning aiming at a single text library, so that the model generalization capability is poor, and the cross-scene adaptation cannot be realized. Moreover, semantic understanding is poor, especially for searches intended by the user, which cannot be well understood by the search engine, resulting in recalled search results that are not intended by the user.

For example, in the field of mobile game search, when a user wants to download a mobile game, the user can generally search for a game that he or she wants to download in an "app store". Through statistics of user input texts (query) of application store search games, the length of the user input texts is generally varied from 1 to 28, namely, long tail distribution exists in the user input texts, and not all the input texts can accurately point to a certain game.

For example, when the user wants to download a game of "prince glory", the user may input "prince" or "prince glory" in the search field of the application store and click on confirm, and then select the corresponding game for download in the search results page.

But when the user needs to perform a general intent search, the user-entered text more closely approximates a conversational search. For example, the user wants to download a game of "principals glory", but does not know the name of the game, but only knows a certain character name in the game, so that the user input text may be "what game has Angela" or "network game with one character called Angela".

For another example, taking the field of film or music search as an example, the user may not have an explicit search intent, but may simply wish to search for general intent based on a tag or current preference, e.g., the user input text may be "hanging movies in love for men" or "comedy movies with highest scores for this year".

At this time, it is difficult to understand the intention of the user based on the conventional search engine, resulting in a large deviation between the game finally recalled and the game that the user actually wants to download, and the search effect is poor.

Based on the defects of the related art, the embodiment of the disclosure provides a vertical type search method, a vertical type search device, a vertical type search system and a storage medium, which aim to realize search in the vertical field by using a large dialogue model as a base model, improve the generalization capability of the search system, only need to perform fine adjustment on the basis of the large dialogue model to be suitable for search tasks in various vertical fields, improve the semantic understanding capability of the search system and improve the search precision.

It is understood that the large model refers to the abbreviation of the large-scale language model (Large Language Model), and features of the large model are expressed in two aspects: firstly, the model is large, namely the parameter quantity of the model is very large, and can reach millions to billions of parameter quantity, and even large models like ChatGPT have billions of parameters; the other is "pre-training", i.e. the model has been pre-trained on a large data set, with only small amounts of data being combined for fine tuning to support applications in various scenarios. Currently, each large manufacturer is deployed with a large model, such as the aforementioned ChatGPT, and further such as ChatGLM, relics, antique, starfire, etc.

In the embodiment of the disclosure, the large dialogue model is the application of the large dialogue model in a dialogue scene, and can realize tasks such as a question-answering robot, text summarization, content generation, data search and the like. For example, using a vertical class search as an example, FIG. 2 illustrates an architecture diagram of a search system in some embodiments of the present disclosure.

As shown in fig. 2, in some embodiments, a search system of examples of the present disclosure may include a client 100 and a server 200, the client 100 and the server 200 establishing a communicable connection through a wireless network.

The client 100 refers to a user device, and the client 100 may be a device type such as a smart phone, a tablet computer, a smart watch, a notebook computer, and the like. Taking a search scenario as an example, a user may input a user query phrase (query) through the client 100 by handwriting, keyboard input, voice input, etc., where the user query phrase (query) refers to search information input by the user.

The server 200 refers to a server, and the server 200 may be, for example, a single server, a server cluster, a cloud server, or the like. In some embodiments, the conversation large model of the disclosed examples may be deployed at the server 200.

Taking a search scenario as an example, when a user wants to search, the user may input a user query phrase at the client 100 and click to determine, the client 100 sends the user query phrase to the server 200, and the server 200 may obtain a final search result by executing a vertical search method described in the disclosure below and return the final search result to the client 100, so that the client 100 presents the search result to the user.

Of course, it should be noted that, under the condition of allowing the hardware performance of the client 100, the session big model described in the present disclosure may be deployed on the client 100, so that in the search scenario, the client 100 may only execute the vertical type search method described in the present disclosure, so that the local search may be implemented, without depending on the server 200, which will be understood by those skilled in the art that the disclosure is not repeated.

Based on the search system illustrated in fig. 2, a description is given below of a vertical class search method according to an embodiment of the present disclosure.

First, it should be noted that the principle of the search system according to the embodiment of the present disclosure is different from that of the conventional search system shown in fig. 1, and in the embodiment of the present disclosure, a large dialogue model is used instead of each neural network model in the conventional scheme. The dialogue large model may use a general large language model as a base model, for example, the base model may be ChatGPT, ghatGLM, the religious, the antique, the starfire, etc., which is not limited in this disclosure.

In the embodiment of the disclosure, for a vertical field scene, a base model can be trained and finely adjusted by combining a small amount of data of the vertical field to obtain a large dialogue model applied to the vertical field. After the large dialogue model is deployed online, the large dialogue model can be combined with certain post-processing logic to realize the vertical class search task.

As can be seen from the foregoing, in the embodiments of the present disclosure, a large dialogue model applied to the vertical domain needs to be trained in advance, and then the large dialogue model is used to implement the search task of the vertical domain, which will be described in the following embodiments of the present disclosure for convenience of understanding and description.

1) Model training

In the embodiment of the present disclosure, a general large language model may be selected as the base model, and for example, the base model may be ChatGPT, ghatGLM, the religious, the antique, the starfire, etc., which is not limited in this disclosure.

For example, in some embodiments, the base model may employ ChatGLM, chatGLM full scale Chat General Language Model, which is an open source and supports Chinese-English bilingual conversational language models. The ChatGLM uses a technology similar to the ChatGPT, optimizes Chinese questions and answers and dialogues, and can generate human preference answers which are quite in line with Chinese environments through the aid of the technology of supervision fine tuning, feedback self-help, human feedback reinforcement learning and the like by the aid of Chinese-English bilingual training of about 1T identifiers.

In addition, chatGLM is smaller than ChatGPT, taking ChatGLM-6B as an example, which has only 62 billion parameters, combined with model quantization technology, can be deployed on consumer-level graphics cards, for example, can be deployed on mobile terminals. Thus, in some embodiments of the present disclosure, the base model may employ ChatGLM-6B.

According to the foregoing, the base model is a model that has been pre-trained on a large-scale data set, which is equivalent to the model having learned a large amount of general basic knowledge, and when applied to the vertical field, the base model needs to be fine-tuned by using the data of the vertical field, so as to obtain a large dialogue model suitable for tasks in the vertical field.

For ease of understanding and explanation, in one example, the drop-type field will be exemplified by a cell phone game search, and the training process of the conversation large model described in this disclosure will be explained with reference to fig. 3.

As shown in fig. 3, in some embodiments, the vertical class search method of the present disclosure includes a process of training to obtain a large dialogue model, including:

s310, acquiring a training data set.

S320, generating dialogue sample data according to the training data set, and carrying out knowledge injection on the pre-trained dialogue large model by utilizing the dialogue sample data to obtain a trained dialogue large model.

In embodiments of the present disclosure, the training data set includes training data for a plurality of drop fields, which in some embodiments may include a plurality of categories, such as in one example, entity profile data, entity tag data, and search click data pairs.

The entity profile data refers to text profiles corresponding to all entities in the vertical domain, the entity tag data refers to tag information corresponding to all entities in the vertical domain, and the search click data pair refers to a data pair formed by a query phrase input by a user and a finally clicked entity.

For example, taking the field of game search as an example, each game included in the game library is an entity, and the entity profile data represents profile information corresponding to a certain game, for example, in an example, the game entity is taken as "dashing fast dashing", and the corresponding entity profile information can be shown in the following table one:

list one

The entity tag data represents tag information corresponding to a game, for example, in one example, the game entity still takes "dashing fast dashing" as an example, and the corresponding entity profile information can be shown in the following table two:

watch II

The search click data pair represents a data pair formed by a query phrase input by a user and a game entity downloaded by a final click, and the search click data pair can be obtained through statistics of search logs in a historical time period, for example, in one example, the search click data pair can be as shown in the following table three:

watch III

In the third table, the first row indicates that the query phrase input by the user is "owner", and finally the game entity selected for clicking and downloading is "owner glowing", so that the query phrase "owner" and the clicking entity "is" owner glowing "which is a search clicking data pair, and the rest is the same.

In the above examples of the present disclosure, entity profile data for a gaming entity may be obtained directly from profile text provided by each game developer. The entity tag data can be obtained according to tag information provided by each game developer, and can also be obtained by marking each game entity. The search click data pairs may be obtained by counting search logs (query) of the application store, which will not be described in detail in this disclosure.

After the training data set is constructed, the training data set can be used for fine tuning training of the base model.

In some embodiments, the model architecture of the base model may be trimmed first, the trimming algorithm may be a trimming algorithm such as P-tuning, adaLoRa, and in addition, to improve training efficiency, a deep speed framework may be used to accelerate the trimming process. The basic principle of fine tuning the base model is to freeze the network parameters of the underlying dialogue large model by adding additional network layers and training only these newly added network layer parameters. For the foregoing fine tuning algorithm and framework acceleration principle, those skilled in the art will certainly understand and fully implement the same with reference to the related art, and this disclosure will not be repeated here.

After the dialogue large model serving as the base model is finely tuned, the model training in the vertical field can be performed on the dialogue large model by utilizing the training data set, and the model training aims at enabling the universal dialogue large model to learn knowledge in the vertical field, so that the model training process is the process of knowledge injection on the dialogue large model.

In some implementations, dialogue sample data may be generated from a training data set based on the Prompt concept, implementing knowledge injection into a large dialogue model in a manner that simulates a contextual dialogue. The promt refers to an AI Prompt word, and is a method for guiding or exciting a model to complete a specific task by using natural language.

For example, in the game search scenario described above, the training data is exemplified by the entity tag data, and the dialogue sample data generated according to the entity tag data may be as shown in table four:

table four

For entity profile data and search click data, the basic principle of knowledge injection is to simulate situational question and answer, give out samples of questions and reference answers, and let a large dialogue model learn knowledge in the vertical field.

After the fine tuning training of the large dialogue model is realized through the process, the general large dialogue model can be combined with the specific vertical field to obtain the large dialogue model applied to the vertical field, and the model training is completed.

According to the embodiment of the disclosure, the general large model is utilized for fine tuning training, the dialogue large model in the vertical field is obtained, only a small amount of vertical field data is needed in the whole training process, and the training process is simple. When the cross-scene application is needed, the combination of the general base model and the new vertical scene can be realized only by acquiring a small amount of vertical field data of the new scene, and the generalization capability of the search system is stronger.

2) Data searching

After the conversation large model of the vertical field is obtained through the process, the conversation large model can be used for realizing data search of the vertical field.

As shown in fig. 4, in some embodiments, the vertical class search method of the present disclosure includes:

s410, acquiring a user query phrase, and carrying out text rewriting on the user query phrase based on a preset limit text to obtain a dialogue query text.

In the embodiment of the disclosure, the user query phrase is the query text input by the user and is used for reflecting the search intention of the user. For example, taking game search as an example, a user may input search terms in a search box of an application store, where the search terms input by the user are user query phrases query described in the present disclosure.

It should be noted that, the user query phrase may be text information input in the search box by handwriting or a keyboard by the user, or may be text information obtained by performing voice recognition on the voice information of the user, which is not limited in this disclosure.

For example, in one example, the user wants to download the game "peace elite", but the user does not know the name of the game, but only knows that there is a game prop called "AWM" in the game. Thus, the user can input "what has AWM in game" in the search box, and the search system can obtain the user query phrase "what has AWM in game". Or, the user can speak the 'what has AWM in the game' through voice, and the search system can obtain the user query phrase 'what has AWM in the game' through voice recognition and text conversion.

It should be noted that, for the game search scenario, it is important to accurately recall the search results desired by the user, while for the game profile, play, etc., there is no need to pay attention. Therefore, in the embodiment of the disclosure, after the user query phrase is obtained, the user query phrase needs to be rewritten, and the purpose of the rewriting is to make a limited answer to the large dialogue model, so that only the game name needs to be quickly given, and other description is not needed, thereby accelerating the search speed.

For example, in the previous example, the user query phrase is "AWM in game", if the user query phrase is directly input as a query into the large dialogue model, the large dialogue model may output a lot of redundant information, for example, in one example, the output (response) of the large dialogue model is as shown in the following table five:

TABLE five

It can be seen that without a restrictive explanation of the user query phrase, the large dialogue model will output much redundant information, resulting in a slow search.

Therefore, in the embodiment of the present disclosure, after obtaining the user query phrase, the user query text needs to be rewritten based on a preset limit text, where the preset limit text is a preset text that limits the number of the answer games or the number of the answers to the large dialogue model.

For example, in the previous example, after the obtained user query phrase is "what AWM is in game", and the user query phrase is rewritten by limiting the text in advance, the obtained dialogue query text may be as shown in the following table six:

TABLE six

In the above table, a preset limit text "show only 1 game name" is used to limit the number of games for the large dialogue model answer, and "limit 5 words" is used to limit the number of words for the large dialogue model answer.

S420, inputting the dialogue query text into the pre-trained obtained dialogue large model to obtain a response result output by the dialogue large model.

In the embodiment of the disclosure, after the user query phrase is subjected to restrictive rewriting, a dialogue query text is obtained, and then the dialogue query text can be input into the dialogue large model obtained through training, and a response result (response) output by the dialogue large model can be obtained.

For example, in the foregoing example, after the dialogue query text shown in the above table is input into the dialogue large model, the response result is shown in the following table seven:

watch seven

By comparing the fifth table with the seventh table, by performing restrictive rewriting on the user query phrase, a great amount of redundant information can be omitted from the response result (response) output by the large dialogue model, and the output efficiency of the large dialogue model is greatly improved.

S430, matching among the entities in the database based on the response result to obtain a target entity.

In the embodiment of the disclosure, after the response result output by the large dialogue model is obtained, the target entity is required to be obtained by matching in the database according to the response result.

It will be appreciated that the response results output by the session large model are not necessarily the physical results desired by the user, for example, as can be seen from the foregoing table five example, the game that the user desires to download is "peace elite", but there are many games that include prop awm, the response results actually output by the session large model are not necessarily "peace elite", and the response results output by the session large model are not necessarily the game entities included in the game library. Therefore, in the embodiment of the present disclosure, after obtaining the response result output by the dialog big model, further post-processing logic needs to be performed according to the response result, so as to obtain the final search result.

Firstly, after a response result output by a large dialogue model is obtained, each entity in a database needs to be matched according to the response result to obtain a target entity.

For example, in some embodiments, text score matching may be performed sequentially with each entity included in the database according to the response result, to obtain text scores between the response result and each entity. In the embodiment of the present disclosure, the text matching algorithm may use, for example, a Jaro-Winkler Distance algorithm, and those skilled in the art will understand with reference to the related art, and the disclosure will not be repeated.

After obtaining the text score corresponding to each entity in the database, each entity can be ordered according to the text score, and then the entity with the highest text score is selected as the target entity.

For example, taking the game search scenario of the foregoing example as an example, after obtaining the response result "and the flat elite" output by the dialogue large model, performing text score matching on the response result and the name of each game entity in the game library, and then determining the game entity with the highest text score as the target entity, for example, in one example, the response result "and the flat elite" are the highest text score of the "flat elite" game in the game library, so that the target entity can be determined to be "flat elite".

S440, determining search results corresponding to the user query phrase according to the target entity.

In some embodiments of the present disclosure, after obtaining the target entity, the target entity may be the only final search result, i.e., the search result includes only one result of the target entity.

In other embodiments, it is contemplated that the search results page may often require multiple results to be presented to the user, and thus after the target entity is obtained, multiple entity results may be further determined according to the target entity, and the multiple entity results may be ranked to obtain the final search result.

For example, in one exemplary embodiment, the number of clicks of the entity results actually clicked by the user in the case of searching for the target entity may be counted according to the search log of the history period, and sorted according to the number of clicks from high to low, and the Top-k entity results ranked earlier are truncated, and the Top-k entity results are determined as the final search result. This is illustrated in the embodiments of the present disclosure below and will not be described in detail here.

It can be appreciated that in the embodiment of the disclosure, the data search in the vertical field is implemented by using the large dialogue model, and compared with the traditional search engine, the method of the disclosure has better semantic understanding capability, and particularly for some dialogue type query phrases, better search results can be obtained.

For example, taking the foregoing user query phrase "what game has awm" as an example, if a traditional search engine is adopted, the user intention cannot be effectively understood, the search result cannot be close to the user search intention, and the search effect is poor. In the embodiment of the disclosure, the user intention can be well understood, and shooting games can be recommended to the user, so that the search result which is more fit with the user intention can be obtained.

Meanwhile, due to the application of the large dialogue model, the semantic representation capability of the search engine is stronger, and under the vertical type search scene, the user can input more general search words, such as 'suspense drama seen by boy loving', 'language theory and novel seen by girl', and the like, without pointing to specific search entities, and can recommend proper search results for the user well.

In addition, in the embodiment of the disclosure, a plurality of neural network models in a traditional search engine are replaced by a general dialogue large model, so that the model training and cross-scene application efficiency is higher, and the generalization capability of the search engine is stronger.

According to the method and the device, in the embodiment of the disclosure, the data search in the vertical field is realized by using the large dialogue model, the semantic understanding capability and the semantic characterization capability of the search engine are improved, the user intention is better understood, and especially the universal intention search has better performance, the search effect is improved, and the generalization capability of the search engine is stronger, so that the cross-scene deployment in different vertical fields can be realized conveniently.

As shown in fig. 5, in some embodiments, in a vertical search method of an example of the present disclosure, a process of determining a search result corresponding to a user query phrase according to a target entity includes:

S441, determining a search log corresponding to the target entity in the history period based on the target entity.

S442, counting the clicking times of each entity result actually clicked by the user under the condition that the user inquiry phrase is the target entity based on the search log.

S443, sorting the entity results according to the clicking times from high to low, and determining the entity results with the preset quantity as search results.

In some embodiments of the present disclosure, after determining the target entity from the database through the foregoing method processes of S410 to S430, the final search result may be determined according to the target entity.

Still taking game search as an example, when a user searches and downloads a game at an application store, the search system records the user's query phrase entered by the user and the game entity results actually clicked on by the user, and records them in a search log (querylog).

For example, in one exemplary scenario, after the user a inputs the user query phrase "king" in the search field of the application store, clicks to confirm, and then selects the downloaded game entity result as "king glory" on the search result page, a log may be recorded in the search log as "{ query: the king; result: "Wang glory }.

In another example scenario, after the user B inputs the user query phrase "owner" in the search field of the application store, it clicks to determine that the game entity result selected for downloading in the search result page is "owner shooter", and then the search log may record a log as "{ query: the king; result: "King shooting }.

In other words, the search system may record the user query phrase and the actual click-down game entity results entered by each user at each search in the corresponding search log.

In the embodiment of the disclosure, after determining the target entity, a search log corresponding to the target entity in the history period may be obtained. The history period may be any period of time over 1 day, 7 days, 1 month, etc., as the disclosure is not limited in this regard.

For example, in the foregoing example, the target entity is "peaceful elite", and in one exemplary embodiment, a search log corresponding to "peaceful elite" in the past 1 month may be obtained, where the search log indicates: when the user queries that the phrase is "peaceful elite", the user actually clicks on the downloaded game entity result.

It will be appreciated that since the games that each user actually wants to download when searching for "and" flat elite "are not necessarily the same, a large number of game entity results may be recorded in the search log. For instance, in one example, the game entity results in the search log that were actually selected for download when the user query phrase was "flat elite" include: "Flat elite", "PUBG Mobile", "pass through Firex", "CS Online".

Then, according to the number of clicks recorded in the search log, the number of times that the user selects to download each entity result when searching for "flat elite" is counted, for example, in one example, when searching for "flat elite" by the user, the number of times that the game entity result "flat elite" is clicked and downloaded is 10 ten thousand times, the number of times that the game entity result "PUBG Mobile" is clicked and downloaded is 1.8 ten thousand times, the number of times that the game entity result "pass through live wire" is clicked and downloaded is 2000 times, and the number of times that the game entity result "CS Online" is clicked and downloaded is 150 times.

And then, sorting according to the clicking times from high to low, and selecting the game entity results with the preset quantity, which are ranked at the front, as the final search results according to the actual search result quantity. For example, in one example, a total of 2000 entity results are included in the search log for the target entity, by counting the number of clicks of the 2000 entity results, and sorting the results according to the number of clicks from high to low, then selecting top-k entity results as the final search result, i.e., selecting k entity results with the top order as the final search result. The value of k may be selected according to specific scene requirements, for example, k=100, 50, 20, 10, etc., which is not limited in this disclosure.

In some embodiments of the present disclosure, after determining the search results, a results page corresponding to the search results may be presented as a user output. For example, taking the scenario shown in fig. 1 as an example, a user inputs a user query phrase at the client 100, the client 100 sends the user query phrase to the server 200, the server 200 obtains a final search result by executing the foregoing method steps, then generates a corresponding result page according to the search result, and then sends the result page to the client 100, so that the client 100 renders and displays the corresponding result page on a display screen, and the user can see the corresponding search result.

As shown in fig. 6, in some embodiments, in the vertical class searching method of the examples of the disclosure, a process of text rewriting a user query phrase to obtain a dialogue query text includes:

S610, performing text detection on the obtained user query phrase.

S620, responding to detection of preset keywords from the user query phrase, and carrying out text rewriting on the user query phrase based on the preset limit text to obtain a dialogue query text.

It should be noted that, in some embodiments, the search system may simultaneously retain the conventional search scheme shown in fig. 1 and the search method described in the present disclosure, so that the search method described in the present disclosure is performed in a case where the call of the large dialogue model needs to be triggered, and the search may be completed according to the conventional scheme in a case where the call of the large dialogue model does not need to be triggered.

For example, in some embodiments, when a user needs to invoke the dialogue large model to execute the search method according to the embodiments of the present disclosure, a preset keyword may be carried in an input user query phrase, where the preset keyword may be any preset keyword.

For example, in one example, the session big model obtained by training in the foregoing disclosure is named as GameChat, so that the preset keyword may be defined as GameChat. When the user desires to invoke the dialogue large model to perform a search task, the GameChat may be carried in front of the input query phrase, e.g., in the previous example, the user input query phrase may be "GameChat has AWM in what game".

After the user query phrase is obtained, text detection may be performed on the user query phrase. If a preset keyword is detected in the user query phrase, for example, in the previous example, the preset keyword "GameChat" is detected in the user query phrase "GameChat what is in AWM", it indicates that the session big model needs to be called to execute the search method described in the disclosure. Otherwise, if the preset keyword is not detected in the query phrase of the user, the description does not need to call the large dialogue model, and only the search task is completed according to the conventional search scheme shown in fig. 1, which is not repeated in the disclosure.

As can be seen from the foregoing, in the embodiments of the present disclosure, the search system has two search modes at the same time, and when a user performs a search with explicit intention, the user can use a conventional search mode without carrying a preset keyword, thereby improving the search efficiency. When the user needs to search for the general intention, the user can carry preset keywords, namely, the large dialogue model can be called according to the method process to realize the search, the semantic understanding capability is improved, and further more accurate search results are obtained.

In some embodiments, the present disclosure provides a vertical search apparatus, as shown in fig. 7, which includes:

The text rewriting module 10 is configured to acquire a user query phrase, and perform text rewriting on the user query phrase based on a preset limit text to obtain a dialogue query text;

a dialogue model module 20, configured to input the dialogue query text into a dialogue large model trained in advance, and obtain a response result output by the dialogue large model;

an entity matching module 30 configured to obtain a target entity by matching among a plurality of entities in a database based on the response result;

a result determination module 40 configured to determine search results corresponding to the user query phrase based on the target entity.

In some embodiments, the entity matching module 30 is configured to:

In some embodiments, the result determination module 40 is configured to:

In some embodiments, the text rewrite module 10 is configured to:

performing text detection on the obtained user query phrase;

In some embodiments, the dialog model module 20 is configured to:

In some embodiments, the text rewrite module 10 is configured to:

In some embodiments, the present disclosure provides a search system comprising:

a processor; and

a memory storing computer instructions for causing a processor to perform the method of any of the preceding embodiments.

As shown in connection with fig. 2, the search system of the example of the present disclosure includes a client 100 and a server 200, and the processor and the memory may be the client 100 or the server 200, which is not limited in this disclosure.

In some embodiments, the present disclosure provides a storage medium storing computer instructions for causing a computer to perform the method of any of the above embodiments.

Specifically, fig. 8 shows a schematic structural diagram of a search system 600 suitable for implementing the method of the present disclosure, and by means of the system shown in fig. 8, the corresponding functions of the processor and the storage medium described above may be implemented.

As shown in fig. 8, the search system 600 includes a processor 601 that can perform various appropriate actions and processes according to a program stored in a memory 602 or a program loaded into the memory 602 from a storage portion 608. In the memory 602, various programs and data required for the operation of the search system 600 are also stored. The processor 601 and the memory 602 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.

The following components are connected to the I/O interface 605: an input portion 606 including a keyboard, mouse, etc.; an output portion 607 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, a speaker, and the like; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The drive 610 is also connected to the I/O interface 605 as needed. Removable media 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is installed as needed on drive 610 so that a computer program read therefrom is installed as needed into storage section 608.

In particular, according to embodiments of the present disclosure, the above method processes may be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program comprising program code for performing the method described above. In such an embodiment, the computer program can be downloaded and installed from a network through the communication portion 609, and/or installed from the removable medium 611.

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

It should be apparent that the above embodiments are merely examples for clarity of illustration and are not limiting of the embodiments. Other variations or modifications of the above teachings will be apparent to those of ordinary skill in the art. It is not necessary here nor is it exhaustive of all embodiments. While still being apparent from variations or modifications that may be made by those skilled in the art are within the scope of the present disclosure.

Claims

1. A method for searching for sags, comprising:

2. The method of claim 1, wherein the matching the target entity among the plurality of entities in the database based on the response result comprises:

3. The method of claim 1, wherein the determining search results corresponding to the user query phrase from the target entity comprises:

4. The method of claim 1, further comprising, after said determining search results corresponding to said user query phrase from said target entity:

5. The method of claim 1, wherein the obtaining the user query phrase and text-rewriting the user query phrase based on the preset constraint text to obtain the dialogue query text comprises:

performing text detection on the obtained user query phrase;

6. The method of claim 1, wherein pre-training the session large model comprises:

7. The method of claim 1, wherein the obtaining a user query phrase comprises:

8. A vertical search device, comprising:

9. A search system, comprising:

a processor; and

memory storing computer instructions for causing a processor to perform the method according to any one of claims 1 to 7.

10. A storage medium having stored thereon computer instructions for causing a computer to perform the method according to any one of claims 1 to 7.