CN110046308B

CN110046308B - Sequencing strategy determination method and device and electronic equipment

Info

Publication number: CN110046308B
Application number: CN201910177973.5A
Authority: CN
Inventors: 张元博; 孙键; 陈炜鹏; 许静芳
Original assignee: Beijing Sogou Technology Development Co Ltd
Current assignee: Beijing Sogou Technology Development Co Ltd
Priority date: 2019-03-07
Filing date: 2019-03-07
Publication date: 2021-05-25
Anticipated expiration: 2039-03-07
Also published as: CN110046308A

Abstract

The embodiment of the invention provides a method, a device and electronic equipment for determining a sorting strategy, wherein the method comprises the steps of obtaining mixed insertion sorting information, sorting and determining search results according to the mixed insertion sorting information and first sorting information and second sorting information, sorting and determining the search results according to the first sorting strategy by the first sorting information, and sorting and determining the search results according to the second sorting strategy by the second sorting information; according to the mixed insertion sorting information, determining first K pieces of search results which are positioned in the first sorting information and are positioned before the second sorting information, and determining second K pieces of search results which are positioned in the second sorting information and are positioned before the first sorting information, wherein K is a positive integer; respectively acquiring user behavior characteristic information corresponding to K first search results and user behavior characteristic information corresponding to K second search results, and determining an optimal ordering strategy; and further improve the accuracy of the search.

Description

Sequencing strategy determination method and device and electronic equipment

Technical Field

The present invention relates to the field of data processing technologies, and in particular, to a method and an apparatus for determining a ranking policy, and an electronic device.

Background

With the development of internet technology and search engine technology, more and more users search information using search platforms. Specifically, a user can input search words in a search platform and execute search operation, the search platform can call a search engine to search, determine search results and sort the search results according to a sorting strategy, then the sorted search results are displayed, and the user can click the search results meeting requirements to enter a corresponding webpage to browse.

In order to enable the sorted search results to better meet the requirements of users, the sorting strategy of the search engine is optimized, and the optimization strategy is determined; after the optimization strategy is obtained, it is necessary to determine which of the current strategy and the optimization strategy is better to determine whether the search engine continues to adopt the current strategy or adopt the optimization strategy, so as to improve the accuracy of the search.

Disclosure of Invention

The embodiment of the invention provides a sequencing strategy determination method to improve the accuracy of searching.

Correspondingly, the embodiment of the invention also provides a sequencing strategy determination device and electronic equipment, which are used for ensuring the realization and application of the method.

In order to solve the above problem, an embodiment of the present invention discloses a method for determining a ranking policy, which specifically includes: acquiring mixed insertion sorting information, wherein the mixed insertion sorting information sorts and determines the search results according to first sorting information and second sorting information, the first sorting information sorts and determines the search results according to a first sorting strategy, and the second sorting information sorts and determines the search results according to a second sorting strategy; according to the mixed insertion sorting information, determining first K pieces of search results which are positioned in the first sorting information and are positioned at the front of the second sorting information, and determining second K pieces of search results which are positioned in the second sorting information and are positioned at the front of the first sorting information, wherein K is a positive integer; and respectively acquiring user behavior characteristic information corresponding to K first search results and user behavior characteristic information corresponding to K second search results, and determining an optimal sorting strategy, wherein the user behavior characteristic information is specific to the search results sorted according to the mixed insertion sorting information.

Optionally, the mixed insertion ranking information includes location information corresponding to each search result, where the location information includes location information of the search result in the first ranking information and location information of the search result in the second ranking information.

Optionally, the determining, according to the mixed insertion ranking information, the first K first search results that are positioned earlier in the first ranking information than in the second ranking information, and determining the first K second search results that are positioned earlier in the second ranking information than in the first ranking information, includes: determining the first M search results in the first ranking information and the first M search results in the second ranking information according to the position information corresponding to each search result; determining the first T search results in the mixed insertion ordering information according to the first M search results in the first ordering information and the first M search results in the second ordering information; determining the number N1 of search results in the first T pieces of search results, the number N2 of search results being in the first ranking information and in the second ranking information, and the number K is determined according to the N1 and the N2; selecting the first K first search results from the first M search results of the first ranking information, and selecting the first K second search results from the first M search results of the second ranking information; wherein M and T are both positive integers, T is greater than or equal to M, and N1 and N2 are integers greater than or equal to zero.

Optionally, the determining the K of the top T search results, the number N1 of search results positioned earlier in the first ranking information than in the second ranking information, and the number N2 of search results positioned earlier in the second ranking information than in the first ranking information, and the determining the K according to N1 and N2 includes: traversing the front T search results from front to back; counting the number N1 of search results positioned earlier in the first ordering information than in the second ordering information and counting the number N2 of search results positioned earlier in the second ordering information than in the first ordering information each time one search result is traversed; comparing the N1 and N2, and recording the equal value of N1 and N2; after traversing the first T search results, selecting a maximum value from the values when N1 is equal to N2 and determining the maximum value as K.

Optionally, the determining, according to the top M search results in the first ranking information and the top M search results in the second ranking information, the top T search results in the mixed insertion ranking information includes:

and after the duplication of the union set of the first M search results of the first sequencing information and the first M search results of the second sequencing information is removed, the first T search results in the mixed insertion sequencing information are obtained.

Optionally, the obtaining user behavior feature information corresponding to the K first search results and the K second search results, and determining an optimal ranking policy respectively includes: respectively acquiring user behavior characteristic information corresponding to K first search results and user behavior characteristic information corresponding to K second search results according to the search logs; according to the user behavior feature information, respectively determining the relevancy information of the K first search results and the relevancy information of the K second search results; and comparing the relevancy information of the K first search results with the relevancy information of the K second search results, and determining the ranking strategy corresponding to the K search results with larger relevancy information as the optimal ranking strategy.

Optionally, the user behavior feature information includes click information; the determining the relevancy information of the K first search results and the relevancy information of the K second search results according to the user behavior feature information respectively includes: determining relevancy information corresponding to the K first search results according to first click information corresponding to the K first search results; and determining the relevancy information corresponding to the K second search results according to the second click information corresponding to the K second search results.

Optionally, the user behavior feature information includes click information and user satisfaction information of a webpage corresponding to the search result; the determining the relevancy information of the K first search results and the relevancy information of the K second search results according to the user behavior feature information respectively includes: determining relevancy information corresponding to the K first search results according to first click information and user satisfaction information corresponding to the K first search results; and determining the relevancy information corresponding to the K second search results according to the second click information and the user satisfaction information corresponding to the K second search results.

The embodiment of the invention also discloses a device for determining the sequencing strategy, which specifically comprises the following steps: the information acquisition module is used for acquiring mixed insertion sorting information, the mixed insertion sorting information is used for sorting and determining the search results according to first sorting information and second sorting information, the first sorting information is used for sorting and determining the search results according to a first sorting strategy, and the second sorting information is used for sorting and determining the search results according to a second sorting strategy; the information determining module is used for determining K first search results which are positioned in the first sorting information and are positioned before the second sorting information according to the mixed insertion sorting information, and determining K second search results which are positioned in the second sorting information and are positioned before the first sorting information, wherein K is a positive integer; and the strategy determining module is used for respectively obtaining user behavior characteristic information corresponding to the K first search results and user behavior characteristic information corresponding to the K second search results and determining an optimal sorting strategy, wherein the user behavior characteristic information is specific to the search results sorted according to the mixed insertion sorting information.

Optionally, the information determining module includes: the first determining submodule is used for determining the first M search results in the first sequencing information and the first M search results in the second sequencing information according to the position information corresponding to each search result; the second determining submodule is used for determining the first T search results in the mixed insertion ordering information according to the first M search results in the first ordering information and the first M search results in the second ordering information; a third determining sub-module, configured to determine, from the top T search results, the number N1 of search results whose positions in the first ranking information are further forward than those in the second ranking information, and the number N2 of search results whose positions in the second ranking information are further forward than those in the first ranking information, and determine K according to the N1 and N2; the search result determining submodule is used for selecting the first K first search results from the first M search results of the first sequencing information and selecting the first K second search results from the first M search results of the second sequencing information; wherein M and T are both positive integers, T is greater than or equal to M, and N1 and N2 are integers greater than or equal to zero.

Optionally, the third determining submodule is configured to traverse the top T search results from front to back; counting the number N1 of search results positioned earlier in the first ordering information than in the second ordering information and counting the number N2 of search results positioned earlier in the second ordering information than in the first ordering information each time one search result is traversed; comparing the N1 and N2, and recording the equal value of N1 and N2; after traversing the first T search results, selecting a maximum value from the values when N1 is equal to N2 and determining the maximum value as K.

Optionally, the second determining sub-module is configured to remove duplicates of a union set of the first M search results of the first ranking information and the first M search results of the second ranking information, and obtain the first T search results in the mixed insertion ranking information.

Optionally, the policy determination module includes: the characteristic information acquisition submodule is used for respectively acquiring user behavior characteristic information corresponding to K first search results and user behavior characteristic information corresponding to K second search results according to the search logs; the relevancy information determining submodule is used for respectively determining the relevancy information of the K first search results and the relevancy information of the K second search results according to the user behavior feature information; and the optimal strategy determining submodule is used for comparing the relevancy information of the K first search results with the relevancy information of the K second search results and determining the sorting strategy corresponding to the K search results with larger relevancy information as the optimal sorting strategy.

Optionally, the user behavior feature information includes click information; the relevancy information determining submodule is used for determining relevancy information corresponding to the K first search results according to the first click information corresponding to the K first search results; and determining the relevancy information corresponding to the K second search results according to the second click information corresponding to the K second search results.

Optionally, the user behavior feature information includes click information and user satisfaction information of a webpage corresponding to the search result; the relevancy information determining submodule is used for determining relevancy information corresponding to the K first search results according to first click information and user satisfaction information corresponding to the K first search results; and determining the relevancy information corresponding to the K second search results according to the second click information and the user satisfaction information corresponding to the K second search results.

The embodiment of the invention also discloses a readable storage medium, and when the instructions in the storage medium are executed by a processor of the electronic equipment, the electronic equipment can execute the sequencing strategy determination method according to any one of the embodiments of the invention.

An embodiment of the present invention also discloses an electronic device, including a memory, and one or more programs, where the one or more programs are stored in the memory, and configured to be executed by one or more processors, and the one or more programs include instructions for: acquiring mixed insertion sorting information, wherein the mixed insertion sorting information sorts and determines the search results according to first sorting information and second sorting information, the first sorting information sorts and determines the search results according to a first sorting strategy, and the second sorting information sorts and determines the search results according to a second sorting strategy; according to the mixed insertion sorting information, determining first K pieces of search results which are positioned in the first sorting information and are positioned at the front of the second sorting information, and determining second K pieces of search results which are positioned in the second sorting information and are positioned at the front of the first sorting information, wherein K is a positive integer; and respectively acquiring user behavior characteristic information corresponding to K first search results and user behavior characteristic information corresponding to K second search results, and determining an optimal sorting strategy, wherein the user behavior characteristic information is specific to the search results sorted according to the mixed insertion sorting information.

Optionally, the determining, according to the top M search results in the first ranking information and the top M search results in the second ranking information, the top T search results in the mixed insertion ranking information includes: and after the duplication of the union set of the first M search results of the first sequencing information and the first M search results of the second sequencing information is removed, the first T search results in the mixed insertion sequencing information are obtained.

The embodiment of the invention has the following advantages:

in the embodiment of the present invention, mixed insertion ordering information for ordering search results according to first ordering information and second ordering information may be obtained, where the first ordering information is determined by ordering search results according to a first ordering policy, and the second ordering information is determined by ordering search results according to a second ordering policy; then, according to the mixed insertion sorting information, determining the first K pieces of first search results, which are positioned in the first sorting information and are positioned at the front of the second sorting information, and determining the first K pieces of second search results, which are positioned in the second sorting information and are positioned at the front of the first sorting information, so that the search results at the front positions with the same number in the two sorting strategies can be determined; respectively acquiring user behavior characteristic information corresponding to K first search results and K second search results, and determining an optimal ordering strategy; and then, an optimal sequencing strategy is determined according to the user behavior characteristics, and the searching accuracy is improved. In addition, in the embodiment of the invention, the search result for determining the optimal sorting strategy is the search result of the same number of front positions in the two sorting strategies, so that the accuracy of determining the optimal sorting strategy can be improved, and the search accuracy is further improved.

Drawings

FIG. 1 is a flow chart of the steps of an embodiment of a ranking policy determination method of the present invention;

FIG. 2 is a flow chart of the steps of an alternative embodiment of a ranking policy determination method of the present invention;

FIG. 3 is a block diagram of an embodiment of an ordering policy determining apparatus according to the present invention;

FIG. 4 is a block diagram of an alternative embodiment of an ordering policy determining apparatus according to the present invention;

FIG. 5 illustrates a block diagram of an electronic device for ordering policy determinations, in accordance with an exemplary embodiment;

fig. 6 is a schematic structural diagram of an electronic device for ranking policy determination according to another exemplary embodiment of the present invention.

Detailed Description

In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.

Referring to fig. 1, a flowchart illustrating steps of an embodiment of a method for determining a ranking policy of the present invention is shown, which may specifically include the following steps:

102, obtaining mixed insertion sorting information, wherein the mixed insertion sorting information sorts and determines search results according to first sorting information and second sorting information, the first sorting information sorts and determines the search results according to a first sorting strategy, and the second sorting information sorts and determines the search results according to a second sorting strategy.

And 104, determining the first K pieces of first search results positioned in the first ordering information and before the second ordering information according to the mixed insertion ordering information, and determining the first K pieces of second search results positioned in the second ordering information and before the first ordering information, wherein K is a positive integer.

And 106, respectively acquiring user behavior characteristic information corresponding to K first search results and user behavior characteristic information corresponding to K second search results, and determining an optimal sorting strategy, wherein the user behavior characteristic information is specific to the search results sorted according to the mixed insertion sorting information.

In the embodiment of the invention, after the optimized sorting strategy for sorting the search engine is determined, the search engine can sort the search results by respectively adopting the optimized sorting strategy and the current sorting strategy after receiving the search instruction and determining the search results according to the preset rule; and then, performing mixed insertion sequencing on the sequencing results obtained by the two sequencing strategies by adopting an Interleaving method, determining a final sequencing result and returning the final sequencing result to the search platform. The search platform displays the corresponding search results according to the final sorting results on the search result page, and records operation information of the user in the process that the user views the search results, such as click operation information, search results corresponding to click operations, click time and the like; therefore, which sort strategy is the optimal strategy is judged according to the operation information of the user. The preset rule can be determined according to requirements, for example, the final sorting result is determined by performing mixed insertion sorting 2 ten thousand times in 100 ten thousand searches, and the final sorting result is determined by performing sorting according to the current sorting strategy in other 98 ten thousand searches.

For convenience of subsequent description, any one of the optimized ranking strategy and the current ranking strategy may be referred to as a first ranking strategy, and the other one may be referred to as a second ranking strategy; and a sorting result obtained by sorting the search results according to the first sorting strategy is called first sorting information, a sorting result obtained by sorting the search results according to the second sorting strategy is called second sorting information, and a sorting result determined by sorting the search results according to the first sorting information and the second sorting information is called mixed insertion sorting information. The first ordering information, the second ordering information and the mixed insertion ordering information all comprise the ordering of the search results, and any two of the first ordering information, the second ordering information and the mixed insertion ordering information may be the same or different; of course, the first sorting information, the second sorting information, and the mixed insertion sorting information may also include other information, such as location information corresponding to the search result.

In the embodiment of the invention, after the search engine performs mixed insertion sequencing each time to determine the final sequencing result, the optimal sequencing strategy corresponding to the search can be determined from two sequencing strategies according to the operation information of the search result obtained by the user aiming at the mixed insertion sequencing; then after determining the optimal sequencing strategy with set times, determining the sequencing strategy with more times determined as the optimal sequencing strategy as a final optimal sequencing strategy; wherein, the set times can be set according to requirements.

In the process of determining the optimal sorting strategy corresponding to each search, mixed insertion sorting information corresponding to the search can be obtained, then, according to the position information of each search result in the mixed insertion sorting information, a search result with a position in the first sorting information being more front than a position in the second sorting information is determined, and a search result with a position in the second sorting information being more front than the position in the first sorting information is determined. After the search results are sorted by using different sorting strategies, the positions of the search results are different, and for distinguishing, the search result with the position earlier than the position in the second sorting information in the first sorting information may be referred to as a first search result, and the search result with the position earlier than the position in the first sorting information in the second sorting information may be referred to as a second search result. Then selecting front K search results from the first search results, and selecting front K search results from the second search results, wherein K is a positive integer; and then the sorting results of the same number of front positions in the two sorting strategies can be determined. Then, determining an optimal ordering strategy according to the analysis of the K first search results and the K second search results; operation information corresponding to the K first search results recorded at this time can be obtained, and the relevance between the K first search results and the user requirements is determined according to the operation information; and acquiring operation information corresponding to the K second search results recorded at this time, and determining the correlation between the K second search results and the user requirements according to the operation information. Then determining the optimal ordering strategy corresponding to the search from the first ordering strategy and the second ordering strategy according to the relevance of the K first search results and the user requirement and the relevance of the K second search results and the user requirement; for example, the ranking policy with high correlation with the user requirement is determined as the optimal policy.

After the final optimal sorting strategy is determined, the search engine can sort the search results by adopting the final optimal strategy in the subsequent search process, and then return the sorted search results to the search platform, so that the search results displayed to the user can better meet the user requirements, and the search accuracy is improved.

In an example of the present invention, a search is taken as an example to illustrate that the optimal ranking strategy corresponding to the search is determined: search results obtained for a certain search term include A, B, C, D, and the first ranking information obtained by ranking the search results by using a first ranking policy is shown in table 1, where table 1 only shows a first ranking order of the search results in the first ranking information, and may of course include other information, which is not limited in this embodiment of the present invention:

first order sequence
	A
B
	C
D

TABLE 1

The table 2 shows that the second ranking information obtained by ranking the search results by using the second ranking policy is shown, and the table 2 only shows the second ranking order of the search results in the second ranking information, and may also include other information, which is not limited in this embodiment of the present invention:

second order of sorting
	B
C
	D
A

TABLE 2

The search engine ranks the search results according to the first ranking information and the second ranking information, and after determining the mixed insertion ranking information, may obtain the mixed insertion ranking information, as shown in table 3, where table 3 only shows the mixed insertion ranking order and corresponding position information of the search results in the mixed insertion ranking information, and may of course include other information, which is not limited in this embodiment of the present invention:

TABLE 3

And then determining the first K pieces of first search results positioned in the first sorting information and the second search results positioned in the second sorting information and the first K pieces of second search results positioned in the first sorting information and the second sorting information according to the mixed insertion sorting information. As can be seen from table 3, for example, the first 1 first search result, such as search result a, positioned earlier in the first ranking information than in the second ranking information may be determined, and the first 1 second search result, such as search result B, positioned earlier in the second ranking information than in the first ranking information may be determined. Acquiring user behavior characteristic information corresponding to K first search results and user behavior characteristic information corresponding to a second search result, and determining an optimal ordering strategy; for example, the user behavior feature information of the search result a and the user behavior feature information of the search result B are obtained, and if the number of clicks of the search result a is 1 and the number of clicks of the search result B is 0, the optimal ranking strategy corresponding to the current search is determined to be the first ranking strategy.

In summary, in the embodiments of the present invention, mixed insertion ordering information that orders search results according to first ordering information and second ordering information may be obtained, where the first ordering information orders search results according to a first ordering policy, and the second ordering information orders search results according to a second ordering policy; then, according to the mixed insertion sorting information, determining the first K pieces of first search results, which are positioned in the first sorting information and are positioned at the front of the second sorting information, and determining the first K pieces of second search results, which are positioned in the second sorting information and are positioned at the front of the first sorting information, so that the search results at the front positions with the same number in the two sorting strategies can be determined; respectively acquiring user behavior characteristic information corresponding to K first search results and K second search results, and determining an optimal ordering strategy; and then, an optimal sequencing strategy is determined according to the user behavior characteristics, and the searching accuracy is improved. In addition, in the embodiment of the invention, the search result for determining the optimal sorting strategy is the search result of the same number of front positions in the two sorting strategies, so that the accuracy of determining the optimal sorting strategy can be improved, and the search accuracy is further improved.

In another embodiment of the present invention, the search results are sorted according to the first sorting information and the second sorting information, and the determining the mixed insertion sorting information may include: and selecting the search result with the top position from the search results except the search results which participate in the sorting to sort according to the first sorting information and the second sorting information in turn. Here, the first sorting information may be prioritized, the first sorting information is shown in table 1, and the second sorting information is shown in table 2. According to the first sorting information, determining that the search result A is positioned at the top in the first sorting information except the search results (including A, B, C, D) which participate in sorting, and selecting the search result A as the first bit of the mixed insertion sorting; then, according to the second ranking information, the search result B with the top position except the search result B which participates in the ranking (comprising B, C, D) in the second ranking information is determined, and the search result B can be selected as the second position of the mixed insertion ranking. According to the first sorting information, determining that the position of a search result C in the first sorting information except the search results (including C, D) participating in sorting is the forefront, and selecting the search result C as a third bit of mixed insertion sorting; then, according to the second sorting information, in addition to the search results (including D) already participating in sorting in the second sorting information, the search result D with the top position is determined, and the search result D can be selected as the fourth bit of the mixed insertion sorting, so that the sorting order in table 3 can be obtained. After each search result is subjected to interpolation sorting, the position of the search result in the first sorting information can be determined, and the position of the search result in the second sorting information can be determined. For example, after the search result a is subjected to mixed insertion sorting, the position of the search result a in the first sorting information is determined to be 1, and the position of the search result a in the second sorting information is determined to be 4; after the search result B is subjected to mixed insertion sorting, the position of the search result B in the first sorting information can be determined to be 2, and the position of the search result B in the second sorting information can be determined to be 1; after the search result C is subjected to mixed insertion sorting, the position of the search result C in the first sorting information is determined to be 3, and the position of the search result C in the second sorting information is determined to be 2; and after the search results D are subjected to mixed insertion sorting, determining that the position of the search results D in the first sorting information is 4, and the position of the search results D in the second sorting information is 3.

In another embodiment of the present invention, for example, by determining an optimal ranking strategy corresponding to one search, how to determine the first K first search results and the first K second search results is described as follows:

referring to fig. 2, a flowchart illustrating steps of an alternative embodiment of the ranking policy determining method according to the present invention is shown, which may specifically include the following steps:

step 202, obtaining mixed insertion sorting information, where the mixed insertion sorting information includes position information of each search result in the first sorting information and position information of the search result in the second sorting information.

In the embodiment of the invention, the mixed insertion sequencing information can be obtained; the mixed insertion ordering information may include an arrangement order of each search result in the mixed insertion ordering, and position information corresponding to each search result; the location information may include location information of the search result in the first ranking information and location information of the search result in the second ranking information. Then, according to the position information corresponding to each search result in the mixed insertion ordering information, the number K of the search results used for determining the optimal ordering strategy is determined, then the first K pieces of first search results, which are positioned in the first ordering information and are positioned before the position in the second ordering information, are determined, and the first K pieces of second search results, which are positioned in the second ordering information and are positioned before the position in the first ordering information, are determined.

In the step 104, determining, according to the interpolation ranking information, the first K first search results whose positions in the first ranking information are earlier than the positions in the second ranking information, and determining the first K second search results whose positions in the second ranking information are earlier than the positions in the first ranking information, may include the following steps: 204-210:

and step 204, determining the first M search results in the first ranking information and the first M search results in the second ranking information according to the position information corresponding to each search result.

And step 206, determining the first T search results in the mixed insertion ordering information according to the first M search results in the first ordering information and the first M search results in the second ordering information.

In the embodiment of the invention, the first M search results of the first ranking information can be determined according to the position information of each search result in the first ranking information in the mixed insertion ranking information; determining the first M search results of the second sorting information according to the position information of each search result in the mixed insertion sorting information in the second sorting information; the M is a positive integer, which may be determined according to requirements, and this is not limited in the embodiment of the present invention. Then, determining the first T search results in the mixed insertion ordering information by combining the first M search results of the first ordering information and the first M search results of the second ordering information; for example, after the union of the first M search results of the first ranking information and the first M search results of the second ranking information is deduplicated, the first T search results in the mixed-insertion ranking information can be obtained. The top T search results in the mixed insertion ranking information may include top M search results in the first ranking information and top M search results in the second ranking information, where T is a positive integer and is greater than or equal to M. For example, M is 3, and as shown in table 1, according to the location information (1, 2, 3, 4) of the search result (A, B, C, D) in the first ranking information, the first 3 search results in the first ranking information are determined to be A, B, C; and determining that the top 3 search results in the second ranking information are B, C, D according to the position information (4, 1, 2, 3) of the search result (A, B, C, D) in the second ranking information as shown in table 2. Further, it may be determined that the part of the mixed and inserted ranking information that includes the first 3 search results of the first ranking information and the first 3 search results of the second ranking information is the first 4 search results, i.e., T is 4, where the first 4 search results of the mixed and inserted ranking information are (A, B, C, D).

And step 208, determining the number N1 of the search results in the first T pieces of search results, wherein the search results are positioned at the position of the first sorting information before the position of the search results in the second sorting information, and the number N2 of the search results in the second sorting information before the position of the search results in the first sorting information, and determining the K according to the N1 and the N2.

In the embodiment of the present invention, the number N1 of search results in the first ranking information that are located earlier than the second ranking information and the number N2 of search results in the second ranking information that are located earlier than the first ranking information in the first ranking information may be determined by traversing the top T search results and comparing the position of each search result in the first ranking information with the position of each search result in the second ranking information; k is then determined from the N1 and N2, where N1 and N2 are integers greater than or equal to zero.

The determination of the number N1 of the top T search results in the above step 208, which is the search result with the position of the first ranking information before the position of the second ranking information, and the number N2 of the search results with the position of the second ranking information before the position of the first ranking information, and the determination of K according to N1 and N2, can be achieved by the following sub-steps 22-28:

and a substep 22, traversing the front T search results from front to back.

Substep 24, counting the number N1 of search results positioned earlier in the first ranking information than in the second ranking information and counting the number N2 of search results positioned earlier in the second ranking information than in the first ranking information each time one search result is traversed.

Substep 26, comparing said N1 and N2, recording the value when said N1 and N2 are equal.

And a substep 28, after traversing the previous T search results, selecting a maximum value from the values when N1 is equal to N2, and determining the maximum value as K.

In the embodiment of the present invention, in the process of traversing the T search results, when a search result is traversed, it may be determined whether the position of the search result in the first ranking information is earlier than the position of the search result in the second ranking information according to the position information of the search result in the first ranking information and the position information of the search result in the second ranking information. If the position of the search result in the first ranking information is earlier, adding 1 to the number N1 of search results which are positioned in the first ranking information and are earlier than the position in the second ranking information; if the position of the search result in the second ranking information is earlier, adding 1 to the number N2 of search results which are positioned in the second ranking information and are earlier than the position in the first ranking information; wherein N1 and N2 are both integers greater than or equal to 0. The N1 and N2 are then compared, and if N1 and N2 are equal, then the value of N1 or N2 at that time can be recorded. After traversing the first T search results, a maximum value may be selected from the values when N1 is equal to N2 and determined to be K.

For example, in table 3, when T is 4, and when the first search result a is traversed, the position of the search result a in the first ranking information is earlier than the position in the second ranking information, N1 may be added by 1, N1 is 1, and N2 is 0. When traversing to the second search result B, the position of the search result B in the second ranking information is earlier than the position in the first ranking information, N2 may be added by 1, so that N1 is equal to 1, N2 is equal to 1, and in this case, N1 is equal to N2, and the value 1 of N1 or N2 may be recorded. When traversing to the third search result C, the position of the search result C in the second ranking information is earlier than the position in the first ranking information, at which time N2 may be added by 1, N1 is 1, and N2 is 2. When traversing to the third search result D, the position of the search result D in the second ranking information is earlier than the position in the first ranking information, at this time, N2 may be added by 1, so that N1 is 1, and N2 is 3; see table 4.

TABLE 4

After traversing the first T search results, in order to determine the ranking results of the same number of front positions from the two ranking strategies, selecting the maximum value from the values when N1 is equal to N2 as K; for example, K ═ 1 in the above example.

Step 210, selecting the first K first search results from the first M search results of the first ranking information, and selecting the first K second search results from the first M search results of the second ranking information.

The step 106 of obtaining user behavior feature information corresponding to the K first search results and the K second search results respectively to determine an optimal ranking strategy includes the following steps: 212-218:

and step 212, respectively acquiring user behavior characteristic information corresponding to the K first search results and user behavior characteristic information corresponding to the K second search results according to the search logs.

Step 214, determining the relevancy information of the K first search results and the relevancy information of the K second search results according to the user behavior feature information.

In the embodiment of the invention, the search log records the operation information of the user, such as the information corresponding to the click operation, so that the operation information of the user can be obtained from the search log, and then the user behavior characteristic information of each first search result in the K first search results and the user behavior characteristic information of each second search result in the K second search results are determined according to the operation information. Then, determining the relevancy information of the K first search results according to the user behavior feature information corresponding to the K first search results, and determining the relevancy information of the K second search results according to the user behavior feature information corresponding to the K second search results; the relevancy information can be used for characterizing the relevancy of the search result and the user requirement, wherein the relevancy information can be in direct proportion to the relevancy of the user requirement.

Wherein the user behavior feature information may include: click information, which may include an identification of whether or not to click, e.g., "1" for clicked, "0" for unchecked; therefore, in an example of the present invention, one way of determining the relevancy information of the K first search results according to the user behavior feature information corresponding to the K first search results may be to determine the relevancy information corresponding to the K first search results according to the first click information corresponding to the K first search results.

In order to distinguish click information corresponding to the first search result from click information corresponding to the second search result, one piece of click information corresponding to the first search result may be referred to as first click information, and the K pieces of first click information corresponding to the first search result may include K pieces of first click information; and the click information corresponding to one piece of second search result is called second click information, and the second click information corresponding to the K pieces of second search result may include K pieces of second click information.

One way of calculating the relevancy information may be to perform weighted calculation on the first click information corresponding to each first search result, and use the result of the weighted calculation as the relevancy information; of course, other ways to calculate the relevancy information may also be adopted, and the embodiment of the present invention is not limited in this respect.

One way of determining the relevancy information of the K second search results according to the user behavior feature information corresponding to the K second search results may be to determine the relevancy information corresponding to the K second search results according to second click information corresponding to the K second search results. Correspondingly, the manner of determining the relevancy information corresponding to the K second search results is similar to the manner of determining the relevancy information corresponding to the K first search results, and is not repeated here.

In the process that a user browses a webpage corresponding to a search result, when the webpage is satisfied, the user is likely to stay in the webpage for a long time, and operations such as page turning, copying, sharing and the like can be performed; when the webpage is not satisfied, the user is likely not to stay on the webpage or stay short, and the user is unlikely to perform certain operations such as page turning, copying, sharing and the like; therefore, the satisfaction information of the user on the webpage corresponding to each search result can be determined according to the operation information of the user in the webpage corresponding to each search result, such as page turning, copying, sharing, staying time and the like, and the satisfaction information is used as one of the user behavior characteristic information.

Therefore, in an example of the present invention, another way of determining the relevancy information of the K first search results according to the user behavior feature information corresponding to the K first search results may be to determine the relevancy information corresponding to the K first search results according to the first click information and the user satisfaction information corresponding to the K first search results. For example, the first click information and the satisfaction information corresponding to each first search result may be weighted to obtain a corresponding weighted result; then, the weighted results corresponding to the K first search results are weighted again to obtain the relevancy information corresponding to the K first search results, wherein the weights of the first click information and the satisfaction information may be set as required.

Correspondingly, in an example of the present invention, another way of determining the relevancy information of the K second search results according to the user behavior feature information corresponding to the K second search results may be to determine the relevancy information corresponding to the K second search results according to second click information and user satisfaction information corresponding to the K second search results.

Step 216, comparing the relevancy information of the K first search results with the relevancy information of the K second search results, and determining the ranking strategy corresponding to the K search results with larger relevancy information as the optimal ranking strategy.

Then, the relevancy information of the K first search results can be compared with the relevancy information of the K second search results to determine a search result with larger relevancy information; if the relevancy information of the K first search results is greater than the relevancy information of the K second search results, determining the first ordering strategy as an optimal ordering strategy; and if the relevancy information of the K second search results is greater than the relevancy information of the K first search results, determining the second sorting strategy as an optimal sorting strategy.

In summary, in the embodiments of the present invention, mixed insertion ordering information that orders search results according to first ordering information and second ordering information may be obtained, where the first ordering information orders search results according to a first ordering policy, and the second ordering information orders search results according to a second ordering policy; then according to the mixed insertion sorting information, determining the first K pieces of first search results, positioned at the front of the second sorting information, in the first sorting information, and determining the first K pieces of second search results, positioned at the front of the first sorting information, in the second sorting information; respectively acquiring user behavior characteristic information corresponding to K first search results and K second search results, and determining an optimal ordering strategy; and then the optimal sorting strategy is selected from the two sorting strategies, so that the searching accuracy is improved. In addition, the embodiment of the invention determines the optimal sorting strategy by analyzing the sorting results from the same number of front positions in the two sorting strategies, thereby improving the accuracy of determining the optimal sorting strategy.

Secondly, in the embodiment of the invention, user behavior characteristic information corresponding to K first search results and user behavior characteristic information corresponding to K second search results can be respectively obtained according to the search logs; according to the user behavior feature information, respectively determining the relevancy information of the K first search results and the relevancy information of the K second search results; furthermore, the relevance information is adopted to represent the relevance between the search result and the user requirement, the quality of the ranking strategy is evaluated, and the accuracy of the ranking strategy evaluation can be improved. Then, the relevancy information of the K first search results and the relevancy information of the K second search results can be compared, and a ranking strategy corresponding to the K search results with larger relevancy information is determined as an optimal ranking strategy; and then, the sorting strategy corresponding to the search result with large relevance information is selected as the optimal sorting strategy, so that the search result sorted by the search engine by adopting the optimal sorting strategy can better meet the user requirement, and the user experience is improved.

It should be noted that, for simplicity of description, the method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the illustrated order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments of the present invention. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no particular act is required to implement the invention.

Referring to fig. 3, a block diagram of a sorting policy determining apparatus according to an embodiment of the present invention is shown, and specifically includes the following modules:

an information obtaining module 302, configured to obtain mixed insertion ranking information, where the mixed insertion ranking information ranks and determines search results according to first ranking information and second ranking information, the first ranking information ranks and determines search results according to a first ranking policy, and the second ranking information ranks and determines search results according to a second ranking policy;

an information determining module 304, configured to determine, according to the mixed insertion sorting information, K first search results that are positioned earlier in the first sorting information than in the second sorting information, and K second search results that are positioned earlier in the second sorting information than in the first sorting information, where K is a positive integer;

and a policy determining module 306, configured to obtain user behavior feature information corresponding to the K first search results and user behavior feature information corresponding to the K second search results, respectively, and determine an optimal ranking policy, where the user behavior feature information is for search results ranked according to the mixed insertion ranking information.

Referring to fig. 4, a block diagram of an alternative embodiment of the ranking policy determination apparatus of the present invention is shown.

In an optional embodiment of the present invention, the mixed insertion ranking information includes location information corresponding to each search result, where the location information includes location information of the search result in the first ranking information and location information of the search result in the second ranking information.

In an optional embodiment of the present invention, the information determining module 304 includes:

a first determining submodule 3042, configured to determine, according to the position information corresponding to each search result, the top M search results in the first ranking information and the top M search results in the second ranking information;

a second determining submodule 3044, configured to determine, according to the top M search results in the first ranking information and the top M search results in the second ranking information, the top T search results in the mixed insertion ranking information;

a third determining sub-module 3046 for determining the number N1 of search results in the top T search results, which are positioned earlier in the first ranking information than in the second ranking information, and the number N2 of search results in the second ranking information than in the first ranking information, and determining the K according to the N1 and N2;

a search result determining sub-module 3048, configured to select the first K first search results from the first M search results of the first ranking information, and select the first K second search results from the first M search results of the second ranking information; wherein M and T are both positive integers, T is greater than or equal to M, and N1 and N2 are integers greater than or equal to zero.

In an optional embodiment of the present invention, the third determining submodule 3046 is configured to traverse the front T search results from front to back; counting the number N1 of search results positioned earlier in the first ordering information than in the second ordering information and counting the number N2 of search results positioned earlier in the second ordering information than in the first ordering information each time one search result is traversed; comparing the N1 and N2, and recording the equal value of N1 and N2; after traversing the first T search results, selecting a maximum value from the values when N1 is equal to N2 and determining the maximum value as K.

In an optional embodiment of the present invention, the second determining submodule 3044 is configured to remove duplicates of a union of the first M search results of the first ordering information and the first M search results of the second ordering information, and obtain the first T search results in the mixed insertion ordering information.

In an optional embodiment of the present invention, the policy determining module 306 comprises:

the feature information obtaining submodule 3062 is configured to obtain, according to the search log, user behavior feature information corresponding to the K first search results and user behavior feature information corresponding to the K second search results, respectively;

the relevancy information determining submodule 3064 is configured to determine relevancy information of the K first search results and relevancy information of the K second search results according to the user behavior feature information;

the optimal policy determining submodule 3066 is configured to compare the relevancy information of the K first search results with the relevancy information of the K second search results, and determine the ranking policy corresponding to the K search results with the greater relevancy information as the optimal ranking policy.

In an optional embodiment of the present invention, the user behavior feature information includes click information;

the relevancy information determining submodule 3064 is configured to determine relevancy information corresponding to the K first search results according to the first click information corresponding to the K first search results; and determining the relevancy information corresponding to the K second search results according to the second click information corresponding to the K second search results.

In an optional embodiment of the present invention, the user behavior feature information includes click information and user satisfaction information of a webpage corresponding to the search result;

the relevancy information determining submodule 3064 is configured to determine relevancy information corresponding to the K first search results according to first click information and user satisfaction information corresponding to the K first search results; and determining the relevancy information corresponding to the K second search results according to the second click information and the user satisfaction information corresponding to the K second search results.

For the device embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, refer to the partial description of the method embodiment.

FIG. 5 is a block diagram illustrating an architecture of an electronic device 500 for ranking policy determination, according to an example embodiment. For example, the electronic device 500 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, an exercise device, a personal digital assistant, and the like.

Referring to fig. 5, electronic device 500 may include one or more of the following components: a processing component 502, a memory 504, a power component 506, a multimedia component 508, an audio component 510, an input/output (I/O) interface 512, a sensor component 514, and a communication component 516.

The processing component 502 generally controls overall operation of the electronic device 500, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing elements 502 may include one or more processors 520 to execute instructions to perform all or a portion of the steps of the methods described above. Further, the processing component 502 can include one or more modules that facilitate interaction between the processing component 502 and other components. For example, the processing component 502 can include a multimedia module to facilitate interaction between the multimedia component 508 and the processing component 502.

The memory 504 is configured to store various types of data to support operation at the device 500. Examples of such data include instructions for any application or method operating on the electronic device 500, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 504 may be implemented by any type or combination of volatile or non-volatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.

The power component 506 provides power to the various components of the electronic device 500. Power components 506 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for electronic device 500.

The multimedia component 508 includes a screen that provides an output interface between the electronic device 500 and a user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 508 includes a front facing camera and/or a rear facing camera. The front camera and/or the rear camera may receive external multimedia data when the electronic device 500 is in an operating mode, such as a shooting mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

The audio component 510 is configured to output and/or input audio signals. For example, the audio component 510 includes a Microphone (MIC) configured to receive external audio signals when the electronic device 500 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may further be stored in the memory 504 or transmitted via the communication component 516. In some embodiments, audio component 510 further includes a speaker for outputting audio signals.

The I/O interface 512 provides an interface between the processing component 502 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.

The sensor assembly 514 includes one or more sensors for providing various aspects of status assessment for the electronic device 500. For example, the sensor assembly 514 may detect an open/closed state of the device 500, the relative positioning of components, such as a display and keypad of the electronic device 500, the sensor assembly 514 may detect a change in the position of the electronic device 500 or a component of the electronic device 500, the presence or absence of user contact with the electronic device 500, orientation or acceleration/deceleration of the electronic device 500, and a change in the temperature of the electronic device 500. The sensor assembly 514 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 514 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 514 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 516 is configured to facilitate wired or wireless communication between the electronic device 500 and other devices. The electronic device 500 may access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication section 514 receives a broadcast signal or broadcast associated information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communications component 514 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the electronic device 500 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components for performing the above-described methods.

In an exemplary embodiment, a non-transitory computer-readable storage medium comprising instructions, such as the memory 504 comprising instructions, executable by the processor 520 of the electronic device 500 to perform the above-described method is also provided. For example, the non-transitory computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

A non-transitory computer readable storage medium having instructions therein which, when executed by a processor of an electronic device, enable the electronic device to perform a ranking policy determination method, the method comprising: acquiring mixed insertion sorting information, wherein the mixed insertion sorting information sorts and determines the search results according to first sorting information and second sorting information, the first sorting information sorts and determines the search results according to a first sorting strategy, and the second sorting information sorts and determines the search results according to a second sorting strategy; according to the mixed insertion sorting information, determining first K pieces of search results which are positioned in the first sorting information and are positioned at the front of the second sorting information, and determining second K pieces of search results which are positioned in the second sorting information and are positioned at the front of the first sorting information, wherein K is a positive integer; and respectively acquiring user behavior characteristic information corresponding to K first search results and user behavior characteristic information corresponding to K second search results, and determining an optimal sorting strategy, wherein the user behavior characteristic information is specific to the search results sorted according to the mixed insertion sorting information.

Fig. 6 is a schematic structural diagram of an electronic device 600 for ranking policy determination according to another exemplary embodiment of the present invention. The electronic device 600 may be a server, which may vary greatly due to different configurations or capabilities, and may include one or more Central Processing Units (CPUs) 622 (e.g., one or more processors) and memory 632, one or more storage media 630 (e.g., one or more mass storage devices) storing applications 642 or data 644. Memory 632 and storage medium 630 may be, among other things, transient or persistent storage. The program stored in the storage medium 630 may include one or more modules (not shown), each of which may include a series of instruction operations for the server. Still further, the central processor 622 may be configured to communicate with the storage medium 630 to execute a series of instruction operations in the storage medium 630 on the server.

The server may also include one or more power supplies 626, one or more wired or wireless network interfaces 650, one or more input-output interfaces 658, one or more keyboards 656, and/or one or more operating systems 641, such as Windows Server, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

An electronic device comprising a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors the one or more programs including instructions for: acquiring mixed insertion sorting information, wherein the mixed insertion sorting information sorts and determines the search results according to first sorting information and second sorting information, the first sorting information sorts and determines the search results according to a first sorting strategy, and the second sorting information sorts and determines the search results according to a second sorting strategy; according to the mixed insertion sorting information, determining first K pieces of search results which are positioned in the first sorting information and are positioned at the front of the second sorting information, and determining second K pieces of search results which are positioned in the second sorting information and are positioned at the front of the first sorting information, wherein K is a positive integer; and respectively acquiring user behavior characteristic information corresponding to K first search results and user behavior characteristic information corresponding to K second search results, and determining an optimal sorting strategy, wherein the user behavior characteristic information is specific to the search results sorted according to the mixed insertion sorting information.

The embodiments in the present specification are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.

Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing terminal to cause a series of operational steps to be performed on the computer or other programmable terminal to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

While preferred embodiments of the present invention have been described, additional variations and modifications of these embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the embodiments of the invention.

Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or terminal that comprises the element.

The foregoing describes in detail a ranking policy determining method, a ranking policy determining apparatus, and an electronic device, which are provided by the present invention, and specific examples are applied in the text to explain the principles and embodiments of the present invention, and the descriptions of the foregoing examples are only used to help understand the method and the core ideas of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.

Claims

1. A ranking policy determination method, comprising:

acquiring mixed insertion sorting information, wherein the mixed insertion sorting information sorts and determines the search results according to first sorting information and second sorting information, the first sorting information sorts and determines the search results according to a first sorting strategy, and the second sorting information sorts and determines the search results according to a second sorting strategy;

according to the mixed insertion sorting information, determining first K pieces of search results which are positioned in the first sorting information and are positioned at the front of the second sorting information, and determining second K pieces of search results which are positioned in the second sorting information and are positioned at the front of the first sorting information, wherein K is a positive integer;

and respectively acquiring user behavior characteristic information corresponding to K first search results and user behavior characteristic information corresponding to K second search results, and determining an optimal sorting strategy, wherein the user behavior characteristic information is specific to the search results sorted according to the mixed insertion sorting information.

2. The method according to claim 1, wherein the interleaving ordering information includes position information corresponding to each search result, and the position information includes position information of the search result in the first ordering information and position information of the search result in the second ordering information.

3. The method according to claim 2, wherein determining the first K first search results positioned earlier in the first ranking information than in the second ranking information and determining the first K second search results positioned earlier in the second ranking information than in the first ranking information according to the shuffle ranking information comprises:

determining the first M search results in the first ranking information and the first M search results in the second ranking information according to the position information corresponding to each search result;

determining the first T search results in the mixed insertion ordering information according to the first M search results in the first ordering information and the first M search results in the second ordering information;

determining the number N1 of search results in the first T pieces of search results, the number N2 of search results being in the first ranking information and in the second ranking information, and the number K is determined according to the N1 and the N2;

selecting the first K first search results from the first M search results of the first ranking information, and selecting the first K second search results from the first M search results of the second ranking information;

wherein M and T are both positive integers, T is greater than or equal to M, and N1 and N2 are integers greater than zero.

4. The method of claim 3, wherein the determining the number N1 of the top T search results, the number N2 of search results with positions earlier in the first ranking information than in the second ranking information, and the K is determined according to the N1 and the N2, comprises:

traversing the front T search results from front to back;

counting the number N1 of search results positioned earlier in the first ordering information than in the second ordering information and counting the number N2 of search results positioned earlier in the second ordering information than in the first ordering information each time one search result is traversed;

comparing the N1 and N2, and recording the equal value of N1 and N2;

after traversing the first T search results, selecting a maximum value from the values when N1 is equal to N2 and determining the maximum value as K.

5. The method according to claim 3, wherein the determining the top T search results in the mixed insertion ranking information according to the top M search results in the first ranking information and the top M search results in the second ranking information comprises:

6. The method according to claim 1, wherein the obtaining user behavior feature information corresponding to the K first search results and the K second search results, respectively, and determining an optimal ranking policy includes:

respectively acquiring user behavior characteristic information corresponding to K first search results and user behavior characteristic information corresponding to K second search results according to the search logs;

according to the user behavior feature information, respectively determining the relevancy information of the K first search results and the relevancy information of the K second search results;

and comparing the relevancy information of the K first search results with the relevancy information of the K second search results, and determining the ranking strategy corresponding to the K search results with larger relevancy information as the optimal ranking strategy.

7. The method of claim 6, wherein the user behavior feature information comprises click information;

the determining the relevancy information of the K first search results and the relevancy information of the K second search results according to the user behavior feature information respectively includes:

determining relevancy information corresponding to the K first search results according to first click information corresponding to the K first search results; and determining the relevancy information corresponding to the K second search results according to the second click information corresponding to the K second search results.

8. The method of claim 6, wherein the user behavior feature information comprises click information and user satisfaction information of a webpage corresponding to the search result;

determining relevancy information corresponding to the K first search results according to first click information and user satisfaction information corresponding to the K first search results; and determining the relevancy information corresponding to the K second search results according to the second click information and the user satisfaction information corresponding to the K second search results.

9. An ordering policy determination apparatus, comprising:

the information acquisition module is used for acquiring mixed insertion sorting information, the mixed insertion sorting information is used for sorting and determining the search results according to first sorting information and second sorting information, the first sorting information is used for sorting and determining the search results according to a first sorting strategy, and the second sorting information is used for sorting and determining the search results according to a second sorting strategy;

the information determining module is used for determining K first search results which are positioned in the first sorting information and are positioned before the second sorting information according to the mixed insertion sorting information, and determining K second search results which are positioned in the second sorting information and are positioned before the first sorting information, wherein K is a positive integer;

and the strategy determining module is used for respectively obtaining user behavior characteristic information corresponding to the K first search results and user behavior characteristic information corresponding to the K second search results and determining an optimal sorting strategy, wherein the user behavior characteristic information is specific to the search results sorted according to the mixed insertion sorting information.

10. The apparatus according to claim 9, wherein the interleaving ordering information includes position information corresponding to each search result, and the position information includes position information of the search result in the first ordering information and position information of the search result in the second ordering information.

11. The apparatus of claim 10, wherein the information determining module comprises:

the first determining submodule is used for determining the first M search results in the first sequencing information and the first M search results in the second sequencing information according to the position information corresponding to each search result;

the second determining submodule is used for determining the first T search results in the mixed insertion ordering information according to the first M search results in the first ordering information and the first M search results in the second ordering information;

a third determining sub-module, configured to determine, from the top T search results, the number N1 of search results whose positions in the first ranking information are further forward than those in the second ranking information, and the number N2 of search results whose positions in the second ranking information are further forward than those in the first ranking information, and determine K according to the N1 and N2;

the search result determining submodule is used for selecting the first K first search results from the first M search results of the first sequencing information and selecting the first K second search results from the first M search results of the second sequencing information;

12. The apparatus of claim 11,

the third determining submodule is used for traversing the front T search results from front to back; counting the number N1 of search results positioned earlier in the first ordering information than in the second ordering information and counting the number N2 of search results positioned earlier in the second ordering information than in the first ordering information each time one search result is traversed; comparing the N1 and N2, and recording the equal value of N1 and N2; after traversing the first T search results, selecting a maximum value from the values when N1 is equal to N2 and determining the maximum value as K.

13. The apparatus of claim 11,

and the second determining submodule is used for removing duplication from a union set of the first M search results of the first ordering information and the first M search results of the second ordering information to obtain the first T search results in the mixed insertion ordering information.

14. The apparatus of claim 9, wherein the policy determination module comprises:

the characteristic information acquisition submodule is used for respectively acquiring user behavior characteristic information corresponding to K first search results and user behavior characteristic information corresponding to K second search results according to the search logs;

the relevancy information determining submodule is used for respectively determining the relevancy information of the K first search results and the relevancy information of the K second search results according to the user behavior feature information;

and the optimal strategy determining submodule is used for comparing the relevancy information of the K first search results with the relevancy information of the K second search results and determining the sorting strategy corresponding to the K search results with larger relevancy information as the optimal sorting strategy.

15. The apparatus of claim 14, wherein the user behavior feature information comprises click information;

the relevancy information determining submodule is used for determining relevancy information corresponding to the K first search results according to the first click information corresponding to the K first search results; and determining the relevancy information corresponding to the K second search results according to the second click information corresponding to the K second search results.

16. The apparatus according to claim 14, wherein the user behavior feature information includes click information and user satisfaction information of a web page corresponding to the search result;

the relevancy information determining submodule is used for determining relevancy information corresponding to the K first search results according to first click information and user satisfaction information corresponding to the K first search results; and determining the relevancy information corresponding to the K second search results according to the second click information and the user satisfaction information corresponding to the K second search results.

17. A readable storage medium, wherein instructions in the storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the ordering policy determination method according to any one of method claims 1-8.

18. An electronic device comprising a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors the one or more programs including instructions for:

19. The electronic device of claim 18, wherein the shuffle and add ordering information includes position information corresponding to each search result, and the position information includes position information of the search result in the first ordering information and position information of the search result in the second ordering information.

20. The electronic device of claim 19, wherein determining, according to the shuffle ordering information, first K first search results positioned earlier in the first ordering information than in the second ordering information, and determining second K second search results positioned earlier in the second ordering information than in the first ordering information, comprises:

21. The electronic device of claim 20, wherein the determining the K from the top T search results, the number N1 of search results positioned earlier in the first ranking information than in the second ranking information, and the number N2 of search results positioned earlier in the second ranking information than in the first ranking information, and the N1 and N2 comprises:

traversing the front T search results from front to back;

comparing the N1 and N2, and recording the equal value of N1 and N2;

22. The electronic device of claim 20, wherein the determining the top T search results in the shuffled ranking information according to the top M search results in the first ranking information and the top M search results in the second ranking information comprises:

23. The electronic device according to claim 18, wherein the obtaining user behavior feature information corresponding to K first search results and K second search results, respectively, and determining an optimal ranking policy includes:

24. The electronic device of claim 23, wherein the user behavior feature information comprises click information;

25. The electronic device of claim 23, wherein the user behavior feature information includes click information and user satisfaction information of a web page corresponding to the search result;