WO2020143346A1 - 排序 - Google Patents

排序 Download PDF

Info

Publication number
WO2020143346A1
WO2020143346A1 PCT/CN2019/120676 CN2019120676W WO2020143346A1 WO 2020143346 A1 WO2020143346 A1 WO 2020143346A1 CN 2019120676 W CN2019120676 W CN 2019120676W WO 2020143346 A1 WO2020143346 A1 WO 2020143346A1
Authority
WO
WIPO (PCT)
Prior art keywords
clicked
candidate
unclicked
neural network
network model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2019/120676
Other languages
English (en)
French (fr)
Inventor
祝升
汤彪
张弓
汪非易
王仲远
朱迪
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Sankuai Online Technology Co Ltd
Original Assignee
Beijing Sankuai Online Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN201910191098.6A external-priority patent/CN111428125B/zh
Application filed by Beijing Sankuai Online Technology Co Ltd filed Critical Beijing Sankuai Online Technology Co Ltd
Priority to KR1020217019287A priority Critical patent/KR20210090706A/ko
Publication of WO2020143346A1 publication Critical patent/WO2020143346A1/zh
Priority to US17/370,084 priority patent/US20210374149A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/29—Geographical information databases

Definitions

  • Embodiments of the present disclosure relate to the field of search recommendation technology, and in particular, to a sorting method, device, electronic device, and readable storage medium.
  • the search recommendation platform can recommend several search results to the user based on the keywords entered by the user, and the search results need to be sorted and displayed to the user, so that the accuracy of sorting directly affects the recommendation effect.
  • deep learning can be applied to sorting, for example, DWN (Deep and Wide Network) model, DFM (Deep Factorization Machines), and DCN (Deep and Cross Network). model.
  • DWN Deep and Wide Network
  • DFM Deep Factorization Machines
  • DCN Deep and Cross Network
  • Embodiments of the present disclosure provide a sorting method, device, electronic device, and readable storage medium to solve the above-mentioned problems of prior art sorting.
  • an embodiment of the present disclosure provides a sorting method, including:
  • Group the data sample set according to the search request to obtain at least one search request group
  • the neural network model is based on the current prediction values of the clicked and unclicked candidates in the same search request group, and the clicked and unclicked
  • the change amount of the normalized discount cumulative gain before and after the clicked candidate sorting positions are swapped, and the parameters of the neural network model are adjusted;
  • the neural network model is used to rank the target objects associated with the target search words.
  • the amount of change and the steps of adjusting the parameters of the neural network model include:
  • the normalized cumulative discount gain of the clicked candidate ranking before the unclicked candidate is calculated, and the clicked candidate ranking is The normalized discount cumulative gain after the candidate that has not been clicked gets the first gain and the second gain;
  • the first product is an exponent to calculate the exponent result, and the first exponent result is obtained;
  • the parameters of the neural network model are adjusted according to the gradient value between the clicked candidate object and the non-clicked candidate object.
  • the gradient value ⁇ i,j between the clicked candidate object and the unclicked candidate object is calculated according to the following formula:
  • ⁇ is the preset coefficient
  • S i and S j are the current predicted values of the clicked candidate and the unclicked candidate, respectively
  • ⁇ NDCG is before and after the ordering positions of the clicked and unclicked candidates are swapped
  • the normalized discount cumulative gain change value is the normalized discount cumulative gain change value.
  • the step of adjusting the parameters of the neural network model according to the gradient value between the clicked candidate object and the unclicked candidate object includes:
  • For each candidate object separately obtain other candidate objects marked before the position of the candidate object, and obtain other candidate objects marked after the position of the candidate object to obtain the first object and the second object;
  • the method further includes:
  • the loss value is calculated according to the current prediction values of the clicked and unclicked candidates in the same search request group and the position markers of the candidates;
  • the training is ended.
  • the step of calculating the loss value according to the current prediction values of the clicked and unclicked candidate objects and the position marks of the candidate objects in the same search request group includes:
  • the average value of the loss values of the clicked and unclicked candidate objects is calculated to obtain the loss value.
  • the loss value is calculated according to the following formula:
  • the method before using the neural network model to sort the target objects associated with the target search words, the method further includes:
  • the trained neural network model is deployed to an application platform, and the application platform calls the neural network model to sort the target objects associated with the target search words.
  • an embodiment of the present disclosure provides a sorting device, including:
  • the sample set grouping module is used to group the data sample set according to the search request to obtain at least one search request group;
  • the neural network model training module is used to train the neural network model by using the search request grouping.
  • the neural network model is based on the current prediction of the clicked and unclicked candidate objects in the same search request grouping
  • the value, and the amount of change in the normalized discount cumulative gain before and after the clicked and unclicked candidate sort positions are swapped, adjusting the parameters of the neural network model;
  • a sorting module is used to sort the target objects associated with the target search words using the neural network model.
  • the neural network model training module includes:
  • the gain calculation submodule is used to calculate the normalized cumulative discounted gain of the clicked candidates before the unclicked candidates for the clicked and unclicked candidates in the same search request group, and, The clicked candidates sort the normalized discount cumulative gain after the unclicked candidates to obtain the first gain and the second gain;
  • An absolute value calculation sub-module for calculating the absolute value of the difference between the first gain and the second gain
  • the first difference calculation sub-module is used to calculate the difference between the current predicted value of the clicked candidate object and the unclicked candidate object to obtain the first difference value
  • a first product calculation submodule used to calculate the product of the difference and the preset coefficient to obtain the first product
  • a first exponential result calculation submodule which is used to calculate the exponential result using the natural constant as the base, and the first product is the exponent, to obtain the first exponent result;
  • a first numerical calculation sub-module used to calculate the sum of the index result and 1, to obtain the first numerical value
  • a second product calculation submodule configured to calculate the product of the preset coefficient and the absolute value to obtain a second product
  • a gradient value calculation sub-module for calculating the ratio of the second product to the first value, and taking the opposite number to obtain the gradient value between the clicked candidate object and the unclicked candidate object;
  • the model parameter adjustment submodule is used to adjust the parameters of the neural network model according to the gradient value between the clicked candidate object and the unclicked candidate object.
  • the gradient value ⁇ i,j between the clicked candidate object and the unclicked candidate object is calculated according to the following formula:
  • ⁇ is the preset coefficient
  • S i and S j are the current predicted values of the clicked candidate and the unclicked candidate, respectively
  • ⁇ NDCG is before and after the ordering positions of the clicked and unclicked candidates are swapped
  • the normalized discount cumulative gain change value is the normalized discount cumulative gain change value.
  • model parameter adjustment sub-module includes:
  • the object acquiring unit is configured to acquire, for each candidate object, other candidate objects marked before the position of the candidate object, and acquire other candidate objects marked after the position of the candidate object, to obtain the first object, Second object
  • a first gradient sum calculation unit configured to calculate a sum of gradient values of the candidate object and the first object to obtain a first gradient sum
  • a second gradient sum calculation unit configured to calculate a sum of gradient values between the candidate object and the second object to obtain a second gradient sum
  • An adjustment gradient calculation unit used to calculate the difference between the second gradient sum and the first gradient sum to obtain the adjustment gradient of the candidate object
  • the model parameter adjustment unit is configured to adjust parameters corresponding to the candidate objects in the neural network model according to the adjustment gradient.
  • the neural network model training module further includes:
  • the loss value calculation submodule is used to calculate the loss value according to the current prediction value of the clicked and unclicked candidate objects and the position markers of the candidate objects in the same search request group after each training;
  • the training end sub-module is used to end training when the loss value is less than or equal to a preset loss value threshold.
  • the loss value calculation sub-module includes:
  • the second difference calculation unit is used to calculate the difference between the position markers of 1 and the candidate for the clicked and unclicked candidates in the same search request group to obtain the second difference;
  • the third difference calculation unit is used to calculate the difference between the current predicted values of the clicked and unclicked candidates in the same search request group, to obtain the third Difference
  • a third product calculation unit configured to calculate the product of the second difference, the third difference, a preset coefficient, and one-half to obtain a third product
  • a fourth product calculation unit configured to calculate the product of the third difference and the preset coefficient, and take the opposite number to obtain the fourth product
  • a second exponential result calculation unit used to calculate the exponential result using the natural constant as the base, and the fourth product is the exponent, to obtain the second exponential result;
  • the logarithmic result calculation unit is used to calculate the sum of 1 and the result of the second index as a true number, and take the logarithm with 10 as the base to obtain the logarithmic result;
  • a first loss value calculation unit configured to calculate the sum of the third product and the logarithmic result to obtain the loss value of the clicked and unclicked candidate objects
  • the second loss value calculation unit is used to calculate the average value of the loss values of the clicked and unclicked candidate objects to obtain the loss value.
  • the loss values of the clicked and unclicked candidate objects are calculated according to the following formula:
  • the device further includes:
  • the model deployment module is used to deploy the trained neural network model to an application platform, and the application platform calls the neural network model to sort the target objects associated with the target search term.
  • an electronic device including:
  • a processor a memory, and a computer program stored on the memory and executable on the processor, and the processor implements the foregoing sequencing method when the processor executes the computer program.
  • an embodiment of the present disclosure provides a computer program, including computer readable code, which, when run on a computing processing device, causes the computing processing device to perform the aforementioned sorting method.
  • embodiments of the present disclosure provide a computer-readable storage medium in which the aforementioned computer program is stored.
  • An embodiment of the present disclosure provides a sorting method by grouping data sample sets according to search requests to obtain at least one search request group; using the search request group to train a neural network model, the neural network model is training During the process, according to the current prediction value of the clicked and unclicked candidates in the same search request group, and the change amount of the normalized discount cumulative gain before and after the exchange of the sorted positions of the clicked and unclicked candidates, Adjust the parameters of the neural network model; use the neural network model to sort the target objects associated with the target search words.
  • the embodiments of the present disclosure can combine with the normalized discount cumulative gain to adjust the neural network model, so that the adjustment result can be more in line with the search recommendation field, which helps to improve the accuracy of the neural network model.
  • FIG. 1 shows a flowchart of specific steps of a sorting method of the present disclosure
  • FIG. 3 is a structural diagram of a sorting device of the present disclosure
  • FIG. 4 is a structural diagram of another sorting device of the present disclosure.
  • FIG. 5 schematically shows a block diagram of an electronic device for performing the method according to the present disclosure.
  • FIG. 6 schematically shows a storage unit for holding or carrying program code implementing the method according to the present disclosure.
  • FIG. 1 shows a flowchart of specific steps of a sorting method of the present disclosure, including:
  • Step 101 Group the data sample set according to the search request to obtain at least one search request group.
  • the data sample set contains a large number of data samples, and each data sample includes: a search request identifier, a keyword input by the user during the search, an object related to the keyword, and an identifier of whether the object is clicked, etc.
  • the search request identifier is the unique identifier of the search request, and multiple data samples with the same search request identifier correspond to the same search request.
  • the embodiment of the present disclosure may group the data samples in the data sample set according to the search request identifier, so that the data samples corresponding to the same search request belong to the same search request group.
  • each search request group is packaged.
  • Step 102 Use the search request group to train a neural network model.
  • the neural network model is based on the current predicted values of clicked and unclicked candidates in the same search request group and clicked The amount of change of the normalized discount cumulative gain before and after the swapping of the sorted position of the candidate object that has not been clicked is adjusted to adjust the parameters of the neural network model.
  • the neural network model is trained in units of search request groups.
  • the neural network model can be a DWN (Deep and Wide Network) model, a DFM (Deep Factorization Machines), and a DCN (Deep and Cross Network) model, etc., which are suitable for search and recommendation Depth model.
  • DWN Deep and Wide Network
  • DFM Deep Factorization Machines
  • DCN Deep and Cross Network
  • Training is to input data samples into the neural network model to obtain the current prediction value of the training, and continuously adjust the parameters of the neural network model according to the current value, repeatedly make predictions and parameter adjustments, and finally make the model optimal. It can be understood that in the initial state, the parameters of the neural network model are random.
  • the embodiments of the present disclosure can implement training through the Tensorflow framework.
  • NDCG Normalized Discounted Cumulative Gain
  • NDCG Normalized Discounted Cumulative Gain
  • the embodiments of the present disclosure may combine with the normalized discount cumulative gain to adjust the neural network model, so that the adjustment result can be more in line with the search recommendation field, which helps to improve the accuracy of the neural network model.
  • Step 103 Use the neural network model to sort the target objects associated with the target search words.
  • the neural network model obtained by training in step 102 of the present disclosure can be used for the ranking of the associated target objects after each user inputs the target search term in actual applications.
  • the target object may be text, video, image, etc.
  • an embodiment of the present disclosure provides a sorting method.
  • the method includes: grouping data sample sets according to search requests to obtain at least one search request group; and using the search request grouping to train a neural network model During the training process, the neural network model is normalized before and after swapping the current predicted values of clicked and unclicked candidates in the same search request group and the sorted positions of clicked and unclicked candidates The amount of change in the cumulative gain of discounts is adjusted to adjust the parameters of the neural network model; the neural network model is used to rank the target objects associated with the target search words.
  • the neural network model can be adjusted in combination with the normalized cumulative discount gain, so that the adjustment result can be more in line with the search recommendation field, which helps to improve the accuracy of the neural network model.
  • the embodiments of the present disclosure describe the optional sorting method.
  • Step 201 Group the data sample set according to the search request to obtain at least one search request group.
  • step 101 can be referred to the detailed description of step 101, and will not be repeated here.
  • Step 202 Use the search request group to train a neural network model.
  • the clicked candidates are calculated separately The normalized discount cumulative gains sorted before the unclicked candidates, and the normalized discount cumulative gains sorted after the unclicked candidates are sorted by the clicked candidates to obtain the first gain and the second gain.
  • calculation formulas of the first gain and the second gain may refer to existing formulas, which are not limited by the embodiment of the present disclosure.
  • Step 203 Calculate the absolute value of the difference between the first gain and the second gain.
  • the absolute value of the difference between the first gain and the second gain It can be calculated by referring to the following formula:
  • NDCG i, j is the normalized discount cumulative gain of the clicked candidate i ranked before the unclicked candidate j
  • NDCG j, i is the clicked candidate i ranked among the unclicked candidates The cumulative gain of normalized discount after j.
  • Step 204 Calculate the difference between the current predicted value of the clicked candidate object and the unclicked candidate object to obtain the first difference value.
  • S i is the current predicted value of the clicked candidate object
  • S j is the current predicted value of the unclicked candidate object.
  • Step 205 Calculate the product of the difference and the preset coefficient to obtain the first product.
  • ⁇ is a preset coefficient, which can be set according to an actual application scenario, which is not limited by the embodiments of the present disclosure.
  • Step 206 Calculate the exponential result using the natural constant as the base and the first product as the exponent to obtain the first exponent result.
  • step 207 the sum of the index result and 1 is calculated to obtain the first value.
  • Step 208 Calculate the product of the preset coefficient and the absolute value to obtain a second product.
  • Step 209 Calculate the ratio of the second product to the first value, and take the opposite number to obtain the gradient value between the clicked candidate object and the unclicked candidate object.
  • Step 210 Adjust the parameters of the neural network model according to the gradient value between the clicked candidate object and the unclicked candidate object.
  • the gradient can reflect the change trend, and thus can be used to guide the adjustment of the model parameters.
  • the embodiments of the present disclosure can accurately adjust the parameters of the model according to the gradient value.
  • step 210 includes sub-steps 2101 to 2105:
  • Sub-step 2101 for each candidate object, separately obtain other candidate objects marked before the position of the candidate object, and obtain other candidate objects marked after the position of the candidate object to obtain the first object and the second object Object.
  • the arrangement order is whether the candidate objects are clicked, the clicked candidate objects are arranged in the front, and the unclicked candidate objects are arranged in the rear.
  • the clicked candidate is marked as 1, and the unclicked candidate is marked as 0, then for the candidate marked as 1, the first object does not exist, only the second object exists; for the marked For a candidate object of 0, the second object does not exist, only the first object exists.
  • the clicked candidate objects can also be marked according to the click-through rate or other guide sorted indicators, so that the first object and the second object can be determined according to the specific values of the specific indicators.
  • Sub-step 2102 calculating the sum of the gradient values of the candidate object and the first object to obtain a first gradient sum.
  • the first gradient sum is the gradient sum between the candidate object and other candidate objects ranked before the candidate object.
  • Sub-step 2103 calculating the sum of gradient values between the candidate object and the second object to obtain a second gradient sum.
  • the second gradient sum is the gradient sum between the candidate object and other candidate objects ranked after the candidate object.
  • Sub-step 2104 calculating the difference between the second gradient sum and the first gradient sum to obtain the adjustment gradient of the candidate object.
  • the adjustment gradient is a unique gradient for each candidate object, and can be used to guide the adjustment model parameters.
  • Sub-step 2105 adjusting parameters corresponding to the candidate objects in the neural network model according to the adjustment gradient.
  • the parameters are adjusted according to the direction of the adjustment gradient.
  • the embodiments of the present disclosure can synthesize all candidate texts, calculate the adjustment gradient, and guide the adjustment of the model parameters, which can accurately adjust the model parameters, which helps to improve the accuracy of the model.
  • Step 211 After each training, calculate the loss value according to the current prediction values of the clicked and unclicked candidate objects and the position marks of the candidate objects in the same search request group.
  • the position marker can mark the front and back positions of the clicked candidate object and the unclicked candidate object, which can be set according to the actual application scenario. For example, when candidate object i precedes candidate object j, the location corresponding to clicked candidate object i and unclicked candidate object j is marked as 1; when candidate object i follows candidate object j, the clicked candidate object The position corresponding to i and the candidate j that has not been clicked is marked as 0.
  • the loss value is used to determine whether the training is over.
  • the step 211 includes sub-steps 2111 to 2118:
  • Sub-step 2111 for the clicked and unclicked candidates in the same search request group, calculate the difference between the position markers of 1 and the candidate to obtain the second difference.
  • S ij is a position mark corresponding to the clicked candidate object i and the unclicked candidate object j.
  • Substep 2112 for the clicked and unclicked candidates in the same search request group, calculate the difference between the current predicted values of the clicked and unclicked candidates to obtain a third difference.
  • Sub-step 2113 calculating the product of the second difference, the third difference, the preset coefficient and one-half to obtain the third product.
  • Sub-step 2114 calculating the product of the third difference and the preset coefficient, and taking the opposite number to obtain the fourth product.
  • the natural constant is used as the base, and the fourth product is an exponent to calculate an exponent result to obtain a second exponent result.
  • sub-step 2116 the sum of 1 and the result of the second exponent is calculated as a true number, and the logarithm is obtained by taking the base 10 as the logarithm to obtain a logarithmic result.
  • Sub-step 2117 calculating the sum of the third product and the logarithmic result to obtain the loss value of the clicked and unclicked candidate objects.
  • Sub-step 2118 the average value of the loss values of the clicked and unclicked candidate objects is calculated to obtain the loss value.
  • a total average value is calculated to obtain a loss value.
  • Step 212 When the loss value is less than or equal to the preset loss value threshold, end the training.
  • the loss value threshold can be set according to actual application scenarios, which is not limited by the embodiments of the present disclosure. It can be understood that when the loss threshold is too large, the accuracy of the trained neural network model is poor, but the training time is short; if the loss threshold is too small, the accuracy of the trained neural network model is higher, but the training time is longer. In practical applications, the loss threshold can be set according to demand.
  • the neural network model using the current parameters at the end of training can be used as the final neural network model, which can be used in practical applications.
  • Step 213 Deploy the trained neural network model to the application platform, and the application platform calls the neural network model to sort the target objects associated with the target search term.
  • the application platform may be a search recommendation platform, and in the embodiment of the present disclosure, the application platform adopts the Tensorflow framework.
  • the neural network model can be packaged, stored, and installed on the application platform, so that when the application platform receives the target search term, it first obtains the associated multiple target objects; then the offline neural network model is called to the target Objects are sorted.
  • the pre-trained neural network model can be deployed on the application platform and called offline for ordering, thereby achieving flexible application of the neural network model.
  • Step 214 Use the neural network model to sort the target objects associated with the target search words.
  • step 103 This step can be referred to the detailed description of step 103, and will not be repeated here.
  • an embodiment of the present disclosure provides a sorting method.
  • the method includes: grouping data sample sets according to search requests to obtain at least one search request group; and using the search request grouping to train a neural network model During the training process, the neural network model is normalized before and after swapping the current predicted values of clicked and unclicked candidates in the same search request group and the sorted positions of clicked and unclicked candidates The amount of change in the cumulative gain of discounts is adjusted to adjust the parameters of the neural network model; the neural network model is used to rank the target objects associated with the target search words.
  • the neural network model can be adjusted in combination with the normalized cumulative discount gain, so that the adjustment result can be more in line with the search recommendation field, which helps to improve the accuracy of the neural network model.
  • the sample set grouping module 301 is configured to group the data sample set according to the search request to obtain at least one search request group.
  • the neural network model training module 302 is used to train the neural network model by using the search request grouping. During the training process, the neural network model is based on the current status of the clicked and unclicked candidate objects in the same search request grouping The predicted value, and the amount of change in the normalized discount cumulative gain before and after the sorted positions of the clicked and unclicked candidates are swapped, adjust the parameters of the neural network model.
  • the sorting module 303 is used to sort the target objects associated with the target search words using the neural network model.
  • the embodiment of the present disclosure provides a sorting device, the device includes: a sample set grouping module for grouping data sample sets according to a search request to obtain at least one search request group; a neural network model training module Is used to train the neural network model by using the search request grouping.
  • the neural network model is based on the current prediction values of the clicked and unclicked candidate objects in the same search request grouping and clicked
  • the amount of change in the normalized discount cumulative gain before and after swapping with the sorted position of the candidate that has not been clicked is adjusted to adjust the parameters of the neural network model;
  • the sorting module is used to use the neural network model to associate the target with the search term Objects are sorted.
  • the neural network model can be adjusted in combination with the normalized discount cumulative gain, so that the adjustment result can be more in line with the search recommendation field, which helps to improve the accuracy of the neural network model.
  • FIG. 4 shows a structural diagram of another sorting device of the present disclosure, specifically as follows.
  • the sample set grouping module 401 is used to group the data sample set according to the search request to obtain at least one search request group.
  • the neural network model training module 402 is used to train the neural network model by using the search request grouping. During the training process, the neural network model is based on the current status of the clicked and unclicked candidate objects in the same search request grouping The predicted value, and the amount of change in the normalized discount cumulative gain before and after the clicked and unclicked candidate sort positions are interchanged, adjust the parameters of the neural network model; optionally, in the embodiment of the present disclosure, the The neural network model training module 402 includes:
  • the gain calculation sub-module 40201 is used to calculate the normalized cumulative discounted gain of the clicked candidates before the unclicked candidates for the clicked and unclicked candidates in the same search request group, and , The clicked candidates sort the normalized discount cumulative gain after the unclicked candidates, to obtain the first gain and the second gain.
  • the absolute value calculation sub-module 40202 is used to calculate the absolute value of the difference between the first gain and the second gain.
  • the first difference calculation submodule 40203 is used to calculate the difference between the current predicted value of the clicked candidate object and the unclicked candidate object to obtain the first difference value.
  • the first product calculation submodule 40204 is used to calculate the product of the difference and the preset coefficient to obtain the first product.
  • the first exponential result calculation sub-module 40205 is used to calculate the exponential result using the natural constant as the base, and the first product is the exponent, to obtain the first exponential result.
  • the first numerical calculation submodule 40206 is used to calculate the sum of the index result and 1 to obtain the first numerical value.
  • the second product calculation submodule 40207 is configured to calculate a product of the preset coefficient and the absolute value to obtain a second product.
  • the gradient value calculation submodule 40208 is configured to calculate the ratio of the second product to the first value, and take the opposite number to obtain the gradient value between the clicked candidate object and the unclicked candidate object.
  • the model parameter adjustment submodule 40209 is configured to adjust the parameters of the neural network model according to the gradient value between the clicked candidate object and the unclicked candidate object.
  • the loss value calculation sub-module 40210 is used to calculate the loss value according to the current prediction values of the clicked and unclicked candidate objects and the position marks of the candidate objects in the same search request group after each training.
  • the training end submodule 40211 is configured to end training when the loss value is less than or equal to a preset loss value threshold.
  • the model deployment module 403 is configured to deploy the trained neural network model to an application platform, and the application platform calls the neural network model to sort the target objects associated with the target search term.
  • the sorting module 404 is used to sort the target objects associated with the target search words using the neural network model.
  • the gradient value ⁇ i,j between the clicked candidate object and the unclicked candidate object is calculated according to the following formula:
  • ⁇ is the preset coefficient
  • S i and S j are the current predicted values of the clicked candidate and the unclicked candidate, respectively
  • ⁇ NDCG is before and after the ordering positions of the clicked and unclicked candidates are swapped Normalized discounted cumulative gain change value.
  • the model parameter adjustment sub-module 40209 includes:
  • the object acquiring unit is configured to acquire, for each candidate object, other candidate objects marked before the position of the candidate object, and acquire other candidate objects marked after the position of the candidate object, to obtain the first object, Second object
  • a first gradient sum calculation unit configured to calculate a sum of gradient values of the candidate object and the first object to obtain a first gradient sum
  • a second gradient sum calculation unit configured to calculate a sum of gradient values between the candidate object and the second object to obtain a second gradient sum
  • An adjustment gradient calculation unit used to calculate the difference between the second gradient sum and the first gradient sum to obtain the adjustment gradient of the candidate object
  • the model parameter adjustment unit is configured to adjust parameters corresponding to the candidate objects in the neural network model according to the adjustment gradient.
  • the loss value calculation sub-module 40210 includes:
  • the second difference calculation unit is used to calculate the difference between the position markers of 1 and the candidate for the clicked and unclicked candidates in the same search request group to obtain the second difference;
  • the third difference calculation unit is used to calculate the difference between the current predicted values of the clicked and unclicked candidates in the same search request group, to obtain the third Difference
  • a third product calculation unit configured to calculate the product of the second difference, the third difference, a preset coefficient, and one-half to obtain a third product
  • a fourth product calculation unit configured to calculate the product of the third difference and the preset coefficient, and take the opposite number to obtain the fourth product
  • a second exponential result calculation unit used to calculate the exponential result using the natural constant as the base, and the fourth product is the exponent, to obtain the second exponential result;
  • the logarithmic result calculation unit is used to calculate the sum of 1 and the result of the second index as a true number, and take the logarithm with 10 as the base to obtain the logarithmic result;
  • a first loss value calculation unit configured to calculate the sum of the third product and the logarithmic result to obtain the loss value of the clicked and unclicked candidate objects
  • the second loss value calculation unit is used to calculate the average value of the loss values of the clicked and unclicked candidate objects to obtain the loss value.
  • the loss values C i,j of the clicked and unclicked candidate objects are calculated according to the following formula:
  • S ij is the difference between the marked values of the clicked candidate text and the unclicked candidate text.
  • the embodiment of the present disclosure provides a sorting device, the device includes: a sample set grouping module for grouping data sample sets according to a search request to obtain at least one search request group; a neural network model training module Is used to train the neural network model by using the search request grouping.
  • the neural network model is based on the current prediction values of the clicked and unclicked candidate objects in the same search request grouping and clicked
  • the amount of change in the normalized discount cumulative gain before and after swapping the sorted positions of candidate objects that have not been clicked is adjusted to adjust the parameters of the neural network model;
  • the sorting module is used to use the neural network model to associate the target with the search term Objects are sorted.
  • the neural network model can be adjusted in combination with the normalized discount cumulative gain, so that the adjustment result can be more in line with the search recommendation field, which helps to improve the accuracy of the neural network model.
  • An embodiment of the present disclosure also provides an electronic device, including: a processor, a memory, and a computer program stored on the memory and executable on the processor, which is implemented when the processor executes the computer program The sorting method of the foregoing embodiment.
  • Embodiments of the present disclosure also provide a computer program, including computer readable code, which, when run on a computing processing device, causes the computing processing device to execute the sorting method of the foregoing embodiment.
  • the embodiment of the present disclosure also provides a computer-readable storage medium in which the computer program of the foregoing embodiment is stored.
  • the device embodiments described above are only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in One place, or can be distributed to multiple network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement without paying creative labor.
  • the various component embodiments of the present disclosure may be implemented in hardware, or implemented in software modules running on one or more processors, or implemented in a combination thereof.
  • a microprocessor or a digital signal processor (DSP) may be used to implement some or all functions of some or all components in a computing processing device according to an embodiment of the present disclosure.
  • DSP digital signal processor
  • the present disclosure may also be implemented as a device or device program (eg, computer program and computer program product) for performing part or all of the methods described herein.
  • Such a program implementing the present disclosure may be stored on a computer-readable storage medium, or may have the form of one or more signals.
  • Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.
  • FIG. 5 shows a computing processing device that can implement the method according to the present disclosure.
  • the computing processing device traditionally includes a processor 510 and a computer program product in the form of a memory 520 or a computer-readable storage medium.
  • the memory 520 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), EPROM, hard disk, or ROM.
  • the memory 520 has a storage space 530 for program code 531 for performing any of the method steps described above.
  • the storage space 530 for program codes may include various program codes 531 for implementing various steps in the above method, respectively. These program codes can be read from or written into one or more computer program products.
  • Such computer program products include program code carriers such as hard disks, compact disks (CDs), memory cards or floppy disks.
  • Such a computer program product is usually a portable or fixed storage unit as described with reference to FIG. 6.
  • the storage unit may have storage sections, storage spaces, and the like arranged similarly to the memory 520 in the computing processing device of FIG. 5.
  • the program code may be compressed in an appropriate form, for example.
  • the storage unit includes computer readable code 531', that is, code that can be read by, for example, a processor such as 510, which, when executed by a computing processing device, causes the computing processing device to perform the method described above The various steps.
  • any reference signs between parentheses should not be constructed as limitations on the claims.
  • the word “comprising” does not exclude the presence of elements or steps not listed in a claim.
  • the word “a” or “one” before an element does not exclude the presence of multiple such elements.
  • the present disclosure can be implemented by means of hardware including several different elements and by means of a suitably programmed computer. In the unit claims enumerating several devices, several of these devices may be embodied by the same hardware item.
  • the use of the words first, second, and third does not indicate any order. These words can be interpreted as names.

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Remote Sensing (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

本公开实施例提供了一种排序方法，涉及搜索推荐技术领域，本公开实施例的所述排序方法包括：对数据样本集按照搜索请求进行分组，得到至少一个搜索请求分组；采用所述搜索请求分组对神经网络模型进行训练，所述神经网络模型在训练过程中，根据同一个搜索请求分组中被点击和未被点击的候选对象的当前预测值、以及被点击和未被点击的候选对象排序位置互换前后的归一化折扣累积增益的变化量，调整所述神经网络模型的参数；采用所述神经网络模型对目标搜索词关联的目标对象进行排序。

Description

排序
本申请要求在2019年01月10日提交中国专利局、申请号为201910024150.9的发明名称为“排序方法、装置、电子设备及可读存储介质”的中国专利申请的优先权，以及，在2019年03月12日提交中国专利局、申请号为201910191098.6、发明名称为“排序方法、装置、电子设备及可读存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中
技术领域
本公开的实施例涉及搜索推荐技术领域，尤其涉及一种排序方法、装置、电子设备及可读存储介质。
背景技术
搜索推荐平台可以根据用户输入的关键词向用户推荐若干搜索结果，而搜索结果需要排序之后展示给用户，从而排序的准确性直接影响推荐效果。
现有技术中，深度学习可以应用于排序，例如，DWN(Deep and Wide Network,深度广度网络)模型、DFM(Deep Factorization Machines,深度因子分解器)和DCN(Deep and Cross Network，深度交叉网络)模型。而上述三种模型均采用对数损失函数，其不能准确的表示搜索效果，导致训练得到的模型的排序准确度较差。
发明内容
本公开的实施例提供一种排序方法、装置、电子设备及可读存储介质，以解决现有技术排序的上述问题。
第一方面，本公开的实施例提供了一种排序方法，包括：
对数据样本集按照搜索请求进行分组，得到至少一个搜索请求分组；
采用所述搜索请求分组对神经网络模型进行训练，所述神经网络模型在训练过程中，根据同一个搜索请求分组中被点击和未被点击的候选对象的当前预测值、以及被点击和未被点击的候选对象排序位置互换前后的归一化折扣累积增益的变化量，调整所述神经网络模型的参数；
采用所述神经网络模型对目标搜索词关联的目标对象进行排序。
可选地，所述根据同一个搜索请求分组中被点击和未被点击的候选对象的当前预测值、以及被点击和未被点击的候选对象排序位置互换前后的归一 化折扣累积增益的变化量，调整所述神经网络模型的参数的步骤，包括：
对于同一个搜索请求分组中被点击和未被点击的候选对象，分别计算被点击的候选对象排序在未被点击的候选对象之前的归一化折扣累计增益，以及，被点击的候选对象排序在未被点击的候选对象之后的归一化折扣累计增益，得到第一增益和第二增益；
计算所述第一增益和第二增益之间的差值的绝对值；
计算被点击的候选对象与未被点击的候选对象的当前预测值之间的差值，得到第一差值；
计算所述差值与预设系数的乘积，得到第一乘积；
以自然常数为底数，所述第一乘积为指数计算指数结果，得到第一指数结果；
计算所述指数结果与1之和，得到第一数值；
计算所述预设系数与所述绝对值的乘积，得到第二乘积；
计算所述第二乘积与所述第一数值的比值，并取相反数得到所述被点击的候选对象和未被点击的候选对象之间的梯度值；
根据所述被点击的候选对象和未被点击的候选对象之间的梯度值调整所述神经网络模型的参数。
可选地，所述被点击的候选对象和未被点击的候选对象之间的梯度值λ i,j根据如下公式计算得到：
Figure PCTCN2019120676-appb-000001
其中，σ为预设系数，S i、S j分别为被点击的候选对象与未被点击的候选对象的当前预测值，Δ NDCG为被点击和未被点击的候选对象排序位置互换前后的归一化折扣累积增益的变化值。
可选地，所述根据所述被点击的候选对象和未被点击的候选对象之间的梯度值调整所述神经网络模型的参数的步骤，包括：
针对每个候选对象，分别获取标记在所述候选对象的位置之前的其他候选对象，以及，获取标记在所述候选对象的位置之后的其他候选对象，得到第一对象、第二对象；
计算所述候选对象与所述第一对象的梯度值之和，得到第一梯度和；
计算所述候选对象与所述第二对象之间的梯度值之和，得到第二梯度 和；
计算所述第二梯度和与第一梯度和的差值，得到所述候选对象的调整梯度；
根据所述调整梯度调整所述神经网络模型中所述候选对象对应的参数。
可选地，所述方法还包括：
在每次训练之后，根据同一个搜索请求分组中被点击和未被点击的候选对象的当前预测值以及候选对象的位置标记计算损失值；
在所述损失值小于或等于预设损失值阈值的情况下，结束训练。
可选地，所述根据同一个搜索请求分组中被点击和未被点击的候选对象的当前预测值以及候选对象的位置标记计算损失值的步骤，包括：
对于同一个搜索请求分组中被点击和未被点击的候选对象，计算1与候选对象的位置标记的差值，得到第二差值；
对于同一个搜索请求分组中被点击和未被点击的候选对象，计算被点击的候选对象和未被点击的候选对象的当前预测值的差值，得到第三差值；
计算所述第二差值、第三差值、预设系数以及二分之一的乘积，得到第三乘积；
计算所述第三差值和预设系数的乘积，并取相反数得到第四乘积；
以自然常数为底数，所述第四乘积为指数计算指数结果，得到第二指数结果；
计算1与所述第二指数结果之和作为真数，并以10为底数取对数，得到对数结果；
计算所述第三乘积和所述对数结果之和，得到被点击和未被点击的候选对象的损失值；
计算所述被点击和未被点击的候选对象的损失值的平均值，得到损失值。
可选地，所述损失值根据如下公式计算得到：
Figure PCTCN2019120676-appb-000002
可选地，在所述采用所述神经网络模型对目标搜索词关联的目标对象进行排序之前，还包括：
将训练得到的神经网络模型部署至应用平台，所述应用平台调用所述神 经网络模型对目标搜索词关联的目标对象进行排序。
第二方面，本公开的实施例提供了一种排序装置，包括：
样本集分组模块，用于对数据样本集按照搜索请求进行分组，得到至少一个搜索请求分组；
神经网络模型训练模块，用于采用所述搜索请求分组对神经网络模型进行训练，所述神经网络模型在训练过程中，根据同一个搜索请求分组中被点击和未被点击的候选对象的当前预测值、以及被点击和未被点击的候选对象排序位置互换前后的归一化折扣累积增益的变化量，调整所述神经网络模型的参数；
排序模块，用于采用所述神经网络模型对目标搜索词关联的目标对象进行排序。
可选地，所述神经网络模型训练模块，包括：
增益计算子模块，用于对于同一个搜索请求分组中被点击和未被点击的候选对象，分别计算被点击的候选对象排序在未被点击的候选对象之前的归一化折扣累计增益，以及，被点击的候选对象排序在未被点击的候选对象之后的归一化折扣累计增益，得到第一增益和第二增益；
绝对值计算子模块，用于计算所述第一增益和第二增益之间的差值的绝对值；
第一差值计算子模块，用于计算被点击的候选对象与未被点击的候选对象的当前预测值之间的差值，得到第一差值；
第一乘积计算子模块，用于计算所述差值与预设系数的乘积，得到第一乘积；
第一指数结果计算子模块，用于以自然常数为底数，所述第一乘积为指数计算指数结果，得到第一指数结果；
第一数值计算子模块，用于计算所述指数结果与1之和，得到第一数值；
第二乘积计算子模块，用于计算所述预设系数与所述绝对值的乘积，得到第二乘积；
梯度值计算子模块，用于计算所述第二乘积与所述第一数值的比值，并取相反数得到所述被点击的候选对象和未被点击的候选对象之间的梯度值；
模型参数调整子模块，用于根据所述被点击的候选对象和未被点击的候 选对象之间的梯度值调整所述神经网络模型的参数。
可选地，所述被点击的候选对象和未被点击的候选对象之间的梯度值λ i,j根据如下公式计算得到：
Figure PCTCN2019120676-appb-000003
其中，σ为预设系数，S i、S j分别为被点击的候选对象与未被点击的候选对象的当前预测值，Δ NDCG为被点击和未被点击的候选对象排序位置互换前后的归一化折扣累积增益的变化值。
可选地，所述模型参数调整子模块，包括：
对象获取单元，用于针对每个候选对象，分别获取标记在所述候选对象的位置之前的其他候选对象，以及，获取标记在所述候选对象的位置之后的其他候选对象，得到第一对象、第二对象；
第一梯度和计算单元，用于计算所述候选对象与所述第一对象的梯度值之和，得到第一梯度和；
第二梯度和计算单元，用于计算所述候选对象与所述第二对象之间的梯度值之和，得到第二梯度和；
调整梯度计算单元，用于计算所述第二梯度和与第一梯度和的差值，得到所述候选对象的调整梯度；
模型参数调整单元，用于根据所述调整梯度调整所述神经网络模型中所述候选对象对应的参数。
可选地，所述神经网络模型训练模块还包括：
损失值计算子模块，用于在每次训练之后，根据同一个搜索请求分组中被点击和未被点击的候选对象的当前预测值以及候选对象的位置标记计算损失值；
训练结束子模块，用于在所述损失值小于或等于预设损失值阈值的情况下，结束训练。
可选地，所述损失值计算子模块，包括：
第二差值计算单元，用于对于同一个搜索请求分组中被点击和未被点击的候选对象，计算1与候选对象的位置标记的差值，得到第二差值；
第三差值计算单元，用于对于同一个搜索请求分组中被点击和未被点击的候选对象，计算被点击的候选对象和未被点击的候选对象的当前预测值的 差值，得到第三差值；
第三乘积计算单元，用于计算所述第二差值、第三差值、预设系数以及二分之一的乘积，得到第三乘积；
第四乘积计算单元，用于计算所述第三差值和预设系数的乘积，并取相反数得到第四乘积；
第二指数结果计算单元，用于以自然常数为底数，所述第四乘积为指数计算指数结果，得到第二指数结果；
对数结果计算单元，用于计算1与所述第二指数结果之和作为真数，并以10为底数取对数，得到对数结果；
第一损失值计算单元，用于计算所述第三乘积和所述对数结果之和，得到被点击和未被点击的候选对象的损失值；
第二损失值计算单元，用于计算所述被点击和未被点击的候选对象的损失值的平均值，得到损失值。
可选地，所述被点击和未被点击的候选对象的损失值根据如下公式计算得到：
Figure PCTCN2019120676-appb-000004
可选地，所述装置还包括：
模型部署模块，用于将训练得到的神经网络模型部署至应用平台，所述应用平台调用所述神经网络模型对目标搜索词关联的目标对象进行排序。
第三方面，本公开的实施例提供了一种电子设备，包括：
处理器、存储器以及存储在所述存储器上并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现前述排序方法。
第四方面，本公开的实施例提供了一种计算机程序，包括计算机可读代码，当所述计算机可读代码在计算处理设备上运行时，导致所述计算处理设备执行前述排序方法。
第五方面，本公开的实施例提供了一种计算机可读存储介质，其中存储了前述计算机程序。
本公开的实施例提供了一种排序方法，通过对数据样本集按照搜索请求进行分组，得到至少一个搜索请求分组；采用所述搜索请求分组对神经网络 模型进行训练，所述神经网络模型在训练过程中，根据同一个搜索请求分组中被点击和未被点击的候选对象的当前预测值、以及被点击和未被点击的候选对象排序位置互换前后的归一化折扣累积增益的变化量，调整所述神经网络模型的参数；采用所述神经网络模型对目标搜索词关联的目标对象进行排序。本公开的实施例可以结合归一化折扣累积增益调整神经网络模型，使得调整结果更能符合搜索推荐领域，有助于提高神经网络模型的准确度。
上述说明仅是本公开技术方案的概述，为了能够更清楚了解本公开的技术手段，而可依照说明书的内容予以实施，并且为了让本公开的上述和其它目的、特征和优点能够更明显易懂，以下特举本公开的具体实施方式。
附图说明
为了更清楚地说明本公开实施例的技术方案，下面将对本公开的实施例的描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本公开的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1示出了本公开的一种排序方法的具体步骤流程图；
图2示出了本公开的另一种排序方法的具体步骤流程图；
图3是本公开的的一种排序装置的结构图；
图4是本公开的的另一种排序装置的结构图；
图5示意性地示出了用于执行根据本公开的方法的电子设备的框图；以及
图6示意性地示出了用于保持或者携带实现根据本公开的方法的程序代码的存储单元。
具体实施例为使本公开的实施例的目的、技术方案和优点更加清楚，下面将结合本公开的实施例中的附图，对本公开的实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本公开的一部分实施例，而不是全部的实施例。基于本公开中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本公开保护的范围。
实施例一
参照图1，其示出了本公开的一种排序方法的具体步骤流程图，包括：
步骤101，对数据样本集按照搜索请求进行分组，得到至少一个搜索请 求分组。
其中，数据样本集包含大量数据样本，每个数据样本包括：搜索请求标识、搜索时用户输入的关键字、与该关键字相关的对象、以及该对象是否被点击的标识等。
在实际应用中，搜索请求标识是搜索请求的唯一标识，搜索请求标识相同的多个数据样本对应同一个搜索请求。本公开实施例可以根据搜索请求标识将数据样本集中的数据样本进行分组，使得同一个搜索请求对应的数据样本属于同一个搜索请求分组。
具体地，在进行分组之后，将每个搜索请求分组进行打包。
步骤102，采用所述搜索请求分组对神经网络模型进行训练，所述神经网络模型在训练过程中，根据同一个搜索请求分组中被点击和未被点击的候选对象的当前预测值、以及被点击和未被点击的候选对象排序位置互换前后的归一化折扣累积增益的变化量，调整所述神经网络模型的参数。
在本公开实施例中，以搜索请求分组为单位对神经网络模型进行训练。
其中，神经网络模型可以为DWN(Deep and Wide Network,深度广度网络)模型、DFM(Deep Factorization Machines,深度因子分解器)和DCN(Deep and Cross Network，深度交叉网络)模型等适用于搜索推荐领域的深度模型。
训练是将数据样本输入至神经网络模型中得到该次训练的当前预测值，并根据当前值不断调整神经网络模型的参数，反复进行预测和参数调整，最终使得模型最优。可以理解，在初始状态下，神经网络模型的参数为随机的。本公开实施例可以通过Tensorflow框架实现训练。
归一化折扣累积增益(NDCG，Normalized Discounted Cumulative Gain)是搜索推荐系统常用的指标，NDCG综合了相关性和位置两个因素，NDCG的公式已经是搜索推荐技术领域的通用公式，本公开实施例对其不再赘述。
具体地，在每次训练结束之后，首先根据同一个搜索请求分组中被点击和未被点击的候选对象的当前预测值、以及被点击和未被点击的候选对象排序位置互换前后的归一化折扣累积增益的变化量，计算每个数据样本的梯度值；然后根据梯度值调整神经网络模型的参数。
本公开实施例可以结合归一化折扣累积增益调整神经网络模型，使得调整结果更能符合搜索推荐领域，有助于提高神经网络模型的准确度。
步骤103，采用所述神经网络模型对目标搜索词关联的目标对象进行排序。
本公开通过步骤102训练得到的神经网络模型可以用于实际应用中每个用户输入目标搜索词之后，关联的目标对象的排序。
其中，目标对象可以为文本、视频、图像等。
综上所述，本公开实施例提供了一种排序方法，所述方法包括：对数据样本集按照搜索请求进行分组，得到至少一个搜索请求分组；采用所述搜索请求分组对神经网络模型进行训练，所述神经网络模型在训练过程中，根据同一个搜索请求分组中被点击和未被点击的候选对象的当前预测值、以及被点击和未被点击的候选对象排序位置互换前后的归一化折扣累积增益的变化量，调整所述神经网络模型的参数；采用所述神经网络模型对目标搜索词关联的目标对象进行排序。可以结合归一化折扣累积增益调整神经网络模型，使得调整结果更能符合搜索推荐领域，有助于提高神经网络模型的准确度。
实施例二
本公开实施例对可选地排序方法进行了描述。
步骤201，对数据样本集按照搜索请求进行分组，得到至少一个搜索请求分组。
该步骤可以参照步骤101的详细说明，在此不再赘述。
步骤202，采用所述搜索请求分组对神经网络模型进行训练，所述神经网络模型在训练过程中，对于同一个搜索请求分组中被点击和未被点击的候选对象，分别计算被点击的候选对象排序在未被点击的候选对象之前的归一化折扣累计增益，以及，被点击的候选对象排序在未被点击的候选对象之后的归一化折扣累计增益，得到第一增益和第二增益。
在本公开实施例中，第一增益和第二增益的计算公式可以参照现有公式，本公开实施例对其不加以限制。
步骤203，计算所述第一增益和第二增益之间的差值的绝对值。
具体地，第一增益和第二增益之间的差值的绝对值
Figure PCTCN2019120676-appb-000005
可以参照如下公式计算得到：
Figure PCTCN2019120676-appb-000006
其中，NDCG i,j为被点击的候选对象i排序在未被点击的候选对象j之前的归一化折扣累计增益，NDCG j,i为被点击的候选对象i排序在未被点击的候选对象j之后的归一化折扣累计增益。
步骤204，计算被点击的候选对象与未被点击的候选对象的当前预测值之间的差值，得到第一差值。
具体地，第一差值M1 i,j的计算公式如下：
M1 i,j＝S i-S j     (2)
其中，S i为被点击的候选对象的当前预测值，S j为未被点击的候选对象的当前预测值。
步骤205，计算所述差值与预设系数的乘积，得到第一乘积。
具体地，第一乘积P1 i,j的计算公式如下：
P1 i,j＝σ·M1 i,j＝σ·(S i-S j)     (3)
其中，σ为预设系数，可以根据实际应用场景设定，本公开实施例对其不加以限制。
步骤206，以自然常数为底数，所述第一乘积为指数计算指数结果，得到第一指数结果。
具体地，第一指数结果I1 i,j的计算公式如下：
Figure PCTCN2019120676-appb-000007
步骤207，计算所述指数结果与1之和，得到第一数值。
具体地，第一数值V1 i,j的计算公式如下：
Figure PCTCN2019120676-appb-000008
步骤208，计算所述预设系数与所述绝对值的乘积，得到第二乘积。
具体地，第二乘积P2 i,j的计算公式如下：
Figure PCTCN2019120676-appb-000009
步骤209，计算所述第二乘积与所述第一数值的比值，并取相反数得到所述被点击的候选对象和未被点击的候选对象之间的梯度值。
具体地，被点击的候选对象和未被点击的候选对象之间的梯度值λ i,j的计算公式如下：
Figure PCTCN2019120676-appb-000010
步骤210，根据所述被点击的候选对象和未被点击的候选对象之间的梯度值调整所述神经网络模型的参数。
可以理解，梯度可以体现变化趋势，从而可以用于指导模型参数的调整。
本公开实施例可以根据梯度值准确调整模型的参数。
可选地，在本公开的另一种实施例中，步骤210包括子步骤2101至2105：
子步骤2101，针对每个候选对象，分别获取标记在所述候选对象的位置之前的其他候选对象，以及，获取标记在所述候选对象的位置之后的其他候选对象，得到第一对象、第二对象。
在实际应用中，排列顺序为候选对象是否被点击，被点击的候选对象排列在前面，未被点击的候选对象排列在后面。特殊地，若被点击的候选对象被标记为1，未被点击的候选对象被标记为0，则对于被标记为1的候选对象，第一对象不存在，仅存在第二对象；对于被标记为0的候选对象，第二对象不存在，仅存在第一对象。
当然，被点击的候选对象还可以按照点击率或其他指导排序的指标进行标记，从而可以根据具体指标的具体取值确定第一对象和第二对象。
子步骤2102，计算所述候选对象与所述第一对象的梯度值之和，得到第一梯度和。
可以理解，对于一个候选对象，第一梯度和为该候选对象与排序在该候选对象之前的其他候选对象之间的梯度和。
子步骤2103，计算所述候选对象与所述第二对象之间的梯度值之和，得到第二梯度和。
可以理解，对于一个候选对象，第二梯度和为该候选对象与排序在该候选对象之后的其他候选对象之间的梯度和。
子步骤2104，计算所述第二梯度和与第一梯度和的差值，得到所述候选对象的调整梯度。
可以理解，调整梯度为每个候选对象的唯一梯度，可以用于指导调整模型的参数。
子步骤2105，根据所述调整梯度调整所述神经网络模型中所述候选对象对应的参数。
具体地，根据调整梯度的方向调整参数。
本公开实施例可以综合所有候选文本，计算得到调整梯度，并指导调整模型参数，可以准确调整模型参数，有助于提高模型的准确度。
步骤211，在每次训练之后，根据同一个搜索请求分组中被点击和未被点击的候选对象的当前预测值以及候选对象的位置标记计算损失值。
其中，位置标可以标记被点击的候选对象和未被点击的候选对象的前后位置，可以根据实际应用场景设定。例如，当候选对象i在候选对象j之前时，被点击的候选对象i和未被点击的候选对象j对应的位置标记为1；当候选对象i在候选对象j之后时，被点击的候选对象i和未被点击的候选对象j对应的位置标记为0。
在实际应用中，损失值用于确定训练是否结束。
可选地，在本公开的另一种实施例中，所述步骤211包括子步骤2111至2118：
子步骤2111，对于同一个搜索请求分组中被点击和未被点击的候选对象，计算1与候选对象的位置标记的差值，得到第二差值。
具体地，第二差值M2 i,j的计算公式如下：
M2 i,j＝1-S ij        (8)
其中，S ij为被点击的候选对象i和未被点击的候选对象j对应的位置标记。
子步骤2112，对于同一个搜索请求分组中被点击和未被点击的候选对象，计算被点击的候选对象和未被点击的候选对象的当前预测值的差值，得到第三差值。
具体地，第三差值M3 i,j的计算公式如下：
M3 i,j＝S i-S j     (9)
子步骤2113，计算所述第二差值、第三差值、预设系数以及二分之一的乘积，得到第三乘积。
具体地，第三乘积P3 i,j的计算公式如下：
Figure PCTCN2019120676-appb-000011
子步骤2114，计算所述第三差值和预设系数的乘积，并取相反数得到第四乘积。
具体地，第四乘积P4 i,j的计算公式如下：
P4 i,j＝-M3 i,j·σ＝-σ(S i-S j)       (11)
子步骤2115，以自然常数为底数，所述第四乘积为指数计算指数结果，得到第二指数结果。
具体地，第二指数结果I2 i,j的计算公式如下：
Figure PCTCN2019120676-appb-000012
子步骤2116，计算1与所述第二指数结果之和作为真数，并以10为底数取对数，得到对数结果。
具体地，对数结果L i,j的计算公式如下：
Figure PCTCN2019120676-appb-000013
子步骤2117，计算所述第三乘积和所述对数结果之和，得到被点击和未被点击的候选对象的损失值。
具体地，被点击和未被点击的候选对象i和j的损失值C i,j的计算公式如下：
Figure PCTCN2019120676-appb-000014
子步骤2118，计算所述被点击和未被点击的候选对象的损失值的平均值，得到损失值。
具体地，对于所有搜索请求下各种被点击和未被点击的候选对象的组合，计算总的平均值得到损失值。
步骤212，在所述损失值小于或等于预设损失值阈值的情况下，结束训练。
其中，损失值阈值可以根据实际应用场景设定，本公开实施例对其不加以限制。可以理解，损失值阈值过大时，训练得到的神经网络模型准确度较差，但训练时间较短；损失值阈值过小时，训练得到的神经网络模型准确度较高，但训练时间较长。在实际应用中可以根据需求设定损失值阈值。
本公开实施例可以将结束训练时使用当前参数的神经网络模型作为最终的神经网络模型，可以用于实际应用中。
步骤213，将训练得到的神经网络模型部署至应用平台，所述应用平台调用所述神经网络模型对目标搜索词关联的目标对象进行排序。
其中，应用平台可以为搜索推荐平台，在本公开实施例中应用平台采用Tensorflow框架。
具体地，可以将神经网络模型进行打包，并存储、安装在应用平台上，从而应用平台在接收到目标搜索词的时候，首先获取关联的多个目标对象；然后离线调用该神经网络模型对目标对象进行排序。
本公开实施例可以将预先训练的神经网络模型部署在应用平台，并离线调用以进行排序，实现了神经网络模型的灵活应用。
步骤214，采用所述神经网络模型对目标搜索词关联的目标对象进行排序。
该步骤可以参照步骤103的详细说明，在此不再赘述。
综上所述，本公开实施例提供了一种排序方法，所述方法包括：对数据样本集按照搜索请求进行分组，得到至少一个搜索请求分组；采用所述搜索请求分组对神经网络模型进行训练，所述神经网络模型在训练过程中，根据同一个搜索请求分组中被点击和未被点击的候选对象的当前预测值、以及被点击和未被点击的候选对象排序位置互换前后的归一化折扣累积增益的变化量，调整所述神经网络模型的参数；采用所述神经网络模型对目标搜索词关联的目标对象进行排序。可以结合归一化折扣累积增益调整神经网络模型，使得调整结果更能符合搜索推荐领域，有助于提高神经网络模型的准确度。
实施例三
参照图3，其示出了本公开的一种排序装置的结构图，具体如下。
样本集分组模块301，用于对数据样本集按照搜索请求进行分组，得到至少一个搜索请求分组。
神经网络模型训练模块302，用于采用所述搜索请求分组对神经网络模型进行训练，所述神经网络模型在训练过程中，根据同一个搜索请求分组中被点击和未被点击的候选对象的当前预测值、以及被点击和未被点击的候选对象排序位置互换前后的归一化折扣累积增益的变化量，调整所述神经网络模型的参数。
排序模块303，用于采用所述神经网络模型对目标搜索词关联的目标对象进行排序。
综上所述，本公开实施例提供了一种排序装置，所述装置包括：样本集分组模块，用于对数据样本集按照搜索请求进行分组，得到至少一个搜索请求分组；神经网络模型训练模块，用于采用所述搜索请求分组对神经网络模型进行训练，所述神经网络模型在训练过程中，根据同一个搜索请求分组中被点击和未被点击的候选对象的当前预测值、以及被点击和未被点击的候选对象排序位置互换前后的归一化折扣累积增益的变化量，调整所述神经网络模型的参数；排序模块，用于采用所述神经网络模型对目标搜索词关联的目标对象进行排序。可以结合归一化折扣累积增益调整神经网络模型，使得调整结果更能符合搜索推荐领域，有助于提高神经网络模型的准确度。
实施例四
参照图4，其示出了本公开的另一种排序装置的结构图，具体如下。
样本集分组模块401，用于对数据样本集按照搜索请求进行分组，得到至少一个搜索请求分组。
神经网络模型训练模块402，用于采用所述搜索请求分组对神经网络模型进行训练，所述神经网络模型在训练过程中，根据同一个搜索请求分组中被点击和未被点击的候选对象的当前预测值、以及被点击和未被点击的候选对象排序位置互换前后的归一化折扣累积增益的变化量，调整所述神经网络模型的参数；可选地，在本公开实施例中，所述神经网络模型训练模块402，包括：
增益计算子模块40201，用于对于同一个搜索请求分组中被点击和未被点击的候选对象，分别计算被点击的候选对象排序在未被点击的候选对象之前的归一化折扣累计增益，以及，被点击的候选对象排序在未被点击的候选对象之后的归一化折扣累计增益，得到第一增益和第二增益。
绝对值计算子模块40202，用于计算所述第一增益和第二增益之间的差值的绝对值。
第一差值计算子模块40203，用于计算被点击的候选对象与未被点击的候选对象的当前预测值之间的差值，得到第一差值。
第一乘积计算子模块40204，用于计算所述差值与预设系数的乘积，得到第一乘积。
第一指数结果计算子模块40205，用于以自然常数为底数，所述第一乘 积为指数计算指数结果，得到第一指数结果。
第一数值计算子模块40206，用于计算所述指数结果与1之和，得到第一数值。
第二乘积计算子模块40207，用于计算所述预设系数与所述绝对值的乘积，得到第二乘积。
梯度值计算子模块40208，用于计算所述第二乘积与所述第一数值的比值，并取相反数得到所述被点击的候选对象和未被点击的候选对象之间的梯度值。
模型参数调整子模块40209，用于根据所述被点击的候选对象和未被点击的候选对象之间的梯度值调整所述神经网络模型的参数。
损失值计算子模块40210，用于在每次训练之后，根据同一个搜索请求分组中被点击和未被点击的候选对象的当前预测值以及候选对象的位置标记计算损失值。
训练结束子模块40211，用于在所述损失值小于或等于预设损失值阈值的情况下，结束训练。
模型部署模块403，用于将训练得到的神经网络模型部署至应用平台，所述应用平台调用所述神经网络模型对目标搜索词关联的目标对象进行排序。
排序模块404，用于采用所述神经网络模型对目标搜索词关联的目标对象进行排序。
可选地，在本公开的另一种实施例中，所述被点击的候选对象和未被点击的候选对象之间的梯度值λ i,j根据如下公式计算得到：
Figure PCTCN2019120676-appb-000015
其中，σ为预设系数，S i、S j分别为被点击的候选对象与未被点击的候选对象的当前预测值，Δ NDCG为被点击和未被点击的候选对象排序位置互换前后的归一化折扣累积增益的变化值。
可选地，在本公开的另一种实施例中，所述模型参数调整子模块40209，包括：
对象获取单元，用于针对每个候选对象，分别获取标记在所述候选对象的位置之前的其他候选对象，以及，获取标记在所述候选对象的位置之后的 其他候选对象，得到第一对象、第二对象；
第一梯度和计算单元，用于计算所述候选对象与所述第一对象的梯度值之和，得到第一梯度和；
第二梯度和计算单元，用于计算所述候选对象与所述第二对象之间的梯度值之和，得到第二梯度和；
调整梯度计算单元，用于计算所述第二梯度和与第一梯度和的差值，得到所述候选对象的调整梯度；
模型参数调整单元，用于根据所述调整梯度调整所述神经网络模型中所述候选对象对应的参数。
可选地，在本公开的另一种实施例中，所述损失值计算子模块40210，包括：
第二差值计算单元，用于对于同一个搜索请求分组中被点击和未被点击的候选对象，计算1与候选对象的位置标记的差值，得到第二差值；
第三差值计算单元，用于对于同一个搜索请求分组中被点击和未被点击的候选对象，计算被点击的候选对象和未被点击的候选对象的当前预测值的差值，得到第三差值；
第三乘积计算单元，用于计算所述第二差值、第三差值、预设系数以及二分之一的乘积，得到第三乘积；
第四乘积计算单元，用于计算所述第三差值和预设系数的乘积，并取相反数得到第四乘积；
第二指数结果计算单元，用于以自然常数为底数，所述第四乘积为指数计算指数结果，得到第二指数结果；
对数结果计算单元，用于计算1与所述第二指数结果之和作为真数，并以10为底数取对数，得到对数结果；
第一损失值计算单元，用于计算所述第三乘积和所述对数结果之和，得到被点击和未被点击的候选对象的损失值；
第二损失值计算单元，用于计算所述被点击和未被点击的候选对象的损失值的平均值，得到损失值。
可选地，在本公开的另一种实施例中，所述被点击和未被点击的候选对象的损失值C i,j根据如下公式计算得到：
Figure PCTCN2019120676-appb-000016
其中，S ij为被点击的候选文本与未被点击的候选文本的标记值的差值。
综上所述，本公开实施例提供了一种排序装置，所述装置包括：样本集分组模块，用于对数据样本集按照搜索请求进行分组，得到至少一个搜索请求分组；神经网络模型训练模块，用于采用所述搜索请求分组对神经网络模型进行训练，所述神经网络模型在训练过程中，根据同一个搜索请求分组中被点击和未被点击的候选对象的当前预测值、以及被点击和未被点击的候选对象排序位置互换前后的归一化折扣累积增益的变化量，调整所述神经网络模型的参数；排序模块，用于采用所述神经网络模型对目标搜索词关联的目标对象进行排序。可以结合归一化折扣累积增益调整神经网络模型，使得调整结果更能符合搜索推荐领域，有助于提高神经网络模型的准确度。
本公开的实施例还提供了一种电子设备，包括：处理器、存储器以及存储在所述存储器上并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现前述实施例的排序方法。
本公开的实施例还提供了一种计算机程序，包括计算机可读代码，当所述计算机可读代码在计算处理设备上运行时，导致所述计算处理设备执行前述实施例的排序方法。
本公开的实施例还提供了一种计算机可读存储介质，其中存储了前述实施例的计算机程序。
以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。本领域普通技术人员在不付出创造性的劳动的情况下，即可以理解并实施。
本公开的各个部件实施例可以以硬件实现，或者以在一个或者多个处理器上运行的软件模块实现，或者以它们的组合实现。本领域的技术人员应当理解，可以在实践中使用微处理器或者数字信号处理器(DSP)来实现根据 本公开实施例的计算处理设备中的一些或者全部部件的一些或者全部功能。本公开还可以实现为用于执行这里所描述的方法的一部分或者全部的设备或者装置程序(例如，计算机程序和计算机程序产品)。这样的实现本公开的程序可以存储在计算机可读存储介质上，或者可以具有一个或者多个信号的形式。这样的信号可以从因特网网站上下载得到，或者在载体信号上提供，或者以任何其他形式提供。
例如，图5示出了可以实现根据本公开的方法的计算处理设备。该计算处理设备传统上包括处理器510和以存储器520形式的计算机程序产品或者计算机可读存储介质。存储器520可以是诸如闪存、EEPROM(电可擦除可编程只读存储器)、EPROM、硬盘或者ROM之类的电子存储器。存储器520具有用于执行上述方法中的任何方法步骤的程序代码531的存储空间530。例如，用于程序代码的存储空间530可以包括分别用于实现上面的方法中的各种步骤的各个程序代码531。这些程序代码可以从一个或者多个计算机程序产品中读出或者写入到这一个或者多个计算机程序产品中。这些计算机程序产品包括诸如硬盘，紧致盘(CD)、存储卡或者软盘之类的程序代码载体。这样的计算机程序产品通常为如参考图6所述的便携式或者固定存储单元。该存储单元可以具有与图5的计算处理设备中的存储器520类似布置的存储段、存储空间等。程序代码可以例如以适当形式进行压缩。通常，存储单元包括计算机可读代码531’，即可以由例如诸如510之类的处理器读取的代码，这些代码当由计算处理设备运行时，导致该计算处理设备执行上面所描述的方法中的各个步骤。
本文中所称的“一个实施例”、“实施例”或者“一个或者多个实施例”意味着，结合实施例描述的特定特征、结构或者特性包括在本公开的至少一个实施例中。此外，请注意，这里“在一个实施例中”的词语例子不一定全指同一个实施例。
在此处所提供的说明书中，说明了大量具体细节。然而，能够理解，本公开的实施例可以在没有这些具体细节的情况下被实践。在一些实例中，并未详细示出公知的方法、结构和技术，以便不模糊对本说明书的理解。
在权利要求中，不应将位于括号之间的任何参考符号构造成对权利要求 的限制。单词“包含”不排除存在未列在权利要求中的元件或步骤。位于元件之前的单词“一”或“一个”不排除存在多个这样的元件。本公开可以借助于包括有若干不同元件的硬件以及借助于适当编程的计算机来实现。在列举了若干装置的单元权利要求中，这些装置中的若干个可以是通过同一个硬件项来具体体现。单词第一、第二、以及第三等的使用不表示任何顺序。可将这些单词解释为名称。
最后应说明的是：以上实施例仅用以说明本公开的技术方案，而非对其限制；尽管参照前述实施例对本公开进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本公开各实施例技术方案的精神和范围。

Claims (11)

  1. 一种排序方法，包括：
    对数据样本集按照搜索请求进行分组，得到至少一个搜索请求分组；
    采用所述搜索请求分组对神经网络模型进行训练，所述神经网络模型在训练过程中，根据同一个搜索请求分组中被点击和未被点击的候选对象的当前预测值、以及被点击和未被点击的候选对象排序位置互换前后的归一化折扣累积增益的变化量，调整所述神经网络模型的参数；
    采用所述神经网络模型对目标搜索词关联的目标对象进行排序。
  2. 根据权利要求1所述的方法，所述根据同一个搜索请求分组中被点击和未被点击的候选对象的当前预测值、以及被点击和未被点击的候选对象排序位置互换前后的归一化折扣累积增益的变化量，调整所述神经网络模型的参数的步骤，包括：
    对于同一个搜索请求分组中被点击和未被点击的候选对象，分别计算被点击的候选对象排序在未被点击的候选对象之前的归一化折扣累计增益，以及，被点击的候选对象排序在未被点击的候选对象之后的归一化折扣累计增益，得到第一增益和第二增益；
    计算所述第一增益和第二增益之间的差值的绝对值；
    计算被点击的候选对象与未被点击的候选对象的当前预测值之间的差值，得到第一差值；
    计算所述差值与预设系数的乘积，得到第一乘积；
    以自然常数为底数，所述第一乘积为指数计算指数结果，得到第一指数结果；
    计算所述指数结果与1之和，得到第一数值；
    计算所述预设系数与所述绝对值的乘积，得到第二乘积；
    计算所述第二乘积与所述第一数值的比值，并取相反数得到所述被点击的候选对象和未被点击的候选对象之间的梯度值；
    根据所述被点击的候选对象和未被点击的候选对象之间的梯度值调整所述神经网络模型的参数。
  3. 根据权利要求2所述的方法，所述被点击的候选对象和未被点击的候选对象之间的梯度值λ i,j根据如下公式计算得到：
    Figure PCTCN2019120676-appb-100001
    其中，σ为预设系数，S i、S j分别为被点击的候选对象与未被点击的候选对象的当前预测值，Δ NDCG为被点击和未被点击的候选对象排序位置互换前后的归一化折扣累积增益的变化值。
  4. 根据权利要求2所述的方法，所述根据所述被点击的候选对象和未被点击的候选对象之间的梯度值调整所述神经网络模型的参数的步骤，包括：
    针对每个候选对象，分别获取标记在所述候选对象的位置之前的其他候选对象，以及，获取标记在所述候选对象的位置之后的其他候选对象，得到第一对象、第二对象；
    计算所述候选对象与所述第一对象的梯度值之和，得到第一梯度和；
    计算所述候选对象与所述第二对象之间的梯度值之和，得到第二梯度和；
    计算所述第二梯度和与第一梯度和的差值，得到所述候选对象的调整梯度；
    根据所述调整梯度调整所述神经网络模型中所述候选对象对应的参数。
  5. 根据权利要求1所述的方法，所述方法还包括：
    在每次训练之后，根据同一个搜索请求分组中被点击和未被点击的候选对象的当前预测值以及候选对象的位置标记计算损失值；
    在所述损失值小于或等于预设损失值阈值的情况下，结束训练。
  6. 根据权利要求5所述的方法，所述根据同一个搜索请求分组中被点击和未被点击的候选对象的当前预测值以及候选对象的位置标记计算损失值的步骤，包括：
    对于同一个搜索请求分组中被点击和未被点击的候选对象，计算1与候选对象的位置标记的差值，得到第二差值；
    对于同一个搜索请求分组中被点击和未被点击的候选对象，计算被点击的候选对象和未被点击的候选对象的当前预测值的差值，得到第三差值；
    计算所述第二差值、第三差值、预设系数以及二分之一的乘积，得到第三乘积；
    计算所述第三差值和预设系数的乘积，并取相反数得到第四乘积；
    以自然常数为底数，所述第四乘积为指数计算指数结果，得到第二指数结果；
    计算1与所述第二指数结果之和作为真数，并以10为底数取对数，得到对数结果；
    计算所述第三乘积和所述对数结果之和，得到被点击和未被点击的候选对象的损失值；
    计算所述被点击和未被点击的候选对象的损失值的平均值，得到损失值。
  7. 根据权利要求5所述的方法，所述被点击和未被点击的候选对象的损失值C i,j根据如下公式计算得到：
    Figure PCTCN2019120676-appb-100002
    其中，S ij为被点击的候选文本与未被点击的候选文本的标记值的差值。
  8. 根据权利要求1所述的方法，在所述采用所述神经网络模型对目标搜索词关联的目标对象进行排序之前，还包括：
    将训练得到的神经网络模型部署至应用平台，所述应用平台调用所述神经网络模型对目标搜索词关联的目标对象进行排序。
  9. 一种排序装置，包括：
    样本集分组模块，用于对数据样本集按照搜索请求进行分组，得到至少一个搜索请求分组；
    神经网络模型训练模块，用于采用所述搜索请求分组对神经网络模型进行训练，所述神经网络模型在训练过程中，根据同一个搜索请求分组中被点击和未被点击的候选对象的当前预测值、以及被点击和未被点击的候选对象排序位置互换前后的归一化折扣累积增益的变化量，调整所述神经网络模型的参数；
    排序模块，用于采用所述神经网络模型对目标搜索词关联的目标对象进行排序。
  10. 一种计算机程序，包括计算机可读代码，当所述计算机可读代码在计算处理设备上运行时，导致所述计算处理设备执行根据权利要求1至8中的任一个所述的排序方法。
  11. 一种计算机可读存储介质，其中存储了如权利要求10所述的计算机程序。
PCT/CN2019/120676 2019-01-10 2019-11-25 排序 Ceased WO2020143346A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
KR1020217019287A KR20210090706A (ko) 2019-01-10 2019-11-25 정렬
US17/370,084 US20210374149A1 (en) 2019-01-10 2021-07-08 Sorting

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN201910024150 2019-01-10
CN201910024150.9 2019-01-10
CN201910191098.6 2019-03-12
CN201910191098.6A CN111428125B (zh) 2019-01-10 2019-03-12 排序方法、装置、电子设备及可读存储介质

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/370,084 Continuation US20210374149A1 (en) 2019-01-10 2021-07-08 Sorting

Publications (1)

Publication Number Publication Date
WO2020143346A1 true WO2020143346A1 (zh) 2020-07-16

Family

ID=71520716

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/120676 Ceased WO2020143346A1 (zh) 2019-01-10 2019-11-25 排序

Country Status (1)

Country Link
WO (1) WO2020143346A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117195957A (zh) * 2022-05-26 2023-12-08 腾讯科技（深圳）有限公司 预测模型的确定方法和相关装置

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150363688A1 (en) * 2014-06-13 2015-12-17 Microsoft Corporation Modeling interestingness with deep neural networks
CN106599577A (zh) * 2016-12-13 2017-04-26 重庆邮电大学 一种结合rbm和特征选择的列表级排序学习方法
CN107679082A (zh) * 2017-08-31 2018-02-09 阿里巴巴集团控股有限公司 问答搜索方法、装置以及电子设备
CN108121814A (zh) * 2017-12-28 2018-06-05 北京百度网讯科技有限公司 搜索结果排序模型生成方法和装置
CN109033140A (zh) * 2018-06-08 2018-12-18 北京百度网讯科技有限公司 一种确定搜索结果的方法、装置、设备和计算机存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150363688A1 (en) * 2014-06-13 2015-12-17 Microsoft Corporation Modeling interestingness with deep neural networks
CN106599577A (zh) * 2016-12-13 2017-04-26 重庆邮电大学 一种结合rbm和特征选择的列表级排序学习方法
CN107679082A (zh) * 2017-08-31 2018-02-09 阿里巴巴集团控股有限公司 问答搜索方法、装置以及电子设备
CN108121814A (zh) * 2017-12-28 2018-06-05 北京百度网讯科技有限公司 搜索结果排序模型生成方法和装置
CN109033140A (zh) * 2018-06-08 2018-12-18 北京百度网讯科技有限公司 一种确定搜索结果的方法、装置、设备和计算机存储介质

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117195957A (zh) * 2022-05-26 2023-12-08 腾讯科技（深圳）有限公司 预测模型的确定方法和相关装置

Similar Documents

Publication Publication Date Title
CN111428125B (zh) 排序方法、装置、电子设备及可读存储介质
TWI689871B (zh) 梯度提升決策樹（ｇｂｄｔ）模型的特徵解釋方法和裝置
JP6964689B2 (ja) サンプル重み設定方法及び装置、電子装置
CN113987161B (zh) 一种文本排序方法及装置
CN110597878B (zh) 一种多模态数据的跨模态检索方法、装置、设备及介质
CN109903086B (zh) 一种相似人群扩展方法、装置及电子设备
CN111931055B (zh) 对象推荐方法、对象推荐装置和电子设备
CN108322317B (zh) 一种账号识别关联方法及服务器
CN109783727B (zh) 检索推荐方法、装置、计算机可读存储介质及电子设备
CN108345702A (zh) 实体推荐方法和装置
CN114528391B (zh) 问答对评分模型的训练方法、装置、设备及存储介质
CN108491540B (zh) 文本信息推送方法、装置及智能终端
CN104750798B (zh) 一种应用程序的推荐方法和装置
CN105894183B (zh) 项目评价方法及装置
JP2014515514A (ja) 提案される語を提供するための方法および装置
WO2018157625A1 (zh) 基于强化学习的排序学习方法及服务器
CN112100482A (zh) 搜索结果排序方法、装置、电子设备及存储介质
CN111126262A (zh) 基于图神经网络的视频精彩片段检测方法及装置
CN114090891A (zh) 个性化内容推荐方法、装置、设备及存储介质
WO2022053018A1 (zh) 一种文本聚类系统、方法、装置、设备及介质
CN107577736A (zh) 一种基于bp神经网络的文件推荐方法及系统
CN113407816A (zh) 对象推荐方法、装置、电子设备及存储介质
WO2020143346A1 (zh) 排序
CN113282831B (zh) 一种搜索信息的推荐方法、装置、电子设备及存储介质
CN111831892A (zh) 信息推荐方法、信息推荐装置、服务器及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19908619

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 20217019287

Country of ref document: KR

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19908619

Country of ref document: EP

Kind code of ref document: A1