CN105718467B - Retrieval algorithm evaluation recommendation method and system - Google Patents

Retrieval algorithm evaluation recommendation method and system Download PDF

Info

Publication number
CN105718467B
CN105718467B CN201410724760.7A CN201410724760A CN105718467B CN 105718467 B CN105718467 B CN 105718467B CN 201410724760 A CN201410724760 A CN 201410724760A CN 105718467 B CN105718467 B CN 105718467B
Authority
CN
China
Prior art keywords
list
retrieval
records
algorithm
evaluation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201410724760.7A
Other languages
Chinese (zh)
Other versions
CN105718467A (en
Inventor
方宁
张侦
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NANJING SUNING ELECTRONIC INFORMATION TECHNOLOGY Co.,Ltd.
Shenzhen yunwangwandian Technology Co.,Ltd.
Original Assignee
Suning Cloud Computing Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Suning Cloud Computing Co Ltd filed Critical Suning Cloud Computing Co Ltd
Priority to CN201410724760.7A priority Critical patent/CN105718467B/en
Publication of CN105718467A publication Critical patent/CN105718467A/en
Application granted granted Critical
Publication of CN105718467B publication Critical patent/CN105718467B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention provides a retrieval algorithm evaluation recommendation method and system, and belongs to the technical field of internet. The method comprises the following steps: s1, determining an evaluation condition according to the retrieval background information; s2, evaluating each retrieval algorithm according to the evaluation condition; and S3, obtaining an optimal retrieval algorithm to carry out retrieval calculation to obtain a retrieval result. According to the technical scheme, the optimal retrieval algorithm is deduced by evaluating each retrieval algorithm, so that a user can obtain better retrieval experience.

Description

Retrieval algorithm evaluation recommendation method and system
Technical Field
The invention relates to the technical field of internet, in particular to a retrieval algorithm evaluation recommendation method and system.
Background
The current criteria for evaluating data mining algorithms for the internet are based on recall and accuracy, or a combination of both. The Recall Rate (Recall Rate, also called Recall Rate) is the ratio of the number of retrieved records to the number of all relevant records in the record library, and the Recall Rate of the retrieval system is measured; the precision is the ratio of the number of relevant records retrieved to the total number of records retrieved, and the precision is measured by the precision of the retrieval system. The evaluation of the ordered queue involves an average sorting score of sorting accuracy or a weighted average sorting score, and the evaluation mode is complex and inaccurate.
Disclosure of Invention
In view of the above-mentioned drawbacks of the prior art, the technical problem to be solved by the present invention is how to derive an optimal search algorithm so as to enable a user to obtain a better search experience.
In order to achieve the above object, in one aspect, the present invention provides a method for evaluating and recommending a search algorithm, including the steps of:
s1, determining an evaluation condition according to the retrieval background information;
s2, evaluating each retrieval algorithm according to the evaluation condition;
and S3, acquiring an optimal retrieval algorithm to perform retrieval calculation, and obtaining a retrieval result.
Preferably, in step S2, the result ranking within a certain time is counted according to the search condition to obtain a real result, the search result is obtained according to each search result algorithm, the search result of each search algorithm is compared with the real result, and the search algorithm with the search result closest to the real result is analyzed.
Preferably, in step S2, a first list is obtained according to a search result, a second list is obtained according to a real result, the first list includes a plurality of records, the second list includes a plurality of records, the first list and the second list are compared, and a search algorithm with a search result closest to the real result is analyzed according to the records only in the first list, the records only in the second list and the records both in the first list and the second list.
Preferably, in step S2, the prediction score RS is calculated by the formula:
adding records in the second list, which do not appear in the first list, to the first list to obtain a third list;
defining a prediction score RS, and obtaining an optimal retrieval algorithm according to the prediction score RS, wherein the calculation formula of the detection score RS is as follows:
Figure BDA0000623699120000021
wherein k is the number of records in the second list; m is the number of records in the third list; h isjScore for the jth record in the third list, tjScores for records that are present in both the first list and the second list; f. ofjA score for a record that exists in the first list but not in the second list; i.e. ijScores for records that are not present in the first list but are present in the second list.
Preferably, said tjAn absolute value equal to a difference value recorded at the sorting positions of the first list and the second list is multiplied by a coefficient, which is confirmed in accordance with the evaluation condition.
Preferably, said fjA (m +1), the said ijB (m +1), wherein A, B was confirmed according to the evaluation conditions, respectively.
In another aspect, the present invention provides a search algorithm evaluation recommendation system, including:
the determining module is used for determining an evaluation condition according to the retrieval background information;
the evaluation module is used for evaluating each retrieval algorithm according to the evaluation conditions;
and the recommendation module is used for acquiring the optimal retrieval algorithm to perform retrieval calculation and obtaining a retrieval result.
Preferably, the evaluation module counts result ranking within a certain time according to the retrieval conditions to obtain real results, respectively obtains retrieval results according to each retrieval result algorithm,
the method comprises the steps of obtaining a first list according to a retrieval result, obtaining a second list according to a real result, wherein the first list comprises a plurality of records, the second list comprises a plurality of records, comparing the first list with the second list, and analyzing a retrieval algorithm with the retrieval result closest to the real result according to the records only existing in the first list, the records only existing in the second list and the records simultaneously existing in the first list and the second list.
Preferably, the evaluation module adds the record in the second list, which is not present in the first list, to the first list to obtain a third list; defining a prediction score RS, and obtaining an optimal retrieval algorithm according to the prediction score RS, wherein the calculation formula of the detection score RS is as follows:
Figure BDA0000623699120000031
wherein k is the number of records in the second list; m is the number of records in the third list; h isjScore for the jth record in the third list; t is tjScores for records that are present in both the first list and the second list; f. ofjA score for a record that exists in the first list but not in the second list; i.e. ijScores for records that are not present in the first list but are present in the second list.
Preferably, said tjMultiplying an absolute value of a difference value corresponding to the sorting positions recorded in the first list and the second list by a coefficient, the coefficient being confirmed according to the evaluation condition; f isjA (m +1), the said ijB (m +1), wherein A, B was confirmed according to the evaluation conditions, respectively.
The invention provides an evaluation method and a system of a mining algorithm with a prediction ordered queue as output, and the application scene of the method can be used in the internet fields of recommendation and prediction of evaluation commodities and the like or other non-internet fields with similar measurement requirements; in the technical scheme, the advantages and disadvantages of the prediction algorithm are comprehensively evaluated by using the size of a numerical value, and an optimal retrieval algorithm is developed, so that a user can obtain better retrieval experience; while reducing the amount of server access.
Drawings
FIG. 1 is a schematic flow chart of a method for evaluating recommendation by a search algorithm according to a first embodiment of the present invention;
fig. 2 is a schematic structural diagram of a retrieval algorithm evaluation recommendation system in the second embodiment of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the present invention will be described in further detail with reference to the accompanying drawings and specific embodiments.
Fig. 1 is a schematic flow chart of a method for evaluating and recommending a search algorithm according to a first embodiment of the present invention, as shown in fig. 1, the method for evaluating and recommending a search algorithm includes the steps of:
s1, determining an evaluation condition according to the retrieval background information;
s2, evaluating each retrieval algorithm according to the evaluation condition;
and S3, acquiring an optimal retrieval algorithm to perform retrieval calculation, and obtaining a retrieval result.
Preferably, in step S2, the result ranking within a certain time is counted according to the search condition to obtain a real result, the search result is obtained according to each search result algorithm, the search result of each search algorithm is compared with the real result, and the search algorithm with the search result closest to the real result is analyzed.
Preferably, in step S2, a first list is obtained according to a search result, a second list is obtained according to a real result, the first list includes a plurality of records, the second list includes a plurality of records, the first list and the second list are compared, and a search algorithm with a search result closest to the real result is analyzed according to the records only in the first list, the records only in the second list and the records both in the first list and the second list.
Preferably, in step S2, a first list is obtained according to a search result, a second list is obtained according to a real result, the first list includes a plurality of records, the second list includes a plurality of records, the first list and the second list are compared, and a search algorithm with a search result closest to the real result is analyzed according to the records only in the first list, the records only in the second list and the records both in the first list and the second list.
Preferably, t isjAn absolute value equal to a difference value recorded at the sorting positions of the first list and the second list is multiplied by a coefficient, which is confirmed in accordance with the evaluation condition.
Preferably, fj=A(m+1),ijB (m +1), wherein A, B was confirmed according to the evaluation conditions, respectively.
Those skilled in the art will appreciate that, in accordance with the method of the present invention, the present invention also includes a search algorithm evaluation recommendation system, which includes, in one-to-one correspondence with the above method steps, as shown in fig. 2: a determining module 201, configured to determine an evaluation condition according to the retrieval background information; an evaluation module 202, configured to evaluate each retrieval algorithm according to an evaluation condition; and the recommending module 203 is used for acquiring an optimal retrieval algorithm to perform retrieval calculation and obtain a retrieval result.
Preferably, the evaluation module 202 counts result ranking within a certain time according to the retrieval conditions to obtain a real result, obtains the retrieval results according to the retrieval result algorithms respectively,
the method comprises the steps of obtaining a first list according to a retrieval result, obtaining a second list according to a real result, wherein the first list comprises a plurality of records, the second list comprises a plurality of records, comparing the first list with the second list, and analyzing a retrieval algorithm with the retrieval result closest to the real result according to the records only existing in the first list, the records only existing in the second list and the records simultaneously existing in the first list and the second list.
Preferably, the evaluation module 202 adds the record in the second list that does not appear in the first list to obtain a third list; defining a prediction score RS, and obtaining an optimal retrieval algorithm according to the prediction score RS, wherein the calculation formula of the detection score RS is as follows:
Figure BDA0000623699120000051
wherein k is the number of records in the second list; m is the number of records in the third list; h isjScore for the jth record in the third list; t is tjScores for records that are present in both the first list and the second list; f. ofjA score for a record that exists in the first list but not in the second list; i.e. ijA score for a record that is not present in the first list but is present in the second list; t is tjMultiplying an absolute value of a difference value corresponding to the sorting positions recorded in the first list and the second list by a coefficient, the coefficient being confirmed according to the evaluation condition; f isjA (m +1), the said ijB (m +1), wherein A, B was confirmed according to the evaluation conditions, respectively.
Wherein, the smaller the value of RS is, the better, if the prediction result is completely correct, the value of RS is 0; k is the number of commodities in the real commodity queue; m is all commodities in the prediction queue and the real queue; h isjIs the ranking score of the jth good, in three cases: correctly predicted as tjWrong prediction is noted as fjNot predicted is denoted as ij;tjIndicating that the jth commodity is a correctly predicted commodity which appears in both queues simultaneously, and the jth commodity is equal to the absolute value of the difference between the sorting position of the commodity in the predicted queue and the sorting position of the commodity in the real queue; f. ofjIndicating that the jth good is not in the real queue but is present in the predicted queue, fj=A(m+1);ijIndicating that the jth good is not in the prediction queue but is present in the real queue, ijB (m + 1). The coefficient A, B is determined according to the evaluation condition.
If A is 1; b is 1;
assuming that the real goods queue is T ═ (B, D, C, E), and the predicted goods queue is P ═ (a, B, C); for queues T and P, then k is 4; m-5 (commercial products a, B, C, D, E); h is16 (article a was misjudged); h is21 (position of article B in P)Is 2, position in T is 1); h is30 (commodity C is 3 in T and 3 in P); h is46 (item D was not predicted); h is56 (item E was not predicted); RS 19/4 4.75.
Assuming that the real goods queue is T ═ D, F, E, and the predicted goods queue is P ═ E, C, F; then k is 3; m-4 (commercial products E, C, F, D); h is12 (article E is 1 in P and 3 in T); h is25 (article C misjudged); h is31 (commodity F is 3 in P and 2 in T); h is44 (item D was not predicted); RS 12/3 4.
Finally, a better retrieval method is introduced to provide retrieval recommendation. The application scene of the invention can be used in the Internet field of the recommendation and prediction of the evaluation commodity and the like, or other non-Internet fields with similar measurement requirements. In the technical scheme, the advantages and disadvantages of the prediction algorithm are comprehensively evaluated by using the size of a numerical value, and an optimal retrieval algorithm is developed, so that a user can obtain better retrieval experience; while reducing the amount of server access.
It will be understood that the above embodiments are merely exemplary embodiments taken to illustrate the principles of the present invention, which is not limited thereto. It will be apparent to those skilled in the art that various modifications and improvements can be made without departing from the spirit and substance of the invention, and these modifications and improvements are also considered to be within the scope of the invention.

Claims (4)

1. A retrieval algorithm evaluation recommendation method is characterized by comprising the following steps:
s1, determining an evaluation condition according to the retrieval background information;
s2, evaluating each retrieval algorithm according to the evaluation condition;
in step S2, counting result ranks within a certain time according to the search conditions to obtain real results, obtaining search results according to each search algorithm, comparing the search results of each search algorithm with the real results, and analyzing the search algorithm with the search results closest to the real results;
the method specifically comprises the following steps: obtaining a first list according to a retrieval result, obtaining a second list according to a real result, wherein the first list comprises a plurality of records, the second list comprises a plurality of records, comparing the first list with the second list, and analyzing a retrieval algorithm with the retrieval result closest to the real result according to the records only existing in the first list, the records only existing in the second list and the records simultaneously existing in the first list and the second list;
further: adding records in the second list, which do not appear in the first list, to the first list to obtain a third list;
defining a prediction score RS, and obtaining an optimal retrieval algorithm according to the prediction score RS, wherein the calculation formula of the prediction score RS is as follows:
Figure FDA0002435383830000011
wherein k is the number of records in the second list; m is the number of records in the third list; h isjScore for the jth record in the third list, tjScores for records that are present in both the first list and the second list; f. ofjA score for a record that exists in the first list but not in the second list; i.e. ijA score for a record that is not present in the first list but is present in the second list; said t isjMultiplying an absolute value of a difference value corresponding to the sorting positions recorded in the first list and the second list by a coefficient, the coefficient being confirmed according to the evaluation condition;
and S3, acquiring an optimal retrieval algorithm to perform retrieval calculation, and obtaining a retrieval result.
2. The search algorithm evaluation recommendation method of claim 1,
f isjA (m +1), the said ijB (m +1), wherein A, B was confirmed according to the evaluation conditions, respectively.
3. A search algorithm evaluation recommendation system, comprising:
the determining module is used for determining an evaluation condition according to the retrieval background information;
the evaluation module is used for evaluating each retrieval algorithm according to the evaluation conditions;
the evaluation module counts result ranking within a certain time according to the retrieval conditions to obtain real results, respectively obtains retrieval results according to each retrieval algorithm,
obtaining a first list according to a retrieval result, obtaining a second list according to a real result, wherein the first list comprises a plurality of records, the second list comprises a plurality of records, comparing the first list with the second list, and analyzing a retrieval algorithm with the retrieval result closest to the real result according to the records only existing in the first list, the records only existing in the second list and the records simultaneously existing in the first list and the second list;
further: adding records in the second list, which do not appear in the first list, to the first list to obtain a third list; defining a prediction score RS, and obtaining an optimal retrieval algorithm according to the prediction score RS, wherein the calculation formula of the prediction score RS is as follows:
Figure FDA0002435383830000021
wherein k is the number of records in the second list; m is the number of records in the third list; h isjScore for the jth record in the third list; t is tjScores for records that are present in both the first list and the second list; f. ofjA score for a record that exists in the first list but not in the second list; i.e. ijA score for a record that is not present in the first list but is present in the second list; said t isjMultiplying an absolute value of a difference value corresponding to the sorting positions recorded in the first list and the second list by a coefficient, the coefficient being confirmed according to the evaluation condition;
and the recommendation module is used for acquiring the optimal retrieval algorithm to perform retrieval calculation and obtaining a retrieval result.
4. The search algorithm evaluation recommendation system according to claim 3, wherein f isjA (m +1), the said ijB (m +1), wherein A, B was confirmed according to the evaluation conditions, respectively.
CN201410724760.7A 2014-12-03 2014-12-03 Retrieval algorithm evaluation recommendation method and system Active CN105718467B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201410724760.7A CN105718467B (en) 2014-12-03 2014-12-03 Retrieval algorithm evaluation recommendation method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201410724760.7A CN105718467B (en) 2014-12-03 2014-12-03 Retrieval algorithm evaluation recommendation method and system

Publications (2)

Publication Number Publication Date
CN105718467A CN105718467A (en) 2016-06-29
CN105718467B true CN105718467B (en) 2020-07-21

Family

ID=56145599

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201410724760.7A Active CN105718467B (en) 2014-12-03 2014-12-03 Retrieval algorithm evaluation recommendation method and system

Country Status (1)

Country Link
CN (1) CN105718467B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109005431B (en) * 2018-09-18 2019-11-12 北京腾信创新网络营销技术股份有限公司 A kind of video evaluations recommender system

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4504441B2 (en) * 2008-06-27 2010-07-14 株式会社トヨタIt開発センター Route search apparatus and route search method
US20140324899A1 (en) * 2013-04-24 2014-10-30 Kitchology Inc. Enhanced food information management and presentation on a selective dynamic basis and associated services
CN102999589B (en) * 2012-11-15 2017-04-19 Tcl集团股份有限公司 Method and system for recommending order
CN103853789A (en) * 2012-12-06 2014-06-11 深圳市点行天下科技有限公司 Method and equipment used for recommending information to user
CN103593411A (en) * 2013-10-23 2014-02-19 江苏大学 Method for testing combination properties of evaluation indexes of search engines and testing device

Also Published As

Publication number Publication date
CN105718467A (en) 2016-06-29

Similar Documents

Publication Publication Date Title
Yin et al. Optimal linear combinations of multiple diagnostic biomarkers based on Youden index
CN104899246B (en) Collaborative filtering recommending method based on blurring mechanism user scoring neighborhood information
RU2017107022A (en) REMOTE CONTENT DETECTION
Mahmoud et al. The performance of multivariate CUSUM control charts with estimated parameters
Olewuezi Note on the comparison of some outlier labeling techniques
CN110008247B (en) Method, device and equipment for determining abnormal source and computer readable storage medium
US20220084102A1 (en) Commodity recommendation method, server, shopping cart and shopping system
US20200219022A1 (en) Method and apparatus for determining similarity between user and merchant, and electronic device
Partington et al. Optimizing data collection for public health decisions: a data mining approach
CN112487283A (en) Method and device for training model, electronic equipment and readable storage medium
Coucke et al. Comparison of different approaches to evaluate External Quality Assessment Data
CN111126119A (en) Method and device for counting user behaviors arriving at store based on face recognition
JP2016177583A (en) Traffic line processing system and traffic line processing method
CN105718467B (en) Retrieval algorithm evaluation recommendation method and system
CN103678709B (en) Recommendation system attack detection method based on time series data
JP6904331B2 (en) Factor analyzers, factor analysis methods, and programs
JP6164993B2 (en) Evaluation system, evaluation program, and evaluation method
KR101879829B1 (en) Method and device for detecting frauds by using click log data
CN106096029B (en) Recommendation method based on user bidirectional relationship
Ohkubo et al. Anomaly detection in high-dimensional data with the Mahalanobis–Taguchi system
CN108681947B (en) Collaborative filtering recommendation method based on time relevance and coverage of articles
Supola et al. The role of secondary assists in basketball–an analysis of its characteristics and effect on scoring
KR20140079639A (en) Method for selecting similar users for collaborative filtering based on earth mover´s distance
KR101689499B1 (en) New thesis recommendation method and new thesis recommendation system for performing the method
WO2017219317A1 (en) Information pushing method and device based on search content

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
TA01 Transfer of patent application right

Effective date of registration: 20200608

Address after: No. 1-1 Suning Avenue, Xuzhuang Software Park, Xuanwu District, Nanjing City, Jiangsu Province, 210000

Applicant after: Suning Cloud Computing Co.,Ltd.

Address before: 210042 Jiangsu, Xuanwu District, Nanjing, Nanjing Road, No. 1, building No. 15

Applicant before: SUNING COMMERCE GROUP Co.,Ltd.

TA01 Transfer of patent application right
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right

Effective date of registration: 20210506

Address after: 518001 unit 3510-130, Luohu business center, 2028 Shennan East Road, Chengdong community, Dongmen street, Luohu District, Shenzhen City, Guangdong Province

Patentee after: Shenzhen yunwangwandian Technology Co.,Ltd.

Patentee after: NANJING SUNING ELECTRONIC INFORMATION TECHNOLOGY Co.,Ltd.

Address before: No.1-1 Suning Avenue, Xuzhuang Software Park, Xuanwu District, Nanjing, Jiangsu Province, 210000

Patentee before: Suning Cloud Computing Co.,Ltd.

TR01 Transfer of patent right