CN110196796B - Effect evaluation method and device for recommendation algorithm - Google Patents
Effect evaluation method and device for recommendation algorithm Download PDFInfo
- Publication number
- CN110196796B CN110196796B CN201910408209.4A CN201910408209A CN110196796B CN 110196796 B CN110196796 B CN 110196796B CN 201910408209 A CN201910408209 A CN 201910408209A CN 110196796 B CN110196796 B CN 110196796B
- Authority
- CN
- China
- Prior art keywords
- recommendation
- target user
- results
- recommended
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3452—Performance evaluation by statistical analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9535—Search customisation based on user profiles and personalisation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Probability & Statistics with Applications (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The disclosure relates to an effect evaluation method and device of a recommendation algorithm. The method comprises the following steps: acquiring recommendation data fed back continuously and repeatedly based on query information provided by a target user, wherein the recommendation data comprises a plurality of recommendation results ordered according to the association degree; n relevance degrees corresponding to N recommendation results fed back each time are obtained, and summation calculation is carried out on the N relevance degrees so as to obtain a plurality of continuous summation results; and drawing a descending curve pointing to the target user based on the continuous multiple addition results so as to determine the effect of the recommendation algorithm on the target user according to the descending curve. The quality change trend of the recommended effect can be well reflected by the method.
Description
Technical Field
The disclosure relates to the technical field of data processing, and in particular relates to an effect evaluation method and device of a recommendation algorithm.
Background
In the development process of the recommendation algorithm, the problem of evaluating the output result of the recommendation algorithm exists. The currently commonly used evaluation method is manual evaluation. By way of example, the evaluating personnel refreshes a recommendation interface, then scores the correlation degree of the first ten or the first twenty recommendation results, and finally selects the result with low correlation degree and feeds the result back to the developer. However, the evaluation method is limited to screening out the results which do not meet the standard, and cannot well reflect the quality change trend of the recommended effect.
Disclosure of Invention
In order to overcome the problems in the related art, the embodiment of the disclosure provides an effect evaluation method and device of a recommendation algorithm. The technical scheme is as follows:
according to a first aspect of an embodiment of the present disclosure, there is provided an effect evaluation method of a recommendation algorithm, including:
acquiring recommendation data fed back continuously and repeatedly based on query information provided by a target user, wherein the recommendation data comprises a plurality of recommendation results ordered according to the association degree;
n relevance degrees corresponding to N recommendation results fed back each time are obtained, and summation calculation is carried out on the N relevance degrees so as to obtain a plurality of continuous summation results;
and drawing a descending curve pointing to the target user based on the continuous multiple addition results so as to determine the effect of the recommendation algorithm on the target user according to the descending curve.
In one embodiment, obtaining recommendation data that is fed back multiple times in succession based on query information provided by a target user includes:
acquiring a current query result returned based on query information provided by the target user in the current query, wherein the current query result is displayed as the recommendation data comprising N recommendation results;
when the current query result is a query result which is not acquired for the first time, judging whether the sum of the relevancy of N recommended results in the recommended data is larger than or equal to a first threshold value;
and when the sum of the relevancy of the N recommended results is greater than or equal to the first threshold, continuously acquiring the query result based on the preset information provided by the target user until the sum of the relevancy of the N recommended results is smaller than the first threshold.
In one embodiment, obtaining recommendation data that is fed back multiple times in succession based on query information provided by a target user includes:
acquiring a current query result returned based on query information provided by the target user in the current query, wherein the current query result is displayed as the recommendation data comprising N recommendation results;
when the current query result is a query result which is not acquired for the first time, judging whether the association degree of each recommended result in the recommended data is larger than or equal to a second threshold value;
and when the association degree of each recommended result is larger than or equal to the second threshold value, continuously acquiring the query result based on the preset information provided by the target user until the association degree of any recommended result is smaller than the second threshold value.
In one embodiment, drawing a decline curve directed to the target user based on the successive plurality of summation results includes:
and drawing a descending curve pointing to the target user based on the continuous multiple addition results and the feedback times, and acquiring slope change data of the descending curve.
In one embodiment, the method further comprises:
and drawing an overall descent curve according to a plurality of descent curves pointing to users, so as to determine the effect of the recommendation algorithm on the plurality of users according to the overall descent curve.
According to a second aspect of the embodiments of the present disclosure, there is provided an effect evaluation apparatus of a recommendation algorithm, including:
the data acquisition module is used for acquiring recommendation data which is fed back continuously and repeatedly based on query information provided by a target user, wherein the recommendation data comprises a plurality of recommendation results which are ordered according to the association degree;
the relevance calculating module is used for acquiring N relevance corresponding to N recommendation results fed back each time, and carrying out addition calculation on the N relevance to obtain a plurality of continuous addition results;
and the first drawing module is used for drawing a descending curve pointing to the target user based on the continuous multiple addition results so as to determine the effect of the recommendation algorithm on the target user according to the descending curve.
In one embodiment, the data acquisition module includes:
the first acquisition unit is used for acquiring a current query result returned based on query information provided by the target user when the current query is performed, and the current query result is displayed as the recommendation data comprising N recommendation results;
the first judging unit is used for judging whether the sum of the relevancy of N recommended results in the recommended data is greater than or equal to a first threshold value when the current query result is a query result which is not acquired for the first time;
and the first limiting unit is used for continuously acquiring the query result based on the preset information provided by the target user when the sum of the relevancy of the N recommended results is greater than or equal to the first threshold value until the sum of the relevancy of the N recommended results is smaller than the first threshold value.
In one embodiment, the data acquisition module includes:
the second acquisition unit is used for acquiring a current query result returned based on the query information provided by the target user when the current query is performed, and the current query result is displayed as the recommendation data comprising N recommendation results;
the second judging unit is used for judging whether the association degree of each recommended result in the recommended data is greater than or equal to a second threshold value when the current query result is a query result which is not acquired for the first time;
and the second limiting unit is used for continuously acquiring the query result based on the preset information provided by the target user when the association degree of each recommended result is larger than or equal to the second threshold value until the association degree of any recommended result is smaller than the second threshold value.
In one embodiment, the first drawing module includes:
and the drawing unit is used for drawing a descending curve pointing to the target user based on the continuous multiple addition results and the feedback times, and acquiring slope change data of the descending curve.
In one embodiment, the apparatus further comprises:
and the second drawing module is used for drawing an overall descent curve according to a plurality of descent curves pointing to the users so as to determine the effect of the recommendation algorithm on the plurality of users according to the overall descent curve.
According to a third aspect of the embodiments of the present disclosure, there is provided an effect evaluation apparatus of a recommendation algorithm, including:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to perform the steps of the method according to any embodiment of the first aspect.
According to a fourth aspect of embodiments of the present disclosure, there is provided a computer readable storage medium having stored thereon computer instructions which, when executed by a processor, implement the steps of the method of any of the embodiments of the first aspect.
The technical scheme provided by the embodiment of the disclosure can comprise the following beneficial effects:
according to the technical scheme, the recommendation data fed back continuously and repeatedly can be obtained according to the query information provided by the target user, and the descending curve pointing to the target user is drawn based on the addition result of the relevance values of N recommendation results in the recommendation data fed back each time, so that the recommendation effect of the current recommendation algorithm on the target user is determined according to the descending curve. The evaluation method can reflect the quality change trend of the recommended effect aiming at the target user, so that the overall effect of the recommended algorithm is improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure.
FIG. 1 is a flowchart of a method for evaluating the effectiveness of a recommendation algorithm, shown in accordance with an exemplary embodiment;
FIG. 2 is a flowchart illustrating a particular method of acquiring recommendation data in accordance with an exemplary embodiment;
FIG. 3 is a flowchart II of a particular method of acquiring recommendation data, according to an example embodiment;
FIG. 4 is a flowchart second of an effect evaluation method of a recommendation algorithm, shown in accordance with an exemplary embodiment;
FIG. 5a is a block diagram of an effect evaluation apparatus of a recommendation algorithm shown in accordance with an exemplary embodiment;
FIG. 5b is a block diagram II of an effect evaluation device of a recommendation algorithm, shown in accordance with an exemplary embodiment;
fig. 5c is a block diagram three of an effect evaluation apparatus of a recommendation algorithm shown according to an exemplary embodiment;
FIG. 5d is a block diagram four of an effect evaluation device of a recommendation algorithm, shown according to an exemplary embodiment;
fig. 5e is a block diagram five of an effect evaluation apparatus of a recommendation algorithm shown according to an exemplary embodiment;
fig. 6 is a block diagram illustrating an effect evaluation apparatus for a recommendation algorithm according to an exemplary embodiment.
Detailed Description
Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as detailed in the accompanying claims.
The technical scheme provided by the embodiment of the disclosure relates to an effect evaluation method of a recommendation algorithm, which can be used for evaluating the output result of the recommendation algorithm. In the related art, an evaluation person can refresh a certain recommended interface, then score the correlation degree of the first ten or the first twenty recommended results, and finally select a result with low correlation degree and feed the result back to a developer. However, the evaluation method is limited to screening out the results which do not meet the standard, and cannot well reflect the quality change trend of the recommended effect. Based on the above, according to the technical scheme of the present disclosure, recommendation data fed back continuously and repeatedly can be obtained according to query information provided by a target user, and a descent curve pointing to the target user is drawn based on the addition result of the relevance values of the N recommendation results in the recommendation data fed back each time, so that the recommendation effect of the current recommendation algorithm on the target user is determined according to the descent curve. The evaluation method can reflect the quality change trend of the recommended effect aiming at the target user, so that the overall effect of the recommended algorithm is improved.
Fig. 1 is a flowchart schematically illustrating an effect evaluation method of a recommendation algorithm according to an embodiment of the present disclosure. According to the embodiment shown in fig. 1, the effect evaluation method specifically includes the following steps S1 to S3:
in step S1, acquiring recommendation data which is fed back continuously and repeatedly based on query information provided by a target user, wherein the recommendation data comprises a plurality of recommendation results which are ordered according to the association degree;
in step S2, N relevance degrees corresponding to N recommendation results fed back each time are obtained, and summation calculation is performed on the N relevance degrees, so as to obtain a plurality of continuous summation results;
in step S3, a descent curve directed to the target user is drawn based on the continuous plurality of addition results, so as to determine a recommendation effect of the recommendation algorithm on the target user according to the descent curve.
Based on the above steps S1 to S3, the technical solution of the present disclosure may obtain recommendation data that is fed back continuously and repeatedly according to query information provided by a target user, and draw a descent curve pointing to the target user based on a result of adding correlation values of N recommendation results in the recommendation data fed back each time, so as to determine a recommendation effect of a current recommendation algorithm on the target user according to the descent curve. The evaluation method can reflect the quality change trend of the recommended effect aiming at the target user, so that the overall effect of the recommended algorithm is improved.
The following describes the steps of the technical scheme of the present disclosure in detail with reference to specific embodiments.
In step S1, recommendation data that is fed back a plurality of times in succession based on query information provided by a target user is acquired, the recommendation data including a plurality of recommendation results ordered according to relevance.
For example, an evaluator may employ a number of accounts registered in advance to evaluate the outcome of the recommendation algorithm. Taking an evaluating person as an example of a commodity required by a target user to query, the target user carries first information such as a user ID and the like, and also provides second information such as keywords, key features, scene values and the like corresponding to the commodity to be queried, wherein the scene values refer to interfaces such as a first page, a commodity domain or a shop domain and the like for the user to query. At this time, the recommendation algorithm queries based on the query information, and feeds back the queried results to the target user as recommendation data continuously for multiple times, where the recommendation data is required to be sorted according to the relevance, for example, the recommendation data is displayed in a descending order in the page of the query result. Therefore, the recommendation data fed back for the first time is the recommendation result combination with the maximum correlation degree. When the recommended data is fed back each time, the recommendation algorithm can display a fixed number of recommended results according to a preset feedback quantity value, for example, each time the recommended data displayed each time contains N recommended results, and the same number of recommended results are continuously returned for multiple times.
In one embodiment, FIG. 2 illustrates a specific method flow diagram for obtaining recommendation data. As can be seen from fig. 2, the acquiring the recommendation data fed back a plurality of times in succession based on the query information provided by the target user includes the following steps S111 to S113:
in step S111, a current query result returned based on the query information provided by the target user in the current query is obtained, where the current query result is displayed as recommendation data including N recommendation results;
in step S112, whether the current query result is the first obtained query result is determined, if yes, the query result based on the preset information provided by the target user is continuously obtained, if not, the next step is executed;
in step S113, it is determined whether the sum of the relevancy of the N recommended results in the recommended data is greater than or equal to the first threshold, and if so, the query result based on the preset information provided by the target user is continuously obtained until the sum of the relevancy of the N recommended results is less than the first threshold.
Specifically, the recommended data including N recommended results are obtained as the current query result during each query, at this time, whether the current query result is the first obtained query result needs to be detected, if yes, the query result based on the preset information provided by the target user is continuously obtained, for example, the query result of the next page is continuously displayed by turning pages, if not, whether the sum of the relevancy of the N recommended results in the currently obtained recommended data is greater than or equal to the first threshold value needs to be determined. And when the sum of the relevancy of the N recommended results is greater than or equal to a first threshold value, continuously acquiring the query result based on the preset information provided by the target user until the sum of the relevancy of the N recommended results is smaller than the first threshold value. It can be seen that the first threshold is a threshold for measuring the degree of association, for example, when the sum of the degrees of association of the N recommended results is very low, or even close to 0, the query can be ended.
In another embodiment, FIG. 3 illustrates a specific method flow diagram for obtaining recommendation data. As can be seen from fig. 3, the acquiring the recommendation data fed back continuously for a plurality of times based on the query information provided by the target user includes the following steps S121 to S123:
in step S121, a current query result returned based on the query information provided by the target user in the current query is obtained, where the current query result is displayed as recommendation data including N recommendation results;
in step S122, it is determined whether the current query result is the first obtained query result, if yes, the query result based on the preset information provided by the target user is continuously obtained, if no, the next step is executed;
in step S123, it is determined whether the association degree of each recommended result in the recommended data is greater than or equal to the second threshold, if yes, the query result based on the preset information provided by the target user is continuously obtained until the association degree of any recommended result is less than the second threshold.
Specifically, the recommended data including N recommended results are obtained as the current query result during each query, at this time, whether the current query result is the first obtained query result needs to be detected, if yes, the query result based on the preset information provided by the target user is continuously obtained, for example, the query result of the next page is continuously displayed by turning pages, if not, whether the association degree of each recommended result in the currently obtained recommended data is greater than or equal to the second threshold value needs to be judged. And when the relevancy of all the recommended results is greater than or equal to the second threshold value, continuously acquiring the query result based on the preset information provided by the target user until the relevancy of any recommended result is smaller than the second threshold value. Because the recommended results are ranked according to the relevancy, when the relevancy of any one recommended result is smaller than the second threshold value, the following recommended results are necessarily smaller than the second threshold value. The second threshold is a threshold for measuring the degree of association, for example, when the degree of association of a recommendation is low or even close to 0, the query can be ended.
It should be noted that: the manner of acquiring the recommended data in the embodiment of the present disclosure is not limited to the above two, and is not particularly limited as long as the recommended result satisfying the condition can be acquired.
In step S2, N relevance degrees corresponding to N recommendation results fed back each time are obtained, and summation calculation is performed on the N relevance degrees, so as to obtain a plurality of continuous summation results.
For example, since the recommended data are sorted according to the relevance degree when being fed back, the relevance degree refers to the relevance degree of the query information provided by the target user, and therefore, each recommended result necessarily has a corresponding relevance degree value. The present embodiment may determine the relevance value of each recommended result according to the relevance value transmitted from the code or the relevance value estimated manually, but is not limited thereto. Based on this, for N recommended results in the recommended data fed back each time, the embodiment may perform addition operation on N relevance values corresponding to the N recommended results to obtain an addition result corresponding to each recommended data, so that for the recommended data fed back continuously multiple times, the embodiment may obtain a plurality of continuous addition results.
In step S3, a descent curve directed to the target user is drawn based on the continuous plurality of addition results, so as to determine a recommendation effect of the recommendation algorithm on the target user according to the descent curve.
For example, after obtaining a plurality of continuous addition results corresponding to the recommended data fed back continuously for a plurality of times, according to the plurality of continuous addition results and the feedback times corresponding to the continuous addition results, a descent curve pointing to the target user can be drawn, and slope change data of the descent curve can be obtained. The descent curve can be used for describing the recommendation effect of the current recommendation algorithm on a single target user, for example, when the slope of the descent curve is too large, the recommendation algorithm is not ideal enough, and when the relevance value of the first half part of the descent curve is relatively high but the second half part of the descent curve descends too fast, the recommendation algorithm can meet the requirement when the quantity requirement on query data is not high, the recommendation algorithm can be used for describing the recommendation effect of the current recommendation algorithm on the single target user.
Considering the evaluation of the recommended effect of a recommendation algorithm for more users, fig. 4 illustrates a flowchart of an effect evaluation method of another recommendation algorithm provided by an embodiment of the present disclosure. As can be seen from fig. 4, the effect evaluation method may further include the following step S4:
in step S4, an overall descent curve is drawn according to the plurality of descent curves pointing to the users, so as to determine recommendation effects of the recommendation algorithm on the plurality of users according to the overall descent curve.
For example, since the method provided in steps S1 to S3 can obtain the descent curve directed to the target user, the present embodiment can obtain a plurality of descent curves directed to the user by using the same method. Based on this, an overall descent curve pointing to the plurality of users can be fitted according to the plurality of descent curves, and in particular, the overall descent curve can be obtained according to the average value of the descent curves of the plurality of users at each feedback point, wherein the number of the descent curves is based on the fact that the recommendation effect of the recommendation algorithm on the majority of users can be reflected, so that the majority of users represent all users to evaluate the recommendation algorithm.
Based on the above, the effect evaluation method provided by the embodiment of the disclosure can reflect the quality change trend of the overall recommendation effect of the recommendation algorithm on most users, so that the comprehensive effect of the recommendation algorithm is improved, and the effect of the recommendation algorithm can be evaluated more comprehensively.
The following are device embodiments of the present disclosure that may be used to perform method embodiments of the present disclosure.
Fig. 5a is a schematic diagram of an effect evaluation apparatus of a recommendation algorithm, which may be implemented as part or all of an electronic device by software, hardware or a combination of both, according to an exemplary embodiment. According to fig. 5a, the effect evaluation device of the recommendation algorithm includes a data acquisition module 510, a relevance calculation module 520 and a first drawing module 530. The data acquisition module 510 is configured to acquire recommendation data that is continuously fed back multiple times based on query information provided by a target user, where the recommendation data includes a plurality of recommendation results ordered according to relevance; the relevance calculating module 502 is configured to obtain N relevance corresponding to N recommendation results fed back each time, and perform addition calculation on the N relevance to obtain a plurality of continuous addition results; the first drawing module 530 is configured to draw a descent curve pointing to the target user based on the continuous multiple addition results, so as to determine an effect of the recommendation algorithm on the target user according to the descent curve.
In one embodiment, referring to fig. 5b, the data acquisition module 510 includes a first acquisition unit 5111, a first determination unit 5112, and a first definition unit 5113. The first obtaining unit 5111 is configured to obtain a current query result returned based on query information provided by the target user at the time of the current query, where the current query result is displayed as the recommendation data including N recommendation results; the first determining unit 5112 is configured to determine, when the current query result is a query result that is not acquired for the first time, whether a sum of relevancy of N recommendation results in the recommendation data is greater than or equal to a first threshold; the first limiting unit 5113 is configured to, when the sum of the relevancy of the N recommendation results is greater than or equal to the first threshold, continue to obtain the query result based on the preset information provided by the target user until the sum of the relevancy of the N recommendation results is less than the first threshold.
In one embodiment, referring to fig. 5c, the data acquisition module 510 includes a second acquisition unit 5121, a second determination unit 5122, and a second definition unit 5123. The second obtaining unit 5121 is configured to obtain a current query result returned based on the query information provided by the target user when the current query is performed, where the current query result is displayed as the recommendation data including N recommendation results; the second judging unit 5122 is configured to judge whether the association degree of each recommended result in the recommended data is greater than or equal to a second threshold when the current query result is a query result that is not acquired for the first time; the second limiting unit 5123 is configured to continuously obtain a query result based on preset information provided by the target user when the association degree of each recommended result is greater than or equal to the second threshold, until the association degree of any one of the recommended results is less than the second threshold.
In one embodiment, referring to fig. 5d, the first drawing module 530 includes a drawing unit 5300, where the drawing unit 5300 is configured to draw a falling curve pointing to the target user based on the continuous multiple addition results and the feedback times, and obtain slope change data of the falling curve.
In one embodiment, referring to fig. 5e, the apparatus further comprises a second drawing module 540, where the second drawing module 540 is configured to draw an overall descent curve according to a plurality of descent curves pointing to users, so as to determine effects of the recommendation algorithm on the plurality of users according to the overall descent curve.
According to the effect evaluation device of the recommendation algorithm, the recommendation data fed back continuously and repeatedly can be obtained according to the query information provided by the target user, and the descending curve pointing to the target user is drawn based on the sum result of the relevance values of N recommendation results in the recommendation data fed back each time, so that the recommendation effect of the current recommendation algorithm on the target user is determined according to the descending curve. The evaluation method can reflect the quality change trend of the recommended effect aiming at the target user, so that the overall effect of the recommended algorithm is improved.
The specific manner in which the respective modules perform the operations of the apparatus in the above embodiments has been described in detail in the embodiments related to the method, and will not be described in detail herein.
The embodiment of the disclosure also provides an effect evaluation device of the recommendation algorithm, which comprises:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to perform:
acquiring recommendation data fed back continuously and repeatedly based on query information provided by a target user, wherein the recommendation data comprises a plurality of recommendation results ordered according to the association degree;
n relevance degrees corresponding to N recommendation results fed back each time are obtained, and summation calculation is carried out on the N relevance degrees so as to obtain a plurality of continuous summation results;
and drawing a descending curve pointing to the target user based on the continuous multiple addition results so as to determine the effect of the recommendation algorithm on the target user according to the descending curve.
In one embodiment, the processor may be further configured to:
acquiring a current query result returned based on query information provided by the target user in the current query, wherein the current query result is displayed as the recommendation data comprising N recommendation results;
when the current query result is a query result which is not acquired for the first time, judging whether the sum of the relevancy of N recommended results in the recommended data is larger than or equal to a first threshold value;
and when the sum of the relevancy of the N recommended results is greater than or equal to the first threshold, continuously acquiring the query result based on the preset information provided by the target user until the sum of the relevancy of the N recommended results is smaller than the first threshold.
In one embodiment, the processor may be further configured to:
acquiring a current query result returned based on query information provided by the target user in the current query, wherein the current query result is displayed as the recommendation data comprising N recommendation results;
when the current query result is a query result which is not acquired for the first time, judging whether the association degree of each recommended result in the recommended data is larger than or equal to a second threshold value;
and when the association degree of each recommended result is larger than or equal to the second threshold value, continuously acquiring the query result based on the preset information provided by the target user until the association degree of any recommended result is smaller than the second threshold value.
In one embodiment, the processor may be further configured to:
and drawing a descending curve pointing to the target user based on the continuous multiple addition results and the feedback times, and acquiring slope change data of the descending curve.
In one embodiment, the processor may be further configured to:
and drawing an overall descent curve according to a plurality of descent curves pointing to users, so as to determine the effect of the recommendation algorithm on the plurality of users according to the overall descent curve.
Fig. 6 is a block diagram illustrating an effect evaluation apparatus for a recommendation algorithm according to an exemplary embodiment. For example, the apparatus 60 may be provided as a server. The apparatus 60 includes a processing component 602 that further includes one or more processors and memory resources represented by a memory 604 for storing instructions, such as application programs, executable by the processing component 602. The application program stored in the memory 604 may include one or more modules each corresponding to a set of instructions. Further, the processing component 602 is configured to execute instructions to perform the above-described methods.
The apparatus 60 may also include a power component 606 configured to perform power management of the apparatus 60, a wired or wireless network interface 608 configured to connect the apparatus 60 to a network, and an input/output (I/O) interface 610. The device 60 may operate based on an operating system stored in memory 604, such as Windows Server, mac OS XTM, unixTM, linuxTM, freeBSDTM, or the like.
The presently disclosed embodiments also provide a non-transitory computer readable storage medium, which when executed by a processor of the apparatus 60, causes the apparatus 60 to perform the effect evaluation method of the recommendation algorithm described above, the method comprising:
acquiring recommendation data fed back continuously and repeatedly based on query information provided by a target user, wherein the recommendation data comprises a plurality of recommendation results ordered according to the association degree;
n relevance degrees corresponding to N recommendation results fed back each time are obtained, and summation calculation is carried out on the N relevance degrees so as to obtain a plurality of continuous summation results;
and drawing a descending curve pointing to the target user based on the continuous multiple addition results so as to determine the effect of the recommendation algorithm on the target user according to the descending curve.
In one embodiment, obtaining recommendation data that is fed back multiple times in succession based on query information provided by a target user includes:
acquiring a current query result returned based on query information provided by the target user in the current query, wherein the current query result is displayed as the recommendation data comprising N recommendation results;
when the current query result is a query result which is not acquired for the first time, judging whether the sum of the relevancy of N recommended results in the recommended data is larger than or equal to a first threshold value;
and when the sum of the relevancy of the N recommended results is greater than or equal to the first threshold, continuously acquiring the query result based on the preset information provided by the target user until the sum of the relevancy of the N recommended results is smaller than the first threshold.
In one embodiment, obtaining recommendation data that is fed back multiple times in succession based on query information provided by a target user includes:
acquiring a current query result returned based on query information provided by the target user in the current query, wherein the current query result is displayed as the recommendation data comprising N recommendation results;
when the current query result is a query result which is not acquired for the first time, judging whether the association degree of each recommended result in the recommended data is larger than or equal to a second threshold value;
and when the association degree of each recommended result is larger than or equal to the second threshold value, continuously acquiring the query result based on the preset information provided by the target user until the association degree of any recommended result is smaller than the second threshold value.
In one embodiment, drawing a decline curve directed to the target user based on the successive plurality of summation results includes:
and drawing a descending curve pointing to the target user based on the continuous multiple addition results and the feedback times, and acquiring slope change data of the descending curve.
In one embodiment, the method further comprises:
and drawing an overall descent curve according to a plurality of descent curves pointing to users, so as to determine the effect of the recommendation algorithm on the plurality of users according to the overall descent curve.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any adaptations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It is to be understood that the present disclosure is not limited to the precise arrangements and instrumentalities shown in the drawings, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the present disclosure should be limited by the attached claims.
Claims (12)
1. The effect evaluation method of the recommendation algorithm is characterized by comprising the following steps of:
acquiring recommendation data fed back continuously and repeatedly based on query information provided by a target user, wherein the recommendation data comprises a plurality of recommendation results ordered according to the association degree;
n relevance degrees corresponding to N recommendation results fed back each time are obtained, and summation calculation is carried out on the N relevance degrees so as to obtain a plurality of continuous summation results;
drawing a descent curve directed to the target user based on the continuous plurality of summation results to determine an effect of the recommendation algorithm on the target user according to the descent curve, comprising: drawing a descent curve pointing to a target user and obtaining slope change data of the descent curve; the descending curve is used for describing the recommending effect of the current recommending algorithm for a single target user, when the slope of the descending curve is overlarge, the recommending algorithm is not ideal, when the relevance value of the first half part of the descending curve is relatively high, but the second half part of the descending curve descends too fast, the recommending algorithm is insufficient in a recommending data set, and at the moment, the recommending algorithm meets the requirement when the requirement on the quantity of query data is not high.
2. The method of claim 1, wherein obtaining recommendation data based on the query information provided by the target user for multiple consecutive feedbacks comprises:
acquiring a current query result returned based on query information provided by the target user in the current query, wherein the current query result is displayed as the recommendation data comprising N recommendation results;
when the current query result is a query result which is not acquired for the first time, judging whether the sum of the relevancy of N recommended results in the recommended data is larger than or equal to a first threshold value;
and when the sum of the relevancy of the N recommended results is greater than or equal to the first threshold, continuously acquiring the query result based on the preset information provided by the target user until the sum of the relevancy of the N recommended results is smaller than the first threshold.
3. The method of claim 1, wherein obtaining recommendation data based on the query information provided by the target user for multiple consecutive feedbacks comprises:
acquiring a current query result returned based on query information provided by the target user in the current query, wherein the current query result is displayed as the recommendation data comprising N recommendation results;
when the current query result is a query result which is not acquired for the first time, judging whether the association degree of each recommended result in the recommended data is larger than or equal to a second threshold value;
and when the association degree of each recommended result is larger than or equal to the second threshold value, continuously acquiring the query result based on the preset information provided by the target user until the association degree of any recommended result is smaller than the second threshold value.
4. The method of claim 1, wherein drawing a decline curve directed to the target user based on the successive plurality of summation results comprises:
and drawing a descending curve pointing to the target user based on the continuous multiple addition results and the feedback times, and acquiring slope change data of the descending curve.
5. The method as recited in claim 1, further comprising:
and drawing an overall descent curve according to a plurality of descent curves pointing to users, so as to determine the effect of the recommendation algorithm on the plurality of users according to the overall descent curve.
6. An effect evaluation device of a recommendation algorithm, comprising:
the data acquisition module is used for acquiring recommendation data which is fed back continuously and repeatedly based on query information provided by a target user, wherein the recommendation data comprises a plurality of recommendation results which are ordered according to the association degree;
the relevance calculating module is used for acquiring N relevance corresponding to N recommendation results fed back each time, and carrying out addition calculation on the N relevance to obtain a plurality of continuous addition results;
the first drawing module is configured to draw a descent curve pointing to the target user based on the continuous multiple addition results, so as to determine an effect of the recommendation algorithm on the target user according to the descent curve, and includes: drawing a descent curve pointing to a target user and obtaining slope change data of the descent curve; the descending curve is used for describing the recommending effect of the current recommending algorithm for a single target user, when the slope of the descending curve is overlarge, the recommending algorithm is not ideal, when the relevance value of the first half part of the descending curve is relatively high, but the second half part of the descending curve descends too fast, the recommending algorithm is insufficient in a recommending data set, and at the moment, the recommending algorithm meets the requirement when the requirement on the quantity of query data is not high.
7. The apparatus of claim 6, wherein the data acquisition module comprises:
the first acquisition unit is used for acquiring a current query result returned based on query information provided by the target user when the current query is performed, and the current query result is displayed as the recommendation data comprising N recommendation results;
the first judging unit is used for judging whether the sum of the relevancy of N recommended results in the recommended data is larger than or equal to a first threshold value when the current query result is a query result which is not acquired for the first time;
and the first limiting unit is used for continuously acquiring the query result based on the preset information provided by the target user when the sum of the relevancy of the N recommended results is greater than or equal to the first threshold value until the sum of the relevancy of the N recommended results is smaller than the first threshold value.
8. The apparatus of claim 6, wherein the data acquisition module comprises:
the second acquisition unit is used for acquiring a current query result returned based on the query information provided by the target user when the current query is performed, and the current query result is displayed as the recommendation data comprising N recommendation results;
the second judging unit is used for judging whether the association degree of each recommended result in the recommended data is greater than or equal to a second threshold value when the current query result is a query result which is not acquired for the first time;
and the second limiting unit is used for continuously acquiring the query result based on the preset information provided by the target user when the association degree of each recommended result is larger than or equal to the second threshold value until the association degree of any recommended result is smaller than the second threshold value.
9. The apparatus of claim 6, wherein the first rendering module comprises:
and the drawing unit is used for drawing a descending curve pointing to the target user based on the continuous multiple addition results and the feedback times, and acquiring slope change data of the descending curve.
10. The apparatus as recited in claim 9, further comprising:
and the second drawing module is used for drawing an overall descent curve according to a plurality of descent curves pointing to the users so as to determine the effect of the recommendation algorithm on the plurality of users according to the overall descent curve.
11. An effect evaluation device of a recommendation algorithm, comprising:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to perform the steps of the method of any of claims 1-5.
12. A computer readable storage medium having stored thereon computer instructions, which when executed by a processor, implement the steps of the method of any of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910408209.4A CN110196796B (en) | 2019-05-15 | 2019-05-15 | Effect evaluation method and device for recommendation algorithm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910408209.4A CN110196796B (en) | 2019-05-15 | 2019-05-15 | Effect evaluation method and device for recommendation algorithm |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110196796A CN110196796A (en) | 2019-09-03 |
| CN110196796B true CN110196796B (en) | 2023-04-28 |
Family
ID=67752767
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910408209.4A Active CN110196796B (en) | 2019-05-15 | 2019-05-15 | Effect evaluation method and device for recommendation algorithm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110196796B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111724238B (en) * | 2020-06-23 | 2022-09-30 | 中国平安人寿保险股份有限公司 | Method, device and equipment for evaluating product recommendation accuracy and storage medium |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101281519A (en) * | 2007-04-02 | 2008-10-08 | 奇智软件（北京）有限公司 | Method for evaluating network resource value and application of searching engine field |
| CN101840535A (en) * | 2010-01-22 | 2010-09-22 | 同济大学 | Integrated supply chain performance indicator evaluation method based on improved grey correlation analysis method |
| CN102313698A (en) * | 2011-07-04 | 2012-01-11 | 天津科技大学 | Quality evaluation method of color ink-jet paper based on multi-layer grey relational evaluation |
| CN103034963A (en) * | 2012-11-28 | 2013-04-10 | 东南大学 | Service selection system and selection method based on correlation |
| CN105488107A (en) * | 2015-11-20 | 2016-04-13 | 天津大学 | Offline evaluation method for recommendation system |
| CN107292097A (en) * | 2017-06-14 | 2017-10-24 | 华东理工大学 | The feature selection approach of feature based group and traditional Chinese medical science primary symptom system of selection |
| EP3267386A1 (en) * | 2016-07-07 | 2018-01-10 | Yandex Europe AG | Method and apparatus for generating a content recommendation in a recommendation system |
| CN107729542A (en) * | 2017-10-31 | 2018-02-23 | 咪咕音乐有限公司 | A kind of information methods of marking and device and storage medium |
| CN108920596A (en) * | 2018-06-27 | 2018-11-30 | 广东亿迅科技有限公司 | A kind of personalized recommendation algorithm and terminal |
| CN109739768A (en) * | 2018-12-29 | 2019-05-10 | 深圳Tcl新技术有限公司 | Search engine evaluating method, device, equipment and readable storage medium storing program for executing |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10373230B2 (en) * | 2012-10-05 | 2019-08-06 | Raise Marketplace, Llc | Computer-implemented method for recommendation system input management |
| RU2632132C1 (en) * | 2016-07-07 | 2017-10-02 | Общество С Ограниченной Ответственностью "Яндекс" | Method and device for creating contents recommendations in recommendations system |
| US20190019158A1 (en) * | 2017-07-13 | 2019-01-17 | Linkedln Corporation | Quality evaluation of recommendation service |
-
2019
- 2019-05-15 CN CN201910408209.4A patent/CN110196796B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101281519A (en) * | 2007-04-02 | 2008-10-08 | 奇智软件（北京）有限公司 | Method for evaluating network resource value and application of searching engine field |
| CN101840535A (en) * | 2010-01-22 | 2010-09-22 | 同济大学 | Integrated supply chain performance indicator evaluation method based on improved grey correlation analysis method |
| CN102313698A (en) * | 2011-07-04 | 2012-01-11 | 天津科技大学 | Quality evaluation method of color ink-jet paper based on multi-layer grey relational evaluation |
| CN103034963A (en) * | 2012-11-28 | 2013-04-10 | 东南大学 | Service selection system and selection method based on correlation |
| CN105488107A (en) * | 2015-11-20 | 2016-04-13 | 天津大学 | Offline evaluation method for recommendation system |
| EP3267386A1 (en) * | 2016-07-07 | 2018-01-10 | Yandex Europe AG | Method and apparatus for generating a content recommendation in a recommendation system |
| CN107292097A (en) * | 2017-06-14 | 2017-10-24 | 华东理工大学 | The feature selection approach of feature based group and traditional Chinese medical science primary symptom system of selection |
| CN107729542A (en) * | 2017-10-31 | 2018-02-23 | 咪咕音乐有限公司 | A kind of information methods of marking and device and storage medium |
| CN108920596A (en) * | 2018-06-27 | 2018-11-30 | 广东亿迅科技有限公司 | A kind of personalized recommendation algorithm and terminal |
| CN109739768A (en) * | 2018-12-29 | 2019-05-10 | 深圳Tcl新技术有限公司 | Search engine evaluating method, device, equipment and readable storage medium storing program for executing |
Non-Patent Citations (1)
| Title |
|---|
| 电子商务推荐系统研究;王化明;《中国优秀硕士学位论文全文数据库》;20160831;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110196796A (en) | 2019-09-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11397772B2 (en) | Information search method, apparatus, and system | |
| US9092549B2 (en) | Recommendation of search keywords based on indication of user intention | |
| US11563818B2 (en) | Generating attribute preference models based on disparate attribute spectrums | |
| WO2020048084A1 (en) | Resource recommendation method and apparatus, computer device, and computer-readable storage medium | |
| JP6152173B2 (en) | Ranking product search results | |
| JP2021533450A (en) | Identification and application of hyperparameters for machine learning | |
| CN105872111B (en) | A kind of control method and server of information push | |
| CN109168047B (en) | Video recommendation method and device, server and storage medium | |
| CN108122611B (en) | Information recommendation method and device, storage medium and program product | |
| CN108833458A (en) | A kind of application recommended method, device, medium and equipment | |
| US10937070B2 (en) | Collaborative filtering to generate recommendations | |
| US20210004379A1 (en) | Utilize high performing trained machine learning models for information retrieval in a web store | |
| CN112269918B (en) | Information recommendation method, device, equipment and storage medium | |
| CN111651669A (en) | Information recommendation method and device, electronic equipment and computer-readable storage medium | |
| JP2021086438A (en) | Image searching apparatus, image searching method, and program | |
| CN110196796B (en) | Effect evaluation method and device for recommendation algorithm | |
| US20180150754A1 (en) | Data analysis method, system and non-transitory computer readable medium | |
| CN111046082A (en) | Data source determination method, device, server and storage medium | |
| CN108647986B (en) | Target user determination method and device and electronic equipment | |
| US10366088B2 (en) | Method and system for mining frequent and in-frequent items from a large transaction database | |
| CN111815258A (en) | Human resource matching method, device, equipment and storage medium | |
| CN103514187B (en) | Method and device for providing search results | |
| JP6751955B1 (en) | Learning method, evaluation device, and evaluation system | |
| KR20230009437A (en) | User search category predictor | |
| CN111581487A (en) | Information processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |