CN108255886B - Evaluation method and device of recommendation system - Google Patents
Evaluation method and device of recommendation system Download PDFInfo
- Publication number
- CN108255886B CN108255886B CN201611254995.XA CN201611254995A CN108255886B CN 108255886 B CN108255886 B CN 108255886B CN 201611254995 A CN201611254995 A CN 201611254995A CN 108255886 B CN108255886 B CN 108255886B
- Authority
- CN
- China
- Prior art keywords
- user
- access information
- recommended
- page
- link
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9535—Search customisation based on user profiles and personalisation
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses an evaluation method and an evaluation device for a recommendation system, relates to the technical field of data processing, and mainly aims to solve the problem that the recommendation effect of the recommendation system is evaluated only through click rate. The main technical scheme of the invention is as follows: acquiring user page access information and user recommended access information; judging whether the user page access information is matched with the user recommended access information or not; and if so, counting direct and indirect flow values brought by the user after the user accesses the recommended content based on a preset access sequence, the user page access information and the user recommended access information to serve as the evaluation value of the recommendation system. The method is mainly used for evaluating the recommendation system.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to an evaluation method and device of a recommendation system.
Background
Along with the gradual enrichment of the webpage content, each large website can establish a recommendation system in the webpage, so that a user can quickly and accurately find the favorite content according to the recommendation system. In order to better analyze the recommendation effect brought by the recommendation system to the user, a developer can perform effect evaluation on the recommendation system.
At present, the recommendation effect is usually evaluated through the click rate generated by the user in the recommendation system, for example, when the recommendation putting amount is 100, the click rate generated by the click recommendation system is 70, and the recommendation effect of the put recommendation system is better evaluated, but it is obvious that the recommendation effect of other resources generated by the recommendation system cannot be taken into consideration only by evaluating the recommendation system through the click rate, which undoubtedly reduces the accuracy of the evaluation effect.
Disclosure of Invention
In view of the above problems, the present invention is provided to provide an evaluation method and an evaluation device for a recommendation system, and mainly aims to solve the problem that the accuracy of an evaluation effect obtained by evaluating the recommendation system through click volume is poor.
By the technical scheme, the evaluation method of the recommendation system provided by the invention comprises the following steps:
acquiring user page access information and user recommended access information, wherein the user page access information comprises access data generated when a user accesses a website page, and the user recommended access information comprises access data generated when the user accesses recommended content provided by a recommendation system in the website page;
judging whether the user page access information is matched with the user recommended access information or not;
and if so, counting direct and indirect flow values brought by the user after the user accesses the recommended content based on a preset access sequence, the user page access information and the user recommended access information to serve as the evaluation value of the recommendation system.
With the above technical solution, the present invention provides an evaluation device for a recommendation system, including:
the system comprises an acquisition unit, a recommendation unit and a recommendation unit, wherein the acquisition unit is used for acquiring user page access information and user recommendation access information, the user page access information comprises access data generated when a user accesses a website page, and the user recommendation access information comprises access data generated when the user accesses recommended content provided by a recommendation system in the website page;
the judging unit is used for judging whether the user page access information is matched with the user recommended access information or not;
and the statistical unit is used for counting direct and indirect flow values brought by the user after accessing the recommended content as the evaluation value of the recommendation system based on a preset access sequence, the user page access information and the user recommended access information if the user page access information is matched with the user recommended access information.
By the technical scheme, the technical scheme provided by the embodiment of the invention at least has the following advantages:
the method and the device for evaluating the recommendation system provided by the embodiment of the invention firstly obtain user page access information and user recommendation access information, wherein the user page access information comprises access data generated when a user accesses a website page, the user recommendation access information comprises access data generated when the user accesses recommended content provided by the recommendation system in the website page, then judge whether the user page access information is matched with the user recommendation access information, and if so, count direct and indirect flow values brought by the user after accessing the recommended content as an evaluation value of the recommendation system based on a preset access sequence, the user page access information and the user recommendation access information. Direct flow brought by the fact that a user accesses recommended content and indirect flow brought by the fact that the recommended content are considered, the recommending effect of the recommending system is accurately evaluated from the flow value perspective, and compared with the existing scheme that the recommending effect of the recommending system is evaluated only through click quantity, accuracy of evaluating the recommending effect of the recommending system is undoubtedly and greatly improved.
The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
FIG. 1 is a flow chart illustrating an evaluation method of a recommendation system according to an embodiment of the present invention;
FIG. 2 is a flow chart of an evaluation method of another recommendation system provided by the embodiment of the invention
FIG. 3 is a block diagram of an evaluation device of a recommendation system according to an embodiment of the present invention;
fig. 4 is a block diagram of an evaluation device of another recommendation system according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
An embodiment of the present invention provides an evaluation method of a recommendation system, as shown in fig. 1, the method includes:
101. and acquiring user page access information and user recommended access information.
The user page access information includes access data generated when a user accesses a website page, the user recommended access information includes access data generated when the user accesses recommended content provided by a recommendation system in the website page, the user page access information may be an ID, time information, a link URL, a page title and the like generated when the user accesses the page, and the user recommended access information may be an ID, time information, a link URL, a page title and the like generated when the user accesses the recommended content in the page.
It should be noted that the information acquisition mode may be information acquisition by using a JS code embedded mode in each website, the user recommended access information is generated by a user accessing recommended content of a recommendation system in a page, or a user accessing the recommendation system again after accessing the recommendation system, the direct user page access information may be multiple user page access information generated for multiple pages accessed by the user, the user recommended access information may be the same as or different from the user page access information, and generally, the user recommended access information is generated by the recommendation system in the user access page.
For example, the obtained user page access information generated when the user accesses the website page for the first time may include the ID1, the time information 9: 20, and the page title movie, the obtained user page access information generated when the user accesses the website page for the second time may include the ID1, the time information 9: 23, and the page title movie, and the obtained user page access information generated when the user accesses the website page for the third time may include the ID1, the time information 9: 26, and the page title avatar; the obtained user recommended access information generated by the user accessing the recommended content provided by the recommendation system in the website page may include the identification ID1, the time information 9, the point 26 and the page title avatar.
102. And judging whether the user page access information is matched with the user recommended access information or not.
The matching may be that all the information in the user page access information and the information in the user recommended access information are the same, or may be that some of the information are the same, and the embodiment of the present invention is not particularly limited.
103. And if so, counting direct and indirect flow values brought by the user after the user accesses the recommended content based on a preset access sequence, the user page access information and the user recommended access information to serve as the evaluation value of the recommendation system.
The preset access sequence may be a time sequence or a page jump sequence, and an embodiment of the present invention is not particularly limited, where the direct traffic value is a traffic value generated when the user directly accesses the recommended content, and the indirect traffic value is a traffic value generated when the user continues to access other recommended content or a web page after accessing the recommended content.
For example, if the user page access information includes ID1, time 9 point 26 minutes, page title avatar and ID2, time 9 point 30 minutes, page title james cameron, and the user recommended access information includes ID1, time 9 point 26 minutes, and page title avatar, then the flow value generated when the user accesses the avatar page and the flow value generated when the user accesses the james cameron page are counted, and the direct flow value brought by the user accessing the avatar page and the indirect flow value brought by the user accessing the james cameron page are taken as the evaluation value of the recommendation system.
According to the method for evaluating the recommendation system provided by the embodiment of the invention, whether the user page access information is the same as the user recommendation access information is compared, if so, direct and indirect flow values brought by the user after accessing the recommendation content are counted as the evaluation value of the recommendation system based on the preset access sequence, the user page access information and the user recommendation access information, so that not only is the direct flow brought by the user accessing the recommendation content considered, but also the indirect flow brought by the recommendation content is considered, the recommendation effect of the recommendation system is accurately evaluated from the flow value perspective, and compared with the existing scheme for evaluating the recommendation effect of the recommendation system only through click quantity, the accuracy for evaluating the recommendation effect of the recommendation system is undoubtedly and greatly improved.
The embodiment of the present invention further provides another method for evaluating a recommendation system, as shown in fig. 2, the method includes:
201. and acquiring user page access information and user recommended access information.
Wherein the access data comprises at least one of: and time and clicked link when the user accesses the website page or the user accesses the recommended content provided by the recommendation system in the website page.
This step is the same as the method described in step 101 of fig. 1, and is not described here again.
202. And orderly storing the time and the link corresponding to the user page access information and the time and the link corresponding to the user recommended access information according to a preset access sequence.
The preset access sequence may be a preset time sequence or a preset jump sequence, and the ordered storage may be storage according to a time sequence or a jump sequence, which is not specifically limited in the embodiment of the present invention.
It should be noted that the storage location may be stored in the cache or in the local memory, and the embodiment of the present invention is not limited specifically, and the storage location may be stored in groups according to a certain type, so that when determining whether the user access page information matches the user recommended access information, the ordered page access information is directly extracted from the storage location, the efficiency of the determining step is improved, the determining step is simplified, and the evaluation efficiency of the recommendation system is improved.
203a, judging whether the time corresponding to the user page access information is matched with the time corresponding to the user recommended access information.
And matching the time corresponding to the user page access information and the time corresponding to the user recommended access information.
It should be noted that, when the number of times corresponding to the user page access information is multiple and the number of times corresponding to the user recommended access information is 1, the times corresponding to the user recommended information need to be compared with the times corresponding to all the user page access information. And judging whether the time corresponding to the user page access information is matched with the time corresponding to the user recommended access information or not so as to extract the matched time and realize the flow value statistics according to the time attribute.
For the embodiment of the present invention, step 203b, which is parallel to step 203a, determines whether the link corresponding to the user page access information matches the link corresponding to the user recommended access information.
And matching the link corresponding to the user page access information and the link corresponding to the user recommended access information if the links are the same.
It should be noted that, when the number of links corresponding to the user page access information is multiple and the number of links corresponding to the user recommended access information is 1, the links corresponding to the user recommended information need to be compared with the links corresponding to all the user page access information. And judging whether the link corresponding to the user page access information is matched with the link corresponding to the user recommended access information or not so as to extract the matched link and realize the flow value statistics according to the skip sequence attribute.
For the embodiment of the present invention, in step 204a after step 203a, if there is a time matching the time corresponding to the user recommended access information in the time corresponding to the user page access information, the time matching the time corresponding to the user recommended access information in the time corresponding to the user page access information is taken as a reference time, and the reference time and the user page access information corresponding to at least one continuous time after the reference time are extracted.
When the preset access sequence is a time sequence of accessing the website page by the user or accessing the recommended content provided by the recommendation system in the website page by the user, at least one continuous time after the reference time is a time after the reference time in the time corresponding to the user page access information stored according to the time sequence. Of course, the deeper the click path after the recommended content is clicked, the smaller the effect of the recommended content is, the depth of statistics can be limited so as to reduce the effect of data exaggeration, and the recommendation effect of the recommendation system can be reflected more truly. For example, a session with 20 pvs, may define a traffic value that is considered to be contributed only 10 pvs near the time of the recommended content.
For example, the user page access information is sorted into ID1-13:24, ID1-13:26, ID1-13:38 and ID1-13:40, if the matched user recommended access information is ID1-13:26, the ID1-13:26 is taken as the reference time, and the user page access information corresponding to the time ID1-13:26 after the reference time and the ID1-13:38 and ID1-13:40 are extracted. The matched time is taken as the reference time, and the user page access information corresponding to the time after the reference time is extracted, so that the user page access information generated by accessing other pages after the user accesses the recommended content is accurately determined, and the direct or indirect flow value brought by the user after accessing the recommended content can be obtained on the basis.
For the embodiment of the present invention, in step 205a after step 204a, the reference time and the page corresponding to the user page access information corresponding to at least one continuous time after the reference time are determined, and the traffic value generated when the user accesses all the pages is counted.
For example, the pages corresponding to the user page access information corresponding to the reference time and at least one continuous time after the reference time are respectively ID1-13:26, ID1-13:38 and ID1-13:40, the flow generated by browsing the pages is respectively 40, 30 and 60, and the statistical flow value is 40+30+60, namely 130. By counting the flow value of the page corresponding to the user page access information corresponding to the user access reference time and at least one continuous time after the reference time, the direct and indirect flow value brought by the user with contribution value after accessing the recommended content is counted as the evaluation value based on the reference time.
For the embodiment of the present invention, step 206a, following step 205a, configures the flow value as the evaluation value of the recommendation system.
By using the flow value as the evaluation value of the recommendation system, the recommendation effect evaluation only according to the click quantity is avoided, and the evaluation accuracy is improved.
For the embodiment of the present invention, in step 204b after step 203b, if there is a link matching with the link corresponding to the user recommended access information in the links corresponding to the user page access information, the link matching with the link corresponding to the user recommended access information in the links corresponding to the user page access information is taken as a reference link, and the reference link and the user page access information corresponding to at least one continuous link after the reference link are extracted.
When the preset access sequence is a jump sequence after the user accesses the website page or the user accesses recommended content provided by a recommendation system in the website page, at least one continuous link after the reference link is a link generated by clicking the current recommended content and then sequentially generated by the recommended content in the page, and so on, the preset access sequence is generated by continuously clicking the recommended content.
It should be noted that the reference link is obtained by identifying a link which is present in the link corresponding to the user page access information and matches with the link corresponding to the user recommended access information, and at least one continuous link after the reference link is also obtained by identifying links generated by recommended content in the page after the user accesses the recommendation system in the page, so that all links generated by accessing the recommended content in the recommendation system are stored as history data.
For example, if the link generated by clicking the recommended content of the recommendation system is a, the link b generated corresponding to the recommended content of the recommendation system in the page is clicked again on the page corresponding to the current link a, the link c generated corresponding to the recommended content of the recommendation system in the page clicked page corresponding to the current link b, and the link d generated corresponding to the content of the non-recommendation system in the page clicked page corresponding to the current link c, then a is used as the reference link, and at least one continuous link after the reference link is only the link b and the link c. And extracting at least one continuous link after the reference link by using the matched link as the reference link so as to extract a flow value generated by the potential user in the user page access information through the page information generated after the recommended content is accessed.
For the embodiment of the present invention, step 205b after step 204b determines the reference link and the page corresponding to the user page access information corresponding to at least one continuous link after the reference link, and counts the traffic value generated when the user accesses all the pages.
For example, the reference link is ss, the continuous links generated by the reference links are ff, ww and hh, the pages s, f, w and h corresponding to the links are determined, and the flow values generated by accessing the pages s, f, w and h are counted. By counting the flow value generated when a user accesses a reference link and a page corresponding to user page access information corresponding to at least one continuous link after the reference link, the flow value corresponding to the user access page with contribution value is counted as an evaluation value through the reference link.
For the embodiment of the present invention, step 206b, which follows step 205b, configures the flow value as the evaluation value of the recommendation system.
By using the flow value as the evaluation value of the recommendation system, the recommendation effect is prevented from being evaluated only according to the click rate, and the accuracy of evaluating the recommendation system is improved.
For the embodiment of the present invention, specific application scenarios may be as follows, but are not limited to the following scenarios, including: the user page access information acquired by adopting the js code is A-15:16, B-15:20, C-15:36, D-15:40, E-15:45, F-16:00, G-16:03, H-16:10 and I-16:20, the time is stored according to a preset time sequence, when the user recommendation information is C1-15:36, the user page access information is judged to be matched with the user recommendation access information, the user page access information corresponding to at least one continuous time after the reference time 15:36 is extracted, namely the sum of flow values generated by the user accessing the pages C-15:36, D-15:40, E-15:45, F-16:00, G-16:03, H-16:10 and I-16:20 is counted to be 90, this was taken as an evaluation value of the evaluation system. According to the evaluation method of the recommendation system provided by the embodiment of the invention, whether the user access page information is the same as the user recommendation access information or not is compared, if so, the flow value generated by the user access page is counted according to the preset time sequence and the skip sequence and is used as another evaluation value except the click rate of the recommendation system, the direct flow brought by the user access of the recommendation content and the indirect flow brought by the recommendation content are considered in a combined manner, and the flow value is used as an angle for evaluating the recommendation effect, so that the accuracy of the recommendation effect evaluation of the recommendation system is improved.
Further, as a specific implementation of the method shown in fig. 1, an embodiment of the present invention provides an evaluation apparatus of a recommendation system, and as shown in fig. 3, the apparatus may include: an acquisition unit 31, a judgment unit 32, and a statistic unit 33.
The acquiring unit 31 is configured to acquire user page access information and user recommended access information, where the user page access information includes access data generated when a user accesses a website page, and the user recommended access information includes access data generated when the user accesses recommended content provided by a recommendation system in the website page; the obtaining unit 31 executes a function module for obtaining the user page access information and the user recommended access information for an evaluation device of the recommendation system.
A judging unit 32, configured to judge whether the user page access information matches the user recommended access information; the determination unit 32 is a functional module for executing, by an evaluation device of the recommendation system, a determination of whether the user page access information matches the user recommendation access information.
And a counting unit 33, configured to count, if the determining unit 32 determines that the user page access information matches the user recommended access information, a direct and indirect traffic value brought by the user after accessing the recommended content as the evaluation value of the recommendation system based on a preset access sequence, the user page access information, and the user recommended access information. The statistical unit 33 is a functional module that performs, for an evaluation device of the recommendation system, statistics of direct and indirect traffic values brought by a user after accessing the recommended content as an evaluation value of the recommendation system based on a preset access order, the user page access information, and the user recommended access information after determining that the user page access information matches the user recommended access information.
Compared with the existing recommendation effect of the recommendation system only through the click volume, the evaluation device of the recommendation system provided by the embodiment of the invention has the advantages that through comparing whether the user access page information is the same as the user recommendation access information or not, and if so, counting the direct and indirect flow values brought by the user after accessing the recommendation content based on the preset access sequence, the user page access information and the user recommendation access information as the evaluation value of the recommendation system, not only the direct flow brought by the user accessing the recommendation content is considered, but also the indirect flow brought by the recommendation content is considered, the recommendation effect of the recommendation system is accurately evaluated from the flow value perspective, and compared with the existing scheme of evaluating the recommendation effect of the recommendation system only through the click volume, the accuracy of evaluating the recommendation effect of the recommendation system is undoubtedly improved.
The embodiment of the apparatus corresponds to the embodiment of the method, and for convenience of reading, details in the embodiment of the apparatus are not repeated one by one, but it should be clear that the apparatus in the embodiment can correspondingly implement all the contents in the embodiment of the method.
Further, as a specific implementation of the method shown in fig. 2, an embodiment of the present invention provides another evaluation apparatus for a recommendation system, and as shown in fig. 4, the apparatus may include: an acquisition unit 41, a judgment unit 42, a statistic unit 43, and a storage unit 44.
An obtaining unit 41, configured to obtain user page access information and user recommended access information, where the user page access information includes access data generated when a user accesses a website page, and the user recommended access information includes access data generated when the user accesses recommended content provided by a recommendation system in the website page; the obtaining unit 41 is a functional module for executing the function of obtaining the user page access information and the user recommended access information for the evaluation device of the recommendation system.
A judging unit 42, configured to judge whether the user page access information matches the user recommended access information; the judging unit 42 is a functional module for executing, by an evaluation device of the recommendation system, judgment on whether the user page access information matches the user recommended access information.
A counting unit 43, configured to count, based on a preset access sequence, the user page access information, and the user recommended access information, a direct and indirect traffic value brought by the user after accessing the recommended content as an evaluation value of the recommendation system if the determining unit 42 determines that the user page access information matches the user recommended access information. The counting unit 43 is a functional module that performs, for the evaluation device of the recommendation system, counting, based on a preset access order, the user page access information, and the user recommended access information, direct and indirect traffic values brought by the user after accessing the recommended content as the evaluation value of the recommendation system after the judging unit 42 judges that the user page access information matches the user recommended access information.
Further, the apparatus further comprises:
and the storage unit 44 is configured to store the time and the link corresponding to the user page access information, and the time and the link corresponding to the user recommended access information in order according to a preset access sequence. The storage unit 44 is a functional module for executing, by an evaluation device of the recommendation system, orderly storing the time and the link corresponding to the user page access information and the time and the link corresponding to the user recommended access information according to a preset access sequence.
The determining unit 42 is specifically configured to determine whether the time corresponding to the user page access information matches the time corresponding to the user recommended access information; the determination unit 42 is a functional module for performing, for an evaluation device of the recommendation system, a determination of whether a time corresponding to the user page access information matches a time corresponding to the user recommended access information.
The determining unit 42 is further specifically configured to determine whether the link corresponding to the user page access information matches with the link corresponding to the user recommended access information. The determining unit 42 is a functional module for performing, for the evaluating device of the recommending system, determination of whether the link corresponding to the user page access information matches with the link corresponding to the user recommended access information.
Further, the statistical unit 43 includes:
an extracting module 4301, configured to, if the determining unit 42 determines that there is a time that matches the time corresponding to the user recommended access information in the time corresponding to the user page access information, take a time that matches the time corresponding to the user recommended access information in the time corresponding to the user page access information as a reference time, and extract user page access information corresponding to the reference time and at least one continuous time after the reference time; the extraction module 4301 is a functional module that, if the determination unit 42 determines that there is a time matching the time corresponding to the user recommended access information in the times corresponding to the user page access information, the extraction module executes, for the evaluation device of the recommendation system, a time matching the time corresponding to the user recommended access information in the times corresponding to the user page access information as a reference time, and extracts the reference time and the user page access information corresponding to at least one continuous time after the reference time.
A determining module 4302, configured to determine the reference time and a page corresponding to the user page access information corresponding to at least one continuous time after the reference time, and count a traffic value generated when the user accesses all the pages; the determining module 4302 is a functional module for executing, by an evaluation device of the recommendation system, a page corresponding to the user page access information corresponding to the reference time and at least one continuous time after the reference time, and counting a traffic value generated when the user accesses all the pages.
A configuration module 4303, configured to configure the flow value as an evaluation value of the recommendation system. The configuration module 4303 is a functional module that performs, for an evaluation device of the recommendation system, configuration of the flow rate value as an evaluation value of the recommendation system.
The extracting module 4301 is further configured to, if the determining unit 42 determines that a link matching the link corresponding to the user recommended access information exists in the links corresponding to the user page access information, use a link matching the link corresponding to the user recommended access information in the links corresponding to the user page access information as a reference link, and extract the reference link and user page access information corresponding to at least one continuous link after the reference link; the extraction module 4301 is a functional module that, when the determination unit 42 determines that the link corresponding to the user page access information exists in the links corresponding to the user recommended access information, the extraction module executes, for the evaluation device of the recommendation system, a link corresponding to the user recommended access information in the links corresponding to the user page access information as a reference link, and extracts the reference link and the user page access information corresponding to at least one continuous link after the reference link.
The determining module 4302 is further configured to determine the reference link and a page corresponding to the user page access information corresponding to at least one continuous link after the reference link, and count a traffic value generated when the user accesses all the pages; the determining module 4302 is a functional module for executing, by an evaluation device of the recommendation system, a page corresponding to the user page access information corresponding to the reference link and at least one continuous link following the reference link, and counting a flow value generated when the user accesses all the pages.
The configuration module 4303 is further configured to configure the flow rate value as an evaluation value of the recommendation system. The configuration module 4303 is a functional module that performs, for an evaluation device of the recommendation system, configuration of the flow rate value as an evaluation value of the recommendation system.
According to the evaluation device of the recommendation system provided by the embodiment of the invention, whether the user page access information is the same as the user recommendation access information or not is compared, if so, the flow value generated by the user access page is counted according to the preset time sequence and the skip sequence and is used as another evaluation value except the click rate of the recommendation system, the direct flow brought by the user access of the recommendation content and the indirect flow brought by the recommendation content are considered in a combined manner, and the flow value is used as an angle for evaluating the recommendation effect, so that the accuracy of the recommendation effect evaluation of the recommendation system is improved.
The embodiment of the apparatus corresponds to the embodiment of the method, and for convenience of reading, details in the embodiment of the apparatus are not repeated one by one, but it should be clear that the apparatus in the embodiment can correspondingly implement all the contents in the embodiment of the method.
The evaluation device of the recommendation system comprises a processor and a memory, wherein the acquisition unit, the judgment unit, the statistic unit and the like are stored in the memory as program units, and the processor executes the program units stored in the memory to realize corresponding functions.
The processor comprises a kernel, and the kernel calls the corresponding program unit from the memory. One or more than one kernel can be set, and the problem that the accuracy of the evaluation effect is poor due to the fact that the recommendation system is evaluated through the click rate is solved by adjusting the kernel parameters.
The memory may include volatile memory in a computer readable medium, Random Access Memory (RAM) and/or nonvolatile memory such as Read Only Memory (ROM) or flash memory (flash RAM), and the memory includes at least one memory chip.
The present application further provides a computer program product adapted to perform program code for initializing the following method steps when executed on a data processing device: acquiring user page access information and user recommended access information, wherein the user page access information comprises access data generated when a user accesses a website page, and the user recommended access information comprises access data generated when the user accesses recommended content provided by a recommendation system in the website page; judging whether the user page access information is matched with the user recommended access information or not; and if so, counting direct and indirect flow values brought by the user after the user accesses the recommended content based on a preset access sequence, the user page access information and the user recommended access information to serve as the evaluation value of the recommendation system.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
The above are merely examples of the present application and are not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims (12)
1. An evaluation method of a recommendation system, comprising:
acquiring user page access information and user recommended access information, wherein the user page access information comprises access data generated when a user accesses a website page, and the user recommended access information comprises access data generated when the user accesses recommended content provided by a recommendation system in the website page;
judging whether the user page access information is matched with the user recommended access information or not;
and if so, counting direct and indirect flow values brought by the user after the user accesses the recommended content as an evaluation value of the recommendation system based on a preset access sequence, the user page access information and the user recommended access information, wherein the direct flow value is a flow value generated by the user after the user directly accesses the recommended content, and the indirect flow value is a flow value generated by the user after the user continuously accesses other recommended content or a webpage after accessing the recommended content.
2. The method of claim 1, wherein the access data comprises at least one of: and time and clicked link when the user accesses the website page or the user accesses the recommended content provided by the recommendation system in the website page.
3. The method of claim 2, wherein after obtaining the user page access information and the user recommended access information, the method further comprises:
orderly storing the time and the link corresponding to the user page access information and the time and the link corresponding to the user recommended access information according to a preset access sequence;
the judging whether the user page access information is matched with the user recommended access information comprises:
judging whether the time corresponding to the user page access information is matched with the time corresponding to the user recommended access information or not; or
And judging whether the link corresponding to the user page access information is matched with the link corresponding to the user recommended access information or not.
4. The method according to claim 3, wherein the preset access sequence is a time sequence of the user accessing the website page or the user accessing recommended content provided by a recommendation system in the website page, and
if the recommended content is matched with the user webpage access information, counting direct and indirect flow values brought by the user after the user accesses the recommended content based on a preset access sequence, the user webpage access information and the user recommended access information as the evaluation value of the recommendation system, wherein the flow values comprise:
if the time matched with the time corresponding to the user recommended access information exists in the time corresponding to the user page access information, taking the time matched with the time corresponding to the user recommended access information in the time corresponding to the user page access information as reference time, and extracting the reference time and the user page access information corresponding to at least one continuous time after the reference time;
determining the reference time and a page corresponding to user page access information corresponding to at least one continuous time after the reference time, and counting flow values generated when a user accesses all the pages;
configuring the flow value as an evaluation value of the recommendation system.
5. The method according to claim 3, wherein the preset access sequence is a jump sequence after the user accesses the website page or the user accesses recommended content provided by a recommendation system in the website page, and
if the recommended content is matched with the user webpage access information, counting direct and indirect flow values brought by the user after the user accesses the recommended content based on a preset access sequence, the user webpage access information and the user recommended access information as the evaluation value of the recommendation system, wherein the flow values comprise:
if the link matched with the link corresponding to the user recommended access information exists in the links corresponding to the user page access information, taking the link matched with the link corresponding to the user recommended access information in the links corresponding to the user page access information as a reference link, and extracting the reference link and the user page access information corresponding to at least one continuous link behind the reference link;
determining a page corresponding to user page access information corresponding to the reference link and at least one continuous link behind the reference link, and counting flow values generated when a user accesses all the pages;
configuring the flow value as an evaluation value of the recommendation system.
6. An evaluation device of a recommendation system, comprising:
the system comprises an acquisition unit, a recommendation unit and a recommendation unit, wherein the acquisition unit is used for acquiring user page access information and user recommendation access information, the user page access information comprises access data generated when a user accesses a website page, and the user recommendation access information comprises access data generated when the user accesses recommended content provided by a recommendation system in the website page;
the judging unit is used for judging whether the user page access information is matched with the user recommended access information or not;
and the statistical unit is used for counting direct and indirect flow values brought by the user after the user accesses the recommended content as the evaluation value of the recommendation system based on a preset access sequence, the user page access information and the user recommended access information if the user page access information is matched with the user recommended access information, wherein the direct flow value is a flow value generated by the user directly accessing the recommended content, and the indirect flow value is a flow value generated by the user after continuing to access other recommended content or a webpage after accessing the recommended content.
7. The apparatus of claim 6, wherein the access data comprises at least one of: and time and clicked link when the user accesses the website page or the user accesses the recommended content provided by the recommendation system in the website page.
8. The apparatus of claim 7, further comprising: the storage unit is used for orderly storing the time and the link corresponding to the user page access information and the time and the link corresponding to the user recommended access information according to a preset access sequence;
the judging unit is specifically configured to judge whether time corresponding to the user page access information matches time corresponding to the user recommended access information; or
The judging unit is specifically configured to judge whether a link corresponding to the user page access information matches a link corresponding to the user recommended access information.
9. The apparatus of claim 8, wherein the statistical unit comprises:
the extraction module is used for taking the time matched with the time corresponding to the user recommended access information in the time corresponding to the user page access information as reference time if the time matched with the time corresponding to the user recommended access information exists in the time corresponding to the user page access information, and extracting the reference time and the user page access information corresponding to at least one continuous time after the reference time;
the determining module is used for determining the reference time and a page corresponding to the user page access information corresponding to at least one continuous time after the reference time, and counting flow values generated when the user accesses all the pages;
a configuration module to configure the flow value as an evaluation value of the recommendation system.
10. The apparatus of claim 8,
the extraction module is used for taking the link matched with the link corresponding to the user recommended access information in the links corresponding to the user page access information as a reference link and extracting the reference link and the user page access information corresponding to at least one continuous link behind the reference link if the link matched with the link corresponding to the user recommended access information exists in the links corresponding to the user page access information;
the determining module is used for determining the reference link and a page corresponding to user page access information corresponding to at least one continuous link after the reference link, and counting flow values generated when a user accesses all the pages;
a configuration module to configure the flow value as an evaluation value of the recommendation system.
11. A storage medium, characterized in that the storage medium comprises a stored program, wherein when the program runs, the device where the storage medium is located is controlled to execute the evaluation method of the recommendation system according to any one of claims 1 to 5.
12. A processor, characterized in that the processor is configured to run a program, wherein the program is configured to execute the evaluation method of the recommendation system according to any one of claims 1 to 5 when running.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611254995.XA CN108255886B (en) | 2016-12-29 | 2016-12-29 | Evaluation method and device of recommendation system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611254995.XA CN108255886B (en) | 2016-12-29 | 2016-12-29 | Evaluation method and device of recommendation system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108255886A CN108255886A (en) | 2018-07-06 |
| CN108255886B true CN108255886B (en) | 2021-08-13 |
Family
ID=62722031
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611254995.XA Active CN108255886B (en) | 2016-12-29 | 2016-12-29 | Evaluation method and device of recommendation system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108255886B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109948056B (en) * | 2019-03-19 | 2022-12-09 | 安庆师范大学 | Evaluation method and device of recommendation system |
| CN111915382A (en) * | 2019-05-08 | 2020-11-10 | 阿里巴巴集团控股有限公司 | Data processing method, system and device |
| CN110765308A (en) * | 2019-09-27 | 2020-02-07 | 上海麦克风文化传媒有限公司 | Evaluation method and system of audio album recommendation system |
| CN111061956B (en) | 2019-12-24 | 2022-08-16 | 北京百度网讯科技有限公司 | Method and apparatus for generating information |
| CN111797310A (en) * | 2020-06-19 | 2020-10-20 | 北京达佳互联信息技术有限公司 | Behavior review method and device, electronic equipment and storage medium |
| CN112784032A (en) * | 2021-01-28 | 2021-05-11 | 上海明略人工智能(集团)有限公司 | Conversation corpus recommendation evaluation method and device, storage medium and electronic equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104123321A (en) * | 2013-04-28 | 2014-10-29 | 百度在线网络技术（北京）有限公司 | Method and device for determining recommended pictures |
| CN104732338A (en) * | 2015-03-06 | 2015-06-24 | 安徽师范大学 | Method for evaluating services based on fuzzy theory in cloud environment |
| CN105005918A (en) * | 2015-07-24 | 2015-10-28 | 金鹃传媒科技股份有限公司 | Online advertisement push method based on user behavior data and potential user influence analysis and push evaluation method thereof |
| CN105069099A (en) * | 2015-08-06 | 2015-11-18 | 北京奇艺世纪科技有限公司 | Information recommendation method and system |
| CN106228392A (en) * | 2016-07-15 | 2016-12-14 | 北京小米移动软件有限公司 | Evaluation management method and device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120143718A1 (en) * | 2010-12-03 | 2012-06-07 | Choicestream, Inc. | Optimization of a web-based recommendation system |
-
2016
- 2016-12-29 CN CN201611254995.XA patent/CN108255886B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104123321A (en) * | 2013-04-28 | 2014-10-29 | 百度在线网络技术（北京）有限公司 | Method and device for determining recommended pictures |
| CN104732338A (en) * | 2015-03-06 | 2015-06-24 | 安徽师范大学 | Method for evaluating services based on fuzzy theory in cloud environment |
| CN105005918A (en) * | 2015-07-24 | 2015-10-28 | 金鹃传媒科技股份有限公司 | Online advertisement push method based on user behavior data and potential user influence analysis and push evaluation method thereof |
| CN105069099A (en) * | 2015-08-06 | 2015-11-18 | 北京奇艺世纪科技有限公司 | Information recommendation method and system |
| CN106228392A (en) * | 2016-07-15 | 2016-12-14 | 北京小米移动软件有限公司 | Evaluation management method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108255886A (en) | 2018-07-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108255886B (en) | Evaluation method and device of recommendation system | |
| CN109561052B (en) | Method and device for detecting abnormal flow of website | |
| US10346496B2 (en) | Information category obtaining method and apparatus | |
| CN106651416A (en) | Analyzing method and analyzing device of application popularization information | |
| CN104902292B (en) | A kind of the analysis of public opinion method and system based on television report | |
| CN110019298B (en) | Data processing method and device | |
| CN106598997B (en) | Method and device for calculating text theme attribution degree | |
| CN106933916B (en) | JSON character string processing method and device | |
| CN106909567B (en) | Data processing method and device | |
| CN106648839B (en) | Data processing method and device | |
| CN110955846A (en) | Propagation path diagram generation method and device | |
| CN106610989B (en) | Search keyword clustering method and device | |
| CN110928636A (en) | Virtual machine live migration method, device and equipment | |
| CN107562703B (en) | Dictionary tree reconstruction method and system | |
| CN106649374B (en) | Navigation tag sequencing method and device | |
| CN108984572B (en) | Website information pushing method and device | |
| CN110908587B (en) | Method and device for storing time sequence data | |
| CN108268475B (en) | Path analysis graph display method and device | |
| CN110019771B (en) | Text processing method and device | |
| CN109213906B (en) | Session duration calculation method, device and system | |
| CN110019198B (en) | Data query method and device | |
| CN110019068B (en) | Log text processing method and device | |
| CN107766216A (en) | It is a kind of to be used to obtain the method and apparatus using execution information | |
| CN111625721A (en) | Content recommendation method and device | |
| CN106547761B (en) | Data processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 100083 No. 401, 4th Floor, Haitai Building, 229 North Fourth Ring Road, Haidian District, Beijing Applicant after: Beijing Guoshuang Technology Co.,Ltd. Address before: 100086 Beijing city Haidian District Shuangyushu Area No. 76 Zhichun Road cuigongfandian 8 layer A Applicant before: Beijing Guoshuang Technology Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |