CN109885504B - Recommendation system test method, device, medium and electronic equipment - Google Patents
Recommendation system test method, device, medium and electronic equipment Download PDFInfo
- Publication number
- CN109885504B CN109885504B CN201910198370.3A CN201910198370A CN109885504B CN 109885504 B CN109885504 B CN 109885504B CN 201910198370 A CN201910198370 A CN 201910198370A CN 109885504 B CN109885504 B CN 109885504B
- Authority
- CN
- China
- Prior art keywords
- tested
- parameter
- measured
- range value
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention relates to the technical field of data processing, in particular to a method, a device, a medium and electronic equipment for testing a recommendation system. The test method comprises the following steps: acquiring a parameter to be measured and a standard range value of the parameter to be measured; acquiring an identifier of a recommender output by a recommendation system, and determining information to be tested according to the identifier and the parameter to be tested; and testing the information to be tested through the standard range value of the parameter to be tested. Compared with the problem of low test accuracy caused by the existing manual test mode, the technical scheme improves the test accuracy by automatically inquiring and counting the to-be-tested information output by the recommendation system. Meanwhile, the problem of high cost caused by manual test of the dynamically recommended information to be tested is solved.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to a test method and a test device for a recommendation system, and a computer readable medium and an electronic device for realizing the test method for the recommendation system.
Background
With the development of science and technology, thousands of people and thousands of application products based on big data and artificial intelligence are more and more pursued by netizens. In many application product recommendation systems, content of interest to a user is often recommended to the user at the entrance of an application (e.g., the top page of the application, etc.), and may include people or things that may be of interest to the user. For example: the system comprises a personalized search engine which accords with different user habits, an e-commerce which provides a personalized recommendation system for the user, or a song listening application which provides the personalized recommendation system for the user, and the like.
For application products with a user recommendation function, for example, a dating application product recommends other users who may be interested in the application to the user currently using the application, a recruiting application product recommends an applicant to a recruiter, and the like. The user (recommender) meeting the selection condition is recommended to the current user, and the current user can know the recommender more through the information of the recommender, so that the current user can be helped to acquire the interested object as soon as possible. In order to measure the recommendation accuracy of the recommendation system, the recommendation system is generally tested by a test method.
For a method for testing a recommendation system of a friend-making application product, the prior art generally adopts a method of comparing information of a recommender recommended by the recommendation system with conditions set by a current user for testing.
However, the accuracy of the test method provided by the prior art is low.
It is to be noted that the information disclosed in the above background section is only for enhancement of understanding of the background of the present invention and therefore may include information that does not constitute prior art known to a person of ordinary skill in the art.
Disclosure of Invention
Embodiments of the present invention provide a method and an apparatus for testing a recommendation system, and a computer-readable medium and an electronic device for implementing the method for testing a recommendation system, so as to overcome at least a disadvantage of a low accuracy of a test method provided in the prior art to a certain extent.
Additional features and advantages of the invention will be set forth in the detailed description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
According to a first aspect of the embodiments of the present invention, there is provided a method for testing a recommendation system, including:
acquiring a parameter to be measured and a standard range value of the parameter to be measured;
acquiring an identifier of a recommender output by a recommendation system, and determining information to be tested according to the identifier and the parameter to be tested;
and testing the information to be tested through the standard range value of the parameter to be tested.
In some embodiments of the present invention, based on the foregoing embodiments, obtaining a parameter to be measured and a standard range value of the parameter to be measured includes:
acquiring a recommendation requirement set by a user, and determining a parameter to be measured according to the recommendation requirement;
and determining a standard range value of the parameter to be measured according to the recommendation requirement and the parameter to be measured.
In some embodiments of the present invention, based on the foregoing embodiments, obtaining an identifier of a recommender output by a recommendation system, and determining information to be tested according to the identifier and the parameter to be tested includes:
creating a class related to recommender information, and taking the parameter to be measured as the attribute of the class;
acquiring the identifier of the recommender from the output data of the recommendation system, and initializing the class through the identifier to determine the recommender instance;
and acquiring the attribute value of the recommender instance as the information to be tested from a database.
In some embodiments of the present invention, based on the foregoing embodiments, the output data of the recommendation system is in JSON format;
obtaining the identifier of the recommender from the output data of the recommendation system includes:
and acquiring item _ id from output data in the JSON format to acquire the identifier of the recommender.
In some embodiments of the present invention, based on the foregoing embodiments, the testing the information to be tested through the standard range value of the parameter to be tested includes:
counting the information to be tested, and determining the range value to be tested of the parameter to be tested, wherein the method comprises the following steps: an upper limit value to be measured and a lower limit value to be measured;
and testing the range value to be tested through the standard range value of the parameter to be tested.
In some embodiments of the present invention, based on the foregoing embodiments, the testing the to-be-tested range value through the standard range value about the to-be-tested parameter includes:
judging whether the upper limit value to be detected and the lower limit value to be detected are both within the standard range value;
and determining that the data driving result is a throw error in response to the fact that the upper limit value to be measured or the lower limit value to be measured is not within the standard range value.
In some embodiments of the present invention, based on the foregoing embodiments, the method further includes:
responding to the fact that the upper limit value to be tested and the lower limit value to be tested are both within the standard range value, and acquiring output data of the recommendation system to expand the information to be tested;
updating the value of the range to be tested according to the expanded information to be tested;
and testing again according to the updated range value to be tested.
In a second aspect of the embodiments of the present invention, a testing apparatus for a recommendation system is provided, including:
the standard range value acquisition module is used for acquiring a parameter to be measured and a standard range value of the parameter to be measured;
the information to be tested acquisition module is used for acquiring the identifier of the recommender output by the recommendation system and determining the information to be tested according to the identifier and the parameter to be tested;
and the test module is used for testing the information to be tested through the standard range value of the parameter to be tested.
According to a third aspect of embodiments of the present invention, there is provided a computer readable medium, on which a computer program is stored, which when executed by a processor, implements the method of testing a recommendation system as described in the first aspect of the embodiments above.
According to a fourth aspect of embodiments of the present invention, there is provided an electronic apparatus, including: one or more processors; storage means for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to implement a method of testing a recommendation system as described in the first aspect of the embodiments above.
The technical scheme provided by the embodiment of the invention has the following beneficial effects:
on one hand, in the technical solutions provided in some embodiments of the present invention, a parameter to be tested is first obtained to determine a standard range value of the parameter to be tested, and then the information to be tested output by the recommendation system is tested according to the standard range value of the parameter to be tested, so that the test on the recommendation system is completed according to the test result. Compared with the problem of low test accuracy caused by the existing manual test mode, the test accuracy is improved by automatically inquiring and counting the to-be-tested information output by the recommendation system in the embodiment. Meanwhile, the problem of high cost caused by manual test of the dynamically recommended information to be tested is solved.
On the other hand, in the technical solutions provided in some embodiments of the present invention, the information to be tested is further determined according to the identifier of the recommender and the parameter to be tested by obtaining the identifier of the recommender output by the recommendation system. Therefore, the information to be tested can be rapidly and accurately determined, the problem of low testing efficiency caused by redundancy of the information to be tested is effectively avoided, and the testing efficiency is improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention. It is obvious that the drawings in the following description are only some embodiments of the invention, and that for a person skilled in the art, other drawings can be derived from them without inventive effort. In the drawings:
FIG. 1 schematically shows a flow diagram of a method of testing a recommendation system according to an embodiment of the invention;
FIG. 2 is a schematic flow chart illustrating a method for determining a standard range value of a parameter to be measured according to an embodiment of the invention;
fig. 3 schematically shows a flowchart of a method for determining information to be measured according to an embodiment of the present invention;
FIG. 4 schematically shows a flow diagram of a data driven test method according to another embodiment of the invention;
FIG. 5 schematically illustrates a flow diagram of a data driven testing method according to yet another embodiment of the invention;
FIG. 6 shows a schematic structural diagram of a testing device of the recommendation system according to an embodiment of the present invention; and the number of the first and second groups,
FIG. 7 illustrates a schematic structural diagram of a computer system suitable for use with the electronic device to implement an embodiment of the invention.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art.
Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to provide a thorough understanding of embodiments of the invention. One skilled in the relevant art will recognize, however, that the invention may be practiced without one or more of the specific details, or with other methods, components, devices, steps, and so forth. In other instances, well-known methods, devices, implementations or operations have not been shown or described in detail to avoid obscuring aspects of the invention.
The block diagrams shown in the figures are functional entities only and do not necessarily correspond to physically separate entities. I.e. these functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor means and/or microcontroller means.
The flow charts shown in the drawings are merely illustrative and do not necessarily include all of the contents and operations/steps, nor do they necessarily have to be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the actual execution sequence may be changed according to the actual situation.
The "recommendation system" described in this embodiment may be a recommendation module of a friend-making application product, and is generally used to dynamically push information that may be of interest to a current user. For example, the recommender system (e.g., a recommender module of a friend-making application) pushes N recommenders and their dynamics to the current user this time (where N is a positive integer), and the recommender system pushes N different recommenders and their dynamics to the user next time. Therefore, if the recommenders are tested one by one, the problems of huge testing workload and huge cost caused by manual query of a large amount of user information are caused. Meanwhile, the randomness of the recommendation causes that manual testing cannot ensure the testing accuracy.
In addition, because the recommendation system dynamically recommends different information, the way of testing through forged data is also not suitable for testing the recommendation system of friend-making application products.
Fig. 1 shows a flow diagram of a method for testing a recommendation system according to an embodiment of the invention. The test method of the recommendation system provided by the embodiment overcomes the above disadvantages of the test method of the existing recommendation system at least to some extent.
The execution subject of the test method of the recommendation system provided in this embodiment may be a device having a calculation processing function, such as a server.
Step S101, acquiring a parameter to be measured and a standard range value of the parameter to be measured;
step S102, acquiring an identifier of a recommender output by a recommendation system, and determining information to be tested according to the identifier and the parameter to be tested; and the number of the first and second groups,
and step S103, testing the information to be tested through the standard range value of the parameter to be tested.
In the technical solution provided in the embodiment shown in fig. 1, on one hand, a parameter to be tested is obtained to determine a standard range value of the parameter to be tested, and then the information to be tested output by the recommendation system is tested according to the standard range value of the parameter to be tested, so that the test on the recommendation system is completed according to the test result. Compared with the problem of low test accuracy caused by the existing manual test mode, the test accuracy is improved by automatically inquiring and counting the to-be-tested information output by the recommendation system in the embodiment. Meanwhile, the problem of high overhead caused by manual testing of the dynamically recommended information to be tested is solved.
On the other hand, the information to be tested is further determined according to the identifier of the recommender and the parameter to be tested by acquiring the identifier of the recommender output by the recommendation system. Therefore, the information to be tested can be rapidly and accurately determined, the problem of low testing efficiency caused by redundancy of the information to be tested is effectively avoided, and the testing efficiency is improved.
Specific embodiments of the individual steps of the embodiment shown in FIG. 1 are described in detail below.
In an exemplary embodiment, fig. 2 schematically shows a flowchart of a method for determining a standard range value of a parameter to be measured according to an embodiment of the present invention, which may be used to explain a specific implementation manner of step S101.
Referring to fig. 2, the method for determining the standard range value of the parameter to be measured according to the embodiment includes:
step S201, acquiring a recommendation requirement set by a user, and determining a parameter to be measured according to the recommendation requirement; and step S202, determining a standard range value of the parameter to be measured according to the recommendation requirement and the parameter to be measured.
In an exemplary embodiment, if the recommendation set by the user is "please recommend me a single man with an age between 20 and 30 years, a height between 170 and 185 centimeters, a master and above academic degree, and a residence in Beijing". In order to recommend a user (recommender) meeting the set recommendation requirement to the user, in the technical solution provided in this embodiment, the following keywords may be extracted from the recommendation requirement set by the user in a keyword extraction manner: age, height, school calendar, residence and gender, so as to determine the parameters to be tested related to the test of the recommendation system according to the keywords.
In an exemplary embodiment, the recommendation requirement is filtered again according to the parameters to be measured determined in the above steps, and a range value (standard range value) corresponding to each parameter to be measured is determined. For example, if the parameter "height" to be measured, the recommendation requirement is screened by the keyword "height", and the standard range value is obtained as follows: between 170 cm and 185 cm. In the following test process, aiming at a certain parameter to be tested, the corresponding information to be tested is compared with the standard range value to realize the test of the recommendation system.
In an exemplary embodiment, after the parameter to be tested and the standard range value thereof are determined, the number of the test combinations is also determined by traversing the parameter to be tested and the standard range value thereof. Illustratively, for the parameter "age" to be measured, the standard range value is 18 to 60 years, but the number of test combinations regarding the parameter "age" to be measured can be determined to be 5. Specifically, the test combination of the parameter "age" to be measured is as follows: testing the upper limit: greater than 18 years old, without upper limit; testing the lower limit: less than 60 years old, no lower limit; testing an upper limit and a lower limit simultaneously: greater than 18 and less than 60; fourthly, the first boundary value is tested to be less than 18; second boundary value test, greater than 60. It can be seen that the number of test combinations for the parameter "age" to be measured is 5 in total.
In an exemplary embodiment, the parameters to be measured are determined with respect to recommended requirements set by a user a of an application product. The standard range values of the parameters to be tested and the number of the test combinations of each parameter to be tested can be referred to table 1.
TABLE 1
In an exemplary embodiment, in the case that the parameter to be measured is greater than one, the total number of parameter combinations may be determined according to the following two ways.
The first mode is a cadier mode, specifically, the number of test combinations corresponding to each parameter to be tested is multiplied, and the product is used as the total number of the parameter combinations. If the number of the test combinations of the parameter to be tested, namely the age, is 5, the number of the test combinations of the parameter to be tested, namely the living area, is 10, the number of the test combinations of the parameter to be tested, namely the height, is 5, the number of the test combinations of the parameter to be tested, namely the academic calendar, is 5, and the number of the test combinations of the parameter to be tested, namely the monthly income is 6, the total number of the corresponding test combinations is as follows: 5 × 10 × 5 × 6 ═ 9000 test combinations.
The second approach is a simplification of the first approach. In view of the fact that the relationships among the parameters to be tested are independent and independent, an independent testing mode can be adopted. Namely, the total number of the test combinations is determined by means of summation: 5+10+5+5+6 are 31 kinds.
In an exemplary embodiment, in the case that the parameter to be measured is more than one, determining the total number of parameter combinations is not limited to the above two ways.
Taking the second method as an example, the 31 test combination forms are stored in a table, and then the 31 test combination forms are traversed and filled in the parameters of the data driving interface to be tested, so that the test taking the 31 test combination forms as the input can be automatically completed.
Illustratively, the unit test framework of python may be utilized to do the driving of data. Storing the data in an excel table, reading out the data line by line, and then calling an interface to execute testing by cycle.
In the technical solution provided by the embodiment shown in fig. 2, the data-driven test is used for testing an application product including a recommended user function, so that the problem of low test efficiency caused by repeated manual tests is solved, meanwhile, the test overhead is saved, and the test accuracy is favorably provided.
In an exemplary embodiment, fig. 3 schematically shows a flowchart of a method for determining information to be measured according to an embodiment of the present invention, which may be used to explain a specific implementation manner of step S102.
Referring to fig. 3, the method for determining information to be measured provided in this embodiment includes:
step S301, creating a class related to recommender information, and taking the parameter to be tested as the attribute of the class; step S302, obtaining the identifier of the recommender from the output data of the recommendation system, and initializing the class through the identifier to determine the recommender instance; and step S303, acquiring the attribute value of the recommender instance as the information to be tested from the database.
In an exemplary embodiment, a class related to recommender information is created in step S301, and the parameter to be measured is taken as an attribute of the class. The embodiment adopts a result data mode to improve the extraction accuracy of the information to be detected.
In an exemplary embodiment, the output data of the recommendation system is in JSON format. Then item _ id can be obtained from the output data in JSON format to obtain the identity of the recommender.
Specifically, the identification of the recommender is extracted:
in order to reduce information redundancy and accurately determine the information to be detected, the identifier ID of the recommender is extracted first in this embodiment for subsequent information arrangement. Specifically, all item _ IDs in the JSON can be read by the ID extraction module and stored in an array for use by subsequent modules.
For example, in the step S302, the class is initialized according to the obtained identifier of the recommender to determine the recommender instance. Further, the information about the parameter to be tested is obtained from the database and is used as the attribute value of the recommender instance, and then the information to be tested of the test is determined.
And extracting recommender information:
the recommender information extraction module is mainly used for extracting the information of the user for statistics and verification of subsequent results. The basic user information that we need statistics is probably as follows: age, sex, place of birth, income, school calendar, height, weight, native place, etc.
In the technical solution provided by the embodiment shown in fig. 3, all recommended information is extracted to avoid manual query one by one, thereby improving the testing efficiency and reducing the manual overhead.
In an exemplary embodiment, fig. 4 schematically shows a flowchart of a data-driven testing method according to another embodiment of the present invention, which may be used to explain a specific implementation manner of step S103.
Referring to fig. 4, the data-driven testing method provided by this embodiment includes:
step S401, counting the information to be tested, and determining the range value to be tested of the parameter to be tested, including: an upper limit value to be measured and a lower limit value to be measured; and step S402, testing the range value to be tested through the standard range value of the parameter to be tested.
In an exemplary embodiment, the information to be tested is counted in step S401, and the parameter "age" to be tested is taken as an example for explanation. For the user B, obtaining and counting recommender information recommended by the recommendation system for 4 times, wherein the specific statistical result of each time is as follows:
the statistical result of the recommender information recommended for the first time is as follows: the range value to be measured: 26-41 years old, the upper limit value to be detected is 41 years old, and the lower limit value to be detected is 26 years old;
and the statistical result of the recommender information recommended for the second time is as follows: the range value to be measured: 23-40 years old, the upper limit value to be tested is 40 years old, and the lower limit value to be tested is 23 years old;
and the statistical result of the third recommended recommender information is as follows: the range value to be measured: 24-41 years old, the upper limit value to be detected is 41 years old, and the lower limit value to be detected is 24 years old;
the statistical result of the recommender information recommended for the fourth time is as follows: the range value to be measured: 26-39 years old, the upper limit value to be measured is 39 years old, and the lower limit value to be measured is 26 years old.
Among the recommendation requests set by the user B, the information "age" to be measured is set to 25 to 40 years old. Whether the recommender information meets the recommendation of the user is verified in the following steps, that is, fig. 5 is used to explain how to judge whether the age of the recommender is 25-40 years old.
In an exemplary embodiment, fig. 5 schematically shows a flowchart of a data-driven testing method according to still another embodiment of the present invention, which may be used to explain a specific implementation manner of step S402.
Referring to fig. 5, the data-driven testing method provided by this embodiment includes:
step S501, judging whether the upper limit value to be detected and the lower limit value to be detected are both within the standard range value;
in an exemplary embodiment, in response to that the upper limit value to be measured or the lower limit value to be measured is not within the standard range value, step S502 is executed: the result of the data drive is determined to be a throw error.
Illustratively, the upper limit value to be measured in the statistical result of the recommender information recommended for the first time/the third time exceeds the standard range value, and the lower limit value to be measured in the statistical result of the recommender information recommended for the second time exceeds the standard range value. That is, the first, second and third cases are that the upper or lower boundary is out of range, so that the recommended user does not meet the recommendation of user B, thereby determining that the data-driven result is a throw error.
In an exemplary embodiment, in response to that the upper limit value to be measured and the lower limit value to be measured are both within the standard range value, for example, the statistical result of the recommender information recommended for the fourth time is that the standard range value does not completely cover the range value to be measured, which may be that the range setting in the code to be measured is problematic, and may also be caused by insufficient data fetching.
Step S503 is executed: acquiring output data of the recommendation system to expand the information to be tested, and updating a range value to be tested according to the expanded information to be tested; and, step S504: and testing again according to the updated range value to be tested.
In an exemplary embodiment, the output data of the recommendation system (i.e., the recommender information of the user B) may be continuously collected to expand the information to be tested, and the specific method for expanding the information to be tested may refer to the specific implementation manner provided in the embodiment shown in fig. 3, which is not described herein again.
In an exemplary embodiment, the specific method for updating the value of the range to be measured through the expanded information to be measured may refer to the specific implementation manner provided in the embodiment shown in step S401, and is not described herein again.
In an exemplary embodiment, the specific method for performing the test again according to the updated value of the range to be tested may refer to the specific implementation manner provided in the embodiment shown in step S402, and is not described herein again.
In the technical solutions provided by the embodiments shown in fig. 4 and 5, the accuracy of the test is improved by performing automatic query statistics on the information to be tested output by the recommendation system. Meanwhile, the problem of high cost caused by manual test of the dynamically recommended information to be tested is solved.
Embodiments of the apparatus of the present invention are described below, which can be used to perform the method for testing the recommendation system of the present invention.
Fig. 6 shows a schematic structural diagram of a testing device of the recommendation system according to an embodiment of the present invention.
Referring to fig. 6, the testing apparatus 600 of the recommendation system includes: a standard range value obtaining module 601, an information to be tested obtaining module 602 and a test module 603.
The standard range value obtaining module 601 is configured to obtain a parameter to be measured and a standard range value of the parameter to be measured;
the information to be tested acquisition module 602 is configured to acquire an identifier of a recommender output by the recommendation system, and determine information to be tested according to the identifier and the parameter to be tested; and the number of the first and second groups,
the testing module 603 is configured to test the information to be tested through the standard range value of the parameter to be tested.
In some embodiments of the present invention, based on the foregoing embodiments, the standard range value obtaining module 601 is specifically configured to:
acquiring a recommendation requirement set by a user, and determining a parameter to be measured according to the recommendation requirement; and the number of the first and second groups,
and determining a standard range value of the parameter to be measured according to the recommendation requirement and the parameter to be measured.
In some embodiments of the present invention, based on the foregoing embodiments, the module 602 for acquiring information to be tested includes: a class creation unit 6021, a recommender instance determination unit 6022, and an information to be measured acquisition unit 6023.
The class creation unit 6021 is configured to: creating a class related to recommender information, and taking the parameter to be measured as the attribute of the class;
the recommender instance determination unit 6022 described above is configured to: acquiring the identifier of the recommender from the output data of the recommendation system, and initializing the class through the identifier to determine the recommender instance; and the number of the first and second groups,
the information to be measured acquisition unit 6023 is configured to: and acquiring the attribute value of the recommender instance from a database as the information to be tested.
In some embodiments of the present invention, based on the foregoing embodiments, the output data of the recommendation system is in JSON format;
the recommender instance determining unit 6022 is specifically configured to:
and acquiring item _ id from output data in the JSON format to acquire an identifier of a recommender, and initializing the class through the identifier to determine a plurality of recommender instances.
In some embodiments of the present invention, based on the foregoing embodiments, the test module 603 includes: a statistics unit 6031 and a test unit 6032.
Wherein the statistical unit 6031 is configured to: counting the information to be tested, and determining the range value to be tested of the parameter to be tested, wherein the method comprises the following steps: an upper limit value to be measured and a lower limit value to be measured; and the number of the first and second groups,
the test unit 6032 is configured to: and testing the range value to be tested through the standard range value of the parameter to be tested.
In some embodiments of the present invention, based on the foregoing embodiments, the test unit 6032 includes: a judgment subunit 321 and a response subunit 322.
Wherein the determining subunit 321 is configured to: judging whether the upper limit value to be detected and the lower limit value to be detected are both within the standard range value; and the number of the first and second groups,
the response subunit 322 is configured to: and determining that the data driving result is a throw error in response to the fact that the upper limit value to be measured or the lower limit value to be measured is not within the standard range value.
In some embodiments of the present invention, based on the foregoing embodiments, the response subunit 322 is further configured to: responding to the fact that the upper limit value to be tested and the lower limit value to be tested are both within the standard range value, and acquiring output data of the recommendation system to expand the information to be tested;
updating the value of the range to be tested according to the expanded information to be tested; and the number of the first and second groups,
and testing again according to the updated range value to be tested.
For details that are not disclosed in the embodiments of the apparatus of the present invention, please refer to the embodiments of the testing method of the recommendation system described above for the details that are not disclosed in the embodiments of the apparatus of the present invention.
Referring now to FIG. 7, shown is a block diagram of a computer system 700 suitable for use with the electronic device implementing an embodiment of the present invention. The computer system 700 of the electronic device shown in fig. 7 is only an example, and should not bring any limitation to the function and the scope of use of the embodiments of the present invention.
As shown in fig. 7, the computer system 700 includes a Central Processing Unit (CPU)701, which can perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)702 or a program loaded from a storage section 708 into a Random Access Memory (RAM) 703. In the RAM703, various programs and data necessary for system operation are also stored. The CPU 701, the ROM 702, and the RAM703 are connected to each other via a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.
The following components are connected to the I/O interface 705: an input portion 706 including a keyboard, a mouse, and the like; an output section 707 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 708 including a hard disk and the like; and a communication section 709 including a network interface card such as a LAN card, a modem, or the like. The communication section 709 performs communication processing via a network such as the internet. A drive 710 is also connected to the I/O interface 705 as needed. A removable medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 710 as necessary, so that the computer program read out therefrom is mounted in the storage section 708 as necessary.
In particular, according to an embodiment of the present invention, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the invention include a computer program product comprising a computer program embodied on a computer-readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication section 709, and/or installed from the removable medium 711. The computer program executes the above-described functions defined in the system of the present application when executed by the Central Processing Unit (CPU) 701.
It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present invention may be implemented by software, or may be implemented by hardware, and the described units may also be disposed in a processor. Wherein the names of the elements do not in some way constitute a limitation on the elements themselves.
As another aspect, the present application also provides a computer-readable medium, which may be contained in the electronic device described in the above embodiment; or may exist separately without being assembled into the electronic device. The computer readable medium carries one or more programs, which when executed by the electronic device, cause the electronic device to implement the method for testing the recommendation system as described in the above embodiments.
For example, the electronic device may implement the following as shown in fig. 1: step S101, acquiring a parameter to be measured and a standard range value of the parameter to be measured; step S102, acquiring an identifier of a recommender output by a recommendation system, and determining information to be tested according to the identifier and the parameter to be tested; and step S103, testing the information to be tested through the standard range value of the parameter to be tested.
As another example, the electronic device may implement the steps shown in any of fig. 2 to 5.
It should be noted that although in the above detailed description several modules or units of the device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functions of two or more modules or units described above may be embodied in one module or unit according to an embodiment of the invention. Conversely, the features and functions of one module or unit described above may be further divided into embodiments by a plurality of modules or units.
Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiment of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiment of the present invention.
Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.
It will be understood that the invention is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the invention is limited only by the appended claims.
Claims (8)
1. A method for testing a recommendation system, comprising:
acquiring a parameter to be measured and a standard range value of the parameter to be measured;
acquiring an identifier of a recommender output by a recommendation system, and determining information to be tested according to the identifier and the parameter to be tested;
testing the information to be tested through the standard range value of the parameter to be tested;
the acquiring of the parameter to be measured and the standard range value of the parameter to be measured include:
acquiring a recommendation requirement, and determining a parameter to be measured according to the recommendation requirement;
determining a standard range value of the parameter to be measured according to the recommendation requirement and the parameter to be measured;
the step of testing the information to be tested through the standard range value of the parameter to be tested comprises the following steps:
counting the information to be tested, and determining the range value to be tested of the parameter to be tested, wherein the step comprises the following steps: an upper limit value to be measured and a lower limit value to be measured;
and testing the range value to be tested through the standard range value of the parameter to be tested.
2. The method for testing the recommendation system according to claim 1, wherein the step of obtaining the identifier of the recommender output by the recommendation system and determining the information to be tested according to the identifier and the parameter to be tested comprises the steps of:
creating a class related to recommender information, and taking the parameter to be tested as the attribute of the class;
acquiring the identifier of the recommender from the output data of the recommendation system, and initializing the class through the identifier to determine the recommender instance;
and acquiring the attribute value of the recommender instance from a database as the information to be tested.
3. The method for testing the recommendation system according to claim 2, wherein the output data of the recommendation system is in a JSON format;
obtaining the identifier of the recommender from the output data of the recommendation system includes:
and acquiring item _ id from output data in the JSON format to acquire the identifier of the recommender.
4. The method for testing the recommendation system according to claim 1, wherein the step of testing the range value to be tested through the standard range value of the parameter to be tested comprises:
judging whether the upper limit value to be detected and the lower limit value to be detected are both within the standard range value;
and determining that the data driving result is a throw error in response to the fact that the upper limit value to be measured or the lower limit value to be measured is not within the standard range value.
5. The method for testing a recommendation system according to claim 4, further comprising:
responding to the fact that the upper limit value to be tested and the lower limit value to be tested are both within the standard range value, and acquiring output data of the recommendation system to expand the information to be tested;
updating the value of the range to be tested according to the expanded information to be tested;
and testing again according to the updated range value to be tested.
6. A testing apparatus for a recommendation system, comprising:
the standard range value acquisition module is used for acquiring a parameter to be measured and a standard range value of the parameter to be measured;
the information to be tested acquisition module is used for acquiring the identifier of the recommender output by the recommendation system and determining the information to be tested according to the identifier and the parameter to be tested;
the test module is used for testing the information to be tested through the standard range value of the parameter to be tested;
the standard range value acquisition module is used for acquiring a recommendation requirement set by a user and determining a parameter to be measured according to the recommendation requirement; determining a standard range value of the parameter to be measured according to the recommendation requirement and the parameter to be measured;
the test module is configured to count the information to be tested, and determine a range value to be tested of the parameter to be tested, including: an upper limit value to be measured and a lower limit value to be measured; and testing the range value to be tested through the standard range value of the parameter to be tested.
7. A computer-readable medium, on which a computer program is stored, which program, when being executed by a processor, carries out a method of testing a recommendation system according to any one of claims 1 to 5.
8. An electronic device, comprising:
one or more processors;
storage means for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to carry out a method of testing a recommendation system as claimed in any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910198370.3A CN109885504B (en) | 2019-03-15 | 2019-03-15 | Recommendation system test method, device, medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910198370.3A CN109885504B (en) | 2019-03-15 | 2019-03-15 | Recommendation system test method, device, medium and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109885504A CN109885504A (en) | 2019-06-14 |
| CN109885504B true CN109885504B (en) | 2022-09-30 |
Family
ID=66932536
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910198370.3A Active CN109885504B (en) | 2019-03-15 | 2019-03-15 | Recommendation system test method, device, medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109885504B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111176965B (en) * | 2019-12-19 | 2023-10-13 | 广州方硅信息技术有限公司 | Recommendation system pre-release test method and device and electronic equipment |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017000743A1 (en) * | 2015-06-29 | 2017-01-05 | 中兴通讯股份有限公司 | Method and device for software recommendation |
| US10223245B1 (en) * | 2016-05-27 | 2019-03-05 | Amdocs Development Limited | System, method, and computer program for identifying tests to automate in a software testing project |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8869116B2 (en) * | 2006-11-13 | 2014-10-21 | Accenture Global Services Limited | Software testing capability assessment framework |
| US9092211B2 (en) * | 2012-12-13 | 2015-07-28 | Microsoft Technology Licensing, Llc | Social-based information recommendation system |
| CN106354841B (en) * | 2016-08-31 | 2020-03-13 | 北京城市网邻信息技术有限公司 | Method and device for testing recommendation algorithm |
| CN106598741B (en) * | 2016-12-16 | 2024-03-01 | 飞狐信息技术（天津）有限公司 | Distributed A/B test method and system for personalized recommendation system and video recommendation system |
| CN107967218B (en) * | 2017-12-26 | 2018-10-30 | 中原工学院 | Boundary value test method in industrial software on-the-spot test based on user's history data |
| CN108845941A (en) * | 2018-06-15 | 2018-11-20 | 郑州云海信息技术有限公司 | A kind of SQL injection test coverage statistical method and system |
-
2019
- 2019-03-15 CN CN201910198370.3A patent/CN109885504B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017000743A1 (en) * | 2015-06-29 | 2017-01-05 | 中兴通讯股份有限公司 | Method and device for software recommendation |
| US10223245B1 (en) * | 2016-05-27 | 2019-03-05 | Amdocs Development Limited | System, method, and computer program for identifying tests to automate in a software testing project |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109885504A (en) | 2019-06-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106651057B (en) | Mobile terminal user age prediction method based on installation package sequence list | |
| US10671684B2 (en) | Method and apparatus for identifying demand | |
| CN112889042A (en) | Identification and application of hyper-parameters in machine learning | |
| US11475055B2 (en) | Artificial intelligence based method and apparatus for determining regional information | |
| CN108959329B (en) | Text classification method, device, medium and equipment | |
| CN103761254A (en) | Method for matching and recommending service themes in various fields | |
| CN111259647A (en) | Question and answer text matching method, device, medium and electronic equipment based on artificial intelligence | |
| CN113688310B (en) | Content recommendation method, device, equipment and storage medium | |
| CN113360711B (en) | Model training and executing method, device, equipment and medium for video understanding task | |
| EP4113376A1 (en) | Image classification model training method and apparatus, computer device, and storage medium | |
| CN111061979A (en) | User label pushing method and device, electronic equipment and medium | |
| CN115730605B (en) | Data analysis method based on multidimensional information | |
| CN117290561B (en) | Service state information feedback method, device, equipment and computer readable medium | |
| CN113393306A (en) | Product recommendation method and device, electronic equipment and computer readable medium | |
| CN116204714A (en) | Recommendation method, recommendation device, electronic equipment and storage medium | |
| CN110929169A (en) | Position recommendation method based on improved Canopy clustering collaborative filtering algorithm | |
| CN109885504B (en) | Recommendation system test method, device, medium and electronic equipment | |
| CN112948526A (en) | User portrait generation method and device, electronic equipment and storage medium | |
| CN111143660A (en) | Intelligent recommendation method and device for knowledge platform and computer readable storage medium | |
| CN116127155A (en) | Method, system, electronic equipment and storage medium for pushing measures | |
| CN113869904B (en) | Suspicious data identification method, device, electronic equipment, medium and computer program | |
| CN111078972B (en) | Questioning behavior data acquisition method, questioning behavior data acquisition device and server | |
| CN114357242A (en) | Training evaluation method and device based on recall model, equipment and storage medium | |
| CN115238165A (en) | Information pushing method and device based on machine learning, storage medium and terminal | |
| CN113704236A (en) | Government affair system data quality evaluation method, device, terminal and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |