CN111190801A - Recommendation system testing method and device and electronic equipment - Google Patents
Recommendation system testing method and device and electronic equipment Download PDFInfo
- Publication number
- CN111190801A CN111190801A CN201911319730.7A CN201911319730A CN111190801A CN 111190801 A CN111190801 A CN 111190801A CN 201911319730 A CN201911319730 A CN 201911319730A CN 111190801 A CN111190801 A CN 111190801A
- Authority
- CN
- China
- Prior art keywords
- tested
- recommendation
- parameter
- test mode
- test
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3409—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment for performance assessment
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2457—Query processing with adaptation to user needs
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Debugging And Monitoring (AREA)
Abstract
The application discloses a recommendation system testing method and device, electronic equipment and a storage medium. The method comprises the following steps: the method comprises the steps of obtaining parameters to be tested corresponding to a target test mode, generating a recommendation request matched with the target test mode, sending the recommendation request carrying the parameters to be tested to a recommendation system, receiving a recommendation result to be tested corresponding to the parameters to be tested returned by the recommendation system, and adjusting a recommendation strategy associated with the parameters to be tested if the recommendation result to be tested does not meet the expected requirement corresponding to the target test mode. According to the scheme, the matched recommendation requests are generated according to different test modes, the recommendation requests matched with the target test mode can be generated after the target test mode and the corresponding parameters to be tested are determined, then different recommendation performances of the recommendation system are tested through various different independent test modes, and the accuracy of the recommendation performance of the recommendation system is improved.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a recommendation system testing method and apparatus, an electronic device, and a storage medium.
Background
A recommendation system (recommendation system) is a complex system for helping users to quickly acquire useful information, and models user figures by analyzing historical behaviors of the users, so that contents meeting the interests and requirements of the users are initiatively recommended to the users. The current recommendation system has various products, relates to various industries of clothes and eating houses, and has wide range of relation. As the sheer volume of recommendation system data grows, it becomes critical how to ensure that the recommendation system accurately recommends to the user the content that is of interest to the user.
Disclosure of Invention
In view of the foregoing problems, the present application provides a recommendation system testing method, device, electronic device and storage medium to improve the foregoing problems.
In a first aspect, an embodiment of the present application provides a recommendation system testing method, where the method includes: acquiring a parameter to be tested corresponding to a target test mode; generating a recommendation request matched with the target test pattern; sending the recommendation request carrying the parameter to be tested to a recommendation system, and receiving a recommendation result to be tested corresponding to the parameter to be tested, which is returned by the recommendation system; and if the recommendation result to be tested does not accord with the expected requirement corresponding to the target test mode, adjusting the recommendation strategy associated with the parameter to be tested.
In a second aspect, an embodiment of the present application provides a recommendation system testing apparatus, where the apparatus includes: the acquisition module is used for acquiring parameters to be tested corresponding to the target test mode; the first processing module is used for generating a recommendation request matched with the target test pattern; the second processing module is used for sending the recommendation request carrying the parameter to be tested to a recommendation system and receiving a recommendation result to be tested corresponding to the parameter to be tested, which is returned by the recommendation system; and the third processing module is used for adjusting the recommendation strategy associated with the parameter to be tested if the recommendation result to be tested does not accord with the expected requirement corresponding to the target test mode.
In a third aspect, an embodiment of the present application provides an electronic device, including a memory and one or more processors; one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to perform the method of the first aspect described above.
In a fourth aspect, embodiments of the present application provide a computer-readable storage medium having program code stored therein, where the program code, when executed by a processor, performs the method of the first aspect.
The embodiment of the application provides a recommendation system testing method and device, electronic equipment and a storage medium. The method comprises the steps of obtaining a parameter to be tested corresponding to a target test mode, then generating a recommendation request matched with the target test mode, sending the recommendation request carrying the parameter to be tested to a recommendation system, receiving a recommendation result to be tested corresponding to the parameter to be tested returned by the recommendation system, and then adjusting a recommendation strategy associated with the parameter to be tested if the recommendation result to be tested does not meet the expected requirement corresponding to the target test mode. According to the scheme, the recommendation request matched with the target test mode can be generated after the target test mode and the corresponding to-be-tested parameters are determined by generating the matched recommendation request aiming at different test modes. By sending the recommendation request carrying the parameter to be tested corresponding to the target test mode to the recommendation system and receiving the recommendation result to be tested corresponding to the parameter to be tested returned by the recommendation system, the recommendation strategy associated with the parameter to be tested is adjusted under the condition that the recommendation result to be tested does not meet the expected requirement corresponding to the target test mode, different recommendation performances of the recommendation system are tested respectively through different independent test modes, and the accuracy of the recommendation performance of the recommendation system is improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 shows a flowchart of a method for recommending a system test method according to an embodiment of the present application.
Fig. 2 shows a flowchart of a method for recommending a system test according to another embodiment of the present application.
Fig. 3 is a diagram illustrating an example of a test interface provided in an embodiment of the present application.
Fig. 4 is a diagram illustrating another example of a test interface provided by an embodiment of the present application.
Fig. 5 is a flowchart illustrating a method for recommending a system test according to another embodiment of the present application.
Fig. 6 is a flowchart illustrating a method for recommending a system test method according to still another embodiment of the present application.
Fig. 7 shows a block diagram of a recommendation system testing device according to an embodiment of the present application.
Fig. 8 shows a block diagram of an electronic device according to an embodiment of the present application.
Fig. 9 illustrates a storage unit for storing or carrying program codes for implementing the recommended system test method according to the embodiment of the present application.
Detailed Description
In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application.
The recommendation system is a complex system which helps a user to quickly acquire useful information by applying technologies such as deep learning and neural networks, and models the user portrait by analyzing historical behaviors of the user, so that contents which can meet the interests and requirements of the user are initiatively recommended to the user. The current recommendation system has various products, relates to various industries of clothes and eating houses, and has wide range of relation. For example, a small video or a live broadcast similar to the browsing content of the user may be recommended according to the browsing content of the user, or a short video ID or the like that is of interest to the user may be recommended.
However, as the amount of data of the recommendation system increases greatly, the existing recommendation system cannot accurately recommend content in which the user is interested to the user, and even some frequent pushing may cause disturbance to the user, thereby greatly reducing the user experience.
Aiming at the problems, the inventor finds out through long-term research that the current recommendation system can be upgraded by making a simulation test and a pressure test before the recommendation system is developed and put on line, so that the accuracy of a recommendation result is ensured, and a good recommendation effect and experience are brought to a user. Therefore, the recommendation system testing method and device, the electronic device and the storage medium provided by the embodiment of the application are provided.
It should be noted that, in this embodiment, the test system may be in communication connection with the recommendation system through a recommendation interface of the recommendation system. Optionally, in this embodiment, the debug system (i.e., the testing system) may simulate the recommendation request, and perform data interaction with the recommendation system through an http protocol or an internally defined protocol (e.g., a yy protocol or a thrift rpc protocol) to complete the test. The debug system requests the service of the recommendation system by setting the parameter to be tested and the variation range of the parameter value to be tested, and the returned recommendation result data can be checked and accepted in the debug system.
The debug system in this embodiment can be regarded as a test tool of a recommendation system corresponding to a short video or live broadcast platform, and can be convenient for developers and product operation testers to test and accept the recommendation system that needs to be online, or to perform system anomaly checking, performance testing and the like on the recommendation system.
Embodiments of the present application will be described in detail below with reference to the accompanying drawings.
Referring to fig. 1, a flowchart of a recommendation system testing method provided in an embodiment of the present application is shown, where the embodiment provides a recommendation system testing method applicable to a testing system, the testing system may be used to test recommendation performances such as recommendation accuracy of a recommendation system, and the recommendation system may be used to recommend various contents that a user is interested in (for example, the contents may include live broadcast, anchor broadcast, video, game), and the method includes:
step S110: and acquiring parameters to be tested corresponding to the target test mode.
The target test mode in this embodiment may include a manual test mode, a semi-automatic test mode, and a fully-automatic test mode. After a certain recommendation strategy of the test system is developed, before the recommendation strategy is online to the recommendation system for use, various recommendation performances of the recommendation system comprising the recommendation strategy can be tested through different test modes.
The parameters to be tested are parameters which can be input into the Debug system and are used for testing the recommendation performance of the recommendation system, and the recommendation performance corresponding to different parameters to be tested can be different. Optionally, different test modes may correspond to different parameters to be tested.
As one way, the test parameters configured by the user may be used as the parameters to be tested. For example, in the manual test mode, the user device number, the user ID, the recommended number of requests, and the current page, which are input by the user, may be used as parameters to be tested. As another way, parameters automatically generated by the test system may be used as parameters to be tested. For example, in the semi-automatic mode, the random batch request number of the Debug system can be used as the parameter to be measured.
Step S120: and generating a recommendation request matched with the target test pattern.
As one way, after obtaining the parameters to be tested corresponding to the target test mode, the Debug system may generate a recommendation request matching the target mode, where the recommendation requests matching different target test modes are different. Optionally, the Debug system may generate an http request matching the target test pattern, so as to request the recommendation system to obtain the recommendation result through the http request.
Step S130: and sending the recommendation request carrying the parameter to be tested to a recommendation system.
Optionally, the Debug system may send a recommendation request (i.e., the http request) carrying the parameter to be tested to the recommendation system through a recommendation interface of the recommendation system.
Step S140: and inquiring a recommendation result corresponding to the recommendation request.
Optionally, the recommendation system stores in advance recommendation results adapted to different recommendation requests, and the recommendation system may query the corresponding recommendation results according to the received recommendation requests. Optionally, the recommendation system may receive different recommendation requests at the same time, and query recommendation results corresponding to the different recommendation requests respectively.
Step S150: and returning the recommendation result.
The recommendation system may return the queried recommendation result to the Debug system.
Step S160: and receiving a recommendation result to be tested corresponding to the parameter to be tested, which is returned by the recommendation system.
It can be understood that, in order to verify the accuracy of the recommendation result of the recommendation system or verify whether the recommendation result meets the expected requirement, after receiving the recommendation result corresponding to the parameter to be tested returned by the recommendation system, the Debug system may use the recommendation result as the recommendation result to be tested, so as to verify whether the recommendation result to be tested meets the expected requirement, thereby evaluating the accuracy of the recommendation system or the recommendation performance of other items.
Step S170: and if the recommendation result to be tested does not accord with the expected requirement corresponding to the target test mode, adjusting the recommendation strategy associated with the parameter to be tested.
Optionally, if the recommended result to be tested does not meet the expected requirement corresponding to the target test mode, the recommendation policy associated with the parameter to be tested may be adjusted. Optionally, the recommendation strategies associated with different parameters to be measured may be different. The recommendation strategy associated with the parameter to be measured can be understood as the recommendation strategy corresponding to the parameter to be measured. For example, if the parameter to be measured includes the user ID level, the number or category of anchor or live broadcasts, the recommendation policy may be "the number or category of anchor or live broadcasts respectively pushed by users of different levels". Optionally, the recommendation policy associated with the parameter to be measured may be adjusted by continuously adjusting the parameter configuration of the recommendation policy.
In the recommendation system testing method provided by this embodiment, a recommendation request matched with a target test mode is generated by obtaining a parameter to be tested corresponding to the target test mode, the recommendation request carrying the parameter to be tested is sent to the recommendation system, a recommendation result to be tested corresponding to the parameter to be tested returned by the recommendation system is received, and then a recommendation strategy associated with the parameter to be tested is adjusted if the recommendation result to be tested does not meet an expected requirement corresponding to the target test mode. According to the scheme, the recommendation request matched with the target test mode can be generated after the target test mode and the corresponding to-be-tested parameters are determined by generating the matched recommendation request aiming at different test modes. By sending the recommendation request carrying the parameter to be tested corresponding to the target test mode to the recommendation system and receiving the recommendation result to be tested corresponding to the parameter to be tested returned by the recommendation system, the recommendation strategy associated with the parameter to be tested is adjusted under the condition that the recommendation result to be tested does not meet the expected requirement corresponding to the target test mode, different recommendation performances of the recommendation system are tested respectively through different independent test modes, and the accuracy of the recommendation performance of the recommendation system is improved.
Referring to fig. 2, a flowchart of a recommended system testing method according to another embodiment of the present application is shown, where the present embodiment provides a recommended system testing method applicable to a testing system, and the method includes:
step S210: a first test pattern is obtained.
Optionally, the target test pattern in this embodiment may include a first test pattern. The first test mode may be used as a functional simulation test, i.e. to test whether the recommended result returned by the recommendation system meets the expected requirements.
Step S220: and acquiring the configured parameters to be tested which are adaptive to the test function of the first test mode.
The parameter to be measured in this embodiment may include a first parameter to be measured. Optionally, the first parameter to be tested may include a parameter for testing accuracy of a recommendation function of the recommendation system.
After the first test mode is obtained, the configured parameter to be tested adapted to the test function of the first test mode may be obtained. For example, in a specific application scenario, the first test mode may be a manual test mode in the above embodiment, and the parameter input by the user may be used as the parameter to be tested adapted to the test function of the first test mode. The parameters input by the user may include a user equipment number, a user ID, a request recommendation number, and a current page (optionally, the default page may be 0, that is, the current page is page 0).
Step S230: and taking the parameter to be tested as a first test parameter corresponding to the first test mode.
Optionally, in order to facilitate verification of the recommendation function of the recommendation system, the parameter to be tested may be used as a first test parameter corresponding to the first test mode.
Step S240: and generating a recommendation request matched with the first test pattern.
Optionally, if the current test mode is the first test mode, the Debug system may generate a recommendation request matching the first test mode. Optionally, in the above example as an example, assuming that, among the parameters input by the user, the user device number is "xxx", the user ID is "0", the request recommendation number is "10", and the current page offset is "0", then the recommendation request matched with the first test pattern that is generated may be "requset: xdix & uid & num & 10& offset & 0 ".
Step S250: and sending the recommendation request carrying the first parameter to be tested to a recommendation system, and receiving a recommendation result to be tested corresponding to the first parameter to be tested, which is returned by the recommendation system.
The Debug system can send a recommendation request carrying the first parameter to be tested to the recommendation system, and then receives a recommendation result to be tested corresponding to the first parameter to be tested, which is returned by the recommendation system.
The following describes the above with a specific example: referring to fig. 3, an exemplary diagram of a test interface of a Debug system is shown, which can be used to test different performances of a recommendation system by switching between different test modes. As shown in fig. 3, assuming that the currently selected test mode is the first test mode, the first parameter to be tested is the user equipment number (i.e. hdid in fig. 3), the user ID (i.e. uid in fig. 3), the number of requested recommendations, and the current page, the user ID and the user equipment number may be input, and the recommendation result returned by the recommendation system may be obtained by clicking the query. Optionally, after the query button is clicked, the number of recommended results can be displayed, and if the number of query results is large, the recommended results can be checked by clicking the next page. It can be understood that, if the currently selected test mode changes, both the interface and the content of the input parameter to be tested may change.
Optionally, as shown in fig. 4, assuming that the input user ID is "123" and the user device number is "123123", the query result interface shown in fig. 4 may be obtained after clicking the query, and the obtained number of recommendation results is "46", which are respectively displayed as "recommendation result 1, recommendation result 2, recommendation result 3, and recommendation result 4. Optionally, the recommendation result may include a video ID and video information, for example, the video information may include live room related information of the anchor, anchor cover page information, or an anchor live source, and the like. The recommendation result arranged in the first is the ID of the first anchor or the live ID. It should be noted that the above examples and specific numerical values are only described as examples, and do not limit the present solution, and the values of the parameters to be measured may be adjusted according to actual situations in actual implementation.
Optionally, if the recommendation result 1 is live broadcast information, by clicking the recommendation result, the user can jump to a live broadcast room to see live broadcast, so as to verify whether the recommendation effect meets expectations.
Step S260: and if the recommendation result to be tested does not meet the expected requirement corresponding to the first test mode, adjusting the recommendation strategy associated with the first parameter to be tested.
Optionally, recommendation strategies associated with different recommendation parameters to be tested may be different.
As one way, if the recommended result to be tested does not meet the expected requirement corresponding to the first test mode, the recommended strategy associated with the first parameter to be tested may be adjusted.
In the method for testing the recommendation system, the first to-be-tested parameter corresponding to the first test mode is obtained, the recommendation request matched with the first test mode is generated, the recommendation request carrying the first to-be-tested parameter is sent to the recommendation system, the to-be-tested recommendation result corresponding to the first to-be-tested parameter returned by the recommendation system is received, and then the recommendation strategy associated with the first to-be-tested parameter is adjusted if the to-be-tested recommendation result does not meet the expected requirement corresponding to the first test mode. According to the method and the device, the recommendation request carrying the first to-be-tested parameter corresponding to the first test mode is sent to the recommendation system, the to-be-tested recommendation result corresponding to the first to-be-tested parameter returned by the recommendation system is received, and the recommendation strategy associated with the first to-be-tested parameter is adjusted under the condition that the to-be-tested recommendation result does not meet the expected requirement corresponding to the first test mode, so that the accuracy of data returned by the recommendation system is tested in a functional simulation test mode, and the accuracy of the recommendation performance of the recommendation system is improved.
Referring to fig. 5, a flowchart of a recommended system testing method according to another embodiment of the present application is shown, where the present embodiment provides a recommended system testing method applicable to a testing system, and the method includes:
step S310: a second test pattern is obtained.
Optionally, the target test pattern in this embodiment may include a second test pattern. The second test mode may be used to test the compression resistance (i.e., compression testing) of the recommended function of the recommendation system. For example, if qps (query rate per second) of the recommendation system is preset, assuming that 1000 requests are available in 1s, a success rate of 100ms can be obtained by counting how many 1000 requests are returned within 100ms, that is, the compression resistance of the recommendation system can be tested by testing "how many 1000 requests are returned within 100 ms".
Step S320: and acquiring the configured parameters to be tested which are adaptive to the test function of the second test mode.
The parameter to be tested in this embodiment may include a second parameter to be tested, and the second parameter to be tested may include a parameter for testing the compression resistance of the recommendation function of the recommendation system. If the currently selected test mode is the second test mode, the configured parameter to be tested which is adaptive to the test function of the second test mode can be obtained.
Step S330: and taking the parameter to be tested as a second test parameter corresponding to the second test mode.
Optionally, in order to verify the compression resistance of the recommendation system, the parameter to be tested may be used as a second test parameter corresponding to the second test mode.
Step S340: and generating a recommendation request matched with the second test pattern.
Similarly, if the current test pattern is the second test pattern, the Debug system may generate a recommendation request matching the second test pattern.
Step S350: and sending the recommendation request carrying the second parameter to be tested to a recommendation system, and receiving a recommendation result to be tested corresponding to the second parameter to be tested, which is returned by the recommendation system.
The Debug system can send a recommendation request carrying the second parameter to be tested to the recommendation system, and then receive a recommendation result to be tested corresponding to the second parameter to be tested, which is returned by the recommendation system.
Step S360: and if the recommendation result to be tested does not meet the expected requirement corresponding to the second test mode, adjusting the recommendation strategy associated with the second parameter to be tested.
If the recommended result to be tested does not meet the expected requirement corresponding to the second test mode, the recommendation strategy associated with the second parameter to be tested can be adjusted.
It should be noted that the second test mode is a semi-manual and semi-automatic mode. After the pressure measurement parameter (i.e., the second parameter to be measured) is set, the Debug system can automatically capture the online input parameter, capture the user ID and the user equipment number ID, and randomly obtain the current page offset and the request recommendation number (which may be 10 or 20), thereby implementing automatic testing. Then, the consumption conditions of the single-click CPU and the memory, the delay conditions and the like are output, and the recommendation result data can be automatically counted for data analysis.
The performance of the current recommendation system is evaluated through the pressure test under the line, the recommendation system can meet the on-line throughput and stability, and if the performance is not enough, capacity expansion preparation can be made in advance. Alternatively, the performance may be judged to be sufficient by comparing whether there is a drop in the same request success rate of qps. Alternatively, if the same request success rate of qps has not dropped, then performance may be deemed adequate, otherwise performance may not be adequate. Alternatively, the determination of adequate performance may be made by determining qps whether a predetermined threshold is met. For example, if 99.999% of requests that can handle 2w/s can be returned at 100ms, then performance may be deemed sufficient, and vice versa.
In the method for testing the recommendation system, the second parameter to be tested corresponding to the second test mode is obtained, the recommendation request matched with the second test mode is generated, the recommendation request carrying the second parameter to be tested is sent to the recommendation system, the recommendation result to be tested corresponding to the second parameter to be tested, which is returned by the recommendation system, is received, and then the recommendation strategy associated with the second parameter to be tested is adjusted if the recommendation result to be tested does not meet the expected requirement corresponding to the second test mode. According to the scheme, the compression resistance of the recommendation system is detected through the simulated pressure test, and the bottleneck of the recommendation system is observed.
Referring to fig. 6, a flowchart of a recommended system testing method according to still another embodiment of the present application is shown, where the present embodiment provides a recommended system testing method applicable to a testing system, and the method includes:
step S410: a third test pattern is obtained.
Optionally, the target test pattern in this embodiment may include a third test pattern. The third test mode may be used to automatically verify the overall operational performance of the recommendation system.
Step S420: and acquiring the configured to-be-tested parameters adaptive to the test function of the third test mode.
The parameter to be measured in this embodiment may include a third parameter to be measured. Optionally, the third parameter to be tested may include a parameter for testing the operation performance of the recommendation system. After the third test mode is obtained, the configured parameter to be tested adapted to the test function of the third test mode may be obtained.
Step S430: and taking the parameter to be tested as a third test parameter corresponding to the third test mode.
Optionally, in order to facilitate verifying the operation performance of the recommendation system, the parameter to be tested may be used as a third test parameter corresponding to the third test mode.
Step S440: and generating a recommendation request matched with the third test pattern.
Optionally, if the current test mode is the third test mode, the Debug system may generate a recommendation request matching the third test mode.
Step S450: and sending the recommendation request carrying the third parameter to be tested to a recommendation system, and receiving a recommendation result to be tested corresponding to the third parameter to be tested, which is returned by the recommendation system.
The Debug system can send a recommendation request carrying the third parameter to be tested to the recommendation system, and then receive a recommendation result to be tested corresponding to the third parameter to be tested, which is returned by the recommendation system.
Step S460: and if the running log does not meet the target condition, judging that the recommendation result to be tested does not meet the expected requirement corresponding to the third test mode, and adjusting the recommendation strategy associated with the third parameter to be tested.
If the operation log of the recommendation system obtained by the test does not meet the target condition, the recommendation result to be tested can be judged not to meet the expected requirement corresponding to the target test mode, and then the recommendation strategy associated with the third parameter to be tested can be adjusted.
In this embodiment, a daily self-check program may be set for the timing task. By acquiring random user equipment ID, user ID, current page and request recommendation number, the live broadcast room can be automatically clicked, online running conditions can be tested, and therefore the whole process is verified and daily reports are generated. Optionally, whether the recommendation system operates normally or not can be verified according to the captured recommendation result, and a wireless bug is tested, so that the operation performance of the recommendation system can be monitored. For example, if a live broadcast that is simulated to be on is not on, it is considered that an online bug exists. Optionally, if there is an on-line bug, an email may be automatically sent to notify the developer, so as to check the error in time.
It should be noted that, in the embodiment of the present application, the corresponding parameter to be tested may be obtained according to the selected test mode as described in the foregoing embodiment, or different test modes may be integrated into one test function module, and the test function module may automatically identify the recommended performance to be tested at present according to the input parameter to be tested, and then jump to the corresponding test mode to complete the test. Optionally, the test result may be displayed in a visual manner, and the specific display manner may not be limited.
In the method for testing the recommendation system provided by this embodiment, a recommendation request matched with the third test mode is generated by obtaining a third parameter to be tested corresponding to the third test mode, the recommendation request carrying the third parameter to be tested is sent to the recommendation system, a recommendation result to be tested corresponding to the third parameter to be tested returned by the recommendation system is received, and then if the operation log of the recommendation system does not satisfy the target condition, it is determined that the recommendation result to be tested does not satisfy the expected requirement corresponding to the third test mode, and a recommendation strategy associated with the third parameter to be tested is adjusted. According to the scheme, whether the recommendation result of the recommendation system meets a normal range or not can be detected in time through daily self-checking of an online system, so that if problems exist, the problems can be checked in time, and the stability and accuracy of the recommendation system are improved.
Referring to fig. 7, a block diagram of a recommendation system testing apparatus according to an embodiment of the present application is shown, where the embodiment provides a recommendation system testing apparatus 500, and the apparatus 500 operates in a recommendation system, and includes: the obtaining module 510, the first processing module 520, the second processing module 530, and the third processing module 540:
an obtaining module 510, configured to obtain a parameter to be tested corresponding to the target test mode.
Optionally, the parameter to be tested includes a first parameter to be tested, the first parameter to be tested includes a parameter for testing accuracy of a recommendation function of the recommendation system, and the target test mode includes a first test mode. In this manner, the obtaining module 510 may be specifically configured to obtain the first test pattern; acquiring a configured parameter to be tested which is adaptive to the test function of the first test mode; and taking the parameter to be tested as a first test parameter corresponding to the first test mode.
Optionally, the parameter to be tested includes a second parameter to be tested, the second parameter to be tested includes a parameter for testing the compression resistance of the recommendation function of the recommendation system, and the target test mode includes a second test mode. In this manner, the obtaining module 510 may be specifically configured to obtain the second test pattern; acquiring a configured parameter to be tested which is adaptive to the test function of the second test mode; and taking the parameter to be tested as a second test parameter corresponding to the second test mode.
Optionally, the parameters to be tested include a third parameter to be tested, the third parameter to be tested includes a parameter for testing the operation performance of the recommendation system, and the target test mode includes a third test mode. In this manner, the obtaining module 510 may be specifically configured to obtain the third test pattern; acquiring a configured parameter to be tested which is adaptive to the test function of the third test mode; and taking the parameter to be tested as a third test parameter corresponding to the third test mode.
The first processing module 520 is configured to generate a recommendation request matching the target test pattern.
The second processing module 530 is configured to send the recommendation request carrying the parameter to be tested to a recommendation system, and receive a recommendation result to be tested corresponding to the parameter to be tested, where the recommendation result is returned by the recommendation system.
The third processing module 540 is configured to adjust the recommendation policy associated with the parameter to be tested if the recommendation result to be tested does not meet the expected requirement corresponding to the target test mode.
Optionally, if the recommendation result to be tested does not meet the expected requirement corresponding to the first test mode, adjusting the recommendation strategy associated with the first parameter to be tested, where recommendation strategies associated with different parameters to be tested are different.
Optionally, if the recommended result to be tested does not meet the expected requirement corresponding to the second test mode, the recommendation strategy associated with the second parameter to be tested is adjusted.
Optionally, if the running log does not satisfy the target condition, determining that the recommended result to be tested does not meet the expected requirement corresponding to the third test mode, and adjusting the recommendation strategy associated with the third parameter to be tested.
The recommendation system testing device provided by this embodiment generates a recommendation request matched with a target test mode by obtaining a parameter to be tested corresponding to the target test mode, sends the recommendation request carrying the parameter to be tested to a recommendation system, receives a recommendation result to be tested corresponding to the parameter to be tested returned by the recommendation system, and adjusts a recommendation strategy associated with the parameter to be tested if the recommendation result to be tested does not meet an expected requirement corresponding to the target test mode. According to the scheme, the recommendation request matched with the target test mode can be generated after the target test mode and the corresponding to-be-tested parameters are determined by generating the matched recommendation request aiming at different test modes. By sending the recommendation request carrying the parameter to be tested corresponding to the target test mode to the recommendation system and receiving the recommendation result to be tested corresponding to the parameter to be tested returned by the recommendation system, the recommendation strategy associated with the parameter to be tested is adjusted under the condition that the recommendation result to be tested does not meet the expected requirement corresponding to the target test mode, different recommendation performances of the recommendation system are tested respectively through different independent test modes, and the accuracy of the recommendation performance of the recommendation system is improved.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described apparatuses and modules may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, the coupling or direct coupling or communication connection between the modules shown or discussed may be through some interfaces, and the indirect coupling or communication connection between the devices or modules may be in an electrical, mechanical or other form.
In addition, functional modules in the embodiments of the present application may be integrated into one processing module, or each of the modules may exist alone physically, or two or more modules are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode.
Referring to fig. 8, based on the recommended system testing method and apparatus, an embodiment of the application further provides an electronic device 100 capable of executing the recommended system testing method. The electronic device 100 includes a memory 102 and one or more processors 104 (only one shown) coupled to each other, the memory 102 and the processors 104 being communicatively coupled to each other. The memory 102 stores therein a program that can execute the contents of the foregoing embodiments, and the processor 104 can execute the program stored in the memory 102.
The processor 104 may include one or more processing cores, among other things. The processor 104 interfaces with various components throughout the electronic device 100 using various interfaces and circuitry to perform various functions of the electronic device 100 and process data by executing or executing instructions, programs, code sets, or instruction sets stored in the memory 102 and invoking data stored in the memory 102. Alternatively, the processor 104 may be implemented in hardware using at least one of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 104 may integrate one or more of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a modem, and the like. Wherein, the CPU mainly processes an operating system, a user interface, an application program and the like; the GPU is used for rendering and drawing display content; the modem is used to handle wireless communications. It is understood that the modem may not be integrated into the processor 104, but may be implemented by a communication chip.
The Memory 102 may include a Random Access Memory (RAM) or a Read-Only Memory (Read-Only Memory). The memory 102 may be used to store instructions, programs, code sets, or instruction sets. The memory 102 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the foregoing embodiments, and the like. The data storage area may also store data created by the electronic device 100 during use (e.g., phone book, audio-video data, chat log data), and the like.
Referring to fig. 9, a block diagram of a computer-readable storage medium according to an embodiment of the present application is shown. The computer-readable storage medium 600 has stored therein program code that can be called by a processor to execute the method described in the above-described method embodiments.
The computer-readable storage medium 600 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read only memory), an EPROM, a hard disk, or a ROM. Alternatively, the computer-readable storage medium 600 includes a non-transitory computer-readable storage medium. The computer readable storage medium 600 has storage space for program code 610 for performing any of the method steps of the method described above. The program code can be read from or written to one or more computer program products. The program code 610 may be compressed, for example, in a suitable form.
In the description herein, reference to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not necessarily depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.
Claims (10)
1. A recommendation system testing method, the method comprising:
acquiring a parameter to be tested corresponding to a target test mode;
generating a recommendation request matched with the target test pattern;
sending the recommendation request carrying the parameter to be tested to a recommendation system, and receiving a recommendation result to be tested corresponding to the parameter to be tested, which is returned by the recommendation system;
and if the recommendation result to be tested does not accord with the expected requirement corresponding to the target test mode, adjusting the recommendation strategy associated with the parameter to be tested.
2. The method of claim 1, wherein the parameter to be tested comprises a first parameter to be tested, the first parameter to be tested comprises a parameter for testing accuracy of a recommended function of the recommendation system, the target test mode comprises a first test mode, and the obtaining the parameter to be tested corresponding to the target test mode comprises:
acquiring a first test mode;
acquiring a configured parameter to be tested which is adaptive to the test function of the first test mode;
and taking the parameter to be tested as a first test parameter corresponding to the first test mode.
3. The method of claim 2, wherein the adjusting the recommendation policy associated with the parameter under test if the recommendation result under test does not meet the expected requirement corresponding to the target test mode comprises:
and if the recommendation result to be tested does not meet the expected requirement corresponding to the first test mode, adjusting the recommendation strategy associated with the first parameter to be tested, wherein the recommendation strategies associated with different recommendation parameters to be tested are different.
4. The method according to claim 1, wherein the parameters to be tested comprise a second parameter to be tested, the second parameter to be tested comprises a parameter for testing a compressive resistance of a recommended function of the recommendation system, the target test mode comprises a second test mode, and the obtaining of the parameters to be tested corresponding to the target test mode comprises:
acquiring a second test mode;
acquiring a configured parameter to be tested which is adaptive to the test function of the second test mode;
and taking the parameter to be tested as a second test parameter corresponding to the second test mode.
5. The method of claim 4, wherein the adjusting the recommendation policy associated with the parameter under test if the recommendation result under test does not meet the expected requirement corresponding to the target test mode comprises:
and if the recommendation result to be tested does not meet the expected requirement corresponding to the second test mode, adjusting the recommendation strategy associated with the second parameter to be tested.
6. The method of claim 1, wherein the parameters to be tested comprise a third parameter to be tested, the third parameter to be tested comprises a parameter for testing the operation performance of the recommendation system, the target test mode comprises a third test mode, and the obtaining the parameters to be tested corresponding to the target test mode comprises:
acquiring a third test mode;
acquiring a configured parameter to be tested which is adaptive to the test function of the third test mode;
and taking the parameter to be tested as a third test parameter corresponding to the third test mode.
7. The method of claim 6, wherein the adjusting the recommendation policy associated with the parameter under test if the recommendation result under test does not meet the expected requirement corresponding to the target test mode comprises:
and if the running log does not meet the target condition, judging that the recommendation result to be tested does not meet the expected requirement corresponding to the third test mode, and adjusting the recommendation strategy associated with the third parameter to be tested.
8. A recommendation system testing apparatus, the apparatus comprising:
the acquisition module is used for acquiring parameters to be tested corresponding to the target test mode;
the first processing module is used for generating a recommendation request matched with the target test pattern;
the second processing module is used for sending the recommendation request carrying the parameter to be tested to a recommendation system and receiving a recommendation result to be tested corresponding to the parameter to be tested, which is returned by the recommendation system;
and the third processing module is used for adjusting the recommendation strategy associated with the parameter to be tested if the recommendation result to be tested does not accord with the expected requirement corresponding to the target test mode.
9. An electronic device, comprising a memory;
one or more processors;
one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to perform the method of any of claims 1-7.
10. A computer-readable storage medium, having program code stored therein, wherein the program code when executed by a processor performs the method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911319730.7A CN111190801A (en) | 2019-12-19 | 2019-12-19 | Recommendation system testing method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911319730.7A CN111190801A (en) | 2019-12-19 | 2019-12-19 | Recommendation system testing method and device and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111190801A true CN111190801A (en) | 2020-05-22 |
Family
ID=70707476
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911319730.7A Pending CN111190801A (en) | 2019-12-19 | 2019-12-19 | Recommendation system testing method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111190801A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115392592A (en) * | 2022-10-25 | 2022-11-25 | 浪潮电子信息产业股份有限公司 | Storage product parameter configuration recommendation method, device, equipment and medium |
| WO2023115831A1 (en) * | 2021-12-23 | 2023-06-29 | 北京百度网讯科技有限公司 | Application testing method and apparatus, electronic device and storage medium |
| JP7511036B2 (en) | 2021-12-23 | 2024-07-04 | ベイジン バイドゥ ネットコム サイエンス テクノロジー カンパニー リミテッド | Application testing method, device, electronic device, and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120284069A1 (en) * | 2011-05-04 | 2012-11-08 | Sony Corporation | Method for optimizing parameters in a recommendation system |
| CN106817296A (en) * | 2017-01-12 | 2017-06-09 | 微梦创科网络科技（中国）有限公司 | The method of testing of information recommendation, device and electronic equipment |
| CN107562632A (en) * | 2017-09-12 | 2018-01-09 | 北京奇艺世纪科技有限公司 | For the A/B method of testings and device of Generalization bounds |
| CN110399290A (en) * | 2019-06-18 | 2019-11-01 | 深圳壹账通智能科技有限公司 | Test device, method and the storage medium of message push |
| CN110415101A (en) * | 2019-06-19 | 2019-11-05 | 深圳壹账通智能科技有限公司 | Products Show test method, device, computer equipment and storage medium |
-
2019
- 2019-12-19 CN CN201911319730.7A patent/CN111190801A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120284069A1 (en) * | 2011-05-04 | 2012-11-08 | Sony Corporation | Method for optimizing parameters in a recommendation system |
| CN106817296A (en) * | 2017-01-12 | 2017-06-09 | 微梦创科网络科技（中国）有限公司 | The method of testing of information recommendation, device and electronic equipment |
| CN107562632A (en) * | 2017-09-12 | 2018-01-09 | 北京奇艺世纪科技有限公司 | For the A/B method of testings and device of Generalization bounds |
| CN110399290A (en) * | 2019-06-18 | 2019-11-01 | 深圳壹账通智能科技有限公司 | Test device, method and the storage medium of message push |
| CN110415101A (en) * | 2019-06-19 | 2019-11-05 | 深圳壹账通智能科技有限公司 | Products Show test method, device, computer equipment and storage medium |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023115831A1 (en) * | 2021-12-23 | 2023-06-29 | 北京百度网讯科技有限公司 | Application testing method and apparatus, electronic device and storage medium |
| JP7511036B2 (en) | 2021-12-23 | 2024-07-04 | ベイジン バイドゥ ネットコム サイエンス テクノロジー カンパニー リミテッド | Application testing method, device, electronic device, and storage medium |
| CN115392592A (en) * | 2022-10-25 | 2022-11-25 | 浪潮电子信息产业股份有限公司 | Storage product parameter configuration recommendation method, device, equipment and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110275958B (en) | Website information identification method and device and electronic equipment | |
| CN102231131B (en) | Testing software in electronic device | |
| KR102058976B1 (en) | Application method and device | |
| CN110413510B (en) | Data processing method, device and equipment | |
| CN107967206B (en) | Data test system, request interception method and request interception device | |
| CN109167812B (en) | Method for evaluating service quality and determining adjustment strategy, server and storage medium | |
| CN113515837B (en) | Method and device for establishing simulation test platform and electronic equipment | |
| CN109583561A (en) | A kind of the activation amount quantization method and device of deep neural network | |
| CN107807841B (en) | Server simulation method, device, equipment and readable storage medium | |
| CN111162934A (en) | Business service test method and device, storage medium and electronic device | |
| CN111190801A (en) | Recommendation system testing method and device and electronic equipment | |
| CN111611140B (en) | Report verification method and device for buried point data, electronic equipment and storage medium | |
| CN109739768A (en) | Search engine evaluating method, device, equipment and readable storage medium storing program for executing | |
| WO2021104387A1 (en) | Method for automatically identifying valid data acquisition module and system | |
| CN111652383A (en) | Data contribution degree evaluation method, device, equipment and storage medium | |
| CN107423176B (en) | Memory test method, control terminal, server and system | |
| CN111736951A (en) | Simulation method for automatic driving, computer device, and storage medium | |
| CN105279061A (en) | Server interface concurrent testing method and apparatus | |
| CN109491702B (en) | Optimization scheme determination method and device, terminal equipment and storage medium | |
| CN112799955B (en) | Method and device for detecting model change, storage medium and electronic equipment | |
| CN202799166U (en) | Mobile application network testing system | |
| CN110688319A (en) | Application keep-alive capability test method and related device | |
| CN110598797B (en) | Fault detection method and device, storage medium and electronic device | |
| CN111176965B (en) | Recommendation system pre-release test method and device and electronic equipment | |
| CN112434717A (en) | Model training method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20210120 Address after: 511442 3108, 79 Wanbo 2nd Road, Nancun Town, Panyu District, Guangzhou City, Guangdong Province Applicant after: GUANGZHOU CUBESILI INFORMATION TECHNOLOGY Co.,Ltd. Address before: 511400 24th floor, building B-1, North District, Wanda Commercial Plaza, Wanbo business district, No.79 Wanbo 2nd Road, Nancun Town, Panyu District, Guangzhou, Guangdong Province Applicant before: GUANGZHOU HUADUO NETWORK TECHNOLOGY Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20200522 Assignee: GUANGZHOU HUADUO NETWORK TECHNOLOGY Co.,Ltd. Assignor: GUANGZHOU CUBESILI INFORMATION TECHNOLOGY Co.,Ltd. Contract record no.: X2021440000054 Denomination of invention: Recommended system test method, device and electronic equipment License type: Common License Record date: 20210208 |
|
| EE01 | Entry into force of recordation of patent licensing contract |