CN111190815B - Recall test method, apparatus, electronic device and storage medium - Google Patents
Recall test method, apparatus, electronic device and storage medium Download PDFInfo
- Publication number
- CN111190815B CN111190815B CN201911320857.0A CN201911320857A CN111190815B CN 111190815 B CN111190815 B CN 111190815B CN 201911320857 A CN201911320857 A CN 201911320857A CN 111190815 B CN111190815 B CN 111190815B
- Authority
- CN
- China
- Prior art keywords
- recall
- test
- channel
- target
- channels
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The embodiment of the application discloses a recall test method and device, electronic equipment and a storage medium, and relates to the technical field of computers. The method comprises the following steps: receiving a test request, wherein the test request carries a test strategy; responding to the test request, and determining a target recall channel from a plurality of recall channels according to a test strategy; recalling the content data corresponding to the test request from the target recall channel, and acquiring performance data of the target recall channel in the recall process of the content data; and acquiring a performance test result of the target recall channel based on the performance data. Based on the method, the application can independently test one or more of the plurality of recall channels, so that the flexibility and comprehensiveness of the test are improved.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a recall test method and apparatus, an electronic device, and a storage medium.
Background
With the prevalence of short videos in networks, recommendation techniques for short videos are also developing faster and faster. The performance test of the recall channel becomes particularly important because current video recommendation systems generally require that short videos be recalled through the recall channel.
However, performance testing for the recall channel is now relatively straightforward.
Disclosure of Invention
In view of the foregoing problems, the present application provides a recall test method, apparatus, electronic device and storage medium to solve the foregoing problems.
In a first aspect, an embodiment of the present application provides a recall test method, where the method includes: receiving a test request, wherein the test request carries a test strategy; responding to the test request, and determining a target recall channel from a plurality of recall channels according to a test strategy; recalling the content data corresponding to the test request from the target recall channel, and acquiring performance data of the target recall channel in the recalling process of the content data; and acquiring a performance test result of the target recall channel based on the performance data.
In a second aspect, an embodiment of the present application provides a recall test apparatus, including: the device comprises a receiving module, a target recall channel determining module, a testing module and a performance testing result acquiring module. The receiving module is used for receiving a test request, and the test request carries a test strategy. And the target recall channel determining module is used for responding to the test request and determining a target recall channel from the plurality of recall channels according to the test strategy. The test module is used for recalling the content data corresponding to the test request from the target recall channel and acquiring the performance data of the target recall channel in the recalling process of the content data. The performance test result acquisition module is used for acquiring a performance test result of the target recall channel based on the performance data.
In a third aspect, embodiments of the present application provide an electronic device, including one or more processors, a memory, and one or more programs, where the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to perform the recall test method described above.
In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a program code is stored, and the program code can be called by a processor to execute the recall test method.
According to the recall test method, the recall test device, the electronic equipment and the storage medium, the test request is received, the test request carries the test strategy, the test request is responded, and the target recall channel is determined from the plurality of recall channels according to the test strategy, so that one or more recall channels needing to be tested can be accurately selected from the plurality of recall channels. And recalling the content data corresponding to the test request from the target recall channel, acquiring performance data of the target recall channel in the recalling process of the content data, and finally acquiring a performance test result of the target recall channel based on the performance data, so that the performance test result of the recall channel selected according to the test strategy can be obtained, testing of a single or multiple recall channels can be realized based on different test strategies, different test results are obtained, and the flexibility of testing of the recall channel is improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 shows a schematic diagram of an application environment according to an embodiment of the present application.
FIG. 2 shows a schematic structural diagram of a recall module according to an embodiment of the present application.
FIG. 3 shows a flow diagram of a recall test method according to one embodiment of the present application.
Fig. 4 is a schematic diagram illustrating a connection relationship between a video recommendation system and a pressure measurement system according to an embodiment of the present application.
FIG. 5 illustrates a flow diagram of a recall test method according to another embodiment of the present application.
FIG. 6 illustrates a flow diagram of a recall test method according to yet another embodiment of the present application.
FIG. 7 illustrates a method flow diagram according to one embodiment of step S360 of the recall test method illustrated in FIG. 5 of the present application.
FIG. 8 is a flow chart of a method according to another embodiment of step S360 of the recall test method of FIG. 5 of the present application.
FIG. 9 is a flowchart of a method according to another embodiment of step S360 of the recall test method of FIG. 5 of the present application.
FIG. 10 illustrates a functional block diagram of a recall test provided by an embodiment of the present application.
Fig. 11 shows a block diagram of an electronic device according to an embodiment of the present application.
Fig. 12 is a storage medium storing or carrying program code implementing a recall test method according to an embodiment of the present application.
Detailed Description
In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application.
With the rapid development of the internet, watching short videos on the network has become a common habit of people in daily life. The current video recommendation system recommends short videos which are relatively concerned or interested to users, and recalling the videos is a very critical step in the process of recommending the videos to the users. The recall of the video is generally implemented by a recall module configured in the video recommendation system, where the recall module corresponds to one or more recall channels, and the recommended video is recalled from the recall channels to the video recommendation system and then recommended to the user by the video recommendation system. Therefore, the performance of the recall channel directly determines the video recommendation quality, so that the performance test of the recall channel is crucial.
However, the inventor found that since different recall channels in a recall module are usually developed by different developers, each recall channel in the recall module cannot be tested individually, and only the performance of multiple recall channels of the recall module can be tested as a whole, which results in a single test result, and only the performance of multiple recall channels can be known, but the performance of single or multiple recall channels cannot be known.
Therefore, in view of the above problems, the inventor proposes a recall test method, apparatus, electronic device and storage medium in the embodiments of the present application, which may be configured to send a test request to a recall module in a recommendation system, where the test request carries a test policy, so that the recall module determines a target recall channel to be tested from a plurality of recall channels according to the test policy to perform a test, so as to perform an individual test on one or more recall channels of the plurality of recall channels, thereby avoiding a problem that a test result is single due to an existing test that only a plurality of recall channels can be integrally tested, and improving flexibility and comprehensiveness of the test.
An application environment of the recall test method provided by the embodiment of the present application is described below.
Referring to fig. 1, fig. 1 shows a video recommendation system and a database 6 for providing recommended videos for the video recommendation system, wherein the video recommendation system may include an experiment module 1, a scheduling module 2, a recall module 3, a ranking module 4, and a mechanism module 5. The experiment module 1 is used for providing experiment configuration data of each module on the line. The scheduling module 2 is used for scheduling the recall module 3, the sorting module 4, the mechanism module 5 and the like, and finally returning the scheduling result to the caller. The recall module 3 is used for recalling the video list from the database 6 and performing rough ranking on the video list. The sorting module 4 is used for performing personalized sorting on the recalled results. The mechanism module 5 is used to add some artificial rules to the sorted results.
As shown in fig. 2, the recall module 3 may be composed of a plurality of recall channels, and each recall channel may be different from each other, for example, in the short video recommendation system, the recall module 3 may include a popular short video recall channel, a personalized short video recall channel, and the like.
Referring to fig. 3, fig. 3 is a flowchart illustrating a recall test method according to an embodiment of the present application, where the method may be applied to an electronic device, where the electronic device may be a networking-enabled electronic device, and the recommendation system shown in fig. 2 may be configured in the electronic device. As shown in fig. 4, the recommendation system may be communicatively connected to a pressure test system 7 and a database 6, respectively, for testing the recall module 3 in the recommendation system. Optionally, the electronic device includes, but is not limited to: smart phones, tablet computers, personal computers, e-book readers, audio players, video players, and the like.
The recall test method may include:
s110, receiving a test request, wherein the test request carries a test strategy.
Wherein the test request may be for recalling content data corresponding to the test request from the recall channel. The testing policy may be used to determine which recall channels from the plurality of recall channels are recall channels that need to be tested, or the number of recall channels that need to be tested. It will be appreciated that the test request may be a request according to a protocol and the test policy may be a field in the test request.
In some embodiments, the pressure testing system 7 may send a test request to the recall module 3, where the recall module 3 may receive the test request and recall content data corresponding to the test request from a different recall channel according to the test request. As an example, for example, the test request may include an Identity Document (ID), and then a video previously associated with the user may be recalled from a different recall channel according to the ID of the user, specifically, a browsing record of the video may be associated according to the ID of the user, and a video of interest to the user may be recalled according to the browsing record. Also for example, the test request may include a search keyword, and the recall module 3 may recall news, articles, videos, and the like associated with the keyword from different recall channels according to the keyword.
As an example, for example, the pressure testing system 7 sends a test request to the recall module 3, and the recall module 3 returns a result to be recommended, specifically, each recall channel corresponding to the recall module 3 recalls content that may be of interest to the corresponding user. For example, an account 123 requests to pull 10 results, the account 123 just sees a video ID which may be denoted as vidxx, and the pull result may be obtained by the following codes:
getRecommend？
flag＝2&uid＝123&getnum＝10&last＝vidxx；
getResponse：vid1：info1；vid2：info2；……vid10：info10；
based on this code, the user can view vid1, vid2 to vid10 by refreshing the video list through account 123. Wherein each recall channel gives 1-N results.
In other embodiments, the test request may further include a recall number for the content data, and as an example, the test request may include a keyword and a recall number, specifically, for example, the keyword is "make" and the recall number is 10, and then 10 pieces of content data related to "make" may be recalled from the recall channel according to the test request. Wherein, the 10 pieces of content data can be recalled from different channels.
Alternatively, the content data may include articles, news, videos, merchandise information, and the like.
And S120, responding to the test request, and determining a target recall channel from the plurality of recall channels according to the test strategy.
In some embodiments, the recall module 3 responds to the test request, and determines one or more recall channels to be tested as target recall channels from a plurality of recall channels corresponding to the recall module 3 according to a test policy carried in the test request. As an example, for example, each of the plurality of recall channels corresponding to the recall module 3 is marked with a serial number, and the test policy may include the serial number of the recall channel to be tested, so that one or more recall channels to be tested may be determined from the plurality of recall channels as the target recall channel according to the serial number in the test policy. As another example, the test policy may include the number of recall channels that need to be tested each time the test is performed, so that the number of recall channels corresponding to the test policy may be selected from the plurality of recall channels as the target recall channel according to the test policy.
And S130, recalling the content data corresponding to the test request from the target recall channel, and acquiring the performance data of the target recall channel in the recall process of the content data.
The performance data may include a response time length of the target recall channel to the test request, and may also include memory occupancy recorded by the target recall channel in the recall process of the content data. It can be understood that the memory occupancy rate in this embodiment refers to a Central Processing Unit (CPU) memory occupancy rate corresponding to the recall channel.
In some embodiments, the test request may include screening condition information for the content data, such as the number of content data to be recalled, the subject of the content data to be recalled, the format of the content data to be recalled, and the like. As an example, if the screening condition information of the test request includes 10 recalls, the subject is classical, and the format is audio, the recall module 3 may recall 10 audios, the subject is classical, from the target recall channel as the content data corresponding to the test request according to the test request. In the process of recalling content data, the state parameter of the target recall channel (such as the memory occupancy rate of the CPU) may also change due to the recall action made by the target recall channel, so the changed state parameter may be recorded as the performance parameter of the target recall channel, that is, the performance parameter is obtained, for example, when the target recall channel does not make a recall action, the memory occupancy rate of the corresponding CPU is 10%, and when the target recall channel makes a recall action, the memory occupancy rate of the corresponding CPU changes from 10% to 25%, then the memory occupancy rate of the 25% CPU may be used as the performance parameter of the target recall channel.
And S140, acquiring a performance test result of the target recall channel based on the performance data.
The performance test result can represent whether the performance and the stability of the target recall channel are in a normal state or not. Specifically, the performance test result of the recall channel can be determined according to the response time, the memory occupancy rate and other performance data. Alternatively, the performance test result may be obtained by determining whether the performance data exceeds the performance data threshold, for example, when the response time length exceeds a preset time length, the performance test result of the target recall channel may be determined to be abnormal, and when the response time length does not exceed the preset time length, the performance test result of the target recall channel may be determined to be normal.
In some embodiments, the performance test results may also include a score made on the performance of the targeted recall channel based on the performance data. As one example, for example, when the performance data includes a response duration, the shorter the response duration, the higher the score corresponding to the performance of the targeted recall channel. For example, when the performance data includes memory occupancy, the smaller the memory occupancy, the higher the score corresponding to the performance of the target recall channel. In some embodiments, the performance data may include a response duration and a memory occupancy rate, and the score corresponding to the performance of the recall target channel may be determined by both the response duration and the memory occupancy rate. Optionally, the scoring weight corresponding to the response duration and the scoring weight corresponding to the memory occupancy rate may be set according to the actual requirement, for example, the scoring weight corresponding to the response duration and the scoring weight corresponding to the memory occupancy rate may both be 50%.
In this embodiment, a test request is received by the recall module 3, where the test request carries a test policy, and then the test request is responded, and a target recall channel is determined from a plurality of recall channels according to the test policy, so that one or more recall channels to be tested can be accurately selected from the plurality of recall channels. And recalling the content data corresponding to the test request from the target recall channel, acquiring the performance data of the target recall channel in the recall process of the content data, and finally acquiring the performance test result of the target recall channel based on the performance data, so that the performance test result of the recall channel selected according to the test strategy can be obtained, the test on a single or a plurality of recall channels can be realized based on different test strategies, different test results can be obtained, the problem that the test result is single because the test can only be carried out on a plurality of recall channels at present is avoided, and the flexibility and the comprehensiveness of the test on the recall channel are improved.
Referring to fig. 5, fig. 5 is a flowchart illustrating a recall test method according to another embodiment of the present application, where the method may include:
s210, receiving a test request, wherein the test request carries a test strategy, and the test strategy comprises the number of the recall channels to be tested.
The recall channel to be tested may be a recall channel that needs to be tested for the current test request. For example, the current test request is directed to a recall channel named "hot video", that is, the current test request is used to request to test the recall channel named "hot video", and the recall channel named "hot video" is the recall channel to be tested.
The number of the recall channels to be tested included in the test strategy may be one, multiple, or the total number of multiple recall channels.
S220, responding to the test request, and selecting the recall channels corresponding to the number from the plurality of recall channels as target recall channels.
In some embodiments, the plurality of recall channels may be numbered in advance, and the recall channels may be sorted in the order from small to large, and then recall channels corresponding to the number of the plurality of recall channels may be sequentially selected as the target recall channel from the plurality of recall channels in the order of the numbers when the test request is responded.
As an example, for example, the plurality of recall channels include recall channel 1, recall channel 2, recall channel 3 … and recall channel 10, assuming that the number of recall channels to be tested in the test policy is two, when the recall module 3 responds to the first test request, recall channel 1 and recall channel 2 may be used as the target recall channels, when the recall module 3 responds to the second test request, recall channel 3 and recall channel 4 may be used as the target recall channels, and so on, and then the plurality of recall channels may be tested comprehensively according to the policy of testing two channels at a time.
As another example, for example, the number of the recall channels to be tested in the test policy is one, when the recall module 3 responds to the first test request, the recall channel 1 may be used as a target recall channel, when the recall module 3 responds to the second test request, the recall channel 2 may be used as a target recall channel, and so on, so that a plurality of recall channels may be comprehensively tested according to the policy of testing one recall channel at a time.
In this embodiment, the test policy includes the number of the recall channels to be tested, and the recall channel corresponding to the number is selected from the plurality of recall channels as the target recall channel. Therefore, the performance of one or more recall channels in the plurality of recall channels can be independently tested, and the flexibility of the test is ensured.
In some embodiments, the test strategy further includes a test label of the recall channel to be tested in addition to the number of the recall channels to be tested, wherein the test label is used for identifying the recall channel to be tested. In an embodiment of S220, the recall channels corresponding to the number of the plurality of recall channels may be selected as the target recall channel according to the test tag.
As a manner, the test tag may be used to identify a name of the recall channel to be detected, where selecting, according to the test tag, the recall channel corresponding to the number from the plurality of recall channels as the target recall channel may be selecting the recall channel corresponding to the specified name as the target recall channel. Alternatively, when the designated name (i.e., the test tag) is one, the recall channel corresponding to the designated name may be directly used as the target recall channel. When there are multiple designated names, the recall channels with the number corresponding to the recall channel to be tested in the test strategy can be selected from the recall channels corresponding to the multiple designated names as the target recall channel. As an example, for example, a test tag includes the recall channel name under test: recall channel a, recall channel B, recall channel C, and recall channel D. If the number of the recall channels to be tested in the test strategy is 3, 3 of the recall channels a, B, C and D can be arbitrarily selected as target recall channels, for example, the recall channels B, C and D are used as target recall channels. Therefore, the recall channel with the designated name in the plurality of recall channels can be tested, the number of the tested recall channels can be adjusted according to actual requirements, and the flexibility of the recall channel test is improved. Optionally, when the number of the designated names is smaller than the number of the recall channels to be tested in the test policy, all of the recall channels corresponding to the designated names may be used as target recall channels.
As another way, the test tag may be used to identify types of the recall channel to be tested, where each type may correspond to a plurality of recall channels to be tested. The method may select the recall channel corresponding to the number from the plurality of recall channels as the target recall channel according to the test tag, and select the recall channel corresponding to the number in the test policy from the plurality of recall channels corresponding to the type of the test tag as the target recall channel. As an example, assuming that the number of recall channels to be tested in the test strategy is 3, and the test label is "sports", each time a test is performed through a test request, 3 recall channels may be selected from a plurality of recall channels of the "sports" type as target recall channels. Therefore, the independent testing of different types of recall channels can be realized, the number of the tested recall channels can be adjusted according to actual requirements, and the accuracy and flexibility of the recall channel testing are improved.
Optionally, when the number of the recall channels corresponding to the test tag is less than the number of the recall channels to be tested in the test policy, all of the recall channels corresponding to the test tag may be used as the target recall channel.
Optionally, the test tag may be used to identify the name and type of the recall channel to be tested, and may also be used to indicate the serial number (e.g., serial numbers of 1, 2 … N, etc.), the format (e.g., video, audio, picture, etc.) of the recall channel to be tested, and so on.
And S230, recalling the content data corresponding to the test request from the target recall channel, and acquiring the performance data of the target recall channel in the recall process of the content data.
And S240, acquiring a performance test result of the target recall channel based on the performance data.
The specific implementation of S230-S240 can refer to S130-S140, and therefore is not described herein.
In this embodiment, the test strategy includes the number of the recall channels to be tested and the test tags, and then the target recall channel is selected from the plurality of recall channels according to the number and the test tags, so that not only can the performance of the designated recall channel be tested according to the test tags, but also the performance of one or more recall channels in the plurality of recall channels can be tested independently, and the flexibility and the accuracy of the test are ensured.
Referring to fig. 6, fig. 6 is a flowchart illustrating a recall test method according to another embodiment of the present application, where the method may include:
s310, receiving a test request, wherein the test request carries a test strategy.
And S320, responding to the test request, and determining a target recall channel from the plurality of recall channels according to the test strategy.
The specific implementation of S310-S320 can refer to S110-S120, and therefore is not described herein.
S330, obtaining a channel control instruction corresponding to the test strategy.
Wherein, the channel control instruction can be used for controlling the opening and closing of the recall channel.
In some embodiments, a plurality of test strategies and a plurality of channel control instructions may be associated in advance, the plurality of test strategies and the plurality of channel control instructions correspond to one another one by one, and a relationship table of the test strategies and the channel control instructions is obtained, so that the channel control instructions corresponding to the current test strategies can be obtained according to the relationship table of the test strategies and the channel control instructions. As an example, for example, there are 7 recall channels, the serial numbers of the 7 recall channels are sequentially 1 to 7, and if the target recall channel corresponding to the current test policy is a recall channel with a serial number of 1, 2, or 4, the control instruction corresponding to the current test policy may be to turn on the switch of the recall channel with a serial number of 1, 2, 4, or 5, and turn off the switch of the recall channel with a serial number of 3, 6, or 7.
S340, closing the recall channels except the target recall channel in the plurality of recall channels according to the channel control instruction.
In some embodiments, when all of the plurality of recall channels are acquiescent with the open state, the recall channels except the target recall channel in the plurality of recall channels can be directly switched to the closed state, and only the target recall channel is kept in the open state to recall the content data, so that the target recall channel can be accurately distinguished from the plurality of recall channels, and the performance test is convenient to perform. Optionally, after the performance test is performed on the target recall channel according to the test request, all of the plurality of recall channels may be restored to the default open state, so that when the performance test is performed on the target recall channel next time, the target recall channel can be quickly made to be in the open state, so as to facilitate recall of the content data.
In other embodiments, the current status of the targeted recall channel and the non-targeted recall channel may be detected when the plurality of recall channels do not default to an open state, wherein the non-targeted recall channel is a recall channel of the plurality of recall channels other than the targeted recall channel. When the current state of the target recall channel is the opening state, the target recall channel can not be processed, otherwise, the target recall channel is controlled to be switched to the opening state according to the control instruction. When the current state of the non-target recall channel is the closing state, the non-target recall channel can not be processed, otherwise, the non-target recall channel is controlled to be switched to the closing state according to the control instruction. Therefore, only the target recall channel in the plurality of recall channels can be reserved quickly and accurately to recall the content data.
And S350, recalling the content data corresponding to the test request from the target recall channel, and acquiring the performance data of the target recall channel in the recalling process of the content data.
The performance data may be obtained from a log generated by the target recall channel in the content data recall process, where the log may include performance parameters such as CPU memory occupancy and input/output (I/O) occupancy corresponding to the target recall channel. In addition, the performance data may also be the response time of the target recall channel to the test request.
And S360, acquiring a performance test result of the target recall channel based on the performance data.
As shown in fig. 7, in some embodiments, the number of the target recall channel is one, and the performance data includes a response duration of the target recall channel, where S360 may include:
S361A, a first time length threshold corresponding to the target recall channel is obtained.
The target recall channel may be associated with the first time threshold in advance, and optionally, the serial number or name of the target recall channel may be associated with the first time threshold, and a relationship table between the target recall channel and the first time threshold is generated. As one approach, the relationship table may be stored locally at the electronic device so that when the relationship table needs to be used, it can be directly invoked locally at the electronic device. As another way, the relationship list may be configured in a cloud server in communication with the electronic device, and when the relationship list needs to be used, the electronic device may send a request to the cloud server to obtain the relationship list, so as to relieve the local storage pressure of the electronic device.
In some embodiments, the first time threshold may be obtained from the relationship table by the serial number or name of the target recall channel currently being tested.
And S362A, when the response time length does not exceed the first time length threshold value, determining that the performance of the target recall channel is normal.
As an example, for example, when the first time threshold corresponding to the target recall channel is 10ms, and the response time of the target recall channel to the test request is 5ms, the performance of the target recall channel may be determined to be normal. When the response duration is 12ms, then the performance of the targeted recall channel may be determined to be abnormal.
In this embodiment, when the number of the target recall channel is one, by acquiring the first time length threshold corresponding to the target recall channel and by determining whether the response time length exceeds the first time length threshold, it is possible to quickly and effectively determine whether the performance test result of the target recall channel is normal or abnormal.
As shown in fig. 8, in some embodiments, the number of the target recall channels is multiple, the performance data includes multiple response durations, and the multiple response durations correspond to the multiple target recall channels one by one, where S360 may include:
and S361B, respectively obtaining second time length thresholds corresponding to the target recall channels to obtain a plurality of second time length thresholds.
In consideration of the fact that different recall channels may have different corresponding performance requirements, for example, the response duration required by a video recall channel may be longer than the response duration required by an article recall channel, and in some embodiments, a relationship table between a plurality of second duration thresholds and a plurality of target recall channels may be pre-established according to the performance requirements of each target recall channel, where the plurality of second duration thresholds and the plurality of target recall channels correspond to one another. As an example, the relationship table of the plurality of second duration thresholds and the plurality of target recall channels may be as shown in table 1:
TABLE 1
| Targeted recall channel | 1 | 2 | 3 | 4 |
| Second duration threshold | 5ms | 10ms | 3ms | 2ms |
And S362B, determining a target duration threshold according to the plurality of second duration thresholds.
In one embodiment, an average value of the plurality of second duration thresholds may be used as the target duration threshold, and as an example, the target duration threshold may be calculated to be 5ms according to the plurality of second duration thresholds in table 1.
In this embodiment, the average value of the second time length thresholds is used as the target time length threshold, so that the target time length threshold can be determined quickly, and the target time length threshold can be used for accurately evaluating the overall performance of the target recall channels.
As an embodiment, the smallest second duration threshold of the plurality of second duration thresholds may be used as the target duration threshold, and as an example, the target duration threshold may be 2ms according to the plurality of second duration thresholds in table 1.
In this embodiment, the target duration threshold can be determined quickly by using the smallest second duration threshold among the plurality of second duration thresholds as the target duration threshold, and the target duration threshold can be used for quickly and effectively evaluating the performance of each target recall channel among the plurality of target recall channels.
As another implementation, the second duration threshold corresponding to the target recall channel may be directly used as the target duration threshold of the target recall channel. For example, as can be seen from table 1, the target duration threshold of the target recall channel 1 is 5ms, the target duration threshold of the target recall channel 2 is 10ms, and the target duration threshold of the target recall channel 3 is 3ms.
And S363B, when each response time length in the plurality of response time lengths does not exceed the target time length threshold value, determining that the performance of the target recall channel is normal.
In some embodiments, when the average value of the second time length thresholds is used as the target time length threshold, and each of the response time lengths does not exceed the average value (e.g., 5 ms), the performance of the target recall channel is determined to be normal. When the minimum second duration threshold of the plurality of second duration thresholds is taken as the target duration threshold, and each response duration of the plurality of response durations does not exceed the average value (e.g., 2 ms), the performance of the target recall channel is determined to be normal. When the second duration threshold corresponding to the target recall channel is directly used as the target duration threshold of the target recall channel, and the response duration of each target recall channel does not exceed the corresponding target duration threshold, for example, when the response duration of the target recall channel 1 does not exceed 5ms, the response duration of the target recall channel 2 does not exceed 10ms, the response duration of the target recall channel 3 does not exceed 3ms, and the response duration of the target recall channel 4 does not exceed 2ms and is satisfied at the same time, it is determined that the performance of the target recall channel is normal.
As shown in fig. 9, in some embodiments, the performance data includes memory occupancy, where S360 may include:
and S361C, acquiring a memory occupancy rate threshold corresponding to the target recall channel.
In some embodiments, a plurality of recall channels and a plurality of memory occupancy thresholds may be associated, where the plurality of recall channels and the plurality of memory occupancy thresholds are in one-to-one correspondence, and a relationship table of the recall channels and the memory occupancy thresholds is obtained. Because the plurality of recall channels include the target recall channel, after the target recall channel is determined, the memory occupancy rate threshold corresponding to the target recall channel can be obtained according to the relationship table of the recall channel and the memory occupancy rate threshold.
And S362C, when the memory occupancy rate does not exceed the memory occupancy rate threshold value, determining that the performance of the target recall channel is normal.
The memory occupancy rate can be obtained from a log generated by the target recall channel in the recall process of the content data. As an example, for example, the memory occupancy threshold corresponding to the target recall channel is 30%, and when the acquired memory occupancy is 25%, the performance of the target recall channel may be normal.
In the embodiment, the memory occupancy rate obtained by the target recall channel during testing is compared with the memory occupancy rate threshold corresponding to the target recall channel to determine whether the performance of the target recall channel is normal or abnormal, so that the performance and recall stability of the target recall channel can be accurately evaluated.
In some embodiments, the performance data includes memory occupancy and IO port occupancy, where the S360 may be implemented by first obtaining an IO port occupancy threshold and a memory occupancy corresponding to the target recall channel. And then judging whether the occupancy rate of the IO port exceeds the occupancy rate threshold of the IO port, and judging whether the occupancy rate of the memory exceeds the occupancy rate threshold of the memory. When the occupancy rate of the IO port does not exceed the occupancy rate threshold of the IO port and the occupancy rate of the memory does not exceed the occupancy rate threshold of the memory, the performance of the target recall channel can be determined to be normal. The obtaining mode of the occupancy rate threshold of the IO port may refer to the obtaining mode of the memory occupancy rate.
In the embodiment, whether the performance of the target recall channel is normal or abnormal is determined by judging whether the memory occupancy rate and the IO port occupancy rate simultaneously meet the IO port occupancy rate threshold and the memory occupancy rate threshold, so that the performance and the recall stability of the target recall channel can be more accurately evaluated.
In other embodiments, the S360 may be implemented by, when the number of the target recall channels is multiple, first specifying a third duration threshold, and as a manner, when the response durations of the multiple target recall channels do not exceed the third duration threshold, determining that the performance of the target recall channel is normal. Alternatively, the number of target recall channels of the plurality of target recall channels whose response time lengths do not exceed the third time length threshold may be obtained, the weight of the number in the total number of the plurality of target recall channels may be calculated, and when the weight is greater than or equal to a preset weight (e.g., 99%), the performance of the target recall channel may be determined to be normal. Therefore, the performance evaluation efficiency of the target recall channel can be improved.
In this embodiment, by obtaining the channel control instruction corresponding to the test policy and closing the recall channels except the target recall channel in the plurality of recall channels according to the channel control instruction, the performance test can be performed on the plurality of recall channels simply and quickly according to the test policy.
Referring to fig. 10, a recall test apparatus 400 according to an embodiment of the present application is shown, where the recall test apparatus 400 includes: the system comprises a receiving module 410, a target recall channel determining module 420, a testing module 430 and a performance test result acquiring module 440.
The target recall channel determining module 420 is configured to, in response to the test request, determine a target recall channel from the plurality of recall channels according to the test policy.
The testing module 430 is configured to recall the content data corresponding to the test request from the target recall channel, and acquire performance data of the target recall channel in a recall process of the content data.
And a performance test result obtaining 440 module, configured to obtain a performance test result of the target recall channel based on the performance data.
Further, the test strategy includes the number of the recall channels to be tested, and the target recall channel determining module 420 is further configured to select a recall channel corresponding to the number from the plurality of recall channels as a target recall channel.
Further, the test strategy further includes test tags of the recall channels to be tested, where the test tags are used to identify the recall channels to be tested, and the target recall channel determining module 420 is further configured to select the recall channels corresponding to the number from the plurality of recall channels as the target recall channels according to the test tags.
Further, the number of the target recall channels is one, the performance data includes a response duration of the target recall channel, and the performance test result obtaining module 440 includes:
and the first time length threshold value acquisition unit is used for acquiring a first time length threshold value corresponding to the target recall channel.
And the first performance result determining unit is used for determining that the performance of the target recall channel is normal when the response time length does not exceed the first time length threshold.
Further, the number of the target recall channels is multiple, the performance data includes multiple response durations, the multiple response durations correspond to the multiple target recall channels one to one, and the performance test result obtaining module 440 includes:
and the second duration threshold acquisition unit is used for respectively acquiring second duration thresholds corresponding to the plurality of target recall channels to obtain a plurality of second duration thresholds.
And the target duration threshold determining unit is used for determining the target duration threshold according to the plurality of second duration thresholds.
And the second performance result determining unit is used for determining that the performance of the target recall channel is normal when each response time length in the plurality of response time lengths does not exceed the target time length threshold.
Further, the performance data includes a memory occupancy rate, and the performance test result obtaining 440 module includes:
and the memory occupancy rate threshold acquisition unit is used for acquiring the memory occupancy rate threshold corresponding to the target recall channel.
And the third performance result determining unit is used for determining that the performance of the target recall channel is normal when the memory occupancy rate does not exceed the memory occupancy rate threshold.
Further, the performance test result obtaining 440 module includes:
and the control instruction acquisition unit is used for acquiring the channel control instruction corresponding to the test strategy.
And the recall channel control unit is used for closing the recall channels except the target recall channel in the plurality of recall channels according to the channel control instruction.
And a performance test result obtaining unit 440, configured to recall, from the target recall channel, content data corresponding to the test request, and obtain performance data of the target recall channel in a recall process of the content data.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described apparatuses and modules may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, the coupling or direct coupling or communication connection between the modules shown or discussed may be through some interfaces, and the indirect coupling or communication connection between the devices or modules may be in an electrical, mechanical or other form.
In addition, functional modules in the embodiments of the present application may be integrated into one processing module, or each of the modules may exist alone physically, or two or more modules are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode.
Referring to fig. 11, a block diagram of an electronic device according to an embodiment of the present application is shown. The electronic device 500 may be the electronic device 500 capable of running the program in the foregoing embodiments. The electronic device 500 in the present application may include one or more of the following components: a processor 510, a memory 520, and one or more programs, wherein the one or more programs may be stored in the memory 520 and configured to be executed by the one or more processors 510, the one or more programs configured to perform a method as described in the aforementioned method embodiments.
Processor 510 may include one or more processing cores. The processor 510 interfaces with various components throughout the electronic device using various interfaces and circuitry to perform various functions of the electronic device and process data by executing or executing instructions, programs, code sets, or instruction sets stored in the memory 520 and invoking data stored in the memory 520. Alternatively, the processor 510 may be implemented in hardware using at least one of Digital Signal Processing (DSP), field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 510 may integrate one or a combination of a Central Processing Unit (CPU) 510, a Graphics Processing Unit (GPU) 510, a modem, and the like. Wherein, the CPU mainly processes an operating system, a user interface, an application program and the like; the GPU is used for rendering and drawing display content; the modem is used to handle wireless communications. It is understood that the modem may not be integrated into the processor 510, but may be implemented by a communication chip.
The Memory 520 may include a Random Access Memory (RAM) 520 and may also include a Read-Only Memory (Read-Only Memory) 520. The memory 520 may be used to store instructions, programs, code sets, or instruction sets. The memory 520 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing various method embodiments described below, and the like. The storage data area may also store data created by the terminal in use, such as a phonebook, audio-video data, chat log data, and the like.
Referring to fig. 12, a block diagram of a computer-readable storage medium according to an embodiment of the present application is shown. The computer readable medium 600 has stored therein a program code 610, the program code 610 being capable of being invoked by a processor to perform the method described in the method embodiments above.
The computer-readable storage medium 600 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer-readable storage medium comprises a non-transitory computer-readable medium. The computer readable storage medium has a storage space for program code for performing any of the method steps of the above-described method. The program code can be read from or written to one or more computer program products. The program code may be compressed, for example, in a suitable form.
In summary, the recall test method, apparatus, electronic device and storage medium provided in the embodiments of the present application receive a test request through a recall module, where the test request carries a test policy, and then respond to the test request to determine a target recall channel from a plurality of recall channels according to the test policy, so that one or more recall channels to be tested can be accurately selected from the plurality of recall channels. And recalling the content data corresponding to the test request from the target recall channel, acquiring the performance data of the target recall channel in the recalling process of the content data, and finally acquiring the performance test result of the target recall channel based on the performance data, so that the performance test result of the recall channel selected according to the test strategy can be obtained, the problem that the test is single because the multiple recall channels can only be integrally tested at present is solved, and the flexibility and the comprehensiveness of the recall channel test are improved.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not necessarily depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.
Claims (10)
1. A recall test method, the method comprising:
receiving a test request, wherein the test request carries a test strategy;
responding to the test request, and determining a target recall channel from a plurality of recall channels according to the test strategy;
recalling content data corresponding to the test request from the target recall channel, and acquiring performance data of the target recall channel in the recalling process of the content data;
and acquiring a performance test result of the target recall channel based on the performance data.
2. The method of claim 1, wherein the test policy comprises a number of recall channels to be tested, and wherein determining a target recall channel from a plurality of recall channels according to the test policy comprises:
and selecting the recall channels corresponding to the number from the plurality of recall channels as target recall channels.
3. The method of claim 2, wherein the test policy further comprises a test tag of the recall channel under test, wherein the test tag is used to identify the recall channel under test, and wherein determining a target recall channel from a plurality of recall channels according to the test policy comprises:
and selecting recall channels corresponding to the quantity from the plurality of recall channels as the target recall channels according to the test labels.
4. The method of claim 1, wherein the number of the target recall channel is one, the performance data comprises a response duration of the target recall channel, and the obtaining the performance test result of the target recall channel based on the performance data comprises:
acquiring a first time length threshold corresponding to the target recall channel;
when the response time length does not exceed the first time length threshold value, determining that the performance of the target recall channel is normal.
5. The method of claim 1, wherein the number of the target recall channels is multiple, the performance data comprises a plurality of response durations, the plurality of response durations correspond to a plurality of target recall channels in a one-to-one manner, and the obtaining the performance test result of the target recall channel based on the performance data comprises:
respectively acquiring second time length thresholds corresponding to the target recall channels to obtain a plurality of second time length thresholds;
determining a target duration threshold according to the plurality of second duration thresholds;
and when each response time length in the plurality of response time lengths does not exceed the target time length threshold value, determining that the performance of the target recall channel is normal.
6. The method according to any one of claims 1-5, wherein the performance data includes memory occupancy, and the obtaining the performance test result of the target recall channel based on the performance data includes:
acquiring a memory occupancy rate threshold corresponding to the target recall channel;
and when the memory occupancy rate does not exceed the memory occupancy rate threshold value, determining that the performance of the target recall channel is normal.
7. The method according to any one of claims 1 to 5, wherein the recalling the content data corresponding to the test request from the target recall channel and acquiring the performance data of the target recall channel during the recalling of the content data comprises:
acquiring a channel control instruction corresponding to the test strategy;
closing recall channels except the target recall channel in the plurality of recall channels according to the channel control instruction;
and recalling the content data corresponding to the test request from the target recall channel, and acquiring performance data of the target recall channel in the recalling process of the content data.
8. A recall test apparatus, the apparatus comprising:
the device comprises a receiving module, a judging module and a judging module, wherein the receiving module is used for receiving a test request, and the test request carries a test strategy;
the target recall channel determining module is used for responding to the test request and determining a target recall channel from a plurality of recall channels according to the test strategy;
the test module is used for recalling the content data corresponding to the test request from the target recall channel and acquiring the performance data of the target recall channel in the recalling process of the content data;
and the performance test result acquisition module is used for acquiring the performance test result of the target recall channel based on the performance data.
9. An electronic device, comprising:
one or more processors;
a memory;
one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to perform the method of any of claims 1-7.
10. A computer-readable storage medium, having stored thereon program code that can be invoked by a processor to perform the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911320857.0A CN111190815B (en) | 2019-12-19 | 2019-12-19 | Recall test method, apparatus, electronic device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911320857.0A CN111190815B (en) | 2019-12-19 | 2019-12-19 | Recall test method, apparatus, electronic device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111190815A CN111190815A (en) | 2020-05-22 |
| CN111190815B true CN111190815B (en) | 2023-03-10 |
Family
ID=70711033
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911320857.0A Active CN111190815B (en) | 2019-12-19 | 2019-12-19 | Recall test method, apparatus, electronic device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111190815B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017181612A1 (en) * | 2016-04-18 | 2017-10-26 | 乐视控股（北京）有限公司 | Personalized video recommendation method and device |
| CN109408703A (en) * | 2018-09-03 | 2019-03-01 | 腾讯科技（深圳）有限公司 | Information recommendation method and its system, device, electronic equipment, storage medium |
| CN110069714A (en) * | 2019-04-25 | 2019-07-30 | 北京酷我科技有限公司 | A kind of video recommendation system |
| CN110221817A (en) * | 2019-06-17 | 2019-09-10 | 北京酷我科技有限公司 | A kind of data recall module and recommender system |
| CN110442512A (en) * | 2019-07-05 | 2019-11-12 | 深圳壹账通智能科技有限公司 | The method for testing reliability and relevant device of information management system by all kinds of means |
-
2019
- 2019-12-19 CN CN201911320857.0A patent/CN111190815B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017181612A1 (en) * | 2016-04-18 | 2017-10-26 | 乐视控股（北京）有限公司 | Personalized video recommendation method and device |
| CN109408703A (en) * | 2018-09-03 | 2019-03-01 | 腾讯科技（深圳）有限公司 | Information recommendation method and its system, device, electronic equipment, storage medium |
| CN110069714A (en) * | 2019-04-25 | 2019-07-30 | 北京酷我科技有限公司 | A kind of video recommendation system |
| CN110221817A (en) * | 2019-06-17 | 2019-09-10 | 北京酷我科技有限公司 | A kind of data recall module and recommender system |
| CN110442512A (en) * | 2019-07-05 | 2019-11-12 | 深圳壹账通智能科技有限公司 | The method for testing reliability and relevant device of information management system by all kinds of means |
Non-Patent Citations (1)
| Title |
|---|
| 基于内容分发平台的短视频产品推荐算法研究与实现;李靖怡;《中国优秀硕士学位论文全文数据库》;20181215;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111190815A (en) | 2020-05-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11032419B2 (en) | Intelligent customer service systems, customer service robots, and methods for providing customer service | |
| CN109803152B (en) | Violation auditing method and device, electronic equipment and storage medium | |
| US9972022B2 (en) | System and method for optimizing access to a resource based on social synchrony and homophily | |
| CN108959319B (en) | Information pushing method and device | |
| CN108696399B (en) | Business service test method and device | |
| WO2020087386A1 (en) | Content recommendation method and device, mobile terminal, and server | |
| CN106874314B (en) | Information recommendation method and device | |
| CN110909205A (en) | Video cover determination method and device, electronic equipment and readable storage medium | |
| CN113127723B (en) | User portrait processing method, device, server and storage medium | |
| CN113422988B (en) | Live broadcast cover generation method and device, electronic equipment and storage medium | |
| US20170169062A1 (en) | Method and electronic device for recommending video | |
| CN113505272B (en) | Control method and device based on behavior habit, electronic equipment and storage medium | |
| WO2020258773A1 (en) | Method, apparatus, and device for determining pushing user group, and storage medium | |
| WO2021081914A1 (en) | Pushing object determination method and apparatus, terminal device and storage medium | |
| CN111182332B (en) | Video processing method, device, server and storage medium | |
| US20200004785A1 (en) | Automatic grouping based on user behavior | |
| CN111090810A (en) | Application message pushing method and device and electronic equipment | |
| CN112507218A (en) | Business object recommendation method and device, electronic equipment and storage medium | |
| CN111190815B (en) | Recall test method, apparatus, electronic device and storage medium | |
| WO2021087970A1 (en) | Method and apparatus for updating tag data, electronic device, and storage medium | |
| JP7354127B2 (en) | Providing activity notifications regarding digital content | |
| CN109829043B (en) | Part-of-speech confirmation method, part-of-speech confirmation device, electronic device, and storage medium | |
| CN115809889A (en) | Intelligent passenger group screening method, system, medium and equipment based on marketing effect | |
| US20190073606A1 (en) | Dynamic content optimization | |
| CN111176965B (en) | Recommendation system pre-release test method and device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20210115 Address after: 511442 3108, 79 Wanbo 2nd Road, Nancun Town, Panyu District, Guangzhou City, Guangdong Province Applicant after: GUANGZHOU CUBESILI INFORMATION TECHNOLOGY Co.,Ltd. Address before: 511400 24th floor, building B-1, North District, Wanda Commercial Plaza, Wanbo business district, No.79 Wanbo 2nd Road, Nancun Town, Panyu District, Guangzhou, Guangdong Province Applicant before: GUANGZHOU HUADUO NETWORK TECHNOLOGY Co.,Ltd. |
|
| EE01 | Entry into force of recordation of patent licensing contract | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20200522 Assignee: GUANGZHOU HUADUO NETWORK TECHNOLOGY Co.,Ltd. Assignor: GUANGZHOU CUBESILI INFORMATION TECHNOLOGY Co.,Ltd. Contract record no.: X2021440000054 Denomination of invention: Recall test method, device, electronic equipment and storage medium License type: Common License Record date: 20210208 |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |