CN109032954B - User selection method and device for A/B test, storage medium and terminal - Google Patents
User selection method and device for A/B test, storage medium and terminal Download PDFInfo
- Publication number
- CN109032954B CN109032954B CN201810933965.4A CN201810933965A CN109032954B CN 109032954 B CN109032954 B CN 109032954B CN 201810933965 A CN201810933965 A CN 201810933965A CN 109032954 B CN109032954 B CN 109032954B
- Authority
- CN
- China
- Prior art keywords
- user
- target user
- step length
- test
- currently selected
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a user selection method, a user selection device, a storage medium and a terminal for an A/B test, wherein the method comprises the following steps: setting a random step length, wherein the random step length is a value added on the basis of the number of the currently selected target user when the next target user is selected; and selecting target users in the same target user set according to the random step length. According to the method and the device, the random step length is set, when the target user is selected, different target users are selected in the same target user set according to the random step length to push the landing page, so that the same or similar attributes among the target users are ensured, a plurality of A/B tests using the same user set are ensured not to be pushed by the same user all the time, and the use experience of the user is optimized on the basis of increasing the accuracy of the test result.
Description
Technical Field
The invention relates to the field of software development, in particular to a user selection method, a user selection device, a storage medium and a terminal for A/B testing.
Background
The A/B test is also called a random experiment or a contrast experiment, and provides a valuable way to evaluate the influence of a new function on the behavior of a client in the software development process, and aims to obtain a representative experimental conclusion through scientific experimental design, sample representativeness, flow segmentation, small flow test and other ways and ensure that the experimental conclusion obtained in the A/B test has reference significance when being formally pushed to all types of users subsequently.
During traffic operation, the experiment that the file to be tested is pushed by the user with small traffic level as the target user is called the push advertisement a/B test, or ab test for short. Generally, in a testing period, aiming at a landing page, a plurality of different case information needs to be sent simultaneously, namely a plurality of A/B tests are carried out, results of the different A/B tests are taken as references for distributing pushing resources, the quality of the case pushed by the landing page is ensured, an effective click rate can be increased in an impulse period, and benefits are increased. When a plurality of target users of the A/B test are selected during the test period in the prior art, the attributes of the selected target users are different among the A/B tests, the results of the A/B tests are not comparable, and the accuracy of the test result is further influenced.
Disclosure of Invention
The invention provides a user selection method, a user selection device, a storage medium and a terminal for an A/B test, which are used for solving the following problems in the prior art: if the attributes of target users selected among the A/B tests are different, the results of the A/B tests are easy to be not comparable, and the accuracy of the test results is further influenced; if the same target user is selected to push the A/B test pattern for multiple times, the user frequently receives the push of the landing page, and the use experience of the user is seriously influenced.
In order to solve the above technical problem, in one aspect, the present invention provides a user selection method for an a/B test, including: setting a random step length, wherein the random step length is a value added on the basis of the number of a currently selected target user when the next target user is selected; and selecting target users in the same target user set according to the random step length.
Further, after selecting the target user in the target user set according to the random step length, the method further includes: detecting whether a currently selected target user exists in a historical user set corresponding to the A/B test; under the condition that the currently selected target user exists in a historical user set corresponding to the A/B test, discarding the currently selected target user, and selecting the next target user in the target user set according to the number of the currently selected target user and the random step length; and under the condition that the currently selected target user does not exist in the historical user set corresponding to the A/B test, pushing the landing page of the A/B test to the currently selected target user, and adding the currently selected target user to the historical user set corresponding to the A/B test.
Further, after pushing the landing page of the a/B test to the currently selected target user, the method further includes: detecting whether the number of target users who have pushed the landing page exceeds a preset sending amount or not; and under the condition that the number of the target users who have pushed the landing page does not exceed the preset sending amount, selecting the next target user in the target user set according to the number of the currently selected target user and the random step length.
Further, the setting a random step size includes: setting a user selection proportion, and setting the reciprocal of the user selection proportion as a user selection step length; and setting the random step length as a random value within a preset step length range according to the user selection step length, wherein the preset step length range takes the difference between the user selection step length and a preset fixed number as a lower limit and the user selection step length as an upper limit.
On the other hand, the invention also provides a user selection device for the A/B test, which comprises the following components: the setting module is used for setting a random step length, wherein the random step length is a value added on the basis of the number of the currently selected target user when the next target user is selected; and the selecting module is used for selecting the target users in the same target user set according to the random step length.
Further, still include: the first detection module is used for detecting whether the currently selected target user exists in a historical user set corresponding to the A/B test; the selecting module is specifically configured to, when a currently selected target user exists in a historical user set corresponding to an a/B test, discard the currently selected target user, and select a next target user in the target user set according to the number of the currently selected target user and the random step length; and the pushing module is used for pushing the landing page of the A/B test to the currently selected target user and adding the currently selected target user to the historical user set corresponding to the A/B test under the condition that the currently selected target user does not exist in the historical user set corresponding to the A/B test.
Further, still include: the second detection module is used for detecting whether the number of the target users who push the landing page exceeds a preset sending amount or not; the selecting module is specifically configured to select a next target user in the target user set according to the random step length when the number of target users who have pushed the landing page does not exceed a preset sending amount.
Further, the setting module is specifically configured to: setting a user selection proportion, and setting the reciprocal of the user selection proportion as a user selection step length; and setting the random step length as a random value within a preset step length range according to the user selection step length, wherein the preset step length range takes the difference between the user selection step length and a preset fixed number as a lower limit and the user selection step length as an upper limit.
In another aspect, the present invention further provides a storage medium storing a computer program, where the computer program is executed by a processor to implement the steps of the user selection method for the a/B test.
On the other hand, the invention also provides a terminal, which at least comprises a memory and a processor, wherein the memory is stored with a computer program, and the processor realizes the steps of the user selection method of the A/B test when executing the computer program on the memory.
According to the method and the device, the random step length is set, when the target user is selected, different target users are selected in the same target user set according to the random step length to push the landing page, so that the same or similar attributes among the target users are ensured, a plurality of A/B tests using the same user set are ensured not to be pushed by the same user all the time, and the use experience of the user is optimized on the basis of increasing the accuracy of the test result.
Drawings
FIG. 1 is a flow chart of a user selection method for A/B testing according to a first embodiment of the present invention;
FIG. 2 is a flow chart of a user selection method for A/B testing according to a second embodiment of the present invention;
FIG. 3 is a flowchart of a user selection method for A/B testing according to a third embodiment of the present invention;
FIG. 4 is a schematic diagram of a user selection apparatus for A/B testing according to a fourth embodiment of the present invention;
FIG. 5 is a schematic diagram of a user selection apparatus for A/B testing according to a fifth embodiment of the present invention;
FIG. 6 is a diagram illustrating a structure of a user selection device for A/B testing according to a sixth embodiment of the present invention.
Detailed Description
In order to solve the following problems in the prior art: if the attributes of target users selected among the A/B tests are different, the results of the A/B tests are easy to be not comparable, and the accuracy of the test results is further influenced; if the same target user is selected to push the A/B test pattern for multiple times, the user frequently receives the push of the landing page, and the use experience of the user is seriously influenced. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the invention.
The first embodiment of the present invention provides a user selection method for a/B test, a flowchart of which is shown in fig. 1, and the method mainly includes steps S101 and S102:
s101, setting a random step size.
When the advertisement push A/B test is performed, before the advertisement is pushed to the same landing page, a plurality of push documents are usually preset, different documents are pushed to users through a plurality of A/B tests, and test results fed back by the users for all the documents are collected to serve as references for distributing push resources. In the existing testing process, target users with different attributes may be selected for a plurality of A/B tests, for example, the target user pushed by a first case is selected as an old person in the Heilongjiang area, the target user pushed by a second case is a young person in the Guangzhou area, the regional consumption levels and the customs of the users are different between the two tests, the age levels of the users are different, and the collected test results are not comparable, so that the accuracy of the test results is influenced. If the users of the two file pushes are completely the same, the users are likely to frequently receive the push of the landing page, the use experience of the users is seriously affected, and even complaints or situations of unloading corresponding application programs and the like may occur.
In order to solve the problems, when the A/B test of the push advertisement is carried out, a target user set is selected firstly, the attributes of the target users of the A/B test carried out for multiple times are ensured to be the same, a conclusion which is more accordant with the preference of the user can be obtained according to the test result, and the quality of the landing page file is improved. When a target user is selected in the current target user set, a random step length is set as a basis for selecting the target user in the target user set, namely, when the next target user is selected, a value is added on the basis of the number of the currently selected target user.
Specifically, after a target user set is determined, a user selection ratio is set, a product obtained by multiplying the number of target users in the target user set by the user selection ratio is used as the final sending user amount in the test, and the reciprocal of the user selection ratio is set as a user selection step length, that is, if the user selection ratio is 2.5%, the user selection step length is 1/0.025-40.
And after the step length selected by the user is determined, setting a random step length according to the step length selected by the user. Specifically, according to the user selection step length, a preset step length range is firstly set, the lower limit of the preset step length range is the difference between the user selection step length and a preset fixed number, the upper limit is the user selection step length, the random step length is randomly selected in the preset step length range, and different random step length values can be used when the user is selected each time. For example, the user selects a step size of 40, a predetermined fixed number of 20, a predetermined step size range of [20, 40], and a random step size of any integer from 20 to 40. It should be understood that there may be a case where the value of the preset fixed number is greater than the user selection step length, the lower limit value of the preset step length range is a negative number, and at this time, the preset step length range includes 0, but in order to ensure normal selection by the user, the random step length may be set to be not zero.
And S102, selecting target users in the same target user set according to the random step length.
When the target user is selected, the number or the coordinate of the currently selected target user is added with the random step length to obtain the number or the coordinate of the next target user. If the selection of the current user is the first selection of the target user in the current target user set, the first user in the set is selected or one user is randomly selected as the start.
Specifically, assuming that the preset step range is [20, 40], the random step is set to 25, and the currently selected target user is numbered 100, then the next target user is numbered 125. It should be understood that there may be a case where the value of the preset fixed number is greater than the user selection step length, and the lower limit value of the preset step length range is a negative number, if the user selection step length is 40, the preset fixed number is 50, the preset step length range is [ -10, 40], the random step length may be-5, and if the number of the currently selected target user is 100, the next target user is a user with a number of 95. In addition, when the lower limit value of the preset step range is a negative number, the preset step range includes 0, but in order to ensure that the next selected user is different from the current selection, the value of the random step cannot be set to 0.
By setting the random step length, when a target user is selected, different target users are selected in the same target user set according to the random step length to push landing pages, so that the same or similar attributes among the target users are ensured, a plurality of A/B tests using the same user set are ensured not to be pushed by the same user all the time, and the use experience of the user is optimized on the basis of increasing the accuracy of a test result.
The second embodiment of the present invention provides a user selection method for a/B test, a flowchart of which is shown in fig. 2, and mainly includes steps S201 and S205:
s201, setting a random step size.
When the A/B test of the push advertisement is carried out, a target user set is selected firstly, the same or similar attributes of the target users of a plurality of times of A/B tests carried out on the same landing page are ensured, the conclusion which is more accordant with the user preference can be obtained according to the test result, and the quality of the landing page file is improved. When a target user is selected in the current target user set, a random step length is set as a basis for selecting the target user in the target user set, namely, when the next target user is selected, a value is added on the basis of the number of the currently selected target user.
Specifically, after a target user set is determined, a user selection ratio is set, the product of the number of target users in the target user set and the user selection ratio is used as the final sending user amount in the test, and the reciprocal of the user selection ratio is set as the user selection step length. And after the step length selected by the user is determined, setting a random step length according to the step length selected by the user. Specifically, according to the user selection step length, a preset step length range is firstly set, the lower limit of the preset step length range is the difference between the user selection step length and a preset fixed number, the upper limit is the user selection step length, the random step length is randomly selected in the preset step length range, and different random step length values can be used when the user is selected each time.
S202, selecting target users in the same target user set according to the random step length.
When the target user is selected, the number or the coordinate of the currently selected target user is added with the random step length to obtain the number or the coordinate of the next target user. If the selection of the current user is the first selection of the target user in the current target user set, the first user in the set is selected or one user is randomly selected as the start. If the value obtained by adding the random step length to the coordinate of the currently selected target user exceeds the maximum number existing in the target user set, calculating the step number between the number of the currently selected target user and the maximum number, subtracting the step number by using the random step length to obtain the residual step number, and taking the target user corresponding to the number with the same value as the residual step number in the target user set as the next selected target user. For example, the current target user set has 1000 target users, that is, the maximum number of the target user is 1000, the number of the currently selected user is 0997, the random step size is 10, the number of steps between the currently selected target user number and the maximum number is 3, and the remaining step size is 7, then the number of the next selected target user is 0007.
S203, detecting whether the currently selected target user exists in the historical user set corresponding to the A/B test, if so, executing the step S204, otherwise, executing the step S205.
S204, abandoning the currently selected target user, and re-executing the step S202 on the basis of the number of the currently selected target user.
S205, pushing the landing page of the A/B test to the currently selected target user, and adding the currently selected target user to a historical user set corresponding to the A/B test.
In order to avoid repeated pushing on the users which have already been pushed, after a target user is selected, whether the current target user exists in a historical user set corresponding to the current A/B test is judged. It should be appreciated that each A/B test should have its corresponding set of historical users, and that the set in the Redis cache storage tool can be used as the storage structure for the set of historical users, the landing page links for the A/B test as the keys for the corresponding set of historical users, or for brevity, the Message Digest Algorithm fifth version value (MD5, Message-Digest Algorithm 5) for the landing page links as the keys for the corresponding set of historical users.
Firstly, searching whether a landing page has a historical user set in Redis according to the MD5 value linked with the landing page when detecting whether a currently selected target user exists in the historical user set corresponding to the A/B test, and if the landing page does not have the historical user set, proving that the landing page is not pushed, establishing the corresponding historical user set and adding the current user to the historical user set; if the landing page exists, detecting whether a currently selected target user exists in a historical user set corresponding to the A/B test, if so, proving that the current user is pushed, abandoning the user at the moment to prevent the user from being repeatedly pushed to the landing page to influence user experience, and selecting a next target user in the target user set according to random step length; if the currently selected target user does not exist in the historical user set, the user is not pushed through the current landing page, the landing page is pushed to the currently selected target user at the moment, the user is added into the historical user set to indicate that the user is pushed through the landing page, and when the user is selected again, the user is discarded.
It should be understood that, in the formal push stage, it is sufficient to filter out the users who have pushed the landing page in the test stage, so as to prevent the users who have received the landing page in the test stage from repeatedly receiving the same test content, which affects the user experience.
According to the method and the device, by setting the random step length, when the target user is selected, different target users are selected in the same target user set according to the random step length to carry out floor page pushing, and whether the currently selected user is pushed through the current floor page or not is judged through the historical user set, so that the same or similar attributes among the target users are ensured, the same user is ensured not to be pushed through the same floor page for multiple times, and the use experience of the user is optimized on the basis of increasing the accuracy of the test result.
The third embodiment of the present invention provides a user selection method for a/B test, a flowchart of which is shown in fig. 3, and the method mainly includes steps S301 and S306:
s301, setting a random step size.
When the A/B test of the push advertisement is carried out, a target user set is selected firstly, the same or similar attributes of the target users of a plurality of times of A/B tests carried out on the same landing page are ensured, the conclusion which is more accordant with the user preference can be obtained according to the test result, and the quality of the landing page file is improved. When a target user is selected in the current target user set, a random step length is set as a basis for selecting the target user in the target user set, namely, when the next target user is selected, a value is added on the basis of the number of the currently selected target user.
Specifically, after a target user set is determined, a user selection ratio is set, the product of the number of target users in the target user set and the user selection ratio is used as the final sending user amount in the test, and the reciprocal of the user selection ratio is set as the user selection step length. And after the step length selected by the user is determined, setting a random step length according to the step length selected by the user. Specifically, according to the user selection step length, a preset step length range is firstly set, the lower limit of the preset step length range is the difference between the user selection step length and a preset fixed number, the upper limit is the user selection step length, the random step length is randomly selected in the preset step length range, and different random step length values can be used when the user is selected each time.
S302, selecting target users in the same target user set according to the random step length.
When the target user is selected, the number or the coordinate of the currently selected target user is added with the random step length to obtain the number or the coordinate of the next target user. If the selection of the current user is the first selection of the target user in the current target user set, the first user in the set is selected or one user is randomly selected as the start. If the value obtained by adding the random step length to the coordinate of the currently selected target user exceeds the maximum number existing in the target user set, calculating the step number between the number of the currently selected target user and the maximum number, subtracting the step number by using the random step length to obtain the residual step number, and taking the target user corresponding to the number with the same value as the residual step number in the target user set as the next selected target user.
S303, detecting whether the currently selected target user exists in the historical user set corresponding to the A/B test, if so, executing the step S304, otherwise, executing the step S305.
S304, abandoning the currently selected target user, and re-executing the step S302 on the basis of the number of the currently selected target user.
S305, pushing the landing page of the A/B test to the currently selected target user, and adding the currently selected target user to the historical user set corresponding to the A/B test.
In order to avoid repeated pushing on the users which have already been pushed, after a target user is selected, whether the current target user exists in a historical user set corresponding to the current A/B test is judged. It should be appreciated that each A/B test should have its corresponding set of historical users, and that the set in the Redis cache storage tool can be used as the storage structure for the set of historical users, the landing page link for the A/B test can be used as the key for the corresponding set of historical users, or for brevity, the MD5 value of the landing page link can be used as the key for the corresponding set of historical users.
Firstly, searching whether a landing page has a historical user set in Redis according to the MD5 value linked with the landing page when detecting whether a currently selected target user exists in the historical user set corresponding to the A/B test, and if the landing page does not have the historical user set, proving that the landing page is not pushed, establishing the corresponding historical user set and adding the current user to the historical user set; if the landing page exists, detecting whether a currently selected target user exists in a historical user set corresponding to the A/B test, if so, proving that the current user is pushed, and abandoning the user to prevent the user from being repeatedly pushed to the landing page to influence user experience; if the currently selected target user does not exist in the historical user set, the user is not pushed through the current landing page, the landing page is pushed to the currently selected target user at the moment, the user is added into the historical user set to indicate that the user is pushed through the landing page, and when the user is selected again, the user is discarded.
S306, detecting whether the number of the target users of the pushed landing page exceeds a preset sending amount, if not, executing the step S302 again on the basis of the number of the currently selected target user, and if so, stopping the user selection.
Specifically, the preset sending amount may be a maximum value of the number of pushed target users that is set for the tester, that is, when the number of pushed target users is greater than or equal to the preset sending amount, the obtained test result may satisfy the test result analysis of the a/B test. The preset sending quantity value can also be the product of the number of the target users in the target user set and the user selection ratio, namely the test requirement is met after the number of the pushed users reaches the preset ratio of the number of the users in the current target user set, and the excessive test is prevented from causing unnecessary test result waste. And when the number of the currently pushed target users does not exceed the preset sending amount, re-executing the step S302, and selecting a next target user in the target user set according to the random step length for pushing until the number of the pushed target users exceeds the preset sending amount.
It should be understood that, in the formal push stage, it is sufficient to filter out the users who have pushed the landing page in the test stage, so as to prevent the users who have received the landing page in the test stage from repeatedly receiving the same test content, which affects the user experience.
According to the method and the device, by setting the random step length, when the target user is selected, different target users are selected in the same target user set according to the random step length to carry out floor page pushing, whether the currently selected user is pushed to pass through the current floor page is judged through the historical user set, and pushing is stopped when the pushed target user amount exceeds the preset sending amount, so that the same or similar attributes among the target users are ensured, the same user is prevented from being pushed to pass through the same floor page for many times, the test repetition and waste are avoided, and the use experience of the user is optimized on the basis of increasing the accuracy of the test result.
A fourth embodiment of the present invention provides a user selecting device for an a/B test, which is specifically installed on a terminal or a server for performing a test, and a schematic structural diagram of the device is shown in fig. 4, and the device mainly includes: the device comprises a setting module 10 and a selecting module 20 which are coupled with each other, wherein the setting module 10 is used for setting a random step length, and the random step length is a value which is increased on the basis of the number of a currently selected target user when the next target user is selected; and the selecting module 20 is configured to select the target users from the same target user set according to the random step length.
When the advertisement push A/B test is performed, before the advertisement is pushed to the same landing page, a plurality of push documents are usually preset, different documents are pushed to users through a plurality of A/B tests, and test results fed back by the users for all the documents are collected to serve as references for distributing push resources. In the existing testing process, target users with different attributes may be selected for a plurality of A/B tests, for example, the target user pushed by a first case is selected as an old person in the Heilongjiang area, the target user pushed by a second case is a young person in the Guangzhou area, the regional consumption levels and the customs of the users are different between the two tests, the age levels of the users are different, and the collected test results are not comparable, so that the accuracy of the test results is influenced. If the users of the two file pushes are completely the same, the users are likely to frequently receive the push of the landing page, the use experience of the users is seriously affected, and even complaints or situations of unloading corresponding application programs and the like may occur.
In order to solve the above problems, when a push advertisement A/B test is performed, a target user set is selected by the setting module 10, so that the attributes of target users of a plurality of A/B tests are the same or similar, a conclusion more conforming to the preference of the user can be obtained according to the test result, and the quality of the landing page file is improved. And when the target user is selected in the current target user set, a random step length is set as a basis for selecting the target user in the target user set, namely, when the next target user is selected, a value is added on the basis of the number of the currently selected target user.
Specifically, after the target user set is determined, the setting module 10 sets a user selection ratio, a product obtained by multiplying the number of target users in the target user set by the user selection ratio is used as the final sending user amount in the test, and the reciprocal of the user selection ratio is set as the user selection step length. And after the step length selected by the user is determined, setting a random step length according to the step length selected by the user. Specifically, the setting module 10 firstly sets a preset step range according to the user selection step, the lower limit of the preset step range is the difference between the user selection step and a preset fixed number, the upper limit is the user selection step, the random step is randomly selected in the preset step range, and different random step values can be used each time the user is selected.
When selecting a target user, the selecting module 20 adds a random step to the number or coordinate of the currently selected target user to obtain the number or coordinate of the next target user. If the selection of the current user is the first selection of the target user in the current target user set, the first user in the set is selected or one user is randomly selected as the start.
By setting the random step length, when a target user is selected, different target users are selected in the same target user set according to the random step length to push landing pages, so that the same or similar attributes among the target users are ensured, a plurality of A/B tests using the same user set are ensured not to be pushed by the same user all the time, and the use experience of the user is optimized on the basis of increasing the accuracy of a test result.
A fifth embodiment of the present invention provides an a/B test user selection apparatus, specifically installed on a terminal or a server for performing a test, and a schematic structural diagram of the apparatus is shown in fig. 5, which mainly includes: a setting module 10, configured to set a random step length, where the random step length is a value added on the basis of a currently selected target user number when a next target user is selected; a selecting module 20, coupled to the setting module 10, configured to select a target user from the same set of target users according to a random step length, and further configured to discard the currently selected target user when the currently selected target user exists in a historical user set corresponding to the a/B test, and select a next target user from the set of target users according to a number of the currently selected target user and the random step length; a first detecting module 30, coupled to the selecting module 20, for detecting whether the currently selected target user exists in the historical user set corresponding to the a/B test; and the pushing module 40 is coupled to the first detecting module 30, and configured to push the landing page of the a/B test to the currently selected target user and add the currently selected target user to the historical user set corresponding to the a/B test when the currently selected target user does not exist in the historical user set corresponding to the a/B test.
When the A/B test of the push advertisement is carried out, the setting module 10 firstly selects a target user set, the attributes of the target users of a plurality of A/B tests carried out on the same landing page are ensured to be the same or similar, a conclusion which is more in line with the preference of the users can be obtained according to the test result, and the quality of the landing page file is improved. The setting module 10 sets a random step length as a basis for selecting a target user in the target user set when the target user is selected in the current target user set, that is, a value added on the basis of the number of the currently selected target user when the next target user is selected.
Specifically, after the target user set is determined, the setting module 10 sets a user selection ratio, a product obtained by multiplying the number of target users in the target user set by the user selection ratio is used as the final sending user amount in the test, and the reciprocal of the user selection ratio is set as the user selection step length. And after the step length selected by the user is determined, setting a random step length according to the step length selected by the user. Specifically, according to the user selection step length, a preset step length range is firstly set, the lower limit of the preset step length range is the difference between the user selection step length and a preset fixed number, the upper limit is the user selection step length, the random step length is randomly selected in the preset step length range, and different random step length values can be used when the user is selected each time.
When the selection module 20 selects a target user, the number or the coordinate of the currently selected target user is added with the random step length to obtain the number or the coordinate of the next target user. If the selection of the current user is the first selection of the target user in the current target user set, the first user in the set is selected or one user is randomly selected as the start. If the value obtained by adding the random step length to the coordinate of the currently selected target user exceeds the maximum number existing in the target user set, calculating the step number between the number of the currently selected target user and the maximum number, subtracting the step number by using the random step length to obtain the residual step number, and taking the target user corresponding to the number with the same value as the residual step number in the target user set as the next selected target user.
In order to avoid repeated pushing on the users that have already been pushed, after selecting a target user, the first detection module 30 detects whether the current target user exists in the historical user set corresponding to the current a/B test. It should be appreciated that each A/B test should have its corresponding set of historical users, and that the set in the Redis cache storage tool can be used as the storage structure for the set of historical users, the landing page link for the A/B test can be used as the key for the corresponding set of historical users, or for brevity, the MD5 value of the landing page link can be used as the key for the corresponding set of historical users.
Detecting whether a currently selected target user exists in a historical user set corresponding to an A/B test or not by a first detection module 30, firstly, searching whether the landing page has the historical user set or not in Redis according to the MD5 value linked with the landing page, if the landing page does not have the historical user set, proving that the landing page is not pushed, establishing the historical user set corresponding to the landing page, and adding the current user to the historical user set; if the landing page exists, detecting whether a currently selected target user exists in a historical user set corresponding to the A/B test, if so, verifying that the current user is pushed, abandoning the user to prevent the user from being repeatedly pushed to the landing page to influence user experience, and selecting a next target user in the target user set by the selection module 20 according to the random step length; if the currently selected target user does not exist in the historical user set, the user is not pushed through the current landing page, at this time, the landing page is pushed to the currently selected target user through the pushing module 40, the user is added into the historical user set to indicate that the user has been pushed through the landing page, and when the user is selected again, the user is discarded.
It should be understood that, in the formal push stage, it is sufficient to filter out the users who have pushed the landing page in the test stage, so as to prevent the users who have received the landing page in the test stage from repeatedly receiving the same test content, which affects the user experience.
According to the method and the device, by setting the random step length, when the target user is selected, different target users are selected in the same target user set according to the random step length to carry out floor page pushing, and whether the currently selected user is pushed through the current floor page or not is judged through the historical user set, so that the same or similar attributes among the target users are ensured, the same user is ensured not to be pushed through the same floor page for multiple times, and the use experience of the user is optimized on the basis of increasing the accuracy of the test result.
A sixth embodiment of the present invention provides a user selecting device for an a/B test, which is specifically installed on a terminal or a server for performing a test, and a schematic structural diagram of the device is shown in fig. 6, and mainly includes: a setting module 10, configured to set a random step length, where the random step length is a value added on the basis of a currently selected target user number when a next target user is selected; a selecting module 20, coupled to the setting module 10, configured to select a target user in the target user set according to the random step length, and further configured to discard the currently selected target user when the currently selected target user exists in the historical user set corresponding to the a/B test, select a next target user in the target user set according to the number of the currently selected target user and the random step length, and select a next target user in the target user set according to the number of the currently selected target user and the random step length when the number of the target users that have pushed the landing page does not exceed a preset sending amount; a first detecting module 30, coupled to the selecting module 20, for detecting whether the currently selected target user exists in the historical user set corresponding to the a/B test; the pushing module 40 is coupled to the first detecting module 30, and configured to push the landing page of the a/B test to the currently selected target user and add the currently selected target user to the historical user set corresponding to the a/B test when the currently selected target user does not exist in the historical user set corresponding to the a/B test; and the second detection module 50 is coupled to the pushing module 40 and the selecting module 20, respectively, and is configured to detect whether the number of target users who have pushed the landing pages exceeds a preset sending amount.
When the A/B test of the push advertisement is carried out, the setting module 10 firstly selects a target user set, the attributes of the target users of a plurality of A/B tests carried out on the same landing page are ensured to be the same or similar, a conclusion which is more in line with the preference of the users can be obtained according to the test result, and the quality of the landing page file is improved. The setting module 10 sets a random step length as a basis for selecting a target user in the target user set when the target user is selected in the current target user set, that is, a value added on the basis of the number of the currently selected target user when the next target user is selected.
Specifically, after the target user set is determined, the setting module 10 sets a user selection ratio, a product obtained by multiplying the number of target users in the target user set by the user selection ratio is used as the final sending user amount in the test, and the reciprocal of the user selection ratio is set as the user selection step length. And after the step length selected by the user is determined, setting a random step length according to the step length selected by the user. Specifically, according to the user selection step length, a preset step length range is firstly set, the lower limit of the preset step length range is the difference between the user selection step length and a preset fixed number, the upper limit is the user selection step length, the random step length is randomly selected in the preset step length range, and different random step length values can be used when the user is selected each time.
When the selection module 20 selects a target user, the number or the coordinate of the currently selected target user is added with the random step length to obtain the number or the coordinate of the next target user. If the selection of the current user is the first selection of the target user in the current target user set, the first user in the set is selected or one user is randomly selected as the start. If the value obtained by adding the random step length to the coordinate of the currently selected target user exceeds the maximum number existing in the target user set, calculating the step number between the number of the currently selected target user and the maximum number, subtracting the step number by using the random step length to obtain the residual step number, and taking the target user corresponding to the number with the same value as the residual step number in the target user set as the next selected target user.
In order to avoid repeated pushing on the users that have already been pushed, after selecting a target user, the first detection module 30 detects whether the current target user exists in the historical user set corresponding to the current a/B test. It should be appreciated that each A/B test should have its corresponding set of historical users, and that the set in the Redis cache storage tool can be used as the storage structure for the set of historical users, the landing page link for the A/B test can be used as the key for the corresponding set of historical users, or for brevity, the MD5 value of the landing page link can be used as the key for the corresponding set of historical users.
Detecting whether a currently selected target user exists in a historical user set corresponding to an A/B test or not by a first detection module 30, firstly, searching whether the landing page has the historical user set or not in Redis according to the MD5 value linked with the landing page, if the landing page does not have the historical user set, proving that the landing page is not pushed, establishing the historical user set corresponding to the landing page, and adding the current user to the historical user set; if the landing page exists, detecting whether a currently selected target user exists in a historical user set corresponding to the A/B test, if so, verifying that the current user is pushed, abandoning the user to prevent the user from being repeatedly pushed to the landing page to influence user experience, and selecting a next target user in the same target user set by the selection module 20 again according to the number and the random step length of the currently selected target user; if the currently selected target user does not exist in the historical user set, the user is not pushed through the current landing page, at this time, the landing page is pushed to the currently selected target user through the pushing module 40, the user is added into the historical user set to indicate that the user has been pushed through the landing page, and when the user is selected again, the user is discarded.
Specifically, after the push module 40 pushes the landing page, it is detected by the second detection module 50 whether the number of target users who have pushed the landing page exceeds a preset sending amount. The preset sending amount can be the maximum value of the number of the pushed target users set for the tester, namely when the number of the pushed target users is larger than or equal to the preset sending amount, the obtained test result can meet the test result analysis of the A/B test. The preset sending quantity value can also be the product of the number of the target users in the target user set and the user selection ratio, namely the test requirement is met after the number of the pushed users reaches the preset ratio of the number of the users in the current target user set, and the excessive test is prevented from causing unnecessary test result waste. When the number of the currently pushed target users does not exceed the preset sending amount, the selection module 20 reselects the next target user in the same target user set for pushing according to the number of the currently selected target users and the random step length until the number of the pushed target users exceeds the preset sending amount.
It should be understood that, in the formal push stage, it is sufficient to filter out the users who have pushed the landing page in the test stage, so as to prevent the users who have received the landing page in the test stage from repeatedly receiving the same test content, which affects the user experience.
According to the method and the device, by setting the random step length, when the target user is selected, different target users are selected in the same target user set according to the random step length to carry out floor page pushing, whether the currently selected user is pushed to pass through the current floor page is judged through the historical user set, and pushing is stopped when the pushed target user amount exceeds the preset sending amount, so that the same or similar attributes among the target users are ensured, the same user is prevented from being pushed to pass through the same floor page for many times, the test repetition and waste are avoided, and the use experience of the user is optimized on the basis of increasing the accuracy of the test result.
The seventh embodiment of the present invention provides a storage medium storing a computer program which, when executed by a processor, realizes the following steps S11 and S12:
s11, setting a random step length, wherein the random step length is a value added on the basis of the number of the currently selected target user when the next target user is selected;
and S12, selecting the target users in the same target user set according to the random step length.
In the present embodiment, the storage medium may be installed in a server having an a/B test function. Since the specific steps of the user selection method for the a/B test have been described in detail in the first embodiment, details are not described in this embodiment.
Optionally, in this embodiment, the storage medium may include, but is not limited to: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes. Optionally, in this embodiment, the processor executes the method steps described in the above embodiments according to the program code stored in the storage medium. Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation manners, and this embodiment is not described herein again. It will be apparent to those skilled in the art that the modules or steps of the present invention described above may be implemented by a general purpose computing device, they may be centralized on a single computing device or distributed across a network of multiple computing devices, and alternatively, they may be implemented by program code executable by a computing device, such that they may be stored in a storage device and executed by a computing device, and in some cases, the steps shown or described may be performed in an order different than that described herein, or they may be separately fabricated into individual integrated circuit modules, or multiple ones of them may be fabricated into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.
An eighth embodiment of the present invention provides a terminal including at least a memory on which a computer program is stored, and a processor that implements the following steps S21 and S22 when executing the computer program on the memory:
s21, setting a random step length, wherein the random step length is a value added on the basis of the number of the currently selected target user when the next target user is selected;
and S22, selecting the target users in the same target user set according to the random step length.
In this embodiment, the terminal may be a server having an AR function. Since the specific steps of the user selection method for the a/B test have been described in detail in the first embodiment, details are not described in this embodiment.
Although the preferred embodiments of the present invention have been disclosed for illustrative purposes, those skilled in the art will appreciate that various modifications, additions and substitutions are possible, and the scope of the invention should not be limited to the embodiments described above.
Claims (6)
1. A user selection method for A/B test is characterized by comprising the following steps:
setting a random step length, wherein the random step length is a value added on the basis of the number of a currently selected target user when the next target user is selected;
selecting target users in the same target user set according to the number of the currently selected target users and the random step length; wherein the target user attributes in the same target user set are the same, and the attributes include at least one of: region, age group;
the setting of the random step size includes:
setting a user selection proportion, and setting the reciprocal of the user selection proportion as a user selection step length;
setting the random step length as a random value in a preset step length range according to the user selection step length, wherein the preset step length range takes the difference between the user selection step length and a preset fixed number as a lower limit and the user selection step length as an upper limit;
after the target user is selected from the target user set according to the random step length, the method further includes:
detecting whether a currently selected target user exists in a historical user set corresponding to the A/B test;
under the condition that the currently selected target user exists in a historical user set corresponding to the A/B test, discarding the currently selected target user, and selecting the next target user in the target user set according to the number of the currently selected target user and the random step length; wherein, the landing page link of the A/B test or the MD5 value of the landing page link is used as a key corresponding to the historical user set;
and under the condition that the currently selected target user does not exist in the historical user set corresponding to the A/B test, pushing the landing page of the A/B test to the currently selected target user, and adding the currently selected target user to the historical user set corresponding to the A/B test.
2. The user selection method of claim 1, wherein after pushing the landing page for the a/B test to the currently selected target user, further comprising:
detecting whether the number of target users who have pushed the landing page exceeds a preset sending amount or not;
and under the condition that the number of the target users who have pushed the landing page does not exceed the preset sending amount, selecting the next target user in the target user set according to the number of the currently selected target user and the random step length.
3. A user selection apparatus for a/B testing, comprising:
the setting module is used for setting a random step length, wherein the random step length is a value added on the basis of the number of the currently selected target user when the next target user is selected;
the selecting module is used for selecting target users in the same target user set according to the random step length;
the setting module is specifically configured to:
setting a user selection proportion, and setting the reciprocal of the user selection proportion as a user selection step length;
setting the random step length as a random value in a preset step length range according to the user selection step length, wherein the preset step length range takes the difference between the user selection step length and a preset fixed number as a lower limit and the user selection step length as an upper limit;
the device further comprises:
the first detection module is used for detecting whether the currently selected target user exists in a historical user set corresponding to the A/B test;
the selecting module is specifically configured to, when a currently selected target user exists in a historical user set corresponding to an a/B test, discard the currently selected target user, and select a next target user in the target user set according to the number of the currently selected target user and the random step length; wherein, the landing page link of the A/B test or the MD5 value of the landing page link is used as a key corresponding to the historical user set;
and the pushing module is used for pushing the landing page of the A/B test to the currently selected target user and adding the currently selected target user to the historical user set corresponding to the A/B test under the condition that the currently selected target user does not exist in the historical user set corresponding to the A/B test.
4. The user selection apparatus of claim 3, further comprising:
the second detection module is used for detecting whether the number of the target users who push the landing page exceeds a preset sending amount or not;
the selecting module is specifically configured to select a next target user in the target user set according to the number of the currently selected target user and the random step length when the number of the target users who have pushed the landing page does not exceed a preset sending amount.
5. A storage medium storing a computer program, characterized in that the computer program, when being executed by a processor, carries out the steps of the user selection method of a/B testing of any one of claims 1 to 2.
6. A terminal comprising at least a memory, a processor, said memory having stored thereon a computer program, characterized in that said processor, when executing the computer program on said memory, is adapted to carry out the steps of the user selection method of a/B testing of any of claims 1 to 2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810933965.4A CN109032954B (en) | 2018-08-16 | 2018-08-16 | User selection method and device for A/B test, storage medium and terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810933965.4A CN109032954B (en) | 2018-08-16 | 2018-08-16 | User selection method and device for A/B test, storage medium and terminal |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109032954A CN109032954A (en) | 2018-12-18 |
| CN109032954B true CN109032954B (en) | 2022-04-05 |
Family
ID=64630562
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810933965.4A Active CN109032954B (en) | 2018-08-16 | 2018-08-16 | User selection method and device for A/B test, storage medium and terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109032954B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110262958B (en) * | 2019-04-28 | 2023-05-16 | 创新先进技术有限公司 | Determination method and device for test user, server and storage medium |
| CN112882929B (en) * | 2021-02-02 | 2023-08-08 | 网易（杭州）网络有限公司 | Test method, test device, computer equipment and storage medium |
| CN112905476B (en) * | 2021-03-12 | 2023-08-11 | 网易（杭州）网络有限公司 | Test execution method and device, electronic equipment and storage medium |
| CN113064820B (en) * | 2021-03-26 | 2022-09-16 | 深圳依时货拉拉科技有限公司 | Method, apparatus and computer readable storage medium for updating A/B experiment SDK |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101141326A (en) * | 2007-09-29 | 2008-03-12 | 北京启明星辰信息技术有限公司 | Flux detecting method and system for self-adaptive sampling |
| CN101159673A (en) * | 2007-11-01 | 2008-04-09 | 杭州华三通信技术有限公司 | Arbitrary sampling method and apparatus |
| CN104102576A (en) * | 2013-04-12 | 2014-10-15 | 阿里巴巴集团控股有限公司 | Multi-version test method and device |
| CN105429950A (en) * | 2015-10-29 | 2016-03-23 | 国家计算机网络与信息安全管理中心 | Network flow identification system and method based on dynamic data packet sampling |
| CN107885808A (en) * | 2017-11-03 | 2018-04-06 | 四川文轩教育科技有限公司 | Shared resource file anti-cheating method |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6996805B2 (en) * | 2001-06-28 | 2006-02-07 | Microsoft Corporation | Methods and systems of testing software, and methods and systems of modeling user behavior |
| US7975000B2 (en) * | 2005-01-27 | 2011-07-05 | Fmr Llc | A/B testing of a webpage |
| US20090259525A1 (en) * | 2008-04-14 | 2009-10-15 | Harrington Daniel J | Internet Probability Sampling |
| KR101784105B1 (en) * | 2016-06-15 | 2017-10-10 | 에스케이 주식회사 | Service Providing Method and System based on AB Testing |
| CN108108175B (en) * | 2016-11-24 | 2021-05-25 | 百度在线网络技术（北京）有限公司 | Method and device for gray scale publishing |
-
2018
- 2018-08-16 CN CN201810933965.4A patent/CN109032954B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101141326A (en) * | 2007-09-29 | 2008-03-12 | 北京启明星辰信息技术有限公司 | Flux detecting method and system for self-adaptive sampling |
| CN101159673A (en) * | 2007-11-01 | 2008-04-09 | 杭州华三通信技术有限公司 | Arbitrary sampling method and apparatus |
| CN104102576A (en) * | 2013-04-12 | 2014-10-15 | 阿里巴巴集团控股有限公司 | Multi-version test method and device |
| CN105429950A (en) * | 2015-10-29 | 2016-03-23 | 国家计算机网络与信息安全管理中心 | Network flow identification system and method based on dynamic data packet sampling |
| CN107885808A (en) * | 2017-11-03 | 2018-04-06 | 四川文轩教育科技有限公司 | Shared resource file anti-cheating method |
Non-Patent Citations (3)
| Title |
|---|
| A Limit Theorem for Random Interval Sampling of a Stochastic Process;Richard F.Gebhard;《The Annals of Mathematical Statistics》;19641231;第35卷(第2期);全文 * |
| 一种改进的随机抽样算法;王军锋 贾建华等;《电脑与信息技术》;20060228;第14卷(第2期);3 * |
| 现代抽样技术介绍(一);吴辉;《统计研究》;19850131;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109032954A (en) | 2018-12-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109032954B (en) | User selection method and device for A/B test, storage medium and terminal | |
| CN106294120B (en) | Method, apparatus and computer program product for testing code | |
| CN106294508B (en) | Brushing amount tool detection method and device | |
| CN104994060B (en) | It is a kind of to provide the method and apparatus of verifying for logging in for user | |
| KR101904436B1 (en) | Machine learning based identification of broken network connections | |
| WO2020119426A1 (en) | Gray scale testing method, server and computer readable storage medium | |
| US9971892B2 (en) | Method, apparatus and computer device for scanning information to be scanned | |
| CN111124911A (en) | Automatic testing method, device, equipment and readable storage medium | |
| CN102256337A (en) | Message processing method and equipment in wireless local area network (WLAN) | |
| CN103198243A (en) | Method and device for identifying emulational application programs | |
| WO2022095847A1 (en) | System upgrading method and apparatus, device, and storage medium | |
| CN112333246B (en) | ABtest experiment method and device, intelligent terminal and storage medium | |
| CN108632085B (en) | Gray level user management method, device, platform and storage medium | |
| CN110209548B (en) | Service control method, system, electronic device and computer readable storage medium | |
| CN115756812A (en) | Resource adjusting method and device and storage medium | |
| CN105138451A (en) | Method, apparatus and system for testing plugin function | |
| CN114064445A (en) | Test method, device, equipment and computer readable storage medium | |
| CN109726124B (en) | Test system, test method, management device, test device and computing equipment | |
| CN110019054B (en) | Log duplicate removal method and system, and content distribution network system | |
| CN111194059A (en) | Data connection method and device, electronic equipment and readable storage medium | |
| CN107908555B (en) | SQL script abnormity detection method and terminal thereof | |
| CN111625720B (en) | Method, device, equipment and medium for determining execution strategy of data decision item | |
| CN105630503B (en) | A kind of application and development mobile device choosing method based on user operation records | |
| CN111142898B (en) | Data anti-leakage terminal upgrading method and system based on group intelligent mode | |
| CN109803160B (en) | Resource allocation method, device and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |