CN113254882B - Method, device and equipment for determining experimental result and storage medium - Google Patents
Method, device and equipment for determining experimental result and storage medium Download PDFInfo
- Publication number
- CN113254882B CN113254882B CN202110633686.8A CN202110633686A CN113254882B CN 113254882 B CN113254882 B CN 113254882B CN 202110633686 A CN202110633686 A CN 202110633686A CN 113254882 B CN113254882 B CN 113254882B
- Authority
- CN
- China
- Prior art keywords
- experimental
- group under
- index
- control group
- under
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 60
- 238000009826 distribution Methods 0.000 claims abstract description 211
- 238000002474 experimental method Methods 0.000 claims abstract description 67
- 230000006870 function Effects 0.000 claims description 94
- 238000004590 computer program Methods 0.000 claims description 14
- 238000012545 processing Methods 0.000 claims description 12
- 238000012360 testing method Methods 0.000 claims description 6
- 239000002131 composite material Substances 0.000 claims description 3
- 238000009827 uniform distribution Methods 0.000 claims description 3
- 230000008569 process Effects 0.000 description 18
- 238000011156 evaluation Methods 0.000 description 10
- 238000005070 sampling Methods 0.000 description 9
- 238000013398 bayesian method Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000010354 integration Effects 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 230000015572 biosynthetic process Effects 0.000 description 2
- 230000014759 maintenance of location Effects 0.000 description 2
- 238000003786 synthesis reaction Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000004744 fabric Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000013441 quality evaluation Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/18—Complex mathematical operations for evaluating statistical data, e.g. average values, frequency distributions, probability functions, regression analysis
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Optimization (AREA)
- Pure & Applied Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Mathematical Physics (AREA)
- Computational Mathematics (AREA)
- Mathematical Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Operations Research (AREA)
- Probability & Statistics with Applications (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Algebra (AREA)
- Evolutionary Biology (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Computational Biology (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Complex Calculations (AREA)
Abstract
The application discloses a method, a device, equipment and a storage medium for determining an experimental result, and belongs to the technical field of computers. The method comprises the following steps: acquiring user experiment data in an AB experiment; based on the user experiment data, obtaining mean posterior distribution of the experiment group under n grading indexes and mean posterior distribution of the contrast group under n grading indexes; determining the score corresponding to the experimental group based on the mean posterior distribution of the experimental group under the n scoring indexes, and determining the score corresponding to the control group based on the mean posterior distribution of the control group under the n scoring indexes; and comparing the scores corresponding to the experimental group with the scores corresponding to the control group to obtain an AB experimental result. According to the method and the device, the respective scores of the experimental group and the control group are determined through mean posterior distribution based on a plurality of scoring indexes, and then a better group is determined based on the scores, so that the automatic determination of the AB experimental result under the plurality of scoring indexes is realized, and the determination efficiency of the AB experimental result is improved.
Description
Technical Field
The embodiment of the application relates to the technical field of computers, in particular to a method, a device, equipment and a storage medium for determining an experimental result.
Background
In an internet scene, designers can test the advantages and disadvantages of new products, new schemes and the like through an AB experimental platform.
Taking a live broadcast scene as an example, designers compare a new product with a current online product in a randomized experiment mode through an AB experiment platform. For example, the users are divided into an experimental group and a control group in a uniform division mode, the experimental group corresponds to a new product, the control group corresponds to a current online product, then data under scoring indexes such as user browsing volume, user remaining time, per-capita watching time, basic interaction rate and the like are counted, and finally the data under a certain scoring index of the experimental group and the data under a certain scoring index of the control group are compared to determine an AB experimental result under the scoring index.
However, the above randomization experiment mode is based on univariate hypothesis verification, that is, only one dimensional scoring index can be compared at a time, and in the case of multidimensional scoring indexes, the AB experiment results under each scoring index need to be manually combined for comparison to obtain the final AB experiment result, and the determination efficiency of the AB experiment results is not high.
Disclosure of Invention
The embodiment of the application provides a method, a device, equipment and a storage medium for determining an experimental result, which can be used for automatically generating an AB experimental result in a quantitative mode based on a multidimensional scoring index, and improving the determining efficiency and the reasonability of the AB experimental result. The technical scheme is as follows:
according to an aspect of an embodiment of the present application, there is provided a method for determining an experimental result, the method including:
acquiring user experiment data in an AB experiment; the user experimental data comprise experimental data of an experimental group and a control group under n grading indexes respectively, wherein n is a positive integer;
acquiring mean posterior distribution of the experimental group under the n scoring indexes based on the experimental data of the experimental group under the n scoring indexes, and acquiring mean posterior distribution of the control group under the n scoring indexes based on the experimental data of the control group under the n scoring indexes, wherein the mean posterior distribution is posterior probability distribution of the mean of the experimental data under the scoring indexes;
determining the score corresponding to the experimental group based on the mean posterior distribution of the experimental group under the n scoring indexes, and determining the score corresponding to the control group based on the mean posterior distribution of the control group under the n scoring indexes;
and comparing the scores corresponding to the experimental group with the scores corresponding to the control group to obtain the AB experimental result.
According to an aspect of an embodiment of the present application, there is provided an apparatus for determining an experimental result, the apparatus including:
acquiring user experiment data in an AB experiment; the user experimental data comprises experimental data of an experimental group and experimental data of a control group under n grading indexes, wherein n is a positive integer;
acquiring mean posterior distribution of the experimental group under the n scoring indexes based on the experimental data of the experimental group under the n scoring indexes, and acquiring mean posterior distribution of the control group under the n scoring indexes based on the experimental data of the control group under the n scoring indexes, wherein the mean posterior distribution is posterior probability distribution of the mean of the experimental data under the scoring indexes;
determining the scores corresponding to the experimental group based on the mean posterior distribution of the experimental group under the n scoring indexes, and determining the scores corresponding to the control group based on the mean posterior distribution of the control group under the n scoring indexes;
and comparing the scores corresponding to the experimental group with the scores corresponding to the control group to obtain the AB experimental result.
According to an aspect of the embodiments of the present application, there is provided a computer device, the computer device including a processor and a memory, the memory storing a computer program, the computer program being loaded and executed by the processor to implement the above determination method of experimental results.
According to an aspect of the embodiments of the present application, there is provided a computer-readable storage medium, in which a computer program is stored, the computer program being loaded and executed by a processor to implement the method for determining the experimental result.
According to an aspect of an embodiment of the present application, there is provided a computer program product or a computer program comprising computer instructions stored in a computer readable storage medium. The computer instructions are read by a processor of the computer device from the computer-readable storage medium, and the computer instructions are executed by the processor to cause the computer device to perform the method for determining the experimental result.
The technical scheme provided by the embodiment of the application can bring the following beneficial effects:
the method comprises the steps of determining the mean posterior distribution of an experimental group under each grading index and the mean posterior distribution of a contrast group under each grading index based on user experimental data in an AB experiment, determining the grade of the experimental group and the grade of the contrast group respectively based on the mean posterior distribution of the experimental group under each grading index and the mean posterior distribution of the contrast group under each grading index, and finally determining an AB experimental result based on the grade of the experimental group and the grade of the contrast group.
In addition, the score is determined by combining the mean posterior distribution under the multidimensional scoring index, so that the reasonability and comprehensiveness of the score are improved, and the reasonability and comprehensiveness of the AB experimental result are further improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the description below are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
FIG. 1 is a schematic illustration of an environment for implementing an embodiment provided by an embodiment of the present application;
FIG. 2 is a flow chart of a method of determining experimental results provided by one embodiment of the present application;
FIG. 3 is a chart of user experimental data provided in one embodiment of the present application;
FIG. 4 is a graph of a mean posterior distribution provided by one embodiment of the present application;
FIG. 5 is a schematic illustration of an experimental interface of an AB experimental platform provided in one embodiment of the present application;
fig. 6 is a block diagram of an experimental result determination apparatus according to an embodiment of the present application.
Detailed Description
To make the objects, technical solutions and advantages of the present application more clear, the following detailed description of the embodiments of the present application will be made with reference to the accompanying drawings.
Refer to fig. 1, which illustrates a schematic diagram of an environment for implementing an embodiment of the present application. The embodiment implementation environment can be realized to be the architecture of the AB experiment system. The embodiment implementation environment may include: terminal 10 and AB experiment platform 20.
The terminal 10 refers to a terminal device used by a user. The terminal 10 may be an electronic device such as a mobile phone, a tablet Computer, a PC (Personal Computer), a wearable device, and the like. A client running a target application may be installed in the terminal 10. A user may access a client of a target application through the terminal 10. The target application may be a live application, a video application, a shopping application, and the like, which is not limited in this embodiment of the present application. Alternatively, a client running a target application under different versions of the schema may be installed in the terminal 10, such as a client corresponding to a new schema, a client corresponding to a current online schema, and so on.
The AB experiment platform 20 is a test platform, and the AB experiment platform 20 can provide test services for products, solutions, and the like. In one example, the AB experiment platform 20 determines the quality of the product solution by means of a randomized experiment. For example, the advantages and disadvantages of the new product are determined by comparing the new product with the current online product based on the data of the new product under the multiple scoring indexes and the data of the current online product under the multiple scoring indexes. Optionally, the AB experiment platform 20 includes one or more servers, which may be background servers of the AB experiment platform 20, for providing background services, such as data statistics, data calculations, data ratio peers, to the AB experiment platform 20.
The terminal 10 and the AB experiment platform 20 can communicate via a network.
Illustratively, the AB experiment platform 20 summarizes the user experiment data from the terminal 10 corresponding to the new product and the user experiment data from the terminal 10 corresponding to the current online product, and compares the new product with the current online product based on the user experiment data to determine the quality of the new product.
Please refer to fig. 2, which illustrates a flowchart of a method for determining an experimental result according to an embodiment of the present application. The execution subject of each step of the method may be the AB experiment platform 20 described above, such as a server corresponding to the AB experiment platform 20. The method can comprise the following steps (201-204):
In the embodiment of the present application, the AB experiment refers to an experiment for judging the quality of a product, a scheme, and the like by using the AB experiment platform. In the AB experiment, a user is divided into an experiment group and a control group by an AB experiment platform in a uniform division mode. Optionally, the user corresponding to the experimental group uses a new product, a new scheme, and the like, and the user corresponding to the control group uses a current online product, a current online scheme, and the like. The user experiment data includes data formed when the user corresponding to the experiment group uses a new product, a new scheme and the like and data formed when the user corresponding to the comparison group uses a current online product, a current online scheme and the like, that is, the user experiment data includes experiment data corresponding to the experiment group and experiment data corresponding to the comparison group. The experimental data corresponding to the control group is used as a comparison object of the experimental data corresponding to the experimental group, that is, the experimental data corresponding to the current online product, the current online scheme and the like is used as a quality evaluation criterion of the new product and the new scheme.
In one example, the groupings may be evenly grouped by mantissa of the user's ID (Identity document). For example, taking the live application as an example, each user is assigned an ID when registering in the client corresponding to the live application, and users whose mantissas of the ID are odd may be grouped into one group, and users whose mantissas of the ID are even may be grouped into one group.
Optionally, the scoring index is a consideration index of the AB experiment platform in a scoring process. And under the condition that n is equal to 1, the AB experiment platform only needs to respectively acquire the experiment data corresponding to the scoring index of the experiment group and the control group. And under the condition that n is greater than 1, the AB experimental platform needs to acquire experimental data of the experimental group and the control group under n scoring indexes respectively. For example, taking a live/short video application as an example, important scoring indexes of the live/short video application may include a user browsing amount, a user retention time, a per-person watching time, a basic interaction rate, and the like, and the AB experimental platform needs to acquire experimental data of the experimental group and the control group respectively under the scoring indexes of the user browsing amount, the user retention time, the per-person watching time, the basic interaction rate, and the like.
In one example, the process of obtaining the mean posterior distribution may be as follows: for a target scoring index in the n scoring indexes, determining the probability density function of the experimental data of the experimental group under the target scoring index and the prior probability distribution of the comprehensive parameter based on the experimental data of the experimental group under the target scoring index, and determining the probability density function of the experimental data of the control group under the target scoring index and the prior probability distribution of the comprehensive parameter based on the experimental data of the control group under the target scoring index; the comprehensive parameters are used for constructing probability density functions corresponding to the experimental data under the grading indexes, and the comprehensive parameters comprise variable parameters used for representing the mean value of the experimental data under the grading indexes; determining the probability density function of the comprehensive parameters of the experimental group under the target scoring index based on the probability density function of the experimental data of the experimental group under the target scoring index and the prior probability distribution of the comprehensive parameters, and determining the probability density function of the comprehensive parameters of the control group under the target scoring index based on the probability density function of the experimental data of the control group under the target scoring index and the prior probability distribution of the comprehensive parameters; and performing integral processing on the probability density functions of the comprehensive parameters of the experimental group under the target scoring index to obtain mean posterior distribution of the experimental group under the target scoring index, and performing integral processing on the probability density functions of the comprehensive parameters of the comparison group under the target scoring index to obtain mean posterior distribution of the comparison group under the target scoring index.
The target scoring index may be any one of the n scoring indexes. The probability density function, which is used to describe the probability density of each data value in the experimental data, may be influenced by one or more variable parameters, which are collectively referred to herein as synthesis parameters.
Illustratively, the determination process of the mean posterior distribution of the experimental group under the target score index is taken as an example. The distribution obeyed by the experimental data of the experimental group under the target scoring index, the probability density function of the experimental data of the experimental group under the target scoring index and the prior probability distribution of the comprehensive parameters corresponding to the probability density function can be determined based on the experimental data of the experimental group under the target scoring index according to expert knowledge. The prior probability distribution of the comprehensive parameters comprises the prior probability distribution of each variable parameter.
Optionally, the distribution to which the experimental data of the experimental group under the target scoring index obeys can be determined from a plurality of preset distributions. For example, if the preset distribution a considers that the probability of the batch of experimental data is 0.6, the preset distribution B considers that the probability of the batch of experimental data is 0.8, and the preset distribution C considers that the probability of the batch of experimental data is 0.95, the preset distribution C with the highest probability can be determined as the distribution to which the experimental data obeys. After the distribution obeyed by the experimental data is obtained, the probability density function corresponding to the experimental data and the prior probability distribution of the comprehensive parameters corresponding to the probability density function can be further determined.
According to the Bayesian method, the posterior probability distribution of comprehensive parameters corresponding to the experimental data of the experimental group under the target scoring index can be learned. The posterior probability distribution of the integrated parameter is proportional to the product of the likelihood function corresponding to the integrated parameter and the prior probability value of the integrated parameter, which can be expressed by the following formula:
P(θ|D)∝P(D|θ)P(θ)；
wherein, P (D | theta) is posterior probability distribution of the comprehensive parameter, theta is the comprehensive parameter, D is experimental data, P (D | theta) is a likelihood function corresponding to the comprehensive parameter, and P (theta) is prior probability value of the comprehensive parameter. And the likelihood function corresponding to the comprehensive parameters is the probability density function corresponding to the experimental data.
For any one of the determined integrated parameters, the posterior probability value corresponding to the integrated parameter can be calculated based on the Bayesian method. Therefore, the embodiment of the application can determine the probability density function of the comprehensive parameters by adopting an equidistant sampling mode, and the specific contents are as follows:
1. based on the probability value of each data value in the experimental data under the comprehensive parameter of the determined value, the probability value corresponding to the likelihood function under the comprehensive parameter of the determined value is determined, and the process can be represented by the following relational expression:
wherein, theta 0 For defining the combined parameter of values, f is the probability density function of the experimental data, x N Is the nth data value.
2. Determining the prior probability value of the integrated parameter of the determined values based on the prior probability value of the variable parameter of each determined value included in the integrated parameter of the determined values, wherein the process can be represented by the following relational expression:
wherein the content of the first and second substances,
a variable parameter for the mth determined value,
a priori probability values of the variable parameters for the mth determined value. Optionally, the prior probability distribution of the mth variable parameter may be determined according to expert knowledge, and then the prior probability value of the mth variable parameter is determined based on the prior probability distribution of the mth variable parameter.
3. And determining the posterior probability value of the comprehensive parameter of the determined value based on the product of the probability value corresponding to the likelihood function under the comprehensive parameter of the determined value and the prior probability value of the comprehensive parameter of the determined value.
We consider the integrated parameter as a parameter space, and for any point in the parameter space (i.e. any integrated parameter of a certain value), we can obtain a posterior probability value of the integrated parameter of a corresponding certain value. Dividing a sufficient number of points (e.g., 1000000) at uniform intervals in this parameter space, the probability density function for the composite parameter can be determined.
After the probability density function of the comprehensive parameters is obtained, the mean posterior distribution of the experimental group under the target scoring index can be obtained only by removing the variable parameter integration except the mean variable parameter.
In another example, the process of obtaining the mean posterior distribution may also be as follows: for a target scoring index in the n scoring indexes, determining a probability density function of a comprehensive parameter of the experimental group under the target scoring index based on experimental data of the experimental group under the target scoring index by adopting a Markov Chain Monte Carlo (MCMC) method, and determining a probability density function of a comprehensive parameter of the contrast group under the target scoring index based on experimental data of the contrast group under the target scoring index; the comprehensive parameters are used for constructing probability density functions corresponding to the experimental data under the grading indexes, and the comprehensive parameters comprise variable parameters used for representing the mean value of the experimental data under the grading indexes; and performing integral processing on the probability density functions of the comprehensive parameters of the experimental group under the target scoring index to obtain the mean posterior distribution of the experimental group under the target scoring index, and performing integral processing on the probability density functions of the comprehensive parameters of the comparison group under the target scoring index to obtain the mean posterior distribution of the comparison group under the target scoring index.
Illustratively, the determination process of the mean posterior distribution of the experimental group under the target score index is taken as an example. The distribution obeyed by the experimental data of the experimental group under the target scoring index, the probability density function of the experimental data of the experimental group under the target scoring index and the prior probability distribution of the comprehensive parameters corresponding to the probability density function can be determined based on the experimental data of the experimental group under the target scoring index according to expert knowledge.
Based on the MCMC (Markov Chain Monte Carlo) method, we can construct a Markov Chain representing a sequence of values of a synthesis parameter over a period of time that satisfies Markov properties. Sampling enough sampling values of the comprehensive parameters from the Markov chain, and enabling the distribution of the sampling values of the comprehensive parameters to be approximate to the distribution corresponding to the probability density function of the comprehensive parameters through continuous convergence in the sampling process, thereby obtaining the probability density function of the comprehensive parameters. By adopting the MCMC method, the method and the device can improve the efficiency of acquiring the probability density function of the comprehensive parameters.
After the probability density function of the comprehensive parameters is obtained, the mean posterior distribution of the experimental group under the target scoring index can be obtained only by removing the variable parameter integration except the mean variable parameter.
And 203, determining the scores corresponding to the experimental group based on the mean posterior distribution of the experimental group under the n scoring indexes, and determining the scores corresponding to the control group based on the mean posterior distribution of the control group under the n scoring indexes.
Alternatively, the process of obtaining the scores corresponding to the experimental group and the scores corresponding to the control group may be as follows: acquiring mean values respectively corresponding to the mean posterior distribution of the experimental group under the n scoring indexes, and acquiring mean values respectively corresponding to the mean posterior distribution of the control group under the n scoring indexes; acquiring weight parameters corresponding to the n scoring indexes respectively; based on the weight parameters respectively corresponding to the n scoring indexes, carrying out weighted summation on mean values respectively corresponding to the posterior distribution of the mean values of the experimental group under the n scoring indexes to obtain scores corresponding to the experimental group; and based on the weight parameters corresponding to the n scoring indexes, carrying out weighted summation on the mean values corresponding to the mean posterior distribution of the comparison group under the n scoring indexes to obtain the scores corresponding to the comparison group.
Wherein, the weight parameter corresponding to each scoring index can be determined according to expert knowledge. Alternatively, the weighting parameter corresponding to the important score index may be set to be higher than the weighting parameter corresponding to the second important score index.
Illustratively, the calculation method of the score may be represented by the following formula:
U＝c 1 ∫f 1 (x)xdx+c 2 ∫f 2 (x)xdx+...+c K ∫f K (x)xdx；
wherein U is the score, c K The weight parameter, which is the kth scoring index, [ integral ] f K (x) xdx is the mean of the mean posterior distribution under the Kth scoring index, f K (x) The K-th score index is the mean posterior distribution, and x is the possible mean of the experimental data under the score index.
Alternatively, the scores of the experimental group under the n scoring indexes and the scores of the control group under the n scoring indexes can be obtained based on the above-mentioned scoring calculation method.
And 204, comparing the scores corresponding to the experimental group with the scores corresponding to the control group to obtain an AB experimental result.
Optionally, if the score corresponding to the experimental group is larger than the score corresponding to the control group, taking the product corresponding to the experimental group as the AB experimental result; and if the score corresponding to the experimental group is smaller than the score corresponding to the control group, taking the product corresponding to the control group as the AB experimental result. Wherein, the AB experimental result can be used for expressing products, schemes and the like with better effect; the AB test results can also be used to indicate products, schemes, etc. with better yield, and the examples of the present application are not limited herein. Optionally, the case that the score corresponding to the experimental group is equal to the score corresponding to the control group may be classified as the case that the score corresponding to the experimental group is greater than the score corresponding to the control group, or may be classified as the case that the score corresponding to the experimental group is less than the score corresponding to the control group.
To sum up, the technical scheme that this application embodiment provided, through based on the user experiment data in the AB experiment, confirm that the mean posterior distribution of experimental group under each mark index and the mean posterior distribution of contrast group under each mark index, again based on the mean posterior distribution of experimental group under each mark index and the mean posterior distribution of contrast group under each mark index, confirm the grade of experimental group and the grade of contrast group respectively, confirm the AB experimental result based on the grade of experimental group and the grade of contrast group at last, the automatic of AB experimental result under the mark index of multidimension degree is confirmed, and need not to carry out manual comparison, thereby the definite efficiency of AB experimental result has been improved.
In addition, the score is determined by combining the mean posterior distribution under the multidimensional score index, so that the reasonability and the comprehensiveness of the score are improved, and the reasonability and the comprehensiveness of the AB experiment result are further improved.
In an exemplary embodiment, the generation process of the experimental data of each scoring index can be regarded as a process of sampling from a distribution of random variables, and the experimental data under each scoring index can be assumed to be subject to a t-distribution according to expert knowledge.
For example, the experimental data under the target evaluation index described above are taken as an example. It can be assumed from expert knowledge that the experimental data under the target scoring index obey a t-distribution including a first variable parameter and a second variable parameter for describing the shape of the t-distribution, and a third variable parameter for describing the mean of the experimental data under the target scoring index; the prior distribution of the first variable parameters is exponential distribution, the prior distribution of the second variable parameters is uniform distribution, and the prior distribution of the third variable parameters is Gaussian distribution.
Alternatively, the process of obtaining the corresponding mean posterior distribution of the experimental group and the control group under the target evaluation index may be as follows:
1. the probability density function of the t-distribution is:
where f (x | μ, λ, v) is a probability density function of the t distribution, μ is used to represent a mean value (i.e., a third variable parameter) corresponding to the t distribution, and λ and v are used to describe the shape of the probability density function (i.e., the first variable parameter and the second variable parameter).
2. The second variable parameters of the experimental group and the control group under the target scoring index share a set of prior probability distribution, which is an index distribution with a parameter of 1/30 (the parameter is an empirical value, and is only exemplary and can be adjusted according to actual conditions), and can be represented as follows:
3. the prior probability distribution of the first variable parameter under the target scoring index of the experimental group and the control group respectively can be expressed as follows:
σ g1 ∝U[1，10]
σ g2 ∝U[1，10]
λ A ＝σ g1 -2
λ B ＝σ g2 -2 ；
wherein λ is A Is the first variable parameter, lambda, of the experimental group under the target scoring index B Is the first variable parameter of the control group under the target scoring index.
4. The prior probability distribution of the third variable parameter under the target scoring index of the experimental group and the control group can be expressed as follows:
wherein, mu A Is the third variable parameter, mu, of the experimental group under the target scoring index B Is the third variable parameter of the control group under the target scoring index,
the mean value corresponding to the total experimental data of the experimental group and the control group under the target scoring index, and s (X) is the variance value corresponding to the total experimental data of the experimental group and the control group under the target scoring index.
Optionally, the obtaining process of the probability density function of the first variable parameter, the second variable parameter and the third variable parameter may be as follows: considering the parameter space as a three-dimensional space, each point in the parameter space can be represented by a first variable parameter, a second variable parameter and a third variable parameter. Enough grid points (for example, 1000000) are divided in the parameter space at uniform intervals, for each grid point, a bayesian method is adopted to calculate a posterior probability value corresponding to the grid point, and then a probability density function of the first variable parameter, the second variable parameter and the third variable parameter can be obtained based on the enough grid points and the posterior probability values corresponding to the enough grid points. Alternatively, the posterior probability value for each point may be calculated as follows:
1. and determining the probability value corresponding to the target grid point based on the probability value of each data value in the experimental data under the target scoring index under the target grid point.
2. And determining the prior probability value of the target grid point based on the prior probability distribution of the first variable parameter, the second variable parameter and the third variable parameter.
3. And determining the posterior probability value of the target lattice point by using a Bayesian method based on the product of the probability value corresponding to the target lattice point and the prior probability value of the target lattice point.
After the probability density functions of the first variable parameter, the second variable parameter and the third variable parameter are obtained, the posterior probability distribution of the third variable parameter, that is, the mean posterior distribution under the target scoring index, can be obtained by only removing the first variable parameter and the second variable parameter through integration, and the process can be expressed as follows:
wherein v is a first variable parameter, lambda is a second variable parameter, mu is a third variable parameter, and D is experimental data under a target scoring index.
Optionally, the obtaining process of the probability density function of the first variable parameter, the second variable parameter and the third variable parameter may further include the following steps: based on the MCMC method, a three-dimensional known distribution can be selected, a Markov chain about a first variable parameter, a second variable parameter and a third variable parameter is constructed based on the known distribution, enough combined sampling value sequences of the first variable parameter, the second variable parameter and the third variable parameter are sampled from the Markov chain, and the distribution of the combined sampling values of the first variable parameter, the second variable parameter and the third variable parameter is enabled to be approximate to the distribution corresponding to the probability density functions corresponding to the first variable parameter, the second variable parameter and the third variable parameter through continuous convergence in the sampling process, so that the probability density functions about the first variable parameter, the second variable parameter and the third variable parameter can be obtained.
After the probability density functions of the first variable parameter, the second variable parameter and the third variable parameter are obtained, the posterior probability distribution of the third variable parameter, namely the mean posterior distribution under the target scoring index, can be obtained only by removing the integration of the first variable parameter and the second variable parameter.
Finally, based on the method, the mean posterior distribution of the experimental group under the target evaluation index and the mean posterior distribution of the contrast group under the target evaluation index can be determined, and further based on the method, the mean posterior distribution of the experimental group under each evaluation index and the mean posterior distribution of the contrast group under each evaluation index can be determined.
Optionally, after determining the mean posterior distribution of the experimental group under each evaluation index and the mean posterior distribution of the control group under each evaluation index, the score corresponding to the experimental group may be calculated based on the mean posterior distribution of the experimental group under each evaluation index, and the score corresponding to the control group may be calculated based on the mean posterior distribution of the control group under each evaluation index, where the specific process may be represented as follows:
wherein U (A) is the score corresponding to the experimental group, U (B) is the score corresponding to the control group, and c K Is composed of
The average value of the cloth is calculated,
the mean posterior distribution of the experimental groups under the kth scoring index,
the mean posterior distribution of the control group under the Kth scoring index is shown, and x is the possible mean of the experimental data under the scoring index. c. C 1 ，c 2 ，...，c K The weight parameter of each scoring index evaluated by the industry experts represents the importance degree of each scoring index.
Finally, the grade corresponding to the experimental group and the grade corresponding to the control group are only needed to be compared, and the product, scheme and the like corresponding to the group can be determined to be better. Exemplarily, if the score corresponding to the experimental group is larger than the score corresponding to the control group, the product, the scheme and the like corresponding to the experimental group are better; if the score corresponding to the experimental group is smaller than or equal to the score corresponding to the control group, the products, schemes and the like corresponding to the control group are better.
To sum up, the technical scheme that this application embodiment provided, through based on the user experiment data in the AB experiment, confirm that the mean posterior distribution of experimental group under each mark index and the mean posterior distribution of contrast group under each mark index, again based on the mean posterior distribution of experimental group under each mark index and the mean posterior distribution of contrast group under each mark index, confirm the grade of experimental group and the grade of contrast group respectively, confirm the AB experimental result based on the grade of experimental group and the grade of contrast group at last, the automatic of AB experimental result under the mark index of multidimension degree is confirmed, and need not to carry out manual comparison, thereby the definite efficiency of AB experimental result has been improved.
In addition, the score is determined by combining the mean posterior distribution under the multidimensional scoring index, so that the reasonability and comprehensiveness of the score are improved, and the reasonability and comprehensiveness of the AB experimental result are further improved.
In an exemplary embodiment, the short video application is taken as an example, and the effect of two different configuration schemes on the short video platform is compared. The AB experiment included two core scoring indices: the number of video views per user per week and the number of hours each user logs in per week. Referring to fig. 3, a graph 301 is experimental data of an experimental group under a rating index of the number of videos watched per week by each user, a graph 302 is experimental data of a control group under a rating index of the number of videos watched per week by each user, a graph 303 is experimental data of an experimental group under a rating index of the number of hours each user logs in per week, and a graph 304 is experimental data of a control group under a rating index of the number of hours each user logs in per week.
Assuming that the experimental data all obey t distribution, by adopting the technical scheme provided by the embodiment of the application, the mean posterior distribution of the experimental group under the two core indexes and the mean posterior distribution of the control group under the two core indexes can be obtained. Referring to fig. 4, a graph 401 is a mean posterior distribution of an experimental group under a rating index of a number of video views per week of each user (94% of possible means are distributed in a range of 3.2 to 4.3), a graph 402 is a mean posterior distribution of a control group under a rating index of a number of video views per week of each user (94% of possible means are distributed in a range of 2.8 to 3.8), a graph 403 is a mean posterior distribution of an experimental group under a rating index of a number of hours each user logs in per week (94% of possible means are distributed in a range of 2.6 to 3.6), and a graph 404 is a mean posterior distribution of a control group under a rating index of a number of hours each user logs in per week (94% of possible means are distributed in a range of 3.3 to 4.9).
The mean value of the mean posterior distribution may be calculated after the mean posterior distribution is obtained. Referring to fig. 4, the mean value of the mean posterior distribution of the experimental group in the scoring index of the number of video views per week of each user is 3.8, the mean value of the mean posterior distribution of the control group in the scoring index of the number of video views per week of each user is 3.3, the mean value of the mean posterior distribution of the experimental group in the scoring index of the number of hours per week of login of each user is 3.1, and the mean value of the mean posterior distribution of the scoring index of the number of hours per week of login of each user of the control group is 4.1.
Based on expert knowledge, the short video application program pays more attention to the score index of the number of video views per week of each user, so that a weight parameter of 0.7 can be allocated to the score index of the number of video views per week of each user, and a weight parameter of 0.3 can be allocated to the score index of the number of hours of login per week of each user. Then the corresponding score of the experimental group is 0.7 + 3.8+0.3 + 3.1=3.59, the corresponding score of the control group is 0.7 + 3.3+0.3 + 4.1=3.54, and the corresponding score of the experimental group is greater than the corresponding score of the control group, so that the corresponding configuration scheme of the experimental group has better effect on the short video platform.
In an exemplary embodiment, reference is made to fig. 5, which illustrates a schematic diagram of an experimental interface of an AB experimental platform provided in an embodiment of the present application. In the experiment interface 501, a user may select experiment data for a target time period required for an experiment; the user can also select the scoring indexes to form a scoring index group, and the scoring indexes in the scoring index group can be switched and selected according to requirements. And the AB experimental platform determines the scores of the control group under each score index and the scores of the control group under the score index group, the scores of the experimental group under each score index and the scores of the experimental group under the score index group according to the experimental data under each score index, and gives the final AB experimental result, namely which group of the experimental group and the control group corresponds to a better product, scheme and the like. And the scores respectively corresponding to each score index can be used for the reference of the user.
To sum up, the technical scheme provided by the embodiment of the application determines the mean posterior distribution of the experimental group under each scoring index and the mean posterior distribution of the control group under each scoring index based on the user experimental data in the AB experiment, determines the scores of the experimental group and the scores of the control group respectively based on the mean posterior distribution of the experimental group under each scoring index and the mean posterior distribution of the control group under each scoring index, and determines the AB experimental result based on the scores of the experimental group and the scores of the control group.
In addition, the score is determined by combining the mean posterior distribution under the multidimensional scoring index, so that the reasonability and comprehensiveness of the score are improved, and the reasonability and comprehensiveness of the AB experimental result are further improved.
Please refer to fig. 6, which shows a block diagram of an experimental result determination apparatus provided in an embodiment of the present application. The device has the function of realizing the determination method example of the experimental result, and the function can be realized by hardware or by hardware executing corresponding software. The device can be a computer device, and can also be arranged in the computer device. The apparatus 600 may include: an experimental data acquisition module 601, a mean distribution acquisition module 602, a score determination module 603, and an experimental result acquisition module 604.
An experiment data acquisition module 601, configured to acquire user experiment data in an AB experiment; the user experimental data comprise experimental data of an experimental group and a control group under n grading indexes respectively, wherein n is a positive integer.
The mean distribution obtaining module 602 is configured to obtain mean posterior distribution of the experimental group under the n scoring indicators based on the experimental data of the experimental group under the n scoring indicators, and obtain mean posterior distribution of the control group under the n scoring indicators based on the experimental data of the control group under the n scoring indicators, where the mean posterior distribution is posterior probability distribution of a mean of the experimental data under the scoring indicators.
A score determining module 603, configured to determine a score corresponding to the experimental group based on the mean posterior distribution of the experimental group under the n scoring indexes, and determine a score corresponding to the control group based on the mean posterior distribution of the control group under the n scoring indexes.
An experimental result obtaining module 604, configured to compare the score corresponding to the experimental group with the score corresponding to the control group, so as to obtain the AB experimental result.
In an exemplary embodiment, the score determining module 603 is configured to:
acquiring mean values respectively corresponding to the mean posterior distribution of the experimental group under the n grading indexes, and acquiring mean values respectively corresponding to the mean posterior distribution of the control group under the n grading indexes;
acquiring weight parameters corresponding to the n scoring indexes respectively;
based on the weight parameters corresponding to the n scoring indexes, respectively, carrying out weighted summation on the mean values corresponding to the mean posterior distribution of the experimental group under the n scoring indexes to obtain the scores corresponding to the experimental group;
and based on the weight parameters corresponding to the n scoring indexes, carrying out weighted summation on the mean values corresponding to the mean posterior distribution of the comparison group under the n scoring indexes to obtain the scores corresponding to the comparison group.
In an exemplary embodiment, the mean distribution obtaining module 602 is configured to:
for a target scoring index in the n scoring indexes, determining a probability density function and a prior probability distribution of a comprehensive parameter of experimental data of the experimental group under the target scoring index based on the experimental data of the experimental group under the target scoring index, and determining a probability density function and a prior probability distribution of a comprehensive parameter of experimental data of the control group under the target scoring index based on the experimental data of the control group under the target scoring index; the comprehensive parameters are used for constructing probability density functions corresponding to the experimental data under the grading indexes, and the comprehensive parameters comprise variable parameters used for representing the mean value of the experimental data under the grading indexes;
determining a probability density function of the comprehensive parameters of the experimental group under the target scoring index based on the probability density function of the experimental data of the experimental group under the target scoring index and the prior probability distribution of the comprehensive parameters, and determining a probability density function of the comprehensive parameters of the comparison group under the target scoring index based on the probability density function of the experimental data of the comparison group under the target scoring index and the prior probability distribution of the comprehensive parameters;
and performing integral processing on the probability density function of the comprehensive parameters of the experimental group under the target scoring index to obtain the mean posterior distribution of the experimental group under the target scoring index, and performing integral processing on the probability density function of the comprehensive parameters of the comparison group under the target scoring index to obtain the mean posterior distribution of the comparison group under the target scoring index.
In an exemplary embodiment, the mean distribution obtaining module 602 is further configured to:
for a target score index in the n score indexes, determining a probability density function of comprehensive parameters of the experimental group under the target score index based on experimental data of the experimental group under the target score index by adopting a Markov Chain Monte Carlo (MCMC) method, and determining a probability density function of the comprehensive parameters of the comparison group under the target score index based on experimental data of the comparison group under the target score index; the comprehensive parameters are used for constructing probability density functions corresponding to the experimental data under the grading indexes, and the comprehensive parameters comprise variable parameters used for representing the mean value of the experimental data under the grading indexes;
and performing integral processing on the probability density function of the comprehensive parameters of the experimental group under the target scoring index to obtain the mean posterior distribution of the experimental group under the target scoring index, and performing integral processing on the probability density function of the comprehensive parameters of the control group under the target scoring index to obtain the mean posterior distribution of the control group under the target scoring index.
In one exemplary embodiment, the experimental data under the target score index obeys a t-distribution, the t-distribution including a first variable parameter and a second variable parameter for describing a shape of the t-distribution, and a third variable parameter for describing a mean value of the experimental data under the target score index;
the prior distribution of the first variable parameters is exponential distribution, the prior distribution of the second variable parameters is uniform distribution, and the prior distribution of the third variable parameters is Gaussian distribution.
In one exemplary embodiment, the mean posterior distribution under the target score index is represented as follows:
wherein v is the first variable parameter, λ is the second variable parameter, μ is the third variable parameter, and D is experimental data under the target scoring index.
In an exemplary embodiment, the experimental result obtaining module 604 is configured to:
if the score corresponding to the experimental group is larger than the score corresponding to the control group, taking the product scheme corresponding to the experimental group as the AB experimental result;
and if the score corresponding to the experimental group is less than or equal to the score corresponding to the control group, taking the product scheme corresponding to the control group as the AB experimental result.
To sum up, the technical scheme that this application embodiment provided, through based on the user experiment data in the AB experiment, confirm that the mean posterior distribution of experimental group under each mark index and the mean posterior distribution of contrast group under each mark index, again based on the mean posterior distribution of experimental group under each mark index and the mean posterior distribution of contrast group under each mark index, confirm the grade of experimental group and the grade of contrast group respectively, confirm the AB experimental result based on the grade of experimental group and the grade of contrast group at last, the automatic of AB experimental result under the mark index of multidimension degree is confirmed, and need not to carry out manual comparison, thereby the definite efficiency of AB experimental result has been improved.
In addition, the score is determined by combining the mean posterior distribution under the multidimensional scoring index, so that the reasonability and comprehensiveness of the score are improved, and the reasonability and comprehensiveness of the AB experimental result are further improved.
It should be noted that, when the apparatus provided in the foregoing embodiment implements the functions thereof, only the division of the functional modules is illustrated, and in practical applications, the functions may be distributed by different functional modules according to needs, that is, the internal structure of the apparatus may be divided into different functional modules to implement all or part of the functions described above. In addition, the apparatus and method embodiments provided by the above embodiments belong to the same concept, and specific implementation processes thereof are described in the method embodiments for details, which are not described herein again.
In one exemplary embodiment, a computer device is provided, which comprises a processor and a memory, wherein the memory stores a computer program, and the computer program is loaded and executed by the processor to realize the determination method of the experimental result.
In an exemplary embodiment, a computer-readable storage medium is also provided, in which a computer program is stored, which when executed by a processor, implements the above-described determination method of experimental results.
Optionally, the computer-readable storage medium may include: ROM (Read-Only Memory), RAM (Random-Access Memory), SSD (Solid State drive), or optical disc. The Random Access Memory may include a ReRAM (resistive Random Access Memory) and a DRAM (Dynamic Random Access Memory).
In an exemplary embodiment, a computer program product or a computer program is also provided, which comprises computer instructions, which are stored in a computer-readable storage medium. The computer instructions are read from the computer readable storage medium by a processor of a computer device, and the processor executes the computer instructions to cause the computer device to execute the method for determining the experimental result.
It should be understood that reference herein to "a plurality" means two or more. "and/or" describes the association relationship of the associated object, indicating that there may be three relationships, for example, a and/or B, which may indicate: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. In addition, the step numbers described herein only exemplarily show one possible execution sequence among the steps, and in some other embodiments, the steps may also be executed out of the numbering sequence, for example, two steps with different numbers are executed simultaneously, or two steps with different numbers are executed in a reverse order to the order shown in the figure, which is not limited by the embodiment of the present application.
The above description is only exemplary of the application and should not be taken as limiting the application, and any modifications, equivalents, improvements and the like that are made within the spirit and principle of the application should be included in the protection scope of the application.
Claims (10)
1. A method for determining experimental results, said method comprising:
acquiring user experiment data in an AB experiment; the user experimental data comprises experimental data of an experimental group and experimental data of a control group under n grading indexes, wherein n is a positive integer;
for a target scoring index of the n scoring indexes, determining a probability density function of a comprehensive parameter of the experimental group under the target scoring index based on experimental data of the experimental group under the target scoring index, and determining a probability density function of a comprehensive parameter of the control group under the target scoring index based on experimental data of the control group under the target scoring index; the comprehensive parameters are used for constructing a probability density function corresponding to the experimental data under the grading index, and the comprehensive parameters comprise variable parameters for representing the mean value of the experimental data under the grading index;
obtaining the mean posterior distribution of the experimental group under the target scoring index based on the probability density function of the comprehensive parameters of the experimental group under the target scoring index, and obtaining the mean posterior distribution of the control group under the target scoring index based on the probability density function of the comprehensive parameters of the control group under the target scoring index; wherein, the mean posterior distribution refers to the posterior probability distribution of the mean of the experimental data under the grading index;
determining the scores corresponding to the experimental group based on the mean posterior distribution of the experimental group under the n scoring indexes, and determining the scores corresponding to the control group based on the mean posterior distribution of the control group under the n scoring indexes;
and comparing the scores corresponding to the experimental group with the scores corresponding to the control group to obtain the AB experimental result.
2. The method of claim 1, wherein determining the score corresponding to the experimental group based on the mean posterior distribution of the experimental group over the n scoring indicators and determining the score corresponding to the control group based on the mean posterior distribution of the control group over the n scoring indicators comprises:
acquiring mean values respectively corresponding to the mean posterior distribution of the experimental group under the n grading indexes, and acquiring mean values respectively corresponding to the mean posterior distribution of the control group under the n grading indexes;
acquiring weight parameters corresponding to the n scoring indexes respectively;
based on the weighting parameters respectively corresponding to the n scoring indexes, carrying out weighted summation on mean values respectively corresponding to the posterior distribution of the mean values of the experimental group under the n scoring indexes to obtain scores corresponding to the experimental group;
and based on the weight parameters corresponding to the n scoring indexes, carrying out weighted summation on the mean values corresponding to the mean posterior distribution of the comparison group under the n scoring indexes to obtain the scores corresponding to the comparison group.
3. The method of claim 1, wherein determining the probability density function of the combined parameter of the experimental group under the target scoring index based on the experimental data of the experimental group under the target scoring index, and determining the probability density function of the combined parameter of the control group under the target scoring index based on the experimental data of the control group under the target scoring index comprises:
determining the prior probability distribution of the probability density function and the comprehensive parameter of the experimental data of the experimental group under the target scoring index based on the experimental data of the experimental group under the target scoring index, and determining the prior probability distribution of the probability density function and the comprehensive parameter of the experimental data of the control group under the target scoring index based on the experimental data of the control group under the target scoring index;
determining the probability density function of the comprehensive parameters of the experimental group under the target scoring index based on the probability density function of the experimental data of the experimental group under the target scoring index and the prior probability distribution of the comprehensive parameters, and determining the probability density function of the comprehensive parameters of the comparison group under the target scoring index based on the probability density function of the experimental data of the comparison group under the target scoring index and the prior probability distribution of the comprehensive parameters;
the obtaining of the mean posterior distribution of the experimental group under the target scoring index based on the probability density function of the comprehensive parameters of the experimental group under the target scoring index and the obtaining of the mean posterior distribution of the control group under the target scoring index based on the probability density function of the comprehensive parameters of the control group under the target scoring index include:
and performing integral processing on the probability density function of the comprehensive parameters of the experimental group under the target scoring index to obtain the mean posterior distribution of the experimental group under the target scoring index, and performing integral processing on the probability density function of the comprehensive parameters of the control group under the target scoring index to obtain the mean posterior distribution of the control group under the target scoring index.
4. The method of claim 1, wherein determining the probability density function of the combined parameter of the experimental group under the target scoring index based on the experimental data of the experimental group under the target scoring index, and determining the probability density function of the combined parameter of the control group under the target scoring index based on the experimental data of the control group under the target scoring index comprises:
determining a probability density function of comprehensive parameters of the experimental group under the target scoring index based on experimental data of the experimental group under the target scoring index by adopting a Markov Chain Monte Carlo (MCMC) method, and determining a probability density function of comprehensive parameters of the control group under the target scoring index based on experimental data of the control group under the target scoring index;
the obtaining of the mean posterior distribution of the experimental group under the target scoring index based on the probability density function of the comprehensive parameters of the experimental group under the target scoring index and the obtaining of the mean posterior distribution of the control group under the target scoring index based on the probability density function of the comprehensive parameters of the control group under the target scoring index include:
and performing integral processing on the probability density function of the comprehensive parameters of the experimental group under the target scoring index to obtain the mean posterior distribution of the experimental group under the target scoring index, and performing integral processing on the probability density function of the comprehensive parameters of the control group under the target scoring index to obtain the mean posterior distribution of the control group under the target scoring index.
5. The method of claim 4, wherein the experimental data under the target scoring index is subject to a t-distribution, the t-distribution comprising a first variable parameter and a second variable parameter for describing a shape of the t-distribution, and a third variable parameter for describing a mean of the experimental data under the target scoring index;
the prior distribution of the first variable parameter is exponential distribution, the prior distribution of the second variable parameter is uniform distribution, and the prior distribution of the third variable parameter is gaussian distribution.
6. The method of claim 5, wherein the mean posterior distribution under the target score index is represented as follows:
wherein v is the first variable parameter, λ is the second variable parameter, μ is the third variable parameter, and D is experimental data under the target scoring index.
7. The method of any one of claims 1 to 6, wherein comparing the scores corresponding to the test group and the scores corresponding to the control group to obtain the AB test result comprises:
if the score corresponding to the experimental group is larger than the score corresponding to the control group, taking the product scheme corresponding to the experimental group as the AB experimental result;
and if the score corresponding to the experimental group is smaller than the score corresponding to the control group, taking the product scheme corresponding to the control group as the AB experimental result.
8. An apparatus for determining experimental results, the apparatus comprising:
the experimental data acquisition module is used for acquiring user experimental data in the AB experiment; the user experimental data comprise experimental data of an experimental group and a control group under n grading indexes respectively, wherein n is a positive integer;
a mean distribution obtaining module, configured to, for a target score index of the n score indexes, determine, based on experimental data of the experimental group under the target score index, a probability density function of a composite parameter of the experimental group under the target score index, and determine, based on experimental data of the control group under the target score index, a probability density function of a composite parameter of the control group under the target score index; the comprehensive parameters are used for constructing a probability density function corresponding to the experimental data under the grading index, and the comprehensive parameters comprise variable parameters for representing the mean value of the experimental data under the grading index; obtaining the mean posterior distribution of the experimental group under the target scoring index based on the probability density function of the comprehensive parameters of the experimental group under the target scoring index, and obtaining the mean posterior distribution of the control group under the target scoring index based on the probability density function of the comprehensive parameters of the control group under the target scoring index; wherein, the mean posterior distribution refers to the posterior probability distribution of the mean of the experimental data under the grading index;
the score determining module is used for determining the scores corresponding to the experimental group based on the mean posterior distribution of the experimental group under the n scoring indexes, and determining the scores corresponding to the control group based on the mean posterior distribution of the control group under the n scoring indexes;
and the experimental result acquisition module is used for comparing the scores corresponding to the experimental group with the scores corresponding to the control group to obtain the AB experimental result.
9. A computer device, characterized in that it comprises a processor and a memory, in which a computer program is stored, which computer program is loaded and executed by the processor to implement the method of determining an experimental result according to any one of claims 1 to 7.
10. A computer-readable storage medium, in which a computer program is stored which is loaded and executed by a processor to implement the method of determining an experimental result of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110633686.8A CN113254882B (en) | 2021-06-07 | 2021-06-07 | Method, device and equipment for determining experimental result and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110633686.8A CN113254882B (en) | 2021-06-07 | 2021-06-07 | Method, device and equipment for determining experimental result and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113254882A CN113254882A (en) | 2021-08-13 |
| CN113254882B true CN113254882B (en) | 2023-02-10 |
Family
ID=77186872
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110633686.8A Active CN113254882B (en) | 2021-06-07 | 2021-06-07 | Method, device and equipment for determining experimental result and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113254882B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115049327B (en) * | 2022-08-17 | 2022-11-15 | 阿里巴巴（中国）有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN115331156A (en) * | 2022-10-17 | 2022-11-11 | 成都西交智汇大数据科技有限公司 | Carbon dioxide preparation experiment scoring method, device, equipment and readable storage medium |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110392740A (en) * | 2017-01-25 | 2019-10-29 | 深圳华大生命科学研究院 | The method and its application for determining crowd's sample Biological indicators collection, predicting biological age |
| WO2018151708A1 (en) * | 2017-02-14 | 2018-08-23 | Webtrends, Inc. | Method and system for web-site testing |
| US20180341873A1 (en) * | 2017-05-24 | 2018-11-29 | Streamlet Data | Adaptive prior selection in online experiments |
| EP3503016B1 (en) * | 2017-12-19 | 2021-12-22 | Nokia Technologies Oy | Apparatus, method and computer program for processing a piecewise-smooth signal |
| CN108845936B (en) * | 2018-05-31 | 2021-10-01 | 创新先进技术有限公司 | AB testing method and system based on massive users |
| CN110377521B (en) * | 2019-07-23 | 2022-11-18 | 郑州阿帕斯科技有限公司 | Target object verification method and device |
| CN112799971B (en) * | 2021-04-09 | 2021-06-22 | 腾讯科技（深圳）有限公司 | Data processing method, data processing equipment and computer readable storage medium |
-
2021
- 2021-06-07 CN CN202110633686.8A patent/CN113254882B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN113254882A (en) | 2021-08-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109284417B (en) | Video pushing method and device, computer equipment and storage medium | |
| CN113254882B (en) | Method, device and equipment for determining experimental result and storage medium | |
| CN109033408B (en) | Information pushing method and device, computer readable storage medium and electronic equipment | |
| CN111899882B (en) | Method and system for predicting cancer | |
| EP3549032B1 (en) | Determination of similarity between videos using shot duration correlation | |
| CN110209946B (en) | Social and community-based product recommendation method, system and storage medium | |
| CN112434213B (en) | Training method of network model, information pushing method and related devices | |
| CN115600691A (en) | Client selection method, system, device and storage medium in federal learning | |
| CN115600017A (en) | Feature coding model training method and device and media object recommendation method and device | |
| CN115272797A (en) | Training method, using method, device, equipment and storage medium of classifier | |
| CN113470799B (en) | Intelligent editor of comprehensive quality supervision platform for hospitals | |
| CN117056591A (en) | Intelligent electric power payment channel recommendation method and system based on dynamic prediction | |
| CN111461188A (en) | Target service control method, device, computing equipment and storage medium | |
| CN111506624A (en) | Electric power missing data identification method and related device | |
| CN114093521B (en) | Random forest based method and system for estimating blood sugar by reconstructing homogenized samples | |
| CN112711739A (en) | Data processing method and device, server and storage medium | |
| CN115618121A (en) | Personalized information recommendation method, device, equipment and storage medium | |
| CN112988186B (en) | Updating method and device of abnormality detection system | |
| CN117097789A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN111784381B (en) | Power customer subdivision method and system based on privacy protection and SOM network | |
| CN113723514A (en) | Safe access log data balance processing method based on hybrid sampling | |
| CN113177613A (en) | System resource data distribution method and device | |
| CN110275986A (en) | Video recommendation method, server and computer storage medium based on collaborative filtering | |
| CN115278374B (en) | Video recall method and device | |
| CN117278485A (en) | Flow segmentation method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |