CN112181782A - Adaptive gray function issuing method and device based on AB test - Google Patents
Adaptive gray function issuing method and device based on AB test Download PDFInfo
- Publication number
- CN112181782A CN112181782A CN202011109194.0A CN202011109194A CN112181782A CN 112181782 A CN112181782 A CN 112181782A CN 202011109194 A CN202011109194 A CN 202011109194A CN 112181782 A CN112181782 A CN 112181782A
- Authority
- CN
- China
- Prior art keywords
- gray
- function
- log
- test
- issuing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012360 testing method Methods 0.000 title claims abstract description 127
- 238000000034 method Methods 0.000 title claims abstract description 79
- 230000003044 adaptive effect Effects 0.000 title claims description 62
- 238000004458 analytical method Methods 0.000 claims abstract description 58
- 230000006870 function Effects 0.000 claims description 284
- 238000004364 calculation method Methods 0.000 claims description 35
- 238000003860 storage Methods 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 17
- 238000007781 pre-processing Methods 0.000 claims description 15
- 238000012216 screening Methods 0.000 claims description 13
- 238000007689 inspection Methods 0.000 claims description 9
- 238000005259 measurement Methods 0.000 claims description 9
- 238000013075 data extraction Methods 0.000 claims description 7
- 238000009826 distribution Methods 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 22
- 238000012545 processing Methods 0.000 description 11
- 230000008569 process Effects 0.000 description 10
- 230000000694 effects Effects 0.000 description 8
- 238000002474 experimental method Methods 0.000 description 7
- 238000004891 communication Methods 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 4
- 238000006243 chemical reaction Methods 0.000 description 3
- 239000003086 colorant Substances 0.000 description 3
- 238000007405 data analysis Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000005457 optimization Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000005096 rolling process Methods 0.000 description 2
- 241001513109 Chrysocephalum apiculatum Species 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 239000012530 fluid Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
- 230000033764 rhythmic process Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000000551 statistical hypothesis test Methods 0.000 description 1
- 238000005728 strengthening Methods 0.000 description 1
- 238000010998 test method Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
- G06F11/3072—Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computer Hardware Design (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention provides a self-adaptive gray scale function issuing method and a device based on AB test, wherein the method comprises the following steps: acquiring gray function embedded point logs sent by the SDK embedded points of all the service nodes and front end embedded point logs sent by the SDK embedded points of all the front end nodes; analyzing the gray function embedded point log and the front end embedded point log based on a preset AB test strategy; the gray function issuing is carried out according to the log analysis result, by adopting the technical scheme, the gray function of the optimal strategy can be selected in a self-adaptive mode aiming at the gray function and issued formally, the transaction data of the link gray function can be collected through service embedding points according to the scene function level, the optimal weight matching result is calculated according to the ABTEST data of the same function level and combined with the preset AB test strategy, and the optimal strategy is pushed by each node through a configuration center to issue in real time, so that the aim of automatically and formally issuing the gray function is fulfilled.
Description
Technical Field
The invention relates to the field of distributed technology, in particular to an adaptive gray scale function issuing method and device based on AB test.
Background
Under the development mode background of continuous integration of projects, especially distributed services in the current industry are more and more, a scene of testing multiple gray scale functions by using the same function in different nodes and by using the same function is more and more common, and especially, the gray scale function of multiple scenes also exists in the fine-grained function. As the service scene is more and more complex, after the project is released and iterated for a period of time, after a large number of ABTEST gray-scale functions are required to be on-line, the gray-scale optimal function is released manually, and the release and rollback of the gray-scale function are interfered after data analysis, so that the workload of manual participation in judgment operation is greatly increased, the iteration efficiency of a product is poor, and the flow conversion efficiency is low.
Disclosure of Invention
In view of the problems in the prior art, the present invention provides an adaptive gray function issuing method and apparatus based on an AB test, an electronic device, and a computer-readable storage medium, which can at least partially solve the problems in the prior art.
In order to achieve the purpose, the invention adopts the following technical scheme:
in a first aspect, an adaptive gray function issuing method based on an AB test is provided, which includes:
acquiring gray function embedded point logs sent by the SDK embedded points of all the service nodes and front end embedded point logs sent by the SDK embedded points of all the front end nodes;
analyzing the gray function embedded point log and the front end embedded point log based on a preset AB test strategy;
and issuing a gray function according to the log analysis result.
Further, the preset AB test strategy includes: a transaction parameter calculation mode and a parameter expected value;
analyzing the gray function point-burying log and the front end point-burying log based on a preset AB test strategy comprises the following steps:
summarizing and counting the gray function embedded point log and the front end embedded point log based on the transaction parameter calculation mode to obtain a transaction parameter value;
and comparing the transaction parameter value with the parameter expected value to obtain a log analysis result.
Further, the preset AB test strategy includes: a transaction parameter calculation mode, a parameter expected value, an expected U value and a gray release time node requirement;
analyzing the gray function point-burying log and the front end point-burying log based on a preset AB test strategy comprises the following steps:
summarizing and counting the gray function embedded point log and the front end embedded point log based on the transaction parameter calculation mode to obtain a transaction parameter value;
judging whether the gray scale release time node requirement is met or not according to the gray scale function embedded point log and the front end embedded point log;
if yes, obtaining a significance level U value according to the transaction parameter value and the parameter expected value based on a U inspection method;
and comparing the significance level U value with the expected U value to obtain a log analysis result.
Further, the transaction parameters include: mean, variance, and/or number of samples.
Further, the adaptive gray function issuing method based on the AB test further comprises the following steps:
screening the highest value of the significance level U values of the service nodes corresponding to the several parallel alternative gray scale functions;
and issuing the gray function corresponding to the highest value as an optimal gray strategy.
Further, the preset AB test strategy includes: starting the service node identification of each gray scale function;
the method further comprises the following steps:
and controlling the corresponding service node to start the corresponding gray function according to the service node identification.
Further, before analyzing the gray-scale function embedded point log and the front-end embedded point log based on a preset AB test strategy, the method further includes:
and preprocessing the gray function point-burying log and the front end point-burying log.
Further, the preprocessing the gray-scale function embedded point log and the front-end embedded point log includes:
collecting the gray function buried point log and the front end buried point log by utilizing kafka;
and performing efficiency data extraction on the gray function buried point log and the front end buried point log to obtain transaction key measurement data which is used as basic data for log analysis.
Further, the gray-scale function buried point log comprises: background transaction data; the front-end buried point log comprises: the transaction instance tracks the identity.
Further, the issuing of the gray function according to the log analysis result includes:
judging whether the log analysis result meets the requirement of a preset AB test strategy or not;
if yes, the gray function is issued;
if not, controlling the service node starting the gray scale function to perform function rollback.
In a second aspect, an adaptive gray function issuing apparatus based on an AB test is provided, including:
the log acquisition module is used for acquiring gray function embedded point logs sent by the SDK embedded points of all the service nodes and front end embedded point logs sent by the SDK embedded points of all the front end nodes;
the log analysis module is used for analyzing the gray function embedded point log and the front end embedded point log based on a preset AB test strategy;
and the gray function issuing control module is used for issuing the gray function according to the log analysis result.
Further, the preset AB test strategy includes: a transaction parameter calculation mode and a parameter expected value;
the log analysis module comprises:
the parameter calculation unit collects and counts the gray function embedded point log and the front end embedded point log based on the transaction parameter calculation mode to obtain a transaction parameter value;
and the parameter comparison and analysis unit is used for comparing the transaction parameter value with the parameter expected value to obtain a log analysis result.
Further, the preset AB test strategy includes: a transaction parameter calculation mode, a parameter expected value, an expected U value and a gray release time node requirement;
the log analysis module comprises:
the parameter calculation unit collects and counts the gray function embedded point log and the front end embedded point log based on the transaction parameter calculation mode to obtain a transaction parameter value;
the time judgment unit judges whether the gray scale release time node requirement is met or not according to the gray scale function embedded point log and the front end embedded point log;
the U inspection unit is used for obtaining a significance level U value according to the transaction parameter value and the parameter expected value based on a U inspection method if the requirement of the gray level release time node is met;
and the U value comparison and analysis unit is used for comparing the significance level U value with the expected U value to obtain a log analysis result.
Further, the transaction parameters include: mean, variance, and/or number of samples.
Further, the adaptive gray function issuing device based on the AB test further includes:
the maximum value screening module is used for screening the maximum value of the significance level U values of the service nodes corresponding to the several parallel alternative gray level functions;
and the optimal gray level issuing module is used for issuing the gray level function corresponding to the highest value as an optimal gray level strategy.
Further, the preset AB test strategy includes: starting the service node identification of each gray scale function;
the adaptive gray function issuing device based on the AB test further comprises:
and the starting control module controls the corresponding service node to start the corresponding gray function according to the service node identification.
Further, the adaptive gray function issuing device based on the AB test further includes:
and the log data preprocessing module is used for preprocessing the gray function embedded point log and the front end embedded point log.
Further, the log data preprocessing module comprises:
a log collection unit which collects the gray function buried point log and the front end buried point log by using kafka;
and the data extraction unit is used for extracting efficiency data of the gray function embedded point log and the front end embedded point log to obtain transaction key measurement data which is used as basic data of log analysis.
Further, the gray-scale function buried point log comprises: background transaction data; the front-end buried point log comprises: the transaction instance tracks the identity.
Further, the gray function issuing control module includes:
the result judging unit is used for judging whether the log analysis result meets the requirement of a preset AB test strategy or not;
the gray function issuing unit issues the gray function if the requirement of a preset AB test strategy is met;
and the function rollback unit controls the service node starting the gray function to perform function rollback if the requirement of the preset AB test strategy is not met.
In a third aspect, an electronic device is provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the foregoing adaptive gray function issuing method based on the AB test when executing the program.
In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, which, when being executed by a processor, implements the steps of the above-mentioned AB test-based adaptive gray function issuing method.
The invention provides an adaptive gray scale function issuing method and device based on AB test, wherein the method comprises the following steps: acquiring gray function embedded point logs sent by the SDK embedded points of all the service nodes and front end embedded point logs sent by the SDK embedded points of all the front end nodes; analyzing the gray function embedded point log and the front end embedded point log based on a preset AB test strategy; the gray function issuing is carried out according to the log analysis result, by adopting the technical scheme, the gray function of the optimal strategy can be selected in a self-adaptive mode aiming at the gray function and issued formally, the transaction data of the link gray function can be collected through service embedding points according to the scene function level, the optimal weight matching result is calculated according to the ABTEST data of the same function level and combined with the preset AB test strategy, and the optimal strategy is pushed by each node through a configuration center to issue in real time, so that the aim of automatically and formally issuing the gray function is fulfilled.
In order to make the aforementioned and other objects, features and advantages of the invention comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts. In the drawings:
fig. 1 is a schematic diagram of a system architecture for implementing an AB test-based adaptive gray scale function issuing method in an embodiment of the present invention;
fig. 2 is a first flowchart of an adaptive gray scale function issuing method based on an AB test in an embodiment of the present invention;
FIG. 3 shows a specific step of step S200 in the embodiment of the present invention;
FIG. 4 shows another specific step of step S200 in the embodiment of the present invention;
fig. 5 is a schematic flow chart diagram of an adaptive gray scale function issuing method based on an AB test in the embodiment of the present invention;
fig. 6 is a third schematic flow chart of an adaptive gray scale function issuing method based on AB test in the embodiment of the present invention;
fig. 7 shows the specific steps of step S500 in the embodiment of the present invention;
fig. 8 shows the specific steps of step S300 in the embodiment of the present invention;
FIG. 9 shows a flow of a log data processing procedure in an embodiment of the invention;
fig. 10 is a first block diagram of the adaptive gray function issuing apparatus based on the AB test in the embodiment of the present invention;
FIG. 11 is a block diagram illustrating an architecture of a log analysis module in an embodiment of the invention;
FIG. 12 is a block diagram showing another configuration of a log analysis module in the embodiment of the present invention;
fig. 13 is a block diagram of a second structure of the adaptive gray function issuing apparatus based on the AB test in the embodiment of the present invention;
fig. 14 is a block diagram of a third structural block diagram of an adaptive gray scale function issuing apparatus based on an AB test in the embodiment of the present invention;
FIG. 15 is a block diagram illustrating the structure of a log data preprocessing module in an embodiment of the invention;
fig. 16 is a block diagram showing the configuration of a gradation function issuance control module in the embodiment of the present invention;
fig. 17 is a block diagram of an electronic device according to an embodiment of the invention.
Detailed Description
In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only partial embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
It should be noted that the terms "comprises" and "comprising," and any variations thereof, in the description and claims of this application and the above-described drawings, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
Due to the fact that service scenes are more and more complex, after a project is released and iterated for a period of time, after a large number of ABTEST gray-scale functions need to be on-line, the gray-scale optimal function releasing is manually intervened (report analysis is manually performed according to data feedback, and a releasing strategy is adjusted according to an analysis result), the releasing and rollback of the gray-scale function are intervened after data analysis, the workload of manual participation in judgment operation is greatly increased, and the research and development period is long. The iteration efficiency of the product is poor, the flow conversion efficiency is low, the application side needs a great deal of attention data analysis while the AB gray scale function is on line, the data effect works, and especially under the condition that the fine-grained AB test and the data volume reach a certain order of magnitude, the analysis workload is greatly increased.
In order to solve at least part of the technical problems, the embodiment of the invention provides an adaptive gray function issuing method based on an AB test, which can adaptively select and formally issue a gray function with an optimal strategy aiming at a gray function, can support the acquisition of transaction data of a link gray function through service embedding points according to scene function levels, calculates and matches an optimal weight result according to ABTEST data with the same function level and in combination with a preset AB test strategy, and pushes each node to issue the optimal strategy in real time through a configuration center, thereby achieving the purpose of automatically formally issuing the gray function.
It is worth mentioning that in the background of a development mode of continuous project integration, applications such as APP need to be continuously updated in function, for example, the currently enabled solution B is, a solution a replacing the solution B is developed for the solution B, but the market effect of the solution a is unknown, then, after the solution a is online, part of users of APP are selected to enable the solution a for trial use, and then whether the function of the solution a meets the preset requirement is analyzed according to the data of the trial user, if so, the solution a is fully enabled, and if not, the solution of the trial user is rolled back to the solution B, and the solution B is fully used.
Fig. 1 is a schematic diagram of a system architecture for implementing an AB test-based adaptive gray scale function issuing method in an embodiment of the present invention; as shown in fig. 1, the adaptive grayscale function issuing server based on the AB test may include a policy platform and a data platform, where the configuration administrator configures an AB test policy on the policy platform, where the AB test policy may include: the service node identification or ID of the gray scale function, a calculation formula and a threshold value during log data processing and the like are required to be started, wherein the gray scale function can comprise a plurality of gray scale functions, and different service nodes and the like can be started for each gray scale function in the AB test strategy. A strategy configuration management module of the strategy platform controls each corresponding service system (namely service node) to start the corresponding gray scale function according to the service node identification or ID needing to start the gray scale function in the AB test strategy; it should be noted that each service system includes a scheme a (corresponding to the grayscale function) and a scheme B (corresponding to the original function), and only needs to be controlled to be enabled as required.
In addition, there may be two ways to control each corresponding service system (i.e., service node, which may also be referred to as test node) to enable the corresponding grayscale function, one way is to send a control command to enable the grayscale function to the corresponding service system, for example, if the a1 service node needs to start the grayscale functions of B1 and B2, and the A3 node needs to start the grayscale function of B3, then an indication to start the grayscale functions of B1 and B2 is sent to the a1 node, and an indication to start the grayscale function of B3 is sent to the A3 node; the other is to send an enabling control command to all the service systems, the enabling control command includes enabling nodes and corresponding gray-scale functions, each service system controls whether the node enables the gray-scale function according to the enabling control command, and which gray-scale function is enabled, in the above example, the enabling control command includes information that the A1 service node needs to enable B1 and B2 gray-scale functions, and the A3 node needs to enable the B3 gray-scale function, and after the enabling control command is sent to the A1-A3 nodes, the A1 enables the B1 and B2 gray-scale functions according to the enabling control command; the A2 does not enable the grayscale function according to the enable control command, and the A3 enables the B3 grayscale function according to the enable control command.
And the service system acquires the indication of the strategy platform through the service system SDK embedded point.
An application side user (namely a person 2) triggers a page effect by clicking a page request on the front end; the front end request is sent to a service system; the service system executes the gray function and returns data to the front end, the SDK embedded point of the service system sends the gray function embedded point log to the embedded point system of the data platform, and the SDK embedded point of the front end sends the front end embedded point log to the embedded point system of the data platform; the data are collected and preprocessed by a buried point system of the data platform, for example, after the data are regularly sorted, the data are sent to a strategy calculation module of the strategy platform, the strategy platform calculates the log data of the gray function in the scene in a gathering mode based on a preset AB test strategy, an analysis result is sent to a strategy configuration management module, the strategy configuration management module issues the latest optimization strategy to each node, the latest strategy or rollback is started, and a closed loop of strategy optimization and gray function release is formed.
The strategy platform is a core module and mainly comprises two parts: the system comprises a policy configuration management module and a policy calculation module.
The policy configuration management module is mainly responsible for configuration management and configuration push through Apollo. An administrator user can complete initialization configuration of a scene to be gray-level on a strategy platform, wherein the initialization configuration comprises initialization configuration of application, the scene, strategy access control, the scene and strategy relation and the like, and the configuration after strategy optimization is issued to each service node to implement function release. After the strategy platform is released in the experiment, the data of the service buried point only needs to be sent upwards, the data platform is fed back, and the strategy platform does not need to be manually intervened to configure, modify or push the releasing strategy.
The data platform module mainly realizes collection, cleaning and pushing of SDK data, collects data sent by a front end and a service node mainly through kafka, calculates and generates efficiency data through the fluid, stores the efficiency data into the Druid as transaction key measurement data and informs the providing strategy calculation module.
The strategy calculation module is mainly responsible for classifying, analyzing and calculating transaction data (or called log data) to obtain the optimal experimental data.
It is to be understood that the front end may include a smartphone, a tablet electronic device, a laptop computer, a desktop computer, etc.
The server and the head-end or business system may communicate using any suitable network protocol, including network protocols not yet developed at the filing date of this application. The network protocol may include, for example, a TCP/IP protocol, a UDP/IP protocol, an HTTP protocol, an HTTPS protocol, or the like. Of course, the network Protocol may also include, for example, an RPC Protocol (Remote Procedure Call Protocol), a REST Protocol (Representational State Transfer Protocol), and the like used above the above Protocol.
In order to make the present invention better understood by those skilled in the art, the following detailed description of the specific implementation process of the present invention is provided with reference to fig. 2 to 17:
fig. 2 is a first flowchart of an adaptive gray scale function issuing method based on an AB test in an embodiment of the present invention; as shown in fig. 2, the method for issuing the adaptive gray function based on the AB test may include the following steps:
step S100: acquiring gray function embedded point logs sent by the SDK embedded points of all the service nodes and front end embedded point logs sent by the SDK embedded points of all the front end nodes;
wherein, grey scale function buried point log includes: background transaction data; the front-end buried point log comprises: the transaction instance tracks the identity.
For example, a user clicks on a page to generate a traceId (tracking identifier) for identifying an ID of a transaction instance in a certain functional scenario.
The front-end SDK embedded point and the SDK embedded point of the service node mainly realize the uploading of experimental data and the receiving of experimental configuration. The SDK can realize the labeling of the shunting of the experiment users according to the specific experiment scene, and the strategy platform carries out dynamic directional distribution on the experiment configuration. In addition, based on the front-end buried point identifier, the SDK generates tracking identifiers including traceId, abrSult (request result data) and the like, and the tracking identifiers are used as statistical indexes of a strategy analysis platform by using a transparent transmission function of an uploading data platform and a service system.
Step S200: analyzing the gray function embedded point log and the front end embedded point log based on a preset AB test strategy;
and analyzing the gray function embedded point log and the front end embedded point log to obtain the effect of the gray function.
Presetting the AB test strategy, which is set by a test administrator based on test requirements in advance, specifically comprises the following steps: 1. a transaction parameter calculation mode and a parameter expected value, or, 2, a transaction parameter calculation mode, a parameter expected value, an expected U value and a gray scale release time node requirement, and a service node identifier for starting each gray scale function.
Step S300: and issuing a gray function according to the log analysis result.
By adopting the technical scheme, the gray function of the optimal strategy can be selected and released formally in a self-adaptive manner aiming at the gray function, the transaction data of the link gray function can be collected through service embedding points according to the scene function level, the optimal weight matching result is calculated according to the ABTEST data of the same function level and in combination with the preset AB test strategy, and the optimal strategy is pushed by the configuration center to release in real time by each node, so that the aim of automatically and formally releasing the gray function is fulfilled.
It is worth to be noted that the adaptive gray scale function issuing method based on the AB test is suitable for issuing and rolling back of the AB test function of multiple nodes and multiple scenes, can automatically screen and issue the optimal strategy, avoids the strategy of which the mis-issuing conversion rate is lower than the set expected value, and automatically completes the issuing or rolling back strategy, thereby accelerating the rhythm of business iterative development and strengthening the automation capacity of operation and production.
In an alternative embodiment, referring to fig. 3, this step S200 may include the following:
step S210: summarizing and counting the gray function embedded point log and the front end embedded point log based on the transaction parameter calculation mode to obtain a transaction parameter value;
wherein the transaction parameters include: mean, variance, and/or number of samples, etc.
Step S220: and comparing the transaction parameter value with the parameter expected value to obtain a log analysis result.
Specifically, when the transaction parameter value meets the requirement of the expected value of the parameter, the gray scale function effect is considered to meet the requirement, and the online operation can be performed, otherwise, the online operation cannot be performed.
By adopting the technical scheme, the effect of the gray scale function is objectively and reasonably automatically evaluated, and automatic gray scale strategy screening is realized.
In another alternative embodiment, referring to fig. 4, this step S200 may include the following:
step S210': summarizing and counting the gray function embedded point log and the front end embedded point log based on the transaction parameter calculation mode to obtain a transaction parameter value;
wherein the transaction parameters include: mean, variance, and/or number of samples, etc.
Step S220': judging whether the gray scale release time node requirement is met or not according to the gray scale function embedded point log and the front end embedded point log;
specifically, the log includes the log occurrence time, and whether the log occurrence time meets the gray release time node requirement is judged, and if the log occurrence time does not meet the time requirement, the original strategy is maintained, and data collection is continued.
If yes, go to step S230'; if not, the process returns to step S210'.
Step S230': obtaining a significance level U value according to the transaction parameter value and the parameter expected value based on a U inspection method;
since the AB test is consistent with the sampling experiment in principle, the optimal situation of the strategy is usually reflected on the user performance, when the expected value is designed, the target statistic is calculated by adopting a U check method according to the threshold and the log data, and the experiment comparison data under the multi-experiment scene is scientifically judged by using a statistical hypothesis test method.
Step S240': and comparing the significance level U value with the expected U value to obtain a log analysis result.
Wherein, the screening and publishing of the optimal scene are completed by judging the statistic (i.e. the significance level U value) and the expected U value.
By adopting the technical scheme, experimental comparison data under multiple experimental scenes can be scientifically judged by using a statistical hypothesis testing method, the accuracy of screening strategies is improved, and the optimal strategies can be obtained more accurately.
In an optional embodiment, the method for issuing the adaptive gray function based on the AB test may further include:
screening the highest value of the significance level U values of the service nodes corresponding to the several parallel alternative gray scale functions;
and issuing the gray function corresponding to the highest value as an optimal gray strategy.
It should be noted that, each time the AB test is performed, multiple gray functions may be involved in the same scene and simultaneously tested, so as to screen the optimal gray function, and at this time, the gray function corresponding to the highest value of the significance level U value needs to be selected as the gray function of the final on-line.
For example, there is a black button on the existing page, according to the user feedback or design requirement, the visibility of the black button is irrational, and cannot attract the attention of the user, the black button can be changed into three colors of red, yellow or green, the three colors of buttons are corresponding three gray functions, but it is unknown how the market effect of the three colors is, at this time, a part is selected from the users to perform trial test, part of users who select APP (select control is performed according to the service node) use the red button, part of users use the yellow button, part of users use the green button, and part of users do not start the gray function, but continue to use the original black button, then, by analyzing the user data of the three gray functions, the optimal gray function is selected to issue, if the effects of the three gray functions are not ideal, a rollback setting is made to roll back the test user's button to the black button along.
In an alternative embodiment, referring to fig. 5, the method for issuing an adaptive gray function based on an AB test may further include:
step S400: and controlling the corresponding service node to start the corresponding gray function according to the service node identification.
By adopting the technical scheme, the service node is automatically controlled to perform AB test, and the automation level is further improved.
In an alternative embodiment, referring to fig. 6, the method for issuing an adaptive gray function based on an AB test may further include:
step S500: and preprocessing the gray function point-burying log and the front end point-burying log.
Specifically, referring to fig. 7, the step S500 may include:
step S510: collecting the gray function buried point log and the front end buried point log by utilizing kafka;
step S520: and performing efficiency data extraction on the gray function buried point log and the front end buried point log to obtain transaction key measurement data which is used as basic data for log analysis.
In an alternative embodiment, referring to fig. 8, this step S300 may include the following:
step S310: whether the log analysis result meets the requirement of a preset AB test strategy or not;
if yes, go to step S320; if not, go to step S330.
Step S320: issuing the gray function;
specifically, controlling the node that does not execute the gray function to execute the gray function may be implemented by sending a control instruction to the service node.
Step S330: and controlling the service node starting the gray function to perform function rollback.
Specifically, function recovery is achieved by sending a rollback instruction to the service node that starts the grayscale function.
In order to make those skilled in the art better understand the present invention, fig. 9 shows a flow of a log data processing procedure in an embodiment of the present invention, and as shown in fig. 9, the processing procedure includes:
step 1: acquiring transaction data of a data platform;
specifically, the policy platform obtains transaction result data from the Druid.
Step 2: measuring index statistics is obtained;
specifically, the transaction result data is summarized by calculating statistics such as average value, variance, sample number and the like.
And step 3: whether the measurement time condition requirement is met;
specifically, whether the gray-scale function issuing condition meets the gray-scale issuing time node requirement is judged.
If yes, executing step 5, otherwise, executing step 4.
And 4, step 4: the strategy remains unchanged;
and 5: calculating all historical data statistics by a Z test method;
specifically, a U check method is carried out on the measurement data and the scene expected value data, and a significance level U value is generated.
Step 6: whether the expectation significance is met;
specifically, it is determined whether the significance level U value satisfies the expected significance.
If yes, go to step 8, otherwise go to step 7.
And 7: the node policy is rolled back.
And 8: a strategy to filter MAX (statistics) scenarios;
and step 9: and pushing the optimal strategy.
Based on the same inventive concept, the embodiment of the present application further provides an adaptive gray scale function issuing apparatus based on an AB test, which can be used to implement the methods described in the above embodiments, as described in the following embodiments. Because the principle of solving the problems of the adaptive gray scale function issuing device based on the AB test is similar to that of the method, the implementation of the adaptive gray scale function issuing device based on the AB test can refer to the implementation of the method, and repeated parts are not described again. As used hereinafter, the term "unit" or "module" may be a combination of software and/or hardware that implements a predetermined function. Although the means described in the embodiments below are preferably implemented in software, an implementation in hardware, or a combination of software and hardware is also possible and contemplated.
Fig. 10 is a first block diagram of the adaptive grayscale function issuing apparatus based on the AB test in the embodiment of the present invention. As shown in fig. 10, the adaptive gray function issuing apparatus based on the AB test includes: a log obtaining module 10, a log analyzing module 20 and a gray function issuing control module 30.
The log obtaining module 10 obtains a gray function embedded point log sent by the SDK embedded point of each service node and a front end embedded point log sent by the SDK embedded point of each front end node;
the log analysis module 20 analyzes the gray function buried point log and the front end buried point log based on a preset AB test strategy;
the gray function issuing control module 30 issues a gray function according to the log analysis result.
By adopting the technical scheme, the gray function of the optimal strategy can be selected and released formally in a self-adaptive manner aiming at the gray function, the transaction data of the link gray function can be collected through service embedding points according to the scene function level, the optimal weight matching result is calculated according to the ABTEST data of the same function level and in combination with the preset AB test strategy, and the optimal strategy is pushed by the configuration center to release in real time by each node, so that the aim of automatically and formally releasing the gray function is fulfilled.
In an alternative embodiment, the preset AB test strategy comprises: a transaction parameter calculation mode and a parameter expected value; referring to fig. 11, the log analysis module may include: a parameter calculation unit 21 and a parameter comparison and analysis unit 22.
The parameter calculation unit 21 collects and counts the gray function embedded point log and the front end embedded point log based on the transaction parameter calculation mode to obtain a transaction parameter value;
the parameter comparison and analysis unit 22 compares the transaction parameter value with the expected parameter value to obtain a log analysis result.
In another alternative embodiment, the preset AB test strategy comprises: the transaction parameter calculation method, the parameter expected value, the expected U value and the gray scale distribution time node requirement are as shown in fig. 12, and the log analysis module includes: a parameter calculation unit 21 ', a time judgment unit 22', a U check unit 23 ', and a U value comparison and analysis unit 24'.
The parameter calculating unit 21' collects and counts the gray function embedded point log and the front end embedded point log based on the transaction parameter calculating mode to obtain a transaction parameter value;
the time judging unit 22' judges whether the gray scale release time node requirement is met according to the gray scale function embedded point log and the front end embedded point log;
if the U inspection unit 23' meets the requirement of the gray level release time node, a significance level U value is obtained according to the transaction parameter value and the parameter expected value based on a U inspection method;
the U-value comparison and analysis unit 24' compares the significance level U-value with the expected U-value to obtain a log analysis result.
Notably, the transaction parameters include: mean, variance, and/or number of samples.
In an optional embodiment, the apparatus for issuing adaptive gray scale function based on AB test may further include: a maximum value screening module and an optimal gray level publishing module.
The maximum value screening module is used for screening the maximum value of the significance level U values of the service nodes corresponding to the several parallel alternative gray level functions;
and the optimal gray level issuing module is used for issuing the gray level function corresponding to the highest value as an optimal gray level strategy.
In an alternative embodiment, the preset AB test strategy comprises: starting the service node identification of each gray scale function; referring to fig. 13, the apparatus for issuing an adaptive gray function based on an AB test further includes: the control module 40 is activated.
And the starting control module 40 controls the corresponding service node to start the corresponding gray function according to the service node identifier.
In an alternative embodiment, referring to fig. 14, the apparatus for issuing an adaptive gray scale function based on an AB test further includes: a log data pre-processing module 50.
The log data preprocessing module 50 preprocesses the gray function embedded point log and the front end embedded point log.
In an alternative embodiment, referring to fig. 15, the log data preprocessing module includes: a log collection unit 51 and a data extraction unit 52.
The log collection unit 51 collects the grayscale function buried point log and the front end buried point log by using kafka;
the data extraction unit 52 extracts performance data from the gray function buried point log and the front end buried point log to obtain transaction key measurement data, which is used as basic data for log analysis.
It should be noted that the gray function buried point log includes: background transaction data; the front-end buried point log comprises: the transaction instance tracks the identity.
In an alternative embodiment, referring to fig. 16, the gray function issuance control module includes: a result determination unit 31, a gradation function issuing unit 32, and a function rollback unit 33.
The result judging unit 31 judges whether the log analysis result meets the requirement of a preset AB test strategy;
if the gray function issuing unit 32 meets the requirement of a preset AB test strategy, the gray function is issued;
if the function rollback unit 33 does not meet the requirement of the preset AB test policy, the service node that starts the grayscale function is controlled to perform function rollback.
The apparatuses, modules or units illustrated in the above embodiments may be implemented by a computer chip or an entity, or implemented by a product with certain functions. A typical implementation device is an electronic device, which may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
In a typical example, the electronic device specifically includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor executes the program to implement the steps of the above-mentioned adaptive gray scale function issuing method based on the AB test.
Referring now to FIG. 17, shown is a schematic diagram of an electronic device 600 suitable for use in implementing embodiments of the present application.
As shown in fig. 17, the electronic apparatus 600 includes a Central Processing Unit (CPU)601 that can perform various appropriate works and processes according to a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage section 608 into a Random Access Memory (RAM)) 603. In the RAM603, various programs and data necessary for the operation of the system 600 are also stored. The CPU601, ROM602, and RAM603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
The following components are connected to the I/O interface 605: an input portion 606 including a keyboard, a mouse, and the like; an output portion 607 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The driver 610 is also connected to the I/O interface 605 as needed. A removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 610 as necessary, so that a computer program read out therefrom is mounted as necessary on the storage section 608.
In particular, according to an embodiment of the present invention, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, an embodiment of the present invention includes a computer-readable storage medium on which a computer program is stored, which, when being executed by a processor, implements the steps of the above-described adaptive gray function issuing method based on the AB test.
In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 609, and/or installed from the removable medium 611.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
For convenience of description, the above devices are described as being divided into various units by function, and are described separately. Of course, the functionality of the units may be implemented in one or more software and/or hardware when implementing the present application.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The application may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The application may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims (22)
1. An adaptive gray scale function issuing method based on AB test is characterized by comprising the following steps:
acquiring gray function embedded point logs sent by the SDK embedded points of all the service nodes and front end embedded point logs sent by the SDK embedded points of all the front end nodes;
analyzing the gray function embedded point log and the front end embedded point log based on a preset AB test strategy;
and issuing a gray function according to the log analysis result.
2. The adaptive gray function issuing method based on the AB test according to claim 1, wherein the preset AB test policy includes: a transaction parameter calculation mode and a parameter expected value;
analyzing the gray function point-burying log and the front end point-burying log based on a preset AB test strategy comprises the following steps:
summarizing and counting the gray function embedded point log and the front end embedded point log based on the transaction parameter calculation mode to obtain a transaction parameter value;
and comparing the transaction parameter value with the parameter expected value to obtain a log analysis result.
3. The adaptive gray function issuing method based on the AB test according to claim 1, wherein the preset AB test policy includes: a transaction parameter calculation mode, a parameter expected value, an expected U value and a gray release time node requirement;
analyzing the gray function point-burying log and the front end point-burying log based on a preset AB test strategy comprises the following steps:
summarizing and counting the gray function embedded point log and the front end embedded point log based on the transaction parameter calculation mode to obtain a transaction parameter value;
judging whether the gray scale release time node requirement is met or not according to the gray scale function embedded point log and the front end embedded point log;
if yes, obtaining a significance level U value according to the transaction parameter value and the parameter expected value based on a U inspection method;
and comparing the significance level U value with the expected U value to obtain a log analysis result.
4. The AB test-based adaptive gray scale function issuing method according to claim 2 or 3, wherein the transaction parameters include: mean, variance, and/or number of samples.
5. The AB test-based adaptive gray scale function issuing method according to claim 3, further comprising:
screening the highest value of the significance level U values of the service nodes corresponding to the several parallel alternative gray scale functions;
and issuing the gray function corresponding to the highest value as an optimal gray strategy.
6. The adaptive gray function issuing method based on the AB test according to claim 1, wherein the preset AB test policy includes: starting the service node identification of each gray scale function;
the method further comprises the following steps:
and controlling the corresponding service node to start the corresponding gray function according to the service node identification.
7. The adaptive gray function issuing method based on the AB test according to claim 1, wherein before analyzing the gray function buried point log and the front end buried point log based on a preset AB test policy, the method further comprises:
and preprocessing the gray function point-burying log and the front end point-burying log.
8. The AB test-based adaptive gray function issuing method according to claim 7, wherein the preprocessing the gray function buried point log and the front end buried point log includes:
collecting the gray function buried point log and the front end buried point log by utilizing kafka;
and performing efficiency data extraction on the gray function buried point log and the front end buried point log to obtain transaction key measurement data which is used as basic data for log analysis.
9. The AB test-based adaptive gray function issuing method according to any one of claims 1 to 3 and 5 to 8, wherein the gray function buried point log comprises: background transaction data; the front-end buried point log comprises: the transaction instance tracks the identity.
10. The AB test-based adaptive gray function issuing method according to claim 1, wherein the issuing of the gray function according to the log analysis result includes:
judging whether the log analysis result meets the requirement of a preset AB test strategy or not;
if yes, the gray function is issued;
if not, controlling the service node starting the gray scale function to perform function rollback.
11. An adaptive gray scale function issuing device based on an AB test is characterized by comprising:
the log acquisition module is used for acquiring gray function embedded point logs sent by the SDK embedded points of all the service nodes and front end embedded point logs sent by the SDK embedded points of all the front end nodes;
the log analysis module is used for analyzing the gray function embedded point log and the front end embedded point log based on a preset AB test strategy;
and the gray function issuing control module is used for issuing the gray function according to the log analysis result.
12. An adaptive gray-scale function issuing device based on AB test according to claim 11, characterized in that the preset AB test strategy includes: a transaction parameter calculation mode and a parameter expected value;
the log analysis module comprises:
the parameter calculation unit collects and counts the gray function embedded point log and the front end embedded point log based on the transaction parameter calculation mode to obtain a transaction parameter value;
and the parameter comparison and analysis unit is used for comparing the transaction parameter value with the parameter expected value to obtain a log analysis result.
13. An adaptive gray-scale function issuing device based on AB test according to claim 11, characterized in that the preset AB test strategy includes: a transaction parameter calculation mode, a parameter expected value, an expected U value and a gray release time node requirement;
the log analysis module comprises:
the parameter calculation unit collects and counts the gray function embedded point log and the front end embedded point log based on the transaction parameter calculation mode to obtain a transaction parameter value;
the time judgment unit judges whether the gray scale release time node requirement is met or not according to the gray scale function embedded point log and the front end embedded point log;
the U inspection unit is used for obtaining a significance level U value according to the transaction parameter value and the parameter expected value based on a U inspection method if the requirement of the gray level release time node is met;
and the U value comparison and analysis unit is used for comparing the significance level U value with the expected U value to obtain a log analysis result.
14. An AB test-based adaptive gray function issuing device according to claim 12 or 13, wherein said transaction parameters include: mean, variance, and/or number of samples.
15. The apparatus for issuing an adaptive gray scale function based on an AB test as claimed in claim 13, further comprising:
the maximum value screening module is used for screening the maximum value of the significance level U values of the service nodes corresponding to the several parallel alternative gray level functions;
and the optimal gray level issuing module is used for issuing the gray level function corresponding to the highest value as an optimal gray level strategy.
16. An adaptive gray-scale function issuing device based on AB test according to claim 11, characterized in that the preset AB test strategy includes: starting the service node identification of each gray scale function;
the adaptive gray function issuing device based on the AB test further comprises:
and the starting control module controls the corresponding service node to start the corresponding gray function according to the service node identification.
17. The apparatus for issuing an adaptive gray scale function based on an AB test as claimed in claim 11, further comprising:
and the log data preprocessing module is used for preprocessing the gray function embedded point log and the front end embedded point log.
18. The apparatus for issuing an adaptive gray scale function based on an AB test as claimed in claim 17, wherein the log data preprocessing module comprises:
a log collection unit which collects the gray function buried point log and the front end buried point log by using kafka;
and the data extraction unit is used for extracting efficiency data of the gray function embedded point log and the front end embedded point log to obtain transaction key measurement data which is used as basic data of log analysis.
19. An adaptive gray function issuing device based on an AB test according to any one of claims 11 to 13 and 15 to 18, wherein the gray function buried point log comprises: background transaction data; the front-end buried point log comprises: the transaction instance tracks the identity.
20. The device for issuing the adaptive gray scale function based on the AB test according to claim 11, wherein the gray scale function issuing control module comprises:
the result judging unit is used for judging whether the log analysis result meets the requirement of a preset AB test strategy or not;
the gray function issuing unit issues the gray function if the requirement of a preset AB test strategy is met;
and the function rollback unit controls the service node starting the gray function to perform function rollback if the requirement of the preset AB test strategy is not met.
21. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the AB test-based adaptive gray scale function distribution method of any one of claims 1 to 10 when executing the program.
22. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the AB test-based adaptive gray function issuing method according to any one of claims 1 to 10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011109194.0A CN112181782A (en) | 2020-10-16 | 2020-10-16 | Adaptive gray function issuing method and device based on AB test |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011109194.0A CN112181782A (en) | 2020-10-16 | 2020-10-16 | Adaptive gray function issuing method and device based on AB test |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112181782A true CN112181782A (en) | 2021-01-05 |
Family
ID=73950592
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011109194.0A Pending CN112181782A (en) | 2020-10-16 | 2020-10-16 | Adaptive gray function issuing method and device based on AB test |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112181782A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112995310A (en) * | 2021-02-10 | 2021-06-18 | 中国工商银行股份有限公司 | Multi-hierarchy calling gray level publishing method and device based on custom routing |
| CN113434135A (en) * | 2021-06-28 | 2021-09-24 | 青岛海尔科技有限公司 | Interface call repetition determination method and device, storage medium and electronic device |
| CN113673997A (en) * | 2021-08-27 | 2021-11-19 | 深圳鼎盛电脑科技有限公司 | Visualized processing method, device, equipment and medium of fund calculation engine |
| CN115408285A (en) * | 2022-08-31 | 2022-11-29 | 北京发现角科技有限公司 | Gray scale testing method and device, electronic equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090182868A1 (en) * | 2000-04-17 | 2009-07-16 | Mcfate Marlin Popeye | Automated network infrastructure test and diagnostic system and method therefor |
| US20170048338A1 (en) * | 2015-08-13 | 2017-02-16 | Molbase (Shanghai) Biotechnology Co., Ltd. | Online testing system and method thereof |
| CN107832210A (en) * | 2017-10-27 | 2018-03-23 | 广州四三九九信息科技有限公司 | Access method of testing, device and a server are buried in daily record |
| CN108763828A (en) * | 2018-06-21 | 2018-11-06 | 哈尔滨工业大学 | A kind of Small Sample Database model verification method based on statistical analysis |
| CN109101425A (en) * | 2018-08-14 | 2018-12-28 | 阿里巴巴集团控股有限公司 | A kind of index of the page AB test of mobilism buries point methods and device |
-
2020
- 2020-10-16 CN CN202011109194.0A patent/CN112181782A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090182868A1 (en) * | 2000-04-17 | 2009-07-16 | Mcfate Marlin Popeye | Automated network infrastructure test and diagnostic system and method therefor |
| US20170048338A1 (en) * | 2015-08-13 | 2017-02-16 | Molbase (Shanghai) Biotechnology Co., Ltd. | Online testing system and method thereof |
| CN107832210A (en) * | 2017-10-27 | 2018-03-23 | 广州四三九九信息科技有限公司 | Access method of testing, device and a server are buried in daily record |
| CN108763828A (en) * | 2018-06-21 | 2018-11-06 | 哈尔滨工业大学 | A kind of Small Sample Database model verification method based on statistical analysis |
| CN109101425A (en) * | 2018-08-14 | 2018-12-28 | 阿里巴巴集团控股有限公司 | A kind of index of the page AB test of mobilism buries point methods and device |
Non-Patent Citations (1)
| Title |
|---|
| WEIXIN_34221276: "数据采集与分析的那些事——从数据埋点到AB测试", 《CSDN》, pages 2 - 6 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112995310A (en) * | 2021-02-10 | 2021-06-18 | 中国工商银行股份有限公司 | Multi-hierarchy calling gray level publishing method and device based on custom routing |
| CN113434135A (en) * | 2021-06-28 | 2021-09-24 | 青岛海尔科技有限公司 | Interface call repetition determination method and device, storage medium and electronic device |
| CN113434135B (en) * | 2021-06-28 | 2023-06-16 | 青岛海尔科技有限公司 | Method and device for determining call repeatability of interface, storage medium and electronic device |
| CN113673997A (en) * | 2021-08-27 | 2021-11-19 | 深圳鼎盛电脑科技有限公司 | Visualized processing method, device, equipment and medium of fund calculation engine |
| CN115408285A (en) * | 2022-08-31 | 2022-11-29 | 北京发现角科技有限公司 | Gray scale testing method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112181782A (en) | Adaptive gray function issuing method and device based on AB test | |
| CN110399550B (en) | Information recommendation method and device | |
| CN108629413B (en) | Neural network model training and transaction behavior risk identification method and device | |
| CN110428127B (en) | Automatic analysis method, user equipment, storage medium and device | |
| CN110912737A (en) | Dynamic perception performance early warning method based on hybrid model | |
| CN105281981A (en) | Data traffic monitoring method and device for network service | |
| CN115511501A (en) | Data processing method, computer equipment and readable storage medium | |
| CN105468161A (en) | Instruction execution method and device | |
| CN112948695A (en) | User portrait based general financial fast loan product recommendation method and device | |
| CN111932020A (en) | Customer churn prediction method and device | |
| CN112750030A (en) | Risk pattern recognition method, risk pattern recognition device, risk pattern recognition equipment and computer readable storage medium | |
| CN111159241B (en) | Click conversion estimation method and device | |
| CN109299975B (en) | Object characteristic parameter determination method and device, electronic equipment and readable storage medium | |
| CN112910708B (en) | Distributed service calling method and device | |
| CN112269937B (en) | Method, system and device for calculating user similarity | |
| CN113850669A (en) | User grouping method and device, computer equipment and computer readable storage medium | |
| WO2019062404A1 (en) | Application program processing method and apparatus, storage medium, and electronic device | |
| CN113609393A (en) | Digital platform based on data service and data management | |
| CN116668331A (en) | Distributed performance monitoring system and method | |
| CN111368413A (en) | Tracking management method and system for clothing production plan | |
| CN110855474A (en) | Network feature extraction method, device, equipment and storage medium of KQI data | |
| CN116155541A (en) | Automatic machine learning platform and method for network security application | |
| CN105405220A (en) | Method and device for automated test | |
| CN113657817B (en) | Transaction processing method and device, electronic equipment and readable storage medium | |
| CN115827232A (en) | Method, device, system and equipment for determining configuration for service model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |