CN110008131A - A kind of method and device for the region AB experiment management realized based on algorithm - Google Patents
A kind of method and device for the region AB experiment management realized based on algorithm Download PDFInfo
- Publication number
- CN110008131A CN110008131A CN201910294985.6A CN201910294985A CN110008131A CN 110008131 A CN110008131 A CN 110008131A CN 201910294985 A CN201910294985 A CN 201910294985A CN 110008131 A CN110008131 A CN 110008131A
- Authority
- CN
- China
- Prior art keywords
- experiment
- configuration
- user
- api
- application
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a kind of method and devices of region AB experiment management realized based on algorithm；Method includes the following steps: one experiment of creation, receives initial experiment configuration, generates experiment ID；Each language access SDK is stored in file system by the configuration that API pulls all experiment ID that application configuration uses in application starting, new experimental configuration is periodically pulled from API；According to user access information, the calculating tested returns to access and applies currently used A scheme or B scheme, and records, stores the scheme that User ID uses；The information of record is sent to Data Analysis Platform by result collection system, and the achievement data for analyzing result and experiment is shown by Visualization Platform；Developer does not need interventional procedure after function is online, and an application supports multiple experiments are parallel to carry out, and multiple applications can share the same experiment and carry out, and will not influence business because of the failure of API system.
Description
Technical field
The present invention relates to AB test experiments technical fields on line, and in particular to a kind of region AB experiment realized based on algorithm
The method and device of management.
Background technique
AB experiment method general at present is the page that two difference URL are using the function of traffic statistics platform, removes root
Summarize according to the statistics that different URL does effect, be existing for the program problem be user is shown actually which page without
Method control, flexibility is not strong, and AB test can only be done based on full page；AB test needs again function to adjust again after completing
It is whole online or do page jump, when an application is upper needed more than the page for doing to jump after manage will be pretty troublesome.It is another
Scheme is to do different disposal in load balancing layer, and AB is issued using different application versions respectively, the program there are the problem of
It is to need to run two versions, O&M is needed to execute change operation on line, different flowrate proportionings is set in allotment, and final publication also needs
It does O&M on line to change, tests its AB at high cost.
Summary of the invention
For the defects in the prior art, the present invention provides a kind of method of region AB experiment management realized based on algorithm
And device.
On the one hand, the present invention provides a kind of method of region AB experiment management realized based on algorithm, the method packets
Include following step:
An experiment is created, initial experiment configuration is received, generates experiment ID；
Each language access SDK is protected in application starting by the configuration that API pulls all experiment ID that application configuration uses
There are in file system, after starting, new experimental configuration is periodically pulled from API；
According to user access information, the calculating tested returns to access and applies currently used A scheme or B scheme,
And it records, store the scheme that User ID uses；
The information of record is sent to Data Analysis Platform, and the finger that will analyze result and experiment by result collection system
Data are marked to show by Visualization Platform.
Optionally, initial experiment configuration include the logins account number ID of user conversation or user, the application of configuration experiment it is regional,
Test customer flow accounting, the AB option weight accounting used.
Optionally, before being stored in file system by the configuration that API pulls all experiment ID that application configuration uses
Further include: API obtains the initial experiment configuration, and initial experiment configuration is compiled into the configuration text of multiple format respectively
Part；API receives the reading initial experiment configuring request of SDK initiation, by corresponding experiment ID with specified format configuration file,
And respond to SDK request；API caches application ID nearest renewal time, and application ID generates ETag label according to renewal time, and
The ETag label of generation is attached to inside response SDK request response message, subsequent SDK request needs to take ETag value and exists
In SDK subsequent request.
Optionally, each language access SDK is matched in application starting by experiment the newest of ID that API pulls all configurations
It sets, and the step of periodically pulling new experimental configuration from API, comprising: when the experiment ID for downloading all configurations for the first time fails
When, then application starting failure；Periodically whether continuous detection configuration updates, and updates and be configured to file；If subsequent update checks
Success, then cover original configuration；If subsequent update checks failure, original configuration is not covered, and charges to error log using announcement
The function of alert system notifies system maintenance personnel to check response in time, guarantees that access application system is unaffected.
Optionally, identity information is accessed according to user, the step of the calculating tested, comprising: incoming user identity ID,
The IP of user, user session id；Whether the IP based on user judges current accessed user in the Experimental Area of configuration；If no
In test area, then returns using original default option, walk the logic of default, no log recording；If in test area,
Into the experimental calculation stage.
On the other hand, the present invention provides a kind of device of region AB experiment management realized based on algorithm, described devices
Include:
Experimental considerations unit is created, for creating an experiment, initial experiment configuration is received, generates experiment ID；
Experimental configuration unit is obtained, for each language access SDK in application starting, pulling application configuration by API makes
The configuration of all experiment ID is stored in file system, after starting, is periodically pulled new experiment from API and is matched
It sets；
Experimental calculation storage unit, for according to user access information, the calculating tested to return to access application and works as
It is preceding to use A scheme or B scheme, and record, store the scheme that User ID uses；
Data analyze display unit, for the information of record to be sent to Data Analysis Platform by result collection system,
And the achievement data for analyzing result and experiment is shown by Visualization Platform.
Optionally, the creation experimental considerations unit is also used to obtain login account number ID, the configuration experiment of user conversation or user
Application area, experiment the customer flow accounting, the AB option weight accounting that use.
Optionally, the acquisition experimental configuration unit is also used to API and obtains the initial experiment configuration, and will be described first
Beginning experimental configuration is compiled into the configuration file of multiple format respectively；The reading initial experiment configuration that API receives SDK initiation is asked
It asks, by corresponding experiment ID with specified format configuration file, and responds to SDK request；When API caches application ID nearest update
Between, application ID generates ETag label according to renewal time, and the ETag label of generation is attached to response SDK request response letter
Breath the inside, subsequent SDK request need to take ETag value in SDK subsequent request.
Optionally, the acquisition experimental configuration unit is also used to when the experiment ID for downloading all configurations for the first time fails, then
Fail using starting；Periodically whether continuous detection configuration updates, and updates and be configured to file；If subsequent update checks successfully,
Then cover original configuration；If subsequent update checks failure, original configuration is not covered, and is charged to error log and utilized warning system
Function notify that system maintenance personnel check response in time, guarantee that access application system is unaffected.
Optionally, the experimental calculation storage unit is also used to the session of incoming user identity ID, the IP of user, user
ID；Whether the IP based on user judges current accessed user in the Experimental Area of configuration；If being returned not in test area
Using original default option, the logic of default, no log recording are walked；If entering the experimental calculation stage in test area.
The beneficial effects of the present invention are embodied in:
In the present invention, it is managed collectively and is tested by management backstage, authority configuration is convenient real with departmental staff's cooperation management
It tests；Developer only needs in experiment access phase intervention, and developer, operation maintenance personnel do not need intervention to grasp after function is online
Make, can be operated completely by product or operation personnel, expands scope of experiment, it is online using issuing that final publication selects A or B not to need；
Experiment can be based on user, can also be based on the access session of user, and under same subscriber or same session, user is used always
It can be A or B, guarantee the consistency of experience.One application supports multiple experiments are parallel to carry out, and the same page holds multiple experiments more
It carries out simultaneously, multiple applications can share the same experiment and carry out.Experiment can control application range, and only reality is done in some area
It tests, reduces influence face, experiment can be stopped at any time and continue to use original logic.Calculating logic is all distributed to each SDK processing, calculates
Resource dispersion will not impact management backstage or API.Multiple language SDK are supported to access, such front end rear end O&M line produces
Product all can access use and do AB test.
Detailed description of the invention
It, below will be to specific in order to illustrate more clearly of the specific embodiment of the invention or technical solution in the prior art
Embodiment or attached drawing needed to be used in the description of the prior art are briefly described.In all the appended drawings, similar element
Or part is generally identified by similar appended drawing reference.In attached drawing, each element or part might not be drawn according to actual ratio.
Fig. 1 is a kind of method flow signal of region AB experiment management realized based on algorithm provided in an embodiment of the present invention
Figure；
Fig. 2 is that a kind of structure of the device of region AB experiment management realized based on algorithm provided in an embodiment of the present invention is shown
It is intended to.
Specific embodiment
It is described in detail below in conjunction with embodiment of the attached drawing to technical solution of the present invention.Following embodiment is only used for
Clearly illustrate technical solution of the present invention, therefore be only used as example, and cannot be used as a limitation and limit protection model of the invention
It encloses.
It should be noted that unless otherwise indicated, technical term or scientific term used in this application should be this hair
The ordinary meaning that bright one of ordinary skill in the art are understood.
Referenced herein " embodiment " is it is meant that a particular feature, structure, or characteristic described can wrap in conjunction with the embodiments
Containing at least one embodiment of the present invention.Each position in the description occur the phrase might not each mean it is identical
Embodiment, nor the independent or alternative embodiment with other embodiments mutual exclusion.Those skilled in the art explicitly and
Implicitly understand, embodiment described herein can be combined with other embodiments.
On line in AB test experiments, AB experiment method general at present is to do two using the function of traffic statistics platform
The page of different URL goes the statistics for doing effect according to different URL to summarize, and be problem existing for the program is studied carefully to user
Unexpectedly show which page is unable to control, flexibility is not strong, and AB test can only be done based on full page；After AB test is completed
It needs again function to adjust online again or do page jump, is managed after having applied more than the upper page for needing to do to jump for one
It will be pretty troublesome.Another scheme is to do different disposal in load balancing layer, and AB is issued using different application versions respectively,
The program needs O&M to execute change operation on line the problem is that need to run two versions, and different flows is set in allotment
Proportion, final publication are also required to do O&M on line and change, and test its AB at high cost；To solve the above-mentioned problems, so having
Necessity develops a kind of method and device of region AB experiment management realized based on algorithm, by management backstage creation experiment, repairs
Change experiment, regulation experiment flow accounting, regulation experiment application region, publication using final experiment option.
The specific embodiment of the invention provides a kind of method of region AB experiment management realized based on algorithm, and this method is such as
Shown in Fig. 1, include the following steps:
In step s101, an experiment is created, initial experiment configuration is received, generates experiment ID.
In embodiments of the present invention, product operation personnel creates an experiment by management backstage, and initial experiment configuration is
Based on user conversation still based on the identity of user, configure the application area of experiment, configuration for the first time using this area number
Flow is tested, this part respective weight of AB into experiment is how many；To generate experiment ID.
In step s 102, each language access SDK is in application starting, by API pull application configuration use it is all
The configuration of experiment ID is stored in file system, and after starting, new experimental configuration is periodically pulled from API.
In embodiments of the present invention, each language access SDK pulls the institute that application configuration uses by API in application starting
There is the configuration of experiment ID to be stored in file system, if it is wrong to pull unsuccessfully application starting failure exposure experiment in time ID configuration
Accidentally, after starting, new experimental configuration is periodically pulled from API, pull failure at this time and then record error log holding
Former configuration file is constant；Experiment, which creates, can successfully trigger experiment update event, and the rule processor in API triggers after receiving event
Rule reads compiler, and API obtains the initial experiment configuration, and initial experiment configuration is compiled into multiple format respectively
At file；API receives the reading initial experiment configuring request of SDK initiation, and corresponding experiment ID is matched with specified format
File is set, and responds to SDK request；API caches application ID nearest renewal time, and application ID generates ETag according to renewal time
Label, and the ETag label of generation is attached to inside response SDK request response message, subsequent SDK request needs to take
ETag value is in SDK subsequent request, so that ETag is consistent, then the configuration consistency of incoming end Yu the end API is then thought at the end API,
It is not responding to content and reduces transmitted data on network.
When the experiment ID for downloading all configurations for the first time fails, then application starting failure, thus solution of finding the problem in time
Problem；Periodically whether continuous detection configuration updates, and updates and be configured to file；If subsequent update checks successfully, original is covered
There is configuration；If subsequent update checks failure, original configuration is not covered, and it is logical using the function of warning system to charge to error log
Know that system maintenance personnel check response in time, guarantees that access application system is unaffected.
In step s 103, according to user access information, the calculating tested returns to access and applies currently used A
Scheme or B scheme, and record, store the scheme that User ID uses.
In embodiments of the present invention, the connecing using the access SDK insert experiment for corresponding to language using the developer of experiment
Enter.SDK encapsulates experimental configuration download function, and the calculating return of Command Line Parsing function and experiment should use A or B.Access personnel
Enter different logics further according to A or B judgement；User can run access code when requesting, access code can parse experimental configuration
File；It is passed to the session id of user identity ID, the IP of user, user；IP based on user judge current accessed user whether
In the Experimental Area of configuration；If returning using original default option not in test area, walking the logic of default, no log
Record；If entering the experimental calculation stage in test area, recording the experimental diary so as to subsequent analysis.
The key point of experimental calculation be hash calculate hexadecimal representation such as length be 1, then value range be 0x0~
0xF.Value range is 0x0000~0xFFFF if hash value is truncated to 4, and hash value/0xFFFF is then one [0,1]
The the length of value range, Hash interception the long, this value more disperses.If existing traffic partition is 100 parts by we, base
It is tested in User ID.Experiment ID is constant, and User ID will not become, then the hash value for testing ID+ User ID will not become, by this
A value takes the result of fixed-length value that will not become.If Hash method is calculated using SHA1, experiment ID is 1, User ID
It is 9678, sha1 (" 1 | 9678 ") result is the hexadecimal expression " 580b9483935ed29d66024789 that length is 40
553bb8229f1d0506 ", 40 value ranges are too wide, do a degree of interception, such as only take 20.Take one [1,
100] random number y can then be rewritten as y=1+ (100-1) * x (x is one 0 to 1 random number), y=1+ (100-1) *
sha1("xxxx")/0xffffn.This mode calculates user can be fixed on one 1 to 100 integer.Experiment is initial
Change initial stage presses the AB weight of experimental configuration using fixed mode, and such as 40% uses A, and 60% uses B, then initializes 40 for 100 parts
A is A, and 60 are B.Judge that integer that user arrives at random is on A or on B, then responding to called side return is A
Or B.Flowrate proportioning can also use identical strategy.It will if the experiment into calculating logic has been handled in issued state
The option of publication result returns to the processing of access side and executes corresponding service logic.
Before SDK response computation result by experimental data be included in log (id containing user or session id, into experiment when
Between), log collection component is responsible for sending Log Analysis System for the daily record data of record.
In step S104, the information of record is sent to Data Analysis Platform by result collection system, and will analysis
As a result it is shown with the achievement data of experiment by Visualization Platform.
In embodiments of the present invention, Log Analysis System combination user access path, the given datas such as page click data
The experiment effect of each option of AB can be analyzed, carries out data mart modeling in real time or offline, passes through experiment effect Visualization Platform exhibition
Show influence of each area AB option to each experimental index；By backstage publication experiment after final confirmation AB scheme, advance into
The all unified scheme used using publication executes logic to the user of experiment now.
It is managed collectively and is tested by management backstage, authority configuration is convenient to test with departmental staff's cooperation management；Developer
It only needs in experiment access phase intervention, developer, operation maintenance personnel do not need interventional procedure after function is online, completely can be by producing
Product or operation personnel's operation expand scope of experiment, and it is online using issuing that final publication selects A or B not to need；Experiment can be based on use
Family, can also be based on the access session of user, and under same subscriber or same session, that user uses can be A or B always, guarantee
The consistency of experience.One application supports multiple experiments are parallel to carry out, and the same page is held multiple experiments more while being carried out, multiple
It is carried out using the same experiment can be shared.Experiment can control application range, and only some area is tested, and reduce influence face,
It can stop experiment at any time and continue to use original logic.Calculating logic is all distributed to each SDK processing, and computing resource dispersion will not be right
Management backstage or API are impacted.Multiple language SDK are supported to access, for example PHP, Node.js, Java, Python can be propped up
Access test platform is held, such front end rear end O&M line product all can access use and do AB test.
Referring to Fig.2, Fig. 2 provides a kind of device of region AB experiment management realized based on algorithm, device includes:
Experimental considerations unit 21 is created, for creating an experiment, initial experiment configuration is received, generates experiment ID.
Optionally, the creation experimental considerations unit 21, login account number ID, the configuration for being also used to obtain user conversation or user are real
Customer flow accounting, the AB option weight accounting that the application tested is regional, experiment uses.
Experimental configuration unit 22 is obtained, for each language access SDK in application starting, application configuration is pulled by API
The configuration of all experiment ID used is stored in file system, after starting, is periodically pulled new experiment from API and is matched
It sets.
Optionally, the acquisition experimental configuration unit 22 is also used to API and obtains the initial experiment configuration, and will be described
Initial experiment configures the configuration file for being compiled into multiple format respectively；API receives the reading initial experiment configuration of SDK initiation
Request by corresponding experiment ID with specified format configuration file, and responds to SDK request；API caches the nearest update of application ID
Time, application ID generates ETag label according to renewal time, and the ETag label of generation is attached to response SDK request response
Inside information, subsequent SDK request needs to take ETag value in SDK subsequent request.
Optionally, the acquisition experimental configuration unit 22 is also used to when the experiment ID for downloading all configurations for the first time fails,
Then application starting failure；Periodically whether continuous detection configuration updates, and updates and be configured to file；If subsequent update check at
Function then covers original configuration；If subsequent update checks failure, original configuration is not covered, and charges to error log using alarm
The function of system notifies system maintenance personnel to check response in time, guarantees that access application system is unaffected.
Experimental calculation storage unit 23, for according to user access information, the calculating tested to return to access application
Currently used A scheme or B scheme, and record, store the scheme that User ID uses；
Optionally, the experimental calculation storage unit 23 is also used to the meeting of incoming user identity ID, the IP of user, user
Talk about ID；Whether the IP based on user judges current accessed user in the Experimental Area of configuration；If being returned not in test area
It returns and uses original default option, walk the logic of default, no log recording；If entering the experimental calculation stage in test area.
Data analyze display unit 24, flat for the information of record to be sent to data analysis by result collection system
Platform, and the achievement data for analyzing structure and experiment is shown by Visualization Platform.
In embodiments of the present invention, each unit of the device for the region AB experiment management realized based on algorithm can be by corresponding
Hardware or software unit realize that each unit can be independent soft and hardware unit, also can integrate as a soft and hardware unit,
Herein not to limit the present invention.
Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention., rather than its limitations；To the greatest extent
Pipe present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that: its according to
So be possible to modify the technical solutions described in the foregoing embodiments, or to some or all of the technical features into
Row equivalent replacement；And these are modified or replaceed, various embodiments of the present invention technology that it does not separate the essence of the corresponding technical solution
The range of scheme should all cover within the scope of the claims and the description of the invention.
Claims (10)
1. a kind of method for the region AB experiment management realized based on algorithm, which is characterized in that described method includes following steps:
An experiment is created, initial experiment configuration is received, generates experiment ID；
Each language access SDK is stored in application starting by the configuration that API pulls all experiment ID that application configuration uses
In file system, after starting, new experimental configuration is periodically pulled from API；
According to user access information, the calculating tested returns to access and applies currently used A scheme or B scheme, and remembers
The scheme that record, storage User ID use；
The information of record is sent to Data Analysis Platform, and the index number that will analyze result and experiment by result collection system
It is shown according to by Visualization Platform.
2. the method according to claim 1, wherein initial experiment configuration includes user conversation or the login of user
Customer flow accounting, the AB option weight accounting that account number ID, the application area of configuration experiment, experiment use.
3. the method according to claim 1, wherein pulling all experiment ID that application configuration uses by API
Configuration be stored in file system before further include:
API obtains the initial experiment configuration, and initial experiment configuration is compiled into the configuration file of multiple format respectively；
API receives the reading initial experiment configuring request of SDK initiation, and corresponding experiment ID is configured text with specified format
Part, and respond to SDK request；
API caches application ID nearest renewal time, and application ID generates ETag label according to renewal time, and will be described in generation
ETag label is attached to inside response SDK request response message, and subsequent SDK request needs to take ETag value in SDK subsequent request
On.
4. being pulled in application starting by API the method according to claim 1, wherein each language accesses SDK
The latest configuration of the experiment ID of all configurations, and the step of periodically pulling new experimental configuration from API, comprising:
When the experiment ID for downloading all configurations for the first time fails, then application starting failure；More whether periodically continuous detection configuration
Newly, and update is configured to file；
If subsequent update checks successfully, original configuration is covered；
If subsequent update checks failure, original configuration is not covered, and is charged to error log and notified using the function of warning system
Response is checked in time to system maintenance personnel, guarantees that access application system is unaffected.
5. the method according to claim 1, wherein accessing identity information, the calculating tested according to user
The step of, comprising:
It is passed to the session id of user identity ID, the IP of user, user；
Whether the IP based on user judges current accessed user in the Experimental Area of configuration；
If returning using original default option not in test area, walking the logic of default, no log recording；
If entering the experimental calculation stage in test area.
6. a kind of device for the region AB experiment management realized based on algorithm, which is characterized in that described device includes:
Experimental considerations unit is created, for creating an experiment, initial experiment configuration is received, generates experiment ID；
Experimental configuration unit is obtained, for each language access SDK in application starting, pulls what application configuration used by API
The configuration of all experiment ID is stored in file system, and after starting, new experimental configuration is periodically pulled from API；
Experimental calculation storage unit, for according to user access information, the calculating tested, returning to access application currently to make
With A scheme or B scheme, and records, stores the scheme that User ID uses；
Data analyze display unit, for the information of record to be sent to Data Analysis Platform by result collection system, and will
The achievement data for analyzing result and experiment is shown by Visualization Platform.
7. device according to claim 6, which is characterized in that initial experiment configuration includes user conversation or the login of user
Customer flow accounting, the AB option weight accounting that account number ID, the application area of configuration experiment, experiment use.
8. device according to claim 6, which is characterized in that the acquisition experimental configuration unit is also used to API and obtains institute
Initial experiment configuration is stated, and initial experiment configuration is compiled into the configuration file of multiple format respectively；API receives SDK hair
The reading initial experiment configuring request risen, by corresponding experiment ID with specified format configuration file, and responds to SDK and asks
It asks；API caches application ID nearest renewal time, and application ID generates ETag label according to renewal time, and will be described in generation
ETag label is attached to inside response SDK request response message, and subsequent SDK request needs to take ETag value in SDK subsequent request
On.
9. device according to claim 6, which is characterized in that the acquisition experimental configuration unit is also used under first
When carrying the experiment ID failure of all configurations, then application starting failure；Periodically whether continuous detection configuration updates, and updates configuration
To file；If subsequent update checks successfully, original configuration is covered；If subsequent update checks failure, original configuration is not covered,
And charge to error log and notify system maintenance personnel to check response in time using the function of warning system, guarantee access application system
It unites unaffected.
10. device according to claim 6, which is characterized in that the experimental calculation storage unit is also used to incoming user
Identity ID, the IP of user, user session id；Whether the IP based on user judges current accessed user in the Experimental Area of configuration
It is interior；If returning using original default option not in test area, walking the logic of default, no log recording；If in trial zone
In domain, then enter the experimental calculation stage.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910294985.6A CN110008131B (en) | 2019-04-12 | 2019-04-12 | Method and device for managing area AB experiment based on algorithm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910294985.6A CN110008131B (en) | 2019-04-12 | 2019-04-12 | Method and device for managing area AB experiment based on algorithm |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110008131A true CN110008131A (en) | 2019-07-12 |
| CN110008131B CN110008131B (en) | 2020-08-04 |
Family
ID=67171428
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910294985.6A Active CN110008131B (en) | 2019-04-12 | 2019-04-12 | Method and device for managing area AB experiment based on algorithm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110008131B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110532192A (en) * | 2019-09-06 | 2019-12-03 | 北京温杜科技有限公司 | A kind of api interface performance optimized experimental facility |
| CN111708689A (en) * | 2020-05-19 | 2020-09-25 | 北京奇艺世纪科技有限公司 | Method and device for modifying AB experiment and electronic equipment |
| CN112817856A (en) * | 2021-02-04 | 2021-05-18 | 上海哔哩哔哩科技有限公司 | AB experiment integration method and system |
| CN113064820A (en) * | 2021-03-26 | 2021-07-02 | 深圳依时货拉拉科技有限公司 | Method, apparatus and computer readable storage medium for updating A/B experiment SDK |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020072873A1 (en) * | 2000-06-14 | 2002-06-13 | Liu Yi Bo | Method for managing and using test system |
| CN103577321A (en) * | 2012-08-07 | 2014-02-12 | 阿里巴巴集团控股有限公司 | Multi-version parallel test method for web pages and server |
| US20150012852A1 (en) * | 2013-07-08 | 2015-01-08 | Kobo Incorporated | User interface tool for planning an ab type of test |
| CN105718251A (en) * | 2014-12-03 | 2016-06-29 | 苏宁云商集团股份有限公司 | Multilanguage SDK (software development kit) automatic implementation method and system |
| CN106341290A (en) * | 2016-08-31 | 2017-01-18 | 北京城市网邻信息技术有限公司 | Flow distribution server |
| CN106856439A (en) * | 2016-12-07 | 2017-06-16 | 武汉斗鱼网络科技有限公司 | The method and server of a kind of scheme test |
| CN107451020A (en) * | 2017-06-28 | 2017-12-08 | 北京五八信息技术有限公司 | A kind of AB test systems and method of testing |
| CN107957938A (en) * | 2016-10-14 | 2018-04-24 | 北京酷我科技有限公司 | A kind of method and system for obtaining website test data |
| CN108345539A (en) * | 2018-01-31 | 2018-07-31 | 北京云测信息技术有限公司 | A kind of method and apparatus for carrying out AB tests |
-
2019
- 2019-04-12 CN CN201910294985.6A patent/CN110008131B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020072873A1 (en) * | 2000-06-14 | 2002-06-13 | Liu Yi Bo | Method for managing and using test system |
| CN103577321A (en) * | 2012-08-07 | 2014-02-12 | 阿里巴巴集团控股有限公司 | Multi-version parallel test method for web pages and server |
| US20150012852A1 (en) * | 2013-07-08 | 2015-01-08 | Kobo Incorporated | User interface tool for planning an ab type of test |
| CN105718251A (en) * | 2014-12-03 | 2016-06-29 | 苏宁云商集团股份有限公司 | Multilanguage SDK (software development kit) automatic implementation method and system |
| CN106341290A (en) * | 2016-08-31 | 2017-01-18 | 北京城市网邻信息技术有限公司 | Flow distribution server |
| CN107957938A (en) * | 2016-10-14 | 2018-04-24 | 北京酷我科技有限公司 | A kind of method and system for obtaining website test data |
| CN106856439A (en) * | 2016-12-07 | 2017-06-16 | 武汉斗鱼网络科技有限公司 | The method and server of a kind of scheme test |
| CN107451020A (en) * | 2017-06-28 | 2017-12-08 | 北京五八信息技术有限公司 | A kind of AB test systems and method of testing |
| CN108345539A (en) * | 2018-01-31 | 2018-07-31 | 北京云测信息技术有限公司 | A kind of method and apparatus for carrying out AB tests |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110532192A (en) * | 2019-09-06 | 2019-12-03 | 北京温杜科技有限公司 | A kind of api interface performance optimized experimental facility |
| CN111708689A (en) * | 2020-05-19 | 2020-09-25 | 北京奇艺世纪科技有限公司 | Method and device for modifying AB experiment and electronic equipment |
| CN112817856A (en) * | 2021-02-04 | 2021-05-18 | 上海哔哩哔哩科技有限公司 | AB experiment integration method and system |
| CN113064820A (en) * | 2021-03-26 | 2021-07-02 | 深圳依时货拉拉科技有限公司 | Method, apparatus and computer readable storage medium for updating A/B experiment SDK |
| CN113064820B (en) * | 2021-03-26 | 2022-09-16 | 深圳依时货拉拉科技有限公司 | Method, apparatus and computer readable storage medium for updating A/B experiment SDK |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110008131B (en) | 2020-08-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110008131A (en) | A kind of method and device for the region AB experiment management realized based on algorithm | |
| Cheng et al. | Architecture-based self-adaptation in the presence of multiple objectives | |
| Spanoudakis et al. | Non-intrusive monitoring of service-based systems | |
| CN102089775B (en) | There is the service performance manager for alleviating restricted responsibility service-level agreement with automatic protection and pattern | |
| CN104363264B (en) | The SDK access systems by all kinds of means of mobile terminal software | |
| CN110971614A (en) | Internet of things adaptation method and system, computer equipment and storage medium | |
| US8739129B1 (en) | Multi-domain unified debugger | |
| CN106100902A (en) | High in the clouds index monitoring method and apparatus | |
| Garousi et al. | Traffic-aware stress testing of distributed systems based on UML models | |
| CN110287052A (en) | A kind of root of abnormal task determines method and device because of task | |
| US8904357B2 (en) | Dashboard for architectural governance | |
| CN108521353A (en) | Processing method, equipment and the readable storage medium storing program for executing of positioning performance bottleneck | |
| CN109739877A (en) | Database Systems and data managing method | |
| CN109032825A (en) | A kind of fault filling method, device and equipment | |
| CN110515855B (en) | Simulation control method, server and system for application program | |
| CN105224296B (en) | Web service Qos attribute evaluation system and its evaluation method based on independent third party | |
| Vinarskii et al. | A model checking based approach for detecting sdn races | |
| CN105511914B (en) | Using update method, device and system | |
| Brückmann et al. | Towards real-time monitoring and controlling of enterprise architectures using business software control centers | |
| Bertolino et al. | Dependability and performance assessment of dynamic connected systems | |
| Losavio et al. | Designing quality architecture: incorporating ISO standards into the unified process | |
| Netes | Modern network technologies and dependability | |
| CN106452815B (en) | A kind of informatization management method, apparatus and system | |
| Rosa et al. | Lightweight formalisation of adaptive middleware | |
| CN103414717A (en) | Simulation monitoring method and system in regard to C / S structure service system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |