CN109583670A - Control experiment method, system and computer installation based on round-robin - Google Patents
Control experiment method, system and computer installation based on round-robin Download PDFInfo
- Publication number
- CN109583670A CN109583670A CN201710899136.4A CN201710899136A CN109583670A CN 109583670 A CN109583670 A CN 109583670A CN 201710899136 A CN201710899136 A CN 201710899136A CN 109583670 A CN109583670 A CN 109583670A
- Authority
- CN
- China
- Prior art keywords
- experiment
- group
- round
- experimental data
- robin
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0631—Resource planning, allocation, distributing or scheduling for enterprises or organisations
- G06Q10/06311—Scheduling, planning or task assignment for a person or group
Abstract
The embodiment of the present disclosure proposes a kind of control experiment method, system, computer installation and computer readable storage medium based on round-robin, applied to experiment the field of test technology, the control experiment method based on round-robin comprises determining that experiment test information；Information is tested according to experiment, access time piece simultaneously determines round-robin rule；It is tested according to timeslice and round-robin rule, obtains first group of experimental data and second group of experimental data；After experiment, according to first group of the experimental data and second group of experimental data statistical experiment index as a result, generating experiment conclusion.Experimental data statistical accuracy is improved in such a way that experimental data to be grouped to control statistics, avoids the problem of interfering between traditional experiment test method (random shunt) bring group.
Description
Technical field
The embodiment of the present disclosure is related to testing the field of test technology, in particular to a kind of pair based on round-robin
According to experimental method, system, computer installation and computer readable storage medium.
Background technique
AB Test (control experiment) is randomly divided into two groups frequently by by user, wherein one group is control group, one group is real
Group is tested, the effect of experiment is assessed by the performance of different groups of users of comparison.In the scene of trip, passenger's bill, driver's quilt
After assigning order or active competition for orders, passenger shares driver together, if divided passenger in such a way that passenger shunts at random
For experimental group and control group, the result tested will receive interference effect between experimental group and control group passenger, and unreliable.
Summary of the invention
The embodiment of the present disclosure aims to solve at least one of technical problems existing in the prior art or related technologies.
For this purpose, the one aspect of the embodiment of the present disclosure is to propose a kind of control experiment side based on round-robin
Method.
The other side of the embodiment of the present disclosure is to propose a kind of control experiment system based on round-robin.
Another aspect of the embodiment of the present disclosure is to propose a kind of computer installation.
The another aspect of the embodiment of the present disclosure is to propose a kind of computer readable storage medium.
In view of this, according to the one aspect of the embodiment of the present disclosure, propose it is a kind of based on round-robin to according to the facts
Proved recipe method, comprising: determine experiment test information；Information is tested according to experiment, access time piece simultaneously determines round-robin rule；
It is tested according to timeslice and round-robin rule, obtains first group of experimental data and second group of experimental data；It is real
After testing, according to first group of the experimental data and second group of experimental data statistical experiment index as a result, generating experiment
Conclusion.
The control experiment method based on round-robin that the embodiment of the present disclosure provides, it is determined that the experiment carried out is needed to survey
After trying information, selects reasonable timeslice and determine that round-robin rule guarantees the absolute change of two groups of the experimental data obtained
Change is smaller, conspicuousness is not significant, and wherein timeslice is time interval, and an experiment is divided into multiple timeslices, round-robin
Rule is to be tested according to timeslice according to which kind of and which kind of experimental period rotation period.Experiment is carried out again obtains first group
Experimental data and second group of experimental data, and then count experimental index result and obtain experiment conclusion, by the way that number will be tested
According to be grouped control statistics mode improve experimental data statistical accuracy, avoid traditional experiment test method (with
Machine shunts) the problem of interfering between bring group.
According to the above-mentioned control experiment method based on round-robin of the embodiment of the present disclosure, there can also be following technology
Feature:
In the above-mentioned technical solutions, it is preferable that information is tested according to experiment, access time piece simultaneously determines round-robin rule
Then the step of, specifically includes: testing information access time piece range according to experiment；Obtain first group history experimental data and
Second group of history experimental data calculates the absolute change of first group of history experimental data and second group of history experimental data
Value and significance value；According to absolute change value and significance value in timeslice range access time piece, while when determining
Between piece rotation rule.
In the technical scheme, the approximate range for the test content access time piece for including according to experiment test information, example
As if the timeslice interference of the test content of experiment may be larger, with regard to timeslice as big as possible, if test content is main
What is influenced is short-term effect, then can choose relatively small timeslice.According to history after the approximate range of timeslice determines
Experimental data calculates separately the absolute change and conspicuousness of two groups of experimental datas, selects one according to absolute change and conspicuousness
Timeslice as big as possible, then determine round-robin rule, it is advised by selecting reasonable timeslice and round-robin
It can then guarantee that the absolute change of two groups of the experimental data obtained is smaller, conspicuousness is not significant, between the group for avoiding experimental data
It interferes and experiment is caused to be judged by accident.
In any of the above-described technical solution, it is preferable that round-robin rule includes default experimental period and default rotation
Period；It is tested according to the timeslice and round-robin rule, obtains first group of experimental data and second group of reality
The step of testing data specifically includes: test period being divided into time interval T according to timeslice and round-robin ruleij；Such as
Fruit (i, j) ∈ S1, then will be in time interval TijThe experimental data got is as first group of experimental data；(if i, j) ∈
S2, then will be in time interval TijThe experimental data got is as second group of experimental data；Wherein, time interval TijTime
Length is timeslice, i=1,2 ..., n, j=1,2 ..., m, n are the default rotation period, and m is default experimental period,
S1={ (i, j): (i mod 2=0 ∩ j mod 2=0) ∪ (i mod 2=1 ∩ j mod 2=1) }, S2={ (i, j): (i
Mod 2=0 ∩ j mod 2=1) ∪ (i mod 2=1 ∩ j mod 2=0) }.
In the technical scheme, when round-robin rule is according to timeslice according to default experimental period and default rotation
Duan Jinhang experiment, i.e. experiment are carried out according to two rotation, and a rotation is experimental period, and for example, on January 1st, 2017 is extremely
On January 14th, 2017, another rotation are to rotate the period, for example, 00:00 to 00:59,01:00 to 01:59,02:00 to 02:
59 etc..When that will be tested after selecting reasonable timeslice and round-robin rule according to timeslice and round-robin rule
Between be divided into time interval Tij, time interval TijTime span be the timeslice that selects, as (i, j) ∈ S1, i.e. i with
It, will be in time interval T when j is even number or is odd numberijThe experimental data got as first group of experimental data, when
(i, j) ∈ S2, i.e. i with j mono- be odd number another be even number when, will be in time interval TijThe experimental data got is as
Two groups of experimental data realizes that first group replaces acquisition experimental data with second group to keep two groups of data interference-free.
In any of the above-described technical solution, it is preferable that experiment test information includes test content, test index, test city
City.
It in the technical scheme, include test content, test index, test city, test content in experiment test information
Such as dividing single distance, dynamic tune multiple, test index such as order response rate, test city is then selected in conjunction with specific business.
According to the other side of the embodiment of the present disclosure, a kind of control experiment system based on round-robin is proposed,
Unit is comprised determining that, for determining that information is tested in experiment；Selection unit, for testing information according to experiment, access time piece is simultaneously
Determine round-robin rule；Experimental considerations unit obtains first group for being tested according to timeslice and round-robin rule
Experimental data and second group of experimental data；Statistic unit, after testing, according to first group of experimental data and the
Two groups of experimental data statistical experiment index is as a result, generate experiment conclusion.
The control experiment system based on round-robin that the embodiment of the present disclosure provides, determination unit, which has determined, to be needed to carry out
Experiment test information after, by selection unit select reasonable timeslice and determine round-robin rule guarantee obtain two
The absolute change of the experimental data of group is smaller, conspicuousness is not significant, and wherein timeslice is time interval, and an experiment is divided into more
A timeslice, round-robin rule are to be tested according to timeslice according to which kind of and which kind of experimental period rotation period.
Experimental considerations unit carries out experiment and obtains first group of experimental data and second group of experimental data, and then counts reality by statistic unit
It tests index result and obtains experiment conclusion, experimental data statistics is improved in such a way that experimental data to be grouped to control statistics
Accuracy, avoid the problem of interfering between traditional experiment test method (random shunt) bring group.
According to the above-mentioned control experiment system based on round-robin of the embodiment of the present disclosure, there can also be following technology
Feature:
In the above-mentioned technical solutions, it is preferable that selection unit, for testing information access time piece range according to experiment；
Control experiment system based on round-robin, further includes: computing unit, for obtain first group history experimental data and
Second group of history experimental data calculates the absolute change of first group of history experimental data and second group of history experimental data
Value and significance value；Selection unit, when being also used to be chosen in timeslice range according to absolute change value and significance value
Between piece, while determine round-robin rule.
In the technical scheme, selection unit according to experiment test information include test content access time piece substantially
Range, for example, if the timeslice interference of the test content of experiment may be larger, with regard to timeslice as big as possible, if in test
What appearance mainly influenced is short-term effect, then can choose relatively small timeslice.It is logical after the approximate range of timeslice determines
Absolute change and conspicuousness that computing unit calculates separately two groups of experimental datas according to history experimental data are crossed, by selection unit root
A timeslice as big as possible is selected according to absolute change and conspicuousness, then determines round-robin rule, passes through selection
Reasonable timeslice and round-robin rule can guarantee that the absolute change of two groups of the experimental data obtained is smaller, significant out
Property it is not significant, avoid interfering between the group of experimental data and experiment being caused to be judged by accident.
In any of the above-described technical solution, it is preferable that round-robin rule includes default experimental period and default rotation
Period；Experimental considerations unit is specifically used for: test period being divided into time interval according to timeslice and the round-robin rule
Tij；(if i, j) ∈ S1, then will be in time interval TijThe experimental data got is as first group of experimental data；If (i,
j)∈S2, then will be in time interval TijThe experimental data got is as second group of experimental data；Wherein, time interval Tij's
Time span is timeslice, i=1,2 ..., n, j=1,2 ..., m, n are the default rotation period, and m is default experiment
Period, S1={ (i, j): (i mod 2=0 ∩ j mod 2=0) ∪ (i mod 2=1 ∩ j mod 2=1) }, S2=(i,
J): (i mod 2=0 ∩ j mod 2=1) ∪ (i mod 2=1 ∩ j mod 2=0) }.
In the technical scheme, when round-robin rule is according to timeslice according to default experimental period and default rotation
Duan Jinhang experiment, i.e. experiment are carried out according to two rotation, and a rotation is experimental period, and for example, on January 1st, 2017 is extremely
On January 14th, 2017, another rotation are to rotate the period, for example, 00:00 to 00:59,01:00 to 01:59,02:00 to 02:
59 etc..When that will be tested after selecting reasonable timeslice and round-robin rule according to timeslice and round-robin rule
Between be divided into time interval Tij, time interval TijTime span be the timeslice that selects, as (i, j) ∈ S1, i.e. i with
It, will be in time interval T when j is even number or is odd numberijThe experimental data got as first group of experimental data, when
(i, j) ∈ S2, i.e. i with j mono- be odd number another be even number when, will be in time interval TijThe experimental data got is as
Two groups of experimental data realizes that first group replaces acquisition experimental data with second group to keep two groups of data interference-free.
In any of the above-described technical solution, it is preferable that experiment test information includes test content, test index, test city
City.
It in the technical scheme, include test content, test index, test city, test content in experiment test information
Such as dividing single distance, dynamic tune multiple, test index such as order response rate, test city is then selected in conjunction with specific business.
According to another aspect of the embodiment of the present disclosure, propose a kind of computer installation, including memory, processor and
The computer program that can be run on a memory and on a processor is stored, processor is realized when executing computer program as above-mentioned
The step of control experiment method based on round-robin of any one.
The computer installation that the embodiment of the present disclosure provides, realization, which has determined, when processor executes computer program needs to carry out
Experiment test information after, select reasonable timeslice and determine round-robin rule guarantee obtain two groups of experimental data
Absolute change is smaller, conspicuousness is not significant, wherein timeslice is time interval, and an experiment is divided into multiple timeslices, when
Between piece rotation rule be to be tested according to timeslice according to which kind of and which kind of experimental period rotation period.Experiment is carried out again to obtain
First group of experimental data and second group of experimental data are taken, and then counts experimental index result and obtains experiment conclusion, is passed through
The mode that experimental data is grouped control statistics is improved into experimental data statistical accuracy, avoids traditional experiment test
The problem of being interfered between method (random to shunt) bring group.
According to the another aspect of the embodiment of the present disclosure, a kind of computer readable storage medium is proposed, is stored thereon with
Computer program realizes the control experiment based on round-robin such as any of the above-described when computer program is executed by processor
The step of method.
The computer readable storage medium that the embodiment of the present disclosure provides is realized when computer program is executed by processor and is determined
After the experiment test information for needing to carry out, selects reasonable timeslice and determine that round-robin rule guarantees two groups obtained
Experimental data absolute change is smaller, conspicuousness is not significant, wherein timeslice is time interval, and an experiment is divided into multiple
Timeslice, round-robin rule are to be tested according to timeslice according to which kind of and which kind of experimental period rotation period.Again
It carries out experiment and obtains first group of experimental data and second group of experimental data, and then count experimental index result and obtain experiment
Conclusion improves experimental data statistical accuracy in such a way that experimental data to be grouped to control statistics, avoids biography
The problem of being interfered between system experimental test procedures (random to shunt) bring group.
The additional aspect and advantage of the embodiment of the present disclosure will become obviously in following description section, or pass through the disclosure
The practice of embodiment is recognized.
Detailed description of the invention
The above-mentioned and/or additional aspect and advantage of the embodiment of the present disclosure are from the description of the embodiment in conjunction with the following figures
It will be apparent and be readily appreciated that, in which:
Fig. 1 shows the process of the control experiment method based on round-robin of one embodiment of the embodiment of the present disclosure
Schematic diagram；
Fig. 2 shows the streams of the control experiment method based on round-robin of another embodiment of the embodiment of the present disclosure
Journey schematic diagram；
Fig. 3 a shows showing for the control experiment system based on round-robin of one embodiment of the embodiment of the present disclosure
Meaning block diagram；
Fig. 3 b shows the control experiment system based on round-robin of another embodiment of the embodiment of the present disclosure
Schematic block diagram；
Fig. 4 shows the schematic block diagram of the computer installation of one embodiment of the embodiment of the present disclosure.
Specific embodiment
In order to be more clearly understood that the above objects, features, and advantages of the embodiment of the present disclosure, with reference to the accompanying drawing and
The embodiment of the present disclosure is further described in detail in specific embodiment.It should be noted that in the absence of conflict,
Feature in embodiments herein and embodiment can be combined with each other.
Many details are explained in the following description in order to fully understand the embodiment of the present disclosure, still, this public affairs
Opening embodiment can also be implemented using other than the one described here other modes, therefore, the protection of the embodiment of the present disclosure
Range is not limited to the limitation of following public specific embodiment.
The embodiment of embodiment of the present disclosure first aspect proposes a kind of control experiment method based on round-robin, Fig. 1
Show the flow diagram of the control experiment method based on round-robin of one embodiment of the embodiment of the present disclosure.Its
In, this method comprises:
Step 102, experiment test information is determined；
Step 104, information is tested according to experiment, access time piece simultaneously determines round-robin rule；
Step 106, it is tested according to timeslice and round-robin rule, obtains first group of experimental data and second
The experimental data of group；
Step 108, after experiment, according to first group of the experimental data and second group of experimental data statistical experiment
Index is as a result, generate experiment conclusion.
The control experiment method based on round-robin that the embodiment of the present disclosure provides, it is determined that the experiment carried out is needed to survey
After trying information, selects reasonable timeslice and determine that round-robin rule guarantees the absolute change of two groups of the experimental data obtained
Change is smaller, conspicuousness is not significant, and wherein timeslice is time interval, and an experiment is divided into multiple timeslices, round-robin
Rule is to be tested according to timeslice according to which kind of and which kind of experimental period rotation period.Experiment is carried out again obtains first group
Experimental data and second group of experimental data, and then count experimental index result and obtain experiment conclusion, by the way that number will be tested
According to be grouped control statistics mode improve experimental data statistical accuracy, avoid traditional experiment test method (with
Machine shunts) the problem of interfering between traditional experiment test method (random to shunt) bring group is interfered between bring group.
Fig. 2 shows the streams of the control experiment method based on round-robin of another embodiment of the embodiment of the present disclosure
Journey schematic diagram.Wherein, this method comprises:
Step 202, experiment test information is determined；
Step 204, information access time piece range is tested according to experiment；
Step 206, first group of history experimental data and second group of history experimental data are obtained, calculates first group
The absolute change value and significance value of history experimental data and second group of history experimental data；
Step 208, according to absolute change value and significance value in timeslice range access time piece, while when determining
Between piece rotation rule；
Step 210, test period is divided into time interval T according to timeslice and round-robin ruleij；If (i,
j)∈S1, then will be in time interval TijThe experimental data got is as first group of experimental data；(if i, j) ∈ S2, then
It will be in time interval TijThe experimental data got is as second group of experimental data；
Step 212, after experiment, according to first group of the experimental data and second group of experimental data statistical experiment
Index is as a result, generate experiment conclusion.
Wherein, round-robin rule includes default experimental period and default rotation period；Time interval TijTime it is long
Spending is timeslice, i=1,2 ..., n, j=1,2 ..., m, n are the default rotation period, and m is default experimental period, S1
={ (i, j): (i mod 2=0 ∩ j mod 2=0) ∪ (i mod 2=1 ∩ j mod 2=1) }, S2={ (i, j): (i
Mod 2=0 ∩ j mod 2=1) ∪ (i mod 2=1 ∩ j mod 2=0) }.
In this embodiment, the approximate range for the test content access time piece for including according to experiment test information, such as
If the timeslice interference of the test content of experiment may be larger, just timeslice as big as possible, if the main shadow of test content
Loud is short-term effect, then can choose relatively small timeslice.According to history reality after the approximate range of timeslice determines
Absolute change and conspicuousness that data calculate separately two groups of experimental datas are tested, selects one to the greatest extent according to absolute change and conspicuousness
Possible big timeslice, then determine round-robin rule, by selecting reasonable timeslice and round-robin rule
It can guarantee that the absolute change of two groups of the experimental data obtained is smaller, conspicuousness is not significant, avoid interfering between experimental data group
And experiment is caused to be judged by accident.Round-robin rule is real according to default experimental period and default rotation period progress according to timeslice
It tests, i.e. experiment is carried out according to two rotation, and a rotation is experimental period, for example, 1 day to 2017 1 January in 2017
The moon 14, another rotation are rotation period, for example, 00:00 to 00:59,01:00 to 01:59,02:00 to 02:59 etc..?
Test period is divided according to timeslice and round-robin rule after selecting reasonable timeslice and round-robin rule
For time interval Tij, time interval TijTime span be the timeslice that selects, as (i, j) ∈ S1, i.e. i and j is
Even number or when being odd number, will be in time interval TijThe experimental data got is as first group of experimental data, as (i, j)
∈S2, i.e. i with j mono- be odd number another be even number when, will be in time interval TijThe experimental data got is as second group
Experimental data, realize that first group replace acquisition experimental data to keep two groups of data interference-free with second group.
In one embodiment of the embodiment of the present disclosure, it is preferable that experiment test information include test content, test refer to
Mark, test city.
It in this embodiment, include test content, test index, test city, test content example in experiment test information
Such as divide single distance, dynamic tune multiple, test index such as order response rate, test city is then selected in conjunction with specific business.
The embodiment of embodiment of the present disclosure second aspect proposes a kind of control experiment system based on round-robin, figure
3a shows the schematic block diagram of the control experiment system 300 based on round-robin of one embodiment of the embodiment of the present disclosure.
Wherein, which includes:
Determination unit 302, for determining that information is tested in experiment；
Selection unit 304, for testing information according to experiment, access time piece simultaneously determines round-robin rule；
Experimental considerations unit 306 obtains first group of experiment number for being tested according to timeslice and round-robin rule
According to the experimental data with second group；
Statistic unit 308 counts after testing according to first group of experimental data and second group of experimental data
Experimental index is as a result, generate experiment conclusion.
The control experiment system 300 based on round-robin that the embodiment of the present disclosure provides, determination unit 302, which has determined, to be needed
After the experiment to be carried out test information, reasonable timeslice is selected by selection unit 304 and determines that round-robin rule is protected
Demonstrate,prove two groups of the experimental data obtained absolute change is smaller, conspicuousness is not significant, wherein timeslice is time interval, one
Experiment is divided into multiple timeslices, and round-robin rule is according to timeslice according to which kind of and which kind of experimental period rotation period
It is tested.Experimental considerations unit 306 carries out experiment and obtains first group of experimental data and second group of experimental data, and then by counting
Unit 308 counts experimental index result and obtains experiment conclusion, is mentioned in such a way that experimental data to be grouped to control statistics
High experimental data statistical accuracy avoids asking of interfering between traditional experiment test method (random to shunt) bring group
Topic.
Fig. 3 b shows the control experiment system 300 based on round-robin of another embodiment of the embodiment of the present disclosure
Schematic block diagram.Wherein, which includes:
Determination unit 302, for determining that information is tested in experiment；
Selection unit 304, for testing information according to experiment, access time piece simultaneously determines round-robin rule；
Experimental considerations unit 306 obtains first group of experiment number for being tested according to timeslice and round-robin rule
According to the experimental data with second group；
Statistic unit 308 counts after testing according to first group of experimental data and second group of experimental data
Experimental index is as a result, generate experiment conclusion.
Selection unit 304, for testing information access time piece range according to experiment；
Control experiment system 300 based on round-robin, further includes: computing unit 310 is gone through for obtaining first group
History experimental data and second group of history experimental data calculate first group of history experimental data and test with second group of history
The absolute change value and significance value of data；
Selection unit 304 is also used to according to absolute change value and significance value the access time piece in timeslice range,
Round-robin rule is determined simultaneously.
In this embodiment, selection unit 304 is according to the big of the experiment test information test content access time piece that includes
Range is caused, for example, if the timeslice interference of the test content of experiment may be larger, with regard to timeslice as big as possible, if test
What content mainly influenced is short-term effect, then can choose relatively small timeslice.After the approximate range of timeslice determines
The absolute change and conspicuousness for calculating separately two groups of experimental datas according to history experimental data by computing unit 310, by choosing
Unit 304 selects a timeslice as big as possible according to absolute change and conspicuousness, then determines round-robin rule,
It can guarantee the absolute change of two groups of the experimental data obtained by selecting reasonable timeslice and round-robin rule
It is smaller, conspicuousness is not significant, avoid interfering between experimental data group and experiment being caused to be judged by accident.
In one embodiment of the embodiment of the present disclosure, it is preferable that round-robin rule include default experimental period and
The default rotation period；Experimental considerations unit 306, is specifically used for: drawing test period according to timeslice and the round-robin rule
It is divided into time interval Tij；(if i, j) ∈ S1, then will be in time interval TijThe experimental data got is as first group of experiment
Data；(if i, j) ∈ S2, then will be in time interval TijThe experimental data got is as second group of experimental data；Wherein,
Time interval TijTime span be timeslice, i=1,2 ..., n, j=1,2 ..., m, n are the default rotation period,
M is default experimental period, S1={ (i, j): (i mod 2=0 ∩ j mod 2=0) ∪ (i mod 2=1 ∩ j mod 2=
1) }, S2={ (i, j): (i mod 2=0 ∩ j mod 2=1) ∪ (i mod 2=1 ∩ j mod 2=0) }.
In this embodiment, round-robin rule is according to timeslice according to default experimental period and default rotation period
It is tested, i.e. experiment is carried out according to two rotation, and a rotation is experimental period, and for example, on January 1st, 2017 is extremely
On January 14th, 2017, another rotation are to rotate the period, for example, 00:00 to 00:59,01:00 to 01:59,02:00 to 02:
59 etc..When that will be tested after selecting reasonable timeslice and round-robin rule according to timeslice and round-robin rule
Between be divided into time interval Tij, time interval TijTime span be the timeslice that selects, as (i, j) ∈ S1, i.e. i with
It, will be in time interval T when j is even number or is odd numberijThe experimental data got as first group of experimental data, when
(i, j) ∈ S2, i.e. i with j mono- be odd number another be even number when, will be in time interval TijThe experimental data got is as
Two groups of experimental data realizes that first group replaces acquisition experimental data with second group to keep two groups of data interference-free.
In one embodiment of the embodiment of the present disclosure, it is preferable that experiment test information include test content, test refer to
Mark, test city.
It in this embodiment, include test content, test index, test city, test content example in experiment test information
Such as divide single distance, dynamic tune multiple, test index such as order response rate, test city is then selected in conjunction with specific business.
In a specific embodiment of the embodiment of the present disclosure, in trip scene when the strategy such as optimization point list, dynamic tune, simultaneously
Passenger and driver are influenced, using the random distributing strategy such as passenger, driver, order, the data of experimental group and control group are disturbed shadow
It is unreliable to ring data conclusion.Control experiment method based on round-robin is applied in the scene of trip, mainly in short-term
When influence, interference can be preferably solved the problems, such as.In practice, be likely to be encountered in experimental period anomalous weather, festivals or holidays or
When the factors such as restricting the number influence transport power, data result can be unreliable, it may be considered that rejects part Experiment number of days, extends experimental period.
Control experiment is carried out to passenger and acquires data, this method comprises:
(1) content, experimental index and the experimental city of test are clearly tested, test content for example divides single distance, dynamic tune times
Number, experimental index such as order response rate, experimental city combine corresponding service to be selected.
(2) suitable timeslice is chosen, determines round-robin rule.The selection Binding experiment test content of timeslice and
AA analysis.If the different timeslice interference of test content may be larger, big timeslice, such as 3 hours are just selected as far as possible
Within, if experiment test content mainly influences is short-term effect, can choose relatively small timeslice, such as 30
Within minute or within 1 hour.After general timeslice range has been determined according to test content, AA analysis is carried out.AA is analyzed
Want to determine in upper experiment the difference of experimental group and the timeslice of control group whether very little, experimentally avoided after line in this way
Timeslice bring difference itself and the erroneous judgement for affecting experimental result.There are two types of modes for AA analysis:
Mode 1: two all data before statistical experiment, the absolute change and conspicuousness of experiment with computing group and control group,
A timeslice as big as possible is selected, and absolute change is smaller, and statistically not significant.If statistically significant, absolutely
Variation is also very small, nor affects on experiment erroneous judgement, also can choose.It is preferred that the former.
Mode 2: 3 months data before choice experiment, since the time of selection is longer, it should be noted that rejecting festivals or holidays number
According to, and using smooth movement daily, the absolute change data of multiple two weeks experimental groups and control group are obtained, and then depict
The distribution of index absolute change.The mean value and conspicuousness of parameter, select a timeslice as big as possible, and mean value compared with
It is small, not significantly.If it is statistically significant, but absolute change is very small, nor affects on experiment erroneous judgement, also can choose.But it is best
It is the former.Wherein, the smaller result that obtains of 1 calculation amount of mode is very fast, and 2 calculation amount of mode is larger but result information is more.Do not consider
Mode 2 can be considered when calculating speed problem.After determining timeslice, determines experimental period and rotation period, that is, can determine the time
Piece rotation rule.
According to the round-robin grouping selected, T is specially divided time intoij, i=1,2 ..., n, j=1,
2 ..., m, time interval TijTime span be timeslice, Z=1 and Z=0 respectively represent experimental group and control group, if
(i, j) ∈ S1, then Z=0；(if i, j) ∈ S2, then Z=1；Wherein, S1={ (i, j): (i mod 2=0 ∩ j mod 2=0)
∪ (i mod 2=1 ∩ j mod 2=1) }, S2={ (i, j): (i mod 2=0 ∩ j mod 2=1) ∪ (i mod 2=1
∩ j mod 2=0) }.
(3) online experiment waits experiment to terminate.It is tested according to the decision experimental subjects execution of round-robin rule or right
According to strategy, order, passenger etc. is may be selected in experimental subjects.Experimental period is generally chosen 2 weeks, and lesser experiment may be selected 4 weeks.
As shown in table 1, rotation includes two, and first when referring to that experimental group and control group in one day appear alternatively in different
Between interval in, second refers to that experimental group and control group are alternately present in first time interval of every day of experimental period, with 1
For hour, experimental period: 2017/01/01 to 2017/1/14, time interval: 00:00 to 00:59,01:00 to 01:59,
02:00 to 02:59 etc..
Table 1
(4) statistical experiment index is as a result, provide experiment conclusion after testing.Such as experimental index be response rate when, answer
Rate=response order numbers/bill number is answered, response order numbers is defined and bill number is respectively Yij,Xij, experimental group and control group response
Rate is respectivelyAnd calculate conspicuousness T (Z=1)-T (Z=0).
The embodiment of the embodiment of the present disclosure third aspect proposes that a kind of computer installation 400, Fig. 4 show disclosure reality
Apply the schematic block diagram of the computer installation 400 of one embodiment of example.Wherein, which includes:
Memory 402, processor 404 and it is stored in the computer journey that can be run on memory 402 and on processor 404
Sequence, processor 404 are realized when executing computer program such as the control experiment method based on round-robin of any of the above-described
Step.
The computer installation 400 that the embodiment of the present disclosure provides, realization, which has determined, when processor 404 executes computer program needs
After the experiment to be carried out test information, selects reasonable timeslice and determine that round-robin rule guarantees two groups of the reality obtained
The absolute change for testing data is smaller, conspicuousness is not significant, and wherein timeslice is time interval, and an experiment is divided into multiple times
Piece, round-robin rule are to be tested according to timeslice according to which kind of and which kind of experimental period rotation period.It carries out again
Experiment obtains first group of experimental data and second group of experimental data, and then counts experimental index result and obtain experiment knot
By, experimental data is grouped control statistics by way of improve experimental data statistical accuracy, avoid tradition
The problem of being interfered between experimental test procedures (random to shunt) bring group.
The embodiment of embodiment of the present disclosure fourth aspect proposes a kind of computer readable storage medium, is stored thereon with
Computer program realizes the control experiment based on round-robin such as any of the above-described when computer program is executed by processor
The step of method.
The computer readable storage medium that the embodiment of the present disclosure provides is realized when computer program is executed by processor and is determined
After the experiment test information for needing to carry out, selects reasonable timeslice and determine that round-robin rule guarantees two groups obtained
Experimental data absolute change is smaller, conspicuousness is not significant, wherein timeslice is time interval, and an experiment is divided into multiple
Timeslice, round-robin rule are to be tested according to timeslice according to which kind of and which kind of experimental period rotation period.Again
It carries out experiment and obtains first group of experimental data and second group of experimental data, and then count experimental index result and obtain experiment
Conclusion improves experimental data statistical accuracy in such a way that experimental data to be grouped to control statistics, avoids biography
The problem of being interfered between system experimental test procedures (random to shunt) bring group.
In the description of this specification, the description of term " one embodiment ", " some embodiments ", " specific embodiment " etc.
Mean that particular features, structures, materials, or characteristics described in conjunction with this embodiment or example are contained in the embodiment of the present disclosure at least
In one embodiment or example.In the present specification, schematic expression of the above terms are not necessarily referring to identical implementation
Example or example.Moreover, the particular features, structures, materials, or characteristics of description in any one or more embodiments or can be shown
It can be combined in any suitable manner in example.
The foregoing is merely the preferred embodiments of the embodiment of the present disclosure, are not limited to the embodiment of the present disclosure, right
For those skilled in the art, the embodiment of the present disclosure can have various modifications and variations.All essences in the embodiment of the present disclosure
Within mind and principle, any modification, equivalent replacement, improvement and so on should be included in the protection scope of the embodiment of the present disclosure
Within.
Claims (10)
1. a kind of control experiment method based on round-robin characterized by comprising
Determine experiment test information；
Information is tested according to the experiment, access time piece simultaneously determines round-robin rule；
Tested according to the timeslice and the round-robin rule, obtain first group experimental data and second group
Experimental data；
After experiment, according to first group of the experimental data and second group of the experimental data statistical experiment index knot
Fruit generates experiment conclusion.
2. the control experiment method according to claim 1 based on round-robin, which is characterized in that described according to
Experiment test information, chooses the timeslice and determines the step of the round-robin rule, specifically include:
Information access time piece range is tested according to the experiment；
First group of history experimental data and second group of history experimental data are obtained, first group of the history experiment is calculated
The absolute change value and significance value of data and second group of history experimental data；
The timeslice is chosen in the timeslice range according to the absolute change value and the significance value, while really
The fixed round-robin rule.
3. the control experiment method according to claim 2 based on round-robin, which is characterized in that the timeslice wheel
Turning rule includes default experimental period and default rotation period；
It is described to be tested according to the timeslice and the round-robin rule, obtain first group of experimental data and second
It the step of experimental data of group, specifically includes:
Test period is divided into time interval T according to the timeslice and the round-robin ruleij；
(if i, j) ∈ S1, then will be in the time interval TijThe experimental data got is as described first group of experiment number
According to；
(if i, j) ∈ S2, then will be in the time interval TijThe experimental data got is as described second group of experiment number
According to；
Wherein, the time interval TijTime span be the timeslice, i=1,2 ..., n, j=1,2 ...,
M, n are the default rotation period, and m is the default experimental period, S1={ (i, j): (i mod 2=0 ∩ j mod 2=0)
∪ (i mod 2=1 ∩ j mod 2=1) }, S2={ (i, j): (i mod 2=0 ∩ j mod 2=1) ∪ (i mod 2=1
∩ j mod 2=0) }.
4. the control experiment method according to any one of claim 1 to 3 based on round-robin, which is characterized in that
The experiment test information includes test content, test index, test city.
5. a kind of control experiment system based on round-robin characterized by comprising
Determination unit, for determining that information is tested in experiment；
Selection unit, for testing information according to the experiment, access time piece simultaneously determines round-robin rule；
Experimental considerations unit obtains first group of experiment for being tested according to the timeslice and the round-robin rule
Data and second group of experimental data；
Statistic unit is united after testing according to first group of the experimental data and second group of the experimental data
Experimental index is counted as a result, generating experiment conclusion.
6. the control experiment system according to claim 5 based on round-robin, which is characterized in that
The selection unit, for testing information access time piece range according to the experiment；
The control experiment system based on round-robin, further includes:
Computing unit calculates described for obtaining first group of history experimental data and second group of history experimental data
The absolute change value and significance value of one group of history experimental data and second group of history experimental data；
The selection unit is also used to be selected in the timeslice range according to the absolute change value and the significance value
The timeslice is taken, while determining the round-robin rule.
7. the control experiment system according to claim 6 based on round-robin, which is characterized in that the timeslice wheel
Turning rule includes default experimental period and default rotation period；
The experimental considerations unit, is specifically used for:
Test period is divided into time interval T according to the timeslice and the round-robin ruleij；
(if i, j) ∈ S1, then will be in the time interval TijThe experimental data got is as described first group of experiment number
According to；
(if i, j) ∈ S2, then will be in the time interval TijThe experimental data got is as described second group of experiment number
According to；
Wherein, the time interval TijTime span be the timeslice, i=1,2 ..., n, j=1,2 ...,
M, n are the default rotation period, and m is the default experimental period, S1={ (i, j): (i mod 2=0 ∩ j mod 2=0)
∪ (i mod 2=1 ∩ j mod 2=1) }, S2={ (i, j): (i mod 2=0 ∩ j mod 2=1) ∪ (i mod 2=1
∩ j mod 2=0) }.
8. the control experiment system according to any one of claims 5 to 7 based on round-robin, which is characterized in that
The experiment test information includes test content, test index, test city.
9. a kind of computer installation, including memory, processor and it is stored on the memory and can be on the processor
The computer program of operation, which is characterized in that the processor realizes such as Claims 1-4 when executing the computer program
Any one of described in the control experiment method based on round-robin the step of.
10. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the computer program
Control experiment method based on round-robin according to any one of claims 1 to 4 is realized when being executed by processor
Step.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710899136.4A CN109583670B (en) | 2017-09-28 | 2017-09-28 | Control experiment method, system and computer device based on time slice rotation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710899136.4A CN109583670B (en) | 2017-09-28 | 2017-09-28 | Control experiment method, system and computer device based on time slice rotation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109583670A true CN109583670A (en) | 2019-04-05 |
| CN109583670B CN109583670B (en) | 2020-12-04 |
Family
ID=65913917
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710899136.4A Active CN109583670B (en) | 2017-09-28 | 2017-09-28 | Control experiment method, system and computer device based on time slice rotation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109583670B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112465604A (en) * | 2020-12-14 | 2021-03-09 | 深圳依时货拉拉科技有限公司 | Order distribution method and device, computer equipment and computer readable storage medium |
| CN113448876A (en) * | 2021-08-31 | 2021-09-28 | 腾讯科技（深圳）有限公司 | Service testing method, device, computer equipment and storage medium |
| CN115361340A (en) * | 2022-08-24 | 2022-11-18 | 深圳依时货拉拉科技有限公司 | AB experiment shunting method and device, computer equipment and storage medium |
| CN117033946A (en) * | 2023-09-21 | 2023-11-10 | 深圳依时货拉拉科技有限公司 | AB experiment method, AB experiment device, computer equipment and computer readable storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1436005A (en) * | 2002-01-30 | 2003-08-13 | 华邦电子股份有限公司 | Method and system for measuring accuracy of video/audio output synchronization, and analysis means |
| US7043343B1 (en) * | 2005-01-31 | 2006-05-09 | The United States Of America As Represented By The Secretary Of The Navy | Method for testing a missile time-space-position unit for missile |
| CN101727383A (en) * | 2008-10-16 | 2010-06-09 | 上海市医疗保险信息中心 | Simulation test method and system of database |
| CN101785022A (en) * | 2007-07-03 | 2010-07-21 | 3M创新有限公司 | System and method for generating time-slot samples to which content may be assigned for measuring effects of the assigned content |
| US8374597B2 (en) * | 2003-09-22 | 2013-02-12 | United Parcel Service Of America, Inc. | Symbiotic system for testing electromagnetic signal coverage in areas near transport routes |
| CN104915726A (en) * | 2015-05-11 | 2015-09-16 | 四川汇源吉迅数码科技有限公司 | Traffic flow database generation method based on time slice division |
-
2017
- 2017-09-28 CN CN201710899136.4A patent/CN109583670B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1436005A (en) * | 2002-01-30 | 2003-08-13 | 华邦电子股份有限公司 | Method and system for measuring accuracy of video/audio output synchronization, and analysis means |
| US8374597B2 (en) * | 2003-09-22 | 2013-02-12 | United Parcel Service Of America, Inc. | Symbiotic system for testing electromagnetic signal coverage in areas near transport routes |
| US7043343B1 (en) * | 2005-01-31 | 2006-05-09 | The United States Of America As Represented By The Secretary Of The Navy | Method for testing a missile time-space-position unit for missile |
| CN101785022A (en) * | 2007-07-03 | 2010-07-21 | 3M创新有限公司 | System and method for generating time-slot samples to which content may be assigned for measuring effects of the assigned content |
| CN101727383A (en) * | 2008-10-16 | 2010-06-09 | 上海市医疗保险信息中心 | Simulation test method and system of database |
| CN104915726A (en) * | 2015-05-11 | 2015-09-16 | 四川汇源吉迅数码科技有限公司 | Traffic flow database generation method based on time slice division |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112465604A (en) * | 2020-12-14 | 2021-03-09 | 深圳依时货拉拉科技有限公司 | Order distribution method and device, computer equipment and computer readable storage medium |
| CN113448876A (en) * | 2021-08-31 | 2021-09-28 | 腾讯科技（深圳）有限公司 | Service testing method, device, computer equipment and storage medium |
| CN113448876B (en) * | 2021-08-31 | 2021-11-19 | 腾讯科技（深圳）有限公司 | Service testing method, device, computer equipment and storage medium |
| CN115361340A (en) * | 2022-08-24 | 2022-11-18 | 深圳依时货拉拉科技有限公司 | AB experiment shunting method and device, computer equipment and storage medium |
| CN115361340B (en) * | 2022-08-24 | 2024-02-27 | 深圳依时货拉拉科技有限公司 | AB experiment shunting method and device, computer equipment and storage medium |
| CN117033946A (en) * | 2023-09-21 | 2023-11-10 | 深圳依时货拉拉科技有限公司 | AB experiment method, AB experiment device, computer equipment and computer readable storage medium |
| CN117033946B (en) * | 2023-09-21 | 2023-12-22 | 深圳依时货拉拉科技有限公司 | AB experiment method, AB experiment device, computer equipment and computer readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109583670B (en) | 2020-12-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109583670A (en) | Control experiment method, system and computer installation based on round-robin | |
| Dube et al. | Pooling multiple case studies using synthetic controls: An application to minimum wage policies | |
| Jerven | The quest for the African dummy: explaining African post‐colonial economic performance revisited | |
| Natanelov et al. | Is there co-movement of agricultural commodities futures prices and crude oil? | |
| CN105610654B (en) | Server, and method and system for policy online testing | |
| CN104281611B (en) | Customer flow distribution method and device in Website testing system | |
| CN104765751A (en) | Application recommendation method and device | |
| CN105975433B (en) | A kind of message processing method and device | |
| Kam et al. | The electoral system, the party system and accountability in parliamentary government | |
| KR20200079157A (en) | System and method for stock analysis with stock index | |
| Dube et al. | Pooled synthetic control estimates for recurring treatments: an application to minimum wage case studies | |
| CN109086339A (en) | It is a kind of for generating the data processing method and device of index recombination rate | |
| CN108304482A (en) | The recognition methods and device of broker, electronic equipment and readable storage medium storing program for executing | |
| CN110333984A (en) | Interface method for detecting abnormality, device, server and system | |
| CN109597744A (en) | Data unusual fluctuation analysis method and device | |
| CN109165251A (en) | A kind of data information dynamic display method, electronic equipment and readable storage medium storing program for executing | |
| CN114723071A (en) | Federal learning method and device based on client classification and information entropy | |
| Cooray | Economic Growth and the Size and Quality of the Government | |
| CN110782276B (en) | Access shunting policy interference judgment method and device and electronic equipment | |
| CN107491378A (en) | A kind of determination method and device of CPU usage | |
| CN110648181B (en) | Method and device for controlling delivery channel based on monitoring delivery efficiency and electronic equipment | |
| CN110472827B (en) | System policy early warning method, device, server and readable storage medium | |
| Moradi et al. | Impact of information and communication technology on economic growth in selected Islamic countries | |
| CN110390594A (en) | Method for visualizing, device and the medium of economy and finance data cycle variation law | |
| CN106909982A (en) | A kind of method and device for optimizing advertisement keyword match pattern |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |