CN109039800A

CN109039800A - The method, apparatus and computer equipment of assignment of traffic are carried out in flux experiment

Info

Publication number: CN109039800A
Application number: CN201810686804.XA
Authority: CN
Inventors: 樊春霖; 丁如敏; 叶方正; 赵田
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2018-06-28
Filing date: 2018-06-28
Publication date: 2018-12-18

Abstract

This application involves a kind of to carry out method, apparatus, computer equipment and the readable storage medium storing program for executing of assignment of traffic in flux experiment, this method comprises: obtaining the corresponding user identifier of customer flow and user property；Respectively according to the hash algorithm of flow layer each in flux experiment, Hash operation is carried out to user identifier, obtains the corresponding cryptographic Hash of customer flow；The hash algorithm of any two flow layer meets orthogonal principle；According to cryptographic Hash and user property, customer flow matched experiment content in each flow layer is determined；Customer flow is distributed to matched experiment content.By considering user property when carrying out assignment of traffic, customer flow distribution is carried out using user property, the experiment that can be supported more various dimensions and more refine, customer flow match corresponding experiment content according to the user property of itself, realize and carry out individual demand experiment for different user.

Description

The method, apparatus and computer equipment of assignment of traffic are carried out in flux experiment

Technical field

This application involves website the field of test technology, more particularly to a kind of side for carrying out assignment of traffic in flux experiment Method, device, computer equipment and readable storage medium storing program for executing.

Background technique

With the growth to network upgrade demand in internet, new strategy exploitation finishes it in website new function either station Afterwards, it needs to carry out flux experiment to new function or new strategy before full flow is online.Flux experiment refers to customer flow point It is assigned to new function or the corresponding control experiment of old function, and according to the data of customer flow feedback to new function or new strategy Performance, effect carry out test assessment, with according to assessment result determine it is final whether the online new function of full flow or new strategy, or Determine final full flow online new function or new strategy.

When carrying out website test based on flux experiment, need to be allocated customer flow, so that certain customers' flow Using the new website with new function or new strategy, certain customers' flow still uses old website.In traditional flux experiment, press Assignment of traffic is carried out according to User ID, the dimension for distributing flow is fixed, and it is real can not to carry out individual demand for different user simultaneously It tests.

Summary of the invention

Based on this, it is necessary to real in traditional flux experiment individual demand can not be carried out for different user simultaneously The technical issues of testing provides a kind of method, apparatus, computer equipment and readable storage that assignment of traffic is carried out in flux experiment Medium.

A method of carrying out assignment of traffic in flux experiment, which comprises

Obtain the corresponding user identifier of customer flow and user property；

Respectively according to the hash algorithm of flow layer each in flux experiment, Hash operation is carried out to the user identifier, is obtained The corresponding cryptographic Hash of the customer flow；The hash algorithm of any two flow layer meets orthogonal principle；

According to the cryptographic Hash and the user property, determine the customer flow in each flow layer in matched experiment Hold；

The customer flow is distributed to the matched experiment content.

A kind of device carrying out assignment of traffic in flux experiment, described device include:

Module is obtained, for obtaining the corresponding user identifier of customer flow and user property；

Hash module, for according to the hash algorithm of flow layer each in flux experiment, being carried out respectively to the user identifier Hash operation obtains the corresponding cryptographic Hash of the customer flow；The hash algorithm of any two flow layer meets orthogonal principle；

Determining module, for determining the customer flow in each flow layer according to the cryptographic Hash and the user property In matched experiment content；

Distribution module, for distributing the customer flow to the matched experiment content.

A kind of computer equipment, including memory and processor, the memory are stored with computer program, the processing Device performs the steps of when executing the computer program

Obtain the corresponding user identifier of customer flow and user property；

The customer flow is distributed to the matched experiment content.

A kind of computer readable storage medium, is stored thereon with computer program, and the computer program is held by processor It is performed the steps of when row

Obtain the corresponding user identifier of customer flow and user property；

The customer flow is distributed to the matched experiment content.

Above-mentioned method, apparatus, computer equipment and the readable storage medium storing program for executing that assignment of traffic is carried out in flux experiment, obtains The corresponding user identifier of customer flow and user property, and respectively according to the hash algorithm of flow layer each in flux experiment, to Family mark carries out Hash operation, obtains the corresponding cryptographic Hash of customer flow, wherein the hash algorithm of any two flow layer meets Orthogonal principle.Then according to cryptographic Hash and user property, determines customer flow matched experiment content in each flow layer, will use Family assignment of traffic is to matched experiment content.By considering user property when carrying out assignment of traffic, carried out using user property Customer flow distribution, the experiment that can be supported more various dimensions and more refine, customer flow are matched according to the user property of itself Corresponding experiment content realizes and carries out individual demand experiment for different user.

Detailed description of the invention

Fig. 1 is the applied environment figure for carrying out the method for assignment of traffic in one embodiment in flux experiment；

Fig. 2 is the flow diagram for carrying out the method for assignment of traffic in one embodiment in flux experiment；

Fig. 3 is the flow diagram that matched experiment content step is determined in one embodiment；

Fig. 4 is the interface schematic diagram that control experiment initialization is carried out in one embodiment；

Fig. 5 is the interface schematic diagram that control experiment initialization is carried out in one embodiment；

Fig. 6 is the flow diagram for carrying out the method for assignment of traffic in one embodiment in flux experiment；

Fig. 7 is the interface schematic diagram for carrying out the method for assignment of traffic in one embodiment in flux experiment；

Fig. 8 is the structural block diagram for carrying out the device of assignment of traffic in one embodiment in flux experiment；

Fig. 9 is the structural block diagram for carrying out the device of assignment of traffic in another embodiment in flux experiment；

Figure 10 is the structural block diagram of computer equipment in one embodiment.

Specific embodiment

It is with reference to the accompanying drawings and embodiments, right in order to which the objects, technical solutions and advantages of the application are more clearly understood The application is further elaborated.It should be appreciated that specific embodiment described herein is only used to explain the application, and It is not used in restriction the application.

Fig. 1 is the applied environment figure for carrying out the method for assignment of traffic in one embodiment in flux experiment.Referring to Fig.1, The method that assignment of traffic is carried out in flux experiment is applied to flux experiment system.The flux experiment system includes requesting terminal 110 and flux experiment manage platform 120.Requesting terminal 110 and flux experiment management platform 120 pass through network connection.Request is eventually End 110 specifically can be terminal console or mobile terminal, and mobile terminal specifically can be with mobile phone, tablet computer, laptop etc. At least one of, for sending access request, the business ginseng that platform 120 responds access request feedback is managed according to flux experiment Number carry out page presentation, and when experiment content is exposed, clicks to flux experiment management platform 120 report experiment mark and Target component.Flux experiment manages the interactive terminal 121 and server 122 that platform 120 includes front end interaction, and server 122 can To be realized with the independent server either server cluster that forms of multiple servers.Interactive terminal 121 is for being flowed The configuration of experiment is measured, server 122 is used to execute the method that assignment of traffic is carried out in flux experiment.

As shown in Fig. 2, in one embodiment, providing a kind of method for carrying out assignment of traffic in flux experiment.This Embodiment is applied to the flux experiment management platform 120 in above-mentioned Fig. 1 in this way mainly to illustrate.Referring to Fig. 2, it is somebody's turn to do The method that assignment of traffic is carried out in flux experiment specifically comprises the following steps:

S202 obtains the corresponding user identifier of customer flow and user property.

Wherein, customer flow refers to that the business targeted to flux experiment carries out the network access user of network access.Into One step, business refers to the module in website or software with different function, for example, in a browser, business includes big information Business, searching service, advertising business etc..Further, business is subdivided into multiple subservices again, for example, big information business is again It is divided into picture and text subservice, atlas subservice, video show subservice etc..When the business of flux experiment is big information in browser When business, then customer flow refers to the network access user for accessing big information business；When the business of flux experiment is big information industry When the lower video show subservice of business, then customer flow refers to that the network of access video show subservice accesses user.User identifier is Finger can be used for indicating unique variable, such as User ID, cookie, cell-phone number of user etc..User property refers to access user institute The feature having, including but not limited to: gender, age, location, educational background etc..Further, in the present embodiment, user The user property that is configured attribute is carries out flow configuration in the flux experiment when.

Specifically, when receiving customer flow, related information entrained by customer flow is analyzed, user is obtained The corresponding user identifier of flow and user property.Wherein, related information can be information entrained by access request, be also possible to The information that requesting terminal reports simultaneously when sending access request.

S204 carries out Hash operation to user identifier, obtains respectively according to the hash algorithm of flow layer each in flux experiment The corresponding cryptographic Hash of customer flow.Wherein, the hash algorithm of any two flow layer meets orthogonal principle.

Multiple flow layers are provided in flux experiment, each flow layer shares identical customer flow, a flow layer An experiment purpose is provided with a control experiment group, again includes a pair about old strategy in a control experiment group The control experiment at least one about new strategy is according to the facts tested, different control experiments may include same or different experiment Content.For example, based on search button color, button-shape, recalling algorithm model point when carrying out flux experiment for searching service Three groups of control experiment groups, the corresponding flow layer of each control experiment group are not set.Wherein, the control experiment of search button color Group includes two control experiments of green (new strategy) of the red (old strategy) of default, correcting；The control experiment group of button-shape Three rectangular, correcting circle including default, diamond shape control experiments；The control experiment group for recalling algorithm model includes default A model, correcting two control experiments of B model.

As noted earlier, when red (old strategy), the green of correcting that the control experiment group of search button color includes default When (new strategy) two control experiments, the corresponding experiment content of two control experiments is different, respectively red search button, green Search button；And when the control experiment group of search button color includes red (old strategy), the user property X hit green of default When (new strategy), user property Y hit three control experiments of green (new strategy), user property X hit green (new strategy), The corresponding experiment content of green (new strategy) two control experiments of user property Y hit is identical, is green search button.It can be with Understand, the customer flow with different user attribute hits identical experiment content and can distinguish when carrying out experimental result assessment User property is assessed respectively, to determine influence of the different user attribute to same experiment content performance or effect.

Wherein, hash algorithm refers to data of the input data random length by a functional transformation at regular length String, is a unique mapping algorithm.It is appreciated that being directed to the same hash algorithm, input data and the serial data being converted to The two uniquely corresponds to.In the present embodiment, each flow layer is configured with a hash algorithm, using hash algorithm to user identifier Hash operation is carried out, so that the distribution that customer flow can be uniform, random is to different control experiments.

Further, the settable multiple flow layers of quantity based on control experiment group, same customer flow can order simultaneously Control experiment in middle different flow layer, namely keep limited customer flow support as much as possible in such a way that flow is multiplexed Control experiment group improves the utilization rate of customer flow and the scalability of flux experiment.

To keep the control experiment group of each flow layer relatively independent, to reduce the experiment between any two flows layer to experiment As a result influence must satisfy orthogonality between any two flows layer.Wherein, orthogonality is instigate a certain flow layer any one A flow subinterval can be random, uniformly correspond to above other flow layers, by a flow subinterval to other The influence of layer is uniformly distributed to above entire flow layer, so that experiment is tied in the interference reduced between different control experiment groups The influence of fruit.Flow subinterval refers to flow layer is divided according to the customer flow accounting situation of control experiment after obtain Multiple subintervals, for example, control experiment a and control experiment b distinguish accounting 50%, then wherein in a flow layer [1,100] The flow layer [1,100] can be divided, control experiment a is made to occupy flow subinterval [1,50], control experiment b occupies flow Subinterval [51,100].At this point, orthogonality refer to each customer flow in the flow subinterval [1,50] of the flow can uniformly, with Distribute to machine into each flow subinterval of another flow layer, each customer flow in flow subinterval [51,100] can uniformly, It is randomly assigned into each flow subinterval of another flow layer.In the present embodiment, by the way that any two flow layer is arranged Meet the hash algorithm of orthogonal principle, to guarantee the orthogonality between each flow layer.

Specifically, the hash algorithm for obtaining each flow layer carries out Hash fortune to user identifier by each hash algorithm It calculates, obtains cryptographic Hash, the corresponding cryptographic Hash quantity of each customer flow is identical as the number of plies of flow layer.

S206 determines customer flow matched experiment content in each flow layer according to cryptographic Hash and user property.

Wherein, experiment content refers to the content of each control experiment in process layer, in particular to carries out to website or software Improvement or increased new function, new strategy, or old function corresponding with new function, new strategy, old strategy.For example, being searched one In rope website in the flux experiment of search button color, the control experiment group of search button color includes red (the old plan of default Slightly), two control experiments of green (new strategy) of correcting, then red search button, green search button are respectively two controls The experiment content of experiment.

Specifically, it according to cryptographic Hash and user property, is matched respectively with the control experiment of corresponding flow layer, when the use When family flow meets the matching condition of control experiment, the experiment mark of the matched control experiment of customer flow is determined, according to experiment Mark determines matched experiment content.Wherein, experiment mark refers to for distinguishing the mark for indicating each control experiment.Experiment mark It is corresponded with control experiment, different control experiments are configured with different experiment marks.Specifically, experiment mark can be reality Title, experiment numbers etc. are tested, as long as control experiment can be indicated uniquely.

Since there is one-to-one correspondence for same hash algorithm, user identifier and cryptographic Hash, therefore, in the flux experiment phase Between, matched experiment content is identical experiment content to same customer flow every time, is not in this matching experiment content I, the case where matching experiment content II when accessing next time again, it is consistent to guarantee that influence of the same user to flow experimental result has Property.

S208 distributes customer flow to matched experiment content.

After the matched experiment content of customer flow institute has been determined, which is distributed to matched experiment content, The related data for feeding back to customer flow corresponding requests terminal is obtained based on matched experiment content, and is carried out by requesting terminal It shows.

In one embodiment, as shown in figure 3, according to cryptographic Hash and user property, determine customer flow in each flow layer The step of matched experiment content, comprising:

S302 obtains the value range in full flow section, obtains the interval value for indicating full flow section size.

Full flow section refers to each occupied flow rate zone of flow layer.For example, configuring full stream in a flux experiment Measuring section is [1, n].For indicating that the interval value of full flow section size can subtract section minimum value simultaneously by section maximum value It is obtained after adding 1.It is appreciated that the value range in full flow section is [1, n], the interval value of full flow section size is (n- 1)+1=n.

Cryptographic Hash is carried out modulus to interval value, obtains the corresponding flow value of customer flow by S304.

Flow value refers to the variable for indicating the minimum unit in flow layer.For example, in full flow section [1, n], Flow value includes: 1,2 ..., n.

In this example, the cryptographic Hash obtained by different hash algorithms is subjected to modulus to interval value respectively, by modulus Obtained numerical value, in the flow value of corresponding flow layer, is realized as the customer flow and maps to customer flow in each flow layer.

S306 determines customer flow matched experiment content in each flow layer according to flow value and user property.

Specifically, according to the corresponding user property of customer flow and in the flow value of different flow layer, to each flow layer reality It tests content to be matched, determines the matched experiment content of institute.

In one embodiment, according to flow value and user property, customer flow matched experiment in each flow layer is determined The step of content, comprising: obtain the configuration information of each control experiment in each flow layer；By flow value and user property, with configuration Information is matched, and determines matched experiment content.

Wherein, configuration information refers to matched information required for customer flow hit control experiment, includes at least but unlimited In flow subinterval and user property.Flow subinterval refers to the customer flow accounting feelings by full flow section according to control experiment Multiple subintervals that condition obtains after being divided.For example, wherein in a flow layer (full flow section [1,100]), control It tests a and control experiment b and distinguishes accounting 50%, then full flow section [1,100] can be divided into [1,50], [51,100] two Subinterval, wherein flow subinterval [1,50] is the flow subinterval for control experiment a, flow subinterval [51,100] be for The flow subinterval of control experiment b configuration.The flow subinterval of different control experiment configurations can be the same or different, and have Body is configured according to experiment demand.

Specifically, flow value and user property and are matched confidence by the configuration information for obtaining each control experiment in each flow layer Breath is matched, when flow value belongs to the corresponding flow subinterval of one of control experiment, and user's category of the customer flow When property is identical as the user property that the control experiment is configured, it is determined that the customer flow hits the control experiment namely the use Flow life in family is matched with the experiment content of the control experiment.

Further, configuration information may also include user's white list and/or subscriber blacklist.Wherein, user's white list packet The user identifier for determining and hitting the control experiment is included, subscriber blacklist includes the user identifier for not allowing to hit the control experiment. It is appreciated that as long as user identifier is recorded in user's white list of a certain control experiment, regardless of the customer flow whether The other configurations information for meeting the control experiment, can all hit the control experiment；As long as user identifier is to being recorded in a certain control In the subscriber blacklist of experiment, then the customer flow will not centainly hit the control experiment.

When configuration information further includes user's white list and/or subscriber blacklist, by flow value and user property, with configuration The step of information is matched, and determines matched experiment content, comprising: by user identifier, flow value and user property, with configuration Information is matched, and determines matched experiment content.

Specifically, judge whether user identifier is the user identifier that records in user's white list of control experiment, if so, The control experiment is hit, matched experiment content is determined according to the experiment of the control experiment of hit mark；And/or judge user Identify whether the user identifier recorded in the subscriber blacklist for each control experiment, and by flow value and user property, and it is each right The configuration information according to the facts tested is matched, when flow value belongs to the corresponding flow subinterval of one of control experiment, user belongs to Property is identical as the user property that the control experiment is configured, and when user identifier is not belonging to subscriber blacklist, it is determined that the user Flow hits the control experiment, determines matched experiment content according to the experiment of the control experiment of hit mark.

It in one embodiment, further include being initialized to the control experiment of new strategy, just before carrying out flux experiment It include the configuration information of control experiment in beginningization data.The configuration information default of the control experiment of old strategy includes: in full flow In section, meet the flow other than the configuration information of new strategy control experiment.

Specifically, experiment mark, the hash algorithm of the control experiment of new strategy are set first, as shown in figure 4, experiment mark Including module number (i.e. experiment numbers) and module title (i.e. Experiment name), hash algorithm selects BKDRHash algorithm, in addition, It can also configure relevant remark information.Further, to the configuration information of each control experiment, experiment content, time started, end Time, type of service, subservice type, evaluation index, target component etc. are configured.It is mentioned with the Guangdong Province user for configuring 10% For the control experiment of green login button, as shown in figure 5, experiment content corresponds to the module parameter information in Fig. 5, specifically For " background color green "；Configuration information corresponds to the distributing policy in Fig. 5, specifically includes the corresponding flow of the control experiment Subinterval accounts for the ratio 10% in full flow section, user property " Guangdong " etc.；Evaluation index corresponds to the observation index in Fig. 5； Subservice type corresponds to the channel type in Fig. 5, etc..By being initialized to all data involved in control experiment, When carrying out flux experiment, the matching of experiment content is carried out based on initialization data.

In one embodiment, before obtaining the corresponding user identifier of customer flow and user property, further includes: receive request The access request that terminal is sent；When the corresponding access business of access request belongs to the scope of business of flux experiment, access is obtained The corresponding flux experiment of business.

Wherein, access request refers to the request of a certain website of access or software initiated by requesting terminal.Specifically it can be By inputting network address, the request that the network address corresponds to website is accessed to server initiation.

Specifically, the access request that requesting terminal is sent is received, access request is analyzed, the access request pair is obtained The access business answered, and the access business is matched with the type of service of configured flux experiment, when configured flow is real In all types of service tested include the access business when, by access business with matched type of service subservice type into Row matching, obtains the initialization data of each control experiment corresponding with the matched subservice type of the access business, according to each right The flux experiment for the initialization data composition access business according to the facts tested.It is appreciated that access business may include one or more Type of service can specifically include one or more subservice types under a type of service, for example, leading in browser homepage It often include simultaneously searching service and advertising business, searching service includes search engine subservice and search box subservice, advertising Business further includes that video ads, picture-text advertisement access business then when access request is the request of access browser homepage The video ads subservice and picture and text of search engine subservice and search box subservice and advertising business including searching service The video of advertisement subservice, then the search engine subservice and search box subservice and advertising business that obtain searching service is wide The initialization data for accusing each control experiment of subservice and picture-text advertisement subservice, according to the initialization data group of each control experiment At the flux experiment of access business.

It is appreciated that executing the visit when the corresponding access business of access request is not belonging to the scope of business of flux experiment Ask business corresponding default policy.

Further, the step of obtaining the corresponding user identifier of customer flow and user property, comprising: parsing access is asked It asks；The user identifier and user property of the corresponding customer flow of access request are obtained according to parsing result.

Wherein, user identifier and user property are carried in access request.When the corresponding access business of access request belongs to When the scope of business of flux experiment, further access request is parsed, obtain the corresponding user identifier of the customer flow and User property.The matched experiment content of institute is determined according to obtained user identifier and user property.

In one embodiment, the step of customer flow being distributed to matched experiment content, comprising: obtain experiment content pair The experiment mark and service parameter answered；Experiment mark with service parameter is sent to the corresponding requesting terminal of customer flow.

Wherein, service parameter refers to being obtained according to the corresponding access business of access request and including in matched experiment The service related data of appearance.For example, matched experiment content includes that green search is pressed when access request is access search website Button, search button are located at the page upper right corner, then service parameter refers to searching including green search button, search button position Rope Website page display data.

Specifically, matched experiment content is obtained and in addition to experiment content, other business phases of response access request Data are closed, and are assembled into service parameter, service parameter and experiment mark are back to the corresponding requesting terminal of the customer flow, So that requesting terminal carries out page presentation according to service parameter is received.

In one embodiment, this method further include: receive after requesting terminal carries out page presentation according to service parameter and feed back Experiment mark and target component.

Target component required data, target of different business when referring to through flux experiment progress experimental result assessment Parameter may be different.For example, target component can include but is not limited to: user identifier, cell phone system, browser version number, property Not, age and/or order price etc..

It, can be actively anti-when exposing and clicking experiment content after requesting terminal carries out page presentation according to service parameter Feedback experiment mark and target component to flux experiment manage platform, and flux experiment manages the experiment that platform receives requesting terminal feedback Mark and target component, with for statistical analysis according to experiment mark and target component, for carrying out effect to each control experiment Assessment.

In one embodiment, this method further include: when flux experiment reaches termination condition, respectively to each experiment mark pair The target component answered is for statistical analysis, obtains each experiment and identifies corresponding evaluation index value；Evaluation index value for determine to The experiment content of publication.

Wherein, termination condition refers to the condition for being used to indicate flux experiment failure.It can be the preconfigured end time, The quantity for being also possible to access request or customer flow reaches preset value.Evaluation index value refers to for each control experiment The correlation values that performance, effect are assessed.The evaluation index of same control experiment group is identical, the evaluation of different control experiment groups Index may be same or different.Specifically, evaluation index includes but is not limited to one in exposure rate, clicking rate, conversion ratio etc. Kind is a variety of.For example, the conversion ratio of advertisement is often most paid close attention in advertising business, it can be using conversion ratio as advertising business correlation Control experiment group evaluation index；It, can be relevant right using the clicking rate most paid close attention to as news in news According to the evaluation index of experimental group.

Specifically, corresponding evaluation index value operation method is identified according to each experiment, united to corresponding target component Meter analysis obtains each experiment and identifies corresponding evaluation index value.In general, evaluation index value more meets expectation development, then explanation should The corresponding control experiment of evaluation index value is more preferable compared to the effect in other control experiments in same group.For example, for wide Accuse display location setting control experiment in, advertisement show position setting in the upper left corner the corresponding conversion ratio highest of control experiment, Indicate that the advertising results for obtaining advertising display in the page upper left corner are best, it therefore, can be by advertisement show position setting in the upper left corner It is determined as experiment content to be released, and full flow is online.

Further, since customer flow is distributed according to user property to matched experiment content, therefore, in conjunction with user property Corresponding evaluation index value compares and analyzes, and can also determine in the experiment of interest of the user with different user attribute Hold, and then determines the online experiment content of full flow or carry out the online experiment content of partial discharge by user property.Also It is to say, carries out assignment of traffic in conjunction with user property, it is whether online or online to each experiment content to improve evaluation index value Range of flow can guiding value.

It is above-mentioned in flux experiment carry out assignment of traffic method, when carrying out assignment of traffic simultaneously consider user identifier with User property carries out customer flow distribution using user identifier and user property, can support more various dimensions and more refine Experiment, customer flow match corresponding experiment content according to the user property of itself, realize and carry out individual character for different user The experiment of change demand, and improve the evaluation index value range of flow whether online or online to each experiment content can guiding price Value.And making it possible to carry out the different control experiment of multiple groups simultaneously by the way that multilayer flow layer is arranged, same customer flow can match The control experiment of different flow layer improves the utilization rate of customer flow and the scalability of flux experiment, and each layer is matched Set the hash algorithm for meeting orthogonality, guarantee between each layer control experiment group it is mutually indepedent, be independent of each other, it is ensured that experiment As a result accuracy.

Below by taking the corresponding flux experiment of the searching service of website W as an example, to being flowed in flux experiment for the application The method of amount distribution is illustrated.Its full flow section is [1,100], and one of flow layer includes control experiment A, B, C.It is false If the experiment content of control experiment A and control experiment B are improved search interface, control experiment C original search interface tests A's Configuration information includes: flow subinterval [1,10], user property " male "；Experiment B configuration information include: flow subinterval [1, 10], user property " female "；The configuration information default for testing C includes: to meet the configuration of control experiment A, B in full flow section Flow other than information, in the present embodiment, it includes: flow subinterval [11,100], user that the configuration information for testing C, which is equivalent to, Attribute " male, female ".As shown in fig. 6, this method comprises:

S601 receives the access request that requesting terminal is sent.

Wherein, access request refers to the request of a certain website of access or software initiated by requesting terminal.Specifically it can be By inputting network address, the request that the network address corresponds to website is accessed to server initiation.Assuming that in the present embodiment, access request be to Request that server is initiated, accessing to the searching service of website W.

S602 obtains access business pair when the corresponding access business of access request belongs to the scope of business of flux experiment The flux experiment answered.

Access request is analyzed, obtains the corresponding access business of the access request, and by the access business with matched The type of service for setting flux experiment is matched, when in all types of service of configured flux experiment include the access business When, by access business with the subservice type of matched type of service match, obtain and the matched son of access business The initialization data of the corresponding each control experiment of type of service forms access business according to the initialization data of each control experiment Flux experiment.

Specifically, the type of service for obtaining configured flux experiment, by the searching service of website W and configured flux experiment Type of service and subservice type be compared, wrapped when in the type of service and/or subservice type of configured flux experiment When including the searching service of website W, the configuration information of the corresponding each control experiment of the searching service is obtained, according to each control experiment The flux experiment of configuration information composition access business.

S603 parses access request.

S604 obtains the user identifier and user property of the corresponding customer flow of access request according to parsing result.

When the corresponding access business of access request belongs to the scope of business of flux experiment, further access request is carried out Parsing, obtains the corresponding user identifier of the customer flow and user property.As shown in fig. 7, by received multiple visits It asks that request is parsed, obtains corresponding user identifier 10001~10008,20001~20008 and the customer flow is corresponding Gender attribute.

S605 carries out Hash operation to user identifier, obtains respectively according to the hash algorithm of flow layer each in flux experiment The corresponding cryptographic Hash of customer flow；The hash algorithm of any two flow layer meets orthogonal principle.

S606 obtains the value range in full flow section, obtains the interval value for indicating full flow section size.

Specifically, the value range for obtaining full flow section, after subtracting section minimum value by section maximum value and add 1 To the interval value of full flow section size.In this embodiment, interval value 100.

Cryptographic Hash is carried out modulus to interval value, obtains the corresponding flow value of customer flow by S607.

The cryptographic Hash obtained by different hash algorithms is subjected to modulus to interval value respectively, the numerical value that modulus is obtained is made It is the customer flow in the flow value of corresponding flow layer, customer flow is mapped in each flow layer.

As shown in fig. 7, including control experiment A, B, C flow layer in, using the hash algorithm that this layer is configured to Family mark carries out Hash operation and obtains cryptographic Hash, and cryptographic Hash is carried out modulus to 100, customer flow is obtained and maps to the flow layer Flow value.For example, obtaining the flow value 11 of the flow value 3,10002 of user identifier 10001 by Hash and modulo operation Deng.

S608 obtains the configuration information of each control experiment in each flow layer.

Referring to Fig. 7, configuration information includes flow subinterval, user property, user's white list and subscriber blacklist.Flow Section refers to full flow section is divided according to the customer flow accounting situation of control experiment after obtained multiple subintervals. Specifically, the configuration information of control experiment A includes: flow subinterval [1,10], user property " male ", user's white list (10001,10002,10003) and subscriber blacklist (10004,10005,10006)；The configuration information of control experiment B includes: Flow subinterval [1,10], user property " female ", user's white list (20001,20002,20003) and subscriber blacklist (20004,20005,20006)；The configuration information for testing C includes: to meet the configuration of control experiment A, B in full flow section Flow other than information.

S609 is matched flow value and user property with configuration information, determines matched experiment content.

With continued reference to Fig. 7, user 10001 (male) is recorded in user's white list of control experiment A, therefore hits control Test A；The flow value of user 10007 (female) belongs to flow subinterval [1,10], user property is " female " and is not recorded in pair It according to the facts tests in the blacklist of B, i.e., customer flow 10007 (female) is matched with the configuration information of control experiment B, determines customer flow 10007 (female) hit control experiment B；Customer flow 20008 (female) is unsatisfactory for the configuration information of control experiment A and control experiment B, Therefore, control experiment C is hit.

S610 obtains the corresponding experiment mark of experiment content and service parameter.

Experiment mark with service parameter is sent to the corresponding requesting terminal of customer flow by S611.

Specifically, matched experiment content is obtained and in addition to experiment content, other business phases of response access request Data are closed, and are assembled into service parameter, service parameter and experiment mark are back to the corresponding requesting terminal of the customer flow, So that requesting terminal carries out page presentation according to service parameter is received.For example, being directed to user 10001 (male), reality has been hit A is tested, then obtains experiment mark A service parameter corresponding with experiment mark A, and mark A industry corresponding with experiment mark A will be tested Business parameter is back to the corresponding requesting terminal of user 10001, and service parameter carries out page exhibition based on the received for the requesting terminal Show.

S612 receives requesting terminal and carries out the experiment mark and target component fed back after page presentation according to page parameter.

S613 identifies corresponding target component to each experiment respectively and counts when flux experiment reaches termination condition Analysis obtains each experiment and identifies corresponding evaluation index value；Evaluation index value is for determining experiment content to be released.

Wherein, termination condition refers to the condition for being used to indicate flux experiment failure.It can be the preconfigured end time, The quantity for being also possible to access request or customer flow reaches preset value.Evaluation index value refers to for carrying out to control experiment The numerical value of the evaluation index of recruitment evaluation.The evaluation index of same group of control experiment is identical, and the evaluation of difference group control experiment refers to Mark may be different.Specifically, evaluation index includes but is not limited to exposure rate, clicking rate, conversion ratio etc..

Specifically, corresponding evaluation index value operation method is identified according to each experiment, united to corresponding target component Meter analysis obtains each experiment and identifies corresponding evaluation index value.In general, evaluation index value more meets expectation development, then explanation should The corresponding control experiment of evaluation index value is more preferable compared to the effect in other control experiments in same group, can more meet expectation The experiment content of the control experiment of development is as experiment content to be released.

By carrying out customer flow distribution using user identifier and user property, it can support more various dimensions and more refine Experiment, customer flow matches corresponding experiment content according to the user property of itself, realizes and carries out for different user Property demand experiment, and improve instructing for the evaluation index value range of flow whether online or online to each experiment content Value.And make it possible to carry out the different control experiment of multiple groups simultaneously by the way that multilayer flow layer is arranged, same customer flow can Control experiment with different flow layer improves the utilization rate of customer flow and the scalability of flux experiment, and each layer The hash algorithm that configuration one meets orthogonality, guarantee between each layer control experiment group it is mutually indepedent, be independent of each other, it is ensured that it is real Test the accuracy of result.

Fig. 6 is the flow diagram for carrying out the method for assignment of traffic in one embodiment in flux experiment.It should be understood that Although each step in the flow chart of Fig. 6 successively shows that these steps are not certainty according to the instruction of arrow It is successively executed according to the sequence that arrow indicates.Unless expressly stating otherwise herein, there is no stringent for the execution of these steps Sequence limits, these steps can execute in other order.Moreover, at least part step in Fig. 6 may include multiple Perhaps these sub-steps of multiple stages or stage are not necessarily to execute completion in synchronization to sub-step, but can be Executed at the time of different, the execution in these sub-steps or stage sequence, which is also not necessarily, successively to be carried out, but can with it is other At least part of the sub-step or stage of step or other steps executes in turn or alternately.

In one embodiment, a kind of device that assignment of traffic is carried out in flux experiment is provided, as shown in figure 8, the device It include: to obtain module 802, hash module 804, determining module 806 and distribution module 808.Wherein:

Module 802 is obtained, for obtaining the corresponding user identifier of customer flow and user property.Receiving customer flow When, related information entrained by customer flow is analyzed as acquisition module 802, obtains the corresponding user identifier of customer flow And user property.Wherein, related information can be information entrained by access request, be also possible to requesting terminal and sending access The information reported simultaneously when request.

Hash module 804, for according to the hash algorithm of flow layer each in flux experiment, being breathed out respectively to user identifier Uncommon operation, obtains the corresponding cryptographic Hash of customer flow；The hash algorithm of any two flow layer meets orthogonal principle.

The settable multiple flow layers of quantity based on control experiment group, same customer flow can hit different flow simultaneously Control experiment in layer, namely so that limited customer flow is supported control experiment as much as possible in such a way that flow is multiplexed Group improves the utilization rate of customer flow and the scalability of flux experiment.In the present embodiment, by any two flow Layer setting meets the hash algorithm of orthogonal principle, to guarantee the orthogonality between each flow layer, make each flow layer to according to the facts It is relatively independent to test group, to reduce influence of the experiment between any two flows layer to experimental result.

Specifically, hash module 804 obtains the hash algorithm of each flow layer, by each hash algorithm to user identifier Hash operation is carried out, obtains cryptographic Hash, the corresponding cryptographic Hash quantity of each customer flow is identical as the number of plies of flow layer.

Determining module 806, for determining customer flow matched reality in each flow layer according to cryptographic Hash and user property Test content.Specifically, it according to cryptographic Hash and user property, is matched respectively with the control experiment of corresponding flow layer, when the use When family flow meets the matching condition of control experiment, the experiment mark of the matched control experiment of customer flow is determined, according to experiment Mark determines matched experiment content.

Distribution module 808, for distributing customer flow to matched experiment content.It is being determined that customer flow is matched Experiment content after, the customer flow is distributed to matched experiment content by distribution module 808, be based on matched experiment content The related data for feeding back to customer flow corresponding requests terminal is obtained, to be shown by requesting terminal.

The above-mentioned device that assignment of traffic is carried out in flux experiment, obtains the corresponding user identifier of customer flow and user belongs to Property, and respectively according to the hash algorithm of flow layer each in flux experiment, Hash operation is carried out to user identifier, obtains customer flow Corresponding cryptographic Hash, wherein the hash algorithm of any two flow layer meets orthogonal principle.Then belonged to according to cryptographic Hash and user Property, it determines customer flow matched experiment content in each flow layer, customer flow is distributed to matched experiment content.Pass through Consider user property when carrying out assignment of traffic, using user property carry out customer flow distribution, can support more various dimensions and The experiment more refined, customer flow match corresponding experiment content according to the user property of itself, realize for different use Family carries out individual demand experiment.

In one embodiment, user identifier and user property are carried in access request.Module 802 is obtained to be also used to parse Access request obtains the user identifier and user property of the corresponding customer flow of access request according to parsing result.

In one embodiment, determining module 806 includes: interval value determining module, modulus module and determining submodule.Its In:

Interval value determining module is obtained for obtaining the value range in full flow section for indicating that full flow section is big Small interval value.For example, configuration full flow section is [1, n] in a flux experiment.For indicating full flow section size Interval value can be subtracted section minimum value by section maximum value and be obtained after adding 1.It is appreciated that the value range in full flow section As [1, n], the interval value of full flow section size are (n-1)+1=n.

Modulus module obtains the corresponding flow value of customer flow for cryptographic Hash to be carried out modulus to interval value.In this reality Example in, respectively by the cryptographic Hash obtained by different hash algorithms to interval value progress modulus, the numerical value that modulus is obtained as The customer flow is realized in the flow value of corresponding flow layer and maps to customer flow in each flow layer.

Submodule is determined, for determining customer flow matched reality in each flow layer according to flow value and user property Test content.Specifically, according to the corresponding user property of customer flow and in the flow value of different flow layer, each flow layer is tested Content is matched, and determines the matched experiment content of institute.

In one embodiment, determine that submodule includes: that configuration information obtains module and matching module.Wherein, configuration information Module is obtained, for obtaining the configuration information of each control experiment in each flow layer；Matching module, for belonging to flow value and user Property, it is matched with configuration information, determines matched experiment content.

Specifically, configuration information obtains module and obtains the configuration information of each control experiment in each flow layer, and will acquire Configuration information is sent to matching module and is matched.Matching module is matched flow value and user property with configuration information, When flow value belongs to the corresponding flow subinterval of one of control experiment, and the user property of the customer flow and this to according to the facts Test configured user property it is identical when, it is determined that the customer flow hit the control experiment namely the customer flow life and should The experiment content of control experiment matches.

Further, configuration information may also include user's white list and/or subscriber blacklist.Matching module is also used to use Family mark, flow value and user property, are matched with configuration information, determine matched experiment content.

Specifically, matching module is used to judge whether user identifier to be the user that records in user's white list of control experiment Mark determines matched experiment content according to the experiment of the control experiment of hit mark if so, hitting the control experiment；With/ Or, judge whether user identifier is the user identifier that records in the subscriber blacklist of each control experiment, and by flow value and user Attribute is matched with the configuration information of each control experiment, when flow value belongs to corresponding flow of one of control experiment Section, user property are identical as the user property that the control experiment is configured, and when user identifier is not belonging to subscriber blacklist, then It determines that the customer flow hits the control experiment, matched experiment content is determined according to the experiment of the control experiment of hit mark.

In one embodiment, distribution module is also used to obtain the corresponding experiment mark of experiment content and service parameter；It will be real Standard inspection, which is known, is sent to the corresponding requesting terminal of customer flow with service parameter.Specifically, it obtains matched experiment content and removes Except experiment content, other service related datas of access request are responded, and are assembled into service parameter, by service parameter and reality Standard inspection knowledge is back to the corresponding requesting terminal of the customer flow, so that requesting terminal carries out page exhibition according to service parameter is received Show.

In one embodiment, a kind of device that assignment of traffic is carried out in flux experiment is provided, as shown in figure 9, the device Include: request receiving module 902, flux experiment module 904, obtain module 906, hash module 908, determining module 910 and divide With module 912.Wherein, request receiving module 902, for receiving the access request of requesting terminal transmission；Flux experiment module 904, for when the corresponding access business of access request belongs to the scope of business of flux experiment, obtaining the corresponding stream of access business Amount experiment.

Specifically, flux experiment module 904 obtains the corresponding access of the access request for analyzing access request Business, and the access business is matched with the type of service of configured flux experiment, when owning for configured flux experiment When including the access business in type of service, by access business with the subservice type of matched type of service match, The initialization data for obtaining each control experiment corresponding with the matched subservice type of the access business, according to each control experiment The flux experiment of initialization data composition access business.

In one embodiment, a kind of device carrying out assignment of traffic in flux experiment further includes feedback reception module, should Module is used to receive requesting terminal and carries out the experiment mark and target component fed back after page presentation according to service parameter.

It, can be actively anti-when exposing and clicking experiment content after requesting terminal carries out page presentation according to service parameter Feedback experiment mark and target component to flux experiment manage platform, and the feedback reception module reception in flux experiment management platform is asked The experiment mark and target component for asking terminal to feed back, with for statistical analysis according to experiment mark and target component, for each Control experiment carries out recruitment evaluation.

Further, which further includes statistical analysis module, right respectively for when flux experiment reaches termination condition Each corresponding target component of mark of testing is for statistical analysis, obtains each experiment and identifies corresponding evaluation index value；Evaluation index Value is for determining experiment content to be released.

Specifically, corresponding evaluation index value operation method is identified according to each experiment, united to corresponding target component Meter analysis obtains each experiment and identifies corresponding evaluation index value.In general, evaluation index value more meets expectation development, then explanation should The corresponding control experiment of evaluation index value is more preferable compared to the effect in other control experiments in same group.For example, for wide Accuse display location setting control experiment in, advertisement show position setting in the upper left corner the corresponding conversion ratio highest of control experiment, Indicate that the advertising results for obtaining advertising display in the page upper left corner are best, it therefore, can be by advertisement show position setting in the upper left corner It is determined as experiment content to be released.

It is above-mentioned in flux experiment carry out assignment of traffic device, when carrying out assignment of traffic simultaneously consider user identifier with User property carries out customer flow distribution using user identifier and user property, can support more various dimensions and more refine Experiment, customer flow match corresponding experiment content according to the user property of itself, realize and carry out individual character for different user The experiment of change demand.Also, make it possible to carry out the different control experiment of multiple groups, same user simultaneously by the way that multilayer flow layer is arranged Flow can match the control experiment of different flow layer, improve the utilization rate of customer flow and the scalability of flux experiment, And the hash algorithm that each layer of configuration one meets orthogonality, guarantee between each layer control experiment group it is mutually indepedent, be independent of each other, Ensure the accuracy of experimental result.

Figure 10 shows the internal structure chart of computer equipment in one embodiment.The computer equipment specifically can be figure The server 122 of flux experiment management platform 120 in 1.As shown in Figure 10, which includes being connected by system bus Processor, memory, network interface and the database connect.Wherein, the processor of the computer equipment is calculated and is controlled for providing Ability processed.The memory of the computer equipment includes non-volatile memory medium, built-in storage.The non-volatile memory medium is deposited Contain operating system, computer program and database.The built-in storage is operating system and meter in non-volatile memory medium The operation of calculation machine program provides environment.The database of the computer equipment is used to store the data of flux experiment management platform.It should The network interface of computer equipment is used to communicate with external terminal by network connection.The computer program is executed by processor When with realize it is a kind of in flux experiment carry out assignment of traffic method.

It will be understood by those skilled in the art that structure shown in Figure 10, only part relevant to application scheme The block diagram of structure, does not constitute the restriction for the computer equipment being applied thereon to application scheme, and specific computer is set Standby may include perhaps combining certain components or with different component layouts than more or fewer components as shown in the figure.

In one embodiment, the device provided by the present application that assignment of traffic is carried out in flux experiment can be implemented as one The form of kind computer program, computer program can be run in computer equipment as shown in Figure 10.Computer equipment is deposited Composition can be stored in reservoir, and this carries out each program module of the device of assignment of traffic in flux experiment, for example, shown in Fig. 8 Obtain module 802, hash module 804, determining module 806 and distribution module 808.The computer program that each program module is constituted So that processor executes the side for carrying out assignment of traffic in flux experiment of each embodiment of the application described in this specification Step in method.

In one embodiment, a kind of computer equipment, including memory and processor are provided, is stored in memory Computer program, the processor perform the steps of when executing computer program

Obtain the corresponding user identifier of customer flow and user property；

Respectively according to the hash algorithm of flow layer each in flux experiment, Hash operation is carried out to user identifier, obtains user The corresponding cryptographic Hash of flow；The hash algorithm of any two flow layer meets orthogonal principle；

According to cryptographic Hash and user property, customer flow matched experiment content in each flow layer is determined；

Customer flow is distributed to matched experiment content.

In one embodiment, it is also performed the steps of when processor executes computer program

The value range for obtaining full flow section, obtains the interval value for indicating full flow section size；

Cryptographic Hash is subjected to modulus to interval value, obtains the corresponding flow value of customer flow；

According to flow value and user property, customer flow matched experiment content in each flow layer is determined.

Obtain the configuration information of each control experiment in each flow layer；

It by flow value and user property, is matched with configuration information, determines matched experiment content.

Receive the access request that requesting terminal is sent；

When the corresponding access business of access request belongs to the scope of business of flux experiment, the corresponding stream of access business is obtained Amount experiment.

Parse access request；

The user identifier and user property of the corresponding customer flow of access request are obtained according to parsing result.

Obtain the corresponding experiment mark of experiment content and service parameter；

Experiment mark with service parameter is sent to the corresponding requesting terminal of customer flow.

It receives requesting terminal and carries out the experiment mark and target component fed back after page presentation according to service parameter.

It is for statistical analysis to the corresponding target component of each experiment mark respectively when flux experiment reaches termination condition, It obtains each experiment and identifies corresponding evaluation index value；Evaluation index value is for determining experiment content to be released.

In one embodiment, a kind of computer readable storage medium is provided, computer program is stored thereon with, is calculated Machine program performs the steps of when being executed by processor

Obtain the corresponding user identifier of customer flow and user property；

Customer flow is distributed to matched experiment content.

In one embodiment, it is also performed the steps of when computer program is executed by processor

Receive the access request that requesting terminal is sent；

Parse access request；

Those of ordinary skill in the art will appreciate that realizing all or part of the process in above-described embodiment method, being can be with Instruct relevant hardware to complete by computer program, program can be stored in a non-volatile computer storage can be read In medium, the program is when being executed, it may include such as the process of the embodiment of above-mentioned each method.Wherein, provided herein each To any reference of memory, storage, database or other media used in embodiment, may each comprise it is non-volatile and/ Or volatile memory.Nonvolatile memory may include read-only memory (ROM), programming ROM (PROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM) or flash memory.Volatile memory may include random access memory (RAM) or external cache.By way of illustration and not limitation, RAM is available in many forms, such as static state RAM (SRAM), dynamic ram (DRAM), synchronous dram (SDRAM), double data rate sdram (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronization link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) directly RAM (RDRAM), straight Connect memory bus dynamic ram (DRDRAM) and memory bus dynamic ram (RDRAM) etc..

Each technical characteristic of above embodiments can be combined arbitrarily, for simplicity of description, not to above-described embodiment In each technical characteristic it is all possible combination be all described, as long as however, the combination of these technical characteristics be not present lance Shield all should be considered as described in this specification.

Above embodiments only express the several embodiments of the application, and the description thereof is more specific and detailed, but can not Therefore it is interpreted as the limitation to the application the scope of the patents.It should be pointed out that for those of ordinary skill in the art, Without departing from the concept of this application, various modifications and improvements can be made, these belong to the protection model of the application It encloses.Therefore, the scope of protection shall be subject to the appended claims for the application patent.

Claims

1. a kind of method for carrying out assignment of traffic in flux experiment, which is characterized in that the described method includes:

Obtain the corresponding user identifier of customer flow and user property；

Respectively according to the hash algorithm of flow layer each in flux experiment, Hash operation is carried out to the user identifier, is obtained described The corresponding cryptographic Hash of customer flow；The hash algorithm of any two flow layer meets orthogonal principle；

According to the cryptographic Hash and the user property, the customer flow matched experiment content in each flow layer is determined；

The customer flow is distributed to the matched experiment content.

2. the method according to claim 1, wherein described according to the cryptographic Hash and the user property, really The fixed customer flow matched experiment content in each flow layer, comprising:

The cryptographic Hash is subjected to modulus to the interval value, obtains the corresponding flow value of the customer flow；

According to the flow value and the user property, the customer flow matched experiment content in each flow layer is determined.

3. according to the method described in claim 2, it is characterized in that, described according to the flow value and the user property, really The fixed customer flow matched experiment content in each flow layer, comprising:

It by the flow value and the user property, is matched with the configuration information, determines matched experiment content.

4. the method according to claim 1, wherein the corresponding user identifier of the acquisition customer flow and user Before attribute, further includes:

Receive the access request that requesting terminal is sent；

When the corresponding access business of the access request belongs to the scope of business of flux experiment, it is corresponding to obtain the access business Flux experiment.

5. according to the method described in claim 4, it is characterized in that, the corresponding user identifier of the acquisition customer flow and user Attribute, comprising:

Parse the access request；

The user identifier and user property of the corresponding customer flow of the access request are obtained according to parsing result.

6. the method according to claim 1, wherein described distribute the customer flow to the matched reality Test content, comprising:

Obtain the corresponding experiment mark of the experiment content and service parameter；

Experiment mark is sent to the corresponding requesting terminal of the customer flow with the service parameter.

7. according to the method described in claim 6, it is characterized in that, the method also includes:

It receives requesting terminal and carries out the experiment mark and target component fed back after page presentation according to the service parameter.

8. the method according to the description of claim 7 is characterized in that the method also includes:

When the flux experiment reaches termination condition, statistical is carried out to the corresponding target component of each experiment mark respectively Analysis obtains each experiment and identifies corresponding evaluation index value；The evaluation index value is for determining experiment content to be released.

9. a kind of device for carrying out assignment of traffic in flux experiment, which is characterized in that described device includes:

Hash module, for according to the hash algorithm of flow layer each in flux experiment, carrying out Hash to the user identifier respectively Operation obtains the corresponding cryptographic Hash of the customer flow；The hash algorithm of any two flow layer meets orthogonal principle；

Determining module, for according to the cryptographic Hash and the user property, determining the customer flow in each flow layer The experiment content matched；

10. device according to claim 9, which is characterized in that the determining module includes:

Interval value determining module is obtained for obtaining the value range in full flow section for indicating full flow section size Interval value；

Modulus module obtains the corresponding flow of the customer flow for the cryptographic Hash to be carried out modulus to the interval value Value；

Submodule is determined, for determining the customer flow in each flow layer according to the flow value and the user property Matched experiment content.

11. device according to claim 10, which is characterized in that the determining submodule includes:

Configuration information obtains module, for obtaining the configuration information of each control experiment in each flow layer；

Matching module, for being matched the flow value and the user property, being determined matched with the configuration information Experiment content.

12. device according to claim 9, which is characterized in that described device further include:

Request receiving module, for receiving the access request of requesting terminal transmission；

Flux experiment module, for obtaining when the corresponding access business of the access request belongs to the scope of business of flux experiment Take the corresponding flux experiment of the access business.

13. device according to claim 9, which is characterized in that described device further include:

Statistical analysis module, for identifying corresponding target to each experiment respectively when the flux experiment reaches termination condition Parameter is for statistical analysis, obtains each experiment and identifies corresponding evaluation index value；The evaluation index value is to be released for determining Experiment content.

14. a kind of computer equipment, including memory and processor, the memory are stored with computer program, feature exists In the step of processor realizes any one of claims 1 to 8 the method when executing the computer program.

15. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the computer program The step of method described in any item of the claim 1 to 8 is realized when being executed by processor.