CN108255472B - Layered experiment frame system - Google Patents
Layered experiment frame system Download PDFInfo
- Publication number
- CN108255472B CN108255472B CN201810045691.5A CN201810045691A CN108255472B CN 108255472 B CN108255472 B CN 108255472B CN 201810045691 A CN201810045691 A CN 201810045691A CN 108255472 B CN108255472 B CN 108255472B
- Authority
- CN
- China
- Prior art keywords
- request
- experiment
- algorithm
- processing method
- program processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/20—Software design
- G06F8/24—Object-oriented
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/30—Creation or generation of source code
- G06F8/31—Programming languages or programming paradigms
- G06F8/315—Object-oriented languages
Abstract
The invention discloses a layered experiment frame system, and relates to the field of data processing. The system comprises: the device comprises a distribution server, a first storage unit, a second storage unit, a packaging unit, a display entry unit and a judgment unit. The system is developed by adopting java language, has clear structural design, and has high readability and high maintainability of codes. The system is used for multi-layer experiments of a business system, and not only supports the request of a web end, but also supports the selection of the algorithm logic in the middle section of business operation. The system has a perfect data recovery scheme, can record all indexes needing to be counted in detail in a log mode, finally uses an offline data analysis platform for analysis, generates a data report and reveals the effect of specific indexes of each strategy or the operation effect of algorithm logic.
Description
Technical Field
The invention relates to the field of data processing, in particular to a layered experiment frame system.
Background
In most business systems at present, a plurality of implementation schemes are stored in some view pages directly facing users, such as a lower single-page landing page, and a decision maker can only decide one implementation scheme by means of perceptual experience under the condition of no data support. Similarly, in some cases, background algorithms invisible to the user, such as CTR prediction algorithms, are also in the same situation, and a decision maker can select one algorithm to be on-line only by virtue of perceptual experience.
The existing selection of a plurality of strategies has no proper solution, and most of the strategies are finally decided by decision makers according to the sensory experience. Therefore, with the rapid development of data processing technology, it is more desirable to make decisions more in line with the interests of the enterprise based on the quantitative comparison of data accumulation.
The prior art greedy release system, ABTestingGateWay, of the new wave, which is mainly used for gray release and developed at ngx-lua, provides help for making final decisions, and the main development language is scripting language lua. The ABTesinggateway supports four modes of ip segment splitting (iprange), uid user segment splitting (uidrange), uid mantissa splitting (uidusuffix) and specified special uid splitting (uidappoint). The ABTestingGateWay realizes shunting work according to runtime information runtimeInfo configured in the system; by setting runtimeInfo as different shunting strategies, dynamic updating of shunting strategies in operation is realized, and the purpose of dynamic scheduling is achieved. However, the existing gray scale publishing system ABTestingGateWay has the following defects: the system is mainly used for gray level release of the system, and has high performance because of being developed in a mode of Nginx + lua extension, but because of the reason, the system only processes distribution and forwarding of flow, only can process requests of a front end, and cannot be embedded into a logic layer; however, the lua script language is not very popular, and the system code composed of the lua script language is not strong in structure and readability. Based on the system theme, there is no complete data analysis process.
Disclosure of Invention
The invention aims to provide a layered experimental framework system, so as to solve the problems of poor universality, limited application range and no data recovery function of the conventional gray scale release system.
In order to achieve the above object, the layered experimental framework system of the present invention comprises:
the distribution server is responsible for pulling the experimental metadata related to the program processing method, verifying the data acquired from the program processing method, providing a shunting result according to the result of the program processing method, mapping experiments and collecting logs;
the system comprises a first storage unit, a second storage unit and a third storage unit, wherein the first storage unit stores a program processing method package for Client integration, and the program processing method package comprises a web-based Filter program processing method and a Client program processing method selected by an application algorithm;
a second storage unit storing data structures related to all experiments;
the packaging unit is used for packaging operation and tool classes aiming at redis and zookeeper;
the display input unit is used for providing an operation interface facing experiment operators and a report view displayed to a decision maker, and is also used for recording the configuration of experiment metadata;
the judging unit is used for judging whether the request needing to be tested is based on a request of a Web page end or based on a request of a selection algorithm logic of a business middle section, and if the request is based on the Web page end, a Web-based Filter program processing method is called; and if the request is based on the selection algorithm logic in the service middle section, calling a Client program processing method applied to algorithm selection.
Preferably, the web-based Filter program processing method is a process of acquiring a processing result based on data recovery, and specifically includes:
a1, intercepting an operation request sent by an operator from a Web page end, acquiring a user ID and request information which send the operation request, and taking the user ID and the request information as request parameters; meanwhile, asynchronously acquiring all experiment metadata related to the operation request information;
a2, performing matching verification on the acquired request parameters and all experimental metadata related to the request information, and if the matching verification is successful, sending the request parameters and the experimental metadata to a distribution server; if the matching verification is not successful, returning to A1;
a3, the distribution server calculates the experiment layer ID that the operation request needs to enter according to the received request parameters;
a4, the operation request needs the entered experimental layer ID to be sent to the display entry unit.
More preferably, a4 is followed by:
a5, an operator enters an experiment layer ID display interface at a Web page end according to an experiment layer ID displayed by a display entry unit, the layered experiment frame system sets a touch point on the experiment layer ID display interface, obtains an operation behavior generated on the experiment layer ID display interface through the touch point, sends a recording request carrying operation behavior data to a distribution server, and the distribution server receives and forms a log.
Preferably, the Client program processing method applied to algorithm selection is based on a process of acquiring a processing result by data recovery, and specifically includes:
b1, receiving a selection request for selecting which algorithm sent by a caller calling the algorithm; acquiring a distribution parameter of any algorithm m to be selected assembled by a caller, and performing a Client program processing method on experimental metadata in a cache or related to the algorithm m to be selected;
b2, carrying out matching verification on the acquired distribution parameters and the experiment metadata related to the algorithm m to be selected, and if the matching verification is successful, sending the distribution parameters and the experiment metadata of the algorithm to be selected, which are successfully matched and verified, to a distribution server; if the match verification is not successful, return to B1;
b3, the distribution server calculates the experiment layer ID required to enter the selection request according to the received distribution parameters, and feeds back the experiment layer ID to the caller.
More preferably, B3 is followed by:
and B4, acquiring a signal that the caller selects an algorithm corresponding to the experimental layer ID to complete the subsequent business operation according to the experimental layer ID, and sending the signal to the distribution server for day recording.
Preferably, the layered experiment framework system further comprises a flow verification unit:
the flow rate checking unit acquires a signal for completing data recovery from the distribution server, calculates the ratio of the flow rate used by the current data in the preset flow rate, sends a prompt to an operator or a caller if the ratio exceeds the preset ratio, and continues to judge if the ratio does not exceed the preset ratio.
The invention has the beneficial effects that:
the system is developed by adopting java language, has clear structural design, and has high readability and high maintainability of codes. The system is used for multi-layer experiments of a business system, and not only supports the request of a web end, but also supports the selection of the algorithm logic in the middle section of business operation.
Because the strategy is decided according to the data analysis result, the system has a perfect data recovery scheme, can record all indexes needing to be counted in detail in a log mode, finally uses an offline data analysis platform for analysis, generates a data report and reveals the effect of the specific index of each strategy or the operation effect of algorithm logic.
Drawings
FIG. 1 is a schematic structural diagram of a layered experimental framework system.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail below with reference to the accompanying drawings. It should be understood that the detailed description and specific examples, while indicating the invention, are intended for purposes of illustration only and are not intended to limit the scope of the invention.
The Croissant layered experimental framework system has the name that the source of the name of the Croissant is a Croissant favored by an author, and the author just eats the bread and drinks a coffee finished primary edition design, so the name is taken.
A Client, which is used for providing a package for Client integration, and currently provides two integration modes, one is a web Filter mode and the other is a Client mode.
The Croissant-core (core logic) is a core part of the whole system and is responsible for pulling of experiment metadata, data verification, a shunting algorithm, experiment mapping, log collection and the like.
Croissant-util (tool class), encapsulates operations on redis and zookeeper with some tool classes.
Croissant-model (data structure), data structure relevant for all experiments.
The system comprises a Croissant-web (management end), an operation interface facing experiment operators and a report view displayed to a decision maker, wherein the report view is used for recording the configuration of experiment metadata.
Examples
The Croissant flow process describes:
(1) the method comprises the steps that a Web page end request arrives at a server, a service end only needs to integrate a CroisantFilter in Web.xml and specify a current experiment layer Gate, the front end request of a user is intercepted through the filter, the front end request is obtained according to a user Id transmitted by the user, the IP of the request and the like, all experiment metadata stored in Redis are obtained asynchronously at the same time, the experiment metadata and request parameters are matched and verified, after the verification is finished, the data are transmitted to a distribution service, the distribution service calculates the experiment layer to which the current request needs to arrive according to the current parameters, and then the system automatically forwards the calculated result.
(2) And integrating the Croissantclient in the algorithm logic in the middle section of the service, wherein a caller is responsible for the assembly of distribution parameters, simultaneously acquiring metadata of an experiment from a cache after the parameter assembly is completed, matching and checking the metadata, transmitting the data to a distribution service after the verification is completed, calculating an experiment layer to which the current request needs to go by the distribution service according to the current parameters, then returning the ID of the experiment layer to the caller, and using the experiment ID by the caller to adopt a proper calling method (such as MQ and RMI) to complete the next service operation.
(3) After the web front end is finished, points are buried in a page, when a user operates on the page, a request is sent to a background by the rear end JS, the request carries behavior data of the user, and a log service of the background records the behavior data. In the algorithm logic, after the caller completes the business operation, the log service needs to be called back, and the log is recorded by Croissant.
(4) And when the data recovery is finished once, the ratio of the current data to the flow set by the experiment is checked for controlling the flow. And when the flow reaches the specified proportion, sending a notice to request a decision maker to pay attention to the collected report data.
The Croissant module is divided:
the Client-Client is used for providing a package for Client integration, and two integration modes are provided at present, one is a web Filter mode, and the other is a Client mode.
And ii, a Croissant-core, which is a core part of the whole system and is responsible for pulling of experiment metadata, data verification, a shunting algorithm, experiment mapping, log collection and the like.
Iii, Croissant-util, which encapsulates the operations and drink of redis and zookeeper.
Iv, Croissant-model, data structure relevant for all experiments.
And v, the Croissant-web, an operation interface facing experiment operators and a report view displayed to a decision maker, and the report view is used for recording the configuration of experiment metadata.
The Croissant system is developed by adopting java language, has clear structural design, and has high readability and high maintainability of codes. The system is mainly used for multi-layer experiments of a business system, can support the request of a web end and the algorithm logic in the middle section of business operation, and has a relatively perfect data recovery scheme because the strategy is finally decided by a data analysis result, and can record various indexes needing to be counted in a log form in detail, and finally use an offline data analysis platform for analysis to generate a data report and reveal the effect of the specific index of each strategy.
1. The method has the advantages that the flow distribution strategy is flexible, a user can use a designated user ID to carry out flow distribution and can also use other parameters to carry out flow distribution, the flow distribution algorithm uses Hash (FNV1Hash algorithm) mapping, and the flow is introduced into different experimental layers.
2. According to the large and complete data recovery scheme, the log records the user behaviors in detail, records key parameters, collects the user behaviors through the embedded point TracePoint, uploads log data to a data processing platform by using kafka, pulls statistical indexes and generates a multi-dimensional data report.
3. The convenient client integration scheme provides different access modes aiming at two types of pages and algorithms respectively, is simple and easy to use, is convenient and clear, and can complete integration only by a few lines of codes.
4. The good basic experiment data entry system and the multi-dimensional report display system provide a customer interface for operators, facilitate the operators to conveniently enter metadata of experiments, including the number of the experiments, the main entrance of each experiment, the shunt parameters and the statistical indexes of the experiments, and provide reports in a visual form after the data statistical analysis is finished.
The system provides data support for making final decisions, and can clearly distinguish which strategy is more suitable after quantitative comparison of data accumulation of each strategy, or which algorithm can improve the conversion rate, so that a decision maker can select a strategy or algorithm with more benefits to deploy.
By adopting the technical scheme disclosed by the invention, the following beneficial effects are obtained: the system of the invention provides a most convincing scheme on data based on the statistics of the data for a decision maker to select, and the scheme is directly related to enterprise interests. In a technical aspect, zookeeper, redis and other technologies used by the system can complete the whole experimental process under the condition of slightly influencing the performance of the service system.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and improvements can be made without departing from the principle of the present invention, and such modifications and improvements should also be considered within the scope of the present invention.
Claims (4)
1. A layered laboratory frame system, the system comprising:
the distribution server is responsible for pulling the experimental metadata related to the program processing method, verifying the data acquired from the program processing method, providing a shunting result according to the result of the program processing method, mapping experiments and collecting logs; the user uses the appointed user ID to shunt or uses other parameters to shunt, and the shunting algorithm uses Hash mapping to introduce the flow into different experimental layers;
the system comprises a first storage unit, a second storage unit and a third storage unit, wherein the first storage unit stores a program processing method package for Client integration, and the program processing method package comprises a web-based Filter program processing method and a Client program processing method selected by an application algorithm;
the web-based Filter program processing method is a process of acquiring a processing result based on data recovery, and specifically comprises the following steps:
a1, intercepting an operation request sent by an operator from a Web page end, acquiring a user ID and request information which send the operation request, and taking the user ID and the request information as request parameters; meanwhile, asynchronously acquiring all experiment metadata related to the operation request information;
a2, performing matching verification on the acquired request parameters and all experimental metadata related to the request information, and if the matching verification is successful, sending the request parameters and the experimental metadata to a distribution server; if the matching verification is not successful, returning to A1;
a3, the distribution server calculates the experiment layer ID that the operation request needs to enter according to the received request parameters;
a4, sending the experiment layer ID required to be entered by the operation request to a display entry unit;
a4 is followed by:
a5, an operator enters an experiment layer ID display interface at a Web page end according to an experiment layer ID displayed by a display entry unit, the layered experiment frame system sets a touch point on the experiment layer ID display interface, acquires an operation behavior generated on the experiment layer ID display interface through the touch point, sends a recording request carrying operation behavior data to a distribution server, and the distribution server receives and forms a log;
a second storage unit storing data structures related to all experiments;
the packaging unit is used for packaging operation and tool classes aiming at redis and zookeeper;
the display input unit is used for providing an operation interface facing experiment operators and a report view displayed to a decision maker, and is also used for recording the configuration of experiment metadata;
the judging unit is used for judging whether the request needing to be tested is based on a request of a Web page end or based on a request of a selection algorithm logic of a business middle section, and if the request is based on the Web page end, a Web-based Filter program processing method is called; and if the request is based on the selection algorithm logic in the service middle section, calling a Client program processing method applied to algorithm selection.
2. The layered experimental framework system of claim 1, wherein the Client program processing method applied to algorithm selection is based on a process of acquiring processing results by data recovery, and specifically comprises:
b1, receiving a selection request for selecting which algorithm sent by a caller calling the algorithm; acquiring a distribution parameter of any algorithm m to be selected assembled by a caller, and performing a Client program processing method on experimental metadata in a cache or related to the algorithm m to be selected;
b2, carrying out matching verification on the acquired distribution parameters and the experiment metadata related to the algorithm m to be selected, and if the matching verification is successful, sending the distribution parameters and the experiment metadata of the algorithm to be selected, which are successfully matched and verified, to a distribution server; if the match verification is not successful, return to B1;
b3, the distribution server calculates the experiment layer ID required to enter the selection request according to the received distribution parameters, and feeds back the experiment layer ID to the caller.
3. The layered experimental framework system of claim 2, further comprising after B3:
and B4, acquiring a signal that the caller selects an algorithm corresponding to the experimental layer ID to complete the subsequent business operation according to the experimental layer ID, and sending the signal to the distribution server for day recording.
4. The layered experiment framework system of claim 1, further comprising a flow verification unit:
the flow rate checking unit acquires a signal for completing data recovery from the distribution server, calculates the ratio of the flow rate used by the current data in the preset flow rate, sends a prompt to an operator or a caller if the ratio exceeds the preset ratio, and continues to judge if the ratio does not exceed the preset ratio.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810045691.5A CN108255472B (en) | 2018-01-17 | 2018-01-17 | Layered experiment frame system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810045691.5A CN108255472B (en) | 2018-01-17 | 2018-01-17 | Layered experiment frame system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108255472A CN108255472A (en) | 2018-07-06 |
| CN108255472B true CN108255472B (en) | 2021-06-01 |
Family
ID=62726713
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810045691.5A Active CN108255472B (en) | 2018-01-17 | 2018-01-17 | Layered experiment frame system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108255472B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113660331A (en) * | 2021-08-16 | 2021-11-16 | 杭州安恒信息技术股份有限公司 | Method for adding source information in data and related device |
| CN113965522B (en) * | 2021-11-11 | 2024-04-05 | 湖南快乐阳光互动娱乐传媒有限公司 | Behavior log grouping method, device, storage medium and equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103699678A (en) * | 2013-12-31 | 2014-04-02 | 苏州大学 | Hierarchical clustering method and system based on multistage layered sampling |
| CN104765617A (en) * | 2015-05-04 | 2015-07-08 | 北京奇虎科技有限公司 | Stream application function interface distributing method and system realized on basis of HTML5 |
| CN105357296A (en) * | 2015-10-30 | 2016-02-24 | 河海大学 | Elastic caching system based on Docker cloud platform |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9015672B2 (en) * | 2012-01-31 | 2015-04-21 | The United States Of America As Represented By The Secretary Of The Navy | Interface simulator for test rig in data distribution service |
-
2018
- 2018-01-17 CN CN201810045691.5A patent/CN108255472B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103699678A (en) * | 2013-12-31 | 2014-04-02 | 苏州大学 | Hierarchical clustering method and system based on multistage layered sampling |
| CN104765617A (en) * | 2015-05-04 | 2015-07-08 | 北京奇虎科技有限公司 | Stream application function interface distributing method and system realized on basis of HTML5 |
| CN105357296A (en) * | 2015-10-30 | 2016-02-24 | 河海大学 | Elastic caching system based on Docker cloud platform |
Non-Patent Citations (1)
| Title |
|---|
| 移动互联网程序的灰度发布;周智等;《电脑知识与技术》;20170502;第13卷(第9期);第47-49页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108255472A (en) | 2018-07-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105959318B (en) | Service processing method based on robot | |
| CN105117437B (en) | Method for assisting business handling administrative examination and approval based on artificial intelligence platform | |
| CN109711655B (en) | Dual-recording quality inspection method, device, equipment and computer readable storage medium | |
| CN107784593A (en) | Settlement of insurance claim interactive approach and system | |
| CN106407078B (en) | Client performance monitoring device and method based on information exchange | |
| CN109948815A (en) | Express delivery cabinet fault handling method, device, express delivery cabinet, manager and storage medium | |
| CN108255472B (en) | Layered experiment frame system | |
| CN109255623A (en) | A kind of business approval method, server, client and storage medium | |
| CN113051094A (en) | Supervision data submission testing method and device | |
| CN103580951B (en) | Output comparative approach, test migration householder method and the system of multiple information systems | |
| CN112799949A (en) | Model optimization method, device, system and equipment | |
| US20170244658A1 (en) | Processing system for multivariate segmentation of electronic message content | |
| CN110659270A (en) | Data processing and transmitting method and device | |
| CN109101964A (en) | Determine the method, equipment and storage medium in head and the tail region in multimedia file | |
| US9171292B1 (en) | Method and system for providing electronic delivery of regulated shareholder communications to account electronic mail addresses | |
| CN111222846B (en) | Method, device, computer equipment and storage medium for tracking reimbursement flow | |
| CN108600113B (en) | Preliminary auditing method and device for data to be issued and storage medium | |
| CN112950226A (en) | Industrial and commercial cluster registration method and device, electronic equipment and readable storage medium | |
| US7734496B1 (en) | Service provider and client survey method | |
| CN105743848B (en) | Multimedia content selective transmission method and device and remote control equipment | |
| US20130073338A1 (en) | Customer input application | |
| US11601384B2 (en) | Processing system performing dynamic training response output generation control | |
| CN110109842B (en) | Method and device for testing numerical system | |
| CN106487865A (en) | The method for uploading of record screen file and system | |
| KR20090114281A (en) | Questionaire survey method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |