CN113760725A - Data processing method, device and storage medium - Google Patents
Data processing method, device and storage medium Download PDFInfo
- Publication number
- CN113760725A CN113760725A CN202110061066.1A CN202110061066A CN113760725A CN 113760725 A CN113760725 A CN 113760725A CN 202110061066 A CN202110061066 A CN 202110061066A CN 113760725 A CN113760725 A CN 113760725A
- Authority
- CN
- China
- Prior art keywords
- experiment
- layer
- experimental
- service request
- external service
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The application discloses a data processing method, a data processing device and a storage medium. The method comprises the following steps: acquiring an external service request to be distributed; determining at least one experimental layer corresponding to the external service request to be distributed according to identification information carried by the external service request to be distributed; the identification information represents at least one service line corresponding to the external service request to be distributed; the service line corresponding to each experimental layer in the at least one experimental layer is different; for each experiment layer in the at least one experiment layer, determining an experiment object corresponding to the external service request to be shunted in each experiment of the at least one experiment contained in the corresponding experiment layer according to a shunting strategy corresponding to the corresponding experiment layer; the service type corresponding to each experiment in the at least one experiment is different; each experiment of the at least one experiment comprises at least two subjects for performing a comparative test.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a data processing method, an apparatus, and a storage medium.
Background
The AB Test (which may also be referred to as AB experiment, and may be expressed as a/B Test) is a method of performing a comparative experiment on at least two protocols, and for example, as shown in fig. 1, an optimal protocol may be determined from among protocol a, protocol B, and protocol C using the AB Test. The AB test can be widely applied to various Application scenarios, for example, when a new User Interface (UI) needs to be online in an Application program (APP, Application), for a newly designed UI a and UI B, the AB test can be used to online the UI a and UI B in a grayscale environment, and segment User traffic, that is, a certain proportion of users see the UI a, and the other users see the UI B, and then perform multidimensional statistical analysis on the User traffic access result corresponding to the UI a and data of User behavior, traffic benefit, and the like included in the User traffic access result corresponding to the UI B, so as to determine whether the online UI a is the UI B. For another example, when the algorithm needs to be updated, AB test can be performed on the new algorithm and the old algorithm within a certain time range to compare and analyze whether the new algorithm can replace the old algorithm. In addition, the AB test can also be used for application scenes such as visual design, page layout, file content design, recommendation algorithm, gray release and the like.
However, in the related art, when performing the AB test, the data processing method needs to be optimized.
Disclosure of Invention
In order to solve the related technical problems, embodiments of the present application provide a data processing method, an apparatus, and a storage medium.
The technical scheme of the embodiment of the application is realized as follows:
the embodiment of the application provides a data processing method, which comprises the following steps:
acquiring an external service request to be distributed;
determining at least one experimental layer corresponding to the external service request to be distributed according to identification information carried by the external service request to be distributed; the identification information represents at least one service line corresponding to the external service request to be distributed; the service line corresponding to each experimental layer in the at least one experimental layer is different;
for each experiment layer in the at least one experiment layer, determining an experiment object corresponding to the external service request to be shunted in each experiment of the at least one experiment contained in the corresponding experiment layer according to a shunting strategy corresponding to the corresponding experiment layer; the service type corresponding to each experiment in the at least one experiment is different; each experiment of the at least one experiment comprises at least two subjects for performing a comparative test.
In addition, according to at least one embodiment of the present application, the determining, according to identification information carried by the external service request to be distributed, at least one experimental layer corresponding to the external service request to be distributed includes:
and determining at least one experimental layer corresponding to the external service request to be distributed by utilizing a preset corresponding relation between the service line identifier and the experimental layer according to at least one service line identifier contained in the identifier information.
Further, in accordance with at least one embodiment of the present application, the method further comprises:
and aiming at each experimental layer in the at least one experimental layer, determining a shunting strategy corresponding to the corresponding experimental layer by utilizing a preset corresponding relation between the service line identification and the shunting strategy according to the service line identification corresponding to the corresponding experimental layer.
Furthermore, according to at least one embodiment of the present application, the determining, according to a splitting policy corresponding to a corresponding experimental layer, an experimental object corresponding to the external service request to be split in each experiment of at least one experiment included in the corresponding experimental layer includes:
for each experiment in at least one experiment contained in the corresponding experiment layer, determining the split ratio between at least two experimental objects contained in the corresponding experiment according to the split strategy corresponding to the corresponding experiment layer; and determining an experimental object corresponding to the external service request to be shunted in the corresponding experiment according to the determined shunting proportion.
Furthermore, according to at least one embodiment of the present application, the determining, according to the determined splitting ratio, an experimental object corresponding to the external service request to be split in the corresponding experiment includes:
and according to the determined distribution proportion, determining an experimental object corresponding to the external service request to be distributed in the corresponding experiment by utilizing a Murmur Hash3 algorithm.
Further, in accordance with at least one embodiment of the present application,
and determining an experiment object corresponding to the external service request to be shunted in the corresponding experiment aiming at each experiment in at least one experiment contained in the corresponding experiment layer in a role chain calling mode.
Further, in accordance with at least one embodiment of the present application, the method further comprises:
and displaying preset alarm information under the condition that the experimental layer corresponding to the external service request to be distributed is not successfully determined.
An embodiment of the present application further provides a data processing apparatus, including:
the acquisition unit is used for acquiring an external service request to be distributed;
the first processing unit is used for determining at least one experimental layer corresponding to the external service request to be distributed according to the identification information carried by the external service request to be distributed; the identification information represents at least one service line corresponding to the external service request to be distributed; the service line corresponding to each experimental layer in the at least one experimental layer is different;
the second processing unit is configured to determine, for each experiment layer in the at least one experiment layer, an experiment object corresponding to the external service request to be shunted in each experiment of the at least one experiment included in the corresponding experiment layer according to a shunting policy corresponding to the corresponding experiment layer; the service type corresponding to each experiment in the at least one experiment is different; each experiment of the at least one experiment comprises at least two subjects for performing a comparative test.
An embodiment of the present application further provides a data processing apparatus, including: a processor and a memory for storing a computer program capable of running on the processor;
wherein the processor is configured to perform the steps of any of the above methods when running the computer program.
An embodiment of the present application further provides a storage medium, where a computer program is stored, and when the computer program is executed by a processor, the steps of any one of the above methods are implemented.
The data processing method, the data processing device and the storage medium provided by the embodiment of the application acquire an external service request to be distributed; determining at least one experimental layer corresponding to the external service request to be distributed according to identification information carried by the external service request to be distributed; the identification information represents at least one service line corresponding to the external service request to be distributed; the service line corresponding to each experimental layer in the at least one experimental layer is different; for each experiment layer in the at least one experiment layer, determining an experiment object corresponding to the external service request to be shunted in each experiment of the at least one experiment contained in the corresponding experiment layer according to a shunting strategy corresponding to the corresponding experiment layer; the service type corresponding to each experiment in the at least one experiment is different; each experiment of the at least one experiment comprises at least two subjects for performing a comparative test. According to the scheme of the embodiment of the application, at least one experiment layer corresponding to the external service request to be shunted is determined, and the corresponding experiment object in each experiment of at least one experiment contained in the corresponding experiment layer of the external service request to be shunted is determined according to the corresponding shunting strategy for each experiment layer, so that the flow division of at least one service line and the flow division of at least one AB test on each service line can be simultaneously realized, multi-level and multi-angle AB test is realized, and the efficiency of the AB test is improved.
Drawings
FIG. 1 is a schematic diagram of an application scenario of an AB test;
FIG. 2 is a schematic flow chart illustrating a data processing method according to an embodiment of the present application;
FIG. 3 is a block diagram of a data processing system according to an embodiment of the present application;
FIG. 4 is a schematic diagram of an internal interaction of a data processing system according to an embodiment of the present application;
FIG. 5 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application;
fig. 6 is a schematic diagram of a hardware structure of a data processing apparatus according to an embodiment of the present application.
Detailed Description
The technical solution of the present application is further described in detail below with reference to the accompanying drawings and embodiments.
In the related art, an AB test module or an AB test system is usually developed for a specific service type, that is, the AB test module or the AB test system is seriously coupled with the corresponding service type and cannot be applied to the test of other service types by means of configuration, extension, and the like. Moreover, one AB test module or one AB test system usually supports only one service type at the same time, and the test efficiency is poor.
Based on this, in various embodiments of the present application, at least one experiment layer corresponding to an external service request to be distributed is determined, and for each experiment layer, an experiment object corresponding to the external service request to be distributed in each experiment of at least one experiment included in the corresponding experiment layer is determined according to a corresponding distribution strategy, so that traffic division of at least one service line and traffic division of at least one AB test on each service line can be simultaneously achieved, thereby achieving multi-level and multi-angle AB tests and improving efficiency of the AB test.
An embodiment of the present application provides a data processing method, as shown in fig. 2, the method includes:
step 201: acquiring an external service request to be distributed;
step 202: determining at least one experimental layer corresponding to the external service request to be distributed according to identification information carried by the external service request to be distributed;
here, the identification information represents at least one service line corresponding to the external service request to be distributed; the service line corresponding to each experimental layer in the at least one experimental layer is different;
step 203: for each experiment layer in the at least one experiment layer, determining an experiment object corresponding to the external service request to be shunted in each experiment of the at least one experiment contained in the corresponding experiment layer according to a shunting strategy corresponding to the corresponding experiment layer;
here, the service type corresponding to each experiment in the at least one experiment is different; each experiment of the at least one experiment comprises at least two subjects for performing a comparative test.
Here, it should be noted that, in various embodiments of the present application, an experiment refers to an AB test, and an experimental subject refers to one of at least two test protocols included in one AB test.
In step 201, in actual application, the manner of obtaining the external service request to be distributed may be set according to requirements, for example, a user with similar hobbies or other similar characteristics may be preset as a target user group, and the external service request to be distributed is obtained from the client of the target user through an interface provided by the client of the target user in the target user group. For another example, the service request of the user may be directed through an interface provided by the target web page to obtain the external service request to be distributed.
For step 202, in an embodiment, the determining, according to the identification information carried by the external service request to be distributed, at least one experimental layer corresponding to the external service request to be distributed may include:
and determining at least one experimental layer corresponding to the external service request to be distributed by utilizing a preset corresponding relation between the service line identifier and the experimental layer according to at least one service line identifier contained in the identifier information.
In practical application, the corresponding relationship between the service line identifier and the experimental layer may be set as required, and stored in a local database in any form (i.e., file type), so the method may further include: and acquiring the corresponding relation between the service line identification and the experimental layer from the local. Of course, the corresponding relationship between the service line identifier and the experimental layer may also be stored in the local cache according to the requirement.
In step 203, in actual application, before determining, according to a splitting policy corresponding to a corresponding experimental layer, an experimental object corresponding to the external service request to be split in each experiment of at least one experiment included in the corresponding experimental layer, a splitting policy corresponding to the corresponding experimental layer needs to be determined.
Based on this, in an embodiment, before determining, according to a offloading policy corresponding to a corresponding experimental layer, an experimental object corresponding to the external service request to be offloaded in each experiment of at least one experiment included in the corresponding experimental layer, the method may further include:
and aiming at each experimental layer in the at least one experimental layer, determining a shunting strategy corresponding to the corresponding experimental layer by utilizing a preset corresponding relation between the service line identification and the shunting strategy according to the service line identification corresponding to the corresponding experimental layer.
In practical application, the correspondence between the service line identifier and the offloading policy may be set according to a requirement, and stored in a local database in any form (i.e., file type), so the method may further include: and acquiring the corresponding relation between the service line identification and the shunting strategy from the local. Of course, the correspondence between the service line identifier and the offloading policy may also be stored in the local cache according to the requirement.
In step 203, the split strategy at least comprises a split ratio between at least two experimental objects contained in each experiment of at least one experiment contained in the corresponding experimental layer.
Based on this, in an embodiment, the determining, according to the offloading policy corresponding to the corresponding experimental layer, an experimental object corresponding to the external service request to be offloaded in each experiment of at least one experiment included in the corresponding experimental layer may include:
for each experiment in at least one experiment contained in the corresponding experiment layer, determining the split ratio between at least two experimental objects contained in the corresponding experiment according to the split strategy corresponding to the corresponding experiment layer; and determining an experimental object corresponding to the external service request to be shunted in the corresponding experiment according to the determined shunting proportion.
In practical application, the proportion information included in the offloading policy may be set according to a requirement, and the offloading policy may also be stored in a local database in any form (i.e., file type), so the method may further include: and acquiring the shunting strategy corresponding to the corresponding experimental layer from the local. Here, the local database may provide an external interface for a user to perform a check, modify and delete (CRUD) process on a correspondence between the service line identifier and the experimental layer, a correspondence between the service line identifier and the offloading policy, which are stored in the local database, through a client. Of course, the offloading policy may also be stored in a local cache according to the requirement.
In an embodiment, the determining, according to the determined splitting ratio, an experimental object corresponding to the external service request to be split in the corresponding experiment may include:
and according to the determined distribution proportion, determining an experimental object corresponding to the external service request to be distributed in the corresponding experiment by utilizing a Murmur Hash3 algorithm.
Here, the Murmur Hash3 algorithm is used to determine the experimental object corresponding to the external service request to be distributed in the corresponding experiment, so that the distribution efficiency can be improved. In practical application, other algorithms may be used for splitting according to requirements, such as Message Digest Algorithm (MD5, Message Digest Algorithm MD 5).
In an embodiment, when determining an experiment object corresponding to the external service request to be distributed in each experiment of the at least one experiment included in the corresponding experiment layer, the experiment object corresponding to the external service request to be distributed in the corresponding experiment may be determined for each experiment of the at least one experiment included in the corresponding experiment layer in a manner of chain of responsibility call.
Here, each experiment in at least one experiment included in the corresponding experiment layer is shunted in a role chain calling mode, pre-positioned and/or post-positioned enhancement processing can be performed according to needs, and expandability is strong.
In practical application, in order to improve user experience and avoid the situation that a user cannot know the current data processing state due to program errors or other reasons, alarm information can be presented to the user under the situation that the experimental layer corresponding to the external service request to be shunted is not successfully determined, so that the user can know the current data processing state.
Based on this, in an embodiment, the method may further include:
and displaying preset alarm information under the condition that the experimental layer corresponding to the external service request to be distributed is not successfully determined.
The data processing method provided by the embodiment of the application acquires an external service request to be distributed; determining at least one experimental layer corresponding to the external service request to be distributed according to identification information carried by the external service request to be distributed; the identification information represents at least one service line corresponding to the external service request to be distributed; the service line corresponding to each experimental layer in the at least one experimental layer is different; for each experiment layer in the at least one experiment layer, determining an experiment object corresponding to the external service request to be shunted in each experiment of the at least one experiment contained in the corresponding experiment layer according to a shunting strategy corresponding to the corresponding experiment layer; the service type corresponding to each experiment in the at least one experiment is different; each experiment of the at least one experiment comprises at least two experimental subjects for performing a comparative test; therefore, the traffic division of at least one service line and the traffic division of at least one AB test on each service line can be realized simultaneously, so that multi-level and multi-angle AB tests are realized, and the AB test efficiency is improved.
The present application will be described in further detail with reference to the following application examples.
The present application provides a data processing system, and the structure of the system is described below with reference to fig. 3.
As shown in fig. 3, the system includes four layers: an algorithm recommendation layer, an Application Programming Interface (API) layer, a business layer and a data layer.
The algorithm recommendation layer comprises an algorithm recommendation module 301 for acquiring the external service request to be distributed and sending the acquired external service request to be distributed to the API layerThat is, through the API included in the API layer, such as a Remote Procedure Call (RPC) API, a HyperText Transfer Protocol (HTTP) API, a Dubbo API, etc., the information parsing module 302 included in the Call service layer parses the external service request to be shunted; the information analysis module 302 may determine, according to identification information carried by the external service request to be distributed, at least one experimental layer corresponding to the external service request to be distributed from a correspondence between service line identifications stored in a Remote Dictionary service (REDIS) 304 included in a data layer and the experimental layer, and obtain a distribution policy corresponding to the corresponding experimental layer from the REDIS 304; sending the determined experimental layer information (specifically, the service line identifier) and the offloading policy to the offloading module 303; the offloading module 303 may determine, for each of the at least one experimental layer, an experimental object corresponding to the external service request to be offloaded in each experiment of the at least one experiment included in the corresponding experimental layer according to an offloading policy corresponding to the corresponding experimental layer; here, the splitting module 303 includes m experimental layers (m is an integer greater than 1), each experimental layer includes at least two splitting dyeing modules, and the number of the splitting dyeing modules included in each experimental layer may be the same or different (for example, the experimental layer 1 may include N)1The test layer m can contain NmIndividual split dyeing module, NmMay be greater than or equal to or less than N1) (ii) a For each experiment layer of the at least one experiment layer, the shunting module 303 calls each shunting and dyeing module in a role chain calling (english can be expressed as Handler-chains) mode, each shunting and dyeing module corresponds to one experiment (namely, one AB test) and is used for determining a shunting proportion between at least two experiment objects contained in the corresponding experiment according to a shunting strategy corresponding to the corresponding experiment layer, determining an experiment object corresponding to the external service request to be shunted in the corresponding experiment according to the determined shunting proportion, returning the determined experiment object to the algorithm recommending module 301, and the algorithm recommending module 301 diverts the corresponding external service request to be shunted to the corresponding experiment object corresponding to the corresponding experiment objectThe algorithm module of (1).
Here, the information parsing module 302, the splitting module 303, and the REDIS 304 are core modules of the system. In practical application, in order to improve system performance, the service layer may further include a logging module 305 and an authority control module 306; the log recording module 305 is used for recording the running log and daily problem troubleshooting information of the system and monitoring the running state of the system; the authority control module 306 is used for controlling basic authority of the system, such as API access authentication, security authentication, and the like.
In addition, for the m experiment layers included in the shunting module 303, parameters of experiments included in the same experiment layer cannot be coupled, and parameters of experiments included in different experiment layers can be coupled.
Fig. 4 is a schematic diagram of the internal interaction of the system, and as shown in fig. 4, the system may perform data processing including the following steps:
step 401: the algorithm recommending module 301 calls the information analyzing module 302 through the API.
Specifically, the information analysis module 302 may determine at least one experimental layer corresponding to the external service request to be distributed by reading a corresponding relationship between the service line identifier configured in the background (i.e., data stored in the REDIS 304) and the experimental layer, read a distribution policy corresponding to the corresponding experimental layer configured in the background (i.e., data stored in the REDIS 304), and send the determined experimental layer information and distribution policy to the distribution module 303; the shunting module 303 may perform splitting of the call chain mode according to a rule of same-layer traffic mutual exclusion and hierarchical traffic multiplexing, that is, for each experiment layer in the at least one experiment layer, the shunting module 303 calls each shunting dyeing module in a role chain calling manner, determines a shunting ratio between at least two experiment objects included in an experiment corresponding to the corresponding dyeing module according to a shunting strategy corresponding to the corresponding experiment layer, and determines an experiment object corresponding to the external service request to be shunted in the corresponding experiment according to the determined shunting ratio.
Here, the peer traffic mutual exclusion means: aiming at each experiment in at least one experiment contained in one experiment layer, one external service request to be shunted can only correspond to one experiment object; the layered traffic multiplexing means: an external service request to be shunted can correspond to at least one experimental object in each experimental layer.
Step 402: the information analysis module 302 returns the analysis result to the algorithm recommendation module 301.
Specifically, the splitting module 303 returns the splitting result of the multi-layer experiment corresponding to each external service request to be split (that is, at least one experiment object corresponding to each experiment layer of one external service request to be split) to the information analyzing module 302, and the information analyzing module 302 returns the splitting result to the algorithm recommending module 301.
Step 403: the algorithm recommending module 301 recommends an algorithm according to the analysis result returned by the information analyzing module 302.
Specifically, the algorithm recommending module 301 drains the external service request to be shunted to the algorithm module corresponding to the corresponding experimental object according to at least one experimental object corresponding to each experimental layer of the external service request to be shunted.
In this embodiment, the data processing system includes a hierarchical model, i.e., a offload module 303. In order to solve the problem that limited traffic cannot support multiple experiment requirements simultaneously and avoid the phenomenon of parameter coupling caused by analyzing multiple variables simultaneously by using the same experimental objects, the offloading module 303 follows the concept of traffic orthogonality and mutual exclusion, that is, in a plurality of experimental layers included in the offloading module 303, each experiment of each experimental layer has all traffic, all experimental objects in the same experiment share 100% of traffic (for example, one experiment includes 3 experimental objects, the first experimental object uses 20% of traffic, the second experimental object uses 50% of traffic, the third experimental object can only use 30% of traffic at most), the traffic between the experimental objects is mutually exclusive, that is, the traffic of the same layer is mutually exclusive (it can be understood that an external service request to be offloaded can only correspond to one experimental object in one experiment), and the traffic of hierarchical multiplexing (it can be understood that an external service request to be offloaded can correspond to at least one real external service request in each experimental layer) Test object). Therefore, under the conditions that more experiment flows are needed and mutual interference among experiments is not needed, a layered experiment can be selected, so that the same external service request enters a plurality of experiments in different layers.
In this embodiment, the data stored by the REDIS 304 is configurable. Specifically, when traffic is sent (i.e., when an external service request is obtained), each independent request (which may carry identification information, such as a service id, a device id, and the like) may be matched according to a corresponding experimental policy, where the experimental policy is read from a background configuration (i.e., data stored in the REDIS 304), the background configuration provides a background management page (english may be denoted as Dashboard), a user may perform CRUD processing on the data stored in the REDIS 304 through the background management page, and the data configured by the user may be actively refreshed and pushed to a REDIS cluster and a local cache of a server (i.e., a server of the data processing system).
In the embodiment of the application, the data processing system has the advantage of easy expansion. Specifically, the shunting module 303 adopts a design mode of responsibility chain calling, and can perform pre-and/or post-enhancement processing during service processing, read multi-layer traffic segmentation configuration (i.e., shunting policy) to perform chain calling, thereby facilitating dynamic expansion.
In the embodiment of the application, the data processing system has the advantage of high performance. Specifically, the system can provide two layers of caches, the REDIS 304 and the local cache, a database interface is not needed, time-consuming operation is avoided, and system performance is improved. In addition, the murmurr Hash3 algorithm can be used as a split-flow algorithm, so that processing based on a 32-bit (low-latency) 128-bit Hash Key (Key) can be realized, and particularly for large blocks of data, the method has high balance and low collision rate.
In the embodiment of the application, the data processing system has the advantage of being capable of crossing platforms. Particularly, the RPC API and HTTP API calling mode is provided, so that full-platform cross-language calling can be supported.
In addition, in practical application, when the system fails to read the background configuration (i.e. data in the REDIS 304 and the local cache is not successfully read), the return can be degraded, the non-shunting mode is automatically entered, and an alarm is triggered. Retrying to read the data in the following period; or, all or corresponding shunt tests can be closed by one key of the background, and the test switch can be dynamically configured, so that the user experience is further improved.
In order to implement the method according to the embodiment of the present application, an embodiment of the present application further provides a data processing apparatus, as shown in fig. 5, the apparatus includes:
an obtaining unit 501, configured to obtain an external service request to be distributed;
a first processing unit 502, configured to determine, according to identification information carried by the external service request to be distributed, at least one experiment layer corresponding to the external service request to be distributed; the identification information represents at least one service line corresponding to the external service request to be distributed; the service line corresponding to each experimental layer in the at least one experimental layer is different;
a second processing unit 503, configured to determine, for each experiment layer in the at least one experiment layer, an experiment object corresponding to the external service request to be shunted in each experiment of the at least one experiment included in the corresponding experiment layer according to a shunting policy corresponding to the corresponding experiment layer; the service type corresponding to each experiment in the at least one experiment is different; each experiment of the at least one experiment comprises at least two subjects for performing a comparative test.
In an embodiment, the first processing unit 502 is specifically configured to determine, according to at least one service line identifier included in the identification information, at least one experiment layer corresponding to the external service request to be distributed by using a preset correspondence between the service line identifier and the experiment layer.
In an embodiment, the second processing unit 503 is further configured to, for each experiment layer in the at least one experiment layer, determine, according to a service line identifier corresponding to the corresponding experiment layer, a shunting policy corresponding to the corresponding experiment layer by using a preset correspondence between the service line identifier and the shunting policy.
In an embodiment, the second processing unit 503 is specifically configured to, for each experiment in at least one experiment included in the corresponding experiment layer, determine a split ratio between at least two experimental objects included in the corresponding experiment according to a split strategy corresponding to the corresponding experiment layer; and determining an experimental object corresponding to the external service request to be shunted in the corresponding experiment according to the determined shunting proportion.
In an embodiment, the second processing unit 503 is specifically configured to determine, according to the determined splitting ratio, an experimental object corresponding to the external service request to be split in the corresponding experiment by using a murmurr Hash3 algorithm.
In an embodiment, the second processing unit 503 is specifically configured to determine, by means of a chain of responsibility call, for each experiment in at least one experiment included in the corresponding experiment layer, an experiment object corresponding to the external service request to be shunted in the corresponding experiment.
In an embodiment, the apparatus further includes an alarm unit, configured to present preset alarm information when the experimental layer corresponding to the external service request to be shunted is not successfully determined.
In practical application, the obtaining unit 501 and the alarm unit may be implemented by a processor in a data processing apparatus in combination with a communication interface; said first processing unit 502 and said second processing unit 503 may be implemented by a processor in a data processing device.
It should be noted that: the data processing apparatus provided in the above embodiment is only illustrated by the division of the program modules when processing data, and in practical applications, the processing may be distributed to different program modules according to needs, that is, the internal structure of the apparatus may be divided into different program modules to complete all or part of the processing described above. In addition, the data processing apparatus and the data processing method provided by the above embodiments belong to the same concept, and specific implementation processes thereof are described in the method embodiments for details, which are not described herein again.
Based on the hardware implementation of the program modules, and in order to implement the method according to the embodiment of the present application, an embodiment of the present application further provides a data processing apparatus, as shown in fig. 6, where the data processing apparatus 600 includes:
the communication interface 601 can perform information interaction with other electronic equipment;
the processor 602 is connected with the communication interface 601 to implement information interaction with other electronic devices, and is used for executing the method provided by one or more technical solutions when running a computer program;
a memory 603 for storing a computer program capable of running on the processor 602.
In particular, the processor 602 is configured to perform the following operations:
acquiring an external service request to be distributed through a communication interface 601;
determining at least one experimental layer corresponding to the external service request to be distributed according to identification information carried by the external service request to be distributed; the identification information represents at least one service line corresponding to the external service request to be distributed; the service line corresponding to each experimental layer in the at least one experimental layer is different;
for each experiment layer in the at least one experiment layer, determining an experiment object corresponding to the external service request to be shunted in each experiment of the at least one experiment contained in the corresponding experiment layer according to a shunting strategy corresponding to the corresponding experiment layer; the service type corresponding to each experiment in the at least one experiment is different; each experiment of the at least one experiment comprises at least two subjects for performing a comparative test.
In an embodiment, the processor 602 is further configured to perform the following operations:
and determining at least one experimental layer corresponding to the external service request to be distributed by utilizing a preset corresponding relation between the service line identifier and the experimental layer according to at least one service line identifier contained in the identifier information.
In an embodiment, the processor 602 is further configured to perform the following operations:
and aiming at each experimental layer in the at least one experimental layer, determining a shunting strategy corresponding to the corresponding experimental layer by utilizing a preset corresponding relation between the service line identification and the shunting strategy according to the service line identification corresponding to the corresponding experimental layer.
In an embodiment, the processor 602 is further configured to perform the following operations:
for each experiment in at least one experiment contained in the corresponding experiment layer, determining the split ratio between at least two experimental objects contained in the corresponding experiment according to the split strategy corresponding to the corresponding experiment layer; and determining an experimental object corresponding to the external service request to be shunted in the corresponding experiment according to the determined shunting proportion.
In an embodiment, the processor 602 is further configured to perform the following operations:
and according to the determined distribution proportion, determining an experimental object corresponding to the external service request to be distributed in the corresponding experiment by utilizing a Murmur Hash3 algorithm.
In an embodiment, the processor 602 is further configured to perform the following operations:
and determining an experiment object corresponding to the external service request to be shunted in the corresponding experiment aiming at each experiment in at least one experiment contained in the corresponding experiment layer in a role chain calling mode.
In an embodiment, the processor 602 is further configured to perform the following operations:
and displaying preset alarm information under the condition that the experimental layer corresponding to the external service request to be distributed is not successfully determined.
It should be noted that: the process of the processor 602 specifically executing the above operations is detailed in the method embodiment, and is not described here again.
Of course, in practice, the various components in the data processing device 600 are coupled together by a bus system 604. It is understood that the bus system 604 is used to enable communications among the components. The bus system 604 includes a power bus, a control bus, and a status signal bus in addition to a data bus. For clarity of illustration, however, the various buses are labeled as bus system 604 in fig. 6.
The memory 603 in the embodiments of the present application is used to store various types of data to support the operation of the data processing apparatus 600. Examples of such data include: any computer program for operating on the data processing apparatus 600.
The methods disclosed in the embodiments of the present application may be applied to the processor 602, or implemented by the processor 602. The processor 602 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware or instructions in the form of software in the processor 602. The Processor 602 may be a general purpose Processor, a Digital Signal Processor (DSP), or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like. The processor 602 may implement or perform the methods, steps, and logic blocks disclosed in the embodiments of the present application. A general purpose processor may be a microprocessor or any conventional processor or the like. The steps of the method disclosed in the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium located in the memory 603, and the processor 602 reads the information in the memory 603 and performs the steps of the aforementioned methods in conjunction with its hardware.
In an exemplary embodiment, the data processing Device 600 may be implemented by one or more Application Specific Integrated Circuits (ASICs), DSPs, Programmable Logic Devices (PLDs), Complex Programmable Logic Devices (CPLDs), Field Programmable Gate Arrays (FPGAs), general purpose processors, controllers, Micro Controllers (MCUs), microprocessors (microprocessors), or other electronic components for performing the foregoing methods.
It is to be appreciated that the memory 603 in the embodiments of the subject application can be either volatile memory or nonvolatile memory, and can include both volatile and nonvolatile memory. Among them, the nonvolatile Memory may be a Read Only Memory (ROM), a Programmable Read Only Memory (PROM), an Erasable Programmable Read-Only Memory (EPROM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), a magnetic random access Memory (FRAM), a Flash Memory (Flash Memory), a magnetic surface Memory, an optical disk, or a Compact Disc Read-Only Memory (CD-ROM); the magnetic surface storage may be disk storage or tape storage. Volatile Memory can be Random Access Memory (RAM), which acts as external cache Memory. By way of illustration and not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), Enhanced Synchronous Dynamic Random Access Memory (Enhanced DRAM), Synchronous Dynamic Random Access Memory (SLDRAM), Direct Memory (DRmb Access), and Random Access Memory (DRAM). The memories described in the embodiments of the present application are intended to comprise, without being limited to, these and any other suitable types of memory.
In an exemplary embodiment, the present application further provides a storage medium, i.e. a computer storage medium, in particular a computer readable storage medium, for example comprising a memory 603 storing a computer program, which is executable by a processor 602 of the data processing apparatus 600 to perform the steps of the foregoing method. The computer readable storage medium may be Memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface Memory, optical disk, or CD-ROM.
It should be noted that: "first," "second," and the like are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order.
The technical means described in the embodiments of the present application may be arbitrarily combined without conflict.
The above description is only a preferred embodiment of the present application, and is not intended to limit the scope of the present application.
Claims (10)
1. A data processing method, comprising:
acquiring an external service request to be distributed;
determining at least one experimental layer corresponding to the external service request to be distributed according to identification information carried by the external service request to be distributed; the identification information represents at least one service line corresponding to the external service request to be distributed; the service line corresponding to each experimental layer in the at least one experimental layer is different;
for each experiment layer in the at least one experiment layer, determining an experiment object corresponding to the external service request to be shunted in each experiment of the at least one experiment contained in the corresponding experiment layer according to a shunting strategy corresponding to the corresponding experiment layer; the service type corresponding to each experiment in the at least one experiment is different; each experiment of the at least one experiment comprises at least two subjects for performing a comparative test.
2. The method according to claim 1, wherein the determining, according to identification information carried by the external service request to be offloaded, at least one experimental layer corresponding to the external service request to be offloaded includes:
and determining at least one experimental layer corresponding to the external service request to be distributed by utilizing a preset corresponding relation between the service line identifier and the experimental layer according to at least one service line identifier contained in the identifier information.
3. The method of claim 1, further comprising:
and aiming at each experimental layer in the at least one experimental layer, determining a shunting strategy corresponding to the corresponding experimental layer by utilizing a preset corresponding relation between the service line identification and the shunting strategy according to the service line identification corresponding to the corresponding experimental layer.
4. The method according to claim 1, wherein the determining, according to a splitting policy corresponding to a corresponding experimental layer, an experimental object corresponding to the external service request to be split in each experiment of at least one experiment included in the corresponding experimental layer includes:
for each experiment in at least one experiment contained in the corresponding experiment layer, determining the split ratio between at least two experimental objects contained in the corresponding experiment according to the split strategy corresponding to the corresponding experiment layer; and determining an experimental object corresponding to the external service request to be shunted in the corresponding experiment according to the determined shunting proportion.
5. The method according to claim 4, wherein the determining, according to the determined splitting ratio, an experimental object corresponding to the external service request to be split in the corresponding experiment includes:
and according to the determined distribution proportion, determining an experimental object corresponding to the external service request to be distributed in the corresponding experiment by utilizing a Murmur Hash3 algorithm.
6. The method according to any one of claims 1 to 5,
and determining an experiment object corresponding to the external service request to be shunted in the corresponding experiment aiming at each experiment in at least one experiment contained in the corresponding experiment layer in a role chain calling mode.
7. The method according to any one of claims 1 to 5, further comprising:
and displaying preset alarm information under the condition that the experimental layer corresponding to the external service request to be distributed is not successfully determined.
8. A data processing apparatus, comprising:
the acquisition unit is used for acquiring an external service request to be distributed;
the first processing unit is used for determining at least one experimental layer corresponding to the external service request to be distributed according to the identification information carried by the external service request to be distributed; the identification information represents at least one service line corresponding to the external service request to be distributed; the service line corresponding to each experimental layer in the at least one experimental layer is different;
the second processing unit is configured to determine, for each experiment layer in the at least one experiment layer, an experiment object corresponding to the external service request to be shunted in each experiment of the at least one experiment included in the corresponding experiment layer according to a shunting policy corresponding to the corresponding experiment layer; the service type corresponding to each experiment in the at least one experiment is different; each experiment of the at least one experiment comprises at least two subjects for performing a comparative test.
9. A data processing apparatus, comprising: a processor and a memory for storing a computer program capable of running on the processor;
wherein the processor is adapted to perform the steps of the method of any one of claims 1 to 7 when running the computer program.
10. A storage medium storing a computer program, characterized in that the computer program realizes the steps of the method of any one of claims 1 to 7 when executed by a processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110061066.1A CN113760725A (en) | 2021-01-18 | 2021-01-18 | Data processing method, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110061066.1A CN113760725A (en) | 2021-01-18 | 2021-01-18 | Data processing method, device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113760725A true CN113760725A (en) | 2021-12-07 |
Family
ID=78786400
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110061066.1A Pending CN113760725A (en) | 2021-01-18 | 2021-01-18 | Data processing method, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113760725A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114760251A (en) * | 2022-04-13 | 2022-07-15 | 北京泰迪熊移动科技有限公司 | Data distribution method and device, electronic equipment and computer readable storage medium |
-
2021
- 2021-01-18 CN CN202110061066.1A patent/CN113760725A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114760251A (en) * | 2022-04-13 | 2022-07-15 | 北京泰迪熊移动科技有限公司 | Data distribution method and device, electronic equipment and computer readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8332952B2 (en) | Time window based canary solutions for browser security | |
| WO2021073156A1 (en) | Short link generation method, server, storage medium and computer device | |
| CN112015674A (en) | Cache resource access method and device based on multiple hierarchies and computer equipment | |
| CN113595788B (en) | API gateway management method and device based on plug-in | |
| DE112011104787T5 (en) | Use of content via personal clouds | |
| CN111770022B (en) | Capacity expansion method, system, equipment and computer storage medium based on link monitoring | |
| CN112651001A (en) | Access request authentication method, device, equipment and readable storage medium | |
| CN108021339B (en) | Method and device for reading and writing magnetic disk and computer readable storage medium | |
| CN107844542A (en) | A kind of distributed document storage method and device | |
| CN113760725A (en) | Data processing method, device and storage medium | |
| CN111291083A (en) | Webpage source code data processing method and device and computer equipment | |
| CN112835885A (en) | Processing method, device and system for distributed table storage | |
| CN111382179B (en) | Data processing method and device and electronic equipment | |
| CN112350997A (en) | Database access right control method and device, computer equipment and storage medium | |
| CN113553332B (en) | Decentralized data storage access method and device | |
| CN113076380B (en) | Data synchronization method, device, system, equipment and storage medium | |
| CN111708795B (en) | Object identification generation method, object identification updating device, computer equipment and medium | |
| CN112153103B (en) | Session management method, device, computer equipment and storage medium | |
| CN110493326B (en) | Zookeeper-based cluster configuration file management system and method | |
| CN111338848B (en) | Failure application copy processing method and device, computer equipment and storage medium | |
| CN113722225A (en) | Page testing method and device, computer equipment and storage medium | |
| CN116339767B (en) | Application resource allocation method, device, computer equipment and storage medium | |
| CN116305218B (en) | Data link tracking and data updating method, device and data management system | |
| CN115242874B (en) | Network request agent optimization method, system, equipment and storage medium in application development stage | |
| CN110287265B (en) | Login request processing method and device, server and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |