CN117033183A

CN117033183A - Test method, test device, electronic equipment and computer readable medium

Info

Publication number: CN117033183A
Application number: CN202310908586.0A
Authority: CN
Inventors: 鲁俊杉; 刘昊骋
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2023-07-21
Filing date: 2023-07-21
Publication date: 2023-11-10

Abstract

The disclosure provides a testing method, a testing device, an electronic device and a computer readable medium, and relates to the field of data processing, in particular to the field of artificial intelligence and model testing. The specific implementation scheme is as follows: obtaining a test sample corresponding to a first data processing node, wherein the first data processing node is the last data processing node of a data processing link formed by a plurality of data processing nodes according to a preset data processing sequence; determining a test sample corresponding to a data processing node of the data processing link according to the test sample corresponding to the first data processing node and the data conversion rate corresponding to the data processing node of the data processing link; testing at least one data processing model according to a test sample corresponding to a data processing node of the data processing link, and obtaining a model index of the data processing model; the data processing model corresponds to one of the data processing nodes in the data processing link.

Description

Test method, test device, electronic equipment and computer readable medium

Technical Field

The present disclosure relates to the field of data processing technologies, and in particular, to the technical fields of artificial intelligence, model testing, and the like. In particular, the disclosure relates to a test method, an apparatus, an electronic device, and a computer readable medium.

Background

With the development of internet technology, more and more information can be recommended to clients via an information recommendation system.

Along with the development of artificial intelligence and other technologies, an artificial intelligence technology can be used for establishing a recommendation model, and the model is used for determining the clients to which the information should be recommended from a plurality of clients, so that the information recommendation efficiency is improved.

Disclosure of Invention

The present disclosure provides a test method, apparatus, electronic device, computer readable medium.

According to a first aspect of the present disclosure, there is provided a test method comprising:

obtaining a test sample corresponding to a first data processing node, wherein the first data processing node is the last data processing node of a data processing link formed by a plurality of data processing nodes according to a preset data processing sequence;

determining a test sample corresponding to a data processing node of the data processing link according to the test sample corresponding to the first data processing node and the data conversion rate corresponding to the data processing node of the data processing link;

testing at least one data processing model according to a test sample corresponding to a data processing node of the data processing link, and obtaining a model index of the data processing model; the data processing model corresponds to one of the data processing nodes in the data processing link.

According to a second aspect of the present disclosure, there is provided a test apparatus comprising:

the node module is used for acquiring a test sample corresponding to a first data processing node, wherein the first data processing node is the last data processing node of a data processing link formed by a plurality of data processing nodes according to a preset data processing sequence;

the sample module is used for determining the test sample corresponding to the data processing node of the data processing link according to the test sample corresponding to the first data processing node and the data conversion rate corresponding to the data processing node of the data processing link;

the test module is used for testing at least one data processing model according to the test sample corresponding to the data processing node of the data processing link, and obtaining the model index of the data processing model; the data processing model corresponds to one of the data processing nodes in the data processing link.

According to a third aspect of the present disclosure, there is provided an electronic device comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the test method.

According to a fourth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the above-described test method.

According to a fifth aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the above-described test method.

It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.

Drawings

The drawings are for a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:

FIG. 1 is a flow chart of a test method provided by an embodiment of the present disclosure;

FIG. 2 is a schematic diagram of data processing link composition in the marketing field in one specific implementation of a testing method provided by embodiments of the present disclosure;

FIG. 3 is a flow chart illustrating some steps of another testing method provided by an embodiment of the present disclosure;

FIG. 4 is a flow chart of some steps of another testing method provided by an embodiment of the present disclosure;

FIG. 5 is a flow chart of some steps of another testing method provided by an embodiment of the present disclosure;

FIG. 6 is a flow chart of some steps of another testing method provided by an embodiment of the present disclosure;

FIG. 7 is a schematic diagram of the composition of a data processing link of one embodiment of another test method provided by an embodiment of the present disclosure;

FIG. 8 is a schematic structural view of a testing device according to an embodiment of the present disclosure;

fig. 9 is a block diagram of an electronic device for implementing a test method of an embodiment of the present disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

In some related technologies, AUC (the area enclosed by the coordinate axis under the working characteristic curve of the subject) and KS (the difference between cumulative distributions between good and bad samples) are used as model effect test indexes, i.e., offline modeling indexes, in the process of establishing a recommended model.

The online application service uses ROI (return on investment) as a service index, wherein the ROI is calculated by (number of effective marketing clients) v (((number of ineffective marketing clients + number of effective marketing clients)) v × marketing cost.

And performing online application service by using a model with high offline modeling index, wherein the obtained service index ROI is very low, i.e. the offline modeling index is inconsistent with the online service index.

The embodiment of the disclosure provides a testing method and device, electronic equipment and a computer readable storage medium, which aim to solve at least one of the technical problems in the prior art.

The test method provided by the embodiment of the disclosure may be performed by an electronic device such as a terminal device or a server, where the terminal device may be a vehicle-mounted device, a user device (UserEquipment, UE), a mobile device, a user terminal, a cellular phone, a cordless phone, a Personal digital assistant (Personal DigitalAssistant, PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, or the like, and the method may be implemented by a processor invoking computer readable program instructions stored in a memory. Alternatively, the method may be performed by a server.

Fig. 1 shows a flow chart of a test method provided by an embodiment of the present disclosure, and as shown in fig. 1, the test method provided by the embodiment of the present disclosure may include step S110, step S120, and step S130.

In step S110, a test sample corresponding to a first data processing node is obtained;

in step S120, determining a test sample corresponding to a data processing node of the data processing link according to the test sample corresponding to the first data processing node and a data conversion rate corresponding to the data processing node of the data processing link;

in step S130, at least one data processing model is tested according to a test sample corresponding to a data processing node of the data processing link, and a model index of the data processing model is obtained.

For example, the first data processing node is the last data processing node of a data processing link composed of a plurality of data processing nodes according to a predetermined data processing sequence.

In some specific implementations, the testing method provided by the embodiments of the present disclosure may be applied to the marketing field.

Fig. 2 shows a schematic diagram of data processing link composition in the marketing field in a specific implementation manner, and as shown in fig. 2, data processing nodes such as delivery (e.g., sending marketing information to customers after preliminary screening), touching (e.g., smoothly sending marketing information to customers), clicking (e.g., clicking on marketing information by customers), downloading/registering (e.g., registering by customers on websites corresponding to marketing information), entering/finishing (e.g., filling relevant information by customers on websites corresponding to marketing information), passing credit (e.g., checking and passing relevant information of customers) and the like sequentially form a data processing link.

The first data processing node is the last data processing node of the data processing link, i.e. the trusted pass.

The clients which are not put after preliminary screening belong to non-marketing clients, and the non-putting, non-clicking, non-registering, non-finished goods and credit non-passing clients corresponding to the input/finished goods data processing nodes belong to invalid marketing clients, and the authorized passing clients (including credit clients and non-using clients) belong to valid marketing clients.

In some possible implementations, in step S110, the test samples corresponding to the first data processing node may be obtained by sampling from OOT samples (out-of-time samples, i.e., samples that are not used in the model development and training process) corresponding to the first data processing node.

In some possible implementations, each of the OOT samples corresponding to the data processing node may include data related to a service corresponding to the data processing node, such as identity data, behavior data, and the like of a client corresponding to the data processing node.

In some possible implementations, the first data processing node includes a plurality of data processing levels, and the OOT samples corresponding to the first data processing node are composed of OOT samples corresponding to the plurality of data processing levels.

In some possible implementations, the number of test samples of each data processing level corresponding to the first data processing node may be determined according to a ratio of the number of test samples corresponding to a certain data processing level and the historical traffic data number of each data processing level of the first data processing node.

In some possible implementations, the test samples corresponding to the first data processing node may be sampled from the OOT samples corresponding to the first data processing node according to the number of test samples corresponding to the first data processing node.

The embodiments of the present disclosure are not limited to the method of sampling, and any method that may implement sampling is within the scope of the embodiments of the present disclosure.

In some possible implementations, in step S120, the data conversion rate corresponding to the data processing node of the data processing link may be used to characterize the client conversion rate of the current data processing node during the actual business process.

The data conversion rate of the data processing node can be determined by the ratio of the number of clients of the next data processing node to the number of all clients entering the current data processing node, which can be obtained by analyzing the historical service data generated in the actual service process, when the service corresponding to the current data processing node is completed in the actual service process.

In some possible implementations, determining the test sample corresponding to the data processing node of the data processing link according to the test sample corresponding to the first data processing node and the data conversion rate corresponding to the data processing node of the data processing link may be determining the test sample corresponding to the second-to-last data processing node according to the test sample corresponding to the first data processing node and the data conversion rate corresponding to the second-to-last data processing node of the data processing link; determining a test sample corresponding to a third-last data processing node according to the test sample corresponding to the second-last data processing node and the data conversion rate corresponding to the third-last data processing node of the data processing link; and so on until test samples corresponding to all the data processing nodes of the data processing link are obtained.

In some possible implementations, in step S130, the number of data processing models may be one or more, each data processing model corresponding to one data processing node of the data processing link, which may be used to predict whether a client of the data processing node that inputs the data processing model will complete the corresponding data processing according to the data processing link, becoming a client of another data processing node.

As shown in fig. 2, the data processing model a corresponds to a touch data processing node, and is used for predicting whether a client corresponding to the touch data processing node will complete clicking, downloading/registering, and entering/finishing a business, and becomes a client corresponding to the entering/finishing data processing node. The data processing model B is corresponding to the feed/completion data processing node and is used for predicting whether a client corresponding to the feed/completion data processing node can complete the credit passing service or not and becomes a client corresponding to the credit passing data processing node.

In some possible implementations, the output of the data processing model may be a score representing a probability that a client of the data processing node that inputs the data processing model completes a corresponding data process according to the data processing link, and becomes a client of another data processing node, the higher the score output by the data processing model, the higher the probability that the client of the input data processing model completes a corresponding data process according to the data processing link, and becomes a client of another data processing node.

In some possible implementations, the testing of the at least one data processing model according to the test sample corresponding to the data processing node of the data processing link may be performed, and the obtaining of the model index of the data processing model may be inputting the test sample of the data processing node corresponding to the data processing model into the data processing model, determining a model precipitation sample from the test sample according to the output of the data processing model, and determining the model index of the data processing model according to the model precipitation sample.

In some possible implementations, the number of data processing models corresponding to the data processing link is a plurality of, and determining the model index of the data processing model according to the model precipitation samples may be determining a model precipitation sample corresponding to each data processing model according to an output of each data processing model, determining a final precipitation sample according to all the model precipitation samples, and determining the model index of all the data processing models according to the final precipitation sample.

In some possible implementations, the model index of the data processing model may be the ROI calculated from the final precipitated samples.

In the actual business process, the client of the first data processing node must go through the complete data processing flow of the data processing link, and the client of a certain data processing node of the data processing link must go through the data processing node before the data processing link.

As in some specific implementations, the trusted clients must be clients that experience drops, touches, clicks, downloads/registrations, and entry/completion data processing nodes; the client of the in/out part must also be the client that experiences the drop, reach, click, download/register data processing node.

Similarly, in the actual business process, the number of clients corresponding to the data processing nodes of the data processing link is continuously reduced along with the position of the data processing nodes in the data processing link, and the data processing nodes are in a funnel shape.

In the actual business process, the data volume corresponding to each data processing node of the data processing link is huge. On one hand, with the increase of data used for modeling and testing, the requirement on the performance of a computer increases exponentially, on the other hand, in consideration of business reasons such as business confidentiality, all data generated in actual business cannot be used in the modeling and testing process of a data processing model, all data generated in the actual business are sampled, and part of data is extracted to perform modeling and testing of the business model.

However, the integrity of the data is destroyed by the sampling behavior, and the client obtained by node sampling is trusted, possibly because the client is not sampled by other data processing nodes and cannot appear in the sampled data corresponding to the input, touch, click, download/registration, and entry/completion data processing nodes; the fact that the client sampled by the delivery node does not appear in the sampled data corresponding to the touch-up node may be due to the fact that the client is not sampled, rather than the client not completing the delivery. That is, sampling results in the data corresponding to each data processing node being split, the data processing nodes' data cannot form a data funnel, which is not true.

In the test method provided by the embodiment of the disclosure, the test samples corresponding to other data processing nodes of the data processing link are obtained according to the test sample corresponding to the last data processing node of the data processing link and the data conversion rate corresponding to the data processing node of the data processing link, so that the test samples corresponding to the data processing nodes can form a data funnel, and the test samples more consistent with the real situation are obtained; and testing the data processing model through the constructed test sample which is more consistent with the real situation so as to improve the consistency degree of the acquired model index and the on-line service index, and further improve the performance of the service model acquired by training by using the acquired model index.

The test method provided by the embodiment of the present disclosure is specifically described below.

As described above, in some possible implementations, the first data processing node includes a plurality of data processing levels.

In some specific implementations, the first data processing node includes an a-class, a B-class, and a C-class, where the a-class may be a higher-class client (e.g., a higher credit-class may be credited with a higher credit line), the B-class may be a lower-class client than the a-class, a credit-class may be credited with a lower credit line than the a-class client, the C-class may be a lower-class, a credit-class may be less than the B-class, and a credit-class may be credited with a lower credit line than the B-class client.

FIG. 3 is a flow diagram of one particular implementation of obtaining test samples corresponding to a first data processing node where the first data processing node includes a plurality of data processing levels. As shown in fig. 3, acquiring the test sample corresponding to the first data processing node may include step S310 and step S320.

In step S310, determining the number of test samples corresponding to the plurality of data processing levels according to the sample number ratio and the number of test samples corresponding to the first data processing level in the plurality of data processing levels;

in step S320, test samples corresponding to the plurality of data processing levels are sampled from the training unused samples corresponding to the first data processing node according to the number of test samples corresponding to the plurality of data processing levels.

In some possible implementations, in step S310, the sample number ratio is determined according to the number ratio of the historical service data corresponding to the plurality of data processing levels in the historical service data corresponding to the first data processing node.

In some specific implementations, the sample number ratio may be a ratio of the number of historical traffic data corresponding to each data processing level.

In some possible implementations, the first data processing level may be any one of a plurality of data processing levels corresponding to the first data processing node, for example, in a case where the first data processing node includes an a level, a B level, and a C level, the first data processing level may be an a level, a B level, or a C level.

In some specific implementations, the first data processing level may be an a level, where the number of test samples corresponding to the first data processing level is 5, and the ratio of the number of samples is 1:2:3, that is, the number of service data corresponding to the a level: the number of service data corresponding to class B: and the number of the service data corresponding to the class c=1:2:3, and if the number of the test samples corresponding to the class a is 5, the number of the test samples corresponding to the class B is 10, and the number of the test samples corresponding to the class C is 15.

In some possible implementations, in step S320, sampling the test samples corresponding to the plurality of data processing levels from the OOT samples corresponding to the first data processing node according to the number of test samples corresponding to the plurality of data processing levels may be extracting a corresponding number of test samples from the OOT samples corresponding to each data processing level according to the number of test samples corresponding to each data processing level.

In some specific implementations, in the case that the number of test samples corresponding to the B-level is 10 and the number of test samples corresponding to the C-level is 15, which are calculated in step S310, 10 samples are extracted from the OOT samples corresponding to the B-level as the test samples corresponding to the B-level, and 15 samples are extracted from the OOT samples corresponding to the C-level as the test samples corresponding to the C-level.

The method for sampling test samples from OOT samples is not limited in the embodiments of the present disclosure, and any method that can implement sampling is within the scope of the embodiments of the present disclosure.

The test samples corresponding to the data processing grades are determined through the sample number ratio, so that the ratio of the number of the test samples corresponding to the data processing grades in the obtained test samples is consistent with the number of the service data corresponding to the data processing grades in the real service data, the test samples are enabled to be as close to the real service data as possible, and the consistency degree of the obtained model indexes and the on-line service indexes is further improved.

FIG. 4 illustrates a flow diagram of one particular implementation of determining test samples corresponding to data processing nodes of a data processing link based on test samples corresponding to a first data processing node and data conversion rates corresponding to data processing nodes of the data processing link, as shown in FIG. 4, may include step S410.

In step S410, according to the test sample corresponding to the n+1th data processing node of the data processing link and the data conversion rate corresponding to the N data processing node of the data processing link, determining the test sample corresponding to the N data processing node of the data processing link;

where N is an integer greater than 0 and less than the number of data processing nodes of the data processing link.

In some possible implementations, the data conversion rate corresponding to the nth data processing node of the data processing link may be determined according to a ratio of the number of historical service data corresponding to the nth data processing node and the number of historical service data corresponding to the n+1th data processing node.

The historical service data corresponding to the data processing node may be service data generated in a real service process. For example, the historical service data corresponding to the authorized passing node can be the data of the authorized passing client in the real service process.

In some possible implementations, the test samples corresponding to the data processing nodes may include test positive samples and test positive samples. Determining, according to the test sample corresponding to the n+1th data processing node of the data processing link and the data conversion rate corresponding to the N data processing node of the data processing link, the test sample corresponding to the N data processing node of the data processing link may include:

Taking a test sample corresponding to an (n+1) th data processing node of the data processing link as a test positive sample corresponding to an (N) th data processing node of the data processing link;

determining the number of test negative samples corresponding to the N data processing nodes of the data processing link according to the number of test samples corresponding to the (n+1) th data processing node of the data processing link and the data conversion rate corresponding to the N data processing node of the data processing link;

and sampling and acquiring the test negative samples corresponding to the Nth data processing node of the data processing link from the training unused samples corresponding to the Nth data processing node of the data processing link according to the number of the test negative samples corresponding to the Nth data processing node of the data processing link.

For the nth data processing node, the data corresponding to the successfully converted client (i.e., the client completes the corresponding data processing and becomes the client of the n+1th data processing node) is the target of the data processing node, and the data corresponding to the successfully converted client can be taken as a positive sample of the data processing node, while the data corresponding to the unsuccessfully converted client can be taken as a negative sample of the data processing node.

The n+1th data processing node is the N data processing node, so the test sample corresponding to the n+1th data processing node is necessarily the test positive sample corresponding to the N data processing node.

The data conversion rate corresponding to the nth data processing node is a ratio of the number of historical service data corresponding to the nth data processing node to the number of historical service data corresponding to the n+1th data processing node, and the number of historical service data corresponding to the nth data processing node can be regarded as the number of all samples corresponding to the nth data processing node, and the number of historical service data corresponding to the n+1th data processing node is the number of positive samples corresponding to the nth data processing node.

In order to maintain consistency of the test and real business processes, the data conversion rate corresponding to the nth data processing node can be regarded as a ratio of the number of test samples of the nth data processing node to the number of test positive samples of the nth data processing node.

Under the condition that the test sample corresponding to the (n+1) th data processing node is taken as the test positive sample corresponding to the (N) th data processing node, the number of the test negative samples corresponding to the (N) th data processing node can be calculated according to the data conversion rate (namely the ratio of the number of the test samples of the (N) th data processing node to the number of the test positive samples of the (N) th data processing node).

After the number of the test negative samples corresponding to the nth data processing node is determined, a corresponding number of negative samples can be extracted from the OOT samples corresponding to the nth data processing node according to the number of the test negative samples corresponding to the nth data processing node to serve as the test negative samples. The extracting a corresponding number of negative samples may be extracting a corresponding number of samples from the OOT samples corresponding to the nth data processing node that are not successfully converted.

By taking the test sample corresponding to the (n+1) th data processing node of the data processing link as the test positive sample corresponding to the (N) th data processing node of the data processing link, and using the data conversion rate corresponding to the (N) th data processing node to obtain the test negative sample corresponding to the (N) th data processing node, a data funnel consistent with the conversion rate in the real business process can be constructed, so that the test sample is as close to the real business data as possible, and the consistency degree of the obtained model index and the on-line business index is further improved.

In some possible implementations, assuming that the data processing link has M data processing nodes in total, the first data processing node is an mth data processing node of the data processing link, and the process of constructing a test sample corresponding to each data processing node of the entire data processing link may be: firstly, determining a test sample corresponding to an M-1 data processing node according to the test sample corresponding to a first data processing node and the data conversion rate corresponding to the M-1 data processing node; determining test samples corresponding to the M-2 data processing nodes according to the test samples corresponding to the M-1 data processing nodes and the data conversion rate corresponding to the M-2 data processing nodes; and so on, directly determining the test sample corresponding to the 1 st data processing node.

FIG. 5 is a flow chart of one particular implementation of testing a data processing model based on test samples corresponding to data processing nodes to obtain model metrics corresponding to the data processing model. As shown in fig. 5, step S510, step S520 may be included.

In step S510, a test sample of a data processing node corresponding to the data processing model is input into the data processing model, and a model precipitation sample is determined from the test sample of the data processing node corresponding to the data processing model according to the output of the data processing model;

in step S520, model indices of the data processing model are determined from the model precipitation samples.

In some possible implementations, the output of the data processing model may be a score representing a probability that a client corresponding to a test sample of the input data processing model completes a corresponding data process according to the data processing link and becomes a client of another data processing node, and the higher the score output by the data processing model, the higher the probability that a client corresponding to a test sample of the input data processing model completes a corresponding data process according to the data processing link and becomes a client of another data processing node.

In some possible implementations, in step S510, determining, from the test samples of the data processing nodes corresponding to the data processing model, the model-precipitated samples according to the output of the data processing model may be to sort the test samples of the data processing nodes according to the score of the output of the data processing model after the test samples are input into the data processing model, and determine, from the sorting result, the model-precipitated samples, e.g., determine, as the model-precipitated samples, the test samples of the first 20% of the sorting result.

In some possible implementations, in step S520, in the case where the number of data processing models is one, determining the model index of the data processing model according to the model precipitation sample may be calculating the ROI according to the model precipitation sample, and taking the ROI as the model index of the data processing model.

In some possible implementations, the clients corresponding to the intersection of the test samples corresponding to the first data processing node may be used as effective marketing clients according to the model precipitation samples, the clients corresponding to other test samples in the model precipitation samples may be used as ineffective marketing clients, and the ROI may be calculated according to the set return price and marketing cost.

In some possible implementations, the first data processing node includes a plurality of data processing levels, and the ROI may be calculated from a customer-corresponding return price for each data processing level.

As in some particular implementations, each level a customer can win 1800, each level B customer can win 700, each level C customer can win 250, the level a customer corresponds to a return price of 1800, the level B customer corresponds to a return price of 700, and the level C customer corresponds to a return price of 250.

In some possible implementations, the number of data processing models corresponding to the data processing links is multiple. The determining the model index of the data processing model according to the model precipitation sample may be determining a final precipitation sample according to the model precipitation sample corresponding to the plurality of data processing models, and determining the model index of the plurality of data processing models according to the final precipitation sample.

In some possible implementations, the final precipitated samples may be determined according to model precipitated samples corresponding to all data processing models, and model indexes of all data processing models may be determined according to the final precipitated samples.

In some possible implementations, determining the final extraction sample from the model extraction samples corresponding to all data processing models may be determining the final extraction sample from an intersection of the model extraction samples corresponding to all data processing models.

That is, the final sample is a test sample that all data processing models consider that the client corresponding to the test sample can complete the corresponding data processing according to the data processing link, and thus, the final sample can be considered as a test sample with the highest probability of completing the data processing corresponding to all data processing nodes of the data processing link.

In the real business process, the final precipitated sample may be data corresponding to the client having the highest probability of becoming authorized to pass through the data processing node, that is, data corresponding to the client having the highest probability of successful marketing.

Therefore, calculating the ROI according to the final precipitated sample is equivalent to calculating the ROI according to the data corresponding to the customer who has succeeded in marketing in the real business process, that is, the calculation method of the offline modeling index is consistent with the calculation method of the online business index, and the consistency of the calculation method also improves the consistency degree of the acquired model index and the online business index, thereby improving the performance of the data processing model acquired by using the acquired model index to guide training.

In some possible implementations, the test samples corresponding to the data processing nodes may be grouped through the output of the data processing model, the groupings corresponding to different data processing models are combined, the ROIs are calculated according to the different combinations, and the highest ROI is used as the model index of the data processing model.

FIG. 6 is a flow chart of a specific implementation of grouping test samples corresponding to data processing nodes through output of data processing models, combining the groups corresponding to different data processing models, calculating ROIs according to the different combinations, and using the highest ROI as a model index of the data processing model, where as shown in FIG. 6, steps S610 and S620 may be included.

In step S610, according to the output of the data processing model, test samples corresponding to the data processing nodes of the data processing model are grouped, and a plurality of groups of model precipitation samples corresponding to the data processing model are obtained;

in step S620, a plurality of candidate model indexes of the data processing model are determined from the plurality of sets of model precipitation samples, and model indexes of the data processing model are determined from the plurality of candidate model indexes.

In some possible implementations, in step S610, the grouping of the test samples corresponding to the data processing nodes of the data processing model may be based on the score output by the data processing model, with the test samples being grouped in 0.1 steps.

That is, the corresponding test samples with a score of 0.9-1.0 are in one group, the corresponding test samples with a score of 0.8-0.9 are in one group, the corresponding test samples with a score of 0.7-0.8 are in one group, the corresponding test samples with a score of 0.6-0.7 are in one group, the corresponding test samples with a score of 0.5-0.6 are in one group, the corresponding test samples with a score of 0.4-0.5 are in one group, the corresponding test samples with a score of 0.3-0.4 are in one group, the corresponding test samples with a score of 0.2-0.3 are in one group, the corresponding test samples with a score of 0.1-0.2 are in one group, and the corresponding test samples with a score of 0-0.1 are in one group.

If there are multiple data processing models, the grouping rules of the test samples of the data processing nodes of the multiple data processing models may or may not be consistent.

In some possible implementations, in step S620, in a case where the data processing model corresponding to the data processing link is one, determining, from the plurality of sets of model precipitation samples, a plurality of candidate model indexes of the data processing model may be to calculate, as the candidate model indexes, ROIs with each grouping result as a final precipitation sample, respectively.

In some possible implementations, the data processing links correspond to a plurality of data processing models; determining a plurality of candidate model indexes of the data processing model according to the plurality of groups of model precipitation samples may be to respectively select a group of model precipitation samples from the plurality of groups of model precipitation samples corresponding to the plurality of data processing models to combine, so as to obtain a plurality of model precipitation sample combinations; and determining one candidate model index corresponding to all the data processing models according to the intersection set of each group of model precipitation samples in the model precipitation sample combination.

In some possible implementation manners, a group of model precipitation samples are selected from a plurality of groups of model precipitation samples corresponding to each data processing model respectively to be combined, so that a plurality of model precipitation sample combinations are obtained; and determining a candidate model index corresponding to all the data processing models according to the intersection of each group of model precipitation samples in the model precipitation sample combination.

And selecting a group of model precipitation samples from a plurality of groups of model precipitation samples corresponding to each data processing model for combination, wherein the obtaining of the plurality of model precipitation sample combinations can be to select a group of model precipitation samples from a plurality of groups of model precipitation samples corresponding to each data processing model for combination according to the sequence.

If the number of the data processing models is 2, namely, a data processing model a and a data processing model B, the data processing model a corresponds to 2 groups of model precipitation samples, namely, a model precipitation sample a and a model precipitation sample B, and the data processing model B corresponds to 2 groups of model precipitation samples, namely, a model precipitation sample C and a model precipitation sample D, the obtained plurality of model precipitation sample combinations may include a combination of the model precipitation sample a and the model precipitation sample C, a combination of the model precipitation sample a and the model precipitation sample D, a combination of the model precipitation sample B and the model precipitation sample C, and a combination of the model precipitation sample B and the model precipitation sample D.

For each model precipitation sample combination, determining one candidate model index corresponding to all the data processing models according to the intersection of each group of model precipitation samples in the model precipitation sample combination may be determining a final precipitation sample according to the intersection of each group of model precipitation samples in the model precipitation sample combination, and calculating an ROI as the candidate model index according to the final precipitation sample. The calculation of the ROI from the final precipitated samples is as described above and will not be described in detail here.

The optimal mode of determining the model precipitation sample according to the output of the data processing model can be determined in a grouping mode, namely, the best ROI can be obtained only by determining the model precipitation sample according to the mode, so that the performance of the data processing model is improved.

In some possible implementations, the test method provided by the embodiments of the present disclosure may be repeatedly executed to test the model index of the data processing model, and determine the model index of the data processing model by taking an average value through multiple tests, so as to reduce the influence of sampling contingency on the acquired model index.

The test method provided by the embodiment of the present disclosure is described in one specific embodiment.

Fig. 7 shows a schematic diagram of a data processing node of a data processing link in a specific embodiment, where, as shown in fig. 7, the data processing link includes 7 data processing nodes to be served, send a short message successfully, click a short message, register, log in, finish, and send a message through, where a short message sending success rate is 80%, that is, a data conversion rate of the data processing node to be served is 80%, a probability of a client click short message is 0.3%, that is, a data conversion rate of the data processing node to send a short message successfully is 0.3%, a registration success rate and a log in success rate are both 50%, that is, a data conversion rate of the click short message data processing node and a data conversion rate of the registered data processing node are both 50%, a finish rate and a message sending rate are respectively 33% and 60%, and a data conversion rate of the log in data processing node and the finish data processing node are respectively 33% and 60%.

The trusted passing data processing node comprises A, B, C three data processing grades, and the ratio of the number of samples is 1:2: and 3, under the condition that the number of test samples corresponding to the data processing grade A is 1, the number of test samples corresponding to the data processing grade B is 2, the number of test samples corresponding to the data processing grade C is 3, the number of test samples passing through the data processing node in a trusted manner is 6, and the test samples are obtained by extracting the corresponding number of samples from the OOT samples corresponding to the three data processing grades A, B, C.

And taking the positive samples corresponding to the trusted passing data processing nodes as the test positive samples of the finished data processing nodes, wherein the number of the test positive samples of the finished data processing nodes is 6, the data conversion rate of the finished data processing nodes is 60%, the number of the test negative samples corresponding to the finished data processing nodes is 4, and extracting the corresponding number of samples from the corresponding OOT samples to obtain the test negative samples.

And taking positive samples corresponding to the completion data processing nodes as test positive samples of the login data processing nodes, wherein the number of the test positive samples of the login data processing nodes is 10, the data conversion rate of the login data processing nodes is 33%, the number of the test negative samples corresponding to the login data processing nodes is 20, and extracting samples with corresponding numbers from the corresponding OOT samples to obtain the test negative samples.

And taking positive samples corresponding to the login data processing nodes as test positive samples of the login data processing nodes, wherein the number of the test positive samples of the login data processing nodes is 30, the data conversion rate of the login data processing nodes is 50%, the number of the test negative samples corresponding to the login data processing nodes is 60, and extracting samples with the corresponding number from the corresponding OOT samples to obtain the test negative samples.

And taking the positive samples corresponding to the registered data processing nodes as test positive samples of the successful data processing nodes for sending the short messages, wherein the number of the test positive samples of the successful data processing nodes for sending the short messages is 120, the data conversion rate of the successful data processing nodes for sending the short messages is 0.3%, the number of the test negative samples corresponding to the successful data processing nodes for sending the short messages is 39880, and the test negative samples are obtained by extracting the samples with the corresponding number from the corresponding OOT samples.

And taking the positive samples corresponding to the data processing nodes which send the short messages successfully as the test positive samples of the data processing nodes to be put in, wherein the number of the test positive samples of the data processing nodes to be put in is 40000, the data conversion rate of the data processing nodes to be put in is 50%, the number of the test negative samples corresponding to the data processing nodes to be put in is 10000, and extracting the samples with the corresponding number from the corresponding OOT samples to obtain the test negative samples.

After the data processing node test sample is built, testing the data processing model according to the test sample corresponding to the data processing node.

As shown in fig. 7, the data processing link includes 2 data processing models, namely an intent model and a qualification model, where the intent model corresponds to a data processing node to be put in, the qualification model corresponds to a registered data processing node, and the testing of the data processing model according to a test sample corresponding to the data processing node may be to input the test sample corresponding to the data processing node to the intent model, so as to obtain an output of the intent model; inputting the registered corresponding test sample into a qualification model to obtain the output of the qualification model; determining a model precipitation sample of the intent model from test samples corresponding to the data processing nodes to be put in according to the output of the intent model, and determining a model precipitation sample of the qualification model from test samples corresponding to the registration service nodes according to the output of the qualification model; and determining a final precipitation sample according to the intersection of the model precipitation sample of the intention model and the model precipitation sample of the qualification model, and calculating the ROI according to the final precipitation sample.

And determining a model precipitation sample of the intent model from the test samples corresponding to the data processing nodes to be put in according to the output of the intent model by using 4 different modes, and determining a model precipitation sample of the qualification model from the test samples corresponding to the registration service nodes according to the output of the qualification model.

The 1 st is a random determination, i.e. a random acquisition of model-precipitated samples from test samples.

The 2 nd is to take a test sample corresponding to the 10% of the highest output score of the intention model as a model precipitation sample of the intention model; and taking a test sample corresponding to 50% of the highest output score of the qualification model as a model precipitation sample of the qualification model.

The 3 rd is to group the test samples corresponding to the data processing nodes to be put in according to the output scores of the intention models, and determine the group with the highest ROI as the model precipitation sample of the intention model and the model precipitation sample of the qualification model according to the result of the ROI corresponding to the model precipitation sample of the qualification model of each group; and taking a test sample corresponding to 50% of the highest output score of the qualification model.

And 4. Respectively grouping and combining the test samples corresponding to the data processing nodes to be put in and the test samples corresponding to the registered data processing nodes according to the output scores of the intention model and the qualification model, and determining the combination with the highest ROI as a model precipitation sample of the intention model and a model precipitation sample of the qualification model according to the result of the ROI corresponding to each combination.

Through multiple experiments, the model index obtained by the 4 th method is the most accurate, and the accuracy is increased along with the increase of the number of test samples corresponding to the trusted passing data processing nodes.

Based on the same principle as the method shown in fig. 1, fig. 8 shows a schematic structural diagram of a test device provided by an embodiment of the present disclosure, and as shown in fig. 8, the test device 80 may include:

the node module 810 is configured to obtain a test sample corresponding to a first data processing node, where the first data processing node is a last data processing node of a data processing link formed by a plurality of data processing nodes according to a preset data processing sequence;

the sample module 820 is configured to determine a test sample corresponding to a data processing node of the data processing link according to the test sample corresponding to the first data processing node and a data conversion rate corresponding to the data processing node of the data processing link;

the test module 830 is configured to test at least one data processing model according to a test sample corresponding to a data processing node of the data processing link, and obtain a model index of the data processing model; the data processing model corresponds to one of the data processing nodes in the data processing link.

In the test device provided by the embodiment of the disclosure, test samples corresponding to other data processing nodes of the data processing link are obtained according to the test sample corresponding to the last data processing node of the data processing link and the data conversion rate corresponding to the data processing node of the data processing link, so that the test samples corresponding to the data processing nodes can form a data funnel, and the test samples more consistent with the real situation are obtained; and testing the data processing model through the constructed test sample which is more consistent with the real situation so as to improve the consistency degree of the acquired model index and the on-line service index, and further improve the performance of the data processing model acquired by training by using the acquired model index.

In some possible implementations, the sample module 820 is to: determining a test sample corresponding to an N data processing node of the data processing link according to the test sample corresponding to an n+1 data processing node of the data processing link and the data conversion rate corresponding to the N data processing node of the data processing link; where N is an integer greater than 0 and less than the number of data processing nodes of the data processing link.

In some possible implementations, the test samples corresponding to the nth data processing node of the data processing link include a test positive sample and a test negative sample; the sample module 820 includes: the positive sample unit is used for taking a test sample corresponding to an (n+1) th data processing node of the data processing link as a test positive sample corresponding to an (N) th data processing node of the data processing link; the negative sample unit is used for determining the number of the test negative samples corresponding to the N data processing nodes of the data processing link according to the number of the test samples corresponding to the (n+1) th data processing nodes of the data processing link and the data conversion rate corresponding to the N data processing nodes of the data processing link; and sampling and acquiring the test negative samples corresponding to the Nth data processing node of the data processing link from the training unused samples corresponding to the Nth data processing node of the data processing link according to the number of the test negative samples corresponding to the Nth data processing node of the data processing link.

In some possible implementations, the first data processing node includes a plurality of data processing levels; the node module 810 includes: the number calculating unit is used for determining the number of the test samples corresponding to the plurality of data processing levels according to the sample number ratio and the number of the test samples corresponding to the first data processing level in the plurality of data processing levels; the sample number ratio is determined according to the number ratio of the historical service data corresponding to a plurality of data processing grades in the historical service data corresponding to the first data processing node; and the sampling unit is used for sampling and acquiring the test samples corresponding to the data processing grades from the training unused samples corresponding to the first data processing node according to the number of the test samples corresponding to the data processing grades.

In some possible implementations, the test module 830 includes: the extraction unit is used for inputting the test sample of the data processing node corresponding to the data processing model into the data processing model, and determining a model extraction sample from the test sample of the data processing node corresponding to the data processing model according to the output of the data processing model; and the calculation unit is used for determining model indexes of the data processing model according to the model precipitation samples.

In some possible implementations, the data processing links correspond to a plurality of data processing models; the calculation unit includes: a first intersection calculating subunit, configured to determine a final precipitated sample according to an intersection of model precipitated samples corresponding to the plurality of data processing models; and the second intersection calculation subunit is used for determining model indexes of the plurality of data processing models according to the intersection of the final precipitated sample and the test sample corresponding to the first data processing node.

In some possible implementations, the extraction unit includes: the grouping subunit is used for grouping the test samples of the data processing nodes corresponding to the data processing model according to the output of the data processing model, and obtaining a plurality of groups of model precipitation samples corresponding to the data processing model; the calculation unit includes: and the screening subunit is used for determining a plurality of candidate model indexes of the data processing model according to the plurality of groups of model precipitation samples, and determining the model indexes of the data processing model from the plurality of candidate model indexes.

In some possible implementations, the data processing links correspond to a plurality of data processing models; the calculation unit includes: the combining subunit is used for respectively selecting a group of model precipitation samples from a plurality of groups of model precipitation samples corresponding to the plurality of data processing models to combine to obtain a plurality of model precipitation sample combinations; and the calculating subunit is used for determining one candidate model index corresponding to the plurality of data processing models according to the intersection of each group of model precipitation samples in the model precipitation sample combination.

It will be appreciated that the above-described modules of the test apparatus in the embodiments of the present disclosure have the function of implementing the corresponding steps of the test method in the embodiment shown in fig. 1. The functions can be realized by hardware, and can also be realized by executing corresponding software by hardware. The hardware or software includes one or more modules corresponding to the functions described above. The modules may be software and/or hardware, and each module may be implemented separately or may be implemented by integrating multiple modules. For the functional description of each module of the above-mentioned testing device, reference may be specifically made to the corresponding description of the testing method in the embodiment shown in fig. 1, which is not repeated herein.

In the technical scheme of the disclosure, the acquisition, storage, application and the like of the related user personal information all conform to the regulations of related laws and regulations, and the public sequence is not violated.

According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

The electronic device includes: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the test methods as provided by the embodiments of the present disclosure.

Compared with the prior art, the electronic equipment acquires test samples corresponding to other data processing nodes of the data processing link according to the test sample corresponding to the last data processing node of the data processing link and the data conversion rate corresponding to the data processing nodes of the data processing link, so that the test samples corresponding to the data processing nodes can form a data funnel, and the test samples more consistent with the real situation are acquired; and testing the data processing model through the constructed test sample which is more consistent with the real situation so as to improve the consistency degree of the acquired model index and the on-line service index, and further improve the performance of the data processing model acquired by training by using the acquired model index.

The readable storage medium is a non-transitory computer readable storage medium storing computer instructions for causing a computer to perform a test method as provided by embodiments of the present disclosure.

Compared with the prior art, the readable storage medium acquires test samples corresponding to other data processing nodes of the data processing link according to the test sample corresponding to the last data processing node of the data processing link and the data conversion rate corresponding to the data processing nodes of the data processing link, so that the test samples corresponding to all the data processing nodes can form a data funnel, and the test samples more consistent with the real situation are acquired; and testing the data processing model through the constructed test sample which is more consistent with the real situation so as to improve the consistency degree of the acquired model index and the on-line service index, and further improve the performance of the data processing model acquired by training by using the acquired model index.

The computer program product comprises a computer program which, when executed by a processor, implements a test method as provided by embodiments of the present disclosure.

Compared with the prior art, the computer program product acquires test samples corresponding to other data processing nodes of the data processing link according to the test sample corresponding to the last data processing node of the data processing link and the data conversion rate corresponding to the data processing nodes of the data processing link, so that the test samples corresponding to all the data processing nodes can form a data funnel, and the test samples more consistent with the real situation are acquired; and testing the data processing model through the constructed test sample which is more consistent with the real situation so as to improve the consistency degree of the acquired model index and the on-line service index, and further improve the performance of the data processing model acquired by training by using the acquired model index.

Fig. 9 shows a schematic block diagram of an example electronic device 900 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 9, the apparatus 900 includes a computing unit 901 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 902 or a computer program loaded from a storage unit 908 into a Random Access Memory (RAM) 903. In the RAM903, various programs and data required for the operation of the device 900 can also be stored. The computing unit 901, the ROM902, and the RAM903 are connected to each other by a bus 904. An input/output (I/O) interface 905 is also connected to the bus 904.

Various components in device 900 are connected to I/O interface 905, including: an input unit 906 such as a keyboard, a mouse, or the like; an output unit 907 such as various types of displays, speakers, and the like; a storage unit 908 such as a magnetic disk, an optical disk, or the like; and a communication unit 909 such as a network card, modem, wireless communication transceiver, or the like. The communication unit 909 allows the device 900 to exchange information/data with other devices through a computer network such as the internet and/or various telecommunications networks.

The computing unit 901 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 901 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 901 performs the respective methods and processes described above, such as a test method. For example, in some embodiments, the test method may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as the storage unit 908. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device 900 via the ROM902 and/or the communication unit 909. When the computer program is loaded into RAM903 and executed by the computing unit 901, one or more steps of the test method described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform the test method by any other suitable means (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.

The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server incorporating a blockchain.

It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel, sequentially, or in a different order, provided that the desired results of the disclosed aspects are achieved, and are not limited herein.

The above detailed description should not be taken as limiting the scope of the present disclosure. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present disclosure are intended to be included within the scope of the present disclosure.

Claims

1. A method of testing, comprising:

2. The method of claim 1, wherein the determining the test sample corresponding to the data processing node of the data processing link according to the test sample corresponding to the first data processing node and the data conversion rate corresponding to the data processing node of the data processing link comprises:

determining a test sample corresponding to an Nth data processing node of the data processing link according to the test sample corresponding to the (n+1) th data processing node of the data processing link and the data conversion rate corresponding to the Nth data processing node of the data processing link; where N is an integer greater than 0 and less than the number of data processing nodes of the data processing link.

3. The method of claim 2, wherein the test samples corresponding to the nth data processing node of the data processing link comprise a test positive sample and a test negative sample;

The determining the test sample corresponding to the nth data processing node of the data processing link according to the test sample corresponding to the (n+1) th data processing node of the data processing link and the data conversion rate corresponding to the nth data processing node of the data processing link includes:

determining the number of test negative samples corresponding to the Nth data processing node of the data processing link according to the number of test samples corresponding to the (n+1) th data processing node of the data processing link and the data conversion rate corresponding to the Nth data processing node of the data processing link;

4. The method of claim 1, wherein the first data processing node comprises a plurality of data processing levels;

The obtaining the test sample corresponding to the first data processing node includes:

determining the number of test samples corresponding to the plurality of data processing levels according to the sample number ratio and the number of test samples corresponding to the first data processing level in the plurality of data processing levels; the sample number ratio is determined according to the number ratio of the historical service data corresponding to a plurality of data processing grades in the historical service data corresponding to the first data processing node;

and sampling test samples corresponding to the data processing grades from the unused training samples corresponding to the first data processing node according to the number of the test samples corresponding to the data processing grades.

5. The method of claim 1, wherein the data conversion rate corresponding to a data processing node of the data processing link is determined according to a ratio of the number of historical traffic data corresponding to the data processing node and the number of historical traffic data corresponding to a subsequent data processing node of the data processing node.

6. The method of claim 1, wherein the testing at least one data processing model according to the test sample corresponding to the data processing node of the data processing link, to obtain the model index of the data processing model, comprises:

Inputting a test sample of a data processing node corresponding to the data processing model into the data processing model, and determining a model precipitation sample from the test sample of the data processing node corresponding to the data processing model according to the output of the data processing model;

and determining model indexes of the data processing model according to the model precipitation samples.

7. The method of claim 6, wherein the data processing link corresponds to a plurality of data processing models;

the determining the model index of the data processing model according to the model precipitation sample comprises the following steps:

determining a final precipitated sample according to the intersection of the model precipitated samples corresponding to the data processing models;

and determining model indexes of a plurality of data processing models according to the intersection set of the final precipitated samples and the test samples corresponding to the first data processing nodes.

8. The method of claim 6, wherein the determining a model-derived sample from test samples of the data processing nodes corresponding to the data processing model according to the output of the data processing model comprises:

grouping test samples of data processing nodes corresponding to the data processing model according to the output of the data processing model, and obtaining a plurality of groups of model precipitation samples corresponding to the data processing model;

and determining a plurality of candidate model indexes of the data processing model according to a plurality of groups of model precipitation samples, and determining the model indexes of the data processing model from the plurality of candidate model indexes.

9. The method of claim 8, wherein the data processing link corresponds to a plurality of data processing models;

the determining a plurality of candidate model indexes of the data processing model according to a plurality of groups of model precipitation samples comprises the following steps:

respectively selecting a group of model precipitation samples from a plurality of groups of model precipitation samples corresponding to a plurality of data processing models to combine to obtain a plurality of model precipitation sample combinations;

and determining a candidate model index corresponding to the data processing models according to the intersection set of each group of model precipitation samples in the model precipitation sample combination.

10. A test apparatus comprising:

11. The apparatus of claim 10, wherein the sample module is to:

12. The apparatus of claim 11, wherein the test samples corresponding to an nth data processing node of the data processing link comprise a test positive sample and a test negative sample;

The sample module includes:

the positive sample unit is used for taking a test sample corresponding to an (N+1) th data processing node of the data processing link as a test positive sample corresponding to an (N) th data processing node of the data processing link;

the negative sample unit is used for determining the number of the test negative samples corresponding to the Nth data processing node of the data processing link according to the number of the test samples corresponding to the (n+1) th data processing node of the data processing link and the data conversion rate corresponding to the Nth data processing node of the data processing link; and sampling and acquiring the test negative samples corresponding to the Nth data processing node of the data processing link from the training unused samples corresponding to the Nth data processing node of the data processing link according to the number of the test negative samples corresponding to the Nth data processing node of the data processing link.

13. The apparatus of claim 10, wherein the first data processing node comprises a plurality of data processing levels;

the node module includes:

the number calculating unit is used for determining the number of the test samples corresponding to the plurality of data processing levels according to the sample number ratio and the number of the test samples corresponding to the first data processing level in the plurality of data processing levels; the sample number ratio is determined according to the number ratio of the historical service data corresponding to a plurality of data processing grades in the historical service data corresponding to the first data processing node;

And the sampling unit is used for sampling and acquiring the test samples corresponding to the data processing grades from the training unused samples corresponding to the first data processing node according to the number of the test samples corresponding to the data processing grades.

14. The apparatus of claim 10, wherein the test module comprises:

the extraction unit is used for inputting the test sample of the data processing node corresponding to the data processing model into the data processing model, and determining a model extraction sample from the test sample of the data processing node corresponding to the data processing model according to the output of the data processing model;

and the calculation unit is used for determining model indexes of the data processing model according to the model precipitation samples.

15. The apparatus of claim 14, wherein the data processing link corresponds to a plurality of data processing models;

the calculation unit includes:

a first intersection calculating subunit, configured to determine a final precipitated sample according to intersections of model precipitated samples corresponding to a plurality of data processing models;

and the second intersection calculating subunit is used for determining model indexes of a plurality of data processing models according to intersections of the final precipitated samples and the test samples corresponding to the first data processing nodes.

16. The apparatus of claim 14, wherein the extraction unit comprises:

the grouping subunit is used for grouping the test samples of the data processing nodes corresponding to the data processing model according to the output of the data processing model, and obtaining a plurality of groups of model precipitation samples corresponding to the data processing model;

the calculation unit includes:

and the screening subunit is used for determining a plurality of candidate model indexes of the data processing model according to a plurality of groups of model precipitation samples, and determining the model indexes of the data processing model from the plurality of candidate model indexes.

17. The apparatus of claim 16, wherein the data processing link corresponds to a plurality of data processing models;

the calculation unit includes:

the combining subunit is used for respectively selecting a group of model precipitation samples from a plurality of groups of model precipitation samples corresponding to the plurality of data processing models to combine to obtain a plurality of model precipitation sample combinations;

and the calculating subunit is used for determining one candidate model index corresponding to the plurality of data processing models according to the intersection of each group of model precipitation samples in the model precipitation sample combination.

18. An electronic device, comprising:

At least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-9.

19. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-9.

20. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any of claims 1-9.