CN116596594A - Method, apparatus, device and storage medium for evaluating data delivery - Google Patents

Abstract

Embodiments of the present disclosure provide methods, apparatuses, devices, and storage medium for evaluating data delivery. The method comprises the following steps: receiving a first selection by a user of at least one indicator for evaluating the data delivery and a second selection of at least one anomaly type for evaluating the data delivery; based on the first selection and the second selection of the user, acquiring user configuration information for evaluating data delivery, wherein the user configuration information at least comprises at least one index to be evaluated and at least one abnormality type; and alerting of an abnormality of the data delivery based on a comparison of the evaluation result for the at least one indicator with a corresponding threshold value, wherein the corresponding threshold value of the at least one indicator is associated with the at least one abnormality type. Therefore, the abnormal situation of data input can be conveniently and rapidly evaluated, an alarm is given, and the evaluation effect is improved.

Description

Method, apparatus, device and storage medium for evaluating data delivery

Technical Field

Example embodiments of the present disclosure relate generally to the field of computers and, more particularly, relate to methods, apparatuses, devices, and computer-readable storage media for evaluating data delivery.

Background

The internet provides access to a wide variety of resources. For example, various applications, goods, audio-visual contents, etc. can be accessed through the internet. In addition, the accessible content may include specific recommended content items associated with various types of objects/resources, including advertisements, for example. A resource provider with resources may provide a content delivery party with a delivery of a recommended content item. For a resource provider (e.g., advertiser), it is desirable that its own resources be able to receive the attention of a large number of customers.

Disclosure of Invention

In a first aspect of the present disclosure, a method for evaluating data placement is provided. The method comprises the following steps: acquiring user configuration information for evaluating data delivery, wherein the user configuration information at least comprises at least one index to be evaluated and at least one abnormality type; and alerting of an abnormality of the data delivery based on a comparison of the evaluation result for the at least one indicator with a corresponding threshold value, wherein the corresponding threshold value of the at least one indicator is associated with the at least one abnormality type.

In a second aspect of the present disclosure, an apparatus for evaluating data delivery is provided. The device comprises: the information acquisition module is configured to acquire user configuration information for evaluating data delivery, wherein the user configuration information at least comprises at least one index to be evaluated and at least one abnormality type; and an anomaly alarm module configured to alarm for anomalies in the data delivery based on a comparison of the evaluation result for the at least one indicator with a respective threshold value, wherein the respective threshold value of the at least one indicator is associated with the at least one anomaly type.

In a third aspect of the present disclosure, an electronic device is provided. The apparatus comprises at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. The instructions, when executed by at least one processing unit, cause the apparatus to perform the method of the first aspect.

In a fourth aspect of the present disclosure, a computer-readable storage medium is provided. The computer readable storage medium has stored thereon a computer program executable by a processor to implement the method of the first aspect.

It should be understood that what is described in this section of the disclosure is not intended to limit key features or essential features of the embodiments of the disclosure, nor is it intended to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following description.

Drawings

The above and other features, advantages and aspects of embodiments of the present disclosure will become more apparent by reference to the following detailed description when taken in conjunction with the accompanying drawings. In the drawings, wherein like or similar reference numerals denote like or similar elements, in which:

FIG. 1 illustrates a schematic diagram of an example environment in which embodiments of the present disclosure may be implemented;

FIG. 2 illustrates a flow chart of a process for evaluating data placement according to some embodiments of the present disclosure;

FIG. 3 illustrates a schematic diagram of a process of data placement assessment, according to some embodiments of the present disclosure;

FIG. 4 illustrates a schematic diagram of a process for evaluating data placement according to some embodiments of the present disclosure;

FIG. 5 illustrates a schematic block diagram of an apparatus for evaluating data impressions in accordance with certain embodiments of the present disclosure; and

fig. 6 illustrates a block diagram of an electronic device in which one or more embodiments of the disclosure may be implemented.

Detailed Description

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure have been illustrated in the accompanying drawings, it is to be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but rather, these embodiments are provided so that this disclosure will be more thorough and complete. It should be understood that the drawings and embodiments of the present disclosure are for illustration purposes only and are not intended to limit the scope of the present disclosure.

In describing embodiments of the present disclosure, the term "comprising" and its like should be taken to be open-ended, i.e., including, but not limited to. The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions are also possible below. The terms "first," "second," and the like, may refer to different or the same object. Other explicit and implicit definitions are also possible below.

It will be appreciated that the data (including but not limited to the data itself, the acquisition or use of the data) involved in the present technical solution should comply with the corresponding legal regulations and the requirements of the relevant regulations.

It will be appreciated that prior to using the technical solutions disclosed in the embodiments of the present disclosure, the user should be informed and authorized of the type, usage range, usage scenario, etc. of the personal information related to the present disclosure in an appropriate manner according to the relevant legal regulations.

For example, in response to receiving an active request from a user, a prompt is sent to the user to explicitly prompt the user that the operation it is requesting to perform will require personal information to be obtained and used with the user. Thus, the user can autonomously select whether to provide personal information to software or hardware such as an electronic device, an application program, a server or a storage medium for executing the operation of the technical scheme of the present disclosure according to the prompt information.

As an alternative but non-limiting implementation, in response to receiving an active request from a user, the manner in which the prompt information is sent to the user may be, for example, a popup, in which the prompt information may be presented in a text manner. In addition, a selection control for the user to select to provide personal information to the electronic device in a 'consent' or 'disagreement' manner can be carried in the popup window.

It will be appreciated that the above-described notification and user authorization process is merely illustrative and not limiting of the implementations of the present disclosure, and that other ways of satisfying relevant legal regulations may be applied to the implementations of the present disclosure.

As mentioned briefly above, the resource provider expects that the self-released resource can be focused by a large number of clients, and in order to ensure the effect of data release, the resource provider needs to evaluate the data release condition of the self-released resource and process the abnormality in time when the data release is abnormal. Traditional evaluation approaches often rely on a launch platform or system, often requiring manual intervention to ascertain anomalies, and giving some ambiguous feedback at the granularity of operation by the launch platform. For example, for ad placement, the ad platform gives "creative repetition high" feedback at the "creative" level. However, the conventional evaluation method cannot give a key cause of the occurrence of the abnormality, and cannot indicate a key problem to the resource provider, and when optimizing the delivery effect, the user is more concerned about the solution of the "key problem". In addition, from the perspective of the platform, the data delivery evaluation of the delivery platform often determines evaluation results and suggestions based on the data of the platform, and the resource provider often considers more sexualized delivery factors. Therefore, the conventional platform-dependent data delivery evaluation method cannot meet the evaluation requirements of the resource provider.

The embodiment of the disclosure provides a scheme for evaluating data delivery. According to the scheme, user configuration information comprising indexes to be evaluated and anomaly types is obtained, the anomaly condition of data delivery is determined based on the comparison result of the evaluation result of the indexes and the corresponding threshold value, and the anomaly is alarmed when the anomaly is determined to occur. In this way, the abnormal situation of the data input can be evaluated based on the index to be evaluated configured by the user, so that the flexibility and the efficiency of the evaluation are improved, the evaluation effect is improved, and the user experience is improved.

FIG. 1 illustrates a schematic diagram of an example environment 100 in which embodiments of the present disclosure may be implemented. As shown in fig. 1, an example environment 100 may include an electronic device 110.

In this example environment 100, an electronic device 110 may be running with an application 115. The application 115 may be any suitable type of application for evaluating data impressions, such as a simulation application, an analog application, a data statistics class application, and the like. The user 120 may be a resource provider that needs to make data placement evaluations. In the context of advertisement delivery, user 120 may also be referred to as an advertiser. The user 120 may interact with the application 115 via the electronic device 110 and/or its attached device.

In some embodiments, in the environment 100 of fig. 1, the electronic device 110 may present the interface 140 of the application 115 if the application 115 is in an active state. Via interface 140, application 115 can provide one or more services related to the data placement assessment to user 120, e.g., where interface 140 is a user-configured interface, electronic device 110 can determine metrics and anomaly types associated with data detection in response to receiving user 120 settings in the user-configured interface.

The electronic device 110 may obtain data in the drop platform 150 associated with drop data 160, which may include, for example, a number of customers, customer retention, customer consumption data, and so forth. The launch platform 150 may include any suitable type of application for data launch, examples of which may include, but are not limited to: content sharing class applications, social class applications, chat applications, shopping applications, and the like. The delivery data 160 may include, for example, various objects, examples of which may include applications, physical merchandise, virtual merchandise, audiovisual content, and so forth. In some embodiments, the impression data 160 refers to content presented to recommend corresponding resources. Examples of impression data 160 may include advertisements. A customer group may include one or more customer members, which may be any potential consumer of a resource, such as a user, community, organization, entity, or the like.

Electronic device 110 may include any computing system having computing capabilities, such as various computing devices/systems, terminal devices, server devices, and the like. The terminal device may be any type of mobile terminal, fixed terminal, portable terminal, or the like, including a cell phone, desktop computer, laptop computer, notebook computer, netbook computer, tablet computer, media computer, multimedia tablet, personal Communication System (PCS) device, personal navigation device, personal Digital Assistant (PDA), audio/video player, digital camera/camcorder, positioning device, television receiver, radio broadcast receiver, electronic book device, virtual Reality (VR) all-in-one, game console, game book, or any combination of the foregoing, including accessories and peripherals for these devices, or any combination thereof. The server device may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, a content distribution network, basic cloud computing services such as big data and an artificial intelligent platform. The server devices may include, for example, computing systems/servers, such as mainframes, edge computing nodes, computing devices in a cloud environment, and so forth.

It should be understood that the structure and function of the various elements in environment 100 are described for illustrative purposes only and are not meant to suggest any limitation as to the scope of the disclosure.

Some example embodiments of the present disclosure will be described below with continued reference to fig. 2-4.

Fig. 2 illustrates a flow chart of a process 200 for evaluating data placement according to some embodiments of the present disclosure. Process 200 may be implemented at electronic device 110. For ease of discussion, the process 200 will be described with reference to the environment 100 of FIG. 1.

At block 205, the electronic device 110 receives a first selection by a user of at least one indicator for evaluating data impressions and a second selection of at least one anomaly type for evaluating data impressions. At block 210, the electronic device 110 obtains user configuration information for evaluating the delivery of data based on the first selection and the second selection of the user, the user configuration information including at least one indicator and at least one anomaly type to be evaluated.

In some embodiments, electronic device 110 may present a user interface, such as interface 140 in fig. 1, for obtaining user configuration information and determine the user configuration information based on user input received in the user interface. In addition to the index and the anomaly type, other configuration information for detecting the data delivery may be included in the user configuration information, for example, a standard or a reference value or a threshold value for determining whether an anomaly occurs.

In some embodiments, the electronic device 110 may present the plurality of metrics and the plurality of anomaly types as options in a user interface for selection by a user. The electronic device 110 may receive a first selection of at least one indicator by a user and a second selection of at least one anomaly type by the user and determine user configuration information based on the first selection and the second selection. For example, electronic device 110 may present 4 indicators, indicator a, indicator B, indicator C, and indicator D, and three exception types, exception type a, exception type B, and exception type C, in a user interface. The electronic device 110 may determine that user configuration information including the index a and the anomaly type a is received in response to receiving a user selection operation of the index a and the anomaly type a in the user interface.

Alternatively or additionally, the user may input user configuration information regarding the metrics and anomaly types through other means than user interface operations. For example, the user may input user configuration information through a voice instruction (e.g., "evaluate index a, anomaly type is determined to be anomaly type a"). Accordingly, electronic device 110 may determine that user configuration information is received in response to receiving a voice instruction associated with the user configuration. For another example, the user may input user configuration information through an instruction or program (e.g., a string of codes). In response to receiving the instructions or programs associated with the user configuration, electronic device 110 may determine that the user configuration information was received.

In some embodiments, the configuration of the indicators and/or the anomaly categories may also be determined by the electronic device 110 based on the delivery effect required by the delivery data. For example, if the placement data is an advertisement, the electronic device 110 may itself use the input-to-output Ratio (ROI) as an indicator.

The index to be evaluated may include any suitable index capable of reflecting an abnormal condition of data delivery. In some embodiments, at least one indicator in the user configuration information obtained by the electronic device 110 may be associated with at least one of a new customer number, customer retention, input-to-output ratio, cold start. The customer is, for example, the audience who puts in the data. The new customer number may include a Day New User (DNU). The input-output ratio is, for example, a ratio between the benefit and the cost when the user performs data delivery. Cold start is for example a way of starting, meaning starting from a 0 base. During the data delivery process, a "cold start" may be an initial phase in which data has not been sufficiently accumulated.

The metrics to be evaluated may include one or more of a new customer number, customer retention, input-to-output ratio, cold start, and may also include metrics related to one or more of these four. For example, the metrics to be evaluated may include the success rate of the cold start associated with the cold start, the speed of the cold start, the survival rate of the cold start (which, for example, indicates that the cold start was successfully stabilized or even increased or decayed), and the like. Alternatively or additionally, the metrics to be evaluated may include a number of customer reservations associated with the customer reservations, a customer reservation rate, and the like. Alternatively or additionally, the indicator associated with the number of newly added users may include a proportion of newly added users to active users. Alternatively or additionally, the ROI may comprise a ROI difference rate.

The anomaly type may reflect the class or classification of various anomalies of the data feed. In some embodiments, the anomaly type may include at least one of absolute anomalies, relative anomalies, trend anomalies.

An absolute anomaly may reflect a situation where a certain indicator does not reach a predetermined target. For example, for an index of the number of new customers, the predetermined target level is 1000, but the actual realized level is 800, and the predetermined target is not reached. Thus, the electronic device 110 may determine that an absolute abnormality occurs with respect to the index.

The relative anomalies may reflect anomalies of one data delivery means relative to other data delivery means. The data may be delivered by a plurality of delivery means, for example, by different delivery platforms, or by using different delivery means, for example, by web pages, links, programs, etc. The delivery results obtained by different delivery means for the same delivery data may be different, for example, the number of clients to which the same delivery data is attracted at different delivery platforms may be different. If the difference of the throwing results obtained by different throwing means is larger for the same index, for example, the corresponding value of the index obtained by one means is far lower than the corresponding value of the index obtained by other means, the relative abnormality can be judged.

For example, when the target value of the index a is 500, the specific value of the index a obtained by the dispensing means a is 600, and the specific value of the index a obtained by the dispensing means B is 100. Since the value of the index a obtained by the throwing means B is far lower than the value of the index a obtained by the throwing means a, the electronic device 110 can determine that the index a obtained by the throwing means B is relatively abnormal.

A trend anomaly may refer to a situation where the difference in variation between two values corresponding to the same indicator over different periods is greater than a large (e.g., greater than a threshold value). The value of the specific threshold may be configured by the user or set automatically by the electronic device 110. For example, if the index a corresponds to a value of 1000 in a first period of time (e.g., a first week) and the index a corresponds to a value of 800 in a second, subsequent period of time (e.g., a second week), the index a corresponds to a value difference of 200 between the first week and the second week, which is greater than a threshold set to 100. Thus, the electronic device 110 may determine that the index a is trending abnormal.

In some embodiments, the exception type may include an exception determined by the launch platform, which may be an exception condition evaluated by the launch platform based on its own policies and logic. For example, a user may configure electronic device 110 to evaluate and alert for anomalies determined and reported by the drop platform.

At block 220, electronic device 110 alerts of anomalies in the data delivery based on a comparison of the evaluation result for the at least one indicator with a corresponding threshold value, wherein the corresponding threshold value of the at least one indicator is associated with the at least one anomaly type. For example, the electronic device 110 may evaluate, based on the indicators to be evaluated and the anomaly type included in the user configuration information, for each indicator to determine whether an anomaly condition corresponding to the anomaly type occurs in the data delivery. For example, the user configuration information indicates an evaluation index a and an absolute abnormality, and the electronic device 110 monitors the index a to determine whether the index a is absolutely abnormal. In this way, the index configured by the user and the anomaly type can be evaluated, so that targeted evaluation is performed on the data delivery, the calculation amount is reduced, and the evaluation efficiency is improved.

In some embodiments, electronic device 110 may perform an evaluation on the delivery data in response to receiving user configuration information including at least one indicator to be evaluated and at least one anomaly type, resulting in an evaluation result. In some embodiments, the electronic device 110 may continuously or periodically perform the data delivery evaluation and obtain an evaluation result of at least one index in the user configuration information. For example, the electronic device 110 may perform data delivery evaluation and acquire an evaluation result periodically on a monthly basis. The electronic device 110 may also determine and alter the evaluation period itself based on user demand or data placement. For example, for a first delivery of a certain data, the electronic device 110 may determine a short evaluation period (e.g., one week) so as to timely obtain the evaluation result and determine an abnormal situation and adjust accordingly, which helps to ensure the delivery effect of the data delivery.

In some embodiments, the electronic device 110 may pre-obtain a threshold value corresponding to a certain indicator and associated with an anomaly type, e.g., a threshold value a for absolute anomalies for indicator a, a threshold value B for relative anomalies for indicator a, etc. These thresholds may be statically configured or dynamically configured (e.g., by a user). The electronic device 110 may compare the evaluation result of the at least one index with a corresponding threshold value, and alarm the abnormality of the data delivery if the comparison result indicates that the abnormality is determined to occur.

The electronic device 110 may alert in a variety of ways including, but not limited to, sending a pop-up window containing information related to the anomaly, voice, messaging, etc. For example, electronic device 110 may present a message such as "XX index causes XX abnormality" on the user interface to alert. Alternatively or additionally, in some embodiments, to enhance the alert effect, the electronic device 110 may also perform a vibration, a screen blink, display a special color, etc. to alert.

In some embodiments, the electronic device 110 may also provide the user with information about the anomaly, such as the particular dimension in which the anomaly occurred, the particular index, the particular type of anomaly, and so forth. Based on such information, the user can optimize for the abnormal situation.

Data placement may be performed in multiple dimensions from top to bottom. The plurality of dimensions may include at least one of resources for which the delivery of data is used for recommendation, delivery platforms, delivery accounts, delivery plans, delivery targets, delivery means. For example, the data can be launched on different platforms according to different resources to be recommended, the data can be launched on the same platform according to the same resource to be recommended through different launching accounts, or the data can be launched by using different launching plans or launching targets. One low-level dimension may be referred to as a "sub-item" or "sub-hierarchy" of the high-level dimension. For example, one drop platform may contain multiple drop accounts, which may be referred to as sub-items or sub-tiers of the drop platform.

The resources to which the data is put for recommendation may include applications with which the data is associated. For example, data placement may be performed for an application that may be considered as a resource for which placement of data is used for recommendation. The delivery platform may include a platform or system for data delivery, which may include various application class platforms, such as content sharing class applications, social class applications, chat applications, shopping applications, and so forth. The data delivery can be performed on the delivery platform by means of links, web pages, audio and video contents and the like. The delivery account may include a user account for logging into or using the delivery platform.

The delivery plan may refer to a specific plan for data delivery, and may include, for example, material for data delivery. For example, in an advertising scenario, the placement plan may be an advertising plan, which may include a plurality of advertising materials. The drop target may indicate a user's desired target for the data drop, e.g., how many orders of magnitude new customers are desired, or how much ratio customers remain desired, etc. Different desired objectives may correspond to different bid types. For example, in the context of advertising, two bid types may be provided, the first bid type corresponding to the desired goal of one indicator of the new number of customers and the second bid type corresponding to the desired goal of both the new number of customers and customer retention.

In some embodiments, the effectiveness of data delivery in each of a plurality of dimensions may be evaluated and anomalies subsequently determined based on the evaluation of those dimensions. As an example, the electronic device 110 may evaluate the index by a specific frequency based on the user configuration information, and perform an alarm in a case where it is determined that the index is abnormal corresponding to the type of abnormality included in the user configuration information, which may be referred to as abnormality discovery. In the anomaly discovery process, if the threshold is reached at the evaluation result, an alarm may be given.

The evaluation result of a certain index can be determined based on the following formula:

taking the ROI index as an example, the evaluation result of the index can be determined based on the following formula:

wherein an analysis group indicates results within an analysis range (e.g., one or more dimensions), and a benchmark group indicates a comparison benchmark, which may:

1) In the same analysis range as the analysis group, but for different time periods: is commonly found in ring ratio fluctuation analysis;

2) Different analysis ranges but for the same time period as the analysis group: common to both sets of comparative analyses;

3) Absolute value: common to absolute target achievement analysis.

If Δroi% is below the threshold, it may be determined that an abnormality in the ROI index occurs.

The absolute value result of the index can be determined by the following formula:

where j=1, …, n, represents the j-th evaluation period, and n is a positive integer.

In some embodiments, the electronic device 110 may determine respective contributions of impressions of data to anomalies (also referred to as "contribution rates") in each of a plurality of dimensions. The key dimension may be determined from the plurality of dimensions based on a comparison of the respective contribution of each dimension to the respective threshold contribution. Such comparison may include a comparison of the contribution (or contribution rate) to a threshold. For example, a dimension with a higher contribution may be determined as the critical dimension. The comparison may also include a comparison of the contribution change to a threshold. For example, if the contribution of a dimension changes from 60% to 40%, decreasing by 20%, by more than a 10% decrease threshold ratio, then that dimension can be determined to be the critical dimension.

The electronic device 110 may determine respective contributions of data in multiple dimensions for respective metrics in the user configuration information. The contribution degree includes, but is not limited to, a specific value corresponding to the index change, a ratio of change of the specific value corresponding to the index (the larger the ratio of change is, the larger the contribution degree is), a ratio of the specific value corresponding to the index, and the like.

In some embodiments, the electronic device 110 may determine the contribution degree for one dimension of the one index based on a first evaluation value obtained in one of the plurality of dimensions for the one index and a second evaluation value obtained in the plurality of dimensions. For example, for a certain index, the ratio of the first evaluation value in a certain dimension to the second evaluation values in a plurality of dimensions may be determined as the contribution degree of the dimension. As an example, data placement can be performed in 4 dimensions A, B, C and D and an index of the new customer number can be evaluated. If the number of new clients corresponding to the dimension a is 1500, the number of new clients corresponding to the dimension B is 3500, and the number of new clients corresponding to the dimension C and the dimension D is 2500, the respective contribution degrees of the 4 dimensions from the dimension a to the dimension D may be 15%, 35%, 25%.

As yet another example, for a customer retention indicator, if a first evaluation value obtained in a dimension corresponding to a certain delivery account of a certain delivery platform is 2000 and a second evaluation value obtained in a plurality of dimensions corresponding to a plurality of accounts on the delivery platform is 10000, the contribution of the delivery account is 20% for the customer retention indicator.

In some embodiments, anomalies of a certain dimension may be associated with changes in the overall evaluation value of all dimensions (also referred to as "large disks"). For example, the duty ratio of the variation of the first evaluation value in one dimension with respect to the variation of the second evaluation value in a plurality of dimensions may be determined as the contribution degree of the dimension.

As an example, the contribution of sub-item i may be determined as:

the comparison period may refer to a current evaluation period, the reference period may refer to a previous evaluation period, X with a subscript i represents an evaluation value of a sub item, and X without a subscript i represents an evaluation value of a large disc. The larger the duty cycle of the variation of the sub-item i, the higher the contribution of the sub-item; and vice versa. By comparing the data of a certain period with the data of a previous period, the increase and decrease change condition can be evaluated, and the trend change percentage can be calculated. Thus, the anomaly analysis can be performed by ring ratio fluctuation analysis.

For example, in the last period, the first evaluation value obtained on a certain delivery account on a certain delivery platform for the customer retention index is 2000, and the second evaluation value obtained on the delivery platform is 10000. In the current period, a first evaluation value obtained on a certain delivery account on a certain delivery platform aiming at a customer retention index is 1000, and a second evaluation value obtained on the delivery platform is 8000. Between two periods, the change of the first evaluation value is-1000, and the change of the second evaluation value is-2000, the contribution degree of the change of the first evaluation value to the change of the second evaluation value may be 50%.

In some embodiments, the anomaly may be associated with a change in the ratio of the first and second evaluation values. In this case, the electronic device 110 may determine the contribution of the variation of the first evaluation value to the variation of the ratio and the contribution of the variation of the second evaluation value to the variation of the ratio, respectively.

As an example, the electronic device 110 may determine the duty cycle index (e.g., DNU duty cycle) of the sub-term by the following formula:

where X represents an evaluation value.

The change in the ratio of the first evaluation value and the second evaluation value, for example, the change in the ratio of the sub item and the large disk can be expressed as:

The contribution degree of the variation of the first evaluation value, for example, the contribution degree of the variation of the sub item may be determined as:

the contribution degree of the change of the second evaluation value, for example, the contribution degree of the large disc change may be determined as:

in some embodiments, the anomaly may be associated with a change in the second evaluation value over multiple dimensions. In this case, the contribution degree of one dimension may be determined by comprehensively considering both the variation of the first evaluation value of the dimension and the variation of the duty ratio of the first evaluation value with respect to the second evaluation value.

For example, the electronic device 110 may determine the ratio index (e.g., retention or ROI) by the following formula:

where p represents a weight.

The contribution of sub-item i may be determined as:

wherein the contribution rate _{iX difference} Representing the contribution degree and the contribution rate of the change of the sub item i _{i New increase in the duty cycle difference} The contribution of the newly added duty cycle of sub-item i is represented.

In some embodiments, the electronic device 110 may sequentially disassemble the difference rates of the analysis group and the reference group into sub-level contributions in the form of a decision tree based on the selected control group. For example, the difference rate of one delivery platform can be disassembled into the contribution degree of each delivery account. The decision tree may be pruned, e.g., to remove dimensions that have been compared to determine that no anomaly has occurred, based on the threshold contribution determined in advance. The pruned path may be considered a core problem path (i.e., critical dimension). This process performed by electronic device 110 may also be referred to as dimension drill down. After the dimension drill-down is performed, the electronic device 110 can evaluate only the index of the key dimension, which is helpful to reduce the calculation amount of the electronic device 110 and improve the evaluation efficiency of the electronic device 110.

In some embodiments, the sum of contributions of multiple dimensions corresponding to the same index is 100%. The sum of the contribution degrees of the sub-levels is consistent with the contribution degree of the level above. For example, the contribution of one drop platform may be equal to the sum of the contribution of the individual drop accounts in the drop platform for an indicator of the number of new customers.

The contribution of the sub-level can be broken down into the sum of the single index rates of change:

the contribution ratio = control influence index a x change ratio of influence index B + control influence index B x change ratio of influence index a (11) sub-level contribution may reflect the influence on the overall index. The contribution degree of the disassembly can guide the subsequent optimization.

The dimension contribution degree calculation concept is as follows:

contribution ratio i=control duty ratio sub-dimension index difference magnitude + control sub-dimension index level sub-dimension duty ratio difference magnitude

The excess contribution (12) of analysis group i to the excess difference of the reference group + the difference of the reference group i to the reference group, wherein a factor with i is associated with a sub-level and a factor without i is associated with a sub-level upper level.

In some embodiments, the user configuration information may include a plurality of metrics, and the electronic device 110 may determine a respective contribution of each of the plurality of metrics to the anomaly and determine a key metric from the plurality of metrics based on a comparison of the respective contribution of each of the metrics to the respective threshold contribution. This process performed by electronic device 110 may be referred to as index drill-down. Index drill-down may be performed in various dimensions (or sub-items or sub-levels).

As an example, the electronic device 110 may use an index having a respective contribution greater than a respective threshold contribution as a key index. The electronic device 110 can further evaluate only the key indicators, so as to further reduce the calculation amount and power consumption of the electronic device 110 and improve the evaluation efficiency.

In some embodiments, after determining the key indicator, the indicator may be further broken down into a plurality of lower indicators for anomaly analysis. This process is also known as abnormal drill-down. For example, the contribution of the ratio index (e.g., ROI, persistence, etc.) to the numerator and denominator can be further broken down. As an example, the electronic device 110 may determine the contribution of the ratio index (e.g., ROI, persistence, etc.) of the sub-term i by the following formula:

taking the ROI index as an example, after determining that the ROI index is abnormal, the electronic device 110 may disassemble the ROI into two lower-level indexes of revenue and yield, and further determine whether the revenue and/or yield is abnormal based on comparing the determined contribution of the revenue and/or yield to the abnormality with the corresponding threshold contribution. Because revenue is associated with customer retention and revenue, in some embodiments, after determining that revenue is abnormal, the electronic device 110 may further disassemble revenue into two underlying indicators of customer retention and revenue. As an example, the electronic device 110 may determine whether the customer retention is abnormal based on a comparison of the customer retention difference rate to a first threshold difference rate and whether the revenue is abnormal based on a comparison of the revenue difference rate to a second threshold difference rate in response to determining that the revenue is abnormal. Because yield is associated with cost, in some embodiments, electronic device 110 may further disassemble the yield into at least one cost item. As an example, electronic device 110 may determine that at least one cost item is abnormal based on a comparison of a difference rate of the at least one cost item to a third threshold difference rate in response to determining that the yield is abnormal. .

Fig. 3 illustrates a schematic diagram of a process 300 of data placement evaluation according to some embodiments of the present disclosure.

In process 300, at 305, electronic device 110 further analyzes the revenue contribution and the cost contribution after determining that the input-to-output ratio is abnormal. As shown in fig. 3, at 310, a determination is made as to whether the revenue contribution > X% is true. If it does not, the diagnosis of the leg is ended at 312. If it is, the electronic device 110 further determines 315 to analyze the customer retention.

Because customer retention is greatly affected by the time period, the electronic device 110 may analyze the short retention discrepancy rate (i.e., the customer retention discrepancy rate over a short period of time) first and then analyze the long retention discrepancy rate (i.e., the customer retention discrepancy rate over a long period of time). As shown in FIG. 3, at 320, a determination is made as to whether the shortfall discrepancy rate < _X% is true. If it is, then the shortfall is diagnosed at 325. If it does not, then at 330, it is determined if the long-left difference rate < -X% is true. If so, then a long-lived diagnosis is made at 335. If not, the diagnosis of the customer's surviving leg is ended at 337.

If it is determined at 310 that the revenue contribution > X% is true, an analysis of revenue may also be determined at 340. At 345, it is determined whether the rate of return difference < _X% is true. If so, then a revenue diagnosis is made at 350. If not, the diagnosis of the branch line for revenue is ended at 352.

Regarding analysis of the cost contribution, as shown in fig. 3, at 355, it is determined whether the cost contribution > X% is true. If it does not, the diagnosis of the leg is ended at 357. If it does, the electronic device 110 further analyzes the cost term. At 360, it is determined whether the rate of difference > X% of the cost term M1 is true. If so, then a cost item M1 diagnosis is made at 365. If not, the diagnosis of the branch line for the cost term M1 ends at 367. At 370, it is determined whether the rate of difference > X% of the cost term M2 is true. If so, then a cost item M2 diagnosis is made at 375. If not, the diagnosis of the branch line for the cost term M2 ends at 377.

It should be understood that the various thresholds are represented as x% in fig. 3 for purposes of illustration only and not intended to suggest any limitation. In some embodiments, the thresholds for the various factors may be different. In this way, the electronic device 110 may refine the index gradually in a top-down manner, and determine specific factors that cause the abnormality, so as to provide assistance for the user to solve the abnormality subsequently.

As described above, the electronic device 110 may determine key factors for occurrence of the abnormality through resolution of dimensions and indexes, which helps to provide the user with finer alarm information and make subsequent abnormality diagnosis based on the alarm information.

Fig. 4 illustrates a schematic diagram of an architecture 400 for evaluating data placement according to some embodiments of the present disclosure.

As shown in fig. 4, the electronic device 110 may perform the task of evaluating the data delivery at least through the data module 410, the pre-warning module 420, and the anomaly localization module 430.

The data module 410 may be configured to perform index disassembly 405, which may utilize core data to disassemble the index to be evaluated into new customer numbers, customer retention, input-to-output ratios, cold starts, and so forth. The data module 410 may also utilize an information visualization module to present a user configuration interface to obtain user configuration information for the metrics and anomaly types.

The pre-warning module 420 may be configured to perform anomaly discovery 415, the types of anomalies discovered including, for example, absolute anomalies, relative anomalies, trend anomalies, anomalies determined by the launch platform, and so forth. The early warning module 420 may include an anomaly definition module that may define different anomaly types. The pre-warning module 420 may also include an anomaly capture module that may perform anomaly capture based on the user's assessment configuration in the user's configuration information, such as assessing data anomalies and determining anomaly types. The pre-warning module 420 may also include an information visualization module for presenting the captured anomaly metrics to a user (e.g., via a correlation interface). In some embodiments, the early warning module 420 may also include a statistical analysis module, which may be configured to statistically analyze the anomalies that occur.

Anomaly location module 430 may be configured to perform dimension drill down 425. By way of example, the dimensions may be, in order from high to low, an application (as an example of a resource to be recommended), a bid type (e.g., corresponding to a drop target), a channel (e.g., drop platform), an account (e.g., drop account), a channel package (e.g., data corresponding to different drop platforms), a plan (e.g., drop plan). The anomaly location module 430 can calculate the contribution of each dimension via a drill-down calculation module based on dimension drill-down logic and determine the critical dimension in which the anomaly occurred via a drill-down conclusion determination module.

The anomaly localization module 430 may also be configured to perform the index drill down 435 based on the determined critical dimension. Taking ROI (input-output ratio) as an example, the anomaly localization module 430 can break down it into two lower indicators of revenue and cost. Further, revenue may be broken down into two underlying indicators of retention and user revenue, which may be broken down into specific revenue items that are underlying indicators. The anomaly localization module 430 can also break down costs into specific cost terms that are underlying indicators.

The anomaly location module 430 may calculate the contribution of each of the metrics via a drill-down calculation module based on the metric drill-down logic and determine the key metrics for which anomalies occur via a drill-down conclusion determination module. The anomaly localization module 430 may in turn utilize the information visualization module to present the user with a user interface that includes anomaly metrics, underlying metric contribution rates, underlying contribution rates, in order to present the user with specific factors that lead to anomalies. In some embodiments, anomaly localization module 430 may also include a statistical analysis module that may be configured to statistically analyze the results of dimension drill-down and index drill-down.

In this way, the electronic device 110 can evaluate the delivery data simply and efficiently, and can intuitively display specific factors causing the occurrence of the abnormality for the user through an interface or other modes, which is beneficial to improving the evaluation effect and the evaluation efficiency.

In some embodiments, architecture 400 may also include a problem diagnosis module 440, a policy invocation module 450, and an effects feedback module 460, performing impression element diagnosis 445, auto tuning 455, and effects feedback 465, respectively.

In this way, the electronic device 110 may perform evaluation tasks of data delivery through the data module 410, the pre-warning module 420, and the anomaly localization module 430 in the architecture 400, and perform anomaly handling tasks through the issue diagnosis module 440, the policy invoking module 450, and the effects feedback module 460. The electronic device 110 can simply and efficiently diagnose and adjust the advertisement delivery effect based on the evaluation result, which is helpful for improving the efficiency of corresponding processing of the abnormality, and further improving the data delivery effect.

Fig. 5 illustrates a schematic block diagram of an apparatus 500 for evaluating data placement according to some embodiments of the present disclosure. The apparatus 500 may be implemented as or included in the electronic device 110. The various modules/components in apparatus 500 may be implemented in hardware, software, firmware, or any combination thereof.

As shown in fig. 5, the apparatus 500 comprises a receiving module 505 configured to receive a first selection by a user of at least one indicator for evaluating a delivery of data; and receiving a second selection by the user of at least one anomaly type for evaluating the delivery of data. The apparatus 500 comprises an information acquisition module 510 configured to acquire user configuration information for evaluating the delivery of data based on the first and second selections of the user, the user configuration information comprising at least one indicator to be evaluated and at least one anomaly type. The apparatus 500 further comprises an anomaly alarm module 520 configured to alarm for anomalies in the data feed based on a comparison of the evaluation result for the at least one indicator with a respective threshold value, wherein the respective threshold value of the at least one indicator is associated with the at least one anomaly type.

In some embodiments, the evaluation of the at least one indicator is obtained based on periodic evaluation of the delivery of data.

In some embodiments, the apparatus 500 further comprises: a dimension contribution determination module configured to determine respective contributions of impressions of data on respective ones of the plurality of dimensions to anomalies; and a critical dimension determination module configured to determine a critical dimension from the plurality of dimensions based on a comparison of the respective contribution of each dimension to the respective threshold contribution.

In some embodiments, the plurality of dimensions includes at least one of: the delivery of data is used for recommended resources, delivery platforms, delivery accounts, delivery plans, delivery targets.

In some embodiments, respective contributions of the respective dimensions are determined for respective ones of the at least one index.

In some embodiments, the dimension contribution determination module is further configured to: a contribution degree for one dimension of the one index is determined based on a first evaluation value obtained in the one dimension of the plurality of dimensions for the one index and a second evaluation value obtained in the plurality of dimensions.

In some embodiments, the anomaly is associated with a change in a ratio of the first and second evaluation values, and the dimension contribution determination module is further configured to: determining a contribution of the change in the first evaluation value to the change in the value; and determining a degree of contribution of the change in the second evaluation value to the change in the value.

In some embodiments, the anomaly is associated with a change in the second evaluation value, wherein the dimension contribution determination module is further configured to: determining a contribution degree of the change of the first evaluation value to the change of the second evaluation value; and determining a degree of contribution of the change in the ratio of the first evaluation value to the second evaluation value to the change in the second evaluation value.

In some embodiments, the at least one indicator comprises a plurality of indicators, the apparatus 500 further comprising: an index contribution determination module configured to determine a respective contribution of each of the plurality of indices to the anomaly; and a key indicator determination module configured to determine a key indicator from the plurality of indicators based on a comparison of the respective contribution of each indicator to the respective threshold contribution.

In some embodiments, the plurality of metrics includes input-to-output ratios, and the anomaly alarm module 520 is further configured to: based on a comparison of the determined contribution of revenue and/or yield to the anomaly to the corresponding threshold contribution, it is determined that the revenue and/or yield is abnormal.

In some embodiments, the anomaly alarm module 520 is further configured to: responsive to determining that the revenue generation anomaly, determining that the customer retention anomaly occurred based on a comparison of the customer retention difference rate to a first threshold difference rate; and/or determining a revenue generation anomaly based on a comparison of the revenue rate difference to a second threshold difference rate.

In some embodiments, the anomaly alarm module 520 is further configured to: in response to determining that the yield is abnormal, determining that the at least one cost item is abnormal based on a comparison of the rate of difference of the at least one cost item to a third threshold rate of difference.

In some embodiments, the at least one indicator is associated with at least one of: new customer number, customer retention, input-to-output ratio, cold start.

In some embodiments, the at least one exception type includes at least one of: absolute anomalies, relative anomalies, trend anomalies.

It should be appreciated that the features and operations and corresponding effects of the electronic device 110 referred to in the method 200 and process 300 discussed above with reference to fig. 1-4 are equally applicable to the apparatus 500 and are not described in detail herein.

The elements included in apparatus 500 may be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units may be implemented using software and/or firmware, such as machine executable instructions stored on a storage medium. In addition to or in lieu of machine-executable instructions, some or all of the elements in apparatus 500 may be at least partially implemented by one or more hardware logic components. By way of example and not limitation, exemplary types of hardware logic components that can be used include Field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standards (ASSPs), systems On Chip (SOCs), complex Programmable Logic Devices (CPLDs), and the like.

Fig. 6 illustrates a block diagram of an electronic device 600 in which one or more embodiments of the disclosure may be implemented. It should be understood that the electronic device 600 illustrated in fig. 6 is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. The electronic device 600 shown in fig. 6 may be used to implement the electronic device 110 of fig. 1.

As shown in fig. 6, the electronic device 600 is in the form of a general-purpose electronic device. The components of electronic device 600 may include, but are not limited to, one or more processors or processing units 610, memory 620, storage 630, one or more communication units 640, one or more input devices 650, and one or more output devices 660. The processing unit 610 may be an actual or virtual processor and is capable of performing various processes according to programs stored in the memory 620. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to increase the parallel processing capabilities of electronic device 600.

The electronic device 600 typically includes a number of computer storage media. Such a medium may be any available media that is accessible by electronic device 600, including, but not limited to, volatile and non-volatile media, removable and non-removable media. The memory 620 may be volatile memory (e.g., registers, cache, random Access Memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 630 may be a removable or non-removable media and may include machine-readable media such as flash drives, magnetic disks, or any other media that may be capable of storing information and/or data (e.g., training data for training) and may be accessed within electronic device 600.

The electronic device 600 may further include additional removable/non-removable, volatile/nonvolatile storage media. Although not shown in fig. 6, a magnetic disk drive for reading from or writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk may be provided. In these cases, each drive may be connected to a bus (not shown) by one or more data medium interfaces. Memory 620 may include a computer program product 625 having one or more program modules configured to perform the various methods or acts of the various embodiments of the disclosure.

The communication unit 640 enables communication with other electronic devices through a communication medium. Additionally, the functionality of the components of the electronic device 600 may be implemented in a single computing cluster or in multiple computing machines capable of communicating over a communication connection. Thus, the electronic device 600 may operate in a networked environment using logical connections to one or more other servers, a network Personal Computer (PC), or another network node.

The input device 650 may be one or more input devices such as a mouse, keyboard, trackball, etc. The output device 660 may be one or more output devices such as a display, speakers, printer, etc. The electronic device 600 may also communicate with one or more external devices (not shown), such as storage devices, display devices, etc., with one or more devices that enable a user to interact with the electronic device 600, or with any device (e.g., network card, modem, etc.) that enables the electronic device 600 to communicate with one or more other electronic devices, as desired, via the communication unit 640. Such communication may be performed via an input/output (I/O) interface (not shown).

According to an exemplary implementation of the present disclosure, a computer-readable storage medium having stored thereon computer-executable instructions, wherein the computer-executable instructions are executed by a processor to implement the method described above is provided. According to an exemplary implementation of the present disclosure, there is also provided a computer program product tangibly stored on a non-transitory computer-readable medium and comprising computer-executable instructions that are executed by a processor to implement the method described above.

Various aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus, devices, and computer program products implemented according to the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.

These computer readable program instructions may be provided to a processing unit of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processing unit of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable medium having the instructions stored therein includes an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various implementations of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The foregoing description of implementations of the present disclosure has been provided for illustrative purposes, is not exhaustive, and is not limited to the implementations disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the various implementations described. The terminology used herein was chosen in order to best explain the principles of each implementation, the practical application, or the improvement of technology in the marketplace, or to enable others of ordinary skill in the art to understand each implementation disclosed herein.

Claims (17)

1. A method for evaluating data placement, comprising:

receiving a first selection by a user of at least one indicator for evaluating the data delivery and a second selection of at least one anomaly type for evaluating the data delivery;

acquiring user configuration information for evaluating the data delivery based on the first selection and the second selection of the user, wherein the user configuration information at least comprises at least one index to be evaluated and at least one abnormality type; and

based on a comparison of the evaluation result for the at least one indicator with a respective threshold value, the abnormality of the data delivery is alerted, wherein the respective threshold value of the at least one indicator is associated with the at least one abnormality type.

2. The method of claim 1, wherein the evaluation of the at least one indicator is obtained based on periodic evaluation of the data delivery.

3. The method of claim 1, further comprising:

determining respective contribution degrees of the delivery of the data to the anomaly in each of a plurality of dimensions; and

a critical dimension is determined from the plurality of dimensions based on a comparison of the respective contribution of the respective dimension to a respective threshold contribution.

4. A method according to claim 3, wherein the plurality of dimensions comprises at least one of: the delivery of the data is used for recommended resources, delivery platforms, delivery accounts, delivery plans, delivery targets.

5. A method according to claim 3, wherein the respective contribution of the respective dimension is determined for each of the at least one index.

6. The method of claim 5, wherein determining the respective contribution of impressions of the data to the anomaly over each of the plurality of dimensions comprises:

a contribution degree for one of the at least one index is determined based on a first evaluation value obtained in the one of the plurality of dimensions and a second evaluation value obtained in the plurality of dimensions for the one index.

7. The method of claim 6, wherein the anomaly is associated with a change in a ratio of the first and second evaluation values, and wherein determining the contribution to the one dimension of the one indicator comprises:

determining a contribution of the variation of the first evaluation value to the variation of the ratio; and

a degree of contribution of the change in the second evaluation value to the change in the ratio is determined.

8. The method of claim 6, wherein the anomaly is associated with a change in the second evaluation value, wherein determining the contribution to the one dimension of the one indicator comprises:

determining a contribution of the change of the first evaluation value to the change of the second evaluation value; and

a degree of contribution of a change in the ratio of the first evaluation value to the second evaluation value to a change in the second evaluation value is determined.

9. The method of claim 1, wherein the at least one indicator comprises a plurality of indicators, and the method further comprises:

determining respective contributions of each of the plurality of indicators to the anomaly; and

a key indicator is determined from the plurality of indicators based on a comparison of the respective contribution of the respective indicators to a respective threshold contribution.

10. The method of claim 9, wherein the plurality of metrics comprises input-to-output ratios, and wherein alerting the anomaly comprises:

based on a comparison of the determined contribution of revenue and/or yield to the anomaly to the corresponding threshold contribution, it is determined that an anomaly occurred in revenue and/or yield.

11. The method of claim 10, wherein alerting the anomaly further comprises:

in response to determining that the revenue is abnormal,

determining that the customer retention is abnormal based on a comparison of the customer retention difference rate and a first threshold difference rate; and/or

Based on the comparison of the revenue rate difference to the second threshold rate difference, a revenue generation anomaly is determined.

12. The method of claim 10, wherein alerting the anomaly further comprises:

in response to determining that the yield is abnormal, determining that the at least one cost item is abnormal based on a comparison of a difference rate of the at least one cost item to a third threshold difference rate.

13. The method of claim 1, wherein the at least one indicator is associated with at least one of: new customer number, customer retention, input-to-output ratio, cold start.

14. The method of claim 1, wherein the at least one anomaly type comprises at least one of: absolute anomalies, relative anomalies, trend anomalies.

15. An apparatus for evaluating data placement, comprising:

a receiving module configured to receive a first selection by a user of at least one indicator for evaluating the data impression and a second selection of at least one anomaly type for evaluating the data impression;

an information acquisition module configured to acquire user configuration information for evaluating the data delivery, based on the first selection and the second selection of the user, the user configuration information including at least one index to be evaluated and at least one anomaly type; and

an anomaly alarm module configured to alarm for anomalies in the data feed based on a comparison of the evaluation result for the at least one indicator with a respective threshold value, wherein the respective threshold value of the at least one indicator is associated with the at least one anomaly type.

16. An electronic device, comprising:

at least one processing unit; and

at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, which when executed by the at least one processing unit, cause the electronic device to perform the method of any one of claims 1 to 14.

17. A computer readable storage medium having stored thereon a computer program executable by a processor to implement the method of any of claims 1 to 14.