CN112819491B - Method and device for converting data processing, electronic equipment and storage medium - Google Patents
Method and device for converting data processing, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN112819491B CN112819491B CN201911120892.8A CN201911120892A CN112819491B CN 112819491 B CN112819491 B CN 112819491B CN 201911120892 A CN201911120892 A CN 201911120892A CN 112819491 B CN112819491 B CN 112819491B
- Authority
- CN
- China
- Prior art keywords
- data
- index value
- index
- value sequence
- conversion
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012545 processing Methods 0.000 title claims abstract description 66
- 238000000034 method Methods 0.000 title claims abstract description 33
- 238000003860 storage Methods 0.000 title claims abstract description 17
- 238000006243 chemical reaction Methods 0.000 claims abstract description 134
- 230000002159 abnormal effect Effects 0.000 claims abstract description 104
- 238000001514 detection method Methods 0.000 claims abstract description 60
- 238000010606 normalization Methods 0.000 claims abstract description 51
- 230000015654 memory Effects 0.000 claims description 20
- 238000009826 distribution Methods 0.000 claims description 14
- 230000005856 abnormality Effects 0.000 claims description 7
- 238000005516 engineering process Methods 0.000 abstract description 3
- 238000004422 calculation algorithm Methods 0.000 description 9
- 238000004364 calculation method Methods 0.000 description 7
- 238000004891 communication Methods 0.000 description 4
- 238000004590 computer program Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000006399 behavior Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000002123 temporal effect Effects 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001151 other effect Effects 0.000 description 1
- 238000005316 response function Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0241—Advertisements
- G06Q30/0242—Determining effectiveness of advertisements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2474—Sequence data queries, e.g. querying versioned data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06393—Score-carding, benchmarking or key performance indicator [KPI] analysis
Abstract
The application discloses a method, a device, electronic equipment and a storage medium for conversion data processing, and relates to the field of data processing, in particular to a conversion data processing technology in target conversion bidding. The specific implementation scheme is as follows: normalizing the conversion data of various types according to the index parameters to obtain normalized data, wherein the normalized data comprises an index value sequence based on time; judging whether the index value sequence has abnormal data or not according to the time weight; if abnormal data exists, the abnormal data is deleted from the index value sequence. The normalization processing can convert various types of conversion data into normalization data with the same data structure, and abnormal data detection can be carried out on the index value sequence contained in the normalization data based on time weight, so that the accuracy of abnormal data detection can be improved.
Description
Technical Field
The present application relates to data processing technology, and in particular, to internet advertisement conversion data processing technology.
Background
Target conversion bids (optimization cost per click, opcc) are used in internet advertising to calculate rates of return. In the implementation process of the target conversion bidding function, statistics and abnormal data detection are required to be carried out on conversion data. However, when heterogeneous data sources are processed, the detection accuracy of abnormal data is poor.
Disclosure of Invention
The embodiment of the application provides a method, a device, electronic equipment and a storage medium for processing conversion data, which can improve the detection accuracy of abnormal data when heterogeneous data sources are processed.
The embodiment of the application provides a method for processing conversion data, which comprises the following steps:
normalizing the conversion data of various types according to the index parameters to obtain normalized data, wherein the normalized data comprises an index value sequence based on time;
judging whether the index value sequence has abnormal data or not according to the time weight;
if abnormal data exists, the abnormal data is deleted from the index value sequence.
According to the embodiment of the application, various types of conversion data can be normalized according to the index parameters to obtain normalized data, wherein the normalized data comprises an index value sequence based on time; judging whether the index value sequence has abnormal data or not according to the time weight; if abnormal data exists, the abnormal data is deleted from the index value sequence. The normalization processing can convert various types of conversion data into normalization data with the same data structure, and abnormal data detection can be carried out on the index value sequence contained in the normalization data based on time weight, so that the accuracy of abnormal data detection can be improved.
In an embodiment of the above application, normalizing the multiple types of conversion data according to the index parameter to obtain normalized data includes:
acquiring a user identification, a conversion type and an acquisition type of conversion data;
and carrying out normalization processing on the conversion data of various types according to the user identification, the conversion type and the acquisition type to obtain normalized data.
In the embodiment of the application, multiple types of conversion data can be classified according to the user identifier, the conversion type and the acquisition type. And carrying out normalization processing on the conversion data according to the user identification, the conversion type and the acquisition mode of the conversion data, so as to realize the normalization processing of the heterogeneous data source. By recording the user identification, conversion type and acquisition type, the conversion data can be marked more accurately.
In one embodiment of the above application, normalizing the multiple types of conversion data according to the user identifier, the conversion type and the acquisition type to obtain normalized data, where the normalizing includes:
configuring a data unit for the user identifier;
the data unit comprises one or more incidence relations between the acquisition type and the conversion type, and the incidence relations are associated with one or more delivery package information; the delivery package information includes an indicator identification and a time-based indicator value sequence associated with the indicator identification.
In the embodiment of the application, the data structure of the data unit is used to store multiple types of conversion data of one user, and multiple types of delivery package information can be determined according to the association relationship between the acquisition type and the conversion type of the conversion data. Each delivery package information comprises an index value sequence under an index, wherein the index value sequences are ordered based on time, and the index values at different time points are stored. And further, a data structure for locating the index value sequence according to the client index, the acquisition mode, the conversion type and the index identification is realized, and the conversion data of different types can be normalized through the data structure, so that the normalization efficiency is improved.
In one embodiment of the above application, determining whether the index value sequence has abnormal data according to the time weight includes:
calculating a hash value of the normalized data according to the target features contained in the normalized data;
determining a hash bucket where the normalized data are located according to the hash value;
judging whether the index value sequence contained in the normalized data in each hash bucket has abnormal data or not according to the time weight judgment.
In the embodiment of the application, before the normalized data is detected, a hash value is obtained according to the target feature of the normalized data, the normalized data is divided into buckets according to the hash value, and then abnormal data detection is carried out on the normalized data in each bucket, so that the normalized data is divided based on the hash value, and the abnormal detection efficiency is improved.
In an embodiment of the foregoing application, determining whether there is abnormal data in an index value sequence included in the normalized data in each hash bucket according to a time weight determination includes:
processing abnormal data detection in a plurality of hash buckets in parallel;
in each hash bucket, sequentially reading the index value sequence in the hash bucket according to the serial sequence;
judging whether abnormal data exists in the index sequence.
In the embodiment of the application, parallel computation can be performed among all hash buckets, so that high-concurrency anomaly detection is realized. Meanwhile, each index value sequence is sequentially processed in a serial mode in each hash bucket, so that the data processing efficiency is further improved.
In one embodiment of the above application, determining whether the index value sequence has abnormal data according to the time weight includes:
weighting each index value in the index value sequence according to the time weight to obtain the average value of the index value sequence, wherein the average value is the average or median of the index sequence;
calculating standard deviation or absolute middle potential difference of the index value sequence;
traversing the index value sequence according to the mean value and the standard deviation, and determining the residual error with the maximum difference with the mean value;
calculating a critical value of the distribution of the index value sequence t;
And determining whether an abnormal point exists according to the critical value and the standard deviation.
In the embodiment of the application, for an index value sequence, weighting each index value in the index value sequence according to time weight to obtain a mean value of the index value sequence; traversing each index value in the index value sequence according to the mean value and the standard deviation, determining a residual error with the maximum difference from the mean value, and determining an abnormal point according to the critical value of t distribution of the index sequence and the residual error. And further, the average value of the index value sequence is determined based on the time information of the index value, and the abnormal point in the index value sequence is determined based on the average value, so that the abnormal point can be determined more accurately based on the time effectiveness, and the abnormal point detection efficiency is improved.
In one embodiment of the above application, weighting each index value in the sequence of index values according to a temporal weight comprises:
determining a weighting parameter of the index value according to the time interval between the acquisition time of the index value and the current time, wherein the weighting parameter is smaller than 1, and the numerical value of the weighting parameter and the length of the time interval have inverse proportion trend;
the index values are weighted according to the weighting parameters.
In the embodiment of the application, the weighting parameters of the index values are determined according to the time interval between the acquisition time and the current time of the index values, so that a weighting mode that the values of the weighting parameters and the length of the time interval have inverse proportion trend can be realized, and further the accuracy of weighting calculation is improved.
The embodiment of the application also provides a device for converting data processing, which comprises:
the normalization module is used for carrying out normalization processing on various types of conversion data according to the index parameters to obtain normalized data, wherein the normalized data comprises an index value sequence based on time;
the anomaly detection module is used for judging whether the index value sequence has anomaly data according to the time weight;
and the exception processing module is used for deleting the exception data from the index value sequence if the exception data exists.
The embodiment of the application also provides a device for converting data processing, which comprises:
the normalization module is used for carrying out normalization processing on various types of conversion data according to the index parameters to obtain normalized data, wherein the normalized data comprises an index value sequence based on time;
the anomaly detection module is used for judging whether the index value sequence has anomaly data according to the time weight;
and the exception processing module is used for deleting the exception data from the index value sequence if the exception data exists.
The embodiment of the application also provides electronic equipment, which comprises:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the methods of the embodiments described above.
The present application also provides a non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the method of the above embodiments.
Other effects of the above alternative will be described below in connection with specific embodiments.
Drawings
The drawings are for better understanding of the present solution and do not constitute a limitation of the present application. Wherein:
FIG. 1 is a schematic flow chart according to a first embodiment of the present application;
FIG. 2 is a schematic flow chart according to a second embodiment of the present application;
FIG. 3 is a schematic flow chart according to a third embodiment of the present application;
fig. 4 is a schematic structural view according to a fourth embodiment of the present application;
fig. 5 is a block diagram of an electronic device for implementing a method of converting data processing according to an embodiment of the present application.
Detailed Description
Exemplary embodiments of the present application are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present application to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Example 1
Fig. 1 is a flow chart of a method for converting data processing, which is applicable to fluctuation detection of multi-source heterogeneous data in target conversion bidding (optimization cost per click, opcc), and may be executed by an electronic device such as a server or a terminal, and the method may be implemented by the following manner:
and 101, carrying out normalization processing on various types of conversion data according to index parameters to obtain normalized data, wherein the normalized data comprises an index value sequence based on time.
The transformation data are classified into different categories according to the collection type and the transformation type. The acquisition type and the conversion type appear in groups, and when an index value of interest is generated, conversion data is generated by conversion data acquisition equipment such as a mobile terminal.
The collection types of the conversion data of various types are used for representing the collection mode of the conversion data, and the collection types can comprise Application programming interface (Application Programming Interface, API) collection, loading JS (JavaScript) code collection in a landing page, business dialogue platform (Business Conversation Platform) collection, application (APP) activation collection, SDK code collection in an applet and the like.
The conversion type of the conversion data may be a type of operation that is deemed to produce the conversion, such as a user triggering a target action in a search page or in a landing page. The target behavior may be consulting a seek or submit form, etc.
The conversion data of various types are also called heterogeneous data sources, and normalization processing is performed on the conversion data carried by the heterogeneous data sources, so that index value sequences are included in the obtained normalization data. The index value sequence is also called a time sequence, and each element in the index value sequence (or time sequence) includes two pieces of time information and an index value. The time information indicates the time at which the index value is generated, the index value being a user behavior parameter of interest.
Optionally, acquiring a user identifier, a conversion type and an acquisition type of the conversion data; and carrying out normalization processing on the conversion data of various types according to the user identification, the conversion type and the acquisition type to obtain normalized data.
According to the acquisition mode of acquiring the conversion data, for example, an interface for receiving the conversion data can determine the acquisition type, and according to the index value type carried by the conversion data, the conversion type can be determined. And extracting the user identification, the conversion type and the acquisition type from the conversion data, and traversing the conversion data to obtain an index value sequence.
In the embodiment of the application, multiple types of conversion data can be classified according to the user identifier, the conversion type and the acquisition type. And carrying out normalization processing on the conversion data according to the user identification, the conversion type and the acquisition mode of the conversion data, so as to realize the normalization processing of the heterogeneous data source. By recording the user identification, conversion type and acquisition type, the conversion data can be marked more accurately.
Specifically, a data unit is configured for the user identifier;
the data unit comprises one or more incidence relations between the acquisition type and the conversion type, and the incidence relations are associated with one or more delivery package information; the delivery package information includes an indicator identification and a time-based indicator value sequence associated with the indicator identification.
In an internet promotion platform, each account may access data of one or more conversion types (marking a targeted behavior that an advertiser desires to optimize) using one or more collection means. A user identification is used to mark an account. Embodiments of the present application provide a new data structure in which a data unit (also called unit) is configured for an account. Each data unit has one or more drop package information (also known as cells). One data unit may be configured for each account, e.g., if there are N accounts represented by N user identities, N is a positive integer. Alternatively, one data unit may be configured for a portion of the accounts, e.g., if there are N accounts represented by N user identities, then M data units are configured, one data unit for each of the M accounts, M being a non-negative integer less than N.
Each piece of information of the delivery package is configured with an association relation between the collection type and the conversion type, and the association relation indicates what collection mode the information of the delivery package is obtained and what conversion type index value is recorded. Multiple pieces of package information can be configured for the same user identifier (i.e., account), so that aggregation of conversion data is realized in a fixed time period. The information of the put package is an index value sequence (also called a time sequence) acquired according to the corresponding conversion type and acquisition type.
As shown in table 1, a user identifier corresponds to a data unit, and the data unit includes two association relations, which are respectively: acquisition type 1 is used to access conversion type 1 and acquisition type 2 is used to access conversion type 2. The acquisition type 1-conversion type 1 comprises two pieces of throwing package information, and the throwing package information respectively comprises an index 1-1 and an associated index value sequence and an index 1-2 and an associated index value sequence. The acquisition type 2-conversion type 2 comprises two pieces of put package information, and the two pieces of put package information respectively comprise an index 2-1 and an associated index value sequence and an index 2-2 and an associated index value sequence.
TABLE 1
In the embodiment of the application, the data structure of the data unit is used to store multiple types of conversion data of one user, and multiple types of delivery package information can be determined according to the association relationship between the acquisition type and the conversion type of the conversion data. Each delivery package information comprises an index value sequence under an index, wherein the index value sequences are ordered based on time, and the index values at different time points are stored. And further, a data structure for locating the index value sequence according to the client index, the acquisition mode, the conversion type and the index identification is realized, and the conversion data of different types can be normalized through the data structure, so that the normalization efficiency is improved.
Step 102, judging whether the index value sequence has abnormal data according to the time weight.
And respectively configuring time weight for each index value in the index sequence, and calculating to obtain the average value of the index value sequence. And judging abnormal data in the index sequence according to the average value and the index value in the index sequence, and finishing fluctuation detection.
Step 103, if abnormal data exists, deleting the abnormal data from the index value sequence.
If the abnormal data exists, the abnormal data is deleted from the index value sequence, and whether the execution returns to step 102 is judged. If no abnormal data exists, the index value sequence does not contain the abnormal data.
Illustratively, it is determined whether the number of the abnormal data that has been determined satisfies the preset number of the abnormal data, or it is performed whether the number of times of presence of the abnormal data exceeds the preset number of cycles. If the number of the abnormal data is smaller than the number of the preset abnormal data, or the number of times of judging whether the abnormal data exist is not more than the preset cycle number, the step 102 is circularly executed. If the number of the abnormal data is greater than or equal to the preset number of the abnormal data, or the number of times of judging whether the abnormal data exist exceeds the preset cycle number, the execution returns to the step 102.
According to the embodiment of the application, various types of conversion data can be normalized according to the index parameters to obtain normalized data, wherein the normalized data comprises an index value sequence based on time; judging whether the index value sequence has abnormal data or not according to the time weight; if abnormal data exists, the abnormal data is deleted from the index value sequence. The normalization processing can convert various types of conversion data into normalization data with the same data structure, and abnormal data detection can be carried out on the index value sequence contained in the normalization data based on time weight, so that the accuracy of abnormal data detection can be improved.
Example two
Fig. 2 is a flow chart of a method for converting data according to a second embodiment of the present application, which further illustrates the above embodiment, and includes:
step 201, performing normalization processing on multiple types of conversion data according to index parameters to obtain normalized data, wherein the normalized data comprises an index value sequence based on time.
Step 202, calculating a hash value of the normalized data according to the target feature contained in the normalized data.
The target feature may be a user identification or an index value in the normalized data, etc. The hash value is calculated according to the user identification, and normalized data of the user identification with the same hash value can be counted in the same hash bucket. According to the index value calculation hash value, normalized data of index values with the same hash value can be obtained in the same hash bucket.
Step 203, determining a hash bucket where the normalized data is located according to the hash value.
And 204, judging whether the index value sequence contained in the normalized data in each hash bucket has abnormal data or not according to the time weight.
Optionally, the abnormal data detection in the hash buckets is processed in parallel; in each hash bucket, sequentially reading the index value sequence in the hash bucket according to the serial sequence; judging whether abnormal data exists in the index sequence.
After hash buckets are performed according to the target features, a plurality of index value sequences are stored in each hash bucket. The processing object of fluctuation detection is one or a plurality of index value sequences. Alternatively, in a hash bucket, the index value sequences may be sequentially read in a certain order, and each index value sequence is read. Judging whether abnormal data exists in the read index sequence. The order may be the first address order of the sequence of index values in the storage space or may be the storage order of the sequence of index values in the data structure described above.
In the embodiment of the application, when detecting abnormal data of the data in each hash bucket, the data in the hash bucket is split into a size that can be loaded into the memory for processing with the data unit as the minimum granularity of each processing. The data in the hash bucket is stored on disk. And performing parallel fluctuation detection among the hash buckets. And performing parallel computation among all hash buckets to realize high-concurrency anomaly detection. Meanwhile, the index value sequence is sequentially processed in a serial mode in each hash bucket, so that the random input and output times can be reduced, the program concurrency is increased, and the fluctuation detection efficiency is improved.
Step 205, if there is abnormal data, deleting the abnormal data from the index value sequence.
In the embodiment of the application, before the normalized data is detected, a hash value is obtained according to the target feature of the normalized data, the normalized data is divided into buckets according to the hash value, and then abnormal data detection is carried out on the normalized data in each bucket, so that the normalized data is divided based on the hash value, and the abnormal detection efficiency is improved.
Example III
Fig. 3 is a flow chart of a method for converting data according to the third embodiment of the present application, which further illustrates the above embodiment, and includes:
step 301, performing normalization processing on multiple types of conversion data according to index parameters to obtain normalized data, wherein the normalized data comprises an index value sequence based on time.
Step 302, weighting each index value in the index value sequence according to the time weight to obtain a mean value of the index value sequence, wherein the mean value is the average or median of the index sequence.
In an actual popularization platform, conversion data can generate expected fluctuation due to change of landing pages of advertisers or change of delivery plans, and at the moment, the reference meaning of long-term historical data is not great and misjudgment is easy to cause, so that an abnormality detection algorithm is required to be capable of adapting to the latest data distribution more timely. Because each customer has different fluctuation trend of conversion data, the unified fluctuation threshold cannot be applied to all the situations of customers, which can lead to erroneous judgment or missed judgment of fluctuation detection. The embodiment of the application uses a weighted limit standard deviation algorithm to detect fluctuation based on customer data statistics and distribution characteristics. The basic principle of the limit standard deviation algorithm is that firstly, no abnormal value exists in the data set, then, the value (the maximum value or the minimum value) which deviates from the mean value to the maximum value is gradually deleted in the data set, and the corresponding t distribution critical value (used for checking whether the assumption is true) is synchronously updated until the assumption is true or the number of abnormal values exceeds the set k value. The embodiment of the application introduces a time weight beta parameter based on a limit standard deviation algorithm (ESD algorithm) for controlling the time weight, and can be called a weighted limit standard deviation algorithm (W-ESD algorithm).
Optionally, determining a weighted parameter of the index value according to a time interval between the acquisition time and the current time of the index value, wherein the weighted parameter is smaller than 1, and the numerical value of the weighted parameter has an inverse proportion trend with the length of the time interval; the index values are weighted according to the weighting parameters.
The mean value can be calculated by the following formula
Wherein the beta parameter is used to control the importance of the temporal feature in calculating the mean, i.e. the weighting parameter. The closer the beta parameter is to 0, the averageThe more the value is biased toward the index value of the latest time. T (T) 1 ,T 2 …T i …T n Is the index value corresponding to each time information in the index value sequence. N is the number of index values contained in the index value sequence. />The average value of the index value sequence may be represented or the median of the index value sequence may be represented.
In the embodiment of the application, the weighting parameters of the index values are determined according to the time interval between the acquisition time and the current time of the index values, so that a weighting mode that the values of the weighting parameters and the length of the time interval have inverse proportion trend can be realized, and further the accuracy of weighting calculation is improved.
Step 303, calculating the standard deviation or the absolute middle bit difference s of the index value sequence.
When (when)S represents the standard deviation when representing the average value; when->S represents the absolute median difference, abbreviated MAD (for the sequence T, mad=media (|ti-media (T) |))
And 304, traversing the index value sequence according to the mean value and the standard deviation, and determining the residual error with the largest difference with the mean value.
Residual R with maximum difference from mean value j The calculation formula of (2) is as follows:
wherein,representing the average value of the index value sequence; t (T) i Is a certain index value in the index value sequence currently traversed. K is the number of the preset abnormal data.
Step 305, calculating a threshold value of the index value sequence t distribution.
Critical value lambda j The calculation formula of (2) is as follows:
wherein t is p,n-j-1 The right-hand threshold of the t distribution, representing degrees of freedom n-j+1 and significance p.
Step 306, determining whether an abnormal point exists according to the critical value and the standard deviation.
If the difference from the mean value is maximum, the residual R j Greater than a critical value lambda j Then determine the fingerThe scalar value is the outlier. Otherwise, if the difference from the mean value is the largest, the residual R j Less than a critical value lambda j And indicating that the index value queue has no abnormal data.
In the embodiment of the application, for an index value sequence, weighting each index value in the index value sequence according to time weight to obtain a mean value of the index value sequence; traversing each index value in the index value sequence according to the mean value and the standard deviation, determining a residual error with the maximum difference from the mean value, and determining an abnormal point according to the critical value of t distribution of the index sequence and the residual error. And further, the average value of the index value sequence is determined based on the time information of the index value, and the abnormal point in the index value sequence is determined based on the average value, so that the abnormal point can be determined more accurately based on the time effectiveness, and the abnormal point detection efficiency is improved. The fluctuation detection algorithm provided by the embodiment of the application can support the detection of fluctuation at a plurality of index latitudes, is used as a reference basis for fluctuation attribution, and can more effectively locate the problem. The fluctuation detection is carried out by adopting time weight, so that the fluctuation detection system can self-adaptively relax or tighten the abnormal judgment standard according to the historical fluctuation condition of the client; the beta parameter is introduced, so that an application party can adjust the importance of the latest time data distribution in an anomaly detection algorithm according to a specific application scene, and the accuracy and flexibility of fluctuation detection are improved.
Example IV
Fig. 4 is a schematic structural diagram of a device 400 for converting data processing, which is provided in the fourth embodiment of the present application, and is configured to execute the manner shown in the foregoing embodiment to implement a corresponding response function, where the device may be applied to an electronic device such as a server or a terminal, and includes: normalization module 401, anomaly detection module 402, and anomaly handling module 403. Wherein:
the normalization module 401 is configured to normalize multiple types of conversion data according to an index parameter to obtain normalized data, where the normalized data includes an index value sequence based on time;
an anomaly detection module 402, configured to determine whether the index value sequence has anomaly data according to the time weight;
the exception handling module 403 is configured to delete the exception data from the index value sequence if the exception data exists.
In the above embodiment of the present application, the normalization module 401 performs normalization processing on multiple types of conversion data according to the index parameter to obtain normalized data, where the normalized data includes an index value sequence based on time; the anomaly detection module 402 judges whether the index value sequence has anomaly data according to the time weight; if there is abnormal data, the abnormality processing module 403 deletes the abnormal data from the index value sequence. The normalization processing can convert various types of conversion data into normalization data with the same data structure, and abnormal data detection can be carried out on the index value sequence contained in the normalization data based on time weight, so that the accuracy of abnormal data detection can be improved.
On the basis of the above embodiment, the normalization module 401 is configured to:
acquiring a user identification, a conversion type and an acquisition type of conversion data;
and carrying out normalization processing on the conversion data of various types according to the user identification, the conversion type and the acquisition type to obtain normalized data.
In the above application embodiment, the normalization module 401 can classify multiple types of conversion data according to the user identifier, the conversion type, and the acquisition type. And carrying out normalization processing on the conversion data according to the user identification, the conversion type and the acquisition mode of the conversion data, so as to realize the normalization processing of the heterogeneous data source. By recording the user identification, conversion type and acquisition type, the conversion data can be marked more accurately.
On the basis of the above embodiment, the normalization module 401 is configured to:
configuring a data unit for each user identity;
the data unit comprises one or more incidence relations between the acquisition type and the conversion type, and the incidence relations are associated with one or more delivery package information; the delivery package information includes an indicator identification and a time-based indicator value sequence associated with the indicator identification.
In the above application embodiment, the normalization module 401 uses the data structure of the data unit to store multiple types of conversion data of one user, and according to the association relationship between the collection type and the conversion type of the conversion data, multiple types of package information can be determined. Each delivery package information comprises an index value sequence under an index, wherein the index value sequences are ordered based on time, and the index values at different time points are stored. And further, a data structure for locating the index value sequence according to the client index, the acquisition mode, the conversion type and the index identification is realized, and the conversion data of different types can be normalized through the data structure, so that the normalization efficiency is improved.
On the basis of the above embodiment, the abnormality detection module 402 is configured to:
calculating a hash value of the normalized data according to the target features contained in the normalized data;
determining a hash bucket where the normalized data are located according to the hash value;
judging whether the index value sequence contained in the normalized data in each hash bucket has abnormal data or not according to the time weight judgment.
In the embodiment of the application, before detecting the normalized data, the anomaly detection module 402 obtains the hash value according to the target feature of the normalized data, and segments the normalized data according to the hash value, and then detects the abnormal data of the normalized data in each segment, thereby implementing segmentation of the normalized data based on the hash value and improving the anomaly detection efficiency.
On the basis of the above embodiment, the abnormality detection module 402 is configured to:
processing abnormal data detection in a plurality of hash buckets in parallel;
in each hash bucket, sequentially reading an index value sequence in the hash bucket according to a serial sequence;
judging whether abnormal data exists in the index sequence.
In the embodiment of the application, the anomaly detection module 402 may perform parallel computation between hash buckets, so as to implement high-concurrency anomaly detection. Meanwhile, each index value sequence is sequentially processed in a serial mode in each hash bucket, so that the data processing efficiency is further improved.
On the basis of the above embodiment, the abnormality detection module 402 is configured to:
weighting each index value in the index value sequence according to the time weight to obtain the average value of the index value sequence, wherein the average value is the average or median of the index sequence;
calculating standard deviation or absolute middle potential difference of the index value sequence;
traversing the index value sequence according to the mean value and the standard deviation, and determining the residual error with the maximum difference with the mean value;
calculating a critical value of the distribution of the index value sequence t;
and determining whether an abnormal point exists according to the critical value and the standard deviation.
In the above embodiment, for one index value sequence, the anomaly detection module 402 weights each index value in the index value sequence according to a time weight to obtain a mean value of the index value sequence; traversing each index value in the index value sequence according to the mean value and the standard deviation, determining a residual error with the maximum difference from the mean value, and determining an abnormal point according to the critical value of t distribution of the index sequence and the residual error. And further, the average value of the index value sequence is determined based on the time information of the index value, and the abnormal point in the index value sequence is determined based on the average value, so that the abnormal point can be determined more accurately based on the time effectiveness, and the abnormal point detection efficiency is improved.
On the basis of the above embodiment, the abnormality detection module 402 is configured to:
determining a weighting parameter of the index value according to the time interval between the acquisition time of the index value and the current time, wherein the weighting parameter is smaller than 1, and the numerical value of the weighting parameter and the length of the time interval have inverse proportion trend;
the index values are weighted according to the weighting parameters.
In the embodiment of the application, the anomaly detection module 402 determines the weighting parameter of the index value according to the time interval between the acquisition time and the current time of the index value, so as to implement a weighting mode in which the value of the weighting parameter and the length of the time interval have inverse proportion trend, thereby improving the accuracy of the weighting calculation.
Example five
According to embodiments of the present application, an electronic device and a readable storage medium are also provided.
As shown in fig. 5, a block diagram of an electronic device is provided for a method of converting data processing according to an embodiment of the present application. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the application described and/or claimed herein.
As shown in fig. 5, the electronic device includes: one or more processors 501, memory 502, and interfaces for connecting components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions executing within the electronic device, including instructions stored in or on memory to display graphical information of the GUI on an external input/output device, such as a display device coupled to the interface. In other embodiments, multiple processors and/or multiple buses may be used, if desired, along with multiple memories and multiple memories. Also, multiple electronic devices may be connected, each providing a portion of the necessary operations (e.g., as a server array, a set of blade servers, or a multiprocessor system). One processor 501 is illustrated in fig. 5.
Memory 502 is a non-transitory computer readable storage medium provided herein. The memory stores instructions executable by the at least one processor to cause the at least one processor to perform the methods of converting data processing provided herein. The non-transitory computer readable storage medium of the present application stores computer instructions for causing a computer to perform the method of converting data processing provided herein.
The memory 502 is used as a non-transitory computer readable storage medium for storing non-transitory software programs, non-transitory computer-executable programs, and modules, such as program instructions/modules (e.g., the normalization module 401, the anomaly detection module 402, and the anomaly handling module 403 of fig. 4) corresponding to the method of converting data in the embodiments of the present application. The processor 501 executes various functional applications of the server and data processing, i.e., a method of implementing the conversion data processing in the above-described method embodiment, by running non-transitory software programs, instructions, and modules stored in the memory 502.
Memory 502 may include a storage program area that may store an operating system, at least one application program required for functionality, and a storage data area; the storage data area may store data created from the use of the electronic device that converts the data processing, and the like. In addition, memory 502 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory 502 may optionally include memory located remotely from processor 501, which may be connected to the electronic device converting the data processing via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The electronic device of the method of converting data processing may further include: an input device 503 and an output device 504. The processor 501, memory 502, input devices 503 and output devices 504 may be connected by a bus or otherwise, for example in fig. 5.
The input device 503 may receive input numeric or character information and generate key signal inputs related to user settings and function control of the electronic device converting the data processing, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, a pointer stick, one or more mouse buttons, a track ball, a joystick, etc. input devices. The output devices 504 may include a display device, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors), among others. The display device may include, but is not limited to, a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, and a plasma display. In some implementations, the display device may be a touch screen.
Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, application specific ASIC (application specific integrated circuit), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
These computing programs (also referred to as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented in a high-level procedural and/or object-oriented programming language, and/or in assembly/machine language. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.
The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
According to the embodiment of the application, various types of conversion data can be normalized according to the index parameters to obtain normalized data, wherein the normalized data comprises an index value sequence based on time; judging whether the index value sequence has abnormal data or not according to the time weight; if abnormal data exists, the abnormal data is deleted from the index value sequence. The normalization processing can convert various types of conversion data into normalization data with the same data structure, and abnormal data detection can be carried out on the index value sequence contained in the normalization data based on time weight, so that the accuracy of abnormal data detection can be improved.
In the embodiment of the application, multiple types of conversion data can be classified according to the user identifier, the conversion type and the acquisition type. And carrying out normalization processing on the conversion data according to the user identification, the conversion type and the acquisition mode of the conversion data, so as to realize the normalization processing of the heterogeneous data source. By recording the user identification, conversion type and acquisition type, the conversion data can be marked more accurately.
In the embodiment of the application, the data structure of the data unit is used to store multiple types of conversion data of one user, and multiple types of delivery package information can be determined according to the association relationship between the acquisition type and the conversion type of the conversion data. Each delivery package information comprises an index value sequence under an index, wherein the index value sequences are ordered based on time, and the index values at different time points are stored. And further, a data structure for locating the index value sequence according to the client index, the acquisition mode, the conversion type and the index identification is realized, and the conversion data of different types can be normalized through the data structure, so that the normalization efficiency is improved.
In the embodiment of the application, before the normalized data is detected, a hash value is obtained according to the target feature of the normalized data, the normalized data is divided into buckets according to the hash value, and then abnormal data detection is carried out on the normalized data in each bucket, so that the normalized data is divided based on the hash value, and the abnormal detection efficiency is improved.
In the embodiment of the application, parallel computation can be performed among all hash buckets, so that high-concurrency anomaly detection is realized. Meanwhile, each index value sequence is sequentially processed in a serial mode in each hash bucket, so that the data processing efficiency is further improved.
In the embodiment of the application, for an index value sequence, weighting each index value in the index value sequence according to time weight to obtain a mean value of the index value sequence; traversing each index value in the index value sequence according to the mean value and the standard deviation, determining a residual error with the maximum difference from the mean value, and determining an abnormal point according to the critical value of t distribution of the index sequence and the residual error. And further, the average value of the index value sequence is determined based on the time information of the index value, and the abnormal point in the index value sequence is determined based on the average value, so that the abnormal point can be determined more accurately based on the time effectiveness, and the abnormal point detection efficiency is improved.
In the embodiment of the application, the weighting parameters of the index values are determined according to the time interval between the acquisition time and the current time of the index values, so that a weighting mode that the values of the weighting parameters and the length of the time interval have inverse proportion trend can be realized, and further the accuracy of weighting calculation is improved.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present application may be performed in parallel, sequentially, or in a different order, provided that the desired results of the technical solutions disclosed in the present application can be achieved, and are not limited herein.
The above embodiments do not limit the scope of the application. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application are intended to be included within the scope of the present application.
Claims (8)
1. A method of converting data processing, comprising:
normalizing the conversion data of various types according to the index parameters to obtain normalized data, wherein the normalized data comprises an index value sequence based on time;
judging whether the index value sequence has abnormal data according to the time weight, wherein the method comprises the following steps:
determining a weighted parameter of the index value according to the time interval between the acquisition time of the index value and the current time, wherein the weighted parameter is smaller than 1, and the numerical value of the weighted parameter and the length of the time interval have inverse proportion trend;
Weighting the index value according to the weighting parameter to obtain the average value of the index value sequence, wherein the average value is the average value of the index value sequence;
calculating the standard deviation of the index value sequence;
traversing the index value sequence according to the mean value and the standard deviation, and determining a residual error with the maximum difference from the mean value;
calculating a critical value of the index value sequence t distribution;
determining whether an abnormal point exists according to the critical value and the standard deviation;
if abnormal data exists, deleting the abnormal data from the index value sequence.
2. The method for processing conversion data according to claim 1, wherein the normalizing the plurality of types of conversion data according to the index parameter to obtain normalized data comprises:
acquiring a user identification, a conversion type and an acquisition type of conversion data;
and carrying out normalization processing on the conversion data of various types according to the user identification, the conversion type and the acquisition type to obtain normalized data.
3. The method for processing the conversion data according to claim 2, wherein the normalizing the conversion data of the plurality of types according to the user identifier, the conversion type and the collection type to obtain normalized data includes:
Configuring a data unit for the user identifier;
the data unit comprises one or more incidence relations between the acquisition type and the conversion type, and the incidence relations are associated with one or more delivery package information; the delivery package information comprises an index identifier and a time-based index value sequence associated with the index identifier.
4. The method of claim 1, wherein determining whether the index value sequence has abnormal data according to the time weight comprises:
calculating a hash value of the normalized data according to the target features contained in the normalized data;
determining a hash bucket where the normalized data are located according to the hash value;
judging whether the index value sequence contained in the normalized data in each hash bucket has abnormal data or not according to the time weight.
5. The method for converting data according to claim 4, wherein said determining whether the index value sequence included in the normalized data in each hash bucket has abnormal data according to the time weight determination comprises:
processing abnormal data detection in a plurality of hash buckets in parallel;
in each hash bucket, sequentially reading a sequence of index values in the hash bucket according to a serial sequence;
And judging whether abnormal data exists in the index value sequence.
6. An apparatus for converting data processing, comprising:
the normalization module is used for carrying out normalization processing on various types of conversion data according to the index parameters to obtain normalized data, wherein the normalized data comprises an index value sequence based on time;
the abnormality detection module is used for:
determining a weighted parameter of the index value according to the time interval between the acquisition time of the index value and the current time, wherein the weighted parameter is smaller than 1, and the numerical value of the weighted parameter and the length of the time interval have inverse proportion trend;
weighting the index value according to the weighting parameter to obtain the average value of the index value sequence, wherein the average value is the average value of the index value sequence;
calculating the standard deviation of the index value sequence;
traversing the index value sequence according to the mean value and the standard deviation, and determining a residual error with the maximum difference from the mean value;
calculating a critical value of the index value sequence t distribution;
determining whether an abnormal point exists according to the critical value and the standard deviation;
and the exception processing module is used for deleting the exception data from the index value sequence if the exception data exists.
7. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.
8. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911120892.8A CN112819491B (en) | 2019-11-15 | 2019-11-15 | Method and device for converting data processing, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911120892.8A CN112819491B (en) | 2019-11-15 | 2019-11-15 | Method and device for converting data processing, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112819491A CN112819491A (en) | 2021-05-18 |
| CN112819491B true CN112819491B (en) | 2024-02-09 |
Family
ID=75851818
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911120892.8A Active CN112819491B (en) | 2019-11-15 | 2019-11-15 | Method and device for converting data processing, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112819491B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103532940A (en) * | 2013-09-30 | 2014-01-22 | 广东电网公司电力调度控制中心 | Network security detection method and device |
| CN103532776A (en) * | 2013-09-30 | 2014-01-22 | 广东电网公司电力调度控制中心 | Service flow detection method and system |
| CN109257612A (en) * | 2018-08-09 | 2019-01-22 | 广州虎牙信息科技有限公司 | Assessment Method on Potential, device, computer storage medium and server is broadcast live in game |
| CN109347653A (en) * | 2018-09-07 | 2019-02-15 | 阿里巴巴集团控股有限公司 | A kind of Indexes Abnormality discovery method and apparatus |
| CN109582485A (en) * | 2018-10-26 | 2019-04-05 | 阿里巴巴集团控股有限公司 | A kind of configuration change method for detecting abnormality and device |
| CN109740811A (en) * | 2018-12-28 | 2019-05-10 | 斑马网络技术有限公司 | Passage speed prediction technique, device and storage medium |
| CN110008080A (en) * | 2018-12-25 | 2019-07-12 | 阿里巴巴集团控股有限公司 | Operational indicator method for detecting abnormality, device and electronic equipment based on time series |
| CN110266552A (en) * | 2019-08-15 | 2019-09-20 | 华为技术有限公司 | Method, model training method and the device of Traffic anomaly detection |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8914317B2 (en) * | 2012-06-28 | 2014-12-16 | International Business Machines Corporation | Detecting anomalies in real-time in multiple time series data with automated thresholding |
| US9652354B2 (en) * | 2014-03-18 | 2017-05-16 | Microsoft Technology Licensing, Llc. | Unsupervised anomaly detection for arbitrary time series |
| US9779361B2 (en) * | 2014-06-05 | 2017-10-03 | Mitsubishi Electric Research Laboratories, Inc. | Method for learning exemplars for anomaly detection |
| US10140576B2 (en) * | 2014-08-10 | 2018-11-27 | Palo Alto Research Center Incorporated | Computer-implemented system and method for detecting anomalies using sample-based rule identification |
-
2019
- 2019-11-15 CN CN201911120892.8A patent/CN112819491B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103532940A (en) * | 2013-09-30 | 2014-01-22 | 广东电网公司电力调度控制中心 | Network security detection method and device |
| CN103532776A (en) * | 2013-09-30 | 2014-01-22 | 广东电网公司电力调度控制中心 | Service flow detection method and system |
| CN109257612A (en) * | 2018-08-09 | 2019-01-22 | 广州虎牙信息科技有限公司 | Assessment Method on Potential, device, computer storage medium and server is broadcast live in game |
| CN109347653A (en) * | 2018-09-07 | 2019-02-15 | 阿里巴巴集团控股有限公司 | A kind of Indexes Abnormality discovery method and apparatus |
| CN109582485A (en) * | 2018-10-26 | 2019-04-05 | 阿里巴巴集团控股有限公司 | A kind of configuration change method for detecting abnormality and device |
| CN110008080A (en) * | 2018-12-25 | 2019-07-12 | 阿里巴巴集团控股有限公司 | Operational indicator method for detecting abnormality, device and electronic equipment based on time series |
| CN109740811A (en) * | 2018-12-28 | 2019-05-10 | 斑马网络技术有限公司 | Passage speed prediction technique, device and storage medium |
| CN110266552A (en) * | 2019-08-15 | 2019-09-20 | 华为技术有限公司 | Method, model training method and the device of Traffic anomaly detection |
Non-Patent Citations (2)
| Title |
|---|
| 时间序列不确定数据流中异常数据检测方法;徐雪松;;电子设计工程(第19期);第19-21页 * |
| 轴系测试数据分布特征信息获取方法与应用;孙锋;刘杰;周建辉;杨梓辉;毛伟兰;;中国舰船研究(第S1期);第185-190页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112819491A (en) | 2021-05-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20220171988A1 (en) | Systems and methods for identifying processes for robotic automation and building models therefor | |
| US20200241057A1 (en) | Novel non-parametric statistical behavioral identification ecosystem for electricity fraud detection | |
| CN111738446B (en) | Scheduling method, device, equipment and medium of deep learning reasoning engine | |
| KR102141083B1 (en) | Optimization methods, systems, electronic devices and storage media of database systems | |
| CN106033574B (en) | Method and device for identifying cheating behaviors | |
| CN110852780A (en) | Data analysis method, device, equipment and computer storage medium | |
| CN111291082B (en) | Data aggregation processing method, device, equipment and storage medium | |
| CN112269706A (en) | Interface parameter checking method and device, electronic equipment and computer readable medium | |
| CN111241396B (en) | Information pushing method and device, electronic equipment and storage medium | |
| CN111966767B (en) | Track thermodynamic diagram generation method, device, electronic equipment and storage medium | |
| CN112819491B (en) | Method and device for converting data processing, electronic equipment and storage medium | |
| CN110995687B (en) | Cat pool equipment identification method, device, equipment and storage medium | |
| CN112540904B (en) | Machine operation behavior recognition method, device, electronic equipment and computer medium | |
| CN112561332B (en) | Model management method, device, electronic equipment, storage medium and program product | |
| CN113688202B (en) | Emotion polarity analysis method and device, electronic equipment and computer storage medium | |
| CN112598136B (en) | Data calibration method and device | |
| CN115563310A (en) | Method, device, equipment and medium for determining key service node | |
| CN114706610A (en) | Business flow chart generation method, device, equipment and storage medium | |
| CN111581366B (en) | User intention determining method, device, electronic equipment and readable storage medium | |
| US11481298B2 (en) | Computing CPU time usage of activities serviced by CPU | |
| CN113656422A (en) | Method and device for updating human face base | |
| CN111985760A (en) | Data content evaluation method and device, electronic equipment and storage medium | |
| CN111597461A (en) | Target object aggregation prediction method and device and electronic equipment | |
| CN111523000A (en) | Method, device, equipment and storage medium for importing data | |
| CN114462611B (en) | Method for estimating resources required by quantum gate construction and related device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |