CN112819491A - Method and device for processing conversion data, electronic equipment and storage medium - Google Patents
Method and device for processing conversion data, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN112819491A CN112819491A CN201911120892.8A CN201911120892A CN112819491A CN 112819491 A CN112819491 A CN 112819491A CN 201911120892 A CN201911120892 A CN 201911120892A CN 112819491 A CN112819491 A CN 112819491A
- Authority
- CN
- China
- Prior art keywords
- data
- index value
- index
- value sequence
- conversion
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000006243 chemical reaction Methods 0.000 title claims abstract description 135
- 238000012545 processing Methods 0.000 title claims abstract description 63
- 238000000034 method Methods 0.000 title claims abstract description 34
- 230000002159 abnormal effect Effects 0.000 claims abstract description 110
- 238000001514 detection method Methods 0.000 claims abstract description 61
- 238000010606 normalization Methods 0.000 claims abstract description 46
- 230000015654 memory Effects 0.000 claims description 20
- 230000001131 transforming effect Effects 0.000 claims description 6
- 239000000126 substance Substances 0.000 claims description 3
- 230000002123 temporal effect Effects 0.000 claims description 2
- 238000005516 engineering process Methods 0.000 abstract description 3
- 238000004422 calculation algorithm Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 7
- 238000004364 calculation method Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 238000004590 computer program Methods 0.000 description 4
- 230000006399 behavior Effects 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 238000013519 translation Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000001151 other effect Effects 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 238000005316 response function Methods 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0241—Advertisements
- G06Q30/0242—Determining effectiveness of advertisements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2474—Sequence data queries, e.g. querying versioned data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06393—Score-carding, benchmarking or key performance indicator [KPI] analysis
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- Strategic Management (AREA)
- Development Economics (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Economics (AREA)
- General Physics & Mathematics (AREA)
- Entrepreneurship & Innovation (AREA)
- Game Theory and Decision Science (AREA)
- Marketing (AREA)
- Finance (AREA)
- General Business, Economics & Management (AREA)
- Accounting & Taxation (AREA)
- Educational Administration (AREA)
- Fuzzy Systems (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Probability & Statistics with Applications (AREA)
- Mathematical Physics (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a method and a device for processing conversion data, electronic equipment and a storage medium, and relates to the field of data processing, in particular to a conversion data processing technology in a target conversion bid. The specific implementation scheme is as follows: normalizing various types of conversion data according to the index parameters to obtain normalized data, wherein the normalized data comprises a time-based index value sequence; judging whether the index value sequence has abnormal data or not according to the time weight; and if the abnormal data exist, deleting the abnormal data from the index value sequence. The normalization processing can convert various types of conversion data into normalization data with the same data structure, and abnormal data detection is performed on an index value sequence contained in the normalization data based on the time weight, so that the accuracy of abnormal data detection can be improved.
Description
Technical Field
The application relates to a data processing technology, in particular to an internet advertisement conversion data processing technology.
Background
An optimization cost per click (oCPC) is used in internet advertising to calculate the rate of return. In the implementation process of the target conversion bidding function, statistics and abnormal data detection need to be carried out on conversion data. But when processing heterogeneous data sources, the abnormal data detection accuracy is poor.
Disclosure of Invention
The embodiment of the application provides a method and a device for processing conversion data, an electronic device and a storage medium, which can improve the accuracy of abnormal data detection when processing heterogeneous data sources.
The embodiment of the application provides a method for processing conversion data, which comprises the following steps:
normalizing various types of conversion data according to the index parameters to obtain normalized data, wherein the normalized data comprises a time-based index value sequence;
judging whether the index value sequence has abnormal data or not according to the time weight;
and if the abnormal data exist, deleting the abnormal data from the index value sequence.
According to the application embodiment, various types of conversion data can be normalized according to the index parameters to obtain normalized data, and the normalized data comprises a time-based index value sequence; judging whether the index value sequence has abnormal data or not according to the time weight; and if the abnormal data exist, deleting the abnormal data from the index value sequence. The normalization processing can convert various types of conversion data into normalization data with the same data structure, and abnormal data detection is performed on an index value sequence contained in the normalization data based on the time weight, so that the accuracy of abnormal data detection can be improved.
In an embodiment of the above application, the normalizing the multiple types of transformation data according to the index parameter to obtain normalized data includes:
acquiring a user identifier, a conversion type and an acquisition type of conversion data;
and normalizing the various types of conversion data according to the user identification, the conversion type and the acquisition type to obtain normalized data.
In the above application embodiment, a plurality of types of conversion data can be classified according to the user identification, the conversion type, and the acquisition type. And normalizing the conversion data according to the user identification, the conversion type and the acquisition mode of the conversion data to which the conversion data belongs, so as to realize the normalization of the heterogeneous data source. By recording the user identification, the conversion type and the acquisition type, the conversion data can be marked more accurately.
In an embodiment of the above application, the normalizing the conversion data of the plurality of types according to the user identifier, the conversion type, and the acquisition type to obtain normalized data includes:
configuring a data unit for the user identification;
the data unit comprises an incidence relation between one or more acquisition types and a conversion type, and the incidence relation is associated with one or more delivery package information; the information of the delivery package comprises an index identifier and a time-based index value sequence associated with the index identifier.
In the above application embodiment, a data structure of a data unit is used to store a plurality of types of conversion data of a user, and a plurality of types of delivery package information can be determined according to the association relationship between the acquisition type of the conversion data and the conversion type. Each putting package information comprises an index value sequence under one index, and the index value sequence is sorted based on time and stores index values at different time points. And then realize according to customer's index, acquisition mode, conversion type and index mark positioning index value sequence's data structure, can carry out normalization processing to the conversion data of different grade type through this data structure, improve normalization efficiency.
In an embodiment of the foregoing application, determining whether there is abnormal data in the index value sequence according to the time weight includes:
calculating a hash value of the normalized data according to the target characteristics contained in the normalized data;
determining a hash bucket where the normalized data is located according to the hash value;
and judging whether the index value sequence contained in the normalized data in each hash bucket has abnormal data or not according to the time weight.
In the embodiment of the application, before the normalized data is detected, the hash value is obtained according to the target characteristics of the normalized data, the normalized data is subjected to barrel division according to the hash value, then abnormal data detection is performed on the normalized data in each barrel division, further the normalized data is divided based on the hash value, and the abnormal detection efficiency is improved.
In an embodiment of the foregoing application, determining whether there is abnormal data in an index value sequence included in the normalized data in each hash bucket according to the time weight includes:
processing abnormal data detection in a plurality of hash buckets in parallel;
in each hash bucket, sequentially reading a sequence of index values in the hash bucket according to a serial sequence;
and judging whether abnormal data exist in the index sequence.
In the embodiment of the application, parallel computation can be performed among the hash buckets, and high-concurrency anomaly detection is realized. Meanwhile, inside each hash bucket, each index value sequence is processed in sequence in a serial mode, and the data processing efficiency is further improved.
In an embodiment of the foregoing application, determining whether there is abnormal data in the index value sequence according to the time weight includes:
weighting each index value in the index value sequence according to the time weight to obtain a mean value of the index value sequence, wherein the mean value is a mean number or a median of the index value sequence;
calculating the standard deviation or the absolute median difference of the index value sequence;
traversing the index value sequence according to the mean value and the standard deviation, and determining a residual error with the maximum numerical difference from the mean value;
calculating a critical value of index value sequence t distribution;
and determining whether the abnormal point exists according to the critical value and the standard deviation.
In the embodiment of the application, for an index value sequence, weighting each index value in the index value sequence according to time weight to obtain a mean value of the index value sequence; traversing each index value in the index value sequence according to the mean value and the standard deviation, determining a residual error with the maximum difference with the mean value, and then determining an abnormal point according to a threshold value of t distribution of the index sequence and the residual error. And then the mean value of the index value sequence is determined based on the time information of the index values, and the abnormal point in the index value sequence is determined based on the mean value, so that the abnormal point can be more accurately determined based on time validity, and the abnormal point detection efficiency is improved.
In an embodiment of the above application, weighting each index value in the index value sequence according to a time weight includes:
determining a weighting parameter of the index value according to the time interval between the acquisition time of the index value and the current time, wherein the weighting parameter is less than 1, and the numerical value of the weighting parameter and the length of the time interval are in an inverse proportion trend;
and weighting the index value according to the weighting parameter.
In the embodiment of the application, the weighting parameter of the index value is determined according to the time interval between the acquisition time of the index value and the current time, so that a weighting mode that the numerical value of the weighting parameter and the length of the time interval are in an inverse proportion trend can be realized, and the accuracy of weighting calculation is further improved.
An embodiment of the present application further provides a device for converting data processing, including:
the normalization module is used for carrying out normalization processing on various types of conversion data according to the index parameters to obtain normalized data, and the normalized data comprises an index value sequence based on time;
the anomaly detection module is used for judging whether the index value sequence has abnormal data or not according to the time weight;
and the exception handling module is used for deleting the exception data from the index value sequence if the exception data exists.
An embodiment of the present application further provides a device for converting data processing, including:
the normalization module is used for carrying out normalization processing on various types of conversion data according to the index parameters to obtain normalized data, and the normalized data comprises an index value sequence based on time;
the anomaly detection module is used for judging whether the index value sequence has abnormal data or not according to the time weight;
and the exception handling module is used for deleting the exception data from the index value sequence if the exception data exists.
An embodiment of the present application further provides an electronic device, including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of the above-described embodiments.
Embodiments of the present application also provide a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the method of the above embodiments.
Other effects of the above-described alternative will be described below with reference to specific embodiments.
Drawings
The drawings are included to provide a better understanding of the present solution and are not intended to limit the present application. Wherein:
FIG. 1 is a schematic flow chart diagram according to a first embodiment of the present application;
FIG. 2 is a schematic flow chart diagram according to a second embodiment of the present application;
FIG. 3 is a schematic flow chart diagram according to a third embodiment of the present application;
FIG. 4 is a schematic structural diagram according to a fourth embodiment of the present application;
fig. 5 is a block diagram of an electronic device for implementing the method of converting data processing according to the embodiment of the present application.
Detailed Description
The following description of the exemplary embodiments of the present application, taken in conjunction with the accompanying drawings, includes various details of the embodiments of the application for the understanding of the same, which are to be considered exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Example one
Fig. 1 is a schematic flowchart of a method for converting data processing, which is provided in an embodiment of the present application, and is adapted to detect fluctuation of multi-source heterogeneous data in an objective conversion cost per click (cpcc), and may be executed by an electronic device such as a server, a terminal, and the like, where the method may be implemented by:
step 101, normalizing various types of conversion data according to the index parameters to obtain normalized data, wherein the normalized data comprises an index value sequence based on time.
The conversion data are classified into different categories according to different acquisition types and conversion types. The acquisition type and the conversion type appear in a group, and when an attention index value is generated, conversion data are generated by conversion data acquisition equipment such as a mobile terminal.
The collection types of the multiple types of conversion data are used to represent the collection mode of the conversion data, and the collection types may include Application Programming Interface (API) collection, js (javascript) code collection loaded in a landing page, Business Conversation Platform (Business Conversation Platform) collection, Application (APP) activation collection, SDK code collection in an applet, and the like.
The conversion type of the conversion data may be the type of operation that is identified as producing the conversion, such as a user triggering a target action in a search page or landing page. The target behavior may be consulting or submitting a form, etc.
The method and the device for converting the data comprise the steps that various types of conversion data are also called heterogeneous data sources, the conversion data carried by the heterogeneous data sources are subjected to normalization processing, and the obtained normalization data comprise index value sequences. The index value sequence is also called as a time sequence, and each element in the index value sequence (or the time sequence) comprises two pieces of time information and an index value. The time information indicates a time at which an index value is generated, the index value being a user behavior parameter of interest.
Optionally, obtaining a user identifier, a conversion type and an acquisition type of the conversion data; and normalizing the various types of conversion data according to the user identification, the conversion type and the acquisition type to obtain normalized data.
The acquisition type can be determined according to the acquisition mode of the acquired conversion data, such as an interface for receiving the conversion data, and the conversion type can be determined according to the index value type carried by the conversion data. And abstracting the user identification, the conversion type and the acquisition type from the conversion data, and traversing the conversion data to obtain an index value sequence.
In the above application embodiment, a plurality of types of conversion data can be classified according to the user identification, the conversion type, and the acquisition type. And normalizing the conversion data according to the user identification, the conversion type and the acquisition mode of the conversion data to which the conversion data belongs, so as to realize the normalization of the heterogeneous data source. By recording the user identification, the conversion type and the acquisition type, the conversion data can be marked more accurately.
Specifically, as shown in fig. 2, a data unit is configured for the user identifier;
the data unit comprises an incidence relation between one or more acquisition types and a conversion type, and the incidence relation is associated with one or more delivery package information; the information of the delivery package comprises an index identifier and a time-based index value sequence associated with the index identifier.
In an internet promotional platform, each account may use one or more acquisition modalities to access data for one or more conversion types (marking a target behavior that an advertiser desires to optimize). One user identification is used to sign one account. The embodiment of the present application provides a new data structure, in which a data unit (also called unit) is configured for an account. Each data unit has one or more parcel information (also called cells). One data unit may be configured for each account, for example, if there are N accounts represented by N user identities, N data units may be configured, where N is a positive integer. Or, one data unit may be configured for a part of the accounts, for example, if there are N accounts represented by N user identifiers, M data units are configured, and one data unit is configured for each account of the M accounts, where M is a non-negative integer smaller than N.
Each delivery package information is configured with an incidence relation between the acquisition type and the conversion type, and the incidence relation represents the acquisition mode of the delivery package information and records the index value of the conversion type. The same user identification (namely account) can be configured with a plurality of delivery package information, and further the conversion data can be aggregated in a fixed time period. The information of the put-in package is an index value sequence (also called a time sequence) acquired according to the corresponding conversion type and acquisition type.
As shown in table 1, one user identifier corresponds to one data unit, and the data unit includes two association relations, which are: the acquisition type 1 is used to access the translation type 1 and the acquisition type 2 is used to access the translation type 2. The acquisition type 1-conversion type 1 comprises two pieces of delivery package information, and the two pieces of delivery package information respectively comprise an index 1-1 and an index value sequence associated with the index 1-1, and an index value sequence associated with the index 1-2. The acquisition type 2-the conversion type 2 includes two pieces of delivery package information, which respectively include an index 2-1 and an index value sequence associated therewith, and an index 2-2 and an index value sequence associated therewith.
TABLE 1
In the above application embodiment, a data structure of a data unit is used to store a plurality of types of conversion data of a user, and a plurality of types of delivery package information can be determined according to the association relationship between the acquisition type of the conversion data and the conversion type. Each putting package information comprises an index value sequence under one index, and the index value sequence is sorted based on time and stores index values at different time points. And then realize according to customer's index, acquisition mode, conversion type and index mark positioning index value sequence's data structure, can carry out normalization processing to the conversion data of different grade type through this data structure, improve normalization efficiency.
And 102, judging whether the index value sequence has abnormal data or not according to the time weight.
And respectively configuring time weight for each index value in the index sequence, and calculating to obtain the mean value of the index value sequence. And judging abnormal data in the index sequence according to the average value and the index value in the index sequence to finish the fluctuation detection.
And 103, deleting the abnormal data from the index value sequence if the abnormal data exists.
If the abnormal data exists, deleting the abnormal data from the index value sequence, and judging whether to return to the step 102. If the abnormal data does not exist, the index value sequence does not contain the abnormal data.
For example, it is determined whether the number of abnormal data that has been determined satisfies a preset number of abnormal data, or whether the number of times of determining whether there is abnormal data exceeds a preset number of cycles is performed. If the number of the abnormal data is less than the preset number of the abnormal data, or the number of times of judging whether the abnormal data exists does not exceed the preset cycle number, the step 102 is executed in a loop. And if the number of the abnormal data is larger than or equal to the preset number of the abnormal data, or the number of times of judging whether the abnormal data exists exceeds the preset cycle number, canceling and returning to the step 102.
According to the application embodiment, various types of conversion data can be normalized according to the index parameters to obtain normalized data, and the normalized data comprises a time-based index value sequence; judging whether the index value sequence has abnormal data or not according to the time weight; and if the abnormal data exist, deleting the abnormal data from the index value sequence. The normalization processing can convert various types of conversion data into normalization data with the same data structure, and abnormal data detection is performed on an index value sequence contained in the normalization data based on the time weight, so that the accuracy of abnormal data detection can be improved.
Example two
Fig. 3 is a schematic flow chart of a method for converting data processing provided in the second embodiment of the present application, which further illustrates the above embodiment, and includes:
step 201, normalization processing is performed on multiple types of conversion data according to the index parameters to obtain normalized data, wherein the normalized data comprises an index value sequence based on time.
Step 202, calculating a hash value of the normalized data according to the target characteristics contained in the normalized data.
The target feature may be a user identifier or an index value in the normalized data, or the like. And calculating the hash value according to the user identifier, and counting the normalized data of the user identifiers with the same hash value in the same hash bucket. And calculating the hash value according to the index value, so that the normalized data of the index value with the same hash value can be obtained in the same hash bucket.
And step 203, determining a hash bucket where the normalized data is located according to the hash value.
And step 204, judging whether the index value sequence contained in the normalized data in each hash bucket has abnormal data or not according to the time weight.
Optionally, detecting abnormal data in a plurality of hash buckets in parallel; in each hash bucket, sequentially reading a sequence of index values in the hash bucket according to a serial sequence; and judging whether abnormal data exist in the index sequence.
And after Hash sub-buckets are carried out according to the target characteristics, a plurality of index value sequences are stored in each Hash sub-bucket. The processing object of the fluctuation detection is one or more index value sequences. Optionally, in one hash bucket, the index value sequence may be sequentially read according to a certain order, and each index value sequence is read. And judging whether abnormal data exist in the read index sequence. The order may be the first address order of the index value sequence in the storage space, or may be the storage order of the index value sequence in the above data structure.
In the embodiment of the application, when abnormal data detection is performed on data in each hash bucket, the data in the hash bucket is split into the size capable of being loaded to a memory for processing by taking a data unit as the minimum granularity of each processing. The data in the hash bucket is stored to disk. And parallel fluctuation detection is carried out among the hash buckets. Parallel computation is performed among the hash buckets, and high-concurrency anomaly detection is achieved. Meanwhile, the index value sequence is sequentially processed in each hash bucket in a serial mode, so that the random input and output times can be reduced, the program concurrency is increased, and the fluctuation detection efficiency is improved.
And step 205, if the abnormal data exist, deleting the abnormal data from the index value sequence.
In the embodiment of the application, before the normalized data is detected, the hash value is obtained according to the target characteristics of the normalized data, the normalized data is subjected to barrel division according to the hash value, then abnormal data detection is performed on the normalized data in each barrel division, further the normalized data is divided based on the hash value, and the abnormal detection efficiency is improved.
EXAMPLE III
Fig. 3 is a schematic flow chart of a method for converting data processing provided in the third embodiment of the present application, which serves as a further description of the foregoing embodiment, and includes:
step 301, normalizing the multiple types of conversion data according to the index parameters to obtain normalized data, wherein the normalized data comprises a time-based index value sequence.
And 302, weighting each index value in the index value sequence according to the time weight to obtain a mean value of the index value sequence, wherein the mean value is a mean number or a median of the index value sequence.
In an actual popularization platform, the conversion data can generate expected fluctuation due to the change of the landing page of an advertiser or the change of a delivery plan, and at the moment, the reference meaning of the long-term historical data is not large and misjudgment is easily caused, so that an anomaly detection algorithm is required to be capable of adapting to the latest data distribution more timely. Because the fluctuation trend of each client conversion data is different, the uniform fluctuation threshold value cannot be applied to the conditions of all clients, and the fluctuation detection misjudgment or the judgment omission can be caused. The embodiment of the application uses a weighted limit standard deviation algorithm to carry out fluctuation detection based on the statistics and the distribution characteristics of the client data. The basic principle of the standard deviation algorithm is to assume that no abnormal value exists in the data set, delete the value (maximum value or minimum value) deviating from the mean value gradually in the data set, and synchronously update the corresponding t distribution critical value (for checking whether the hypothesis is established) until the hypothesis is established or the number of abnormal values exceeds the set k value. The embodiment of the application introduces a time weight beta parameter on the basis of an extreme standard deviation algorithm (ESD algorithm) for controlling time weight, and can be called a weighted extreme standard deviation algorithm (W-ESD algorithm).
Optionally, determining a weighting parameter of the index value according to the time interval between the acquisition time of the index value and the current time, wherein the weighting parameter is smaller than 1, and the value of the weighting parameter and the length of the time interval are in an inverse proportion trend; and weighting the index value according to the weighting parameter.
The mean value can be calculated by the following formula
Wherein the beta parameter is used to control the importance of the temporal features in calculating the mean, i.e. the weighting parameter. The closer the beta parameter is to 0, the mean value
The more the value of the index is biased to the latest time. T is1,T2…Ti…TnThe index value is corresponding to each time information in the index value sequence. N is the number of index values included in the index value sequence.
The average value that can represent the series of indicator values may also represent the median of the series of indicator values.
In the embodiment of the application, the weighting parameter of the index value is determined according to the time interval between the acquisition time of the index value and the current time, so that a weighting mode that the numerical value of the weighting parameter and the length of the time interval are in an inverse proportion trend can be realized, and the accuracy of weighting calculation is further improved.
And step 303, calculating the standard deviation or the absolute median difference s of the index value sequence.
When in use
S represents the standard deviation when representing the mean; when in use
When s represents a median indicates an absolute median difference, abbreviated as MAD (for the sequence T, | Ti-mean (T) |)
And step 304, traversing the index value sequence according to the mean value and the standard deviation, and determining the residual error with the maximum numerical difference from the mean value.
Residual error R with maximum numerical difference from mean valuejThe calculation formula of (a) is as follows:
wherein the content of the first and second substances,
a mean value representing a sequence of indicator values; t isiIs a certain index value in the index value sequence traversed currently. K is the number of preset abnormal data.
Step 305, calculating a threshold value of the index value sequence t distribution.
Critical value lambdajThe calculation formula of (2) is as follows:
wherein, tp,n-j-1Representing the right critical value of the t distribution with the degree of freedom n-j +1 and the significance p.
And step 306, determining whether an abnormal point exists according to the critical value and the standard deviation.
Residual error R if the difference from the mean value is maximumjGreater than a critical value lambdajThen the index value is determined to be abnormal data. Otherwise, if the residual error R with the maximum difference with the mean value isjLess than a critical value lambdajThen it indicates that there is no abnormal data in the index value queue.
In the embodiment of the application, for an index value sequence, weighting each index value in the index value sequence according to time weight to obtain a mean value of the index value sequence; traversing each index value in the index value sequence according to the mean value and the standard deviation, determining a residual error with the maximum difference with the mean value, and then determining an abnormal point according to a threshold value of t distribution of the index sequence and the residual error. And then the mean value of the index value sequence is determined based on the time information of the index values, and the abnormal point in the index value sequence is determined based on the mean value, so that the abnormal point can be more accurately determined based on time validity, and the abnormal point detection efficiency is improved. The fluctuation detection algorithm provided by the embodiment of the application can support the fluctuation detection in a plurality of index latitudes, and can be used as a reference basis for attributing the fluctuation, so that the problem can be more effectively positioned. Adopting time weight to carry out fluctuation detection, so that a fluctuation detection system can adaptively relax or tighten an abnormal judgment standard according to the historical fluctuation condition of a client; the beta parameter is introduced, so that an application party can adjust the importance of the latest time data distribution in an anomaly detection algorithm according to a specific application scene, and the accuracy and the flexibility of fluctuation detection are improved.
Example four
Fig. 4 is a schematic structural diagram of an apparatus 400 for transforming data processing according to a fourth embodiment of the present application, where the apparatus is configured to execute the manner shown in the foregoing embodiments to implement a corresponding response function, and the apparatus may be applied to an electronic device such as a server or a terminal, and includes: a normalization module 401, an anomaly detection module 402, and an anomaly handling module 403. Wherein:
the normalization module 401 is configured to perform normalization processing on multiple types of conversion data according to the index parameter to obtain normalized data, where the normalized data includes a time-based index value sequence;
an anomaly detection module 402, configured to determine whether the index value sequence has abnormal data according to the time weight;
and an exception handling module 403, configured to delete the exception data from the index value sequence if the exception data exists.
In the above application embodiment, the normalization module 401 performs normalization processing on multiple types of conversion data according to the index parameter to obtain normalized data, where the normalized data includes a time-based index value sequence; the anomaly detection module 402 determines whether the index value sequence has abnormal data according to the time weight; if there is abnormal data, the abnormal processing module 403 deletes the abnormal data from the index value sequence. The normalization processing can convert various types of conversion data into normalization data with the same data structure, and abnormal data detection is performed on an index value sequence contained in the normalization data based on the time weight, so that the accuracy of abnormal data detection can be improved.
On the basis of the above embodiment, the normalization module 401 is configured to:
acquiring a user identifier, a conversion type and an acquisition type of conversion data;
and normalizing the various types of conversion data according to the user identification, the conversion type and the acquisition type to obtain normalized data.
In the above application embodiment, the normalization module 401 can classify a plurality of types of conversion data according to the user identifier, the conversion type, and the acquisition type. And normalizing the conversion data according to the user identification, the conversion type and the acquisition mode of the conversion data to which the conversion data belongs, so as to realize the normalization of the heterogeneous data source. By recording the user identification, the conversion type and the acquisition type, the conversion data can be marked more accurately.
On the basis of the above embodiment, the normalization module 401 is configured to:
configuring a data unit for each user identifier;
the data unit comprises an incidence relation between one or more acquisition types and a conversion type, and the incidence relation is associated with one or more delivery package information; the information of the delivery package comprises an index identifier and a time-based index value sequence associated with the index identifier.
In the above application embodiment, the normalization module 401 uses the data structure of the data unit to store multiple types of conversion data of one user, and can determine multiple types of delivery package information according to the association relationship between the acquisition type of the conversion data and the conversion type. Each putting package information comprises an index value sequence under one index, and the index value sequence is sorted based on time and stores index values at different time points. And then realize according to customer's index, acquisition mode, conversion type and index mark positioning index value sequence's data structure, can carry out normalization processing to the conversion data of different grade type through this data structure, improve normalization efficiency.
On the basis of the foregoing embodiment, the anomaly detection module 402 is configured to:
calculating a hash value of the normalized data according to the target characteristics contained in the normalized data;
determining a hash bucket where the normalized data is located according to the hash value;
and judging whether the index value sequence contained in the normalized data in each hash bucket has abnormal data or not according to the time weight.
In the embodiment of the application, before the normalized data is detected, the anomaly detection module 402 obtains a hash value according to the target characteristics of the normalized data, performs barrel splitting on the normalized data according to the hash value, and then performs anomaly data detection on the normalized data in each barrel splitting, thereby realizing that the normalized data is divided based on the hash value, and improving the anomaly detection efficiency.
On the basis of the foregoing embodiment, the anomaly detection module 402 is configured to:
processing abnormal data detection in a plurality of hash buckets in parallel;
in each hash bucket, sequentially reading an index value sequence in the hash bucket according to a serial sequence;
and judging whether abnormal data exist in the index sequence.
In the above application embodiment, the anomaly detection module 402 may perform parallel computation among hash buckets, so as to implement highly concurrent anomaly detection. Meanwhile, inside each hash bucket, each index value sequence is processed in sequence in a serial mode, and the data processing efficiency is further improved.
On the basis of the foregoing embodiment, the anomaly detection module 402 is configured to:
weighting each index value in the index value sequence according to the time weight to obtain a mean value of the index value sequence, wherein the mean value is a mean number or a median of the index value sequence;
calculating the standard deviation or the absolute median difference of the index value sequence;
traversing the index value sequence according to the mean value and the standard deviation, and determining a residual error with the maximum numerical difference from the mean value;
calculating a critical value of index value sequence t distribution;
and determining whether the abnormal point exists according to the critical value and the standard deviation.
In the above application embodiment, for an index value sequence, the anomaly detection module 402 weights each index value in the index value sequence according to the time weight to obtain a mean value of the index value sequence; traversing each index value in the index value sequence according to the mean value and the standard deviation, determining a residual error with the maximum difference with the mean value, and then determining an abnormal point according to a threshold value of t distribution of the index sequence and the residual error. And then the mean value of the index value sequence is determined based on the time information of the index values, and the abnormal point in the index value sequence is determined based on the mean value, so that the abnormal point can be more accurately determined based on time validity, and the abnormal point detection efficiency is improved.
On the basis of the foregoing embodiment, the anomaly detection module 402 is configured to:
determining a weighting parameter of the index value according to the time interval between the acquisition time of the index value and the current time, wherein the weighting parameter is less than 1, and the numerical value of the weighting parameter and the length of the time interval are in an inverse proportion trend;
and weighting the index value according to the weighting parameter.
In the above application embodiment, the anomaly detection module 402 determines the weighting parameter of the index value according to the time interval between the acquisition time of the index value and the current time, so that a weighting mode that the value of the weighting parameter and the length of the time interval are in an inverse proportion trend can be realized, and the accuracy of weighting calculation is further improved.
EXAMPLE five
According to an embodiment of the present application, an electronic device and a readable storage medium are also provided.
Fig. 5 is a block diagram of an electronic device according to an embodiment of the present application. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application that are described and/or claimed herein.
As shown in fig. 5, the electronic apparatus includes: one or more processors 501, memory 502, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions for execution within the electronic device, including instructions stored in or on the memory to display graphical information of a GUI on an external input/output apparatus (such as a display device coupled to the interface). In other embodiments, multiple processors and/or multiple buses may be used, along with multiple memories and multiple memories, as desired. Also, multiple electronic devices may be connected, with each device providing portions of the necessary operations (e.g., as a server array, a group of blade servers, or a multi-processor system). In fig. 5, one processor 501 is taken as an example.
The memory 502 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created from use of the electronic device that converts data processing, and the like. Further, the memory 502 may include high speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, memory 502 optionally includes memory located remotely from processor 501, which may be connected to the electronics that translate data processing via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The electronic device of the method of converting data processing may further include: an input device 503 and an output device 504. The processor 501, the memory 502, the input device 503 and the output device 504 may be connected by a bus or other means, and fig. 5 illustrates the connection by a bus as an example.
The input device 503 may receive input numeric or character information and generate key signal inputs related to user settings and function control of the electronic apparatus that converts data processing, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, a pointing stick, one or more mouse buttons, a track ball, a joystick, and the like. The output devices 504 may include a display device, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibrating motors), among others. The display device may include, but is not limited to, a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, and a plasma display. In some implementations, the display device can be a touch screen.
Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, application specific ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
These computer programs (also known as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented using high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
According to the application embodiment, various types of conversion data can be normalized according to the index parameters to obtain normalized data, and the normalized data comprises a time-based index value sequence; judging whether the index value sequence has abnormal data or not according to the time weight; and if the abnormal data exist, deleting the abnormal data from the index value sequence. The normalization processing can convert various types of conversion data into normalization data with the same data structure, and abnormal data detection is performed on an index value sequence contained in the normalization data based on the time weight, so that the accuracy of abnormal data detection can be improved.
In the above application embodiment, a plurality of types of conversion data can be classified according to the user identification, the conversion type, and the acquisition type. And normalizing the conversion data according to the user identification, the conversion type and the acquisition mode of the conversion data to which the conversion data belongs, so as to realize the normalization of the heterogeneous data source. By recording the user identification, the conversion type and the acquisition type, the conversion data can be marked more accurately.
In the above application embodiment, a data structure of a data unit is used to store a plurality of types of conversion data of a user, and a plurality of types of delivery package information can be determined according to the association relationship between the acquisition type of the conversion data and the conversion type. Each putting package information comprises an index value sequence under one index, and the index value sequence is sorted based on time and stores index values at different time points. And then realize according to customer's index, acquisition mode, conversion type and index mark positioning index value sequence's data structure, can carry out normalization processing to the conversion data of different grade type through this data structure, improve normalization efficiency.
In the embodiment of the application, before the normalized data is detected, the hash value is obtained according to the target characteristics of the normalized data, the normalized data is subjected to barrel division according to the hash value, then abnormal data detection is performed on the normalized data in each barrel division, further the normalized data is divided based on the hash value, and the abnormal detection efficiency is improved.
In the embodiment of the application, parallel computation can be performed among the hash buckets, and high-concurrency anomaly detection is realized. Meanwhile, inside each hash bucket, each index value sequence is processed in sequence in a serial mode, and the data processing efficiency is further improved.
In the embodiment of the application, for an index value sequence, weighting each index value in the index value sequence according to time weight to obtain a mean value of the index value sequence; traversing each index value in the index value sequence according to the mean value and the standard deviation, determining a residual error with the maximum difference with the mean value, and then determining an abnormal point according to a threshold value of t distribution of the index sequence and the residual error. And then the mean value of the index value sequence is determined based on the time information of the index values, and the abnormal point in the index value sequence is determined based on the mean value, so that the abnormal point can be more accurately determined based on time validity, and the abnormal point detection efficiency is improved.
In the embodiment of the application, the weighting parameter of the index value is determined according to the time interval between the acquisition time of the index value and the current time, so that a weighting mode that the numerical value of the weighting parameter and the length of the time interval are in an inverse proportion trend can be realized, and the accuracy of weighting calculation is further improved.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present application may be executed in parallel, sequentially, or in different orders, and the present invention is not limited thereto as long as the desired results of the technical solutions disclosed in the present application can be achieved.
The above-described embodiments should not be construed as limiting the scope of the present application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (10)
1. A method of transforming data processing, comprising:
normalizing various types of conversion data according to the index parameters to obtain normalized data, wherein the normalized data comprises an index value sequence based on time;
judging whether the index value sequence has abnormal data or not according to the time weight;
and if the abnormal data exist, deleting the abnormal data from the index value sequence.
2. The method for converting data processing according to claim 1, wherein the normalizing the plurality of types of conversion data according to the index parameter to obtain normalized data includes:
acquiring a user identifier, a conversion type and an acquisition type of conversion data;
and normalizing the various types of conversion data according to the user identification, the conversion type and the acquisition type to obtain normalized data.
3. The method for converting data processing according to claim 2, wherein said normalizing the conversion data of the plurality of types according to the user identifier, the conversion type, and the collection type to obtain normalized data comprises:
configuring a data unit for the user identification;
the data unit comprises an incidence relation between one or more acquisition types and a conversion type, and the incidence relation is associated with one or more delivery package information; the information of the delivery package comprises an index identifier and a time-based index value sequence associated with the index identifier.
4. The method for converting data processing according to claim 1, wherein said determining whether there is abnormal data in the index value sequence according to the time weight includes:
calculating a hash value of the normalized data according to the target characteristics contained in the normalized data;
determining a hash bucket where the normalized data is located according to the hash value;
and judging whether the index value sequence contained in the normalized data in each hash bucket has abnormal data or not according to the time weight.
5. The method for converting data processing according to claim 4, wherein said determining whether there is abnormal data in the index value sequence included in the normalized data in each hash bucket according to the time weight includes:
processing abnormal data detection in a plurality of hash buckets in parallel;
in each hash bucket, sequentially reading a sequence of index values in the hash bucket according to a serial sequence;
and judging whether abnormal data exist in the index sequence.
6. The method for converting data processing according to claim 1, wherein said determining whether there is abnormal data in the index value sequence according to the time weight includes:
weighting each index value in the index value sequence according to the time weight to obtain a mean value of the index value sequence, wherein the mean value is a mean number or a median number of the index value sequence;
calculating the standard deviation or the absolute median difference of the index value sequence;
traversing the index value sequence according to the mean value and the standard deviation, and determining a residual error with the maximum numerical difference from the mean value;
calculating a critical value of the index value sequence t distribution;
and determining whether an abnormal point exists according to the critical value and the standard deviation.
7. The method of transforming data processing according to claim 6, wherein said weighting each metric value in the series of metric values according to a temporal weight comprises:
determining a weighting parameter of the index value according to the time interval between the acquisition time of the index value and the current time, wherein the weighting parameter is less than 1, and the value of the weighting parameter and the length of the time interval are in an inverse proportion trend;
and weighting the index value according to the weighting parameter.
8. An apparatus for transforming data processing, comprising:
the normalization module is used for carrying out normalization processing on various types of conversion data according to the index parameters to obtain normalized data, and the normalized data comprises an index value sequence based on time;
the anomaly detection module is used for judging whether the index value sequence has abnormal data or not according to the time weight;
and the exception handling module is used for deleting the exception data from the index value sequence if the exception data exists.
9. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.
10. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911120892.8A CN112819491B (en) | 2019-11-15 | 2019-11-15 | Method and device for converting data processing, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911120892.8A CN112819491B (en) | 2019-11-15 | 2019-11-15 | Method and device for converting data processing, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112819491A true CN112819491A (en) | 2021-05-18 |
| CN112819491B CN112819491B (en) | 2024-02-09 |
Family
ID=75851818
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911120892.8A Active CN112819491B (en) | 2019-11-15 | 2019-11-15 | Method and device for converting data processing, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112819491B (en) |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140006330A1 (en) * | 2012-06-28 | 2014-01-02 | International Business Machines Corporation | Detecting anomalies in real-time in multiple time series data with automated thresholding |
| CN103532776A (en) * | 2013-09-30 | 2014-01-22 | 广东电网公司电力调度控制中心 | Service flow detection method and system |
| CN103532940A (en) * | 2013-09-30 | 2014-01-22 | 广东电网公司电力调度控制中心 | Network security detection method and device |
| US20150269050A1 (en) * | 2014-03-18 | 2015-09-24 | Microsoft Corporation | Unsupervised anomaly detection for arbitrary time series |
| US20150356421A1 (en) * | 2014-06-05 | 2015-12-10 | Mitsubishi Electric Research Laboratories, Inc. | Method for Learning Exemplars for Anomaly Detection |
| US20160042287A1 (en) * | 2014-08-10 | 2016-02-11 | Palo Alto Research Center Incorporated | Computer-Implemented System And Method For Detecting Anomalies Using Sample-Based Rule Identification |
| CN109257612A (en) * | 2018-08-09 | 2019-01-22 | 广州虎牙信息科技有限公司 | Assessment Method on Potential, device, computer storage medium and server is broadcast live in game |
| CN109347653A (en) * | 2018-09-07 | 2019-02-15 | 阿里巴巴集团控股有限公司 | A kind of Indexes Abnormality discovery method and apparatus |
| CN109582485A (en) * | 2018-10-26 | 2019-04-05 | 阿里巴巴集团控股有限公司 | A kind of configuration change method for detecting abnormality and device |
| CN109740811A (en) * | 2018-12-28 | 2019-05-10 | 斑马网络技术有限公司 | Passage speed prediction technique, device and storage medium |
| CN110008080A (en) * | 2018-12-25 | 2019-07-12 | 阿里巴巴集团控股有限公司 | Operational indicator method for detecting abnormality, device and electronic equipment based on time series |
| CN110266552A (en) * | 2019-08-15 | 2019-09-20 | 华为技术有限公司 | Method, model training method and the device of Traffic anomaly detection |
-
2019
- 2019-11-15 CN CN201911120892.8A patent/CN112819491B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140006330A1 (en) * | 2012-06-28 | 2014-01-02 | International Business Machines Corporation | Detecting anomalies in real-time in multiple time series data with automated thresholding |
| CN103532776A (en) * | 2013-09-30 | 2014-01-22 | 广东电网公司电力调度控制中心 | Service flow detection method and system |
| CN103532940A (en) * | 2013-09-30 | 2014-01-22 | 广东电网公司电力调度控制中心 | Network security detection method and device |
| US20150269050A1 (en) * | 2014-03-18 | 2015-09-24 | Microsoft Corporation | Unsupervised anomaly detection for arbitrary time series |
| US20150356421A1 (en) * | 2014-06-05 | 2015-12-10 | Mitsubishi Electric Research Laboratories, Inc. | Method for Learning Exemplars for Anomaly Detection |
| US20160042287A1 (en) * | 2014-08-10 | 2016-02-11 | Palo Alto Research Center Incorporated | Computer-Implemented System And Method For Detecting Anomalies Using Sample-Based Rule Identification |
| CN109257612A (en) * | 2018-08-09 | 2019-01-22 | 广州虎牙信息科技有限公司 | Assessment Method on Potential, device, computer storage medium and server is broadcast live in game |
| CN109347653A (en) * | 2018-09-07 | 2019-02-15 | 阿里巴巴集团控股有限公司 | A kind of Indexes Abnormality discovery method and apparatus |
| CN109582485A (en) * | 2018-10-26 | 2019-04-05 | 阿里巴巴集团控股有限公司 | A kind of configuration change method for detecting abnormality and device |
| CN110008080A (en) * | 2018-12-25 | 2019-07-12 | 阿里巴巴集团控股有限公司 | Operational indicator method for detecting abnormality, device and electronic equipment based on time series |
| CN109740811A (en) * | 2018-12-28 | 2019-05-10 | 斑马网络技术有限公司 | Passage speed prediction technique, device and storage medium |
| CN110266552A (en) * | 2019-08-15 | 2019-09-20 | 华为技术有限公司 | Method, model training method and the device of Traffic anomaly detection |
Non-Patent Citations (2)
| Title |
|---|
| 孙锋;刘杰;周建辉;杨梓辉;毛伟兰;: "轴系测试数据分布特征信息获取方法与应用", 中国舰船研究, no. 1, pages 185 - 190 * |
| 徐雪松;: "时间序列不确定数据流中异常数据检测方法", 电子设计工程, no. 19, pages 19 - 21 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112819491B (en) | 2024-02-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111428008B (en) | Method, apparatus, device and storage medium for training a model | |
| CN112036509A (en) | Method and apparatus for training image recognition models | |
| EP3356951B1 (en) | Managing a database of patterns used to identify subsequences in logs | |
| CN111738446A (en) | Scheduling method, device, equipment and medium of deep learning inference engine | |
| CN111460384B (en) | Policy evaluation method, device and equipment | |
| CN112561332B (en) | Model management method, device, electronic equipment, storage medium and program product | |
| CN110852780A (en) | Data analysis method, device, equipment and computer storage medium | |
| CN112529181B (en) | Method and apparatus for model distillation | |
| CN111563541A (en) | Training method and device of image detection model | |
| CN111241225A (en) | Resident area change judgment method, resident area change judgment device, resident area change judgment equipment and storage medium | |
| CN112819491B (en) | Method and device for converting data processing, electronic equipment and storage medium | |
| CN112598136B (en) | Data calibration method and device | |
| CN115563310A (en) | Method, device, equipment and medium for determining key service node | |
| CN111339344B (en) | Indoor image retrieval method and device and electronic equipment | |
| US11481298B2 (en) | Computing CPU time usage of activities serviced by CPU | |
| CN111581366B (en) | User intention determining method, device, electronic equipment and readable storage medium | |
| CN114706610A (en) | Business flow chart generation method, device, equipment and storage medium | |
| CN114116688A (en) | Data processing and data quality inspection method, device and readable storage medium | |
| CN112560987A (en) | Image sample processing method, device, equipment, storage medium and program product | |
| CN113220982A (en) | Advertisement searching method, device, electronic equipment and medium | |
| CN111523000A (en) | Method, device, equipment and storage medium for importing data | |
| CN111597461A (en) | Target object aggregation prediction method and device and electronic equipment | |
| CN112734454A (en) | User information determination method and device, electronic equipment and storage medium | |
| CN113159398B (en) | Power consumption prediction method and device and electronic equipment | |
| CN111522837B (en) | Method and apparatus for determining time consumption of deep neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |