CN116383190B - Intelligent cleaning method and system for massive financial transaction big data - Google Patents
Intelligent cleaning method and system for massive financial transaction big data Download PDFInfo
- Publication number
- CN116383190B CN116383190B CN202310537830.7A CN202310537830A CN116383190B CN 116383190 B CN116383190 B CN 116383190B CN 202310537830 A CN202310537830 A CN 202310537830A CN 116383190 B CN116383190 B CN 116383190B
- Authority
- CN
- China
- Prior art keywords
- time sequence
- data
- subsequences
- difference
- subsequence
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 68
- 238000004140 cleaning Methods 0.000 title claims abstract description 45
- 238000005259 measurement Methods 0.000 claims abstract description 51
- 230000000877 morphologic effect Effects 0.000 claims abstract description 42
- 230000002159 abnormal effect Effects 0.000 claims abstract description 28
- 230000011218 segmentation Effects 0.000 claims description 32
- 238000004422 calculation algorithm Methods 0.000 claims description 22
- 230000008859 change Effects 0.000 claims description 19
- 238000004364 calculation method Methods 0.000 claims description 14
- 238000001514 detection method Methods 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 6
- 230000002123 temporal effect Effects 0.000 claims description 6
- 238000003064 k means clustering Methods 0.000 claims description 5
- 238000011524 similarity measure Methods 0.000 claims description 4
- 238000012545 processing Methods 0.000 abstract description 8
- 230000008569 process Effects 0.000 description 27
- 238000009826 distribution Methods 0.000 description 15
- 230000005540 biological transmission Effects 0.000 description 9
- 238000013450 outlier detection Methods 0.000 description 6
- 230000005856 abnormality Effects 0.000 description 4
- 238000000638 solvent extraction Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 238000007635 classification algorithm Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2474—Sequence data queries, e.g. querying versioned data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
- G06F16/285—Clustering or classification
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Quality & Reliability (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Probability & Statistics with Applications (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Preliminary Treatment Of Fibers (AREA)
Abstract
The application relates to the technical field of data processing, in particular to an intelligent cleaning method and system for massive big data, wherein the method comprises the following steps: acquiring a time sequence subsequence corresponding to massive big data; obtaining a morphological similarity measurement index between any two time sequence subsequences according to the difference between the data in any two time sequence subsequences; obtaining importance indexes corresponding to any two time sequence subsequences according to the length difference between any two time sequence subsequences; obtaining a state factor of the time sequence subsequence according to the abnormal condition of the data in the time sequence subsequence; obtaining a difference index between the time sequence subsequences according to the distance between the time sequence subsequences, the morphological similarity measurement index, the importance index and the state factor; and classifying the time sequence subsequences according to the difference indexes, and cleaning the data of massive big data according to the classification results. The application can obtain more accurate data cleaning results of time sequence data.
Description
Technical Field
The application relates to the technical field of data processing, in particular to an intelligent cleaning method and system for massive financial transaction big data.
Background
Data cleaning is an important step in data processing, can improve the quality and accuracy of data, ensures the reliability and effectiveness of the data, and is beneficial to decision making and business application of a clearing center. In the process of data cleaning, large data of mass financial transactions need to be subjected to outlier detection, and in order to improve data processing efficiency, acquired data need to be subjected to data slicing, and outlier detection is performed on each data block in a parallel computing mode, so that outlier of each data block is obtained. And finally, gathering and counting the abnormal values dispersed in different data blocks through abnormal value aggregation to obtain a global abnormal value detection result.
When the collected data is segmented, the existing classification algorithm only considers the data difference between time sequences, but does not consider the difference change between time sequences with different lengths, so that the data classification result is inaccurate, and the accuracy of the data cleaning result of each data block obtained by classification is low.
Disclosure of Invention
In order to solve the technical problem of lower accuracy of data cleaning results of each data block obtained by classification, the application aims to provide a mass financial transaction big data intelligent cleaning method and system, and the adopted technical scheme is as follows:
acquiring a time sequence subsequence corresponding to massive financial transaction big data;
obtaining a morphological similarity measurement index between any two time sequence subsequences according to the difference between the data in any two time sequence subsequences; obtaining importance indexes corresponding to any two time sequence subsequences according to the length difference between any two time sequence subsequences;
obtaining a state factor of the time sequence subsequence according to the abnormal condition of the data in the time sequence subsequence; obtaining a difference index between the time sequence subsequences according to the distance between the time sequence subsequences, the morphological similarity measurement index, the importance index and the state factor; and classifying the time sequence subsequences according to the difference indexes, and cleaning the massive financial transaction big data according to the classification results.
Preferably, the method for obtaining the morphological similarity measurement index between any two time sequence subsequences according to the difference between the data in any two time sequence subsequences specifically includes:
for any two time sequence subsequences, obtaining a change rate corresponding to the data according to the difference between every two adjacent data in the time sequence subsequences; and taking the absolute value of the difference value between all the change rate average values corresponding to the two time sequence subsequences as a morphological similarity measurement index between any two time sequence subsequences.
Preferably, the obtaining the importance index corresponding to any two time sequence subsequences according to the length difference between any two time sequence subsequences specifically includes:
acquiring the absolute value of the difference of the length between every two time sequence subsequences and recording the absolute value as the length difference of the time sequence subsequences;
for any two time sequence subsequences, the ratio between the length difference corresponding to the two time sequence subsequences and the maximum value in all the length differences is used as an importance index corresponding to any two time sequence subsequences.
Preferably, the state factor for obtaining the time sequence subsequence according to the abnormal condition of the data in the time sequence subsequence is specifically: and acquiring the average link distance corresponding to the data in the time sequence subsequence by using a COF outlier factor detection algorithm, and taking a normalized value of the average link distance of all the data in the time sequence subsequence as a state factor of the time sequence subsequence.
Preferably, the obtaining the difference index between the time sequence subsequences according to the distance between the time sequence subsequences, the morphological similarity measurement index, the importance index and the state factor specifically includes:
and acquiring the DTW distance between any two time sequence subsequences, acquiring the difference between state factors corresponding to the two time sequence subsequences, and acquiring a difference index between the time sequence subsequences according to the DTW distance, the difference between the state factors, the morphological similarity measurement index and the importance index.
Preferably, the method for calculating the difference index specifically includes: wherein (1)>Representing the index of difference between the time sequence subsequence u and the time sequence subsequence v,/>An importance index corresponding to the time sequence subsequence u and the time sequence subsequence v is represented by +.>Represents the DTW distance between the time sequence subsequence u and the time sequence subsequence v,/and>represents the DTW distance between the time sequence subsequence u and the time sequence subsequence v,/>A morphological similarity measure indicative of the temporal sub-sequence u and the temporal sub-sequence v, +.>Representing the state factor corresponding to the time sequence sub-sequence u, < ->Representing the state factor corresponding to the time sequence subsequence v.
Preferably, the time sequence subsequence corresponding to the massive financial transaction big data is specifically: and dividing the time sequence formed by massive financial transaction big data to obtain a time sequence sub-sequence.
Preferably, the dividing the time sequence formed by massive financial transaction big data to obtain the time sequence subsequence specifically includes:
acquiring an initial segmentation point corresponding to the time sequence by using a flow algorithm; in the neighborhood of the initial dividing point, marking any data point in the neighborhood as a target data point, and respectively acquiring the number of data points which are positioned at the left side and the right side of the target data point and belong to the neighborhood of the target data; taking the larger value of the number of data points corresponding to the left side of the target data point and the number of data points corresponding to the right side of the target data point as a numerator, taking the total number of the data points contained in the neighborhood of the target data point as a denominator, and taking the normalized value of the ratio of the numerator to the denominator as a single degree of the neighborhood direction of the target data point; and recording the data point corresponding to the maximum value of the neighborhood direction singleness in the neighborhood of the initial segmentation point as a final segmentation point, and segmenting the time sequence by using the final segmentation point to obtain a time sequence subsequence.
Preferably, the step of performing data cleaning on massive financial transaction big data according to the classification result specifically includes:
and taking the local outlier factor of the data point in each category in the classification result as the abnormal degree of the data point, and removing the data point corresponding to the abnormal degree larger than the preset degree threshold.
The application also provides an intelligent cleaning system for mass financial transaction big data, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the computer program realizes the steps of the intelligent cleaning method for mass financial transaction big data when being executed by the processor.
The embodiment of the application has at least the following beneficial effects:
firstly, according to the data difference between time sequence subsequences corresponding to massive financial transaction big data, a morphological similarity measurement index between the time sequence subsequences is obtained, and the morphological similarity of the data difference condition between two time sequence subsequences is reflected by using the morphological similarity measurement index; meanwhile, when the difference index is calculated later, the situation that the two time sequence sub-sequences have the length difference is considered, namely, the importance index corresponding to the two time sequence sub-sequences is obtained according to the length difference; furthermore, considering that the overall morphology between the time sequence subsequences is similar, but the difference of the outlier states also occurs between the time sequence subsequences, further obtaining the state factors of the time sequence subsequences according to the abnormal condition of the data in the time sequence subsequences, and reflecting the outlier states of the whole data in the time sequence subsequences by using the state factors; finally, the measurement distance between the time sequence subsequences, the morphological similarity of the data difference condition between the time sequence subsequences, the length difference between the time sequence subsequences and the outlier state between the time sequence subsequences are combined to obtain a difference index, namely the difference index between the time sequence subsequences, the time sequence subsequences are classified according to the difference index, and a more accurate time sequence data classification result can be obtained. And data cleaning is carried out on massive financial transaction big data according to the accurate classification result, so that the accurate data cleaning result can be obtained, and the data cleaning efficiency is improved. Finally, the method and the device can improve the accuracy of the time sequence data classification result and the data cleaning efficiency and accuracy.
Drawings
In order to more clearly illustrate the embodiments of the application or the technical solutions and advantages of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are only some embodiments of the application, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a method for intelligent cleaning of mass financial transaction big data.
Detailed Description
In order to further describe the technical means and effects adopted by the application to achieve the preset aim, the following is a detailed description of a specific implementation, structure, characteristics and effects of the method and system for intelligently cleaning mass financial transaction big data according to the application in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" means that the embodiments are not necessarily the same. Furthermore, the particular features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.
The application provides a mass financial transaction big data intelligent cleaning method and a system specific scheme by combining the drawings.
The main purpose of the application is as follows: the method comprises the steps of obtaining a time sequence subsequence through a flow algorithm, classifying the time sequence subsequence, transmitting clusters obtained through classification as data blocks of parallel calculation into a calculation node to calculate local outlier factors, and further finishing data cleaning on data. In the process, in order to avoid the waste of calculation power of the parallel calculation nodes, and in order to ensure that the parallel calculation achieves optimal calculation efficiency and accuracy of calculating local outlier factors when data transmission is carried out between the calculation nodes, the data quantity of the data blocks needs to be ensured to be similar.
The specific scene aimed by the application is as follows: in the process of cleaning massive financial transaction big data, the massive financial transaction big data is required to be divided into data blocks in order to ensure the data cleaning efficiency due to overlarge data quantity, and the data blocks are efficiently processed through parallel computing nodes.
Example 1:
referring to fig. 1, a flowchart of a method for intelligently cleaning big data of mass financial transactions according to an embodiment of the present application is shown, the method includes the following steps:
step one, acquiring a time sequence sub-sequence corresponding to massive financial transaction big data.
To ensure the accuracy and reliability of transactions and settlements, and to provide higher quality services to customers. The clearing house needs to analyze and process the transaction data in the financial market, such as risk early warning, transaction verification, settlement processing and the like. For financial assets of customers served by institutions, real-time data of financial markets, such as trade data of securities, futures, foreign exchange, bonds, etc., is required to be analyzed and processed.
In this embodiment, massive financial transaction big data during financial transaction is taken as an example for explanation, namely, a data interface is established between a clearing data center and a financial transaction data source, financial transaction data corresponding to a client of the clearing center is collected, and time sequence data is formed according to a time stamp of the transaction data and recorded as a time sequence.
Before the data block for parallel computation is acquired, the time sequence formed by massive financial transaction big data is firstly required to be divided to obtain a time sequence subsequence. For the segmentation of time series data, the existing segmentation method flow (Fast Low-cost Online Semantic Segmentation) algorithm is commonly used for time series data segmentation because of its versatility and efficiency. In the patent document with publication No. CN113780295a, a method for dividing time series data by using a flow algorithm is disclosed.
When time series division is performed by the flow algorithm, a correction arc crossing (Corrected Arc Crossings, CAC) sequence is acquired, and then a division point is selected by setting a threshold value in a conventional manner. In a scenario where large data of a massive financial transaction are subjected to data division and these data blocks are transferred into parallel computing nodes for outlier detection, since the computing efficiency of the parallel computing nodes needs to be maximized and the detection effect on outliers is optimized, it is necessary to make the time sequence lengths of the data blocks relatively similar, and to ensure that when the parallel computing nodes perform outlier detection, the outlier detection of data points near the division points does not have errors due to the division of the data blocks, and meanwhile, the data transmission between the parallel computing nodes is reduced to ensure the highest efficiency.
In the process of calculating the local outlier of the parallel nodes by using the data blocks obtained by clustering, as the time sequence data in the data blocks need to obtain data points of the neighborhood when calculating the local outlier, if the time sequence data is divided more inaccurately, the data calculated by the current parallel nodes need to obtain the data in other parallel nodes, so that data transmission between the parallel nodes is needed, and for massive financial transaction big data, more data transmission between the parallel nodes is needed, and meanwhile, the data processing efficiency is reduced.
Based on the method, an initial segmentation point is obtained from long time sequence data according to a flow algorithm, a final accurate segmentation point is determined through unidirectionality of a K-distance neighborhood of a data point in a local range of the initial segmentation point, and a sub-sequence is obtained through segmentation. In the embodiment of the present application, for the relevant parameters in the flow algorithm, the number of division points is set to numregions=n/50, where N represents the total number of data points included in the time sequence formed by massive financial transaction big data, numregions represents the number of division points, the forbidden zone range L is set to 20, and the implementer can set according to the specific implementation scenario.
After all the initial segmentation points are obtained through the traditional flow algorithm, since the K-distance neighborhood of the data point o is determined through the K data points nearest to the data point o in the time sequence, for the final accurate segmentation point, in order to ensure that fewer data transmissions are required in the process of calculating the local outlier in parallel calculation of the data point, namely, when the data point is represented to perform parallel calculation on the local outlier, the K-distance neighborhood data of the data point exists in the same subsequence, and the segmentation of the subsequence can also follow the basic principle of the flow algorithm, therefore, one data point with the highest K-distance neighborhood unidirectionality needs to be selected as the final segmentation point in the K-distance neighborhood of the initial segmentation point.
It should be noted that the number of data points located in the same direction within the K-distance neighborhood of data points characterizes the unidirectionality of the K-distance neighborhood of data points. In this embodiment, the K-distance neighborhood refers to a neighborhood range constituted by K data points having the smallest data difference from the data points. In this embodiment, the number K of data points in the neighborhood range has a value of 20, and the practitioner can set the data points according to a specific implementation scenario.
Based on the data points, in the neighborhood of the initial segmentation point, marking any one data point in the neighborhood as a target data point, and respectively acquiring the number of the data points which are positioned at the left side and the right side of the target data point and belong to the neighborhood of the target data; taking the larger value of the number of data points corresponding to the left side of the target data point and the number of data points corresponding to the right side of the target data point as a numerator, taking the total number of the data points contained in the neighborhood of the target data point as a denominator, and taking the normalized value of the ratio of the numerator to the denominator as a single degree of the neighborhood direction of the target data point.
In this embodiment, the data point i in the K-distance neighborhood of the initial division point a is described as the target data point, and is expressed as:wherein (1)>Representing a neighborhood direction single degree of a data point i in a K-distance neighborhood of an initial segmentation point a, namely, a neighborhood direction single degree of a target data point, +.>Represents the number of data points located to the right of and belonging to the K-distance neighborhood of data point i, +.>Represents the number of data points located to the left of and belonging to the K distance neighborhood of data point i, K represents the total number of data points contained in the neighborhood of data point i, +.>Representing a function of maximizing>Representing the normalization function.
The ratio of more data points positioned on the same side in the neighborhood of the data point i is represented, and the larger the ratio is, the more the number of data points positioned on the same side in the neighborhood of the data point i is, the larger the corresponding neighborhood direction is, and the larger the unidirectionality of the data point i in the K-distance neighborhood is.
The neighborhood direction single degree of the data point represents the degree of the direction single degree of the data point in the K-distance neighborhood, and the larger the neighborhood direction single degree value is, the more the number of the data points positioned on the same side in the neighborhood range of the data point is, and the more the degree of the single degree of the direction of the data point in the K-distance neighborhood is. The smaller the one-degree value of the neighborhood direction is, the smaller the number of the data points positioned on the same side in the neighborhood range of the data points is, and the smaller the unidirectional degree of the data points in the K-distance neighborhood is.
According to the method, a neighborhood direction single degree corresponding to the data point of each initial segmentation point in the K-distance neighborhood range is obtained, each data point in the neighborhood is screened according to the neighborhood direction single degree, the data point corresponding to the maximum value of the neighborhood direction single degree in the neighborhood of the initial segmentation point is recorded as a final segmentation point, and the time sequence is segmented by the final segmentation point to obtain a time sequence subsequence.
The final segmentation point is selected according to the unidirectional degree of the data point in the K-distance neighborhood of each segmentation point in the neighborhood range, and compared with the segmentation point in the traditional flow algorithm for segmenting long time sequence data, the method can reduce data transmission among parallel node data in the parallel computing process, and improves the data processing efficiency.
Step two, obtaining morphological similarity measurement indexes between any two time sequence subsequences according to the difference between the data in any two time sequence subsequences; and obtaining importance indexes corresponding to any two time sequence subsequences according to the length difference between any two time sequence subsequences.
After all time sequence sub-sequences corresponding to big data of financial transactions needing to be subjected to data cleaning are obtained, when the time sequence sequences formed by massive big data of financial transactions are segmented by utilizing final segmentation points in the embodiment of the application, although the time sequence sub-sequences obtained by segmentation are considered to enable parallel nodes to carry out less data transmission among nodes in the process of calculation, the situation that the length difference among the time sequence sub-sequences is large can exist, and the distance among the time sequence sub-sequences needs to be optimized in the process of clustering the time sequence sub-sequences.
In the clustering process of the time sequence sub-sequences, the time sequence sub-sequences with different lengths are required to be subjected to distance measurement, and the distance calculation between the long and short sequences can be performed due to the different lengths of the time sequence sub-sequences. In the process of calculating the DTW distance, the distance between the time sequence sub-sequences may have a distance measurement difference due to the different lengths. Therefore, the distance measurement of the subsequences needs to be optimized in the clustering process, and the average length of the time sequence subsequences in the category in the clustering process is limited, so that the data volume in the category obtained by clustering is similar, and the calculated volume of each parallel node is ensured to be similar.
In the conventional K-means clustering, the distance is measured by euclidean distance between data points, and when the time series are clustered, the distance is measured by DTW distance between the time series. The distance measurement between time sequences needs to consider the length difference of the sequences in the clustering process, and the larger the length difference between the two sequences is, the larger the measurement of the data distribution similarity is needed for the two sequences. Therefore, the distance between time sequence subsequences in the clustering process needs to be considered in two aspects, namely data distribution similarity and DTW distance, and the specific gravity between the two can be measured by the length difference.
Based on the above, a morphological similarity measurement index between any two time sequence subsequences is obtained according to the difference between corresponding data in any two time sequence subsequences, specifically, for any two time sequence subsequences, a change rate corresponding to data is obtained according to the difference between every two adjacent data in the time sequence subsequences; taking absolute values of differences between all change rate averages corresponding to the two time sequence subsequences as morphological similarity measurement indexes between any two time sequence subsequences, and expressing the morphological similarity measurement indexes as follows by a formula:
wherein (1)>A morphological similarity measure indicative of the temporal sub-sequence u and the temporal sub-sequence v, +.>Representing the rate of change of the mth data in the time sequence sub-sequence u, <>Indicating the length of the time sequence sub-sequence u, i.e. the total amount of data contained in the time sequence sub-sequence u,/->Representing the rate of change of the nth data correspondence in the time sequence subsequence v, <>The length of the time sequence subsequence v is indicated, i.e. the total amount of data contained in the time sequence subsequence v.
In this embodiment, the method for obtaining the change rate corresponding to the mth data in the time sequence subsequence u may be that a difference value between the mth data and the mth-1 data is calculated, and a ratio between the difference value and the mth-1 data is used as the change rate corresponding to the mth data, and it is to be noted that when the change rate corresponding to the first data is calculated, the value of the last data is defaulted to be 0.
In other embodiments, the method for obtaining the change rate corresponding to the mth data in the time sequence sub-sequence u may further be that, since the time sequence sub-sequence is a time sequence, there is a time interval between two adjacent data in the time sequence sub-sequence, that is, there is a time interval between the mth data and the m-1 data in the time sequence sub-sequence u. Based on the time interval between the mth data and the (m-1) th data in the time sequence subsequence is obtained, the difference value between the mth data and the (m-1) th data is calculated, and the ratio of the difference value to the time interval is taken as the corresponding change rate of the mth data.
The method for acquiring the rate of change corresponding to the data in the time sequence subsequence is the same as the method for acquiring the rate of change corresponding to the mth data in the time sequence subsequence u.
The difference between the data change conditions of the time sequence subsequence u and the time sequence subsequence v is represented, and the larger the difference is, the more dissimilar the data distribution conditions between the two time sequence subsequences are, namely the more dissimilar the morphology is, and the larger the corresponding morphology similarity measurement index is. The smaller the difference is, the more similar the data distribution situation between the two time sequence subsequences is, namely, the more similar the morphology is, and the smaller the corresponding morphology measurement index is.
When there is a difference in length between the time sequence sub-sequences, the last data point of the shorter time sequence sub-sequence corresponds to the data point of the longer time sequence sub-sequence in the process of calculating the DTW distance, and the phenomenon is caused by the difference in length between the time sequence sub-sequences. When the data distribution situation between two time sequence subsequences is more similar, the measurement distance between the two time sequence subsequences needs to be reduced, so that the time sequence subsequences with similar distribution situation but large length difference can be gathered into one type in the clustering process.
When the measurement distance between time sequence subsequences is obtained, the importance degree corresponding to the morphology similarity is obtained by normalizing the lengths of all the subsequences by using the maximum value, and the importance degree is used for measuring the measurement condition of the DTW distance and the morphology similarity. It can be further explained that the larger the difference in length between the time-series sub-sequences, the more important the measure of morphological similarity between the two time-series sub-sequences.
Based on the importance index, the importance index corresponding to any two time sequence subsequences is obtained according to the length difference between any two time sequence subsequences, and specifically, the absolute value of the difference of the length between every two time sequence subsequences is obtained and recorded as the length difference of the time sequence subsequences; for any two time sequence subsequences, the ratio between the length difference corresponding to the two time sequence subsequences and the maximum value in all the length differences is used as an importance index corresponding to any two time sequence subsequences, and the importance index is expressed as follows by a formula:wherein (1)>An importance index corresponding to the time sequence subsequence u and the time sequence subsequence v is represented by +.>Representing the length of the time sequence subsequence u, +.>Representing the length of the time sequence subsequence v, +.>Representing a function of maximizing>Indicating the difference in length->Representing the maximum value of the length differences corresponding to all any two time sequence sub-sequences.
The larger the length difference corresponding to the time sequence sub-sequence u and the time sequence sub-sequence v is, the larger the length difference is, the larger the corresponding importance index is, and the more important the similarity degree of the data distribution condition between the time sequence sub-sequence u and the time sequence sub-sequence v, namely the measurement of the morphological similarity between the two time sequence sub-sequences is required to be concerned.
The importance index characterizes the measurement importance of the similarity of the data distribution conditions among the time sequence subsequences, and the larger the importance index is, the more attention is required to be paid to the data distribution conditions among the time sequence subsequences, namely the morphological similarity among the time sequence subsequences. The smaller the importance index is, the less attention is required to the data distribution condition among time sequence subsequences, namely the morphological similarity among the time sequence subsequences, and the DTW distance is directly calculated when the measurement distance is obtained.
Step three, obtaining a state factor of the time sequence subsequence according to the abnormal condition of the data in the time sequence subsequence; obtaining a difference index between the time sequence subsequences according to the distance between the time sequence subsequences, the morphological similarity measurement index, the importance index and the state factor; and classifying the time sequence subsequences according to the difference indexes, and cleaning the massive financial transaction big data according to the classification results.
It should be noted that, the morphological similarity measure is a measure of the overall state of change between the time-series subsequences, and although the overall morphology between the time-series subsequences is similar, an outlier state difference may also occur between the time-series subsequences. It will be appreciated that, although the difference in the sorting distance between the time-series subsequences is small, there may be a difference between the outliers corresponding to the data in the time-series subsequences, and thus there is a higher overall value of the local outlier in the time-series subsequences with a larger outlier, and a lower overall value of the local outlier in the time-series subsequences with a smaller outlier, when the time-series subsequences with the difference between the two outliers are evaluated for abnormal data points, the local outlier with a lower value may be ignored, so that abnormal data points in the time-series subsequences with lower overall values of the local outlier may not be detected, and thus the data cleaning result is less accurate.
Based on this, the state factor of the time sequence subsequence is obtained according to the abnormal condition of the data in the time sequence subsequence, specifically, in this embodiment, the average link distance corresponding to the data in the time sequence subsequence is obtained by using a COF outlier factor detection algorithm, and the normalized value of the average link distance of all the data in the time sequence subsequence is used as the state factor of the time sequence subsequence.
The method for acquiring the average link distance corresponding to the data in the time sequence subsequence by using the COF outlier factor detection algorithm is a well-known technique, and the specific acquisition method is not described in the embodiment. The average link distance corresponding to the data reflects the abnormal condition of the data, and can reflect the outlier state of the data, namely, the larger the average link distance corresponding to the data is, the more likely the data is the outlier data point, so that the state factor of the time sequence subsequence characterizes the outlier state degree of the whole data in the time sequence subsequence, the larger the value of the state factor is, the larger the outlier state degree of the whole data in the time sequence subsequence is, and the overall numerical value of the local outlier factor of the data in the time sequence subsequence is calculated. The smaller the value of the state factor is, the smaller the degree of the outlier state of the whole data in the time sequence subsequence is, and the lower the whole data of the local outlier factor of the data in the time sequence subsequence is calculated.
Further, the closer the state factors of the two time sequence sub-sequences are, the closer the data in the two time sequence sub-sequences will be in the process of calculating the local outlier state factor. The larger the difference between the state factors of the two time sequence subsequences is, the larger the difference between the data in the two time sequence subsequences appears in the process of calculating the local outlier state factors, so that abnormal data in the time sequence subsequences with smaller state factors can be ignored, and certain data errors exist in the data cleaning process.
Based on the method, on the premise of considering the similarity of the data distribution conditions among the time sequence subsequences, the outlier state of the whole data in the time sequence subsequences is further measured through the state factors corresponding to the time sequence subsequences, so that the situation that abnormal data in the time sequence subsequences with lower values are ignored when abnormal data point evaluation is carried out on the time sequence subsequences with different abnormal values is avoided, and the abnormal data can be detected more accurately in the data cleaning process.
Further, according to the distance between the time sequence subsequences, the morphological similarity measurement index, the importance index and the state factor, a difference index between the time sequence subsequences is obtained, specifically, the DTW distance between any two time sequence subsequences is obtained, the difference between the state factors corresponding to the two time sequence subsequences is obtained, and according to the difference between the DTW distance and the state factor, the morphological similarity measurement index and the importance index, the difference index between the time sequence subsequences is obtained, and expressed as a formula: wherein (1)>Representing the index of difference between the time sequence subsequence u and the time sequence subsequence v,/>An importance index corresponding to the time sequence subsequence u and the time sequence subsequence v is represented by +.>Represents the DTW distance between the time sequence subsequence u and the time sequence subsequence v,/and>represents the DTW distance between the time sequence subsequence u and the time sequence subsequence v,/>Representing the time sequence subsequence u and the time sequence subsequence vMorphological similarity measurement index between ++>Representing the state factor corresponding to the time sequence sub-sequence u, < ->Representing the state factor corresponding to the time sequence subsequence v.
The importance index corresponding to the time sequence subsequence u and the time sequence subsequence vThe larger the weight of the similarity measurement index between the time sequence subsequences, namely the importance index is, the more attention is required to be paid to the data distribution situation between the time sequence subsequences, namely the morphological similarity between the time sequence subsequences. The smaller the importance index is, the larger the weight corresponding to the DTW distance is, the more no attention is required to be paid to the data distribution situation among time sequence subsequences, namely the morphological similarity among the time sequence subsequences, and the DTW distance is directly calculated when the measurement distance is obtained.
The larger the difference of the state factors between the time sequence subsequence u and the time sequence subsequence v is, the larger the difference of the state factors is, which means that the larger the difference of the data in the two time sequence subsequences appears in the process of calculating the local outlier state factors, the larger the corresponding DTW distance is, and the larger the finally calculated measurement distance is, namely the larger the difference index between the two time sequence subsequences is.
The DTW distance between the time sequence subsequences is limited by the length difference between the time sequence subsequences, the measurement distance in the clustering process is obtained through the morphological similarity measurement index of the time sequence subsequences and the DTW distance, the difference between the overall outlier states of all data in the time sequence subsequences is further considered, and finally, the measurement distance index which characterizes the time sequence subsequences accurately and the difference index between the time sequence subsequences are obtained.
And then, acquiring difference indexes between all arbitrary two time sequence subsequences according to the method, classifying the time sequence subsequences according to the difference indexes to obtain a plurality of categories, and forming the time sequence subsequences in each category into a data block for parallel calculation. In this embodiment, the K-means clustering algorithm is used to classify the time sequence subsequences according to the difference index, the difference index is used as an index for measuring the similarity between the time sequence subsequences, the similarity is inversely proportional to the difference index, the larger the similarity is, the smaller the difference index is, in this embodiment, the value of the cluster class number in the K-means clustering algorithm needs to be selected by an implementer according to the number of parallel computing nodes, for example, the cluster class number is set to be the same value as the number of the parallel computing nodes, or the cluster class number is set to be an integer multiple of the number of the parallel computing nodes. Meanwhile, the implementer can select other suitable clustering algorithms for classification according to specific implementation scenes.
It should be noted that parallel computing refers to an operation that can execute a plurality of instructions at a time, so as to increase the computing speed, that is, all data blocks are transferred into parallel computing nodes, and related data is computed in parallel in each computing node, so as to increase the efficiency of data cleaning.
And carrying out data cleaning on massive financial transaction big data according to the classification result, specifically, taking local outlier factors of data points in each category in the classification result as the abnormal degree of the data points, and eliminating the data points corresponding to the abnormal degree being greater than a preset degree threshold.
In this embodiment, a COF outlier detection algorithm is used to obtain a local outlier factor corresponding to each data point, and the algorithm is a well-known technique and will not be described herein.
The larger the local outlier factor corresponding to the data point is, the more likely the data is abnormal, and the greater the corresponding degree of abnormality is, the more the data needs to be removed. The smaller the local outlier factor corresponding to the data point is, the more likely the data is normal data, and the smaller the corresponding abnormality degree is.
Therefore, when the abnormality degree of the data point is greater than the degree threshold, it is indicated that the data point is more likely to be abnormal data, so that the data point needs to be removed, and the average value of the adjacent data points in the time sequence subsequence can be used as the fitting value of the data point, so that the number of the adjacent data points can be set according to the specific implementation scene. In this embodiment, the value of the range threshold is 0.7, and the practitioner may set the value of the range threshold according to a specific implementation scenario.
In summary, the time sequence subsequences are clustered by the difference index obtained by calculation in the embodiment of the application, the time sequence subsequences with more similar data distribution change are divided into the same category to form the data blocks, and compared with the data blocks formed by not classifying the time sequence subsequences, the data distribution in the data blocks in each parallel computing node can be ensured to be in a relatively close fluctuation range, so that deviation in the process of calculating local outliers of the data in the parallel computing node is avoided, and the accuracy in the data cleaning process is improved.
In the method for partitioning the time sequence subsequence, the data transmission quantity among the parallel computing nodes is evaluated when the parallel computing is performed in the K-distance neighborhood of the initial partitioning point, the unidirectional degree of the data point in the K-distance neighborhood of each partitioning point in the neighborhood range is obtained, and the final partitioning point is selected, so that the data transmission among the parallel node data in the parallel computing process can be reduced, and the data processing efficiency is improved.
Further, when the classification measurement distance between the time sequence sub-sequences is obtained, the classification measurement distance is optimized through measurement of the morphological similarity between the time sequence sub-sequences and the importance degree of the morphological similarity, compared with the traditional DTW distance, the classification measurement distance between the time sequence sub-sequences with larger length difference can be calculated, the classification measurement distance is reduced by using the morphological similarity between the time sequence sub-sequences, and therefore the time sequence sub-sequences with larger length difference in the clustering process can be classified into one class according to the morphological similarity.
Finally, the classification measurement distance after optimization of the morphological similarity is further optimized through the outlier states of all the data in the time sequence subsequences, so that when the morphological similarity among the time sequence subsequences is large, the classification measurement distance is further optimized through the outlier states of all the data in the time sequence subsequences, and when the abnormality degree of the data is measured by the parallel computing nodes, abnormal data error detection caused by outlier state difference among different time sequence subsequences in the same data block is avoided.
Example 2:
the embodiment provides an intelligent cleaning system for massive financial transaction big data, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the computer program realizes the steps of the intelligent cleaning method for massive financial transaction big data when being executed by the processor. Since embodiment 1 has already described a detailed description of a method for intelligent cleaning of massive financial transactions big data, it will not be described here too much.
The above embodiments are only for illustrating the technical solution of the present application, and not for limiting the same; although the application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the application and are intended to be included within the scope of the application.
Claims (5)
1. The intelligent cleaning method for mass financial transaction big data is characterized by comprising the following steps:
acquiring a time sequence subsequence corresponding to massive big data; establishing a data interface between a clearing data center and a financial transaction data source, collecting financial transaction data corresponding to a client of the clearing center, forming time sequence data according to a time stamp of the transaction data, and marking the time sequence data as a time sequence; dividing the time sequence to obtain a time sequence sub-sequence, wherein the time sequence sub-sequence is a financial transaction data time sequence sub-sequence; the financial transaction data includes one or more of securities, futures, foreign exchange, bond transaction data;
obtaining a morphological similarity measurement index between any two time sequence subsequences according to the difference between the data in any two time sequence subsequences; obtaining importance indexes corresponding to any two time sequence subsequences according to the length difference between any two time sequence subsequences;
obtaining a state factor of the time sequence subsequence according to the abnormal condition of the data in the time sequence subsequence; obtaining a difference index between the time sequence subsequences according to the distance between the time sequence subsequences, the morphological similarity measurement index, the importance index and the state factor; classifying the time sequence subsequences according to the difference indexes, and cleaning the data of massive big data according to the classification results;
classifying the time sequence subsequences according to the difference index by using a K-means clustering algorithm, wherein the value of the cluster class number in the K-means clustering algorithm is selected by an implementer according to the number of parallel computing nodes, and the cluster class number is set to be the same value as the number of the parallel computing nodes or is set to be an integer multiple of the number of the parallel computing nodes;
the method for obtaining the morphological similarity measurement index between any two time sequence subsequences according to the difference between the data in any two time sequence subsequences comprises the following specific steps:
for any two time sequence subsequences, obtaining a change rate corresponding to the data according to the difference between every two adjacent data in the time sequence subsequences; taking the absolute value of the difference value between all the change rate average values corresponding to the two time sequence subsequences as a morphological similarity measurement index between any two time sequence subsequences;
the importance index corresponding to any two time sequence subsequences is obtained according to the length difference between any two time sequence subsequences, and is specifically as follows:
acquiring the absolute value of the difference of the length between every two time sequence subsequences and recording the absolute value as the length difference of the time sequence subsequences;
for any two time sequence subsequences, the ratio between the length difference corresponding to the two time sequence subsequences and the maximum value in all the length differences is used as an importance index corresponding to any two time sequence subsequences;
the state factors for obtaining the time sequence subsequence according to the abnormal condition of the data in the time sequence subsequence are specifically as follows:
acquiring average link distances corresponding to the data in the time sequence subsequences by using a COF outlier factor detection algorithm, and taking a normalized value of an average value of the average link distances of all the data in the time sequence subsequences as a state factor of the time sequence subsequences;
the method for obtaining the difference index between the time sequence subsequences according to the distance between the time sequence subsequences, the morphological similarity measurement index, the importance index and the state factor specifically comprises the following steps:
acquiring a DTW distance between any two time sequence subsequences, acquiring differences between state factors corresponding to the two time sequence subsequences, and acquiring a difference index between the time sequence subsequences according to the DTW distance, the differences between the state factors, the morphological similarity measurement index and the importance index;
the calculation method of the difference index specifically comprises the following steps: wherein (1)>Representing the index of difference between the time sequence subsequence u and the time sequence subsequence v,/>An importance index corresponding to the time sequence subsequence u and the time sequence subsequence v is represented by +.>Representing DT between time sequence sub-sequence u and time sequence sub-sequence vW distance->Represents the DTW distance between the time sequence subsequence u and the time sequence subsequence v,/>A morphological similarity measure indicative of the temporal sub-sequence u and the temporal sub-sequence v, +.>Representing the state factor corresponding to the time sequence sub-sequence u, < ->Representing the state factor corresponding to the time sequence subsequence v.
2. The method for intelligently cleaning massive financial transaction big data according to claim 1, wherein the time sequence subsequence corresponding to the massive big data is specifically: and dividing the time sequence formed by the massive big data to obtain a time sequence sub-sequence.
3. The method for intelligently cleaning massive financial transaction big data according to claim 2, wherein the time sequence formed by massive big data is divided to obtain a time sequence subsequence specifically comprising:
acquiring an initial segmentation point corresponding to the time sequence by using a flow algorithm;
in the neighborhood of the initial dividing point, marking any data point in the neighborhood as a target data point, and respectively acquiring the number of data points which are positioned at the left side and the right side of the target data point and belong to the neighborhood of the target data;
taking the larger value of the number of data points corresponding to the left side of the target data point and the number of data points corresponding to the right side of the target data point as a numerator, taking the total number of the data points contained in the neighborhood of the target data point as a denominator, and taking the normalized value of the ratio of the numerator to the denominator as a single degree of the neighborhood direction of the target data point;
and recording the data point corresponding to the maximum value of the neighborhood direction singleness in the neighborhood of the initial segmentation point as a final segmentation point, and segmenting the time sequence by using the final segmentation point to obtain a time sequence subsequence.
4. The method for intelligently cleaning massive large data of financial transactions according to claim 1, wherein the step of cleaning massive large data according to classification results is specifically as follows:
and taking the local outlier factor of the data point in each category in the classification result as the abnormal degree of the data point, and removing the data point corresponding to the abnormal degree larger than the preset degree threshold.
5. A mass financial transaction big data intelligent cleaning system comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the computer program when executed by the processor performs the steps of a mass financial transaction big data intelligent cleaning method as claimed in any of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310537830.7A CN116383190B (en) | 2023-05-15 | 2023-05-15 | Intelligent cleaning method and system for massive financial transaction big data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310537830.7A CN116383190B (en) | 2023-05-15 | 2023-05-15 | Intelligent cleaning method and system for massive financial transaction big data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116383190A CN116383190A (en) | 2023-07-04 |
| CN116383190B true CN116383190B (en) | 2023-08-25 |
Family
ID=86964207
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310537830.7A Active CN116383190B (en) | 2023-05-15 | 2023-05-15 | Intelligent cleaning method and system for massive financial transaction big data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116383190B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116612641B (en) * | 2023-07-19 | 2023-09-22 | 天津中德应用技术大学 | Vehicle queue control data processing method based on intelligent network connection |
| CN116703485B (en) * | 2023-08-04 | 2023-10-20 | 山东创亿智慧信息科技发展有限责任公司 | Advertisement accurate marketing method and system based on big data |
| CN117725451B (en) * | 2023-12-15 | 2024-06-21 | 北京微保科技有限责任公司 | Automatic checking method and system for multidimensional transaction data |
| CN117422345B (en) * | 2023-12-18 | 2024-03-12 | 泰安金冠宏食品科技有限公司 | Oil-residue separation quality assessment method and system |
| CN117556108B (en) * | 2024-01-12 | 2024-03-26 | 泰安金冠宏食品科技有限公司 | Abnormal detection method for oil-residue separation efficiency based on data analysis |
| CN117909770B (en) * | 2024-03-20 | 2024-05-24 | 山东德源电力科技股份有限公司 | Intelligent settlement data storage method for single-phase fee-controlled electric energy meter |
| CN118070195B (en) * | 2024-04-16 | 2024-07-19 | 山东艾克索仑电气有限公司 | Mining alternating current frequency converter abnormal data state monitoring system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112966017A (en) * | 2021-03-01 | 2021-06-15 | 北京青萌数海科技有限公司 | Abnormal subsequence detection method with indefinite length in time sequence |
| CN113705726A (en) * | 2021-09-15 | 2021-11-26 | 北京沃东天骏信息技术有限公司 | Traffic classification method and device, electronic equipment and computer readable medium |
| WO2021238455A1 (en) * | 2020-05-29 | 2021-12-02 | 中兴通讯股份有限公司 | Data processing method and device, and computer-readable storage medium |
| CN115982611A (en) * | 2023-03-14 | 2023-04-18 | 北京易能中网技术有限公司 | Clustering algorithm-based power user energy characteristic analysis method |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP6661839B1 (en) * | 2018-07-23 | 2020-03-11 | 三菱電機株式会社 | Time series data diagnosis device, additional learning method, and program |
-
2023
- 2023-05-15 CN CN202310537830.7A patent/CN116383190B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021238455A1 (en) * | 2020-05-29 | 2021-12-02 | 中兴通讯股份有限公司 | Data processing method and device, and computer-readable storage medium |
| CN112966017A (en) * | 2021-03-01 | 2021-06-15 | 北京青萌数海科技有限公司 | Abnormal subsequence detection method with indefinite length in time sequence |
| CN113705726A (en) * | 2021-09-15 | 2021-11-26 | 北京沃东天骏信息技术有限公司 | Traffic classification method and device, electronic equipment and computer readable medium |
| CN115982611A (en) * | 2023-03-14 | 2023-04-18 | 北京易能中网技术有限公司 | Clustering algorithm-based power user energy characteristic analysis method |
Non-Patent Citations (1)
| Title |
|---|
| 基于时间序列挖掘的异常检测关键技术研究;展鹏;《中国博士学位论文全文数据库 基础科学辑》(第04期);第A002-65页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116383190A (en) | 2023-07-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116383190B (en) | Intelligent cleaning method and system for massive financial transaction big data | |
| US9454902B2 (en) | Performing-time-series based predictions with projection thresholds using secondary time-series-based information stream | |
| CN107742127A (en) | A kind of improved anti-electricity-theft intelligent early-warning system and method | |
| US20090222243A1 (en) | Adaptive Analytics | |
| CN109934301B (en) | Power load cluster analysis method, device and equipment | |
| CN111640304B (en) | Automatic quantitative extraction method for traffic jam propagation characteristics of continuous flow traffic facility | |
| CN116011894A (en) | Aluminum alloy rod production data management system | |
| CN117196446B (en) | Product risk real-time monitoring platform based on big data | |
| CN116739645A (en) | Order abnormity supervision system based on enterprise management | |
| CN118378199A (en) | Real-time anomaly detection method in big data analysis platform | |
| CN114722098A (en) | Typical load curve identification method based on normal cloud model and density clustering algorithm | |
| CN118012718B (en) | Real-time monitoring method for distributed storage system | |
| CN117828371A (en) | Intelligent analysis method for business information of comprehensive operation and maintenance platform | |
| CN117591860A (en) | Data anomaly detection method and device | |
| CN116258864B (en) | Village planning construction big data management system | |
| CN116151950B (en) | Intelligent banking outlet scheduling management method, system and storage medium | |
| CN116070150B (en) | Abnormality monitoring method based on operation parameters of breathing machine | |
| CN116295506A (en) | Method, device, equipment and medium for predicting vehicle remaining mileage | |
| KR20220123845A (en) | Meathod and device for measuring similarity between time series data | |
| CN117235651B (en) | Enterprise information data optimization management system based on Internet of things | |
| CN118393290B (en) | Power grid fault detection method based on 5G4G short sharing wireless communication | |
| CN117527523B (en) | Cloud computing-based server security monitoring system | |
| CN117196831B (en) | Financial service-oriented risk prediction method and system | |
| CN117953252B (en) | Automatic acquisition method and system for highway asset data | |
| CN117527523A (en) | Cloud computing-based server security monitoring system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |