CN111026740A - Data reconciliation method, system and data system based on data fingerprints - Google Patents
Data reconciliation method, system and data system based on data fingerprints Download PDFInfo
- Publication number
- CN111026740A CN111026740A CN201911217339.6A CN201911217339A CN111026740A CN 111026740 A CN111026740 A CN 111026740A CN 201911217339 A CN201911217339 A CN 201911217339A CN 111026740 A CN111026740 A CN 111026740A
- Authority
- CN
- China
- Prior art keywords
- data
- reconciliation
- fingerprint
- node
- abnormal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2471—Distributed queries
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/50—Reducing energy consumption in communication networks in wire-line communication networks, e.g. low power modes or reduced link rate
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Quality & Reliability (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Probability & Statistics with Applications (AREA)
- Software Systems (AREA)
- Storage Device Security (AREA)
Abstract
The invention provides a data reconciliation method based on data fingerprints, which comprises the following steps: the method comprises the following steps: a data provider provides data streams under different service scenes and automatically matches a data fingerprint calculation engine for the data streams; step two: dynamically acquiring key attribute and extreme change in the accessed data stream by combining a data characteristic dynamic self-adaptive adjustment algorithm, and generating a data fingerprint of the data stream according to the key attribute; step three: and traversing intermediate nodes of the data provider and the data access party through a transmission node tracing algorithm according to the data fingerprints, and providing the abnormal reconciliation result for the data provider and the data access party.
Description
Technical Field
The present invention relates to the field of information processing technology, and more particularly, to a data reconciliation method, system and data system based on data fingerprints.
Background
As society enters a big data era, the system layers for serving different services are infinite, data generated by the system grows in a geometric layer and has complex and various data structures, how to quickly and accurately access various massive data and automatically and timely provide account checking results of the accessed data become a core point for improving the utilization value of the big data.
The data reconciliation method based on the data fingerprints aims to solve the actual combat requirements, accurately and quickly acquires key business attributes in the data records through a data characteristic dynamic self-adaptive adjustment algorithm, generates the data fingerprints of the data records according to the key attributes and improves the accuracy of the data reconciliation; through a transmission node traceability algorithm, a final account checking result of the data is pushed to a data provider and a data access party in time, the capability of correcting abnormal data and the efficiency of retransmitting correct data are improved, and the data aggregation quality and value utilization capability of big data are improved.
Due to the characteristics of large data volume, complex and various structures and the like of each business system access, the existing data reconciliation method in the current market mainly carries out data reconciliation work in a mode of manually configuring a template, and the technologies have the following defects:
(1) the data reconciliation mode can only carry out data checking through a single manual configuration template, and under the condition that the structure of a data source is changed or the value of a data item is changed, the problem that effective data reconciliation cannot be carried out or reconciliation errors possibly occur exists.
(2) The account checking result can be pushed only to the last sending node, but in a big data era, data is often transmitted and gathered between different networks, the number of the transmission nodes is usually more than two or even more, so that the account checking result cannot be pushed to a real data provider in time, and abnormal data transmission cannot be processed in time.
Disclosure of Invention
Aiming at the problems, the invention mainly utilizes a data characteristic dynamic self-adaptive adjustment algorithm and a transmission node tracing algorithm to solve the problems, thereby improving the utilization value of big data.
Therefore, the invention mainly aims to provide a data reconciliation method, a data reconciliation system and a data system based on data fingerprints, so that automatic reconciliation of mass original data is realized, and the accuracy and timeliness of data reconciliation are improved.
In order to achieve the purpose, the invention provides a data reconciliation method based on data fingerprints, which comprises the following steps:
the method comprises the following steps: a data provider provides data streams under different service scenes and automatically matches a data fingerprint calculation engine for the data streams;
step two: dynamically acquiring key attribute and extreme change in the accessed data stream by combining a data characteristic dynamic self-adaptive adjustment algorithm, and generating a data fingerprint of the data stream according to the key attribute;
step three: and traversing intermediate nodes of the data provider and the data access party through a transmission node tracing algorithm according to the data fingerprints, and providing the abnormal reconciliation result for the data provider and the data access party.
Preferably, in step two, the variation of the key attribute comprises: and changing the key attribute, periodically analyzing the data item definition description of the data source according to the type tag of the key attribute, and automatically adjusting the reconciliation key attribute according to the configuration parameters when the type tag of the reconciliation key attribute is changed.
Preferably, in step two, the changing of the key attribute further comprises:
and when the proportion is compared with the abnormal proportion of the whole transmission period, the probability of occurrence reaches a set threshold value, and then abnormal adjustment is automatically triggered.
Preferably, in the second step, when the key attribute changes, the accurate key attribute information is automatically updated, and then the encryption engine is automatically matched to generate the specific data fingerprint.
Preferably, in step three, the process of traversing the data provider and the data access party by using the transmission node tracing algorithm is as follows:
the system automatically calculates the MD5 value of each transmission data of the data sender from the data, the data provider, the data access party and the transmission time, the MD5 value is used as the fingerprint of the data transmission path, each node of the data transmission automatically generates a number, inherits the number of the last access node to obtain a data set of the path node, and the path fingerprint and the node identification are stored in the data set;
acquiring a path set with an abnormal account checking result, circularly traversing the data set, comparing the path characteristics of the path set and the data set to obtain a transmission node set with an abnormal account checking result, and then acquiring node information of all data sending parties with abnormal account checking according to the node characteristics of the data sending parties;
and pushing the abnormal account checking result to a storage path under an appointed IP and an appointed port according to the node information of the data sender, and finishing the data account checking operation.
Preferably, the data fingerprint is a unique credential for the data reconciliation.
Preferably, in the third step, a warning prompt is given once the reconciliation result is abnormal, otherwise, the reconciliation result is normal, and no warning prompt is given.
Preferably, the present invention further provides a data reconciliation system based on data fingerprints, comprising:
the data access unit is used for acquiring and transmitting a data stream of a data provider;
the data analysis unit analyzes and sorts the data by using a data characteristic dynamic self-adaptive adjustment algorithm and generates data fingerprints;
and the data fingerprint judging unit is used for judging the data fingerprint and pushing an abnormal reconciliation result.
Preferably, the data fingerprint determining unit determines whether the data reconciliation result is abnormal, and if so, the data reconciliation result is further transmitted to the transmission node tracing algorithm unit for processing, otherwise, the data reconciliation result is ended.
Preferably, the present invention further provides a data system characterized by comprising a plurality of transmission nodes including a first transmission node and one or more second transmission nodes, wherein
The first transmission node comprises a data grouping unit, and the data grouping unit is used for grouping and inducing the data flow of the data provider according to the characteristics;
and the second transmission node is used for performing key attribute change judgment and key service attribute abnormity judgment on the received data stream and encrypting the judgment result through the encryption engine.
Compared with the prior art, the invention has the following characteristics:
(1) the method for data reconciliation is different, the invention uses a data characteristic dynamic self-adaptive adjustment algorithm, dynamically acquires key attributes accessed into the data record according to the data source structure and the characteristics of the data item value, and generates the data fingerprint of the data record according to the key attributes, thereby improving the accuracy of the data reconciliation.
(2) The invention adopts a transmission node tracing algorithm to traverse the intermediate nodes of the data provider and the data access party and to push the abnormal reconciliation result to the data provider and the data access party in time, thereby improving the capability of correcting error data and the efficiency of retransmitting abnormal data and improving the data aggregation quality and value utilization capability of big data.
Drawings
Hereinafter, the present invention will be explained in more detail with reference to the accompanying drawings. The same reference numbers will be used throughout the drawings to refer to the same or like features:
FIGS. 1a and 1b are overall analysis flow diagrams of a data reconciliation method based on data fingerprints according to the present invention;
FIG. 2 is a data characteristic dynamic adaptive rectification flowchart of a data reconciliation method based on data fingerprints according to the present invention;
FIG. 3 is a flow chart of a transmission node tracing method for a data reconciliation based on data fingerprints according to the present invention;
fig. 4 is a block diagram of a data reconciliation system based on data fingerprints in accordance with the present invention.
Detailed Description
To further illustrate the various embodiments, the invention provides the accompanying drawings. The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the embodiments. Those skilled in the art will appreciate still other possible embodiments and advantages of the present invention with reference to these figures. Elements in the figures are not drawn to scale and like reference numerals are generally used to indicate like elements.
The invention will now be further described with reference to the accompanying drawings and detailed description.
Referring to fig. 1a and 1b, fig. 1a and 1b are overall analysis flow charts of a data reconciliation method based on data fingerprints according to the present invention. Aiming at the problem of how to efficiently and accurately perform data reconciliation when massive original data are accessed, the invention dynamically obtains key attributes in the accessed data records through 2 algorithms of a data characteristic dynamic self-adaptive adjustment algorithm and a transmission node traceability algorithm, generates data fingerprints of the data records according to the key attributes, traverses intermediate nodes of a data provider and a data access party, and pushes an abnormal reconciliation result to the data provider and the data access party in time, thereby improving the accuracy and the timeliness of the data reconciliation.
According to one embodiment of the invention, a data reconciliation method based on data fingerprints is provided, which comprises the following steps:
the method comprises the following steps: a data provider provides data streams under different service scenes and automatically matches a data fingerprint calculation engine for the data streams; step two: combining a data characteristic dynamic self-adaptive adjustment algorithm, dynamically acquiring key attributes and changes thereof in the accessed data stream, and generating a data fingerprint of the data stream according to the key attributes; step three: and traversing intermediate nodes of the data provider and the data access party through a transmission node tracing algorithm according to the data fingerprints, and providing the abnormal reconciliation result for the data provider and the data access party.
With further reference to fig. 2, fig. 2 is a data characteristic dynamic adaptive rectification flowchart of the data reconciliation method based on data fingerprints according to the present invention. The invention mainly realizes automatic reconciliation of accessing massive original data through 2 algorithms of a data characteristic dynamic self-adaptive adjustment algorithm and a transmission node tracing algorithm, improves the accuracy and timeliness of data reconciliation, and has the following main analysis models:
in the second step, a dynamic self-adaptive adjustment algorithm of data characteristics is mainly applied. With the development of business, the structure of access data is changed frequently, or the value of a certain original important data item is changed, if a certain key data item cannot acquire a new value due to the change of business conditions and is empty all the time, in such a scene, the problem that effective data reconciliation or reconciliation errors cannot be performed can be caused by checking data through a single manual configuration template.
1) Key attribute change: and periodically analyzing the data item definition description of the data source according to the type tags of the key attributes, such as a business main key, a combination main key, a dictionary, change time and the like, and automatically adjusting the reconciliation key attributes according to the configuration parameters when the type tags of the reconciliation key attributes are changed.
2) The key service attribute is abnormal: and (3) acquiring abnormal proportions of the null values, the repetition values and the like within a specific time by using the principle of the service attribute abnormality judgment algorithm, and automatically triggering abnormality adjustment when the proportion is compared with the abnormal proportion in the whole transmission period and the occurrence probability reaches a set threshold value. Describing an algorithm: if the difference between the current abnormal proportion in the Cn in the characteristic rule base and the integral abnormal proportion exceeds 30%, automatically triggering abnormal adjustment and moving the data item out of the key characteristic attribute; if the difference is between 10 and 30 percent, reminding an administrator to perform manual check; if the difference is within 10%, no early warning is executed;
3) an encryption engine: calling the pair to generate a data fingerprint by an encryption engine according to the value distribution condition of the key service attribute of the reconciliation rule base, taking the data fingerprint as a final result set, and storing the data fingerprint in a database;
4) data fingerprint: as the only proof of the data reconciliation.
Referring to fig. 3, fig. 3 is a flow chart of a transmission node tracing of a data reconciliation method based on data fingerprints according to the present invention. After the data fingerprint is obtained in the step two, account checking can be further performed, namely, the step three. In the data access in the third step, the final data access party is often reached through two or more transmission nodes across a plurality of different networks, and the traditional method can only push the reconciliation result to the last data sending party, which is not a data providing party, so that effective correction cannot be performed. According to the method, a unique fingerprint is given to each data transmission path through methods of transmission node fingerprinting, feature matching and the like, so that the node of a data provider can be quickly found, and an abnormal reconciliation result can be pushed.
Specifically, the data reconciliation process is mainly based on the following two core libraries: the account checking rule base is used for acquiring key characteristic attributes of different types of data sources and matching data fingerprint calculation engine information, and is defined as the following table a; a transmission node library: which is used to obtain the transmission node information of the data, is defined as the following table b.
| Attribute name | Attribute description | Remarks for note |
| JLID | Recording ID | |
| GZID | Rule numbering | |
| TGH | Data provider | |
| JLH | Data access side | |
| CJZX | Data resources | |
| GJXWCX | Key business attribute | |
| ZT | Status of state | 0-unavailable, 1-available |
| BGXY | Reason for change | |
| BGXJ | Time of change | |
| YCBL | Abnormal ratio | |
| ZTYCBL | Overall abnormal ratio | |
| ZMYC | Encryption engine |
TABLE a
Table b
By optimizing the matching rules of dynamic changes of different data source characteristics and combining the reconciliation rule base, the high-efficiency real-time reconciliation of various mass original data is automatically and efficiently supported and accessed by a computer program, the requirement of large data aggregation and proofreading is met, and the problem that effective data reconciliation cannot be carried out or reconciliation errors are possible to occur under the condition that the data source which is long-troubled changes or the value of a data item changes is solved.
However, the main approach to data reconciliation relies on:
1) path fingerprint: each data transmission system of the data sender automatically calculates the MD5 value as the fingerprint of the data transmission path by the data, the data provider, the data access party and the transmission time, each node of the data transmission automatically generates the number of the "| + 8-bit random letter" and inherits the number of the last access node to obtain a set S of data path nodes, wherein the set S comprises n subsets { S1, S2, …, Sn }, and the path fingerprint and the node identification are stored in the data set;
2) and (3) feature matching: and acquiring a path set Xn with an abnormal account checking result, circularly traversing the path fingerprint set S, comparing the path characteristics of the Xn and the Sn to obtain a transmission node set Mn with an abnormal account checking result, and acquiring node information of all data sending parties with abnormal account checking according to the node characteristics of the data sending parties.
3) And pushing a result: and pushing the abnormal account checking result to a storage path under an appointed IP and an appointed port according to the node information of the data sender, and finishing the data account checking operation.
By providing 2 optimization algorithms, namely a data characteristic dynamic self-adaptive adjustment algorithm and a transmission node tracing algorithm, key attributes in an access data record are dynamically acquired, data fingerprints of the data record are generated according to the key attributes, intermediate nodes of a data provider and a data access party are traversed, an abnormal reconciliation result is timely pushed to the data provider and the data access party, and the accuracy and the timeliness of data reconciliation are improved.
According to another embodiment of the present invention, there is provided a data reconciliation system based on data fingerprints, comprising: the data access unit is used for acquiring and transmitting a data stream of a data provider; the data analysis unit analyzes and sorts the data by using a data characteristic dynamic self-adaptive adjustment algorithm and generates data fingerprints; and the data fingerprint judging unit is used for judging the data fingerprint and pushing an abnormal reconciliation result. The data fingerprint judging unit judges whether the data reconciliation result is abnormal or not, if so, the data fingerprint judging unit further transmits the data reconciliation result to the transmission node tracing algorithm unit for processing, otherwise, the data fingerprint judging unit ends.
According to yet another embodiment of the present invention, there is provided a data system including a plurality of transmission nodes including a first transmission node and one or more second transmission nodes, wherein the first transmission node includes a data grouping unit for grouping data streams of a data provider according to characteristics; the second transmission node is configured to perform key attribute change determination and key service attribute abnormality determination on a received data stream, and encrypt a determination result by using an encryption engine, where the function of the second transmission node is similar to a set of data characteristic dynamic adaptive tuning routines, but is not limited to this.
Claims (10)
1. The data reconciliation method based on the data fingerprint is characterized by comprising the following steps of:
the method comprises the following steps: a data provider provides data streams under different service scenes and automatically matches a data fingerprint calculation engine for the data streams;
step two: combining a data characteristic dynamic self-adaptive adjustment algorithm, dynamically acquiring key attributes and changes thereof in the accessed data stream, and generating a data fingerprint of the data stream according to the key attributes;
step three: and traversing intermediate nodes of the data provider and the data access party through a transmission node tracing algorithm according to the data fingerprints, and providing the abnormal reconciliation result for the data provider and the data access party.
2. The data reconciliation method based on the data fingerprint of claim 1 wherein in step two, the change of the key attribute comprises: and changing the key attribute, periodically analyzing the data item definition description of the data source according to the type tag of the key attribute, and automatically adjusting the reconciliation key attribute according to the configuration parameters when the type tag of the reconciliation key attribute is changed.
3. The data reconciliation method based on the data fingerprint of claim 2 wherein in step two, the changing of the key attribute further comprises:
and when the proportion is compared with the abnormal proportion of the whole transmission period, the probability of occurrence reaches a set threshold value, and then abnormal adjustment is automatically triggered.
4. The data reconciliation method based on the data fingerprint of claim 3, wherein in step two, when the key attribute changes, the accurate key attribute information is automatically updated, and then the encryption engine is automatically matched to generate the specific data fingerprint.
5. The data reconciliation method based on the data fingerprint of claim 1, wherein in step three, the process of traversing the data provider and the data access party by using the transmission node tracing algorithm comprises the following steps:
the system automatically calculates the MD5 value of each transmission data of the data sender from the data, the data provider, the data access party and the transmission time, the MD5 value is used as the fingerprint of the data transmission path, each node of the data transmission automatically generates a number, inherits the number of the last access node to obtain a data set of the path node, and the path fingerprint and the node identification are stored in the data set;
acquiring a path set with an abnormal account checking result, circularly traversing the data set, comparing the path characteristics of the path set with the path characteristics of the data set to obtain a transmission node set with an abnormal account checking result, and then acquiring node information of all data sending parties with abnormal account checking according to the node characteristics of the data sending parties;
and pushing the abnormal account checking result to a storage path under an appointed IP and an appointed port according to the node information of the data sender, and finishing the data account checking operation.
6. The data reconciliation method based on a data fingerprint of claim 1 wherein the data fingerprint is a unique credential for data reconciliation.
7. The data reconciliation method based on the data fingerprint as claimed in claim 1, wherein in the third step, a warning prompt is performed once the reconciliation result is abnormal, otherwise, the reconciliation result is normal, and no warning prompt is performed.
8. Data reconciliation system based on data fingerprints, characterized by comprising:
the data access unit is used for acquiring and transmitting a data stream of a data provider;
the data analysis unit analyzes and sorts the data by using a data characteristic dynamic self-adaptive adjustment algorithm and generates data fingerprints;
and the data fingerprint judging unit is used for judging the data fingerprint and pushing an abnormal reconciliation result.
9. The data reconciliation system based on the data fingerprint of claim 8, wherein the data fingerprint determining unit determines whether the result of the data reconciliation is abnormal, and if so, the result is further transmitted to the transmission node tracing algorithm unit for processing, otherwise, the data reconciliation system is ended.
10. A data system comprising a plurality of transport nodes including a first transport node and one or more second transport nodes, wherein
The first transmission node comprises a data grouping unit, and the data grouping unit is used for grouping and inducing the data flow of the data provider according to characteristics;
and the second transmission node is used for performing key attribute change judgment and key service attribute abnormity judgment on the received data stream and encrypting the judgment result through the encryption engine.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911217339.6A CN111026740B (en) | 2019-12-03 | 2019-12-03 | Data reconciliation method and system based on data fingerprints |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911217339.6A CN111026740B (en) | 2019-12-03 | 2019-12-03 | Data reconciliation method and system based on data fingerprints |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111026740A true CN111026740A (en) | 2020-04-17 |
| CN111026740B CN111026740B (en) | 2022-07-12 |
Family
ID=70207825
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911217339.6A Active CN111026740B (en) | 2019-12-03 | 2019-12-03 | Data reconciliation method and system based on data fingerprints |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111026740B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101494658A (en) * | 2008-01-24 | 2009-07-29 | 华为技术有限公司 | Method, apparatus and system for implementing fingerprint technology |
| US20100306635A1 (en) * | 2009-05-28 | 2010-12-02 | Emulex Design & Manufacturing Corporation | Method for Verifying Correct Encryption Key Utilization |
| CN104317823A (en) * | 2014-09-30 | 2015-01-28 | 北京合力思腾科技股份有限公司 | Method for carrying out data detection by utilizing data fingerprints |
| CN106506274A (en) * | 2016-11-08 | 2017-03-15 | 东北大学秦皇岛分校 | A kind of efficient single bag source tracing method of dynamic extending |
| CN107016542A (en) * | 2016-12-06 | 2017-08-04 | 阿里巴巴集团控股有限公司 | A kind of business data processing method, verification method, apparatus and system |
| CN109088903A (en) * | 2018-11-07 | 2018-12-25 | 湖南大学 | A kind of exception flow of network detection method based on streaming |
| CN110378778A (en) * | 2019-07-10 | 2019-10-25 | 中信百信银行股份有限公司 | Multi-data source account checking method, system, electronic equipment and storage medium |
-
2019
- 2019-12-03 CN CN201911217339.6A patent/CN111026740B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101494658A (en) * | 2008-01-24 | 2009-07-29 | 华为技术有限公司 | Method, apparatus and system for implementing fingerprint technology |
| US20100306635A1 (en) * | 2009-05-28 | 2010-12-02 | Emulex Design & Manufacturing Corporation | Method for Verifying Correct Encryption Key Utilization |
| CN104317823A (en) * | 2014-09-30 | 2015-01-28 | 北京合力思腾科技股份有限公司 | Method for carrying out data detection by utilizing data fingerprints |
| CN106506274A (en) * | 2016-11-08 | 2017-03-15 | 东北大学秦皇岛分校 | A kind of efficient single bag source tracing method of dynamic extending |
| CN107016542A (en) * | 2016-12-06 | 2017-08-04 | 阿里巴巴集团控股有限公司 | A kind of business data processing method, verification method, apparatus and system |
| CN109088903A (en) * | 2018-11-07 | 2018-12-25 | 湖南大学 | A kind of exception flow of network detection method based on streaming |
| CN110378778A (en) * | 2019-07-10 | 2019-10-25 | 中信百信银行股份有限公司 | Multi-data source account checking method, system, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111026740B (en) | 2022-07-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109510737B (en) | Protocol interface testing method and device, computer equipment and storage medium | |
| US20180302306A1 (en) | Complementary workflows for identifying one-hop network behavior and multi-hop network dependencies | |
| Dai et al. | Finding persistent items in distributed datasets | |
| US10621180B2 (en) | Attribute-based detection of anomalous relational database queries | |
| Mao et al. | Minimax rates and efficient algorithms for noisy sorting | |
| US11250166B2 (en) | Fingerprint-based configuration typing and classification | |
| US9413760B2 (en) | System and method for data quality analysis between untrusted parties | |
| CN110619019B (en) | Distributed storage method and system for data | |
| CN113591485B (en) | Intelligent data quality auditing system and method based on data science | |
| CN116308824A (en) | Knowledge graph-based group fraud risk identification method and related equipment | |
| CN110768856B (en) | Network flow measuring method, network measuring equipment and control plane equipment | |
| CN112560878A (en) | Service classification method and device and Internet system | |
| CN111651741B (en) | User identity recognition method, device, computer equipment and storage medium | |
| CN114189347A (en) | Data safety transmission method combining data granulation and gatekeeper | |
| WO2021000645A1 (en) | Contract generation method and apparatus based on data crawling and computer device | |
| US20230004776A1 (en) | Moderator for identifying deficient nodes in federated learning | |
| CN111026740B (en) | Data reconciliation method and system based on data fingerprints | |
| CN113297253A (en) | Equipment identification method, device, equipment and readable storage medium | |
| CN115934484B (en) | Diffusion model data enhancement-based anomaly detection method, storage medium and apparatus | |
| Qian et al. | Collaborative sampling for partial multi-dimensional value collection under local differential privacy | |
| CN116303379A (en) | Data processing method, system and computer storage medium | |
| CN111209284B (en) | Meta-data-based sub-table method and device | |
| CN112564928B (en) | Service classification method and device and Internet system | |
| CN112560877A (en) | Service classification method and device and Internet system | |
| CN110569435B (en) | Intelligent dual-ended recommendation engine system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |