CN117149842A - Heterogeneous multi-chain data supervision system - Google Patents
Heterogeneous multi-chain data supervision system Download PDFInfo
- Publication number
- CN117149842A CN117149842A CN202210572383.4A CN202210572383A CN117149842A CN 117149842 A CN117149842 A CN 117149842A CN 202210572383 A CN202210572383 A CN 202210572383A CN 117149842 A CN117149842 A CN 117149842A
- Authority
- CN
- China
- Prior art keywords
- data
- database
- heterogeneous
- chain
- abnormal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000002159 abnormal effect Effects 0.000 claims abstract description 39
- 238000012795 verification Methods 0.000 claims abstract description 16
- 238000013524 data verification Methods 0.000 claims abstract description 6
- 238000000034 method Methods 0.000 claims abstract description 5
- 239000000523 sample Substances 0.000 claims abstract description 4
- 238000012545 processing Methods 0.000 claims description 28
- 238000013500 data storage Methods 0.000 claims description 7
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 230000008859 change Effects 0.000 claims description 4
- 238000013523 data management Methods 0.000 claims description 3
- 238000013499 data model Methods 0.000 claims description 3
- 238000013461 design Methods 0.000 claims description 3
- 230000001788 irregular Effects 0.000 claims description 3
- 238000007726 management method Methods 0.000 claims description 3
- 238000012544 monitoring process Methods 0.000 claims description 2
- 230000008569 process Effects 0.000 claims description 2
- 230000005540 biological transmission Effects 0.000 abstract 1
- 238000011161 development Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000005065 mining Methods 0.000 description 2
- 206010063385 Intellectualisation Diseases 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2474—Sequence data queries, e.g. querying versioned data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/283—Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Fuzzy Systems (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a heterogeneous multi-chain data supervision system, which comprises a heterogeneous multi-chain data supervision system and is characterized in that the method of the heterogeneous multi-chain data supervision system is as follows: s1: heterogeneous multi-link data access: the data with different versions, different types and different structures are accessed through a data port, so that the multi-chain data are collected and summarized; s2: heterogeneous data acquisition unit: the heterogeneous data entering through data transmission is subjected to data acquisition, the preposed probe arranged in the acquisition unit can acquire the data, the acquired data are placed in a big data application system or a cloud service system, after the acquisition, the data are subjected to data verification with the heterogeneous multi-chain database through the data supervision and identification unit, after the verification, abnormal heterogeneous data can be rapidly processed through the data supervision and treatment unit, the abnormal processed data are subjected to alarm and notification, and therefore safety and stability of a market data chain are guaranteed and maintained.
Description
Technical Field
The invention relates to the technical field of data supervision, in particular to a heterogeneous multi-chain data supervision system.
Background
Heterogeneous data are different in types and different versions or data with different structures, a data multi-link refers to a link for communicating data, the data link is a data network, like the Internet, only one data terminal is needed to obtain information needed by the user from the data link, like the Internet, the user can use the terminal to add things to the data link network.
With the continuous development of modern blockchain technology, the blockchain is used as a decentralizing infrastructure, the characteristics of distributed type, non-falsification, disclosure transparency and the like solve a plurality of practical problems, but also bring information security problems, at present, no matter a public blockchain system or a licensed blockchain system, each peer node writes data into the blockchain and reads data from the blockchain by itself, so that harmful or illegal information cannot be prevented from being written into the blockchain, and the harmful or illegal information cannot be prevented from being read for propagation, and in order to ensure orderly operation of the type of data chain, the security scene modeling is carried out to realize data depth mining, more detailed data are acquired, the trabecular mark of network attack mixed under massive data is exposed, reliable basis is provided for searching network attack, market data chain security and stability are ensured, and an isomeric multi-chain data system is provided for solving the problems.
Disclosure of Invention
In view of the shortcomings of the prior art, the present invention provides a heterogeneous multi-link data supervision system to solve the above problems.
In order to achieve the above purpose, the present invention adopts the following technical scheme:
a heterogeneous multi-chain data supervision system comprising a heterogeneous multi-chain data supervision system, wherein the heterogeneous multi-chain data supervision system comprises the following steps:
s1: heterogeneous multi-link data access: the data with different versions, different types and different structures are accessed through a data port, so that the multi-chain data are collected and summarized;
s2: heterogeneous data acquisition unit: the heterogeneous data transmitted by the data are subjected to data acquisition, and the preposed probes arranged in the acquisition unit can acquire the data and place the acquired data in a big data application system or a cloud service system;
s3: data supervision and identification unit: the collected multi-chain data is subjected to comparison and verification with a heterogeneous multi-chain database, after data verification, the verified data result can be classified, normal heterogeneous data is verified and transmitted to a data warehouse, abnormal heterogeneous data is sent to an abnormal data storage for storage, and the subsequent data processing is waited;
s4: the data supervision processing unit: and (3) monitoring and processing the abnormal heterogeneous data, wherein in the processing process, firstly, the abnormal data is received by an abnormal data receiving module, after the abnormal data is received, the abnormal data is started by a verification starting module, after the verification starting module is started, the data containing the encrypted module is decrypted by a seal conversion module, and finally, the data is inspected by a data original code.
Preferably, in S3, the heterogeneous multi-chain database comprises a relational database, a graphic database, a time-series database, and an unstructured database;
relational database: relational databases, which are databases that employ a relational model to organize data, store data in rows and columns for ease of user understanding, a series of rows and columns of the relational database are referred to as tables, a set of tables comprising the database;
graphic database: the graph database is a data management system which takes points and edges as basic storage units and takes efficient storage and query graph data as design principles;
time series database: the time sequence database is mainly used for processing data with time labels (which change according to the sequence of time and are instant in time sequence), and the data with the time labels are also called time sequence data and are novel non-relational databases;
unstructured database: unstructured data is data represented by a two-dimensional logical table of a database, which is irregular or incomplete in data structure, has no predefined data model, and is inconvenient.
Preferably, the heterogeneous multi-chain database is a set of a plurality of related database systems, so that sharing and transparent access of data can be realized, a plurality of database systems exist before joining the heterogeneous database systems, each component part of the own database management system and the external database has own autonomy, and each database system still has own application characteristics, integrity control and security control while realizing data sharing.
Preferably, the data warehouse comprises a database conversion module and a database transparent access module, and can convert and load the data with normal verification into the heterogeneous multi-chain database, and meanwhile, the database transparent access module is convenient for direct access after the next verification.
Preferably, the data warehouse can collect information from a plurality of databases in the heterogeneous database system, establish a unified global mode, and simultaneously support access to historical data, and a user performs decision-supported query through a unified data interface provided by the data warehouse.
Preferably, in S4, after the data original code is inspected, qualified heterogeneous data is inspected, and transferred to a data warehouse, so that the data can be stored conveniently, abnormal heterogeneous data is inspected, and the abnormal heterogeneous data is sent to an early warning notification module.
Preferably, the early warning notification module comprises a message early warning system, a message notification system and a task distribution processing system, and sends out message early warning to abnormal heterogeneous data, and then notifies the task distribution system of the early warning message, distributes the early warning message to corresponding processing staff, and performs abnormal processing.
Compared with the prior art, the invention has the beneficial effects that: the heterogeneous multi-chain data supervision system can carry out data access on various heterogeneous multi-chain data, then collect the accessed heterogeneous data, carry out data verification with a heterogeneous multi-chain database through a data supervision and identification unit after collection, and carry out data supervision and processing unit after verification, so that abnormal heterogeneous data can be rapidly processed, the abnormal processed data is warned and notified, and the safety and stability of a market data chain are ensured and maintained;
and after the received heterogeneous data and the heterogeneous multi-chain database are checked to be qualified, the data are input into a data warehouse, so that the data storage of the database is increased, and whether the data are abnormal or not can be quickly identified after the subsequent data are received again.
Drawings
FIG. 1 is a general flow chart of multi-chain data supervision in accordance with the present invention;
FIG. 2 is a flow chart of a heterogeneous data collection unit according to the present invention;
FIG. 3 is a flow chart of a data supervision and identification unit according to the present invention;
FIG. 4 is a flow chart of a data supervision processing unit according to the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Examples: referring to fig. 1-4, a heterogeneous multi-chain data supervision system, comprising a heterogeneous multi-chain data supervision system, wherein the heterogeneous multi-chain data supervision system method is as follows:
s1: heterogeneous multi-link data access: the data with different versions, different types and different structures are accessed through a data port, so that the multi-chain data are collected and summarized;
s2: heterogeneous data acquisition unit: the heterogeneous data transmitted by the data are subjected to data acquisition, and the preposed probes arranged in the acquisition unit can acquire the data and place the acquired data in a big data application system or a cloud service system;
s3: data supervision and identification unit: the collected multi-chain data is subjected to comparison and verification with the heterogeneous multi-chain database, after the data is verified, the verified data result can be classified, normal heterogeneous data is verified and transmitted to the data warehouse, abnormal heterogeneous data is sent to the abnormal data storage for storage, subsequent data processing is waited, after the data is verified to be qualified with the heterogeneous multi-chain database, the data warehouse is entered, the data storage of the database is increased, and whether the data are abnormal or not can be rapidly identified after the subsequent data are received again;
s4: the data supervision processing unit: the method comprises the steps of performing supervision processing on abnormal heterogeneous data, firstly receiving the abnormal heterogeneous data through an abnormal data receiving module, after receiving the abnormal heterogeneous data, performing decryption on the data containing an encrypted module through a sealed conversion module after the verification starting module is started through a verification starting module, finally performing data access on various heterogeneous multi-chain data through data original codes, then collecting the accessed heterogeneous data, performing data verification with a heterogeneous multi-chain database through a data supervision and identification unit after collection, and performing data verification with the heterogeneous multi-chain database after verification, rapidly processing the abnormal heterogeneous data through a data supervision and treatment unit, alarming and notifying the abnormal treated data, so that market data chain safety and stability are guaranteed and maintained.
In S3, the heterogeneous multi-chain database includes a relational database, a graphic database, a time-series database, and an unstructured database;
relational database: relational database, which is a database that employs a relational model to organize data, which stores data in rows and columns for the convenience of user understanding, a series of rows and columns of the relational database are referred to as tables, a set of tables forms the database, a user retrieves data in the database by querying, which is an execution code that defines certain areas in the database, a relational model can be simply understood as a two-dimensional table model, and a relational database is a data organization made up of two-dimensional tables and relationships between them;
graphic database: the graph database is a data management system taking points and edges as basic storage units and taking efficient storage and query graph data as design principles, the graph database belongs to a non-relational database (NoSQL), the graph database is quite different from the relational database in terms of storage, query and data structure of the data, the graph data structure directly stores the dependency relationship among nodes, the relational database and other types of non-relational databases express the relationship among the data in a non-direct mode, the graph database stores the association among the data as a part of the data, labels, directions and attributes can be added on the association, and query of other databases for the relationship is required to be performed in a concrete operation in operation, which is also the reason that the graph database has great performance advantage in terms of the relationship query compared with other types of databases;
time series database: the time sequence database is mainly used for processing data with time labels (which change according to the sequence of time and are time-series), the data with time labels is also called time sequence data, and is a novel non-relational database, the time sequence database is mainly used for processing the data with time labels (which change according to the sequence of time and are time-series), the data with time labels is also called time sequence data, and the time sequence big data is often stored and processed in a relational database way;
unstructured database: unstructured data is irregular or incomplete in data structure, has no predefined data model, is inconvenient to express by using a two-dimensional logic table of a database, is divided into structured data and unstructured data in a computer informatization system, has very various formats and standards, is technically difficult to normalize and understand compared with structured information, and stores, retrieves, distributes and utilizes IT technologies needing more intellectualization, such as mass storage, intelligent retrieval, knowledge mining, content protection and value-added development and utilization of information.
The heterogeneous multi-chain database is a set of a plurality of related database systems, can realize data sharing and transparent access, and a plurality of database systems exist before being added into the heterogeneous database systems, have own database management systems and each component part of an external database and have own autonomy, so that each database system still has own application characteristics, integrity control and security control while realizing data sharing.
The data warehouse comprises a database conversion module and a database transparent access module, can convert the data which is checked to be normal, loads and blends the data into the heterogeneous multi-chain database, and meanwhile, the database transparent access module is convenient for direct access after the next check.
The data warehouse can collect information from a plurality of databases in the heterogeneous database system, establish a unified global mode, support access to historical data, and enable a user to perform decision-supported query through a unified data interface provided by the data warehouse.
In S4, after the data original codes are inspected, qualified heterogeneous data are inspected and transferred into a data warehouse, so that data storage is facilitated, abnormal heterogeneous data are inspected, and the abnormal heterogeneous data enter an early warning notification module.
The early warning notification module comprises a message early warning system, a message notification system and a task distribution processing system, and sends out message early warning to abnormal heterogeneous data, and then notifies the task distribution system of the early warning message, distributes the early warning message to corresponding processing personnel, and performs abnormal processing.
This heterogeneous multi-chain data supervisory systems can carry out data access to multiple heterogeneous multi-chain data, then will insert heterogeneous data and gather, through the back of gathering, and through data supervision recognition element, carry out data check with heterogeneous multi-chain database, after the check, through data supervision processing element, can handle unusual heterogeneous data fast, unusual handled data, carry out the alarm and send out the notice, thereby guarantee and maintain market data chain safety and stability, to the heterogeneous data of receipt, check up qualified with heterogeneous multi-chain database, input data warehouse, the data stock of increase database, the subsequent data of being convenient for is received once more, whether unusual discernment of data can be carried out fast.
Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (7)
1. A heterogeneous multi-chain data supervision system comprising a heterogeneous multi-chain data supervision system, wherein the heterogeneous multi-chain data supervision system comprises the following steps:
s1: heterogeneous multi-link data access: the data with different versions, different types and different structures are accessed through a data port, so that the multi-chain data are collected and summarized;
s2: heterogeneous data acquisition unit: the heterogeneous data transmitted by the data are subjected to data acquisition, and the preposed probes arranged in the acquisition unit can acquire the data and place the acquired data in a big data application system or a cloud service system;
s3: data supervision and identification unit: the collected multi-chain data is subjected to comparison and verification with a heterogeneous multi-chain database, after data verification, the verified data result can be classified, normal heterogeneous data is verified and transmitted to a data warehouse, abnormal heterogeneous data is sent to an abnormal data storage for storage, and the subsequent data processing is waited;
s4: the data supervision processing unit: and (3) monitoring and processing the abnormal heterogeneous data, wherein in the processing process, firstly, the abnormal data is received by an abnormal data receiving module, after the abnormal data is received, the abnormal data is started by a verification starting module, after the verification starting module is started, the data containing the encrypted module is decrypted by a seal conversion module, and finally, the data is inspected by a data original code.
2. The heterogeneous multi-chain data supervision system according to claim 1, wherein in S3, the heterogeneous multi-chain database comprises a relational database, a graph database, a time-series database, and an unstructured database;
relational database: relational databases, which are databases that employ a relational model to organize data, store data in rows and columns for ease of user understanding, a series of rows and columns of the relational database are referred to as tables, a set of tables comprising the database;
graphic database: the graph database is a data management system which takes points and edges as basic storage units and takes efficient storage and query graph data as design principles;
time series database: the time sequence database is mainly used for processing data with time labels (which change according to the sequence of time and are instant in time sequence), and the data with the time labels are also called time sequence data and are novel non-relational databases;
unstructured database: unstructured data is data represented by a two-dimensional logical table of a database, which is irregular or incomplete in data structure, has no predefined data model, and is inconvenient.
3. The heterogeneous multi-chain data supervision system according to claim 1, wherein the heterogeneous multi-chain database is a collection of related database systems, so that sharing and transparent access of data can be realized, and several database systems exist before joining the heterogeneous database systems, and have own database management systems and each component of the external database has own autonomy, so that data sharing is realized, and each database system still has own application characteristics, integrity control and security control.
4. The heterogeneous multi-chain data supervision system according to claim 1, wherein the data warehouse comprises a database conversion module and a database transparent access module, and can convert and load the data with normal verification into the heterogeneous multi-chain database, and meanwhile, the database transparent access module facilitates direct access after the next verification.
5. The heterogeneous multi-chain data administration system according to claim 1, wherein the data warehouse is capable of collecting information from a plurality of databases in the heterogeneous database system and establishing a unified global schema, wherein the collected data also supports access to historical data, and wherein a user performs decision-supported queries via a unified data interface provided by the data warehouse.
6. The heterogeneous multi-chain data supervision system according to claim 1, wherein in S4, after the data source code examines, the qualified heterogeneous data is examined and transferred to the data warehouse, so as to facilitate data storage, and the abnormal heterogeneous data is examined and is sent to the early warning notification module.
7. The heterogeneous multi-link data supervision system according to claim 6, wherein the early warning notification module comprises a message early warning system, a message notification system and a task distribution processing system, and sends out a message early warning to abnormal heterogeneous data, and then the early warning message is notified to the task distribution system and distributed to corresponding processing personnel to perform abnormal processing.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210572383.4A CN117149842A (en) | 2022-05-24 | 2022-05-24 | Heterogeneous multi-chain data supervision system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210572383.4A CN117149842A (en) | 2022-05-24 | 2022-05-24 | Heterogeneous multi-chain data supervision system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117149842A true CN117149842A (en) | 2023-12-01 |
Family
ID=88897367
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210572383.4A Pending CN117149842A (en) | 2022-05-24 | 2022-05-24 | Heterogeneous multi-chain data supervision system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117149842A (en) |
-
2022
- 2022-05-24 CN CN202210572383.4A patent/CN117149842A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106778253A (en) | Threat context aware information security Initiative Defense model based on big data | |
| CN114925045B (en) | PaaS platform for big data integration and management | |
| RO138285A2 (en) | Smart energy management and control platform based on industrial internet | |
| CN114328688A (en) | Management and control platform for electric power energy big data | |
| CN112883001A (en) | Data processing method, device and medium based on marketing and distribution through data visualization platform | |
| CN112446022A (en) | Data authority control method and device, electronic equipment and storage medium | |
| WO2022237506A1 (en) | Method, apparatus, and device for monitoring online diagnosis service, and storage medium | |
| CN118094650B (en) | Agricultural product traceability storage method based on blockchain | |
| CN111708807A (en) | Data flattening processing method, device, equipment and storage medium | |
| CN116961241B (en) | Unified application monitoring platform based on power grid business | |
| CN111858647B (en) | Method for verifying conversion field type between data sources | |
| CN112232843B (en) | Drug supervision system and method based on big data technology | |
| CN112308738A (en) | Water affair data analysis processing system | |
| CN117851389A (en) | Multi-source heterogeneous mass data acquisition and management system based on middleware of Internet of things | |
| CN112784129A (en) | Pump station equipment operation and maintenance data supervision platform | |
| CN117149842A (en) | Heterogeneous multi-chain data supervision system | |
| CN117371945A (en) | One-stop big data management service platform for environmental industry | |
| CN113592210A (en) | Internet of things integrated management platform for water supply non-negative-pressure secondary water supply facility | |
| CN111414355A (en) | Offshore wind farm data monitoring and storing system, method and device | |
| Wang et al. | Block storage optimization and parallel data processing and analysis of product big data based on the hadoop platform | |
| CN104601361B (en) | Power secondary system security event analysis method for non-policy-compliant access | |
| CN112910974B (en) | System and method based on block chain system architecture | |
| CN115168474A (en) | Internet of things center station system building method based on big data model | |
| CN114238027A (en) | Multi-dimensional analysis system based on mass request data | |
| CN113111103A (en) | Intelligent comprehensive big data fusion processing platform |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |