CN103078947B - Mass data exchange and transaction architecture system of internet of things - Google Patents
Mass data exchange and transaction architecture system of internet of things Download PDFInfo
- Publication number
- CN103078947B CN103078947B CN201310009955.9A CN201310009955A CN103078947B CN 103078947 B CN103078947 B CN 103078947B CN 201310009955 A CN201310009955 A CN 201310009955A CN 103078947 B CN103078947 B CN 103078947B
- Authority
- CN
- China
- Prior art keywords
- data
- internet
- things
- module
- transaction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Computer And Data Communications (AREA)
Abstract
The invention relates to a mass data exchange and transaction architecture system of the internet of things. The mass data exchange and transaction architecture system comprises a data storage layer, a data processing layer and a data exchange and transaction layer, wherein the data storage layer is used for receiving data transmitted by a data source and carrying out encryption and sorted storage on the data; the data processing layer is used for receiving the data sorted by the data storage layer, carrying out needed preprocessing and standard formatted conversion on the data and extracting the processed and converted data to obtain the format presentation or data set; and the data exchange and transaction layer is used for carrying out management services of issuing, inquiry, exchange and transaction on the format presentation or data set extracted by the data processing layer and storing the data generated in management. The mass data exchange and transaction architecture system has the advantages that the commercial value of the mass data of the internet of the things is improved, the data processing layer can present the corresponding mass data of the internet of things into the needed form, can carry out transaction inquiry and management on the data processed by the data processing layer, and the security and the reliability of the mass data of the internet of things can be ensured in the whole process.
Description
Technical field
The present invention relates to a kind of system configuration, especially a kind of Internet of Things mass data exchanges transaction architecture system, belongs to the technical field of Internet of Things data processing.
Background technology
Nowadays, use the various smart city application of technology of Internet of things, the moment has a large amount of data to produce.In the development process of the Internet of Things industry to be applied as traction, the magnitude of the mass data produced is also from TB(Trillionbyte) be sharply increased to PB(PetaByte,) rank, in these mass datas, contain the obscure information much very with commercial value, also there is a large amount of invalid informations simultaneously.
These apply the obscure information with commercial value produced from certain Internet of Things, after applying a series of data processing technique, will can be used for other certain industry field, and realize the secondary increment of data.The magnanimity traffic flow data produced in such as Internet of Things intelligent transportation application, by the valuable information obtained after a series of process, can be used in the application of Internet of Things Intelligent logistics and realize more diversified service function.But carry out in the process of changing, process and moving in these Internet of Things mass datas, some sensitive datas or invalid data is comprised in these mass datas, can produce the problem on certain information security and service trust, these problems will affect the immortalized of data.
Summary of the invention
The object of the invention is to overcome the deficiencies in the prior art, provide a kind of Internet of Things mass data to exchange transaction architecture system, its compact conformation, improve the validity of exchanges data transaction after treatment, safe and reliable.
According to technical scheme provided by the invention, described Internet of Things mass data exchanges transaction architecture system, comprising:
Data storage layer, for receiving the data of data source, and stores described data encryption classification;
Data analysis layer, receives data storage layer sorted data, and described data is carried out required preliminary treatment and standard format conversion, and the data after described process and conversion is carried out refining and obtain required form and represent or data set;
Exchanges data transaction layer, refines the form obtained and to represent or data set carries out issuing, inquire about and exchanging the management service of transaction, and store the data produced in management by data analysis layer.
The mode that described data storage layer receives data source data comprises IPV6 network, IPV4 network or wireless network.
Described data storage layer comprises data-privacy protection encrypting module; described data-privacy protection encrypting module is connected with Internet of Things mass data cloud storage subsystem; described Internet of Things mass data cloud storage subsystem is classified to the mass data stored, with the data set that is applied.
Described data analysis layer comprises Internet of Things massive data processing and standardized format modular converter, invalid data in data storage layer and logic error data are removed in described Internet of Things massive data processing and the purification of standardized format modular converter, and the data of needs are carried out standardized format conversion, and by the transfer of data after process in data set.
Described data analysis layer comprises Internet of Things massive data processing and standardized format modular converter, the output of described Internet of Things massive data processing and standardized format modular converter is connected with data scrubbing subsystem and data purification subsystem, the output of described data scrubbing subsystem and data purification subsystem all protects deciphering module to be connected with data-privacy, and the output of described data-privacy protection deciphering module describes with Internet of Things mass data and representation module is connected; The cleaning that the data that data scrubbing subsystem transmits Internet of Things massive data processing and standardized format modular converter need; the data of data purification subsystem to Internet of Things massive data processing and the transmission of standardized format modular converter carry out required refinement; the decrypt data that data-privacy protection deciphering module inputs data scrubbing subsystem and data purification subsystem, the form that the data that Internet of Things mass data describes and data-privacy protects deciphering module to transmit by representation module are expressed as form, figure or index exports.
Described data scrubbing subsystem comprises connection column cleaning module, single-column type cleaning module, setting data cleaning module and data scrubbing abnormality processing module.
Described data purification subsystem comprises association analysis module, Cluster Analysis module, evolution analysis module, special cohort analysis module and data purification abnormality processing module.
Described data analysis layer comprises Internet of Things massive data processing and standardized format modular converter, invalid data in data storage layer and logic error data are removed in described Internet of Things massive data processing and the purification of standardized format modular converter, and the data of needs are carried out standardized format conversion; Output and the data-privacy of Internet of Things massive data processing and standardized format modular converter protect deciphering module to be connected, and the output of data-privacy protection deciphering module describes with Internet of Things mass data and representation module is connected.
Described exchanges data transaction layer comprises WEB Self-Service module, security authentication module, capital settlement module, exchanges transaction engine and trading information data storehouse.
Advantage of the present invention: Internet of Things data source produces Internet of Things mass data, data storage layer is by the encryption of Internet of Things mass data and store classification, Internet of Things mass data can be carried out preliminary treatment and format conversion by data analysis layer, and can clear up and refining data, to improve the commercial value obtaining Internet of Things mass data, corresponding Internet of Things mass data can be expressed as the form of needs by data analysis layer, exchanges data transaction layer can carry out Transaction Inquiries and management to the data after data analysis layer process, and related data is stored, the security reliability of Internet of Things mass data can be guaranteed in whole process.
Accompanying drawing explanation
Fig. 1 is structured flowchart of the present invention.
Description of reference numerals: 1-data source, 2-data storage layer, 3-data analysis layer, 4-exchanges data transaction layer, 5-data-privacy protection encrypting module, 6-Internet of Things mass data cloud storage subsystem, 7-application data set, 8-Internet of Things massive data processing and standardized format modular converter, 9-data scrubbing subsystem, 10-joins column cleaning module, 11-single-column type cleaning module, 12-setting data cleaning module, 13-data scrubbing abnormality processing module, 14-data purification subsystem, 15-association analysis module, 16-Cluster Analysis module, 17-evolution analysis module, the special cohort analysis module of 18-, 19-data purification abnormality processing module, 20-data-privacy protection deciphering module, 21-Internet of Things mass data describes and representation module, 22-data set, 23-WEB Self-Service module, 24-security authentication module, 25-capital settlement module, 26-exchanges transaction engine and 27-trading information data storehouse.
Embodiment
Below in conjunction with concrete drawings and Examples, the invention will be further described.
As shown in Figure 1: in order to the safety solving existing Internet of Things mass data is trusted and increment problem, the present invention includes data storage layer 2, for receiving the data that data source 1 is transmitted, and described data encryption classification is stored;
Data analysis layer 3, receives data storage layer 2 sorted data, and described data is carried out required preliminary treatment and standard format conversion, and the data after described process and conversion is carried out refining and obtain required form and represent or data set 22;
Exchanges data transaction layer 4, refines the form obtained and to represent or data set 22 carries out issuing, inquire about and exchanging the management service of transaction, and store the data produced in management by data analysis layer 3.
Wherein, data source 1 produces Internet of Things mass data, and the Data Source of described data source 1 comprises transducer, database, intelligent mobile terminal and the Internet etc.The data message that these Data Sources produce, it can be the real-time stream generated from data terminal, also can be the database file generated after being converged by the application system of user oneself, no matter be real-time stream or database file, can be transferred to by the multiple network such as Ipv6, Ipv4 or wireless network mode in the data storage layer 2 in architecture system.
Described data storage layer 2 comprises data-privacy protection encrypting module 5; described data-privacy protection encrypting module 5 is connected with Internet of Things mass data cloud storage subsystem 6; described Internet of Things mass data cloud storage subsystem 6 is classified to the mass data stored, with the data set 7 that is applied.After data storage layer 2 receives Internet of Things magnanimity information data; first usage data secret protection encrypting module 5 pairs of data carry out full homomorphic cryptography; enable Internet of Things mass data in the whole data storage and processing procedure of system; can data-privacy be ensured, avoid the distrust problem of data security.Enter in Internet of Things mass data cloud storage subsystem 6 by the Internet of Things mass data after encryption, described Internet of Things mass data cloud storage subsystem 6 can realize high speed storing and the read functions of the data file of single more than 100G in Internet of Things mass data and the data file of single below 1M; Described Internet of Things mass data cloud storage subsystem 6 can utilize existing storage system framework to realize.Internet of Things mass data, after entering Internet of Things mass data cloud storage subsystem 2, leaves the data centralization of different Internet of Things application in respectively, and in addition in order to the convenient management of mass data, data set also can be subdivided into subdata collection again.
Described data analysis layer 3 comprises Internet of Things massive data processing and standardized format modular converter 8, the output of described Internet of Things massive data processing and standardized format modular converter 8 is connected with data scrubbing subsystem 9 and data purification subsystem 14, the output of described data scrubbing subsystem 9 and data purification subsystem 14 all protects deciphering module 20 to be connected with data-privacy, and the output of described data-privacy protection deciphering module 20 describes with Internet of Things mass data and representation module 21 is connected; The cleaning that the data that data scrubbing subsystem 9 pairs of Internet of Things massive data processings and standardized format modular converter 8 transmit need; the data that data purification subsystem 14 pairs of Internet of Things massive data processings and standardized format modular converter 8 transmit carry out required refinement; the decrypt data that data-privacy protection deciphering module 20 pairs of data scrubbing subsystems 9 and data purification subsystem 14 input, the form that Internet of Things mass data describes and data-privacy protects data that deciphering module 20 transmits to be expressed as form, figure or index by representation module 21 exports.
Wherein, Internet of Things massive data processing in described data analysis layer 3 and standardized format modular converter 8 can directly be connected with data set 22; or Internet of Things massive data processing and standardized format modular converter 8 directly protect deciphering module 20 to be connected with data-privacy, data-privacy protection deciphering module 20 describes with Internet of Things mass data and representation module 21 is connected.
Described data scrubbing subsystem 9 comprises connection column cleaning module 10, single-column type cleaning module 11, setting data cleaning module 12 and data scrubbing abnormality processing module 13.Described data purification subsystem 14 comprises association analysis module 15, Cluster Analysis module 16, evolution analysis module 17, special cohort analysis module 18 and data purification abnormality processing module 19.
In data storage layer 2, the mass data of each application data sets is after entering data analysis layer 3, the purification of Internet of Things mass data and standardized format modular converter 8 purify a large amount of invalid datas existed, logic error data, and carry out standardized format conversion to the data of the special format not easily carrying out Data Analysis Services.
Internet of Things mass data after data purification and standardized format conversion, can enter data scrubbing subsystem 9 and data purification subsystem 14, to obtain different data processed result.The user comprised in data scrubbing subsystem 9 pairs of mass datas is reluctant that the sensitive information item externally shared effectively is cleared up, wherein single-column type cleaning module 11 is responsible for the cleaning of data centralization individual event sensitive information, connection column cleaning module 10 is responsible for the multinomial cleaning with the sensitive information of internal logic relation of data centralization, the cleaning of setting data cleaning module in charge 12 data centralization destructuring sensitive data, data scrubbing abnormality processing module 13 is responsible for the process extremely occurs in data scrubbing process.Data purification subsystem 14 pairs of Internet of Things mass datas carry out the association analysis of data, cluster analysis, evolution analysis and special cohort analysis, and find out the inherent law between mass data, finally pool the data set with commercial value.Also can be carried out the refinement of data commercial value again by data purification subsystem 14 by the data set of data scrubbing subsystem 9.In the embodiment of the present invention, in data scrubbing subsystem 14, association analysis module 15 adopts existing conventional association analysis method, Cluster Analysis module 16 adopts conventional clustering method, evolution analysis module 17 adopts existing evolution analysis method, special cohort analysis module 18 adopts existing conventional analytical method, data scrubbing abnormality processing module 13 and data purification abnormality processing module 19 process according to presetting abnormality processing mode, guarantee the continuity to Internet of Things mass data processing process and stability.
The encrypted data set of Internet of Things mass data by being formed after the process of data purification and standardized format modular converter, data scrubbing subsystem 9 and data purification subsystem 14; except directly carrying out exchanges data between Internet of Things application and transaction; deciphering module 20 can also be protected to carry out the deciphering of data set by data-privacy; to be described by Internet of Things mass data again and representation module 21 forms figure, form and the technical indicator etc. that are easier to understand, and these also will become a kind of mode that Internet of Things mass data exchanges transaction.
Described exchanges data transaction layer 4 comprises WEB Self-Service module 23, security authentication module 24, capital settlement module 25, exchanges transaction engine 26 and trading information data storehouse 27.Internet of Things mass data obtains the results such as figure, form, index and data set after being processed by data analysis layer 3, can enter exchanges data transaction layer 4 and carry out business-like exchange transaction.The WEB Self-Service module 23 of exchanges data transaction layer 4 mainly provides user's request to issue and exchanges the guide service functions such as Transaction Information inquiry, WEB Self-Service module 23, by utilizing security authentication module 24 to carry out the access of user security and to exchange the functions such as transaction, utilizes capital settlement module 25 to carry out exchanging the capital management of transaction.Exchange transaction engine 26 mainly to provide and exchange transaction and bring together, entrust and conclude the business and the function such as to complete a business transaction.The all related datas in exchanges data process of exchange are preserved in 27, trading information data storehouse.The whole operation principle of exchanges data transaction layer 4 and method of work all can adopt existingly to search, the mode such as safety certification, capital settlement and management.
Data source 1 of the present invention produces Internet of Things mass data, data storage layer 2 is by the encryption of Internet of Things mass data and store classification, Internet of Things mass data can be carried out preliminary treatment and format conversion by data analysis layer 3, and can clear up and refining data, to improve the commercial value obtaining Internet of Things mass data, corresponding Internet of Things mass data can be expressed as the form of needs by data analysis layer 3, exchanges data transaction layer 4 can carry out Transaction Inquiries and management to the data after data analysis layer 3 process, and related data is stored, the security reliability of Internet of Things mass data can be guaranteed in whole process.
Claims (6)
1. Internet of Things mass data exchanges a transaction architecture system, it is characterized in that, comprising:
Data storage layer (2), for receiving the data that data source (1) is transmitted, and stores described data encryption classification;
Data analysis layer (3), receive data storage layer (2) sorted data, and described data are carried out required preliminary treatment and standard format conversion, and the data after described process and conversion are carried out refining obtain required form and represent or data set (22);
Exchanges data transaction layer (4), refines the form obtained and to represent or data set (22) carries out issuing, inquire about and exchanging the management service of transaction, and store the data produced in management by data analysis layer (3);
The mode that described data storage layer (2) receives data source (1) transmission data comprises IPV6 network, IPV4 network or wireless network;
Described data storage layer (2) comprises data-privacy protection encrypting module (5), described data-privacy protection encrypting module (5) is connected with Internet of Things mass data cloud storage subsystem (6), described Internet of Things mass data cloud storage subsystem (6) is classified to the mass data stored, with the data set that is applied (7);
Described data analysis layer (3) comprises Internet of Things massive data processing and standardized format modular converter (8), invalid data in data storage layer (2) and logic error data are removed in described Internet of Things massive data processing and standardized format modular converter (8) purification, and the data of needs are carried out standardized format conversion, and by the transfer of data after process in data set (22).
2. Internet of Things mass data according to claim 1 exchanges transaction architecture system, it is characterized in that: described data analysis layer (3) comprises Internet of Things massive data processing and standardized format modular converter (8), the output of described Internet of Things massive data processing and standardized format modular converter (8) is connected with data scrubbing subsystem (9) and data purification subsystem (14), the output of described data scrubbing subsystem (9) and data purification subsystem (14) all protects deciphering module (20) to be connected with data-privacy, the output of described data-privacy protection deciphering module (20) describes with Internet of Things mass data and representation module (21) is connected, the cleaning that the data that data scrubbing subsystem (9) transmits Internet of Things massive data processing and standardized format modular converter (8) need, data purification subsystem (14) carries out required refinement to the data that Internet of Things massive data processing and standardized format modular converter (8) transmit, the decrypt data that data-privacy protection deciphering module (20) inputs data scrubbing subsystem (9) and data purification subsystem (14), Internet of Things mass data describes and data-privacy protects the data that deciphering module (20) transmits to be expressed as form by representation module (21), the form of figure or index exports.
3. Internet of Things mass data according to claim 2 exchanges transaction architecture system, it is characterized in that: described data scrubbing subsystem (9) comprises connection column cleaning module (10), single-column type cleaning module (11), setting data cleaning module (12) and data scrubbing abnormality processing module (13).
4. Internet of Things mass data according to claim 2 exchanges transaction architecture system, it is characterized in that: described data purification subsystem (14) comprises association analysis module (15), Cluster Analysis module (16), evolution analysis module (17), special cohort analysis module (18) and data purification abnormality processing module (19).
5. Internet of Things mass data according to claim 1 exchanges transaction architecture system, it is characterized in that: described data analysis layer (3) comprises Internet of Things massive data processing and standardized format modular converter (8), invalid data in data storage layer (2) and logic error data are removed in described Internet of Things massive data processing and standardized format modular converter (8) purification, and the data of needs are carried out standardized format conversion; Output and the data-privacy of Internet of Things massive data processing and standardized format modular converter (8) protect deciphering module (20) to be connected, and the output of data-privacy protection deciphering module (20) describes with Internet of Things mass data and representation module (21) is connected.
6. Internet of Things mass data according to claim 1 exchanges transaction architecture system, it is characterized in that: described exchanges data transaction layer (4) comprises WEB Self-Service module (23), security authentication module (24), capital settlement module (25), exchanges transaction engine (26) and trading information data storehouse (27).
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310009955.9A CN103078947B (en) | 2013-01-11 | 2013-01-11 | Mass data exchange and transaction architecture system of internet of things |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310009955.9A CN103078947B (en) | 2013-01-11 | 2013-01-11 | Mass data exchange and transaction architecture system of internet of things |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103078947A CN103078947A (en) | 2013-05-01 |
| CN103078947B true CN103078947B (en) | 2015-04-22 |
Family
ID=48155349
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310009955.9A Active CN103078947B (en) | 2013-01-11 | 2013-01-11 | Mass data exchange and transaction architecture system of internet of things |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103078947B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103475472B (en) * | 2013-07-22 | 2018-07-20 | 浙江万里学院 | The full homomorphic cryptography method of NTRU types on ring LWE |
| CN104506352B (en) * | 2014-12-24 | 2018-04-20 | 福建江夏学院 | A kind of method and system of Internet of Things data pretreatment |
| CN106951913A (en) * | 2017-02-13 | 2017-07-14 | 上海优刻得信息科技有限公司 | The method for carrying out data exchange, cloud platform and system |
| CN112651032A (en) * | 2020-12-18 | 2021-04-13 | 火眸智媒(苏州)科技有限公司 | Internet of things mass data exchange transaction architecture system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1513083A1 (en) * | 2003-09-03 | 2005-03-09 | Sap Ag | Provision of data for data warehousing applications |
| CN101110079A (en) * | 2007-06-27 | 2008-01-23 | 中国科学院遥感应用研究所 | Digital globe antetype system |
| CN102521040A (en) * | 2011-12-08 | 2012-06-27 | 北京亿赞普网络技术有限公司 | Data mining method and system |
-
2013
- 2013-01-11 CN CN201310009955.9A patent/CN103078947B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1513083A1 (en) * | 2003-09-03 | 2005-03-09 | Sap Ag | Provision of data for data warehousing applications |
| CN101110079A (en) * | 2007-06-27 | 2008-01-23 | 中国科学院遥感应用研究所 | Digital globe antetype system |
| CN102521040A (en) * | 2011-12-08 | 2012-06-27 | 北京亿赞普网络技术有限公司 | Data mining method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103078947A (en) | 2013-05-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Liu et al. | A novel debt-credit mechanism for blockchain-based data-trading in internet of vehicles | |
| CN104022549B (en) | A kind of electric automobile intelligent charge access system | |
| CN103078947B (en) | Mass data exchange and transaction architecture system of internet of things | |
| Singh et al. | Blockchain-enabled secure framework for energy-efficient smart parking in sustainable city environment | |
| CN109670321A (en) | Date storage method, data query method and device | |
| CN110855783A (en) | Smart city data system based on big data | |
| JP2016200938A5 (en) | ||
| CN110224808B (en) | Bank data sharing method and device based on block chain, computer equipment and storage medium | |
| CN103973668A (en) | Server-side personal privacy data protecting method in network information system | |
| CN109063138A (en) | For in block chain, that is, service platform search data method, equipment and storage medium | |
| CN103287359A (en) | Energy consumption detecting method for BEV (Blade Electric Vehicle) | |
| Ayvaz et al. | Witness of Things: Blockchain-based distributed decision record-keeping system for autonomous vehicles | |
| CN102752306B (en) | Digital media management method and system based on mark | |
| ElGhanam et al. | Authentication and billing for dynamic wireless EV charging in an internet of electric vehicles | |
| CN110060361A (en) | Vehicle consumption and payment method and its system based on RFID | |
| Gong et al. | Multi-information location data fusion system of railway signal based on cloud computing | |
| CN202870852U (en) | Face identification system based on cloud computing | |
| Feng et al. | A hybrid cryptography scheme for nilm data security | |
| Huang et al. | Secure data sharing over vehicular networks based on multi-sharding blockchain | |
| Meijers et al. | Blockchain for V2X: A taxonomy of design use cases and system requirements | |
| Zhu et al. | Blockchain technology in internet of things | |
| Zhao et al. | Blockchain for effective renewable energy management in the intelligent transportation system | |
| Wang et al. | A privacy-preserving vehicular data sharing framework atop multi-sharding blockchain | |
| Zupanovic | Implementation model for near field communication in Croatian ferry ticketing system | |
| Sasikumar et al. | A secure big data storage framework based on blockchain consensus mechanism with flexible finality |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20171013 Address after: 201600 room 108, building 4, No. 455, research Exhibition Road, Shanghai, Songjiang District, -85 Patentee after: Shanghai state core Internet of things Technology Co., Ltd. Address before: 214135 Jiangsu New District of Wuxi City Linghu Road No. 200 China Sensor Network International Innovation Park C building 4 floor Patentee before: Jiangsu Internet of Things Research & Develoment Co., Ltd. |
|
| TR01 | Transfer of patent right |