CN110502559A - A kind of data/address bus and transmission method of credible and secure cross-domain data exchange - Google Patents
A kind of data/address bus and transmission method of credible and secure cross-domain data exchange Download PDFInfo
- Publication number
- CN110502559A CN110502559A CN201910677669.7A CN201910677669A CN110502559A CN 110502559 A CN110502559 A CN 110502559A CN 201910677669 A CN201910677669 A CN 201910677669A CN 110502559 A CN110502559 A CN 110502559A
- Authority
- CN
- China
- Prior art keywords
- data
- layer
- module
- database
- credible
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/254—Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/258—Data format conversion from or to a database
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/283—Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
Abstract
The invention discloses the data/address bus and transmission method of a kind of credible and secure cross-domain data exchange, belong to Data Interchange Technology field.Include: (one) data collection layer: utilizing flume log acquisition module and ETL data collecting module collected different scenes data；(2) data administer layer: data collection layer acquisition data are standardized data conversion；(3) data analysis layer: according to service feature relationship, standardized data is subjected to the processing such as non-normal form relationship, data redundancy, forms complete business datum collection.(4) data application layer: each operation system is by shared data；Shared opening development platform.The present invention provides a kind of safe Data Integration modes, and the data that can merge different structure form database, and carry out processing to data and share, and can guarantee the safety of database, avoid by network attack.
Description
Technical field
The present invention relates to Data Interchange Technology fields, and in particular to a kind of data/address bus of credible and secure cross-domain data exchange
And transmission method.
Background technique
Traditional enterprise network security center of gravity is always that controllable environment is closed in construction, each to defend in this way
The network attack of kind various kinds.But in recent years along with the fast of internet, Intelligent mobile equipment, technology of Internet of things and cloud computing
Speed development and extensive use, network boundary are broken and extend again and again, and the opening of network is more and more stronger.Simultaneously with basis
Network technology is constantly progressive, and the means and method of network attack are also constantly being reformed.
The network of present enterprise is in the risk status of duration, any existing in face of so severe security risk
Safety equipment be all difficult to cope with.When especially enterprise and internet, Intelligent mobile equipment, Internet of Things carry out data interaction, pass
It unites the interception based on strategy and defense mechanism is easy to be bypassed by advanced directional attack, for the attack of enterprises
Behavior also becomes progressively more and defies capture.
Summary of the invention
In view of the deficiencies of the prior art, the present invention provides the data/address bus and biography of a kind of credible and secure cross-domain data exchange
Transmission method realizes the standardization fusion in each side's metadata source, basis of formation information data resources bank, in basic information resources library
It is upper that extraction cleaning, intelligent protocol, label application, index modeling, offline fortune are carried out to its metadata by big data improvement service
The big datas technological means such as calculation form application service data warehouse and topic model.
A kind of data/address bus of credible and secure cross-domain data exchange, comprising:
(1) flume log acquisition module and ETL data collecting module collected different scenes data data collection layer: are utilized；
(2) data administer layer: data collection layer acquisition data are standardized data conversion；
(3) data analysis layer: according to service feature relationship, carrying out non-normal form relationship, data redundancy processing for standardized data,
Form complete business datum collection；
(4) data application layer: each operation system is by shared data；Shared opening development platform.
Further, the data main source of the data collection layer is respectively Internet of Things, internet, government intranet, industry
Application system of being engaged in acquires preposition service using different applications to complete respectively according to different data source scene；It is based on
SpringBoot micro services framework is disposed in conjunction with Docker container, unified to carry out service container resource coordination using K8S to realize
Entire big data ETL data acquisition and flume log collection process.
Further, unstructured for Internet of Things, semi-structured low value density data carries out critical data feature and mentions
It takes, forms standardized structural data after data parsing semantic analysis；By web crawlers technology to internet key public sentiment data
It extracts, form standardized structural data after lexical analysis；E-gov Network application and private network business application system are mostly that structuring is closed
It is type data, is standardized unified importing to it by technological means such as duplicate removal, replacement, merging.
Further, the data are administered layer and are filtered by customized conversion process model, key feature, realize to height
Value structure data conversion, and multidimensional meta data information is combined, it is realized using key feature data combination business core algorithm,
It is final to establish the data warehouse for being suitble to solve business-subject analysis, to realize that high performance parallel query analysis statistics provides support.
Further, the log of data collection layer acquisition or data are transferred to message-oriented middleware Kafka, message
Middleware Kafka is based on message subscribing mode, and one is subscribed to the realization that theme carries out a process or algorithm, in message
Between part according to the theme of subscription, data-pushing or pull and subscribe to topic module to this and carry out process or calculation processing,
Using as a result, subscribing to topic module by another to subscribe to for topic module processing is subscribed to, processing result will be pushed to
Next subscribing module goes to handle, and successively executes；
After the daily record data is subscribed to by subscription topic module, stream process module, stream process are carried out through intermediary message part Kafka
Mode can be Kafka stream stream process mode or Spark Streaming stream process mode, processing result storage
To big data storage Hbase database, Hive database and dictionary and part intermediate result storing data library MySQL.
Further, prefabricated intelligent tag processes are carried out for different business scenarios, analysis feature, realizes multidimensional
Spend big data fusion specification processing；Engine Spark computing platform is handled by using big data, to interactive inquiry and stream process
It carries out high-performance large data sets and calculates service；With SparkSQL structuring immediate inquiring, Streaming calculates in real time, machine
The core systems such as study, figure calculating, realize high-performance analytic operation, the construction in supporting business theme warehouse.
Further, the data in data analysis layer dictionary and part intermediate result storing data library by JDBC interface with
Data application layer carries out data interaction；The real-time early warning message push that stream process module generates is through intermediary message part Kafka, transmission
To data application layer.
Data analysis layer is equipped with Spark batch processing job big data analysis and excavates module, and the module is distributed using Redis
The data of formula K/V memory database and big data storage Hbase database, Hive database carry out data analysis and excavate, and
Return result to big data storage Hbase database, Hive database.
Further, the data application layer can be issued to data analysis layer and be requested, and request, which is divided into complicated analysis, asks
Simple queries of summing statistics request:
(1) processing of complicated analysis request: complicated analysis request is transferred to Spark through Spring Dataflow Task api
Instant batch processing job module, the instant batch processing job module of Spark store Hbase database, Hive database from big data
And Redis distribution K/V memory database called data carries out processing calculating, calculated result is transferred to big data storage Hbase
Database, Hive database, and data application layer is returned to through intermediary message part Kafka；
(2) big data directly simple queries statistics request processing: is stored by Hbase database, Hive according to request query requirement
Data in database return to data application layer.
Further, the data application layer is using enterprise level service bussing technique to the business such as each police service, categories within police force system
System carries out unified integrated, and combines Kafka subscribing mechanism, and each operation system is needed big data platform treated body data
Collection is standardized Interface integration, and each operation system is by shared big data theme depot data collection.Furthermore it is based on micro services framework
Opening development platform, realize that police, the open type developings such as industry specialists need.Flattening, What You See Is What You Get fortune
Algorithm model, the customized development that skill tactics is used and rehearsal are carried out with big data resource, improves the investigation of each police really
Means are shared with experience.
A kind of transmission method of credible and secure cross-domain data exchange, comprising:
S1: flume log acquisition module and ETL data collecting module collected different scenes data are utilized；
S2: data collection layer acquisition data are standardized data conversion；
S3: according to service feature relationship, standardized data is subjected to the processing such as non-normal form relationship, data redundancy, is formed complete
Business datum collection；
S4: each operation system is by shared data；Shared opening development platform.
The present invention provides a kind of safe Data Integration mode, the data that can merge different structure form database,
And processing is carried out to data and is shared, it can guarantee the safety of database, avoid by network attack.
Detailed description of the invention
Fig. 1 is treatment process schematic diagram of the invention；
Fig. 2 is K8s service container resource coordination figure.
Specific embodiment
Technical solution of the present invention is described further with reference to the accompanying drawings of the specification.
The present invention provides a kind of data/address bus of credible and secure cross-domain data exchange, as shown in Figure 1, taking into account each side's member
The standardization of data source is merged, basis of formation information data resources bank.It is administered on basic information resources library by big data
Service carries out the big datas technology hands such as extraction cleaning, intelligent protocol, label application, index modeling, off-line operation to its metadata
Section forms application service data warehouse and topic model.Platform mainly constructs on Hadoop big data framework, for various
Different application scene is standardized storage to the data for needing random access to handle, reading and writing in real time using HBase database；
The demands such as parallel computation are realized using the technical system based on MapReduce, Spark etc. simultaneously.Otherwise for specific application
Scene and business demand carry out the construction in Data subject warehouse using Hive technological means based on HBase.The data/address bus knot
Structure mainly includes data collection layer, data improvement layer, data analysis layer, data application layer.
(1) data collection layer
Data main source is respectively Internet of Things, internet, government intranet, business application system.According to different data source
Scape acquires preposition service using different applications to complete respectively.Based on SpringBoot micro services framework, hold in conjunction with Docker
Device deployment, it is unified to carry out service container resource coordination using K8S to realize entire big data ETL data acquisition and flume log
Collection process.The K8s service container resource coordination figure is as shown in Figure 2.
For processing multi-source isomerous environment under structuring, semi-structured, unstructured data, for different data come
Source scene is handled using different acquisition methods of service, and is uniformly unified in using distributed micro services Technical Architecture is based on
Multidimensional holographic perceives in the convergence service of preposition adaptation realization under isomerous environment, the system of different application scene, data source mode
One preposition adaptation servicesization processing.It is special that unstructured for Internet of Things, semi-structured low value density data carries out critical data
Sign is extracted, and forms standardized structural data after data parsing semantic analysis；By web crawlers technology to internet key public sentiment
Data are extracted, form standardized structural data after lexical analysis；E-gov Network application and private network business application system are mostly structure
Change relational data, is standardized unified importing to it by technological means such as duplicate removal, replacement, merging.
(2) data administer layer
It is adapted to meta-data extraction by the environment of preposition service, data conversion will be standardized.Pass through customized conversion process
Model, key feature filtering etc., realize and convert to high value structural data, and combine multidimensional meta data information, utilize key
Characteristic combination business core algorithm is realized, final to establish the data warehouse for being suitble to solve business-subject analysis, high to realize
Performance parallel query analysis statistics provides support.
The conversion process model can be understood as the definition of one business process of processing, and model is by multiple operator tuples
At multiple operators are combined into a model, the starting of a model, referred to as the one of the model fortune by process layout
Row example or task.Such as: database connection is adapted to the connections configuration such as operator definitions common jdbc, odbc, and visualization is matched
It sets；The load of data pick-up operator has parsed source database structure, can be with customized target data knot according to source data structure
Structure finally stores the structural information of the operator definitions；Data filtering, conjunction in data cleansing operator definitions extraction process
And the rules such as fractionation；The target database connection of data loading operator definitions, library structure etc.；It will be upper finally by process layout
It states operator and is combined into data acquisition washing moulding, starting, execution etc..
The log of data collection layer acquisition or data are transferred to message-oriented middleware Kafka, and message-oriented middleware Kafka is base
In message subscribing mode, one is subscribed to the realization that theme carries out a process or algorithm, and message-oriented middleware is according to subscription
Theme data-pushing or is pulled to this subscription topic module progress process or calculation processing, is subscribed at topic module
Using as a result, subscribing to topic module by another to subscribe to for reason, will be pushed to processing result next subscribing module
It goes to handle, successively executes.
After the daily record data is subscribed to by subscription topic module, stream process module, stream are carried out through intermediary message part Kafka
Processing mode can be Kafka stream stream process mode or Spark Streaming stream process mode, processing result
Store big data storage Hbase database, Hive database and dictionary and part intermediate result storing data library MySQL.
(3) data analysis layer
According to service feature relationship, standardized data is subjected to the processing such as non-normal form relationship, data redundancy, forms complete business
Data set.Prefabricated intelligent tag processes are carried out for different business scenarios, analysis feature, realize that various dimensions big data is melted
Close specification processing.Engine Spark computing platform is handled by using big data, high-performance is carried out to interactive inquiry and stream process
Large data sets calculate service.SparkSQL structuring immediate inquiring will be used, Streaming is calculated in real time, machine learning, schemed meter
The core systems such as calculation realize high-performance analytic operation, the construction in supporting business theme warehouse.
Data in dictionary and part intermediate result storing data library carry out data by JDBC interface and data application layer
Interaction.The real-time early warning message that stream process module generates is pushed through intermediary message part Kafka, is transferred to data application layer.
Data analysis layer is equipped with Spark batch processing job big data analysis and excavates module, and the module is distributed using Redis
The data of formula K/V memory database and big data storage Hbase database, Hive database carry out data analysis and excavate, and
Return result to big data storage Hbase database, Hive database.
Data application layer can be issued to data analysis layer and be requested, and request is divided into complicated analysis request and simple queries statistics
Request:
(1) processing of complicated analysis request: complicated analysis request is transferred to Spark through Spring Dataflow Task api
Instant batch processing job module, the instant batch processing job module of Spark store Hbase database, Hive database from big data
And Redis distribution K/V memory database called data carries out processing calculating, calculated result is transferred to big data storage Hbase
Database, Hive database, and data application layer is returned to through intermediary message part Kafka；
(2) big data directly simple queries statistics request processing: is stored by Hbase database, Hive according to request query requirement
Data in database return to data application layer.
(4) data application layer
Application layer mainly builds the content of two aspects.Firstly, using enterprise level service bussing technique to industry such as each police service, categories within police force
Business system carries out unified integrated, and combines Kafka subscribing mechanism, and each operation system is needed big data platform treated main body
Data set is standardized Interface integration, and each operation system is by shared big data theme depot data collection.Furthermore it is based on micro services
The opening development platform of framework realizes that the open type developings such as police, industry specialists need.Flattening, What You See Is What You Get
With big data resource carry out algorithm model, skill tactics use customized development and rehearsal, improve each police's really
Investigation is shared with experience.
A kind of transmission method of credible and secure cross-domain data exchange, comprising:
S1: flume log acquisition module and ETL data collecting module collected different scenes data are utilized；
S2: data collection layer acquisition data are standardized data conversion；
S3: according to service feature relationship, standardized data is subjected to the processing such as non-normal form relationship, data redundancy, is formed complete
Business datum collection；
S4: each operation system is by shared data；Shared opening development platform.
Claims (10)
1. a kind of data/address bus of credible and secure cross-domain data exchange, characterized by comprising:
(1) flume log acquisition module and ETL data collecting module collected different scenes data data collection layer: are utilized；
(2) data administer layer: data collection layer acquisition data are standardized data conversion；
(3) according to service feature relationship, standardized data data analysis layer: is subjected to non-normal form relationship, data redundancy etc.
Reason, forms complete business datum collection；
(4) data application layer: each operation system is by shared data；Shared opening development platform.
2. a kind of data/address bus of credible and secure cross-domain data exchange according to claim 1, it is characterised in that: the number
It is respectively Internet of Things, internet, government intranet, business application system according to the data main source of acquisition layer, according to different data
Source scene acquires preposition service using different applications to complete respectively；Based on SpringBoot micro services framework, in conjunction with
The deployment of Docker container, it is unified using K8S carry out service container resource coordination realize entire big data ETL data acquisition with
Flume log collection process.
3. a kind of data/address bus of credible and secure cross-domain data exchange according to claim 2, it is characterised in that be directed to object
Unstructured, semi-structured low value density data of networking carries out critical data feature extraction, and data parse shape after semantic analysis
At standardized structural data；Internet key public sentiment data is extracted by web crawlers technology, forms standard after lexical analysis
Change structured data；E-gov Network application and private network business application system be mostly structured relations type data, by duplicate removal, replacement,
The technological means such as merging are standardized unified importing to it.
4. a kind of data/address bus of credible and secure cross-domain data exchange according to claim 1, it is characterised in that described
Data are administered layer and are filtered by customized conversion process model, key feature, realize and convert to high value structural data, and tie
Multidimensional meta data information is closed, is realized using key feature data combination business core algorithm, final establish is suitble to solution business master
The data warehouse of analysis is inscribed, to realize that high performance parallel query analysis statistics provides support.
5. a kind of data/address bus of credible and secure cross-domain data exchange according to claim 1, it is characterised in that described
The log of data collection layer acquisition or data are transferred to message-oriented middleware Kafka, and message-oriented middleware Kafka is ordered based on message
Read mode, one is subscribed to the realization that theme carries out a process or algorithm, and message-oriented middleware is according to the theme of subscription, number
Subscribe to topic module according to pushing or pulling to this and carry out process or calculation processing, subscribe to topic module processing as a result,
Topic module is subscribed to by another to use to subscribe to, and processing result will be pushed to next subscribing module and be gone to handle, according to
Secondary execution is gone down；
After the daily record data is subscribed to by subscription topic module, stream process module, stream process are carried out through intermediary message part Kafka
Mode can be Kafka stream stream process mode or Spark Streaming stream process mode, processing result storage
To big data storage Hbase database, Hive database and dictionary and part intermediate result storing data library MySQL.
6. a kind of data/address bus of credible and secure cross-domain data exchange according to claim 1, it is characterised in that for not
With business scenario, analysis feature carry out prefabricated intelligent tag processes, realize the fusion specification processing of various dimensions big data；It is logical
It crosses and handles engine Spark computing platform using big data, the calculating of high-performance large data sets is carried out to interactive inquiry and stream process
Service；With SparkSQL structuring immediate inquiring, Streaming is calculated in real time, machine learning, is schemed the core systems such as calculating,
Realize high-performance analytic operation, the construction in supporting business theme warehouse.
7. a kind of data/address bus of credible and secure cross-domain data exchange according to claim 6, it is characterised in that at data
The data managed in layer dictionary and part intermediate result storing data library carry out data interaction by JDBC interface and data application layer；
The real-time early warning message that stream process module generates is pushed through intermediary message part Kafka, is transferred to data application layer；
Data analysis layer is equipped with Spark batch processing job big data analysis and excavates module, and the module utilizes Redis distribution K/
The data of V memory database and big data storage Hbase database, Hive database carry out data analysis and excavate, and will knot
Fruit returns to big data storage Hbase database, Hive database.
8. a kind of data/address bus of credible and secure cross-domain data exchange according to claim 1, it is characterised in that described
Data application layer can be issued to data analysis layer and be requested, and request is divided into complicated analysis request and simple queries statistics request:
(1) processing of complicated analysis request: complicated analysis request is transferred to Spark through Spring Dataflow Task api
Instant batch processing job module, the instant batch processing job module of Spark store Hbase database, Hive database from big data
And Redis distribution K/V memory database called data carries out processing calculating, calculated result is transferred to big data storage Hbase
Database, Hive database, and data application layer is returned to through intermediary message part Kafka；
(2) big data directly simple queries statistics request processing: is stored by Hbase database, Hive according to request query requirement
Data in database return to data application layer.
9. a kind of data/address bus of credible and secure cross-domain data exchange according to claim 1, it is characterised in that described
Data application layer carries out unification to operation systems such as each police service, categories within police force using enterprise level service bussing technique and integrates, and combines
Kafka subscribing mechanism, needs big data platform treated that subject data set is standardized Interface integration for each operation system,
Each operation system is by shared big data theme depot data collection；Furthermore it based on the opening development platform of micro services framework, realizes
The open type developings such as police, industry specialists need；Flattening, What You See Is What You Get utilization big data resource carry out algorithm mould
The customized development and rehearsal, the investigation for improving each police really that type, skill tactics are used are shared with experience.
10. a kind of transmission method of credible and secure cross-domain data exchange, characterized by comprising:
S1: flume log acquisition module and ETL data collecting module collected different scenes data are utilized；
S2: data collection layer acquisition data are standardized data conversion；
S3: according to service feature relationship, standardized data is subjected to the processing such as non-normal form relationship, data redundancy, is formed complete
Business datum collection；
S4: each operation system is by shared data；Shared opening development platform.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910677669.7A CN110502559A (en) | 2019-07-25 | 2019-07-25 | A kind of data/address bus and transmission method of credible and secure cross-domain data exchange |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910677669.7A CN110502559A (en) | 2019-07-25 | 2019-07-25 | A kind of data/address bus and transmission method of credible and secure cross-domain data exchange |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110502559A true CN110502559A (en) | 2019-11-26 |
Family
ID=68587088
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910677669.7A Pending CN110502559A (en) | 2019-07-25 | 2019-07-25 | A kind of data/address bus and transmission method of credible and secure cross-domain data exchange |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110502559A (en) |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111078781A (en) * | 2019-12-30 | 2020-04-28 | 电信科学技术第五研究所有限公司 | Multi-source streaming big data fusion convergence processing framework model implementation method |
| CN111385143A (en) * | 2020-02-21 | 2020-07-07 | 深圳市天彦通信股份有限公司 | Police affairs information cloud platform |
| CN111667243A (en) * | 2020-06-08 | 2020-09-15 | 甘肃建筑职业技术学院 | Business audit system based on ERP system |
| CN111861805A (en) * | 2020-06-15 | 2020-10-30 | 中国司法大数据研究院有限公司 | Cross-domain data cooperation method and system based on data subject |
| CN111914255A (en) * | 2020-07-14 | 2020-11-10 | 北京人人云图信息技术有限公司 | Semi-automatic anti-climbing system based on behavior characteristics |
| CN112073536A (en) * | 2020-09-21 | 2020-12-11 | 福建威盾科技集团有限公司 | Method for realizing safe data transmission and processing between networks incapable of direct inter-access |
| CN112199206A (en) * | 2020-08-28 | 2021-01-08 | 杭州数云信息技术有限公司 | Method and system for configuring and processing real-time stream event based on event mechanism template |
| CN112350921A (en) * | 2020-09-30 | 2021-02-09 | 北京大米科技有限公司 | Message processing method, terminal and storage medium |
| CN112380189A (en) * | 2020-11-17 | 2021-02-19 | 国网福建省电力有限公司信息通信分公司 | Online management system of data model |
| CN112667590A (en) * | 2021-01-05 | 2021-04-16 | 上海七牛信息技术有限公司 | Efficient fusion online analytical processing (OLAP) storage query system and method for Content Distribution Network (CDN) real-time logs |
| CN112749153A (en) * | 2020-12-30 | 2021-05-04 | 工业互联网创新中心（上海）有限公司 | Industrial network data management system |
| CN113220756A (en) * | 2021-03-25 | 2021-08-06 | 上海东普信息科技有限公司 | Logistics data real-time processing method, device, equipment and storage medium |
| CN113572840A (en) * | 2021-07-23 | 2021-10-29 | 浪潮卓数大数据产业发展有限公司 | Data exchange system and method |
| CN113901042A (en) * | 2021-12-10 | 2022-01-07 | 西安中电环通数字科技有限公司 | Ecological environment data dynamic activity level library and terminal |
| CN114006883A (en) * | 2021-10-15 | 2022-02-01 | 南京三眼精灵信息技术有限公司 | Cross-network data penetration interaction method, device, equipment and storage medium |
| CN114490847A (en) * | 2022-01-17 | 2022-05-13 | 武汉魅客科技有限公司 | Smart energy cloud platform data processing method |
| CN116307345A (en) * | 2023-05-09 | 2023-06-23 | 佛山众陶联供应链服务有限公司 | Ceramic industry data system and acquisition method |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106339509A (en) * | 2016-10-26 | 2017-01-18 | 国网山东省电力公司临沂供电公司 | Power grid operation data sharing system based on large data technology |
| US20180084073A1 (en) * | 2015-03-27 | 2018-03-22 | Globallogic, Inc. | Method and system for sensing information, imputing meaning to the information, and determining actions based on that meaning, in a distributed computing environment |
| CN108694221A (en) * | 2017-04-12 | 2018-10-23 | 中国移动通信集团福建有限公司 | Data real-time analysis method, module, equipment and device |
| CN108768741A (en) * | 2018-06-08 | 2018-11-06 | 上海新炬网络技术有限公司 | A kind of network management data interacted system based on big data |
| CN109189589A (en) * | 2018-08-14 | 2019-01-11 | 北京博睿宏远数据科技股份有限公司 | A kind of distribution big data computing engines and framework method |
| CN109344133A (en) * | 2018-08-27 | 2019-02-15 | 成都四方伟业软件股份有限公司 | A kind of data administer driving data and share exchange system and its working method |
| CN109710215A (en) * | 2018-12-25 | 2019-05-03 | 福建南威软件有限公司 | The visible process processing engine and its application method that distributed stream calculates |
| CN109739549A (en) * | 2018-12-28 | 2019-05-10 | 武汉长光科技有限公司 | A kind of equipment performance acquisition method based on micro services |
| CN109840253A (en) * | 2019-01-10 | 2019-06-04 | 北京工业大学 | Enterprise-level big data platform framework |
| CN110022226A (en) * | 2019-01-04 | 2019-07-16 | 国网浙江省电力有限公司 | A kind of data collection system and acquisition method based on object-oriented |
-
2019
- 2019-07-25 CN CN201910677669.7A patent/CN110502559A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180084073A1 (en) * | 2015-03-27 | 2018-03-22 | Globallogic, Inc. | Method and system for sensing information, imputing meaning to the information, and determining actions based on that meaning, in a distributed computing environment |
| CN106339509A (en) * | 2016-10-26 | 2017-01-18 | 国网山东省电力公司临沂供电公司 | Power grid operation data sharing system based on large data technology |
| CN108694221A (en) * | 2017-04-12 | 2018-10-23 | 中国移动通信集团福建有限公司 | Data real-time analysis method, module, equipment and device |
| CN108768741A (en) * | 2018-06-08 | 2018-11-06 | 上海新炬网络技术有限公司 | A kind of network management data interacted system based on big data |
| CN109189589A (en) * | 2018-08-14 | 2019-01-11 | 北京博睿宏远数据科技股份有限公司 | A kind of distribution big data computing engines and framework method |
| CN109344133A (en) * | 2018-08-27 | 2019-02-15 | 成都四方伟业软件股份有限公司 | A kind of data administer driving data and share exchange system and its working method |
| CN109710215A (en) * | 2018-12-25 | 2019-05-03 | 福建南威软件有限公司 | The visible process processing engine and its application method that distributed stream calculates |
| CN109739549A (en) * | 2018-12-28 | 2019-05-10 | 武汉长光科技有限公司 | A kind of equipment performance acquisition method based on micro services |
| CN110022226A (en) * | 2019-01-04 | 2019-07-16 | 国网浙江省电力有限公司 | A kind of data collection system and acquisition method based on object-oriented |
| CN109840253A (en) * | 2019-01-10 | 2019-06-04 | 北京工业大学 | Enterprise-level big data platform framework |
Non-Patent Citations (2)
| Title |
|---|
| 姚颖颖等: "面向融合业务的有线网大数据平台技术体系研究", 《信息技术与标准化》 * |
| 姚颖颖等: "面向融合业务的有线网大数据平台技术体系研究", 《信息技术与标准化》, no. 11, 10 November 2016 (2016-11-10), pages 46 - 49 * |
Cited By (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111078781A (en) * | 2019-12-30 | 2020-04-28 | 电信科学技术第五研究所有限公司 | Multi-source streaming big data fusion convergence processing framework model implementation method |
| CN111385143A (en) * | 2020-02-21 | 2020-07-07 | 深圳市天彦通信股份有限公司 | Police affairs information cloud platform |
| CN111385143B (en) * | 2020-02-21 | 2023-08-22 | 深圳市天彦通信股份有限公司 | Police information cloud platform |
| CN111667243A (en) * | 2020-06-08 | 2020-09-15 | 甘肃建筑职业技术学院 | Business audit system based on ERP system |
| CN111861805A (en) * | 2020-06-15 | 2020-10-30 | 中国司法大数据研究院有限公司 | Cross-domain data cooperation method and system based on data subject |
| CN111914255B (en) * | 2020-07-14 | 2024-03-22 | 北京人人云图信息技术有限公司 | Semi-automatic anti-climbing system based on behavior characteristics |
| CN111914255A (en) * | 2020-07-14 | 2020-11-10 | 北京人人云图信息技术有限公司 | Semi-automatic anti-climbing system based on behavior characteristics |
| CN112199206A (en) * | 2020-08-28 | 2021-01-08 | 杭州数云信息技术有限公司 | Method and system for configuring and processing real-time stream event based on event mechanism template |
| CN112199206B (en) * | 2020-08-28 | 2023-12-26 | 杭州数云信息技术有限公司 | Method and system for configuring and processing real-time stream event based on event mechanism template |
| CN112073536B (en) * | 2020-09-21 | 2023-01-31 | 福建威盾科技集团有限公司 | Method for realizing safe data transmission and processing between networks incapable of direct inter-access |
| CN112073536A (en) * | 2020-09-21 | 2020-12-11 | 福建威盾科技集团有限公司 | Method for realizing safe data transmission and processing between networks incapable of direct inter-access |
| CN112350921A (en) * | 2020-09-30 | 2021-02-09 | 北京大米科技有限公司 | Message processing method, terminal and storage medium |
| CN112380189A (en) * | 2020-11-17 | 2021-02-19 | 国网福建省电力有限公司信息通信分公司 | Online management system of data model |
| CN112749153A (en) * | 2020-12-30 | 2021-05-04 | 工业互联网创新中心（上海）有限公司 | Industrial network data management system |
| CN112667590A (en) * | 2021-01-05 | 2021-04-16 | 上海七牛信息技术有限公司 | Efficient fusion online analytical processing (OLAP) storage query system and method for Content Distribution Network (CDN) real-time logs |
| CN113220756A (en) * | 2021-03-25 | 2021-08-06 | 上海东普信息科技有限公司 | Logistics data real-time processing method, device, equipment and storage medium |
| CN113572840A (en) * | 2021-07-23 | 2021-10-29 | 浪潮卓数大数据产业发展有限公司 | Data exchange system and method |
| CN114006883A (en) * | 2021-10-15 | 2022-02-01 | 南京三眼精灵信息技术有限公司 | Cross-network data penetration interaction method, device, equipment and storage medium |
| CN114006883B (en) * | 2021-10-15 | 2023-06-27 | 南京三眼精灵信息技术有限公司 | Cross-network data penetration interaction method, device, equipment and storage medium |
| CN113901042A (en) * | 2021-12-10 | 2022-01-07 | 西安中电环通数字科技有限公司 | Ecological environment data dynamic activity level library and terminal |
| CN114490847A (en) * | 2022-01-17 | 2022-05-13 | 武汉魅客科技有限公司 | Smart energy cloud platform data processing method |
| CN116307345A (en) * | 2023-05-09 | 2023-06-23 | 佛山众陶联供应链服务有限公司 | Ceramic industry data system and acquisition method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110502559A (en) | A kind of data/address bus and transmission method of credible and secure cross-domain data exchange | |
| CN105554070B (en) | A method of based on police service large data center Service and Construction | |
| CN104820670B (en) | A kind of acquisition of power information big data and storage method | |
| CN104133858B (en) | Intelligence analysis system with double engines and method based on row storage | |
| CN104850601B (en) | Police service based on chart database analyzes application platform and its construction method in real time | |
| CN109272155A (en) | A kind of corporate behavior analysis system based on big data | |
| CN109582717B (en) | Database unified platform for electric power big data and reading method thereof | |
| CN104331435B (en) | A kind of efficient mass data abstracting method of low influence based on Hadoop big data platforms | |
| CN103246749B (en) | The matrix database system and its querying method that Based on Distributed calculates | |
| CN105183834A (en) | Ontology library based transportation big data semantic application service method | |
| CN103440288A (en) | Big data storage method and device | |
| CN106951552A (en) | A kind of user behavior data processing method based on Hadoop | |
| CN106126601A (en) | A kind of social security distributed preprocess method of big data and system | |
| Scannapieco et al. | Placing big data in official statistics: a big challenge | |
| CN107895046A (en) | A kind of Heterogeneous Database Integration Platform | |
| CN107025298A (en) | A kind of big data calculates processing system and method in real time | |
| Li et al. | The overview of big data storage and management | |
| CN112559634A (en) | Big data management system based on computer cloud computing | |
| CN107766451A (en) | A kind of integration across database associative search method towards electric power big data | |
| CN116775763A (en) | Data braiding system for decentralized distributed symbiotic sharing | |
| CN112905571B (en) | Train rail transit sensor data management method and device | |
| Ravichandran | Big Data processing with Hadoop: a review | |
| CN117493340A (en) | Multi-mode data integration fusion analysis system oriented to public safety field | |
| CN102945270A (en) | Parallel distribution type network public opinion data management method and system | |
| CN109542828A (en) | A kind of electric power big data experiment porch |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |