CN207766288U - A kind of multi-source heterogeneous data Fast Acquisition System - Google Patents
A kind of multi-source heterogeneous data Fast Acquisition System Download PDFInfo
- Publication number
- CN207766288U CN207766288U CN201721668866.5U CN201721668866U CN207766288U CN 207766288 U CN207766288 U CN 207766288U CN 201721668866 U CN201721668866 U CN 201721668866U CN 207766288 U CN207766288 U CN 207766288U
- Authority
- CN
- China
- Prior art keywords
- data
- unit
- server
- cloud server
- processing unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn - After Issue
Links
Landscapes
- Information Transfer Between Computers (AREA)
Abstract
The utility model discloses a kind of multi-source heterogeneous data Fast Acquisition Systems, it includes data acquisition unit, data cleansing unit, data processing unit, communication layers, cloud server and high in the clouds storage unit, the gathered data output end of data acquisition unit is connect with the gathered data input terminal of data cleansing unit, the cleaning data output end of data cleansing unit and the input terminal of data processing unit connect, data processing unit is connected with communication layers communication, communication layers are connected with cloud server communication, the data storage control terminal of cloud server is connect with the control terminal of high in the clouds storage unit, cloud server includes monitoring module, monitor the connection status of module display data processing unit and cloud server.The system carries out data acquisition by data acquisition unit, and the data of acquisition carry out data cleansing by data cleansing unit, and data processing unit progress is normalized, and reduced data is transferred to cloud server, is used so that user accesses.
Description
Technical field
The utility model is related to data collecting field, especially a kind of multi-source heterogeneous data Fast Acquisition System.
Background technology
The rapid advancement of society, the fast development of Internet of Things lead information industry to move towards a new tide now, and
Data acquire core technology as information industry, enable itself and its marginal technology is more intelligent has become Internet of Things system
The Key technique problem of system.What data acquisition faced is the various information from heterogeneous networks different web sites and different industries, and
User is often more likely to when using various data using the data by induction-arrangement, therefore these collected letters
Breath can not be supplied to user directly as data, sorted out or extracted.
In the big data epoch, the timeliness of information is limited, who can be provided to the user in a relatively short period of time becomes apparent from effectively
Information, who can occupy the advantageous market share, and existing data collecting system often can only be directed to single data
It is handled, or processing is sufficiently complete clear, data acquisition and procession is not rapid enough.
Utility model content
The purpose of the utility model is to overcome the deficiencies in the prior art, provide a kind of multi-source heterogeneous data Quick Acquisition system
System, the system carry out data acquisition by data acquisition unit, and the data of acquisition carry out data cleansing by data cleansing unit,
Data processing unit progress is normalized, and reduced data is transferred to cloud server, is used so that user accesses.
The purpose of this utility model is achieved through the following technical solutions：A kind of multi-source heterogeneous data Quick Acquisition system
System, it includes that data acquisition unit, data cleansing unit, data processing unit, communication layers, cloud server and high in the clouds storage are single
Member, the gathered data output end of the data acquisition unit is connect with the gathered data input terminal of data cleansing unit, described
Data cleansing unit cleaning data output end and data processing unit input terminal connect, the data processing unit with
Communication layers communication connection, the communication layers are connected with cloud server communication, the data storage control terminal and cloud of cloud server
It includes monitoring module to hold the control terminal connection of storage unit, the cloud server, monitors module display data processing unit
With the connection status of cloud server.
The system further includes safety certification unit and application server, application server through safety certification unit access cloud
Hold server.
The data processing unit further includes connection status monitoring modular and breakpoint transmission module, the connection status
Connection status between monitoring module monitors data processing unit and cloud server, after breakpoint transmission module is used for transmission processing
Data.
The cloud server further includes correction verification module, and correction verification module verifies the identity of application server.
The high in the clouds storage unit includes Database Administration Server and database server, Database Administration Server
It is connect with the control terminal of database server, the control of data the storage control terminal and Database Administration Server of cloud server
End connection.
The utility model has the beneficial effects that：The utility model provides a kind of multi-source heterogeneous data Fast Acquisition System,
The system carries out data acquisition by data acquisition unit, and the data of acquisition carry out data cleansing, number by data cleansing unit
It is normalized according to processing unit progress, reduced data is transferred to cloud server, is used so that user accesses, user
By application server access high in the clouds storage server, through safety certification unit encrypts certification to access process, has ensured high in the clouds
The safety of the data of storage, while the transmission mode of breakpoint transmission is used between data processing unit and high in the clouds storage server,
The integrality of the data of guarantee.
Description of the drawings
Fig. 1 is the structure diagram of the utility model.
Specific implementation mode
The technical solution of the utility model, but the scope of protection of the utility model are described in further detail below in conjunction with the accompanying drawings
It is not limited to as described below.
As shown in Figure 1, a kind of multi-source heterogeneous data Fast Acquisition System, it includes data acquisition unit, data cleansing list
Member, data processing unit, communication layers, cloud server and high in the clouds storage unit, the gathered data of the data acquisition unit
Output end is connect with the gathered data input terminal of data cleansing unit, the cleaning data output end of the data cleansing unit with
The input terminal of data processing unit connects, and the data processing unit is connected with communication layers communication, the communication layers and high in the clouds
The data storage control terminal of server communication connection, cloud server is connect with the control terminal of high in the clouds storage unit, the cloud
It includes monitoring module to hold server, monitors the connection status of module display data processing unit and cloud server.
The system further includes safety certification unit and application server, application server through safety certification unit access cloud
Hold server.
The data processing unit further includes connection status monitoring modular and breakpoint transmission module, the connection status
Connection status between monitoring module monitors data processing unit and cloud server, after breakpoint transmission module is used for transmission processing
Data.
The cloud server further includes correction verification module, and correction verification module verifies the identity of application server.
The high in the clouds storage unit includes Database Administration Server and database server, Database Administration Server
It is connect with the control terminal of database server, the control of data the storage control terminal and Database Administration Server of cloud server
End connection.
Data acquisition unit acquires the multi-source heterogeneous data Fast Acquisition System of the utility model from each website when in use
The data needed, and be transferred to data cleansing unit and carry out " dirty " data preparation, redundant data is eliminated, after the completion of data cleansing
Data transmission carries out normalized processing to data processing unit, and data that treated are by breakpoint transmission module through communication layers
It is transferred to cloud server, the data received are packaged into data packet and are transferred to Database Administration Server by cloud server,
Database Administration Server, which is controlled database, holds server to carry out data storage.User carries out data visit by application server
It asks, application server through safety certification just can access database server after the certification of unit, pass through the identity of correction verification module
The data stored in ability accessible database server after verification.In data transmission procedure, it is located at the monitoring of cloud server
It module and is monitored between cloud server and data processing unit positioned at the connection status monitoring modular of data processing unit
Connection status.
The multi-source heterogeneous data Fast Acquisition System of the utility model carries out data acquisition, acquisition by data acquisition unit
Data by data cleansing unit carry out data cleansing, data processing unit carry out it is normalized, by reduced data
It is transferred to cloud server, is used so that user accesses, user was accessed by application server access high in the clouds storage server
Through safety certification unit encrypts certification to journey, has ensured the safety of the data of high in the clouds storage, while data processing unit and high in the clouds
The transmission mode of breakpoint transmission, the integrality of the data of guarantee are used between storage server.
Claims (4)
1. a kind of multi-source heterogeneous data Fast Acquisition System, it is characterised in that：It includes data acquisition unit, data cleansing list
Member, data processing unit, communication layers, cloud server and high in the clouds storage unit, the gathered data of the data acquisition unit
Output end is connect with the gathered data input terminal of data cleansing unit, the cleaning data output end of the data cleansing unit with
The input terminal of data processing unit connects, and the data processing unit is connected with communication layers communication, the communication layers and high in the clouds
The data storage control terminal of server communication connection, cloud server is connect with the control terminal of high in the clouds storage unit, the cloud
It includes monitoring module to hold server, monitors the connection status of module display data processing unit and cloud server；
The system further includes safety certification unit and application server, and through safety certification unit access high in the clouds takes application server
Business device.
2. a kind of multi-source heterogeneous data Fast Acquisition System according to claim 1, it is characterised in that：At the data
It further includes connection status monitoring modular and breakpoint transmission module, the connection status monitoring module monitors data processing to manage unit
Connection status between unit and cloud server, data that breakpoint transmission module is used for transmission that treated.
3. a kind of multi-source heterogeneous data Fast Acquisition System according to claim 1, it is characterised in that：The high in the clouds clothes
Business device further includes correction verification module, and correction verification module verifies the identity of application server.
4. a kind of multi-source heterogeneous data Fast Acquisition System according to claim 1, it is characterised in that：It deposits in the high in the clouds
Storage unit includes Database Administration Server and database server, the control of Database Administration Server and database server
End connection, the data storage control terminal of cloud server and the control terminal of Database Administration Server connect.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201721668866.5U CN207766288U (en) | 2017-12-05 | 2017-12-05 | A kind of multi-source heterogeneous data Fast Acquisition System |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201721668866.5U CN207766288U (en) | 2017-12-05 | 2017-12-05 | A kind of multi-source heterogeneous data Fast Acquisition System |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN207766288U true CN207766288U (en) | 2018-08-24 |
Family
ID=63189901
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201721668866.5U Withdrawn - After Issue CN207766288U (en) | 2017-12-05 | 2017-12-05 | A kind of multi-source heterogeneous data Fast Acquisition System |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN207766288U (en) |
-
2017
- 2017-12-05 CN CN201721668866.5U patent/CN207766288U/en not_active Withdrawn - After Issue
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103545924B (en) | Secondary device state on_line monitoring method | |
| CN109547240B (en) | Intelligent device based on edge calculation and access and device analysis method | |
| CN105786592A (en) | Method and device for processing distributed transactions | |
| CN106910146A (en) | A kind of isomery educational data switching plane and method based on Stream Processing technology | |
| CN104484167B (en) | Task processing method and device | |
| CN103384274A (en) | Hyper text transport protocol (http)-based communication method for connection of public network with intranet computer | |
| CN103678522B (en) | A kind of data acquisition of intelligent substation metering system and conversion method | |
| CN103839195A (en) | Power quality management server side and data processing method thereof | |
| CN106790699A (en) | A kind of diesel engine cloud monitoring and cloud management system | |
| CN103905484B (en) | Handle the method and device of hypertext transfer protocol requests | |
| CN107276857A (en) | A kind of method and device for monitoring flow | |
| CN207766288U (en) | A kind of multi-source heterogeneous data Fast Acquisition System | |
| CN106302432B (en) | A kind of communication device and control method based on car networking | |
| CN206236065U (en) | A kind of hand-held GIS data management system | |
| CN107888683A (en) | A kind of client exchange method and device | |
| CN103929488B (en) | The method and device of Distributed Central monitor system data syn-chronization | |
| CN105302664B (en) | A kind of storage snapshot management method and system | |
| CN107770290A (en) | A kind of multi-source heterogeneous data Fast Acquisition System | |
| CN201230328Y (en) | Network system based on three layer architecture | |
| CN104407920A (en) | Data processing method and system based on inter-process communication | |
| CN206135548U (en) | System data on -line monitoring system of distribution automation main website | |
| CN108737568A (en) | A kind of data interaction system and method based on Message Queuing server | |
| CN110244638B (en) | Data monitoring device and method | |
| CN104243253B (en) | The control method and system of a kind of client access device | |
| CN207039637U (en) | A kind of information gathering based on big data, processing system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| AV01 | Patent right actively abandoned |
Granted publication date: 20180824 Effective date of abandoning: 20200305 |
|
| AV01 | Patent right actively abandoned |