CN110113398A - A kind of large-scale wireless sensing network data-storage system based on HBase - Google Patents
A kind of large-scale wireless sensing network data-storage system based on HBase Download PDFInfo
- Publication number
- CN110113398A CN110113398A CN201910328671.3A CN201910328671A CN110113398A CN 110113398 A CN110113398 A CN 110113398A CN 201910328671 A CN201910328671 A CN 201910328671A CN 110113398 A CN110113398 A CN 110113398A
- Authority
- CN
- China
- Prior art keywords
- data
- gateway
- sensor
- storage
- layers
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W84/00—Network topologies
- H04W84/18—Self-organising networks, e.g. ad-hoc networks or sensor networks
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of large-scale wireless sensing network data-storage system based on HBase, is related to big data technical field.The present invention includes gateway layer, LM layers, GM layers, and gateway layer is the access layer of sensing data, and different sensors type has corresponding gateway to receive and process sensing data, and the data of sensor acquisition are obtained by gateway by application program；LM layers are the storage for managing sensing data, obtain sensor from gateway and acquire data, and real-time storage these data；GM layers, to manage and recording global information, are distributed sensing data storage position.The present invention is stored trans-regional wireless sensor network data and global data storage management catalogue using distributed data base HBase, realizes the storage system of a near real-time by proposing two layers of distributed storage architecture.
Description
Technical field
The invention belongs to big data technical fields, more particularly to a kind of large-scale wireless sensing network based on HBase
Data-storage system.
Background technique
As Internet of Things is in the expansion of China, information technology interface faces ultrahigh- density data storage problem, and wireless sense network is made
For the storage of internet of things endpoint information, distributed areas are more and more wider, and scale is increasing, generate information content and are skyrocketed through；
Wireless sensor is made of one group of sensor and aggregation node, passes through wireless communication, a nothing between them
The data that line sensing network has one or more aggregation nodes specially to collect the acquisition of all the sensors node, the distribution of sensor
Extensively, scale is big, deployment be have it is zonal；
Since large-scale sensor is distributed domain of the existence, if all areas sensor acquisition data are stored in one
LHBase Cluster Database necessarily causes network resource consumption big, time-consuming length, and therefore, the data that each region acquires are stored in
Regional database server advantageously reduces network resource consumption, improves the real-time of data storage and access, for asking above
Topic, which provides a kind of large-scale wireless sensing network data-storage system based on HBase, to have great importance.
Summary of the invention
The purpose of the present invention is to provide a kind of the large-scale wireless sensing network data-storage system based on HBase, nothing
There is the trans-regional property of distribution in line sensing network, with the expansion of wireless sensor network, number of sensors increases, and will generate big rule
The sensing data of mould proposes two layers of distributed storage frame aiming at the problem that storing large-scale wireless sensing network data
Structure stores trans-regional wireless sensor network data and global data storage management catalogue using distributed data base HBase, real
The storage system of an existing near real-time.
In order to solve the above technical problems, the present invention is achieved by the following technical solutions: the present invention is based on for one kind
The large-scale wireless sensing network data-storage system of HBase, including gateway layer, LM layers, GM layers；
The gateway layer is the access layer of sensing data, and different sensors type has corresponding gateway to connect
Sensing data is received and handles, the data of sensor acquisition are obtained by gateway by application program；
Described LM layers be manage sensing data storage, from gateway obtain sensor acquire data, and real-time storage these
Data；
Described GM layers, to manage and recording global information, is distributed sensing data storage position.
Further, the gateway presses region division, and each region has LHBbase to be used to store the real-time of gateway collection
Sensor acquires data, saves global information in GHBase, records all LHBase, LM, gateway and sensor information, add when newly
When adding a gateway, new gateway is added to GM application by local LM first, GM is that gateway and sensor distribute one uniquely
Identifier, keep these information in GHBase for the specified storage table name of each sensor and table structure, GM, maintain an overall situation
Information, LM write data to LHBase after GM acquisition data storage table name and table structure.
Further, in system architecture, there is 3 kinds of correspondences:
(1) gateway is to LM, sensor data transmission, and LHBase is written in data in real time；
(2) LM to GM, when newly adding, updating or delete gateway or sensor node, application for registration, update or deletion registration
Record；
(3) GM to LM, response user query request are maintained at LHBase by sensing data, and GHBase record is all
The data information of data is saved in LHBase.Therefore, GM first inquires the address LHBase where GHBase obtains looked into data, so
Acquisition data are communicated with LM afterwards and return to user.
The invention has the following advantages:
1, the data that each region acquires are stored in regional database server, advantageously reduce network resource consumption,
Improve the real-time of data storage and access.
2, two layers of distributed storage architecture stores trans-regional wireless sense network using distributed data base HBase
Network data and global data storage management catalogue, realize the storage system of a near real-time, have good dilatancy, storage and
Search efficiency.
Certainly, it implements any of the products of the present invention and does not necessarily require achieving all the advantages described above at the same time.
Detailed description of the invention
In order to illustrate the technical solution of the embodiments of the present invention more clearly, will be described below to embodiment required
Attached drawing is briefly described, it should be apparent that, drawings in the following description are only some embodiments of the invention, for ability
For the those of ordinary skill of domain, without creative efforts, it can also be obtained according to these attached drawings other attached
Figure.
Fig. 1 is a kind of large-scale wireless sensing network data memory system architecture schematic diagram based on HBase of the invention；
Specific embodiment
The present invention is a kind of large-scale wireless sensing network data-storage system based on HBase, including gateway layer, LM
(Local Manager) layer, GM (Global Manager) layer；
Gateway layer is the access layer of sensing data, and different sensors type has corresponding gateway to receive simultaneously
Sensing data is handled, the data of sensor acquisition are obtained by gateway by application program；
LM layers are the storage for managing sensing data, obtain sensor from gateway and acquire data, and real-time storage these data；
GM layers, to manage and recording global information, are distributed sensing data storage position.
Wherein, gateway presses region division, and each region has LHBbase (Local HBase) to be used to store gateway collection
Real time sensor acquire data, global information is saved in GHBase (Global HBase), records all LHBase, LM, net
It closes and sensor information adds new gateway to GM application by local LM first, GM is gateway when newly one gateway of addition
A unique identifier is distributed with sensor, keeps these information for the specified storage table name of each sensor and table structure, GM
In GHBase, a global information is maintained, LM writes data to LHBase after GM acquisition data storage table name and table structure.
Wherein, in system architecture, there is 3 kinds of correspondences:
(1) gateway is to LM, sensor data transmission, and LHBase is written in data in real time；
(2) LM to GM, when newly adding, updating or delete gateway or sensor node, application for registration, update or deletion registration
Record；
(3) GM to LM, response user query request are maintained at LHBase by sensing data, and GHBase record is all
The data information of data is saved in LHBase.Therefore, GM first inquires the address LHBase where GHBase obtains looked into data, so
Acquisition data are communicated with LM afterwards and return to user.
In the description of this specification, the description of reference term " one embodiment ", " example ", " specific example " etc. means
Particular features, structures, materials, or characteristics described in conjunction with this embodiment or example are contained at least one implementation of the invention
In example or example.In the present specification, schematic expression of the above terms may not refer to the same embodiment or example.
Moreover, particular features, structures, materials, or characteristics described can be in any one or more of the embodiments or examples to close
Suitable mode combines.
Present invention disclosed above preferred embodiment is only intended to help to illustrate the present invention.There is no detailed for preferred embodiment
All details are described, are not limited the invention to the specific embodiments described.Obviously, according to the content of this specification,
It can make many modifications and variations.These embodiments are chosen and specifically described to this specification, is in order to better explain the present invention
Principle and practical application, so that skilled artisan be enable to better understand and utilize the present invention.The present invention is only
It is limited by claims and its full scope and equivalent.
Claims (3)
1. a kind of large-scale wireless sensing network data-storage system based on HBase, it is characterised in that:
Including gateway layer, LM layers, GM layers；
The gateway layer is the access layer of sensing data, and different sensors type has corresponding gateway to receive simultaneously
Sensing data is handled, the data of sensor acquisition are obtained by gateway by application program；
Described LM layers is the storage for managing sensing data, obtains sensor from gateway and acquires data, and real-time storage these data；
Described GM layers, to manage and recording global information, is distributed sensing data storage position.
2. a kind of large-scale wireless sensing network data-storage system based on HBase according to claim 1, feature
Be: for the gateway by region division, the real time sensor that each region has LHBbase to be used to store gateway collection acquires number
According to, global information is saved in GHBase, records all LHBase, LM, gateway and sensor information, when newly addition one gateway
When, new gateway is added to GM application by local LM first, GM is that gateway and sensor distribute a unique identifier, is
The specified storage table name of each sensor and table structure, GM keep these information in GHBase, maintain a global information, and LM is from GM
After obtaining data storage table name and table structure, LHBase is write data to.
3. a kind of large-scale wireless sensing network data-storage system based on HBase according to claim 1, feature
Be: in the system architecture, there is 3 kinds of correspondences:
(1) gateway is to LM, sensor data transmission, and LHBase is written in data in real time；
(2) LM to GM, when newly adding, updating or delete gateway or sensor node, application for registration, update or deletion registration note
Record；
(3) GM to LM, response user query request are maintained at LHBase by sensing data, and GHBase record is all
The data information of data is saved in LHBase.Therefore, GM first inquires the address LHBase where GHBase obtains looked into data, so
Acquisition data are communicated with LM afterwards and return to user.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910328671.3A CN110113398A (en) | 2019-04-23 | 2019-04-23 | A kind of large-scale wireless sensing network data-storage system based on HBase |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910328671.3A CN110113398A (en) | 2019-04-23 | 2019-04-23 | A kind of large-scale wireless sensing network data-storage system based on HBase |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110113398A true CN110113398A (en) | 2019-08-09 |
Family
ID=67486279
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910328671.3A Pending CN110113398A (en) | 2019-04-23 | 2019-04-23 | A kind of large-scale wireless sensing network data-storage system based on HBase |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110113398A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101645922A (en) * | 2009-04-17 | 2010-02-10 | 中国科学院声学研究所 | CDN network system based on geographical position information encoding and distribution treatment method |
| US20150121371A1 (en) * | 2013-10-25 | 2015-04-30 | Vmware, Inc. | Multi-tenant distributed computing and database |
| CN105930384A (en) * | 2016-04-14 | 2016-09-07 | 南京理工大学 | Sensing cloud data storage system based on Hadoop system and implementation method thereof |
| CN108040092A (en) * | 2017-11-28 | 2018-05-15 | 特斯联（北京）科技有限公司 | A kind of management of Internet of Things big data and application platform towards garden |
-
2019
- 2019-04-23 CN CN201910328671.3A patent/CN110113398A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101645922A (en) * | 2009-04-17 | 2010-02-10 | 中国科学院声学研究所 | CDN network system based on geographical position information encoding and distribution treatment method |
| US20150121371A1 (en) * | 2013-10-25 | 2015-04-30 | Vmware, Inc. | Multi-tenant distributed computing and database |
| CN105930384A (en) * | 2016-04-14 | 2016-09-07 | 南京理工大学 | Sensing cloud data storage system based on Hadoop system and implementation method thereof |
| CN108040092A (en) * | 2017-11-28 | 2018-05-15 | 特斯联（北京）科技有限公司 | A kind of management of Internet of Things big data and application platform towards garden |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104486445B (en) | Distributed extendable resource monitoring system based on cloud platform | |
| US8788760B2 (en) | Adaptive caching of data | |
| CN103067433B (en) | A kind of data migration method of distributed memory system, equipment and system | |
| CN103067461B (en) | A kind of metadata management system of file and metadata management method | |
| US7797275B2 (en) | System and method of time-based cache coherency maintenance in user file manager of object-based storage system | |
| CN101047610B (en) | Data storage, reading, transmission method and management server and network node | |
| US20070299804A1 (en) | Method and system for federated resource discovery service in distributed systems | |
| CN103310000B (en) | Metadata management method | |
| CN103166991B (en) | Cross nodal point storage implementation method and device based on P2P and cloud storage | |
| US12032550B2 (en) | Multi-tenant partitioning in a time-series database | |
| WO2010072083A1 (en) | Web application based database system and data management method therof | |
| CN101170452A (en) | Content distribution network service provision node system for enhancing management capability and its affiliated network | |
| CN108920552A (en) | A kind of distributed index method towards multi-source high amount of traffic | |
| US20090210426A1 (en) | System and method for efficient management of distributed spatial data | |
| CN106775446A (en) | Based on the distributed file system small documents access method that solid state hard disc accelerates | |
| CN105635196A (en) | Method and system of file data obtaining, and application server | |
| CN109684282A (en) | A kind of method and device constructing metadata cache | |
| CN102420814A (en) | Data access method and device, and server | |
| CN111966482B (en) | Edge computing system | |
| CN101673272B (en) | Method, system and device for searching information and method for registering vertical search engine | |
| CN105930345A (en) | Hierarchical indexing method based on distributed real-time database system (DRTDBS) | |
| CN102891768A (en) | Method and network element for network management | |
| CN110276713A (en) | A kind of high-efficiency caching method and system towards remote sensing image data | |
| CN101251843A (en) | Full distributed duplicate copy positioning method for data grid | |
| US11599516B1 (en) | Scalable metadata index for a time-series database |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190809 |
|
| RJ01 | Rejection of invention patent application after publication |