CN106844535A - It is a kind of while accessing the database connection method of different cluster Hbase - Google Patents
It is a kind of while accessing the database connection method of different cluster Hbase Download PDFInfo
- Publication number
- CN106844535A CN106844535A CN201611253020.5A CN201611253020A CN106844535A CN 106844535 A CN106844535 A CN 106844535A CN 201611253020 A CN201611253020 A CN 201611253020A CN 106844535 A CN106844535 A CN 106844535A
- Authority
- CN
- China
- Prior art keywords
- hbase
- different cluster
- accessing
- connection method
- cluster hbase
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2471—Distributed queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
Abstract
It is especially a kind of while accessing the database connection method of different cluster Hbase, specific Connection Step is the present invention relates to the technical field of database purchase：S1：Different cluster Hbase profile names are obtained from xml configuration files；S2：Configuration examples are created, calls addResource methods to load different cluster Hbase configuration files；S3：The use of the Configuration examples for creating is initiation parameter, HConnection references to variable are defined by HConnectionManager；S4：HTableInterface references to variable are defined, then table in the different cluster Hbase of correspondence can carry out Hbase table handlings between different clusters；S5：Close methods are called to discharge resource.The database connection method can simultaneously access different cluster Hbase.
Description
Technical field
It is especially a kind of while accessing the data of different cluster Hbase the present invention relates to the technical field of database purchase
Storehouse connection method.
Background technology
Under big data historical background, processing data is all calculated with TB grades, PB grades, and conventional art is to processing so
The data of the order of magnitude are unable to do what one wishes, and what is arisen at the historic moment is exactly the so new-type database of Hadoop clusters and Hbase.Compare
Compared with for, traditional relevant database has the disadvantages that：The two-dimensional table data model that a, relation data place are used is not
Multidimensional data can effectively be processed, it is impossible to effectively process semi-structured and non-structured mass data in the Internet, applications；b、
The performance of high concurrent read-write is low, when relational database reaches certain scale, is very easy to the concurrent problem such as deadlock, causes
Its readwrite performance declines very serious；C, support finite capacity；D, the scalability of database and availability are low, work as application system
Customer volume and when visit capacity growing day by day, traditional Relational DataBase but has no idea simply to lead to as Web Server
Cross and add more hardware and service node comes scalability and load capacity.
Normal conditions access certain cluster Hbase, and the create methods of HbaseConfiguration can be called to create
Configuration examples, create methods can search Hbase-site.xml files under classpath first, by the inside
Information analysis be out encapsulated into Configuration objects, if Hbase-site.xml files do not exist, using acquiescence
Hbase-core.xml files, it is clear that the method obtain be a Hbase configuration file for cluster, it is impossible to it is same when needed
When connect the Hbase of another cluster.Therefore need badly a kind of while accessing the database connection method of different cluster Hbase.
The content of the invention
The technical problem to be solved in the present invention is：In order to overcome the shortcomings of Hbase general connection methods, there is provided one kind is simultaneously
Access the database connection method of different cluster Hbase.
The technical solution adopted for the present invention to solve the technical problems is：It is a kind of while accessing the number of different cluster Hbase
According to storehouse connection method, specific Connection Step is as follows：
S1：Different cluster Hbase profile names are obtained from xml configuration files；
S2：Configuration examples are created, calls addResource methods to load different cluster Hbase configuration files；
S3：The use of the Configuration examples for creating is initiation parameter, is defined by HConnectionManager
HConnection references to variable；
S4：HTableInterface references to variable are defined, then table in the different cluster Hbase of correspondence can carry out different clusters
Between Hbase table handlings；
S5：Close methods are called to discharge resource.
Further specifically, different cluster Hbase configurations are preserved using xml formatted files in S1 in above-mentioned technical proposal
File name.
Further specifically, xml configuration files are read using DOM parser in S1 in above-mentioned technical proposal.
Further specifically, Configuration examples are respectively created in above-mentioned technical proposal in S2, are each called
AddResource methods load different cluster Hbase configuration files.
Further specifically, joined as initialization with the Configuration examples for creating in S3 in above-mentioned technical proposal
Number, defines HConnection references to variable.
Further specifically, HConnection variable-definitions are passed through in above-mentioned technical proposal in S3
HTableInterface references to variable, the different cluster Hbase tables of correspondence, so as to be grasped to different cluster Hbase tables simultaneously
Make.
The beneficial effects of the invention are as follows：The database connection method has advantages below：
First, the profile name of multiple cluster Hbase can be preserved by xml configuration files, and absolute path can be included,
Xml configuration files can very easily be parsed by DOM parser again；
2nd, HTableInterface is built by HConnection and realizes it being unusual lightweight, resource is controllable, so often
It is secondary Hbase connection pools are conducted interviews operation when the resource that is consumed it is all little, and speed is very fast；
3rd, the table in multiple cluster Hbase can simultaneously be had access to by the above method, and is operated on it, for difference
Data interaction is provided a great convenience between cluster.
Brief description of the drawings
The present invention is further described with reference to the accompanying drawings and examples.
Fig. 1 is flow chart of the invention.
Specific embodiment
In conjunction with the accompanying drawings, the present invention is further explained in detail.These accompanying drawings are simplified schematic diagram, only with
Illustration illustrates basic structure of the invention, therefore it only shows the composition relevant with the present invention.
See Fig. 1, it is a kind of while accessing the database connection method of different cluster Hbase, specific Connection Step is as follows：
S1：Different cluster Hbase profile names are obtained from xml configuration files；Preserved not using xml formatted files in S1
With cluster Hbase profile names；Xml configuration files are read using DOM parser in S1, DOM parser is easily used,
And traversal is simple；Cluster and Hbase configuration files are corresponded, and after configuration file is resolved, the corresponding relation is with key-value
Form is stored in Map containers, and cluster name is keyword, and configuration filename is referred to as value；
S2：Configuration examples are created, calls addResource methods to load different cluster Hbase configuration files；S2
In be respectively created Configuration examples, each call addResource methods to load different cluster Hbase configuration files；
One cluster creates a Configuration object, calls addResource methods to load cluster Hbase configuration files；
In addition, addResource methods support absolute path mode specified configuration file, i.e., can be comprising file where it in file name
Directory path；
S3：The use of the Configuration examples for creating is initiation parameter, is defined by HConnectionManager
HConnection references to variable；Configuration examples in S3 with establishment are initiation parameter, define HConnection
Reference to variable；By HConnection variable-definition HTableInterface references to variable in S3, the different cluster Hbase of correspondence
Table, so as to be operated to different cluster Hbase tables simultaneously；HConnectionManager is a class that can not be instantiated,
Dedicated for the connection created and between Hbase connection pools；Due to building HTableInterface realities by HConnection
Now unusual lightweight, so resource is controllable, Hbase connection pools are conducted interviews every time so operation when disappeared
The resource of consumption is all little, and speed is quickly；
S4：HTableInterface references to variable are defined, then table in the different cluster Hbase of correspondence can carry out different clusters
Between Hbase table handlings；Different cluster Hbase tables are carried out by obtaining different cluster HTableInterface variables put,
Get, scan etc. are operated；The reference to variable of multiple cluster Hbase tables is obtained, very easily number can be obtained from different clusters
According to, and simultaneously to multiple cluster logging datas；It is herein service logic part, is emphasis part during actual program, but be not this
Invention focus；
S5：Close methods are called to discharge resource.Due to time-consuming more long when Hbase connection pools are created to be connected, ensure will not
After proceeding table handling, call close methods to discharge resource, it is to avoid to repeat the connection consuming time, influence executing efficiency.
Above-mentioned Connection Step show only a kind of while accessing the possible ways of different cluster Hbase, and having much can be with
Optimization part, for example, when connecting number and being more, be capable of achieving the Hbase connections that connection pool manages different clusters simultaneously.And the present invention
Focus be while the method for accessing different cluster Hbase, to obtain the concrete operations after connection and belong to service logic aspect,
The code write according to different demands will also vary.
With above-mentioned according to desirable embodiment of the invention as enlightenment, by above-mentioned description, relevant staff is complete
Various changes and amendments can be carried out without departing from the scope of the technological thought of the present invention' entirely.The technology of this invention
Property scope is not limited to the content on specification, it is necessary to its technical scope is determined according to right.
Claims (6)
1. a kind of while accessing the database connection method of different cluster Hbase, it is characterised in that：Specific Connection Step is as follows：
S1：Different cluster Hbase profile names are obtained from xml configuration files；
S2：Configuration examples are created, calls addResource methods to load different cluster Hbase configuration files；
S3：The use of the Configuration examples for creating is initiation parameter, is defined by HConnectionManager
HConnection references to variable；
S4：HTableInterface references to variable are defined, then table in the different cluster Hbase of correspondence can carry out different clusters
Between Hbase table handlings；
S5：Close methods are called to discharge resource.
2. according to claim 1 a kind of while accessing the database connection method of different cluster Hbase, its feature exists
In：Different cluster Hbase profile names are preserved using xml formatted files in S1.
3. according to claim 1 a kind of while accessing the database connection method of different cluster Hbase, its feature exists
In：In S1 xml configuration files are read using DOM parser.
4. according to claim 1 a kind of while accessing the database connection method of different cluster Hbase, its feature exists
In：Configuration examples are respectively created in S2, are each called addResource methods to load different cluster Hbase and is configured
File.
5. according to claim 1 a kind of while accessing the database connection method of different cluster Hbase, its feature exists
In：Configuration examples in S3 with establishment are initiation parameter, define HConnection references to variable.
6. according to claim 1 a kind of while accessing the database connection method of different cluster Hbase, its feature exists
In：By HConnection variable-definition HTableInterface references to variable in S3, the different cluster Hbase tables of correspondence, from
And different cluster Hbase tables are operated simultaneously.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611253020.5A CN106844535A (en) | 2016-12-30 | 2016-12-30 | It is a kind of while accessing the database connection method of different cluster Hbase |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611253020.5A CN106844535A (en) | 2016-12-30 | 2016-12-30 | It is a kind of while accessing the database connection method of different cluster Hbase |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106844535A true CN106844535A (en) | 2017-06-13 |
Family
ID=59114711
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611253020.5A Pending CN106844535A (en) | 2016-12-30 | 2016-12-30 | It is a kind of while accessing the database connection method of different cluster Hbase |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106844535A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111737741A (en) * | 2020-06-19 | 2020-10-02 | 中国工商银行股份有限公司 | Distributed database cluster access method and intermediate service layer |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160065627A1 (en) * | 2014-08-29 | 2016-03-03 | Box, Inc. | Configurable metadata-based automation and content classification architecture for cloud-based collaboration platforms |
| CN105938487A (en) * | 2016-04-14 | 2016-09-14 | 北京思特奇信息技术股份有限公司 | Application method and apparatus based on Hbase connection pool |
-
2016
- 2016-12-30 CN CN201611253020.5A patent/CN106844535A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160065627A1 (en) * | 2014-08-29 | 2016-03-03 | Box, Inc. | Configurable metadata-based automation and content classification architecture for cloud-based collaboration platforms |
| CN105938487A (en) * | 2016-04-14 | 2016-09-14 | 北京思特奇信息技术股份有限公司 | Application method and apparatus based on Hbase connection pool |
Non-Patent Citations (2)
| Title |
|---|
| ZHOULIYANG35: "Hbase建立Htable方式分析", 《道客巴巴：HTTP://WWW.DOC88.COM/P-5866236110483.HTML》 * |
| 鞠剑勋: "基于HBase的人口管理系统的设计与实现", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111737741A (en) * | 2020-06-19 | 2020-10-02 | 中国工商银行股份有限公司 | Distributed database cluster access method and intermediate service layer |
| CN111737741B (en) * | 2020-06-19 | 2024-02-27 | 中国工商银行股份有限公司 | Distributed database cluster access method and intermediate service layer |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9619514B2 (en) | Integration of optimization and execution of relational calculation models into SQL layer | |
| CN104346143B (en) | A kind of data transfer device by EBOM to MBOM | |
| US7814045B2 (en) | Semantical partitioning of data | |
| US20150039588A1 (en) | Split processing paths for a database calculation engine | |
| US9767151B2 (en) | Optimizing database queries having hierarchy filters | |
| US9146979B2 (en) | Optimization of business warehouse queries by calculation engines | |
| US11023468B2 (en) | First/last aggregation operator on multiple keyfigures with a single table scan | |
| CN103309958A (en) | OLAP star connection query optimizing method under CPU and GPU mixing framework | |
| US10324930B2 (en) | Database calculation engine with nested multiprovider merging | |
| Wang et al. | Distributed storage and index of vector spatial data based on HBase | |
| US10459760B2 (en) | Optimizing job execution in parallel processing with improved job scheduling using job currency hints | |
| US9613094B2 (en) | Constant mapping optimization in a database | |
| US20150142777A1 (en) | Consistent Aggregation in a Database | |
| US11106666B2 (en) | Integrated execution of relational and non-relational calculation models by a database system | |
| US10275490B2 (en) | Database calculation engine with dynamic top operator | |
| US9715527B2 (en) | Join optimization in a database | |
| US9116953B2 (en) | Calculation engine with dynamic partitioning of intermediate results | |
| US10067980B2 (en) | Database calculation engine integrating hierarchy views | |
| CN106844535A (en) | It is a kind of while accessing the database connection method of different cluster Hbase | |
| US9037570B2 (en) | Optimization of business warehouse filters on complex calculation models | |
| US10255316B2 (en) | Processing of data chunks using a database calculation engine | |
| US10169410B2 (en) | Merge of stacked calculation views with higher level programming language logic | |
| US20140344245A1 (en) | Calculation Engine with Optimized Multi-Part Querying | |
| US11893026B2 (en) | Advanced multiprovider optimization | |
| US10949431B2 (en) | Ranking operations in query processing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170613 |
|
| RJ01 | Rejection of invention patent application after publication |