CN102385628B - Java data base connectivity (JDBC)-based data distributed processing method - Google Patents
Java data base connectivity (JDBC)-based data distributed processing method Download PDFInfo
- Publication number
- CN102385628B CN102385628B CN201110359435.1A CN201110359435A CN102385628B CN 102385628 B CN102385628 B CN 102385628B CN 201110359435 A CN201110359435 A CN 201110359435A CN 102385628 B CN102385628 B CN 102385628B
- Authority
- CN
- China
- Prior art keywords
- data
- sql
- jdbc
- database
- virtual
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a Java data base connectivity (JDBC)-based data distributed processing method, which comprises the following steps of: 1) setting a virtual database JDBC driver packet file in a client or a Web application program, establishing a virtual database connection, and sending a structured query language (SQL) calling request to a virtual database according to the JDBC specifications; 2) setting parameters in the virtual database, establishing a connection between a JDBC driver layer and each entity database, and caching and/or storing the data in a plurality of databases of a distributed system; and 3) receiving the data SQL calling request by a data distributed processor of the virtual database, and performing statement analysis and distribution, so that distributed processing of the data is realized. According to the method, the data processing capacity of the original program can be improved only by adding the virtual database driver packet into a class loading path of the original application program and simply configuring the virtual database; and the method is strong in expansibility, the data storage capacity can be improved in a vertical expansion mode, and the concurrent processing capacity can also be improved in a transverse expansion mode.
Description
Technical field
The present invention relates to data processing, particularly a kind of Data distribution8 formula disposal route based on JDBC, belongs to data storage and retrieval field.
Background technology
When the data scale of application system process constantly increases, the storage and the processing requirements that meet these mass datas generally have two kinds of modes, and a kind of is the minicomputer, the mainframe device that adopt performance higher, and another kind is the cluster adopting common machines composition.Because cluster has outstanding cost performance, instead of large scale computer to a great extent at present, the application on Web server and application server is exactly well embody.JDBC (Java Data Base Connectivity, java DataBase combining) be a kind of Java API for performing SQL statement, can provide unified access for multiple relational database, the class that it is write with Java language by a group and interface form.JDBC provides a kind of benchmark, can build more senior instrument and interface accordingly, be connected with Database, the statement of transmit operation database result.JVM is (Java Virtual Machine, Java Virtual Machine) abbreviation, JVM is a kind of specification for computing equipment, and it is a computing machine fabricating out, is realized by the various computer function of analogue simulation on the computing machine of reality.Java Virtual Machine comprises a set of byte code instruction collection, one group of register, stack, a garbage reclamation heap and a storage means territory.JVM shields the information relevant to specific operating system platform, and the object code (bytecode) java applet only need being generated run on a java virtual machine, just can not run in kinds of platform, with not adding amendment.
Store with process in data, although cluster provides high-performance and strong fault tolerance ability, the instrument of this respect with regard to much less, and mainly towards the solution of large enterprises.Business solution, as Oracle Real Application Clusters, has taken to Application share storage system; Apply network in the DB2 Integrated Cluster environment of IBM and share storage; And in the data-base cluster in camp of increasing income, MySQL reproduction technology application principal and subordinate mechanism realizes.The obvious defect of these technology is the characteristic in order to realize cluster, necessary extending database engine, so some extra API of application program, API (Application Programming Interface, application programming interface) be some predefined functions, object is to provide application program and the developer ability to access one group of routine based on certain software or hardware, and without the need to accessing source code, or understand the details of internal work mechanism.And be difficult to cooperation mutually between the different implementations of these different technologies.
Summary of the invention
The object of the invention is to propose a kind of Data distribution8 formula disposal route based on JDBC, it can convert a unified virtual data base to the entity data bak of series of different, the list structure of each entity data bak is identical, client or web application drive by loading virtual data base JDBC, SQL request can be sent according to JDBC specification to virtual data base, realize the operations such as the data retrieval to each entity data bak, insertion, renewal and deletion.
In order to solve the problems of the technologies described above, technical scheme of the present invention comprises:
Based on a Data distribution8 formula disposal route of JDBC, concrete steps are as follows:
1) virtual data base JDBC is set in the class load path of the client developed based on Java specification or web application and drives APMB package position, make described application program can load this driving bag, connect with virtual data base, and send SQL call request according to JDBC specification to virtual data base;
2) virtual data base is a kind of server-side application based on the exploitation of Java specification, configuration management tool is used to carry out optimum configurations, to drive the driving that provides of Ceng Zhong manufacturer to connect with each entity data bak respectively by JDBC, by data buffer storage and/or be stored in multiple databases of distributed system;
3) the Data distribution8 formula processor of described virtual data base receives data SQL call request, and line statement analysis of going forward side by side, then distributes execution, and inquiry and/or the data upgraded in entity data bak, realize the distributed treatment of data.
Data volume as application is large, has exceeded single node processing power, then data scatter has been stored in multi-node cluster; As application data volume little, then data multiple duplication in multiple node cluster.
Described Data distribution8 formula processor comprises: the modules such as SQL calling interface, SQL statement analyzer, SQL statement actuator, scheduling distributor, data buffer storage, connection pool, Data access plug-ins manager, Oracle data access, MySQL data access, user security, daily record maintenance, configuration management.
Described JDBC drives layer to comprise: the JDBC specification that meets that Oracle drives, MYsql drives layer and other manufacturers to provide drives.
Described database comprises oracle database, MySQL database and other manufacturer data storehouses.
Described virtual data base JDBC drives bag: be a driver meeting JDBC specification, can run in JVM, make user can carry out communication with Virtual database management system, the SQL statement of user is sent in virtual data base, and its result will be sent back to user.
The present invention is mainly used in certain circumstances, and the data volume as application is large, has exceeded single node processing power, this method can have been adopted by extends perpendicular mode, data scatter is stored in multi-node cluster, be satisfied the demands by distributed treatment; Data volume as application is little, but the concurrent request of user is many, and this method can be adopted by mode extending transversely, data multiple duplication in multiple node cluster, improves concurrent responding ability by carrying out load balancing to request.
Beneficial effect of the present invention:
1. application is simple, only needs in the class load path of original application program, add virtual data base and drives bag, carry out easy configuration, can improve the data-handling capacity of original program, adapt to the data processing of different pieces of information amount to virtual data base.
2. extendability is strong, improves data storage capacities, also improve concurrent processing ability by mode extending transversely by extends perpendicular mode.
Accompanying drawing explanation
Fig. 1 is that Data distribution8 formula disposal route of the present invention implements schematic diagram;
Fig. 2 is Data distribution8 formula process flow figure of the present invention;
Embodiment
Below in conjunction with accompanying drawing, preferred embodiment of the present invention is described in further details.
As shown in Figure 1, virtual data base JDBC is set in the class load path of the client developed based on Java specification or web application and drives APMB package position, make described application program can load this driving bag, connect with virtual data base, the operation calls requests such as SQL retrieval, insertion, renewal and deletion that send according to JDBC specification are to virtual data base; Virtual data base is a kind of server-side application based on the exploitation of Java specification, configuration management tool is used to carry out optimum configurations, to drive the driving that provides of Ceng Zhong manufacturer to connect with each entity data bak respectively by JDBC, by data buffer storage and/or be stored in multiple databases of distributed system; After virtual data base receives the SQL call request of application program, Sentence analysis is carried out by Data distribution8 formula processor, then execution is distributed, to realize the distributed treatment of data, Data distribution8 formula processor comprises the module such as SQL calling interface, SQL statement analyzer, SQL statement actuator, scheduling distributor, data buffer storage, connection pool, Data access plug-ins manager, Oracle data access, MySQL data access, user security, daily record maintenance, configuration management.
Entity data bak can be oracle database, MySQL database, SQL Server database, also can be other manufacturer data storehouses, as long as the Data access plug-ins that exploitation is corresponding, is then deployed in Data distribution8 formula processor.
As shown in Figure 2, the present invention proposes a kind of Data distribution8 formula disposal route based on JDBC, and for web application, its treatment scheme is as follows:
(1) in web application, according to operation calls requests such as JDBC specification encapsulation SQL retrieval, insertion, renewal and deletions;
(2) by virtual data base JDBC drive bag, web application and virtual data base connect, by this connect by step 1) in packaged SQL call request send to virtual data base.
(3) virtual data base passes through all connections of connection pool module management and web application foundation, and the SQL operation to web application authorizes audit by user security and daily record maintenance module.
(4) for the SQL call request received, by SQL calling interface, the SQL call statement that virtual data base is supported is converted to.
(5) SQL statement analyzer is according to the parameter configuration of virtual data base, judge which table is this SQL call request relate to, which Data access plug-ins is needed to participate in, and the processing mode of result set, and be assembled into the fill order that each Data access plug-ins can identify, then fill order is sent to SQL statement actuator.According to the concrete condition of service application, the parameter configuration of virtual data base is different: for the table that there is mass data, data scatter can be stored in the database of multiple reality by parameter configuration; For the table of small data quantity, can redundant storage be carried out, and then increase reliability and the concurrency performance of system by multiple duplication.SQL statement analyzer can arrange result set processing mode according to different service application situations, for dispersion memory module, adopts many storehouses result set merger processing mode; For multiple duplication pattern, adopt load balancing, select best database perform and return results.
(6) after SQL statement actuator receives fill order, first judge whether to use buffer memory and effectively data cached, then directly result is returned to web application by calling data cache module in this way, otherwise fill order is given scheduling distributor to perform, according to the result set processing mode that fill order is arranged, the result set that scheduling distributor returns is processed, then result is returned to web application, and calling data cache module upgrades buffer memory.
(7) distributor is dispatched to the fill order received, from Data access plug-ins manager, corresponding Data access plug-ins is obtained according to order classification, then calling data access plug-in unit the corresponding interface fill order, and result is returned to SQL statement actuator and process.If relate to multiple entity data bak, each execution that can be scheduled.
(8) access interface of Data access plug-ins package implementation database, the mode generally driven by JDBC and entity data bak are connected, and send to solid data base management system to perform and collect SQL call request and return results.The database of different vendor has respective Data access plug-ins, as Oracle Data access plug-ins, MySQL Data access plug-ins, SQL Server Data access plug-ins etc., all Data access plug-ins all need to be registered in Data access plug-ins manager by virtual data base configuration management tool.
(9) Data distribution8 formula user security that processor provides, daily record maintenance, Configuration Manager belong to basic module, for other modules provide configuration, authorize auditing service, to increase security, the reliability of system.
Claims (5)
1., based on a Data distribution8 formula disposal route of JDBC, its step comprises:
1) virtual data base JDBC is set in client or web application and drives APMB package, set up virtual data base and connect, and send SQL call request according to JDBC specification to virtual data base;
2) virtual data base carries out optimum configurations, drives the driver in layer and each entity data bak to connect by JDBC, by data buffer storage and/or be stored in multiple databases of distributed system; According to the concrete condition of service application, the parameter configuration of virtual data base is different: for the table that there is mass data, data scatter is stored in the database of multiple reality; For the table of small data quantity, carry out redundant storage, and then increase reliability and the concurrency performance of system by multiple duplication;
3) the Data distribution8 formula processor of described virtual data base receives data SQL call request, and line statement analysis of going forward side by side, then distributes execution, realizes the distributed treatment of data; Described Data distribution8 formula processor comprises:
SQL calling interface, converts the SQL call statement that virtual data base is supported to by the SQL call request received;
SQL statement analyzer, carries out Sentence analysis to the SQL call request received, and arranges result set processing mode according to different service application situations: for dispersion memory module, adopt many storehouses result set merger processing mode; For multiple duplication pattern, adopt load balancing, select best database perform and return results;
SQL statement actuator, receives sql command from SQL statement analyzer and performs;
Scheduling distributor, the SQL realizing multinode entity data bak calls distribution and performs, from Data access plug-ins manager, corresponding Data access plug-ins is obtained according to the fill order classification received, then calling data access plug-in unit the corresponding interface fill order, and result is returned to SQL statement actuator and process;
Data buffer, for the Data Update of buffer memory JDBC.
2. disposal route as claimed in claim 1, is characterized in that, described JDBC drives layer to comprise: Oracle driving, MySQL driving, SQL Server driving and other manufacturers meet the driving of JDBC specification.
3. disposal route as claimed in claim 1, it is characterized in that, described entity data bak comprises oracle database, MySQL database, SQL Server database.
4. disposal route as claimed in claim 1, it is characterized in that, described Data distribution8 formula processor also comprises Data access plug-ins manager.
5. disposal route as claimed in claim 1, it is characterized in that, described Data distribution8 formula processor also comprises user security, daily record maintenance and Configuration Manager.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110359435.1A CN102385628B (en) | 2011-11-14 | 2011-11-14 | Java data base connectivity (JDBC)-based data distributed processing method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110359435.1A CN102385628B (en) | 2011-11-14 | 2011-11-14 | Java data base connectivity (JDBC)-based data distributed processing method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102385628A CN102385628A (en) | 2012-03-21 |
| CN102385628B true CN102385628B (en) | 2015-05-13 |
Family
ID=45825044
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201110359435.1A Active CN102385628B (en) | 2011-11-14 | 2011-11-14 | Java data base connectivity (JDBC)-based data distributed processing method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102385628B (en) |
Families Citing this family (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103678354B (en) * | 2012-09-11 | 2017-05-03 | 中国移动通信集团公司 | Local relation type database node scheduling method and device based on cloud computing platform |
| CN103885995B (en) * | 2012-12-21 | 2017-05-03 | 中国移动通信集团河北有限公司 | List-based database monitoring method and list-based database monitoring device |
| CN103336782B (en) * | 2013-05-30 | 2016-09-21 | 莱诺斯科技（北京）股份有限公司 | A kind of relationship type distributed data base system |
| CN104363165A (en) * | 2014-11-14 | 2015-02-18 | 华东电网有限公司 | Information interactive system under internal and external network isolation environment and data integrating method |
| CN106066890B (en) * | 2016-06-16 | 2020-02-18 | 上海天玑科技股份有限公司 | Distributed high-performance database all-in-one machine system |
| CN107402941A (en) * | 2016-07-22 | 2017-11-28 | 延边众生云计算科技有限公司 | Conventional data Fabric Interface and its implementation |
| CN106874109A (en) * | 2016-12-29 | 2017-06-20 | 朗新科技股份有限公司 | A kind of distributed job distribution processing method and system |
| CN107463511B (en) * | 2017-01-23 | 2020-06-26 | 北京思特奇信息技术股份有限公司 | Data internationalization realization method and device based on multi-level cache |
| CN110147508A (en) * | 2017-10-26 | 2019-08-20 | 北京京东尚科信息技术有限公司 | A kind of method and apparatus of system current limliting |
| CN108762112A (en) * | 2018-06-12 | 2018-11-06 | 哈尔滨理工大学 | A kind of industrial robot emulation and real-time control system based on virtual reality |
| CN109828983A (en) * | 2018-12-15 | 2019-05-31 | 平安科技（深圳）有限公司 | PG data base processing method, device, electronic equipment and storage medium |
| CN112688976A (en) * | 2019-10-17 | 2021-04-20 | 广州迈安信息科技有限公司 | Data processing transmission service system adopting JDBC/HTTP standard |
| CN111367983B (en) * | 2020-03-10 | 2023-08-15 | 中国联合网络通信集团有限公司 | Database access method, system, device and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101093502A (en) * | 2007-06-19 | 2007-12-26 | 深圳市迈科龙电子有限公司 | Security structure of database, and method of use |
| CN101833620A (en) * | 2010-04-28 | 2010-09-15 | 国网电力科学研究院 | Custom security JDBC driver-based database protective method |
| CN101840352A (en) * | 2010-04-29 | 2010-09-22 | 中兴通讯股份有限公司 | Method and device for monitoring database connection pool |
| US20110093435A1 (en) * | 2009-10-21 | 2011-04-21 | Delphix Corp. | Virtual Database System |
| CN102065097A (en) * | 2010-12-27 | 2011-05-18 | 北京像素软件科技股份有限公司 | Synchronous operation method of clients and servers |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101187937A (en) * | 2007-10-30 | 2008-05-28 | 北京航空航天大学 | Mode multiplexing isomerous database access and integration method under gridding environment |
| CN101853274A (en) * | 2010-05-10 | 2010-10-06 | 浪潮电子信息产业股份有限公司 | Method for realizing interconnection of heterogeneous databases |
-
2011
- 2011-11-14 CN CN201110359435.1A patent/CN102385628B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101093502A (en) * | 2007-06-19 | 2007-12-26 | 深圳市迈科龙电子有限公司 | Security structure of database, and method of use |
| US20110093435A1 (en) * | 2009-10-21 | 2011-04-21 | Delphix Corp. | Virtual Database System |
| CN101833620A (en) * | 2010-04-28 | 2010-09-15 | 国网电力科学研究院 | Custom security JDBC driver-based database protective method |
| CN101840352A (en) * | 2010-04-29 | 2010-09-22 | 中兴通讯股份有限公司 | Method and device for monitoring database connection pool |
| CN102065097A (en) * | 2010-12-27 | 2011-05-18 | 北京像素软件科技股份有限公司 | Synchronous operation method of clients and servers |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102385628A (en) | 2012-03-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102385628B (en) | Java data base connectivity (JDBC)-based data distributed processing method | |
| CN100565510C (en) | Data access layer class generator | |
| CN103930875B (en) | Software virtual machine for acceleration of transactional data processing | |
| JP6117378B2 (en) | System and method for a distributed database query engine | |
| CN104484472B (en) | A kind of data-base cluster and implementation method of a variety of heterogeneous data sources of mixing | |
| US7886294B2 (en) | Virtual machine monitoring | |
| US8286127B2 (en) | Mirrored file system | |
| CN112534396A (en) | Diary watch in database system | |
| US6996680B2 (en) | Data prefetching method | |
| WO2020119649A1 (en) | Task scheduling simulation system | |
| CN103399942B (en) | A kind of data engine system supporting the many tenants of SaaS and method of work thereof | |
| CN102479225B (en) | Distributed data analyzing and processing method and system | |
| CN105243155A (en) | Big data extracting and exchanging system | |
| CN111488143A (en) | Automatic code generation device and method based on Springboot2 | |
| CN102420814A (en) | Data access method and device, and server | |
| CN102355499B (en) | Cloud computing system | |
| CN104156300A (en) | Log management system and log management method | |
| US20210165803A1 (en) | Replicating materialized views across heterogeneous target systems | |
| CN111078961A (en) | Multi-data source query driving system, method, device and storage medium | |
| WO2021108582A9 (en) | Managed materialized views created from heterogeneous data sources | |
| CN107590208A (en) | A kind of electric network data heterogeneous database data cut-in method of plug-in type | |
| CN106874067A (en) | Parallel calculating method, apparatus and system based on lightweight virtual machine | |
| GB2509978A (en) | Polymorphic columns in database | |
| CN114443680A (en) | Database management system, related apparatus, method and medium | |
| CN115083538B (en) | Medicine data processing system, operation method and data processing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |