CN101980207A - Method and system for implementing database access - Google Patents
Method and system for implementing database access Download PDFInfo
- Publication number
- CN101980207A CN101980207A CN201010536532.9A CN201010536532A CN101980207A CN 101980207 A CN101980207 A CN 101980207A CN 201010536532 A CN201010536532 A CN 201010536532A CN 101980207 A CN101980207 A CN 101980207A
- Authority
- CN
- China
- Prior art keywords
- bucket
- statement
- database
- data
- database access
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 114
- 230000008569 process Effects 0.000 claims description 26
- 230000014759 maintenance of location Effects 0.000 claims description 7
- 230000001360 synchronised effect Effects 0.000 claims description 6
- 230000005059 dormancy Effects 0.000 claims description 5
- 230000007246 mechanism Effects 0.000 claims description 4
- 230000004044 response Effects 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 13
- 230000006870 function Effects 0.000 description 8
- 238000013507 mapping Methods 0.000 description 6
- 238000004590 computer program Methods 0.000 description 5
- 238000007726 management method Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 3
- 230000005856 abnormality Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 241001269238 Data Species 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000005538 encapsulation Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 230000011514 reflex Effects 0.000 description 1
- 230000033772 system development Effects 0.000 description 1
Images
Abstract
The invention discloses a method and a system for implementing database access. The method comprises the following steps of: forming a Bucket instance by using the same class of structured query language (SQL); configuring various SQL and Bucket in a data access configuration file; if non cross-statement database access is performed, acquiring Pooled Statement by using a borrow Statement () interface method; if cross-statement database access is performed, acquiring the Bucket by using a borrow Bucket () interface method, and acquiring the Pooled Statement therein one by one by using a get Statement () method provided by the Bucket; and performing data operation of the database access by using an interface method provided by the Pooled Statement. The method and the system can implement high-availability, high-performance, low-cost, and easily managed, adjusted and optimized database access for a distributed system of massive data.
Description
Technical field
The present invention relates to the information application technology, be meant a kind of implementation method and system of database access especially.
Background technology
Data access technology is an emphasis problem will considering in the distributed system of mass data.
Present data storage generally adopts relevant database, in realizing with Java, relevant database is called O-R Mapping to the conversion between the Java object, has had at present the scheme of a lot of O-R Mapping, wherein increase income as Hibernate (Object Relation Mapping framework).
The distributed system of the mass data of, high performance requirements available for height, generally there is following deficiency in these existing realizations:
1, be difficult at the characteristics of using the database of different reading and writing features and different priority level be connected and be optimized, exclusively enjoy database connection resource such as a class database access, the shared database of several statements of writing connects; Some database connects always and keeps, and other databases connect release as early as possible etc.
2, SQL (Structured Query Language, Structured Query Language (SQL)) statement is not by centralized configuration, be difficult to carry out the management of other tuning of SQL statement level, fault analysis and concentrated database access statement, the managerial personnel of system development and data base administrator are difficult to manage and tuning with the SQL of a kind of concentrated mode to accessing database.
3, adopt the reflex mechanism (reflect) of Java to realize O-R Mapping, have bigger performance consumption, can not satisfy requirement the extra high system of performance requirement.
4, in the distributed system of the available mass data of height, generally need a lot of database server of quantity, sometimes even thousands of, and data have a plurality of copies.In this case, the mass storage function of dependency database system needs database access layer that corresponding support is provided fully, and the support underaction of general existing realization this respect is as the strategy that does not provide multiple distribution to write.
5, stride the realization of the transaction management dependence application server of statement, performance consumption is bigger.
Summary of the invention
In view of this, the objective of the invention is to propose a kind of implementation method and system of database access, realize for the distributed system of mass data provides the data access layer of high available, high-performance, with low cost, manageability and tuning.
Implementation method based on above-mentioned purpose a kind of database access provided by the invention comprises:
SQL is classified;
Each Bucket of configuration in the data access configuration file comprises among the Bucket that the database of of a sort SQL and this Bucket connects;
If carry out non-database access of striding statement, then obtain Pooled Statement by borrowStatement () interface method; If stride the database access of statement, then obtain Bucket by borrowBucket () interface method, the getStatement () method that provides by Bucket obtains Pooled Statement wherein one by one;
The interface method that uses Pooled Statement to provide carries out the data manipulation of this database access.
Optionally, if carry out non-database access of striding statement in this method, then, obtain Pooled Statement according to statement id by borrowStatement () interface method; If stride the database access of statement, then, obtain Bucket according to statement Bucket id by borrowBucket () interface method.
Optionally, manage the Bucket example by a two-dimensional array in this method, first dimension is the Bucket type under the Bucket example, and second dimension connects for the database that this example takies;
When caller passed through borrowStatement () or borrowBucket () method application statement or Bucket, the parameter of importing into was " Bucket id " or " statement id ", and the pool manager operating process is as follows:
If parameter is " statement id ", then be converted into Bucket id by an index.
In the Bucket pond, find all Bucket examples that belong to this Bucket type, i.e. an one-dimension array according to Bucket id;
Scan this array,, be labeled as " using " if find the Bucket example of " free time ",, then this example is returned if that the caller application is Bucket; If application is statement, then in the statement that this Bucket example comprises, find this statement to return according to " statement id "; If can not find the Bucket example of " free time ", then process dormancy a period of time is waited for,, also can not find the example of " free time " if after the stand-by period, the exception response that the expression of then dishing out " is done ".
Optionally, give back Pooled Statement or Bucket by returnMe () method in this method.
Optionally, this method further comprises: all Bucket centralized configuration in a data access-profile that a database access is required.
Optionally, the described data access configuration file of this method adopts the Properties file of Java, and the collocation form of statement is " statement id=SQL ", and the collocation form of Bucket is as follows:
The statement that comprises: class[bucket id] a plurality of statement id that the .stmts=comma separates, defined the SQL statement that comprises among the Bucket;
The database linking number that can use at most: class[bucket id] the database linking number that can use at most of .conn=;
Whether database connects can write: class[bucket id] .update=true/false, be defaulted as false, promptly read-only.
Optionally, if the described database access of this method is a write operation, then adopt one of following three kinds of modes to carry out:
Only write a database: data are only write a database and are promptly finished;
A plurality of data sync are write: data are wanted a plurality of databases of writing of success, and affairs just are counted as merit;
A plurality of database asynchronous writes: attempt earlier data are write in a plurality of databases, if success, then affairs complete successfully; If one of them database write is then selected in failure, and writes down these data and need be synchronized in other databases, carry out afterwards data sync by the synchronization mechanism on backstage.
Optionally, when appearance in the database access process was unusual, caller was not given back Pooled Statement or Bucket, will carry out mark to the database of makeing mistakes in this method, and refusal carries out write operation to it again.
Optionally, this method still was not used if exceed this retention time free time statement in the idle retention time of described data access configuration file middle finger attribute sentence, then reclaimed the resource of this statement; If all statements in Bucket example all are recovered, then this Bucket example connects together with the data of its use and all can be recovered.
In another aspect of this invention, also provide a kind of realization system of database access, having comprised:
Interface module is used to provide the interface of database access, for non-database access of striding statement, provides borrowStatement () interface method to obtain Pooled Statement; For the database access of striding statement, provide borrowBucket () interface method to obtain Bucket;
The Bucket module is formed a Bucket example with of a sort SQL; Provide getStatement () method to be used to obtain the PooledStatement of Pooled Statement module;
Pooled Statement module, the interface method that provides carries out the data manipulation of database access;
Connect and statement pond administration module, be used for management and connect and the SQL statement pond;
The distributed data storehouse memorizer is used for the data of stored data base.
Optionally, the described interface module of this system, if carry out non-database access of striding statement, then by borrowStatement () interface method, id obtains PooledStatement according to statement; If stride the database access of statement, then, obtain Bucket according to statement Bucket id by borrowBucket () interface method.
Optionally, the described Bucket module of this system is managed the Bucket example by a two-dimensional array, and first dimension is the Bucket type under the Bucket example, and second dimension connects for the database that this example takies;
When caller passed through borrowStatement () or borrowBucket () method application statement or Bucket, the parameter of importing into was " Bucket id " or " statement id ", and the pool manager operating process is as follows:
If parameter is " statement id ", then be converted into Bucket id by an index.
In the Bucket pond, find all Bucket examples that belong to this Bucket type, i.e. an one-dimension array according to Bucket id;
Scan this array,, be labeled as " using " if find the Bucket example of " free time ",, then this example is returned if that the caller application is Bucket; If application is statement, then in the statement that this Bucket example comprises, find this statement to return according to " statement id "; If can not find the Bucket example of " free time ", then process dormancy a period of time is waited for,, also can not find the example of " free time " if after the stand-by period, the exception response that the expression of then dishing out " is done ".
Optionally, the described interface module of this system is given back PooledStatement or Bucket by returnMe () method.
From above as can be seen, there are following advantage in the implementation method of database access provided by the invention and system:
SQL and database are connected centralized configuration in the data access-profile, and configuration file adopts the properties file of Java, and the collocation form of SQL is as " statement id=SQL ".
SQL is classified, of a sort SQL is formed Bucket, and Bucket can be by instantiation, and a Bucket example comprises the SQL statement example that belongs to this Bucket, statement example shared data bank in Bucket example connects, and can finish the affairs of striding statement.Different Bucket uses different databases to connect, and is the database linking number of the maximum of each Bucket assignment of allocation, and the attribute that connects, as reads or writes, keeps time of connecting etc.Bucket is centralized configuration in the data access configuration file also, and form is as " a plurality of statement id that comma separates ".
Do not need to use Java reflect to finish O-R Mapping, to obtain more performance.
The programming mode aspect: all data accesses must be in the data access configuration file centralized configuration, do not allow SQL to occur in program outside the data access configuration file and the configuration file.
Description of drawings
Fig. 1 is the schematic flow sheet of embodiment of the invention database access implementation method;
Fig. 2 is the schematic flow sheet of the database access of another embodiment of the present invention;
Fig. 3 is the synoptic diagram of a data access-profile of embodiment of the invention generation;
Fig. 4 is the structural representation of the realization system of embodiment of the invention database access;
Fig. 5 is the non-schematic flow sheet of striding the statement access process of the embodiment of the invention;
Fig. 6 strides the schematic flow sheet of statement access process for the embodiment of the invention.
Embodiment
Block diagram and flow chart description the present invention below with reference to method according to an embodiment of the invention, device (being system) and computer program.Be appreciated that each piece of block diagram and process flow diagram and the combination of the piece in block diagram and the process flow diagram can be realized by computer program instructions respectively.These computer program instructions can be loaded on multi-purpose computer, special purpose computer or other programmable data treating apparatus, to produce product, so that the instruction of carrying out on computing machine or other programmable data treating apparatus produces the device of the block diagram that is used for executive system or flow chart block appointment.
These computer program instructions also can be stored in the computer-readable memory, this computer-readable memory can instruct computing machine or other programmable data treating apparatus by ad hoc fashion work, so that the instruction that is stored in the computer-readable memory produces a product, wherein comprise the command device of realizing the function of appointment in one or more flow chart blocks.Computer program instructions also can be loaded on computing machine or other programmable data treating apparatus, to cause the sequence of operations step on computing machine or other programmable devices, to be performed, to produce computer implemented process, so that the instruction of carrying out on computing machine or other programmable devices is provided for realizing the step of the function of appointment in one or more flow chart blocks.
Therefore, the piece support of block diagram and process flow diagram is used to carry out the combination of the device of appointed function, the combination of step that is used to carry out appointed function and the program instruction means that is used to carry out appointed function.It is also understood that each piece of block diagram and process flow diagram and the combination of the piece in block diagram and the process flow diagram can be realized by the combination of the hardware based dedicated computer system of carrying out appointed function or step or specialized hardware and computer instruction.
Referring to shown in Figure 1, the implementation method of database access provided by the invention mainly comprises:
Step 101 is classified to SQL;
Step 102 disposes Bucket in the data access configuration file, comprise among the Bucket after the configuration that of a sort SQL is connected with database, and of a sort SQL is subdivided into a Bucket example;
Step 103 when carrying out database access, if carry out non-database access of striding statement, is then obtained Pooled Statement by borrowStatement () interface method; If stride the database access of statement, then obtain Bucket by borrowBucket () interface method, the getStatement () method that provides by Bucket obtains PooledStatement wherein one by one;
Step 104, the interface method that uses Pooled Statement to provide carries out the data manipulation of this database access.
As an embodiment, with the needed a plurality of Bucket of a database access (comprising SQL is connected with database) centralized configuration in a data access-profile, configuration file adopts the properties file of Java, and the collocation form of SQL is as " statement id=SQL ".The programming mode aspect: all data accesses must be in the data access configuration file centralized configuration, do not allow SQL to occur in program outside the data access configuration file and the configuration file.Because SQL and database are connected centralized configuration in the data access configuration file, make the exploitation of software and maintainer's (comprising the data base administrator) come into plain view to the operation to data in the system, conveniently manage and tuning.In addition, the embodiment of the invention is put into a configuration file the inside owing to SQL is connected with database, the user can carry out system configuration by revising configuration file like this, thereby for system configuration provides an approach very easily, can be by revising SQL in this configuration file, make the different data of system's visit, obtain different entity objects.This configuration can also be carried out very complicated expansion by the storing process of database.
As an embodiment, with the SQL classification, of a sort SQL is formed Bucket, Bucket can be by instantiation, a Bucket example comprises the SQL statement example that belongs to this Bucket, and the statement example shared data bank in Bucket example connects, and can finish the affairs of striding statement.Different Bucket example uses different databases to connect, and is the database linking number of the maximum of each Bucket example assignment of allocation, and the attribute that connects, as reads or writes, keeps time of connecting etc.Bucket is centralized configuration in the data access configuration file also, and form is as " a plurality of statement id that comma separates ".
Wherein, the classification to SQL generally has Several principles:
● reading statement and write statement are assigned to different bucket.
● according to the application scenarios of statement, the statement that will will carry out in same things is put into a bucket.
● according to the database resource that these statement needs consume, the statement that the consumption data base resource is many is put into independently bucket the inside, and what the consumption data base resource was few combines.
Do not use Java reflect to finish O-R Mapping in the present invention, to obtain more performance.
The upper strata caller comes the notebook data access layer is called by interface layer, caller can obtain the entity object that Pooled Statement and Bucket two classes are carried out database access, wherein Bucket is the entity that comprises a plurality of Pooled Statement, can stride the transaction operation of statement.The hard core control statement pond of data access layer and data connection pool, finish with the lower floor distributed environment under being connected of a plurality of databases.
Referring to shown in Figure 2, comprise the steps: as a concrete data base call process of embodiment
Step 201 configures Bucket in the data access configuration file, comprise among each Bucket that the database of a class SQL and this Bucket is connected.
Step 205, caller is given back Pooled Statement or Bucket by returnMe () method.
Wherein, the Pooled Statement that uses in the step 204 is the encapsulation to preparedStatment among the Java JDBC, therefore, can support preparedStatment or callableStatment, the storing process of the corresponding database of the latter.Different with the preparedStatment method of JDBC is, for the data access that needs the return data collection, this programme has defined the IResultSetHandler interface, the function that this interface is realized is the delegation in the database data that will use SQL to return, be assembled into an object (Object), these objects be put in the tabulation (List) return to caller.Wherein, described SQL configures in the data access configuration file that generates of described step 201.
As an embodiment, the data access configuration file adopts Java Properties file layout, and preferred version comprises following main points:
The collocation form of SQL statement is " a statement id=SQL statement ", and statement id is used for caller visit statement, is the integer since 0, as " 0=select A; B from TableAwhere id=? ", can specify some parameters for each statement, the retention time during as the free time etc.Can come the retention time of specify default with default.keep.alive, be set to 30 seconds as default.keep.alive=30, keep.alive0=0 represents that No. 0 statement keeps forever, statement and connection pool manager do not discharge the resource of this statement and the connection of used database thereof, can be used for real-time and require extra high occasion.
Bucket also conducts interviews by id, with the statement that comprises among Bucket of " a plurality of statement id that comma separates " definition, can be expressed as class[bucket id] a plurality of statement id of separating of .stmts=comma, as " class0.stmts=0; 1,5 ", defined among No. 0 Bucket and comprised 0,1, No. 5 three SQL statement.Different Bucket uses different databases to connect, in an application, different data accesses has different important and priority levels, can for each Bucket specify at most can with the database linking number ensure, can be expressed as class[bucketid] the database linking number that can use at most of .conn=, as class0.conn=3, specify this Bucket can be instantiated as 3 Bucket examples, each takies a database and connects.
Whether database connects can write: can be expressed as class[bucketid] .update=true/false, as class0.4update=true.The statement among the Bucket or all be the read-only operation statement wherein, or be the statement that comprises write operation all, specify with the 4update attribute of Bucket, be defaulted as read-only operation false.So just reading and writing on database and be connected and separate, so that to connecting the optimization of using.
Referring to shown in Figure 3, be the example of a data access-profile generating according to above-mentioned preferred version.
The present invention towards be distributed application, in these application, data access is usually to read.In the available distributed system of height, critical data generally has a plurality of copies, exists in the different databases.For read operation, only need optional one of them database to carry out read operation.For writing, the present invention provides flexible three kinds tactful selective for application:
Only write a database: data are only write a database and are promptly finished, and are suitable for not too important situations such as data.
A plurality of data sync are write (abbreviation synchronous write): data are wanted a plurality of databases of writing of success, and affairs just are counted as merit.This mode has reduced the availability (because wanting the success simultaneously of a plurality of database writes) of these affairs itself, but has improved the reliability of data simultaneously.
A plurality of database asynchronous writes (abbreviation asynchronous write): attempt earlier data are write in a plurality of databases, if success, then affairs complete successfully; If one of them database write is then selected in failure, and writes down these data and need be synchronized in other databases, carry out afterwards data sync by the synchronization mechanism on backstage.
By three kinds of WriteModes that provide, make that not relying on Database Systems can realize high available data storage, has greatly reduced cost.
In the data access process, all need to carry out complicated abnormality processing, and in the distributed system of integration across database, it is complicated more that these abnormality processing become.This programme is handled common abnormal and is encapsulated significantly, and when appearance was unusual, caller did not need to give back PooledStatement or Bucket, does not need to be concerned about statement and the problems such as recovery that are connected yet.In the preferred embodiment of the present invention data are protected especially, when some taking place can cause database collapse unusual, will be carried out mark to the database of makeing mistakes, refusal carries out write operation to it again.
In another aspect of this invention, also provide a kind of realization system of database access,, having comprised referring to shown in Figure 4:
Interface module is used to provide the interface of database access, for non-database access of striding statement, provides borrowStatement () interface method to obtain Pooled Statement; For the database access of striding statement, provide borrowBucket () interface method to obtain Bucket;
The Bucket module is formed a Bucket example with of a sort SQL; Provide getStatement () method to be used to obtain the PooledStatement of Pooled Statement module;
Pooled Statement module, the interface method that provides carries out the data manipulation of database access;
Connect and statement pond administration module, be used for management and connect and the SQL statement pond.
The distributed data storehouse memorizer is used for the data of stored data base.
Wherein, described Bucket module, Pooled Statement module and be connected and statement pond administration module can be positioned at data access layer.
As a preferred embodiment, described connection and statement pond administration module, regularly statement and the database in (time interval can dispose) scanning pond connects, and discharges no resource, and the database connection of unusual disconnection is attempted reconnecting by certain strategy.In the data access configuration file, can specify the idle retention time of statement, surpassing this time statement is not used, its resource will be recovered, if all statements in Bucket example all are recovered, then this Bucket example connects together with the data of its use and all can be recovered.Application can be set different maintenance strategies for the different pieces of information visit, to obtain best performance.
Connection and statement pond administration module are managed the Bucket example by a two-dimensional array (being the Bucket pond), and first dimension is the Bucket type under the Bucket example, the database connection that second dimension takies for this example.When caller passed through borrowStatement () or borrowBucket () method application statement or Bucket, the parameter of importing into was " Bucket id " or " statement id ", and connection and the administration module operating process of SQL statement pond are generally as follows:
If parameter is " statement id ", then be converted into Bucket id by an index.
In the Bucket pond, find all Bucket examples that belong to this Bucket type, i.e. an one-dimension array according to Bucket id.
Scan this array,, be labeled as " using " if find the Bucket example of " free time ",, then this example is returned if that the caller application is Bucket; If application is statement, then in the statement that this Bucket example comprises, find this statement to return according to " statement id ".If can not find the Bucket example of " free time ", then process dormancy a period of time is waited for (time length can dispose),, also can not find the example of " free time " if after the stand-by period, then dish out " hurry " unusually.
Referring to shown in Figure 5,, comprise the steps: for implementing non-schematic flow sheet of striding the statement access process based on said system
Step 503 uses the returnMe () method of Pooled Statement to give back PooledStatement.
Fig. 6 comprises the steps: for implement to stride the schematic flow sheet of statement access process based on said system
Step 601, the borrowStatement () method of caller calling interface module obtain Pooled Statem ent.
Step 602, the borrowBucket () method of caller calling interface module obtains Bucket.
Step 603, the getStatement () method of use Bucket module obtains Pooled Statement wherein one by one.
Step 604, class JDBCpreparedStatment that use Pooled Statement module provides or the various interface method of callableStatment are carried out data manipulation.
Step 605 uses the returnMe () method of Bucket module to give back Bucket.
Below be system of the present invention example in actual applications:
The Best Tone Service system of certain province, DATA DISTRIBUTION is on 6 nodes, each node use a pair of (two) database (with call in the following text be equipped with mutually database to), critical data keeps two copies at this in to database, have 12 databases altogether, be distributed on 12 PC server hosts, the main frame in the system surpasses 50, is a large-scale Distributed Application.
This Best Tone Service system mainly provides local service for life information to the user, and the data owner in the database will comprise Business Information (information that its products ﹠ services that provide are provided), user profile etc.Use scheme provided by the invention to carry out database access, following several access mode arranged:
Read-only: as to be equipped with the data that database centering all has copy mutually, only to need to select one of them to conduct interviews.
Only write a database: as some access logs, these access logs are by tabulate statistics regularly.
Synchronous write: some change little critical data and adopt synchronous write, as newly-increased businessman, revise merchant visitor's essential information etc.
Asynchronous write: critical datas that some often change, the clicks accessed as businessman, when data break down after in, service needs to continue after the database that the breaks down reparation, to finish data synchronization by background program to obtain higher availability.
Since this Best Tone Service system formally reaches the standard grade; the operation efficient stable; wherein use data access scheme of the present invention to obtain very high performance and availability, and because of the shutdown of a database does not influence the main service of system, the well online database of back-up system upgrading.
This system deployment is used the (SuSE) Linux OS of increasing income and the MySQL Database Systems of increasing income on the PC server cluster, with low cost, has very high cost performance.
In the actual maintenance of this Best Tone Service system, the superiority of centralized configuration SQL of the present invention and database has obtained good embodiment, all SQL come into plain view with being connected in the configuration file, can visit data easily and be configured and tuning, when breaking down, can accomplish to know very well in one's heart more to the adjustment of system; In the large-scale software development that many people participate in, be the link of makeing mistakes easily at data access, the present invention does not allow to introduce database access except that the data access-profile on programming mode in system, is convenient to management and control.
Those of ordinary skill in the field are to be understood that: the above only is specific embodiments of the invention; be not limited to the present invention; within the spirit and principles in the present invention all, any modification of being made, be equal to replacement, improvement etc., all should be included within protection scope of the present invention.
Claims (13)
1. the implementation method of a database access is characterized in that, comprising:
SQL is classified;
Each Bucket of configuration in the data access configuration file comprises among the Bucket that the database of of a sort SQL and this Bucket connects;
If carry out non-database access of striding statement, then obtain Pooled Statement by borrowStatement () interface method; If stride the database access of statement, then obtain Bucket by borrowBucket () interface method, the getStatement () method that provides by Bucket obtains Pooled Statement wherein one by one;
The interface method that uses Pooled Statement to provide carries out the data manipulation of this database access.
2. method according to claim 1 is characterized in that, if carry out non-database access of striding statement, then by borrowStatement () interface method, obtains Pooled Statement according to statement id; If stride the database access of statement, then, obtain Bucket according to statement Bucket id by borrowBucket () interface method.
3. method according to claim 1 is characterized in that, manages the Bucket example by a two-dimensional array, and first dimension is the Bucket type under the Bucket example, and second dimension connects for the database that this example takies;
When caller passed through borrowStatement () or borrowBucket () method application statement or Bucket, the parameter of importing into was " Bucket id " or " statement id ", and the pool manager operating process is as follows:
If parameter is " statement id ", then be converted into Bucket id by an index.
In the Bucket pond, find all Bucket examples that belong to this Bucket type, i.e. an one-dimension array according to Bucket id;
Scan this array,, be labeled as " using " if find the Bucket example of " free time ",, then this example is returned if that the caller application is Bucket; If application is statement, then in the statement that this Bucket example comprises, find this statement to return according to " statement id "; If can not find the Bucket example of " free time ", then process dormancy a period of time is waited for,, also can not find the example of " free time " if after the stand-by period, the exception response that the expression of then dishing out " is done ".
4. method according to claim 1 is characterized in that, gives back Pooled Statement or Bucket by returnMe () method.
5. method according to claim 1 is characterized in that, this method further comprises: all Bucket centralized configuration in a data access-profile that a database access is required.
6. method according to claim 1 is characterized in that, described data access configuration file adopts the Properties file of Java, and the collocation form of statement is " statement id=SQL ", and the collocation form of Bucket is as follows:
The statement that comprises: class[bucket id] a plurality of statement id that the .stmts=comma separates, defined the SQL statement that comprises among the Bucket;
The database linking number that can use at most: class[bucket id] the database linking number that can use at most of .conn=;
Whether database connects can write: class[bucket id] .update=true/false, be defaulted as false, promptly read-only.
7. method according to claim 1 is characterized in that, if described database access is a write operation, then adopts one of following three kinds of modes to carry out:
Only write a database: data are only write a database and are promptly finished;
A plurality of data sync are write: data are wanted a plurality of databases of writing of success, and affairs just are counted as merit;
A plurality of database asynchronous writes: attempt earlier data are write in a plurality of databases, if success, then affairs complete successfully; If one of them database write is then selected in failure, and writes down these data and need be synchronized in other databases, carry out afterwards data sync by the synchronization mechanism on backstage.
8. method according to claim 1 is characterized in that, when appearance in the database access process was unusual, caller was not given back Pooled Statement or Bucket, will carry out mark to the database of makeing mistakes, and refusal carries out write operation to it again.
9. method according to claim 1 is characterized in that, in the idle retention time of described data access configuration file middle finger attribute sentence, still is not used if exceed this retention time free time statement, then reclaims the resource of this statement; If all statements in Bucket example all are recovered, then this Bucket example connects together with the data of its use and all can be recovered.
10. the realization system of a database access is characterized in that, comprising:
Interface module is used to provide the interface of database access, for non-database access of striding statement, provides borrowStatement () interface method to obtain Pooled Statement; For the database access of striding statement, provide borrowBucket () interface method to obtain Bucket;
The Bucket module is formed a Bucket example with of a sort SQL; Provide getStatement () method to be used to obtain the PooledStatement of Pooled Statement module;
Pooled Statement module, the interface method that provides carries out the data manipulation of database access;
Connect and statement pond administration module, be used for management and connect and the SQL statement pond;
The distributed data storehouse memorizer is used for the data of stored data base.
11. system according to claim 10 is characterized in that, described interface module if carry out non-database access of striding statement, then by borrowStatement () interface method, is obtained Pooled Statement according to statement id; If stride the database access of statement, then, obtain Bucket according to statement Bucket id by borrowBucket () interface method.
12. system according to claim 10 is characterized in that, described Bucket module is managed the Bucket example by a two-dimensional array, and first dimension is the Bucket type under the Bucket example, and second dimension connects for the database that this example takies;
When caller passed through borrowStatement () or borrowBucket () method application statement or Bucket, the parameter of importing into was " Bucket id " or " statement id ", and the pool manager operating process is as follows:
If parameter is " statement id ", then be converted into Bucket id by an index.
In the Bucket pond, find all Bucket examples that belong to this Bucket type, i.e. an one-dimension array according to Bucket id;
Scan this array,, be labeled as " using " if find the Bucket example of " free time ",, then this example is returned if that the caller application is Bucket; If application is statement, then in the statement that this Bucket example comprises, find this statement to return according to " statement id "; If can not find the Bucket example of " free time ", then process dormancy a period of time is waited for,, also can not find the example of " free time " if after the stand-by period, the exception response that the expression of then dishing out " is done ".
13. system according to claim 10 is characterized in that, described interface module is given back Pooled Statement or Bucket by returnMe () method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010105365329A CN101980207B (en) | 2010-11-09 | 2010-11-09 | Method and system for implementing database access |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010105365329A CN101980207B (en) | 2010-11-09 | 2010-11-09 | Method and system for implementing database access |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101980207A true CN101980207A (en) | 2011-02-23 |
| CN101980207B CN101980207B (en) | 2012-05-30 |
Family
ID=43600709
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2010105365329A Active CN101980207B (en) | 2010-11-09 | 2010-11-09 | Method and system for implementing database access |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101980207B (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102945264A (en) * | 2012-10-24 | 2013-02-27 | 浪潮集团山东通用软件有限公司 | Method for intelligently starting distributed transaction |
| CN102999392A (en) * | 2012-11-23 | 2013-03-27 | 北京奇虎科技有限公司 | Method and device for allocating database operation request |
| CN102999575A (en) * | 2012-11-12 | 2013-03-27 | 北京华通伟业科技发展有限公司 | Method and system for configuring Hibernate mapping configuration file |
| CN103150344A (en) * | 2013-02-06 | 2013-06-12 | 湖北微驾技术有限公司 | Driving training examination system data access method based on ORM (object relational mapping) framework |
| CN102955851B (en) * | 2012-10-30 | 2016-05-11 | 北京奇虎科技有限公司 | A kind of switching method and apparatus of database |
| CN105653356A (en) * | 2016-01-05 | 2016-06-08 | 世纪禾光科技发展（北京）有限公司 | Method and device for processing multi-server concurrent operation |
| CN106126540A (en) * | 2016-06-15 | 2016-11-16 | 中国传媒大学 | Data base access system and access method thereof |
| WO2018120171A1 (en) * | 2016-12-30 | 2018-07-05 | 华为技术有限公司 | Method, device and system for executing stored procedure |
| CN108255852A (en) * | 2016-12-29 | 2018-07-06 | 中国移动通信集团浙江有限公司 | SQL performs method and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1869939A (en) * | 2005-05-27 | 2006-11-29 | 华为技术有限公司 | Method for access data base by multi-line program applied program |
| CN101395602A (en) * | 2005-12-29 | 2009-03-25 | 亚马逊科技公司 | Method and apparatus for a distributed file storage and indexing service |
| US20100030817A1 (en) * | 2008-07-31 | 2010-02-04 | Sybase, Inc | Statement categorization and normalization |
| CN101840352A (en) * | 2010-04-29 | 2010-09-22 | 中兴通讯股份有限公司 | Method and device for monitoring database connection pool |
-
2010
- 2010-11-09 CN CN2010105365329A patent/CN101980207B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1869939A (en) * | 2005-05-27 | 2006-11-29 | 华为技术有限公司 | Method for access data base by multi-line program applied program |
| CN101395602A (en) * | 2005-12-29 | 2009-03-25 | 亚马逊科技公司 | Method and apparatus for a distributed file storage and indexing service |
| US20100030817A1 (en) * | 2008-07-31 | 2010-02-04 | Sybase, Inc | Statement categorization and normalization |
| CN101840352A (en) * | 2010-04-29 | 2010-09-22 | 中兴通讯股份有限公司 | Method and device for monitoring database connection pool |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102945264B (en) * | 2012-10-24 | 2017-02-15 | 浪潮通用软件有限公司 | Method for intelligently starting distributed transaction |
| CN102945264A (en) * | 2012-10-24 | 2013-02-27 | 浪潮集团山东通用软件有限公司 | Method for intelligently starting distributed transaction |
| CN102955851B (en) * | 2012-10-30 | 2016-05-11 | 北京奇虎科技有限公司 | A kind of switching method and apparatus of database |
| CN102999575A (en) * | 2012-11-12 | 2013-03-27 | 北京华通伟业科技发展有限公司 | Method and system for configuring Hibernate mapping configuration file |
| CN102999575B (en) * | 2012-11-12 | 2015-10-14 | 北京华通伟业科技发展有限公司 | A kind of collocation method of Hibernate Map Profile and system |
| CN102999392B (en) * | 2012-11-23 | 2016-12-21 | 北京奇虎科技有限公司 | The method and apparatus of allocation database operation requests |
| CN102999392A (en) * | 2012-11-23 | 2013-03-27 | 北京奇虎科技有限公司 | Method and device for allocating database operation request |
| CN103150344A (en) * | 2013-02-06 | 2013-06-12 | 湖北微驾技术有限公司 | Driving training examination system data access method based on ORM (object relational mapping) framework |
| CN105653356A (en) * | 2016-01-05 | 2016-06-08 | 世纪禾光科技发展（北京）有限公司 | Method and device for processing multi-server concurrent operation |
| CN105653356B (en) * | 2016-01-05 | 2018-12-11 | 数贸科技（北京）有限公司 | Handle the method and device of multiserver concurrent operations |
| CN106126540A (en) * | 2016-06-15 | 2016-11-16 | 中国传媒大学 | Data base access system and access method thereof |
| CN106126540B (en) * | 2016-06-15 | 2019-07-09 | 中国传媒大学 | Data base access system and its access method |
| CN108255852A (en) * | 2016-12-29 | 2018-07-06 | 中国移动通信集团浙江有限公司 | SQL performs method and device |
| CN108255852B (en) * | 2016-12-29 | 2020-09-11 | 中国移动通信集团浙江有限公司 | SQL execution method and device |
| WO2018120171A1 (en) * | 2016-12-30 | 2018-07-05 | 华为技术有限公司 | Method, device and system for executing stored procedure |
| US11182353B2 (en) | 2016-12-30 | 2021-11-23 | Huawei Technologies Co., Ltd. | Stored-procedure execution method and device, and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101980207B (en) | 2012-05-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101980207B (en) | Method and system for implementing database access | |
| CN107679192B (en) | Multi-cluster cooperative data processing method, system, storage medium and equipment | |
| Mondal et al. | Managing large dynamic graphs efficiently | |
| CN109445802B (en) | Privatized Paas platform based on container and method for publishing application thereof | |
| US5687363A (en) | Distributed database architecture and distributed database management system for open network evolution | |
| US5721909A (en) | Distributed database architecture and distributed database management system for open network evolution | |
| US20110154339A1 (en) | Incremental mapreduce-based distributed parallel processing system and method for processing stream data | |
| US5835757A (en) | Distributed database management system for servicing application requests in a telecommunications switching system | |
| US5764977A (en) | Distributed database architecture and distributed database management system for open network evolution | |
| EP0675451A2 (en) | A distributed database architecture and distributed database management system for open network evolution | |
| CN101360123B (en) | Network system and management method thereof | |
| CN111061788A (en) | Multi-source heterogeneous data conversion integration system based on cloud architecture and implementation method thereof | |
| CN111400326A (en) | Smart city data management system and method thereof | |
| CN100538646C (en) | A kind of method and apparatus of in distributed system, carrying out the SQL script file | |
| CN108073696B (en) | GIS application method based on distributed memory database | |
| CN109885642B (en) | Hierarchical storage method and device for full-text retrieval | |
| CN105069151A (en) | HBase secondary index construction apparatus and method | |
| CN107977446A (en) | A kind of memory grid data load method based on data partition | |
| CN109669975B (en) | Industrial big data processing system and method | |
| US5884311A (en) | Method and system for dynamically configuring a relational database | |
| CN111126852A (en) | BI application system based on big data modeling | |
| Delaney et al. | Microsoft SQL Server 2012 Internals | |
| CN111930354A (en) | Framework component for software development and construction method thereof | |
| CN113051263A (en) | Metadata-based big data platform construction method, system, equipment and medium | |
| CN112925767A (en) | Multi-data-source dynamic data synchronization management method and system based on internet supervision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20110223 Assignee: Tianyiyun Technology Co.,Ltd. Assignor: CHINA TELECOM Corp.,Ltd. Contract record no.: X2024110000020 Denomination of invention: An Implementation Method and System for Database Access Granted publication date: 20120530 License type: Common License Record date: 20240315 |
|
| EE01 | Entry into force of recordation of patent licensing contract |