CN110795419A - Method and device for dynamic database-based routing - Google Patents
Method and device for dynamic database-based routing Download PDFInfo
- Publication number
- CN110795419A CN110795419A CN201910949147.8A CN201910949147A CN110795419A CN 110795419 A CN110795419 A CN 110795419A CN 201910949147 A CN201910949147 A CN 201910949147A CN 110795419 A CN110795419 A CN 110795419A
- Authority
- CN
- China
- Prior art keywords
- data
- horizontal
- segmentation rule
- database
- sub
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/214—Database migration support
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/211—Schema design and management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method and a device for dynamic database-based routing, and relates to the technical field of computers. One embodiment of the method comprises: acquiring dynamic configuration information of a data entity; if the database partitioning mode of the data entity is vertical database partitioning, determining a data sub-table corresponding to the data to be processed according to a vertical segmentation rule; if the database partitioning mode of the data entity is horizontal database partitioning, determining a data sub-table corresponding to the data to be processed according to a horizontal segmentation rule and a global index table; and establishing connection with the data sub-table to realize sub-library routing. The invention can support the rapid transverse expansion of the application server, has rapid migration capability when redistributing data and has good expansibility; various database partitioning modes and partitioning rules are supported, and different service scenes are adapted; the data retrieval condition is flexible, and complex service application is supported.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a method and a device for dynamic database-based routing.
Background
With the development of the internet, especially the mobile internet, the number of mobile applications has increased explosively, the number of users has also increased to an unprecedented scale, and the data volume of billions of users is uniform. An IT technical team almost has the problems of huge data volume, slow business query, large data volume occupying a large amount of storage, large disk IO concurrency and the like. Generally, if the Mysql library exceeds 5000 million records, and the Oracle library exceeds 1 hundred million records, the DB pressure is very large, and of course, the capacity and the specific services, the number of fields, the access mode, the data content and the like have further relationships. If the query is only slow, the method can be firstly solved by means of sql optimization, caching, read-write separation and the like. When the data amount increases beyond the single bank limit, performance problems easily occur, and it is natural to think of sub-bank sub-table storing data, i.e. sharing.
In the prior art, a vertical segmentation or horizontal segmentation mode is generally adopted for library division and table division.
In the process of implementing the invention, the inventor finds that the prior art has at least the following problems:
1. the vertical segmentation can only solve the dynamic routing based on the vertical segmentation model, and the expansibility is limited after the segmentation reaches a certain degree;
2. the horizontal segmentation can only solve the dynamic routing based on the horizontal segmentation model, the data retrieval must be carried with the KEY _ ID condition, and the method has certain limitation and is not suitable for the service application scene of the data retrieval under various conditions.
Disclosure of Invention
In view of this, embodiments of the present invention provide a method and an apparatus for dynamic split-base routing, which can support fast horizontal extension of an application server, and have fast migration capability when redistributing data, and good extensibility; various database partitioning modes and partitioning rules are supported, and different service scenes are adapted; the data retrieval condition is flexible, and complex service application is supported; and the bottom layer data segmentation model is shielded from application service developers, so that the method is easy to use and high in fault tolerance rate.
To achieve the above object, according to an aspect of the present invention, there is provided a method for dynamic split-base routing, including:
acquiring dynamic configuration information of a data entity, wherein the dynamic configuration information comprises: the method comprises the steps of a library partitioning mode, a vertical segmentation rule, a horizontal segmentation rule and a global index table;
if the database partitioning mode of the data entity is vertical database partitioning, determining a data sub-table corresponding to the data to be processed according to the vertical segmentation rule; if the database partitioning mode of the data entity is horizontal database partitioning, determining a data sub-table corresponding to the data to be processed according to the horizontal segmentation rule and the global index table;
and establishing connection with the data sub-table to realize sub-library routing.
Optionally, determining a data sublist corresponding to the to-be-processed data according to the horizontal segmentation rule and the global index table includes:
judging whether the search word of the data to be processed is a database partitioning main key or not;
if so, determining a data sublist corresponding to the data to be processed directly according to the horizontal segmentation rule;
otherwise, inquiring a database partitioning main key corresponding to the search term from the global index table, and then determining a data sub-table corresponding to the data to be processed according to the database partitioning main key and the horizontal segmentation rule.
Optionally, the horizontal slicing rule includes: and according to the segmentation rule of the Hash fragment and the segmentation rule of the time slice.
Optionally, the method of the embodiment of the present invention is implemented in a data DAO layer.
According to another aspect of the present invention, there is provided an apparatus for dynamic split-base routing, including:
an obtaining module, configured to obtain dynamic configuration information of a data entity, where the dynamic configuration information includes: the method comprises the steps of a library partitioning mode, a vertical segmentation rule, a horizontal segmentation rule and a global index table;
the routing module is used for determining a data sublist corresponding to the data to be processed according to the vertical segmentation rule if the sublist mode of the data entity is vertical sublist; if the database partitioning mode of the data entity is horizontal database partitioning, determining a data sub-table corresponding to the data to be processed according to the horizontal segmentation rule and the global index table;
and the connection module is used for establishing connection with the data sub-table so as to realize sub-library routing.
Optionally, the determining, by the routing module, a data sublist corresponding to the to-be-processed data according to the horizontal segmentation rule and the global index table includes:
judging whether the search word of the data to be processed is a database partitioning main key or not;
if so, determining a data sublist corresponding to the data to be processed directly according to the horizontal segmentation rule;
otherwise, inquiring a database partitioning main key corresponding to the search term from the global index table, and then determining a data sub-table corresponding to the data to be processed according to the database partitioning main key and the horizontal segmentation rule.
Optionally, the horizontal slicing rule includes: and according to the segmentation rule of the Hash fragment and the segmentation rule of the time slice.
Optionally, the obtaining module, the routing module, and the connection module are disposed on a data DAO layer.
According to another aspect of the present invention, there is provided an electronic device for dynamic library-based routing, comprising:
one or more processors;
a storage device for storing one or more programs,
when the one or more programs are executed by the one or more processors, the one or more processors implement the method for implementing dynamic split-base routing provided by the first aspect of the embodiments of the present invention.
According to a further aspect of the invention, a computer-readable storage medium is provided, on which a computer program is stored which, when being executed by a processor, carries out the method provided by the invention.
According to the technical scheme of the invention, one embodiment of the invention has the following advantages or beneficial effects: the horizontal database partitioning can support the rapid transverse expansion of the application server, and the data redistribution has rapid migration capability and good expansibility; the configuration information is set to support various database partitioning modes and partitioning rules, so that the method can adapt to different service scenes; by setting the global index table, the data retrieval condition is flexible, and complex service application is supported. By executing the method of the embodiment of the invention on the DAO layer, the bottom layer data segmentation model can be shielded for application service developers, and the method has the advantages of high usability and high fault tolerance rate.
Further effects of the above-mentioned non-conventional alternatives will be described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
FIG. 1 is a schematic diagram of the main steps of a dynamic database-based routing method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a main flow of a method for dynamic database-based routing according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of the main modules of the dynamic database-based routing apparatus according to the embodiment of the present invention;
FIG. 4 is an exemplary system architecture diagram in which embodiments of the present invention may be employed;
fig. 5 is a schematic structural diagram of an electronic device for implementing the method in the embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present invention are described below with reference to the accompanying drawings, in which various details of embodiments of the invention are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
It should be noted that the embodiments of the present invention and the technical features of the embodiments may be combined with each other without conflict.
FIG. 1 is a schematic diagram of the main steps of a method according to an embodiment of the present invention.
As shown in fig. 1, the method according to the embodiment of the present invention may be specifically performed according to the following steps:
step S101, obtaining dynamic configuration information of a data entity, wherein the dynamic configuration information comprises: the system comprises a database partitioning mode, a vertical segmentation rule, a horizontal segmentation rule and a global index table.
S102, judging whether a database partitioning mode of the data entity is horizontal database partitioning or vertical database partitioning; if the database partitioning mode of the data entity is vertical database partitioning, jumping to step S103; otherwise, jumping to step S104;
s103, determining a data sub-table corresponding to the data to be processed according to the vertical segmentation rule;
s104, determining a data sub-table corresponding to the data to be processed according to the horizontal segmentation rule and the global index table;
and step S103, establishing connection with the data sub-table to realize sub-base routing.
In the actual application process, the dynamic configuration information of the data entity can be stored in a cache; obtaining dynamic configuration information for a data entity includes: and acquiring the dynamic configuration information of the data entity from the cache. Therefore, database access operation can be reduced, and database performance is improved.
The banking schema may be a vertical banking schema (i.e., vertical slicing) or a horizontal banking schema (i.e., horizontal slicing). The principle of the vertical segmentation is as follows: the tables with close service and close relation among the tables are divided together, for example, the tables of the same module. Combining the prepared database ER map or the domain model map, following the concept of the lanes in the activity map, one lane represents one shrard, and all tables are divided into different lanes. Each database table vertically split corresponds to a fixed sub-database. After vertical slicing, the data size and acceleration of the tables in the board may be further analyzed to determine whether horizontal slicing is required.
If the data of the database tables divided together slowly increases, after the service function is on line, the single database can bear the load in a long enough future period, so that horizontal segmentation is not needed, and all the tables reside in the same database. Therefore, the incidence relation among the database tables can be reserved to the maximum extent, and the execution flexibility of database operation statements including complex operations such as association, paging and the like is ensured.
If the amount of table data divided together is very large and is expected to grow rapidly in the future, further horizontal division is required. The principle of horizontal segmentation is as follows: in conjunction with business logic and inter-table relationships, a current shelf is divided into a plurality of smaller shelves, each of which typically contains only one primary table (the table to be hashed with its unique ID) and a plurality of secondary tables associated or indirectly associated therewith. This situation of one shard with multiple primary and secondary tables is a corollary of horizontal slicing. Thus, the number of shards is rapidly increased.
After the horizontal splitting is completed, if each shrard represents an independent database, the management and maintenance of the database are very troublesome, and the small shrards usually only have two or three tables, so that a new library is established, and the utilization rate is not high, so that after the horizontal splitting is completed, the merging can be performed once again, and two or more shrards which are similar in service and have similar data growth rates (the data volume of the main table is in the same order of magnitude) are put on the same database.
The horizontal segmentation is based on a certain sub-library main KEY KEY _ ID, and the sub-partitions and the sub-libraries are made according to rules. In some embodiments, the horizontal slicing rule is a slicing rule in accordance with hash slicing. For example, the method comprises the steps of splitting according to a user number HASH, performing modular extraction by using the user number HASH, splitting a single library into n libraries according to data volume evaluation, and storing the n libraries into m groups of servers respectively. The horizontal database partitioning mode is suitable for an internet user system and a business scene with KEY _ ID balanced distribution.
In other embodiments, the horizontal slicing rule may also be a time-sliced slicing rule, such as a rule for performing segmented storage based on a self-increment sequence. The method is suitable for systems such as Internet e-commerce orders and the like, and the KEY _ ID is composed of a pure numerical value type self-increment sequence. It should be noted that, in the actual application process, the horizontal segmentation may also be a self-defined segmentation rule in other special service scenarios.
According to the invention, the application server can be supported to be rapidly and transversely expanded through the horizontal database partitioning, and the data redistribution has rapid migration capability and good expansibility; the configuration information is set to support various database partitioning modes and partitioning rules, so that the method can adapt to different service scenes.
In order to facilitate routing of the table after horizontal segmentation, the embodiment of the invention constructs a global index table. Storing the mapping relation between each search term and the library division main key in the global index table, and directly routing according to the search terms when the search terms input by the user are the library division main keys; and when the search word input by the user is not the sub-base main key, searching the sub-base main key corresponding to the search word from the global index table, and routing according to the searched sub-base main key.
Illustratively, performing hash library division and table division according to a user number can meet the requirements of creating an order and performing query operation through a user identification USERID dimension, but business may have other retrieval conditions, for example, query is performed according to certificate information, a mobile phone number, and the like, and operations such as cut (Create), Update (Update), read (Retrieve), Delete (Delete) and the like of performing an order through other conditions need to be solved, so a global index table needs to be created.
For example, the certificate number index table is a corresponding relation table (see table 1 below) for the user number and the certificate number, and hash and modulo are performed according to the certificate number and the certificate number is placed in the sub-library. When inquiring according to the certificate number, firstly checking out the user number corresponding to the certificate number, and then taking a module according to the user number to inquire the corresponding database to inquire order data.
The relation between the certificate number and the user number is not changed after being established, in order to further improve the performance, a cache is introduced, the relation between the certificate number and the user number is stored in the cache, the table look-up operation is reduced, the performance is improved, the table look-up is carried out when the index is not hit, and the query result is updated in the cache.
TABLE 1 certificate information INDEX Table, CUSTINFO _ INDEX
| Name of field | Type of field | Constraining | Description of the invention |
| Platform client number | Char(20) | ||
| Document type | Char(4) | ||
| Certificate number | Char(20) | ||
| Name of customer |
In table 1, the primary key is the platform client number and the search term is certificate type + certificate number.
Determining a data sublist corresponding to the data to be processed according to the horizontal segmentation rule and the global index table, wherein the data sublist comprises the following steps: judging whether a search word of the data to be processed is a primary key of the database division; if so, determining a data sub-table corresponding to the data to be processed directly according to a horizontal segmentation rule; otherwise, inquiring the database partitioning main key corresponding to the search term from the global index table, and then determining the data partitioning table corresponding to the data to be processed according to the database partitioning main key and the horizontal partitioning rule. By setting the global index table, the data retrieval condition is flexible, and complex service application is supported.
Fig. 2 is a schematic diagram of a main flow of a method for dynamic database-based routing in the embodiment of the present invention. In this example, the attributes of each data entity are defined in advance as follows
1. name is entity name;
2. dsc, description of the entity;
3. db _ rule is a database partitioning mode, a vertical database partitioning or a horizontal database partitioning;
4. if the current sub-library is a horizontal sub-library, defining a horizontal segmentation rule of the current sub-library, and corresponding to a callback method, namely, sepresult (key _ id);
5. the global index table, which may have other retrieval conditions in service, needs to be established.
As shown in fig. 2, the method for dynamic database-based routing includes: the following steps are executed at the data DAO layer:
step S201, obtaining dynamic configuration information of a data entity, where the dynamic configuration information includes: the method comprises the steps of a library partitioning mode, a vertical segmentation rule, a horizontal segmentation rule and a global index table;
step S202, judging whether the database partitioning mode of the data entity is horizontal database partitioning or vertical database partitioning. If the database is horizontally divided, jumping to step S203, otherwise, jumping to step S207;
and step S203, judging whether the search word of the data to be processed is a main key of the database division. If yes, jumping to step S204, otherwise, jumping to step S206;
step S204, inquiring a sub-library main key corresponding to the search term from the global index table;
s205, determining a data sublist corresponding to the data to be processed according to the sublist key and the horizontal segmentation rule;
step S206, determining a data sublist corresponding to the data to be processed according to the search terms and the horizontal segmentation rule
Step S207, determining a data sublist corresponding to the data to be processed according to a vertical segmentation rule
And step S208, establishing connection with the data sub-table to realize sub-base routing.
By implementing the method of the embodiment of the invention on the DAO layer, the bottom layer data segmentation model can be shielded for application service developers, and the method has the advantages of easy use, high fault tolerance and high fault tolerance rate.
According to the invention, the application server can be supported to be rapidly and transversely expanded through the horizontal database partitioning, and the data redistribution has rapid migration capability and good expansibility; the configuration information is set to support various database partitioning modes and partitioning rules, so that the method can adapt to different service scenes; by setting the global index table, the data retrieval condition is flexible, and complex service application is supported. By executing the method of the embodiment of the invention on the DAO layer, the bottom layer data segmentation model can be shielded for application service developers, and the method has the advantages of high usability and high fault tolerance rate.
To facilitate a better implementation of the above-described aspects of embodiments of the present invention, the following also provides relevant means for implementing the above-described aspects.
Referring to fig. 3, an apparatus 300 according to an embodiment of the present invention includes:
the obtaining module 301 obtains dynamic configuration information of a data entity, where the dynamic configuration information includes: the method comprises the steps of a library partitioning mode, a vertical segmentation rule, a horizontal segmentation rule and a global index table;
the routing module 302 determines a data sublist corresponding to the data to be processed according to the vertical segmentation rule if the sublist mode of the data entity is vertical sublist; if the database partitioning mode of the data entity is horizontal database partitioning, determining a data sub-table corresponding to the data to be processed according to the horizontal segmentation rule and the global index table;
and the connection module 303 establishes connection with the data sub-table to realize sub-base routing.
Optionally, the determining, by the routing module, a data sublist corresponding to the to-be-processed data according to the horizontal segmentation rule and the global index table includes:
judging whether the search word of the data to be processed is a database partitioning main key or not;
if so, determining a data sublist corresponding to the data to be processed directly according to the horizontal segmentation rule;
otherwise, inquiring a database partitioning main key corresponding to the search term from the global index table, and then determining a data sub-table corresponding to the data to be processed according to the database partitioning main key and the horizontal segmentation rule.
Optionally, the horizontal slicing rule includes: and according to the segmentation rule of the Hash fragment and the segmentation rule of the time slice.
Optionally, the obtaining module, the routing module, and the connection module are disposed on a data DAO layer.
According to the technical scheme of the invention, one embodiment of the invention has the following advantages or beneficial effects: the horizontal database partitioning can support the rapid transverse expansion of the application server, and the data redistribution has rapid migration capability and good expansibility; the configuration information is set to support various database partitioning modes and partitioning rules, so that the method can adapt to different service scenes; by setting the global index table, the data retrieval condition is flexible, and complex service application is supported. By executing the method of the embodiment of the invention on the DAO layer, the bottom layer data segmentation model can be shielded for application service developers, and the method has the advantages of high usability and high fault tolerance rate.
It should be noted that, for the convenience of description, the foregoing method embodiments are described as a series of acts, but those skilled in the art will appreciate that the present invention is not limited by the order of acts described, and that some steps may in fact be performed in other orders or concurrently. Moreover, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no acts or modules are necessarily required to implement the invention.
Fig. 4 illustrates an exemplary system architecture 400 to which the methods or apparatus of embodiments of the invention may be applied.
As shown in fig. 4, the system architecture 400 may include terminal devices 401, 402, 403, a network 404, and a server 405 (this architecture is merely an example, and the components included in a particular architecture may be adapted according to application specific circumstances). The network 404 serves as a medium for providing communication links between the terminal devices 401, 402, 403 and the server 405. Network 404 may include various types of connections, such as wire, wireless communication links, or fiber optic cables, to name a few.
A user may use terminal devices 401, 402, 403 to interact with a server 405 over a network 404 to receive or send messages or the like. The terminal devices 401, 402, 403 may have various client applications installed thereon, such as a shopping-like application, a web browser application, a search-like application, an instant messaging tool, a mailbox client, social platform software, etc. (by way of example only).
The terminal devices 401, 402, 403 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.
The server 405 may be a server providing various services, such as a background management server (for example only) providing support for shopping websites browsed by users using the terminal devices 401, 402, 403. The backend management server may process the received product information query request and feed back the processing result (e.g., target push information, product information-just an example) to the terminal device 401, 402, 403.
It should be noted that the method for dynamic banking and routing provided by the embodiment of the present invention is generally executed by the server 405, and accordingly, the apparatus for dynamic banking and routing is generally disposed in the server 405.
It should be understood that the number of terminal devices, networks, and servers in fig. 4 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
The invention also provides the electronic equipment. The electronic device of the embodiment of the invention comprises: one or more processors; the storage device is used for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors realize the method provided by the invention.
Referring now to FIG. 5, shown is a block diagram of a computer system 500 suitable for use in implementing an electronic device of an embodiment of the present invention. The electronic device shown in fig. 5 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.
As shown in fig. 5, the computer system 500 includes a Central Processing Unit (CPU)501 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)502 or a program loaded from a storage section 508 into a Random Access Memory (RAM) 503. In the RAM503, various programs and data necessary for the operation of the computer system 500 are also stored. The CPU501, ROM 502, and RAM503 are connected to each other via a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.
The following components are connected to the I/O interface 505: an input portion 506 including a keyboard, a mouse, and the like; an output portion 507 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage portion 508 including a hard disk and the like; and a communication section 509 including a network interface card such as a LAN card, a modem, or the like. The communication section 509 performs communication processing via a network such as the internet. The driver 510 is also connected to the I/O interface 505 as necessary. A removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 510 as necessary, so that a computer program read out therefrom is mounted into the storage section 508 as necessary.
In particular, the processes described in the main step diagrams above may be implemented as computer software programs, according to embodiments of the present disclosure. For example, embodiments of the invention include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the main step diagram. In the above-described embodiment, the computer program can be downloaded and installed from a network through the communication section 509, and/or installed from the removable medium 511. The computer program performs the above-described functions defined in the system of the present invention when executed by the central processing unit 501.
It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in the embodiments of the present invention may be implemented by software or hardware. The described modules may also be provided in a processor, which may be described as: a processor comprising: the acquisition module acquires dynamic configuration information of the data entity; the routing module is used for determining a data sublist corresponding to the data to be processed according to a vertical segmentation rule if the sublist mode of the data entity is vertical sublist; if the database partitioning mode of the data entity is horizontal database partitioning, determining a data sub-table corresponding to the data to be processed according to a horizontal segmentation rule and a global index table; and the connection module is used for establishing connection with the data sub-table so as to realize sub-library routing. The names of these modules do not in some cases form a limitation on the module itself, and for example, the acquiring module may also be described as a "module that establishes a connection with a data sublist".
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be separate and not incorporated into the device. The computer readable medium carries one or more programs which, when executed by the apparatus, cause the apparatus to perform steps comprising: acquiring dynamic configuration information of a data entity, wherein the dynamic configuration information comprises: the method comprises the steps of a library partitioning mode, a vertical segmentation rule, a horizontal segmentation rule and a global index table; the routing module is used for determining a data sublist corresponding to the data to be processed according to the vertical segmentation rule if the sublist mode of the data entity is vertical sublist; if the database partitioning mode of the data entity is horizontal database partitioning, determining a data sub-table corresponding to the data to be processed according to the horizontal segmentation rule and the global index table; and establishing connection with the data sub-table to realize sub-library routing.
According to the technical scheme of the embodiment of the invention, the application server can be supported to be rapidly and transversely expanded through horizontal database partitioning, the data redistribution has rapid migration capability, and the expansibility is good; the configuration information is set to support various database partitioning modes and partitioning rules, so that the method can adapt to different service scenes; by setting the global index table, the data retrieval condition is flexible, and complex service application is supported. By executing the method of the embodiment of the invention on the DAO layer, the bottom layer data segmentation model can be shielded for application service developers, and the method has the advantages of high usability and high fault tolerance rate.
The above-described embodiments should not be construed as limiting the scope of the invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions can occur, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A method for dynamic banking routing, comprising:
acquiring dynamic configuration information of a data entity, wherein the dynamic configuration information comprises: the method comprises the steps of a library partitioning mode, a vertical segmentation rule, a horizontal segmentation rule and a global index table;
if the database partitioning mode of the data entity is vertical database partitioning, determining a data sub-table corresponding to the data to be processed according to the vertical segmentation rule; if the database partitioning mode of the data entity is horizontal database partitioning, determining a data sub-table corresponding to the data to be processed according to the horizontal segmentation rule and the global index table;
and establishing connection with the data sub-table to realize sub-library routing.
2. The method of claim 1, wherein determining a data sublist corresponding to the data to be processed according to the horizontal slicing rule and the global index table comprises:
judging whether the search word of the data to be processed is a database partitioning main key or not;
if so, determining a data sublist corresponding to the data to be processed directly according to the horizontal segmentation rule;
otherwise, inquiring a database partitioning main key corresponding to the search term from the global index table, and then determining a data sub-table corresponding to the data to be processed according to the database partitioning main key and the horizontal segmentation rule.
3. The method of claim 2, wherein the horizontal slicing rule comprises: and according to the segmentation rule of the Hash fragment and the segmentation rule of the time slice.
4. The method of claim 1, wherein the method is implemented at a data DAO layer.
5. An apparatus for dynamic banking routing, comprising:
an obtaining module, configured to obtain dynamic configuration information of a data entity, where the dynamic configuration information includes: the method comprises the steps of a library partitioning mode, a vertical segmentation rule, a horizontal segmentation rule and a global index table;
the routing module is used for determining a data sublist corresponding to the data to be processed according to the vertical segmentation rule if the sublist mode of the data entity is vertical sublist; if the database partitioning mode of the data entity is horizontal database partitioning, determining a data sub-table corresponding to the data to be processed according to the horizontal segmentation rule and the global index table;
and the connection module is used for establishing connection with the data sub-table so as to realize sub-library routing.
6. The apparatus of claim 5, wherein the routing module determines a data sublist corresponding to the data to be processed according to the horizontal slicing rule and the global index table, and comprises:
judging whether the search word of the data to be processed is a database partitioning main key or not;
if so, determining a data sublist corresponding to the data to be processed directly according to the horizontal segmentation rule;
otherwise, inquiring a database partitioning main key corresponding to the search term from the global index table, and then determining a data sub-table corresponding to the data to be processed according to the database partitioning main key and the horizontal segmentation rule.
7. The apparatus of claim 6, wherein the horizontal slicing rule comprises: and according to the segmentation rule of the Hash fragment and the segmentation rule of the time slice.
8. The apparatus of claim 5, wherein the obtaining module, the routing module, and the connection module are disposed at a data DAO layer.
9. An electronic device for dynamic banking routing, comprising:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-4.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910949147.8A CN110795419A (en) | 2019-10-08 | 2019-10-08 | Method and device for dynamic database-based routing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910949147.8A CN110795419A (en) | 2019-10-08 | 2019-10-08 | Method and device for dynamic database-based routing |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110795419A true CN110795419A (en) | 2020-02-14 |
Family
ID=69438814
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910949147.8A Pending CN110795419A (en) | 2019-10-08 | 2019-10-08 | Method and device for dynamic database-based routing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110795419A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111563837A (en) * | 2020-07-16 | 2020-08-21 | 浙江宇视科技有限公司 | Image processing system and method |
| CN112000825A (en) * | 2020-08-25 | 2020-11-27 | 浪潮云信息技术股份公司 | Method and system for establishing electronic license storage model based on sub-warehouse and sub-table |
| CN112925859A (en) * | 2021-03-31 | 2021-06-08 | 中国建设银行股份有限公司 | Data storage method and device |
| CN113486023A (en) * | 2021-07-27 | 2021-10-08 | 中国银行股份有限公司 | Database and table dividing method and device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104408174A (en) * | 2014-12-12 | 2015-03-11 | 用友软件股份有限公司 | Database routing device and method |
| CN107818115A (en) * | 2016-09-14 | 2018-03-20 | 苏宁云商集团股份有限公司 | A kind of method and device of processing data table |
| CN107844490A (en) * | 2016-09-19 | 2018-03-27 | 华为技术有限公司 | A kind of database divides storehouse method and device |
| CN108228848A (en) * | 2018-01-10 | 2018-06-29 | 山东浪潮通软信息科技有限公司 | A kind of expansible business datum divides library implementation method |
| CN109376151A (en) * | 2018-09-04 | 2019-02-22 | 中国建设银行股份有限公司 | Data divide library processing method, system, device and storage medium |
| CN109962951A (en) * | 2017-12-25 | 2019-07-02 | 航天信息股份有限公司 | Cloud platform monitoring data system |
-
2019
- 2019-10-08 CN CN201910949147.8A patent/CN110795419A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104408174A (en) * | 2014-12-12 | 2015-03-11 | 用友软件股份有限公司 | Database routing device and method |
| CN107818115A (en) * | 2016-09-14 | 2018-03-20 | 苏宁云商集团股份有限公司 | A kind of method and device of processing data table |
| CN107844490A (en) * | 2016-09-19 | 2018-03-27 | 华为技术有限公司 | A kind of database divides storehouse method and device |
| CN109962951A (en) * | 2017-12-25 | 2019-07-02 | 航天信息股份有限公司 | Cloud platform monitoring data system |
| CN108228848A (en) * | 2018-01-10 | 2018-06-29 | 山东浪潮通软信息科技有限公司 | A kind of expansible business datum divides library implementation method |
| CN109376151A (en) * | 2018-09-04 | 2019-02-22 | 中国建设银行股份有限公司 | Data divide library processing method, system, device and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| 董献伦: "基于关系型数据库的数据切分问题研究", 《中国优秀博硕士学位论文全文数据库(硕士)信息科技辑（月刊）》 * |
| 黄刚: "基于SQL历史的分布式关系型数据库自动分库的技术与算法", 《中国优秀博硕士学位论文全文数据库(硕士)信息科技辑（月刊）》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111563837A (en) * | 2020-07-16 | 2020-08-21 | 浙江宇视科技有限公司 | Image processing system and method |
| CN112000825A (en) * | 2020-08-25 | 2020-11-27 | 浪潮云信息技术股份公司 | Method and system for establishing electronic license storage model based on sub-warehouse and sub-table |
| CN112925859A (en) * | 2021-03-31 | 2021-06-08 | 中国建设银行股份有限公司 | Data storage method and device |
| CN113486023A (en) * | 2021-07-27 | 2021-10-08 | 中国银行股份有限公司 | Database and table dividing method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107704202B (en) | Method and device for quickly reading and writing data | |
| CN107229718B (en) | Method and device for processing report data | |
| CN109614402B (en) | Multidimensional data query method and device | |
| CN110019080B (en) | Data access method and device | |
| CN107480205B (en) | Method and device for partitioning data | |
| CN110795419A (en) | Method and device for dynamic database-based routing | |
| CN112597126B (en) | Data migration method and device | |
| CN110858194A (en) | Method and device for expanding database | |
| CN111061680A (en) | Data retrieval method and device | |
| CN112835904A (en) | Data processing method and data processing device | |
| CN111400304A (en) | Method and device for acquiring total data of section dates, electronic equipment and storage medium | |
| CN111753019B (en) | Data partitioning method and device applied to data warehouse | |
| CN111401684A (en) | Task processing method and device | |
| CN112100168A (en) | Method and device for determining data association relationship | |
| CN112783887A (en) | Data processing method and device based on data warehouse | |
| CN110851419A (en) | Data migration method and device | |
| CN113312355A (en) | Data management method and device | |
| CN110109919B (en) | Method and device for determining logic information | |
| CN109213815B (en) | Method, device, server terminal and readable medium for controlling execution times | |
| CN112115206A (en) | Method and device for processing object storage metadata | |
| CN113704242B (en) | Data processing method and device | |
| CN112783914B (en) | Method and device for optimizing sentences | |
| CN114443910A (en) | Data storage method, searching device and electronic equipment | |
| CN112711572B (en) | Online capacity expansion method and device suitable for database and table division | |
| CN112988857B (en) | Service data processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20220923 Address after: 25 Financial Street, Xicheng District, Beijing 100033 Applicant after: CHINA CONSTRUCTION BANK Corp. Address before: 25 Financial Street, Xicheng District, Beijing 100033 Applicant before: CHINA CONSTRUCTION BANK Corp. Applicant before: Jianxin Financial Science and Technology Co.,Ltd. |
|
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200214 |