CN112035482B - Service table partitioning method and device - Google Patents
Service table partitioning method and device Download PDFInfo
- Publication number
- CN112035482B CN112035482B CN202010904665.0A CN202010904665A CN112035482B CN 112035482 B CN112035482 B CN 112035482B CN 202010904665 A CN202010904665 A CN 202010904665A CN 112035482 B CN112035482 B CN 112035482B
- Authority
- CN
- China
- Prior art keywords
- partition
- configuration file
- data
- partitioning
- target column
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a business table partitioning method and a business table partitioning device, which relate to the technical field of computers, and the business table partitioning method comprises the following steps: acquiring partition information of a configuration file and a service table; comparing the first target column data of the configuration file with the second target column data of the partition information to obtain partition data to be added; generating a partition adding script file according to the partition data to be added; and partitioning the business table according to the partition added script file. The invention can automatically add the service table in the partition, reduce the workload of developers, reduce the error rate of data addition, promote the operation of adding a new partition more conveniently and rapidly, and further promote the efficiency of the service table partition.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a method and an apparatus for partitioning a service table.
Background
Various fields are included in the service table in the database, for example, for a service table involving multiple regions, a list of regions is included in the table for storing region information to which the service belongs. For systems where the business involves multiple locales and there are a large number of database tables that need to be partitioned by locale, developers often need to define the partition for each locale of each table in the database table creation script, which is labor intensive and prone to error.
Disclosure of Invention
The invention provides a business table partitioning method and a business table partitioning device, which can automatically identify and create column data which is lack in a database table so as to partition according to the column data, thereby improving the partitioning efficiency of the business table.
In a first aspect, an embodiment of the present invention provides a method for partitioning a service table, where the method includes: acquiring configuration files and partition information of the service table; comparing the first target column data of the configuration file with the second target column data of the partition information to obtain partition data to be added; the first target column data includes a plurality of partition data; the second target column data includes a plurality of partition data; generating a partition adding script file according to the partition data to be added; partitioning the business table according to the partition added script file; before acquiring the configuration file, the method further comprises the following steps: generating a region list configuration file; and taking the region list configuration file as a configuration file.
In a second aspect, an embodiment of the present invention further provides a service table partitioning apparatus, where the apparatus includes: the acquisition module is used for acquiring the configuration file and the partition information of the service table; the identification module is used for comparing the first target column data of the configuration file with the second target column data of the partition information to obtain partition data to be added; the first target column data includes a plurality of partition data; the second target column data includes a plurality of partition data; the adding module is used for generating a partition adding script file according to the partition data to be added; the partition module is used for partitioning the business table according to the partition added script file; the device further comprises a configuration module for: generating a region list configuration file; and taking the region list configuration file as a configuration file. In a third aspect, an embodiment of the present invention further provides a computer device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, where the processor implements the above-mentioned service table partitioning method when executing the computer program.
In a fourth aspect, embodiments of the present invention further provide a computer readable storage medium storing a computer program for executing the above-described traffic table partitioning method.
The embodiment of the invention has the following beneficial effects: the embodiment of the invention provides a business table partitioning scheme, which comprises the steps of firstly obtaining partition information of a configuration file and a business table, then comparing first target column data of the configuration file with second target column data of partition information to obtain partition data to be added, generating a partition adding script file according to the partition data to be added, and finally partitioning the business table according to the partition adding script file. The embodiment of the invention can automatically add the service table in the partition, reduce the workload of developers, reduce the error rate of data addition, promote the operation of adding a new partition more conveniently and rapidly, and further promote the efficiency of the service table partition.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and drawings.
In order to make the above objects, features and advantages of the present invention more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the description of the embodiments or the prior art will be briefly described, and it is obvious that the drawings in the description below are some embodiments of the present invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a method for partitioning a service table according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a business table partitioning system module and a relationship provided in an embodiment of the present invention;
FIG. 3 is a block diagram of a service table partitioning apparatus according to an embodiment of the present invention;
FIG. 4 is a block diagram of another service table partitioning apparatus according to an embodiment of the present invention;
fig. 5 is a block diagram of a computer device according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Currently, prior art solutions have the following drawbacks, before the operation of partitioning the service table, by manually maintaining each table by a developer, if the developer manually maintains the creation script or the partition addition script of each table:
1. the script is required to be maintained manually, so that the workload is high;
2. because of relying on manual work, the subareas of some areas are easily missed;
3. it is difficult to add new information to the system because of the need to manually write scripts to add partitions to each table.
Based on the above, the business table partitioning method and the business table partitioning device provided by the embodiment of the invention can reduce the workload of developers; reducing the possibility of error; the operation of adding the new region is more convenient, and the efficiency of the business table partition operation is further improved.
For the sake of understanding the present embodiment, a detailed description is first provided of a method for partitioning a service table disclosed in the present embodiment.
First, the technical terms involved will be explained:
partitioning:
a. refers to an operation performed when a database table is created. This operation generates several partitions in the table and possibly sub-partitions under the partitions or sub-partitions. A column needs to be designated as a partition column when partitioning. For a partitioned table, each record will be partitioned into a partition or sub-partition according to the value of the partition column. According to different record division modes and sub-partition depths, the partitions can be divided into a List partition, a Range partition, a Hash partition and a mixed partition.
b. Also referred to as the partition created by the partition operation.
Sub-partitions: the second level or less of the partitions in the hybrid partition.
List partition: a partition is created for each possible value of the partition column and records having different partition column values are partitioned into different partition ways.
The embodiment of the invention provides a business table partitioning method, referring to a business table partitioning method flow chart shown in fig. 1, comprising the following steps:
step S102, obtaining the configuration file and the partition information of the business table.
In the embodiment of the present invention, the configuration file is preconfigured information, and the configuration file includes information to be added to the service table, for example, the configuration file may be column data, where the column data is used to partition the service table, and the column data is used as the first target column data. The partition information of the service table may be current partition information of the service table, including column data used in performing the partition operation, and the column data is used as second target column data.
Step S104, comparing the first target column data of the configuration file with the second target column data of the partition information to obtain the partition data to be added.
In the embodiment of the invention, the data in the first target column data and the data in the second target column data do not completely overlap, and the data which is not in the second target column data and is not in the first target column data is used as the partition data to be added through comparison.
For example, the first target column data includes an a region, a B region, and a C region, and the second target column data includes an a region, then the B region and the C region are regarded as partition data to be added.
And step S106, generating a partition adding script file according to the partition data to be added.
In an embodiment of the present invention, a partition add script tool is used to dynamically identify and create missing locale partitions in a business table.
And S108, partitioning the business table according to the partition added script file.
In the embodiment of the invention, the partition adding script file is generated according to the second target column data and the partition data to be added, so that the business table is automatically partitioned according to the partition adding script file, and the updating efficiency of the business table is improved.
The embodiment of the invention provides a business table partitioning scheme, which comprises the steps of firstly obtaining partition information of a configuration file and a business table, then comparing first target column data of the configuration file with second target column data of partition information to obtain partition data to be added, generating a partition adding script file according to the partition data to be added, and finally partitioning the business table according to the partition adding script file. The embodiment of the invention can automatically add the service table in the partition, reduce the workload of developers, reduce the error rate of data addition, promote the operation of adding a new partition more conveniently and rapidly, and further promote the efficiency of the service table partition.
Considering that different services relate to different regions, in order to realize automatic partitioning of the multi-region service, the following steps may be further executed before the configuration file is acquired:
generating a region list configuration file; and taking the region list configuration file as a configuration file.
In the embodiment of the invention, the region list configuration file comprises a plurality of region information to be added to the service table, and the region list configuration file is used as the configuration file, so that the service table can be partitioned according to regions.
The region list file may be generated using the region name, or the region list file may be generated using the region code, for example, if the region code of region E is set to "007", the region code "007" may be previously arranged in the region list configuration file.
In order to improve the applicability and flexibility of the method, the method can further execute the following steps:
the locale list configuration file is modified, added or deleted.
In the embodiment of the present invention, the modification operation, the addition operation or the deletion operation may be performed on the region list configuration file, which operation is specifically adopted, and may be set according to actual requirements, which is not particularly limited in the embodiment of the present invention.
In order to facilitate automatic updating of the service table, the service table is partitioned according to the partition added script file, and the method can be implemented as follows:
and executing the partition adding script file to obtain the updated service table.
Referring to the business table partitioning system module and the relationship diagram shown in fig. 2, the method may also be implemented according to a system, where the system includes: and (3) a controller: is responsible for scheduling other modules; profile reader: reading a region list configuration file; database parser: connecting a database to obtain table partition information; partition addition script generator: generating a partition adding script; database script runner: and running a database script. The method is implemented according to the system by the following steps:
1. a partition table is established in the database, and the first-level partition is a List partition taking a region ID as a partition key. Since the partition table must contain partitions when being established, a partition of a real or virtual region ID can be arbitrarily established. The column names of the "region ID" of each table should be identical.
2. The configuration region list configuration file.
Referring to table 1, this table is one possible example (exemplified by YAML format), where "001" and "002" are region numbers:
TABLE 1
3. The partition adding tool reads the configuration file and dynamically adds the partition to the partition table of the database.
The workflow of the tool is as follows:
a. the controller calls a configuration file reader to read a region list configuration file and acquire a region list;
b. the controller calls a database analyzer to acquire all partition tables and existing partitions by taking the region ID as a first-level List partition key;
c. the controller identifies the missing partitions of the tables relative to the locale list in the configuration file;
d. the controller calls a partition adding script generator to dynamically generate partition adding scripts for the missing partitions;
e. the controller invokes the database script runner to execute the script, adding the missing partition to the database table.
The invention provides a business table partitioning method and a business table partitioning device, which can dynamically analyze the partitioning condition of a table in a database and dynamically add the missing regional partitions into the table according to a regional list configured by a user. After each new region or new list is added, the user can complete the addition of the new region by executing the tool. Because the invention does not need to pay attention to a specific region list when a new table is created, only an arbitrary partition is created, and the workload of database table creation script writer is reduced; because the partition to be created is automatically identified by the tool, the possibility of error occurrence of missing partitions is reduced; because the user only needs to update the region list configuration file when adding a new region, and then executes the partition adding tool, the partition script of each table is not required to be manually maintained, and the operation of adding the new region is more convenient.
The embodiment of the invention also provides a business table partitioning device, which is described in the following embodiment. Since the principle of the device for solving the problem is similar to that of the service table partitioning method, the implementation of the device can refer to the implementation of the service table partitioning method, and the repetition is omitted. Referring to fig. 3, a block diagram of a service table partitioning apparatus is shown, the apparatus comprising:
an obtaining module 71, configured to obtain partition information of the configuration file and the service table; the identifying module 72 is configured to compare the first target column data of the configuration file with the second target column data of the partition information to obtain partition data to be added; an adding module 73, configured to generate a partition adding script file according to the partition data to be added; the partition module 74 is configured to partition the service table according to the partition adding script file.
In one embodiment, referring to another block diagram of a traffic table partitioning apparatus shown in fig. 4, the apparatus further comprises a configuration module 75 for: generating a region list configuration file; and taking the region list configuration file as a configuration file.
In one embodiment, the configuration module is further configured to: the locale list configuration file is modified, added or deleted.
In one embodiment, the partitioning module is specifically configured to: and executing the partition adding script file to obtain the updated service table.
The embodiment of the present invention further provides a computer device, referring to the schematic block diagram of the structure of the computer device shown in fig. 5, where the computer device includes a memory 81, a processor 82, and a computer program stored on the memory and capable of running on the processor, and when the processor executes the computer program, the processor implements the steps of any one of the service table partitioning methods described above.
It will be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process of the computer device described above may refer to the corresponding process in the foregoing method embodiment, which is not repeated herein.
The embodiment of the invention also provides a computer readable storage medium, and the computer readable storage medium stores a computer program for executing any business table partitioning method.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
Finally, it should be noted that: the above examples are only specific embodiments of the present invention, and are not intended to limit the scope of the present invention, but it should be understood by those skilled in the art that the present invention is not limited thereto, and that the present invention is described in detail with reference to the foregoing examples: any person skilled in the art may modify or easily conceive of the technical solution described in the foregoing embodiments, or perform equivalent substitution of some of the technical features, while remaining within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention, and are intended to be included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (8)
1. A method for partitioning a service table, comprising:
acquiring configuration files and partition information of the service table;
comparing the first target column data of the configuration file with the second target column data of the partition information to obtain partition data to be added; the first target column data includes a plurality of partition data; the second target column data includes a plurality of partition data;
generating a partition adding script file according to the partition data to be added;
partitioning the business table according to the partition added script file;
before acquiring the configuration file, the method further comprises the following steps:
generating a region list configuration file;
and taking the region list configuration file as a configuration file.
2. The method as recited in claim 1, further comprising:
and modifying, adding or deleting the region list configuration file.
3. The method of claim 1, wherein partitioning the business table according to the partition add script file comprises:
and executing the partition adding script file to obtain an updated service table.
4. A traffic table partitioning apparatus, comprising:
the acquisition module is used for acquiring the configuration file and the partition information of the service table;
the identification module is used for comparing the first target column data of the configuration file with the second target column data of the partition information to obtain partition data to be added; the first target column data includes a plurality of partition data; the second target column data includes a plurality of partition data;
the adding module is used for generating a partition adding script file according to the partition data to be added;
the partition module is used for partitioning the business table according to the partition added script file;
the device further comprises a configuration module for:
generating a region list configuration file;
and taking the region list configuration file as a configuration file.
5. The apparatus of claim 4, wherein the configuration module is further configured to:
and modifying, adding or deleting the region list configuration file.
6. The apparatus of claim 4, wherein the partitioning module is specifically configured to:
and executing the partition adding script file to obtain an updated service table.
7. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method of any of claims 1 to 3 when executing the computer program.
8. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program for executing the method of any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010904665.0A CN112035482B (en) | 2020-09-01 | 2020-09-01 | Service table partitioning method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010904665.0A CN112035482B (en) | 2020-09-01 | 2020-09-01 | Service table partitioning method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112035482A CN112035482A (en) | 2020-12-04 |
| CN112035482B true CN112035482B (en) | 2023-08-18 |
Family
ID=73590861
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010904665.0A Active CN112035482B (en) | 2020-09-01 | 2020-09-01 | Service table partitioning method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112035482B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103699676A (en) * | 2013-12-30 | 2014-04-02 | 厦门市美亚柏科信息股份有限公司 | MSSQL SERVER based table partition and automatic maintenance method and system |
| CN109977145A (en) * | 2019-03-28 | 2019-07-05 | 山东浪潮云信息技术有限公司 | A kind of database auto-partition management method and system based on horizontal partitioning |
| CN109995813A (en) * | 2017-12-29 | 2019-07-09 | 杭州华为数字技术有限公司 | A kind of partition extension method, date storage method and device |
-
2020
- 2020-09-01 CN CN202010904665.0A patent/CN112035482B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103699676A (en) * | 2013-12-30 | 2014-04-02 | 厦门市美亚柏科信息股份有限公司 | MSSQL SERVER based table partition and automatic maintenance method and system |
| CN109995813A (en) * | 2017-12-29 | 2019-07-09 | 杭州华为数字技术有限公司 | A kind of partition extension method, date storage method and device |
| CN109977145A (en) * | 2019-03-28 | 2019-07-05 | 山东浪潮云信息技术有限公司 | A kind of database auto-partition management method and system based on horizontal partitioning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112035482A (en) | 2020-12-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111339041B (en) | File analysis and storage method and device and file generation method and device | |
| CN109670951B (en) | Block chain intelligent contract debugging and publishing method and system | |
| JP2019523462A (en) | Multitask scheduling method, system, application server, and computer-readable storage medium | |
| US8549046B2 (en) | Test data generation and scale up for database testing using unique common factor sequencing | |
| CN110471754B (en) | Data display method, device, equipment and storage medium in job scheduling | |
| US20120158453A1 (en) | Workflow processing system for supporting multi-tenant, and method thereof | |
| CN107423404B (en) | Flow instance data synchronous processing method and device | |
| CN111124872A (en) | Branch detection method and device based on difference code analysis and storage medium | |
| CN109165175B (en) | Equipment identifier generation method and device | |
| CN112925757A (en) | Method, equipment and storage medium for tracking operation log of intelligent equipment | |
| CN104516864A (en) | Report generating method and report generating device | |
| CN112035207A (en) | Page menu dynamic loading method and device based on database | |
| CN113641651A (en) | Business data management method, system and computer storage medium | |
| CN112035482B (en) | Service table partitioning method and device | |
| CN115757174A (en) | Database difference detection method and device | |
| CN115185825A (en) | Interface test scheduling method and device | |
| CN115270731A (en) | Collaborative editing method and device for mixed document | |
| CN111563123B (en) | Real-time synchronization method for hive warehouse metadata | |
| CN113111072A (en) | Business data file generation method and device | |
| CN111352824B (en) | Test method and device and computer equipment | |
| CN113495723A (en) | Method and device for calling functional component and storage medium | |
| CN107463374B (en) | Subset model extraction method and system based on inheritance and association automatic deduction | |
| US8423532B1 (en) | Managing data indexed by a search engine | |
| CN110908898B (en) | Method and system for generating test scheme | |
| CN115712622B (en) | Electric power transaction data processing method, system, computer device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |